Reverse Engineering Spelljammer at Claude for Games

I'm writing this from Claude for Games at 165 Page St in San Francisco. My project for the night: reverse engineer Spelljammer: Pirates of Realmspace, the 1992 SSI game I played as a kid.

Quick backstory, if you missed the first post: I made it all the way to the end of that game, and then a quest bug made it impossible to finish. Thirty-some years later, I want to know exactly what broke. And I want to play it again in a browser.

The rules I'm working under: I use my own copy from GOG, and none of the game's art, maps, music, or extracted data goes on this site or in the repo. Everything you see here is our own tooling and code.

The toolkit

• REA, an agent-driven reverse-engineering toolkit that sits on top of Ghidra. This is the thing I came here to put through its paces.
• Ghidra for the disassembly and decompilation.
• DOSBox Staging to run the original as ground truth.
• Python for the format decoders, and Vite + Three.js for the port.
• Claude as my pair, doing a lot of the hex-staring with me.

Cracking the file formats

The first win was the data files. A few highlights:

• Ship images (MSH) are Huffman-coded, then run-length encoded with an RLE90 scheme. Each ship is stored from nine view angles, so the game can turn it as you maneuver. Getting the two layers in the right order was the unlock.
• Rumors and strings (the RUM files and STR.DAT) are XOR-encrypted with a repeating 8-byte key. Once you spot the period in the ciphertext, it falls right out.
• Music is XMI, the extended-MIDI format a lot of DOS games used. That converts cleanly to standard MIDI.
• Animations live in their own container (MDA), which is next on the list.

Three programs, not one

The game isn't one executable. It's three, handing off to each other:

• SJ.EXE: the front end and character creation.
• SF3.EXE: the heart of it. Flight, ship combat, ports, trade, and save/load.
• OTC.EXE: boarding actions, dungeons, and the finale.

That last one matters to me. The ending I never reached lives in OTC.EXE, and the state it reads gets written by SF3.EXE's save code. If the quest bug is anywhere, it's in the handoff between those two.

The wall: Borland overlays

Then I hit the real boss. Most of the actual game logic isn't in the EXEs the way a decompiler expects. It's packed into Borland VROOMM overlays, code segments that get paged in from the end of the file at runtime (you can spot the FBOV header). Point Ghidra or REA at SF3.EXE and you see the stub, the runtime, and a whole lot of nothing.

So the fix is to build an overlay loader: parse the overlay table, pull each segment out, rebase it, and hand the whole thing back to Ghidra as if the code had been there all along. That's what I'm building now, and then REA gets a second run at SF3.EXE with the logic actually visible.

Update: rather than finishing my own loader, I merged the overlays back into the executable with MZap from devroomm. Ghidra now sees 153 segments in SF3, up from 77, and 292 of the 335 strings are traced to the code that uses them.

Where this is going: Three.js

The end goal is a browser port in Three.js. The catch, and the point, is that it ships no game assets. You point it at your own GOG install, and it decodes the ships, text, and music at runtime on your machine. The repo holds only our code and docs. Anything decoded stays local and git-ignored.

Progress

Update, end of the night: the port is a clean-room Three.js reimplementation, and it's real. The code is all new. Art, text, and music load at runtime from your own GOG copy, and none of it is in the repo.

• The whole main quest, stages 0 through 9, is implemented, from the Waterdeep envoy to the mothership.
• Boarding fights, captain creation, and the ship and crew screens are built.
• The walkthrough is the test suite. I turned the community walkthrough into automated steps, written in my own words. All 20 logic steps pass, and the full suite is 41 of 41.
• A browser test now clicks through the real screens. It's at 8 of 17 steps. It stalls at the dwarven citadel, where the lich wipes a party without spells, so spells are next.

The "flying pyramid" I remembered from childhood turned out to be the dwarven citadel, the same step I'd flagged as the likeliest home of the bug. In the rebuild, losing there doesn't trap you, and the sceptre survives a save and load.

The source is on GitHub, and the project lives at Spelljammer 1992 Reboot.