[Projekt] Extracting Ace Combat

[Krishty]

Habe am Wochenende Flugzeugtexturen entschlüsselt. Von Gebäuden bin ich aber leider noch ein Stück entfernt …


… den ganzen Satz und das Post-Mortem lade ich später hoch, wenn alle Fehler behoben sind.

[marcgfx]

bis auf das cockpit schauts gut aus, könnte da noch ein fehler vorliegen?

[Krishty]

Nee, die sehen alle so aus – ist ja Sci-Fi; die Cockpits sind komplett geschlossen und alles wird via Kamera gesteuert

Nachtrag: Nee warte – habe doch auch konventionell aussehende:

[Krishty]

Let’s extract a plane.

Planes can be found in folders 0066–0102 (Japanese version) and 0047–0068 (EU/US version). These all have the following layout:

• folder with 12 small binary files (from 0 B to 16 KiB)
• folder with a small binary file (~100 B)
• folder with two textures:

  

• a binary file of several KiB

Pretty obvious this is plane data. But where’s the vertices? We should take a look at the binary files from the first folder. You’d probably start with the first file, but I’ll pick #9 here because it contains some data that’s not present in the first file (and it’s not as complex as the other ones). Let’s repeat the process of finding patterns in binary data, and so we’ll find this structure throughout the files:

This looks a lot like the vertex data we found earlier. In fact, it’s pretty easy to recognize vertices in the same XY XY XY Z Z Z format. Triangle indices are pretty easy to spot as well (the consecutive integers from 00 to 0B):

Visualize what we just found:

mesh 0 wireframe.png (3.15 KiB) 1199-mal betrachtet

Makes sense! I’ll take a shortcut here: In some other files, the last block is not empty, but contains quads instead of triangles (4th index not always zero). However, many files come with no triangles/quads at all, just a set of vertices. Let’s have a look at all files from the folder:

Using this identification chart, we can clearly identify it as a F-18 Hornet

Now, what about the other data (X, Y, Z, Q)?

X does not make sense as triangle indices, but it does make sense as indices into block #3. (See how it counts to 0B == 12 decimal, and there’s 13 entries in block #3?)

So what’s in block #3? It’s not too difficult to figure out once you know what you’re looking for. It’s the vertex normals, scaled to 4096 and stored as 16-bit fixed point numbers (because the PSX has no floating-point support). These normals are indeed very pretty (I decided to apply a random color to individual meshes in order to tell them apart):

We have seen Z before: It’s the GPU instruction for drawing a triangle.

There is not much room left for Y & Q: It must be texture coordinates, because where else would they go? But there’s eight numbers for three vertices, so we need to pick the right ones. We can easily get this done by using the individual texture coordinates as red and green colors:

(I chose a different plane here because I was too lazy to take another screenshot for the article, sorry!) Nice smooth shading and the seams are exactly where a designer would place them!

We’ll apply the texture next time. It’s more difficult than one would think, and it requires the last pair of bytes to get it right …

[marcgfx]

Sieht wieder mal gut aus Krishty! Bin schön gespannt wie dein Endprodukt dann ausschaut, bin weiterhin gebannt am lesen

[Krishty]

Just announcing three new "hidden" levels in the Japanese version of Ace Combat 3.


One hidden level has been known for years: Game Show, the level that shipped on demo discs, with minor modifications. It can only be accessed through a GameShark code (i.e. overwriting a memory location) and it's pretty boring. Its internal name is "----", and the developers probably used it as a test bed for AI before deploying the game.


Another hidden level has been known for some time, but nobody ever managed to enter it: Mission Code. There are some snapshots of the briefing on Acepedia, but the game will crash right after that.

I'm pretty sure I got its map decoded, and there's two copies of that map in the Japanese game files. It's Expo City in an early development stage. The differences from the final city are subtle, but it's amazing to see all the improvements that have been done. The map data looks compatible with the final data format, but the only airplane is the carrier for the Sepia spacecraft. This may be the reason why the briefing loads, but the mission doesn't.


Then there's two completely new maps. This time, the binary format of the models is not compatible with the final maps at all. I suspect the game will crash right away, and this might be the reason no one ever investigated the data. It's Expo City in some state after Mission Code and Megafloat in a *very* early stage of development. I have yet to find out how texturing works; even the vertex colors are screwed up.


I'll try to post snapshots of the early Expo City soon. Snapshots of the two other maps follow later. I have so many things to write here, but my PC does not have an internet connection, so it'll take some time to publish it all.

[Krishty]

I proudly present the first snapshot of the Mission Code version of Ace Combat 3's Expo City map!

For comparison, this is the same region in the demo/final version of the game:

• Expo City's iconic underground airport (top-left corner) has been introduced rather late. The early version didn't have one.
• The early version featured more buildings. The remaining buildings have been scaled down to as little as half their original sidelength (bottom-right corner).
• Texture detail has improved significantly (bottom-right corner).
• The final color palettes are much less colorful. Some look just like stills from the Matrix movie (e.g. the mission Awakening:)

  

• Prebaked lighting has improved significantly.
• The early version is brighter (about twice as bright). I tried to correct for it in the snapshots, so it's not obvious. It's not an art thing -- I guess at some point they compressed the color range of the prebaked vertex lighting and instead improved the texture color range. The reasons could be obscure color processing problems with the PSX GPU, or optimizations, or anything else.

I'll resume the plane extraction tutorial shortly, then I'll write about extracting maps.

[scheichs]

Alsways impressive how much NAMCO got out from a system like PS One and what tricks they used to accomplish that. Thanks for sharing your reengineering work with us guys!

[Krishty]

Sieht wieder mal gut aus Krishty! Bin schön gespannt wie dein Endprodukt dann ausschaut, bin weiterhin gebannt am lesen

Freut mich, wenn es unterhält

Alsways impressive how much NAMCO got out from a system like PS One and what tricks they used to accomplish that. Thanks for sharing your reengineering work with us guys!

One of the next write-ups will dive deep into the GPU memory management of the PSX, and this will be really impressive for anybody who’s into low-level stuff. It just takes me a lot of time to illustrate what’s going on. Stay tuned


In other news, I’m close to having decoded all maps, entities, and planes (including the new hidden ones). That’s enough stuff for three more threads like this …

[Krishty]

Extracted the hidden/secret levels/maps/terrains/missions from Ace Combat 3.

I’m now pretty confident we can group the three Expo City versions of the Japanese game version into alpha/beta/final. The hidden version of Megafloat is a beta version (not alpha) for pretty complicated technical reasons I’ll explain later. First, some snapshots:

This is the alpha version of Expo City. I explained its unique features in an earlier post. This is probably what you’d see if the hidden Mission Code level worked.

This is the beta version of Expo City. It is much closer to the final version (iconic underground airport, smaller buildings, new textures, cleaned up level), but there are subtle differences:

• some textures have not yet been replaced
• there are some holes in the terrain in other places
• it still uses the old lighting

But most importantly, its rendering technology is very different from the other levels. It’ll take me some more tutorial posts to explain it in depth, but it looks a lot like they tried to implement some kind of mip mapping or LOD …

Here’s the final version for reference:

Moving on to Megafloat. Here’s the beta:

It’s not as close to the final version as Expo City’s beta is, but … it uses the same exotic mip mapping approach, so it must have been created at the same time.

Again, the final version for reference:

Two conclusions:

1. Megafloat was completed much later than Expo City.
2. At a late point during development (when Expo City was almost in its final state), Namco tried to replace the game’s entire texture management with a more efficient approach, which likely allowed for mip mapping or farther range of sight – but they failed and rolled back to the old system.

Speaking of hidden beta stuff … the Japanese version was to feature a model viewer, but it was cut before release. It is accessable via memory manipulation (cheat/GameShark code; video not mine):

I extracted the model viewer’s data as well, and to my surprise, the models were slightly different from the ones you see in the video. We could have known earlier: the first two planes in the video are not lit dynamically (turning them upside-down does not change lighting), while the 3rd plane (at 1:18) is indeed lit dynamically. The reason I found: some models are so buggy/have so bad resolution that Namco chose to use the in-game models instead!

[scheichs]

Again very cool stuff! Do you have some polygon numbers? It appears to me that the beta versions have higher polygoncount than the final ones.

[Krishty]

polycount Expo City:

• alpha: 261,673 (4,505 unique)
• beta: 261,172 (5,050 unique)
• final: 251,066 (4,856 unique)

I included the unique polygon count because a few (less than 300) tiles are copy-pasted all over the map and it’s a better metric of entropy IMO. E.g this map:

has a large polycount of 524,800, but it’s just a single tile over and over (512 unique triangles), so the total polycount is misleading

Megafloat:

• beta: 153,031 (7,468 unique)
• final: 147,267 (5,495 unique)

So you’re right – the betas were much more detailed!

Another interesting metric is texture load, in bytes. However, I don’t want to comment on this until I wrote the article on the GPU memory system …

[kimmi]

Amazing results, thanks a lot for sharing those results!

Kim

[Krishty]

I’d love to share more, but I fear sitting down and writing tutorials would break my flow

Extracting maps from Ace Combat 2 took me three evenings:

1. analyzing the game archive format – no compression involved this time, and the internal data structures are very similar to Ace Combat 3
2. extracting tile polygons – still way to go here
3. applying textures – such a good thing I made my code from Ace Combat 3 reusable, it fits perfectly

I can only extract terrain, no detailed objects, but here’s two maps:

Details to come.

[Krishty]

In case anyone wants to extract Ace Combat 2, here’s my decompressor (I don’t think one has been written before):

unpack_AC2.zip

(2.2 KiB) 37-mal heruntergeladen

Apart from the compression, both games’ formats are almost identical, so here’s a similar one for Ace Combat 3:

unpack_AC3.zip

(2.41 KiB) 43-mal heruntergeladen

• for Windows
• should work with all regions (EU, JAP, US)
• and versions (demo, 1.0, 1.1)
• place unpack_AC2.exe in one folder with ACE2.DAT (for the demo, rename D.T to ACE2.DAT) and run
• place unpack_AC3.exe in one folder with ACE.BPB & ACE.BPH and run

Source code (public domain; Visual Studio 2017 solution, builds with VS 2015 as well):

2017-08-04 sources of unpack_AC2 & unpack_AC3.zip

(233.03 KiB) 13-mal heruntergeladen

Ace Combat 2 Regional/Version Differences

Ace Combat 2 was re-released on PlayStation 2 in 2005. I was hoping to find bugfixes and other improvements in the game files, but the only difference in the PS2 version is … this:

It’s even more confusing when you see this tweet from the developers, where the trademark was already in place. They probably just forgot it in the original game files.

Speaking of version differences …

• except for a japanese character map, there are no regional differences (Japan/EU/US) in the game resources
• the executables are vastly different – no wonder regarding the different screen resolutions & refresh rates of PAL vs NTSC
• versions 1.0 and 1.1 differ in
  • the translation fixes I mentioned before
  • a string table in 1.0 being padded with “04EAS.PLY P:\UNIX\PX201\BRIEFING\WO” instead of zeroes in 1.1
  • the executables, again (had no look at the disassembly yet)

However, the Demo version does differ from the full games. More specifically, binary data on the levels is slightly different. This may be different placement of enemies, but I’ll have to check further.