Which is the last step on the way to completely position-independent
code, with no random hex numbers that should have been data pointers,
but weren't automatically turned into data pointers by IDA because
they're only ever addressed in the indirect fashion of
mov bx, [bp-array_index]
mov ax, [bx+0D00h] ; 0D00h is obviously an array of some sort
Removing all of these makes it practicable to add or delete code without
breaking the game in the process. Basic "modding", so to speak.
Automatically catching all possible cases where this happens actually
amounts to emulating the entire game, and *even then*, we're not
guaranteed that the *size* of the array just falls out as a byproduct
of this emulation and the tons of heuristics I would have thrown on top
of that. ZUN hates proper bounds checking and the correct size of each
array may simply never be implied anywhere.
So, rather than going through all that trouble of that (and hell, I
haven't even finished *parsing* this nasty MASM assembly format), and
since nothing really has happened in this project for almost two years,
I chose to just turn this into a text manipulation issue and figure out
the rest manually. Yeah, quick and dirty, and it probably won't scale if
I ever end up doing the same for PC-98 Policenauts, but it'd better work
at least for the rest of PC-98 Touhou.
Trying to do one of those per day from now on. Probably won't make it
due to the reverse-engineering effort required for the big main
executables of each game, but it'd sure be cool if I did.
This, hands down, has been the single worst stretch of decompilation so far.
Three extremely difficult functions that each still required inline assembly.
And no, this didn't even work out with any of the optimization features in
Borland C++ that aren't included in Turbo C++.
Oh, right, these functions can have parameters. So, let's turn snd_kaja_func()
into a macro that combines the function number and the parameter into the AX
value for the driver.
And renaming them all to the short filenames they will be decompiled to for
consistency. These functions aren't really immediately hardware-related, as
we've established earlier in the decompilation.
Only one code segment left in both OP and FUUIN! its-happening.gif
Yeah, that commit is way larger than I'm comfortable with, but none of these
functions is particularly large or difficult to decompile (with the exception
of graph_putsa_fx(), which I actually did weeks ago), and OP and MAIN have
their own unique functions in between the shared ones, so…
Which, for some reason, is also found in the MAIN.EXE of every later game
in between completely unrelated hardware and file format functions.
Separate commit because it has its own segment in REIIDEN.EXE, and because
coming up with the nice function names took pretty long, since I haven't done
anything involving trigonometry in the past 5 years...
So yeah, after ignoring this issue for a week, we indeed have no choice but to
decompile these functions into this horrible mess of C and inline assembly.
And you know what? Since the compiled result still matches with ZUN's binary,
it's entirely possible that this *was* the original format this code was
written in! Seriously, how intoxicated do you have to be to write (or rather,
slur) code like this?
Keeping these functions entirely in assembly would have surely been better.
However, it would have made linking practically impossible, especially for the
later games which still need them in the current assembly slice format.
Yet another set of questionable C reimplementations of master.lib functions to
waste my time. And half of them, including z_text_(v)putsa, aren't even called
anywhere.
So apparently, TH01 isn't double-buffered in the usual sense, and instead uses
the second hardware framebuffer (page 1) exclusively to keep the background
image and any non-animated sprites, including the cards. Then, in order to
limit flickering when animating the bullet, character and boss sprites on top
of that (or just to the limit number of VRAM accesses, who knows), ZUN goes to
great lengths and tries to make sure to only copy back the pixels that were
modified on plane 0 in the last frame.
(Which doesn't work that well though. When you play the game, you still notice
tons of flickering whenever sprites overlap.)
And by "great lengths", I mean "having a separate counterpart function for
each shape and sprite animated which recalculates and copies back the same
pixels from plane 1 to plane 0", because that's what the new functions here
lead me to believe. Both of them are only called at one place: the wave
function on the second half of Elis' entrance animation, and the horizontal
masked line function for Reimu's X attack animations.
This function raises one of those essential questions about the eventual ports
we'd like to do. I'll explain everything more thoroughly here, since people
who might complain about the ports not being faithful enough need to
understand this.
----
The original plan was aim for "100% frame-perfect" ports and advertise them as
such. However, the PC-98 is not a console with fixed specs. As the name
implies, it's a computer architecture, and a plethora of different, more and
more powerful PC-98 models were released during its lifespan. Even if we only
consider the subset of products that fulfills the minimum requirements to run
the PC-98 Touhou games, that's still a sizable number of systems.
Therefore, the only true definition of a *frame* can be "everything that is
drawn between two Vsync wait calls". Such a *frame* may contain certain
expensive function calls, and certain systems may run these functions slower
than the developer expected, thus effectively leading to more *frames* than
the developer explicitly specified.
This is one of those functions.
Here, we have a scaling function that appears to be written deliberately to
run very slow, which ends up creating the rolling effect you see in the route
selection and the high score and continue screens of TH01. However, that
doesn't change the fact that the function is still CPU-bound, and neither
waits for Vsync nor is iteratively called by something that does. The faster
your CPU, the faster the rolling effect gets… until ultimately, it's faster
than one frame and therefore vanishes altogether. Mind you, this is true on
both emulators and real hardware. The final PC-98 model, the Ra43, had a CPU
clocked at 433 Mhz, and it may have even been instant there.
If you use more optimized algorithm, it also runs faster on the same CPU (I
tried this, and it worked beautifully)… you get the idea.
Still, it may very well be that this algorithm was not a deliberate choice and
simply resulted from a lack of experience, especially since this was ZUN's
first game.
That leaves us with two approaches to porting functions like these:
1) Look at the recommended system requirements ZUN specified, configure the
PC-98 emulator accordingly, measure how much of the work is done in each
frame, then rewrite the function to be bound to that specific frame rate…
2) …or just continue using a CPU-bound algorithm, which will pretty much
complete instantly on any modern system.
I'd argue that 2) is actually the more "faithful" approach. It will run faster
than the typical clock speeds people emulate the games at, and maybe draw a
bit of criticism because of that, but it seems a lot more rational than the
approximation provided by 1). Not to mention that it's undeniably easier to
implement, and hey, a faster game feels a lot better than a slower one, right?
… Oh well, maybe we'll still encounter some kind of CPU-bound animation that
is so essential to the experience that we do want to lock it to a certain
frame rate…
Fuck TH02 and above and their bizarre assembly code that indeed appears to be,
uh, playfully "optimized" in the most inadequate of places, far away from the
innermost loop. It's ALWAYS just these one or two instructions I just can't
fucking get out of the C compiler, which lead to the conclusion that these
functions must have either been first compiled to assembly, then "fine-tuned"
and then linked into the executable…
… or I'm really just missing some obscure compiler setting.
At least with TH01, you can tell that the source language must have undeniably
been C++, and the decompilation is a breeze.
MAIN.EXE shares most of the code in this segment, but I can't remove it from
there right now due to the weird ordering of the data segments in that
executable…
And yes, once again, those three seemingly random type casts in here are
*necessary* to build a bit-perfect binary.
Small detour into MAINE.EXE because it has all the juicy algorithms that will
explain the remaining unknown members of the highscore data structure, and
there's this one code segment here we need to get out of the way first.
The same function appears unused in TH02's MAINE.EXE. Separate commit because
this was painful enough and we can link the C version into FUUIN.EXE right
now.
Oh, OK, so this is what the PC-98 GRCG is all about. You call grcg_setcolor(),
and that puts the PC-98 hardware in some sort of "monochromatic mode". Then,
you just write your pixels into any *single* one of the 4 VRAM bitplanes. This
causes the hardware to automatically write to *all* bitplanes in such a way
that the final palette index for each of the 8, 16, or 32 pixels you just wrote
a 1 value to will actually end up to match the color you set earlier.
Don't forget to call grcg_off() at the end though, or you can't draw any
non-monochromatic graphics, heh.
Yes, all of it. Including the bouncing polygons, of course. And since it's
placed at the end of ZUN's code inside the executable, the code's already
position-independent and fully hackable.
Don't really understand the other games yet because they start introducing
joystick support and TH03 has multiplayer and then there are these master.lib
modifications that don't really make any sense to me, especially when you add
that TH04 seemingly does not read js_stat *at all*, yet still works just fine
with a gamepad and... urgh.
Well, duh, of course, we *can* do this in order to allow decompilation to be
started at the end (not the beginning) of any segment. In fact, if we hadn't
done this, we would have had to start by moving _TEXT out to libraries....
Well, that became unbearable pretty quickly. Not sure whether I'm doing all
this Makefile business right, but this looks pretty nice.
It doesn't really help much at this point though because the 32-bit part is
still entirely separate and forces everything to rebuild all the time, but at
least it aborts on C compiler errors.
After spending a few hours on correctly decompiling ZUN's bulky custom text
renderer used in TH02 and TH03, it unfortunately turned out that TLINK doesn't
actually give us the fine-grained control over segment ordering we'd like to
have in a project like this, and that we can't slot code from one object file
in between segments from another object file. This means that yes, we really
have to decompile the functions in the order they appear in the executables,
starting on either end.
So, have a boring janitorial commit instead.
This took long enough, so we're not covering the COM files right now. Like, I
can't even tell how you're supposed to work around the forced word alignment
for the _TEXT segment. Guess we'll just have to decompile all of these in one
go, just like we did with ZUNSOFT.COM.
Also, it really seems as if we're merely trading one ugly workaround for
another in our quest for identical binaries.
Yup, we'll be linking against the original binary blob for the time being.
Don't worry though, we will (and in fact, have to) recompile the libraries
from source, separately for each game, as part of the build process in the
future, but we'll get to that once we've decompiled some of the non-TH01 code.
So yeah, that'll be our build environment - just plain batch files calling the
Borland command-line assembler, linker, and eventually C compiler. These are
the exact tools that ZUN used as well. There certainly are other assemblers,
compilers and linkers that could compile this code into 16-bit DOS
executables; Open Watcom is the only free one I know, and the master.lib
manual also mentions C compilers by Microsoft and Symantec. However, I favor
having one clear build path for a single toolchain that will, with the correct
command-line switches for each game, create builds that are bit-perfect to
ZUN's original ones over the possibility of cross-platform builds and the
maintenance nightmare they add.
So, Borland-only it is.
(Also, no Makefile, due to our messy build setup. I think I still prefer this
solution though, as we can have these really nice error messages that double
as build instructions without any dependencies on installed software.)
I kinda wanted to wait with this until I've brought REIIDEN.EXE down to at
least 65,536 lines as well, but that's not going to happen anytime soon, and
this split has annoyed me enough by now...
nmlgc