Cxbx-Reloaded Progress Report (November 2020)

Progress Report (November 2020)

Surprisingly, this month was much more scarier for me than October: the amount of changes and improvements made and their frequency (don’t even get me started on this, I couldn’t look away from Github for more than 6 hours and BANG, already had yet another huge commit merged and ready for testing…) was nothing short of amazing. With a lot of improvements, you’re probably already asking “ok, great, but something else is playable?” and I’m glad you asked (you did, I know), as we got 11 new playable games in this month (and counting, as we couldn’t verify all the results so far). Hmm, I almost forgot: what about a nice performance boost to almost all the games? First, let’s check the numbers… but for a change, not here this time, the new official site is LIVE! https://cxbx-reloaded.co.uk/

Here’s what you can already benefit from it:

-  Future articles covering some of the work being made (outside of the already monthly reports)

-  Up to date compatibility data: no more need to wait an entire month to check what’s improved in your favorite game!

-  Much more detailed compatibility information: a game used to work and broke? It’s now much easier to find out when it was working and potentially what broke it. Finding regressions will take less time as more reports are submitted

-  Direct download to the latest builds: some users aren’t comfortable with using Github to download our releases, so this will make their lives easier.

So right now you can just visit the site and search for the game you want some information about, making my previous work on exporting compatibility pdf files a thing of the past ;)

General Improvements

Miscellaneous Direct3D LTCG fixes [PR #2022]

They told us that it was impossible… we prove them wrong! For a very long time, LTCG was seen as this solid roadblock that would prevent us from getting any game that uses it from working, but this is one of the changes that make it very clear that it’s very possible. LTCG is an optimization made on the code of the game that makes things harder for CxBx-Reloaded (e.g. symbol detection), but not even close to impossible. Results from this latest improvement? Midtown Madness 3 and Kelly Slater's Pro Surfer no longer crashes, Metal Wolf Chaos no longer crashes with the debug version of the emulator and some other games are rendering graphics for the first time (e.g. Jade Empire)

Vertex declaration refactoring [PR #1894]

One of the biggest changes we had been working on for a loooooong time (yes, the PR was open for months). We had very good reasons to keep it under testing for so long: it’s a fundamental change to how rendering works and had the potential to break a lot of things (and before some major fixes, it did). With most of the regressions fixed (we’re aware of some that still need to be addressed), we can finally talk about the positive side of it: a ton of games improved in extreme ways (from black screen with almost nothing visible to almost perfect visible graphics). I recommend that you check the PR link itself for the list of games that were improved (it’s honestly too much to even list in this report), as well some of the regressions introduced (keep in mind that some of those are already fixed).

Misc fixes after the vertex declaration changes [PR #2031]

As said on the previous item, regressions were expected from this huge change, and this following PR was made to address them. As a nice side effect, this fix also improved rendering in other games (Need for Speed Hot Pursuit 2 and Tony Hawk’s Pro Skater 2).

Take vertex stream offset into the account when caching VBs [PR #2034]

Caching vertex buffers in CxbxVertexBufferConverter::ConvertStream ignored the vertex stream offset parameter passed together with the vertex stream, therefore always sampling X vertices from the beginning of the buffer. Adding offset to the guest vertex data allows the cache to treat such buffers with an offset no different than separate buffers. Fixes exploding vertices in TOCA Race Driver/Pro Race Driver.

Remove hacks skipping guest resource removal if they are bound [PR #2036]

This PR removes special cases in patched DestroyResource methods, skipping removal of currently bound resources. It may have been required in the past to avoid crashes, but nowadays it is not only unneeded, but also causes host and guest memory leaks. Because our special case skipped the guest trampoline, games literally ran out of memory as far as they are concerned. Fixes Outrun 2 running out of memory and crashing several seconds after entering the race and Monster Garage running out of memory and crashing as soon as the mission was started. More games could probably be fixed by this changed, but no other title was confirmed.

Allow the cxbxr to work again with wine under Linux [PR #2037]

The flash region number 3 fails reservation when running under wine, which causes the emulation to fail. Since it's still acceptable for this region to fail, we workaround the problem by simply allowing it to fail reservation by adding the MAY_FAIL flag. With this, the loader works again under wine.

Improve viewport calculations [PR #2033]

This was another big improvement that made dozens of games show 2D elements (HUDs and menus) for the first time. Some of the verified improvements were: Need for Speed Hot Pursuit 2 menus and in-game HUD elements, Mafia map and text (in-game and menus), Pirates of the Caribbean dialogue boxes, Mortal Kombat: Deception and Mortal Kombat: Armageddon menus (that were only partially visible before this change) and DRIV3R menus.

Fix free up address range's system allocated [PR #2038]

Now that we have found the cause of address range for wine issue, this can be fixed properly on both ends. The cause was that the index is set to a negative one which then tries to obtain value outside of address ranges' list. With this fix, it no longer causes a loophole on both linux with wine and Windows. Aka clear_systems not being set other than zero.

Palette fixes [PR #2045]

This PR consists of 3 parts:

●  The first commit fixes a stupid copypaste error in D3DDevice_SetRenderTarget_0

●  Fixes D3DDevice_SetPalette and D3DDevice_SetPalette_4 not calling to guest code - because palettes are regular D3D resources, not calling to guest potentially resulted in broken reference counting and leaking of guest palette resources. This should fix guest memory leaks in games using palettes.

●  In DRIV3R I observed a case where the game had a palettized texture bound, but did not bind a palette. This caused a crash on "resolving" palettes, so I added a test case for such occurrences and fill the texture with zeroes - without a palette it is not possible to guess what colors should have been used (AFAIK), so filling with zeroes is the safest choice.

D3D: wrap surfaces in textures [PR #2044]

Fixes when titles attempt to pass a Surface object to SetTexture. Games improved: Midnight Club 3 menus now properly visible and in-game rendering better, Red Dead Revolver rendering much more in-game (only some textures missing), bloom fixed in Sponge Bob: Battle for Bikini Bottom, rain rendered properly on GTA III, Steel Battalion: Line of Contact rendering 3D for the first time on the demo loop.

Update renderstate table with 4034 findings [PR #2049]

This dramatically improves Simpsons: Road Rage

Optimize vertex stream cache for partial buffer access [PR #2048]

Drastically reduces the amount of copied and converted data in cases where any of the following apply:

●  BaseVertexIndex is not 0

●  LowIndex is not 0

●  dwStartVertex is not 0

Also potentially fixes an issue where data past an indexed vertex buffer was hashed and converted, leading to potential false positives on dirty checks, maybe more. A lot of games had a huge performance gain, here’s a small compilation for some of the results https://youtu.be/6qfpQFIeQNU

Z fixes [PR #2040]

●  Calculate Z matching D3D9 docs (combined with Xbox depth scaling)

●  Normalize Z from either (0, 1) or (0, zbuffer depth)

The Rendertarget commit corrected both JSRF test cases, as well as the GunValkyrie one.

Rework texturestate mapping to be more sane + fix unmapped get [PR #2053]

This reworks the TextureState mapping table indexing to be a little saner/easier to grasp what's happening. It also fixes an oversight where TextureState reads were not taking mapping into account.

Preserve original IO status block and context for ApcRoutine [PR #2055]

Fixes cases where guest's APC routine/completion callback does not receive the original IO status block variable, but it assumed it did (it's allowed as per WinAPI docs). Fixes Project Gotham Racing deadlocking after intro movies unless silence.wma was removed.

Remove 3925 fog/alpha hacks [PR #2052]

This PR removes the Fog and Alpha disable hacks for 3925 XDK.

We don't yet understand why these hacks were needed, and while they have been present, a solution has not been investigated. This will cause apparent regressions in some titles using this XDK (Silent Hill 2, perhaps Halo), however, while this hack is present, we cannot understand and fix the proper behavior.

There's also a chance that there are some 3925 games this hack hinders, rather than helps.

Notable Game Improvements

Also doing this section differently from what we’re used to: I’ll be listing only the new playable games we had this month. Reason: a lot of the games couldn’t be tested long enough so there’s probably a lot more being fully playable right now that I’ll be covering on the next report (and for most people playable is the only state that matters).

2002 FIFA World Cup [PLAYABLE] https://youtu.be/_00RiDXBTiA

Verified as fully playable.

AFL Live 2003 [PLAYABLE]

Verified as fully playable.

Dead or Alive 3 [PLAYABLE] https://youtu.be/dm9X93jHpaA

Menus and HUD now visible, verified as fully playable.

ESPN MLS ExtraTime 2002 [PLAYABLE]

Verified as fully playable.

Mortal Kombat Armageddon [PLAYABLE] https://youtu.be/45ppsvhQHcE

Every single game mode 100% tested, every single game mechanic works as intended (including the bonus stuff as the Krypt, character creation and Motor Kombat).

Teenage Mutant Ninja Turtles 2 Battle Nexus [PLAYABLE] https://youtu.be/92iyfYccElE

Verified as playable.

Teenage Mutant Ninja Turtles 3 Mutant Nightmare [PLAYABLE] https://youtu.be/rJuj0Tq2hGU

Verified as playable.

World Series Of Poker [PLAYABLE]

Verified as playable.

Defender [PLAYABLE] https://youtu.be/OBOg73xuyl0

Verified as playable.

The Hulk [PLAYABLE] https://youtu.be/9_D5EXnq0uk

Menus text fixed, verified as playable.

Mojo! [PLAYABLE] https://youtu.be/Sa_x8P9ZY2E

Last gamebreaking issue fixed (crash related to rendering in a specific level o World 2), can be finished properly now without major issues.

Final Words

As I’ve mentioned, the report is over but there’s still a lot of testing required so expect for even more playable games being confirmed on the next one. What’s being worked on right now? This can be checked here, but I can already give a little spoiler that #2014 will be yet another huge incoming improvement. As there’s no more need for me to provide pdf files about the recent compatibility data, all I can recommend is that you all check on the official site from now on to keep up with anything new, see you all next month ;)