Ryujinx - Meet the Team Vol. 3

Interview with Ryujinx Team Developer riperiperi

What is your software development background? How did you find out about Ryujinx?

I'm a hobbyist game developer, and I also do software engineering for a living. I found out about Ryujinx as I've been big into emulation for a while, and was linked to the first post about it on GBATEMP from the emulation subreddit. Not sure how you're meant to capitalize that, but I don't think it matters.

I'm big into C# and dotnet core, as they've been improving rapidly recently. A fast & compatible modern system emulator in an environment, language and project structure like this is really a sight to behold, so of course I started contributing.

As far as past emulation experience goes, I wrote a Gameboy Color emulator in Javascript called amebo with the goal of running on mobile hardware, which was very optimized at the time for being plain JS (javascript). I also wrote an unreleased JS emulator for the Defender arcade cabinet, but that isn't public.

Of all the contributions you’ve added to the emulator thus far, which are you most proud of and why?

My first large contribution to the emulator - the majority of the ARM32 instruction set, and the ability to properly run games using it. A32 emulation was largely shrugged off due to most games being ports of games from the Wii U, which already ran on Cemu. But, all of these games have small tweaks and improvements that make them different or better than their Wii U counterparts, such as MK8D's battle mode. We had a few good months of being the first to have A32, but by the end of last year we had 4 emulators now running Mario Kart 8 in some form, due to overwhelming peer pressure. My ultimate goal was to get Mario Kart 8 running on as many places as possible, so I call that an overwhelming success.

I also added Jump Tables for quick jumps and calls in the JIT immediately after A32 to double CPU performance, and they've served us well until now. I'm a big fan of the texture dependencies stuff as well, but only because I slaved over it for so long.

What are the biggest factors when you are deciding what to tackle next?

It's almost entirely personal interest. I work on things that would improve games that I'm generally interested in, or technical challenges that I want to experiment with. Might seem a little selfish, but usually my interests align with the users’, so it's not too bad. ;) For instance, you'll find that most of my large changes have a positive effect on a certain racing game featuring Waluigi*, which isn't really a coincidence.

Out of the things interesting to me, I like to pick the items that will have the largest impact, rather than spend ages doing a large change with little impact. That's not really specific to me though, you should be doing that anyways.

*Mario & Sonic at the Olympic Games Tokyo 2020

Your reputation in the Ryujinx community is that you mainly work on Ryujinx graphics but you’ve also taken Switch emulation to new heights on several occasions: in early 2020 you added nearly all A32 CPU instructions to the emulator, making most 32-bit games playable. In the middle of last year you added a zero config resolution scaler and were largely responsible (along with fellow dev Ac_K) for the LDN build which enabled internet multiplayer capabilities. Is there any big feature left that hasn’t yet been implemented in a Switch emulator?

In terms of features used by games, I think we're reaching the end of the "big" features, with amiibo wrapped up and LDN functional. But if you think about it, there is one thing that emulators typically don't have that a console usually does, which is the surrounding interface. Wouldn't it be nice to have some of that experience working within the emulator, perfectly faithful to what you'd see on the real thing? Switch emulation is in a unique position where this is possible, and I'd like to see it happen.

In terms of emulator features, I feel like we can still pull some surprises. We recently added a texture replacement goal on Patreon, which will allow users to dump game textures, redraw, recompress or upscale them, then use them in place of the original textures in that game without file mods, similar to many other existing emulators. One thing we want to try out here is changing the replaced texture based on input configuration, which was recently implemented on the Dolphin emulator as "Dynamic Input Textures". Along with upscaled UI textures, these would improve the experience of playing switch games on PC, which usually has games swap out button prompts to match the controller that you've connected. Obviously, we have more ideas on top of this but there's no point blowing everything in one interview.

Graphics accuracy wise, probably just tessellation shaders are left unimplemented. Maybe a big LM3 fan will want to work on that one.

Performance wise, I guess you'll see.

When will we see LDN/LAN mode functionality come to the main build?

When I finish it off! Currently, we're using TCP communication for sending all of the packet data between clients. TCP gives us reliability and ordering, which is great for important packets such as knowing when someone joins or leaves a room; but TCP is rather useless for proxying UDP packets sent by a game, which do not need either of these things. It also gives us a connection lifecycle and is very simple to use, which is why development started here.

However, TCP's reliability and stream behaviour comes with a bunch of downsides that can result in a higher latency over an internet connection, especially if the bandwidth between clients is strained, so it's not exactly ideal for game networking. This is particularly true if the game itself is already using UDP, as nearly all LDN games on the switch are.

On top of this, we want clients to be able to connect P2P, but the only "reliable" way of doing this is to have the host open up a port for outsiders to connect to. We already attempt to do this automatically using UPnP, but testing has proven that it's extremely poorly supported. P2P is important as we want users to have the most direct connection to each other as possible, without us setting up dozens of servers around the globe. P2P solutions on UDP are much more widespread and better supported, as they are typically used for VOIP and video calling, so we'd be able to reap these benefits for more people, without as much reliance on our own network infrastructure.

So, the main blocker for getting LDN finished is performing the switch to UDP. I've already attempted using Valve Networking Sockets for this purpose, as it provides some nice optional reliability that we can use for room messages, with mixed results. I've mostly been distracted by working on other things, such as the recent texture fixes. I should get back to it soon. :woozy:

Why is graphical accuracy so important in emulation? Isn’t it fine to get the image to “good enough” status and then just make sure it runs at a playable FPS?

When people talk about what makes games appealing to them, they often talk about "Immersion". This is essentially losing the awareness that you're playing a game, which is usually facilitated by carefully crafted graphics, sound and gameplay. If anything is wrong with any of these three things, then the player will be jolted back to reality - this often happens with buggy games or poorly crafted assets in one. You can imagine that emulator induced issues with these things would cause the same feeling; the player would notice something is wrong and the experience will change as a result.

Emulators have another layer of deception - the goal is to be as close to playing the original console as possible. Not only does a graphical inaccuracy destroy your immersion in a game, it immediately makes it apparent that you're playing the game in an emulator, and it feels like a degraded experience. Even if you have enhancements provided by modding and resolution scale, any issue aside from that will turn playing on an emulator into a game of tradeoffs. What are you willing to lose for a higher resolution?

It's important to come up with solutions that don't turn into a game of tradeoffs. What good is smooth gameplay if it comes at the cost of sudden bursts of broken graphics? Conversely, what's the point in perfect graphics if there are stutters? I do think the former is more important than the latter in the above analogy, as slowdowns and stutters are more predictable and less surreal than graphical errors, but I do believe that you should aim to improve things without visual compromises.

Can you share a sneak peek of what you’re working on currently?

As I mentioned above, when games stutter or freeze, players lose immersion, audio or networking issues can ensue, and frustration mounts. We want to reduce the duration of these stutters, for as many users as possible (we are planning ARB shaders, but they have a limited audience), without visual artifacts. One of these routes is multithreading shader compilation at runtime, which would greatly reduce the duration of shader stutters on the first run of a game... but I've also made it clear that I think Async Shaders are for chumps. So how do we multithread compilation of shaders without skipping draws? Well, by multithreading the entire graphics abstraction layer, of course. I'm calling this "Threaded GAL".

If you think this is a silly idea, it's not entirely unprecedented! Good OpenGL drivers actually implement a form of this themselves, called "Threaded Optimization" on NVIDIA, where OpenGL calls are sent to be processed on another thread. I'm doing the same thing, but with calls to our graphics abstraction layer. So, this work effectively replaces the driver multithreading, eliminating its quirks and issues as well as giving us way more control over how it works and its optimizations. These issues include periodic stuttering, unpredictable buffer flush timing and unpredictable automatic disable.

So, how does this help implement multithreaded shader compilation? Well, currently when we reach a draw command, we must wait on the currently bound program to finish linking before we can continue processing graphics commands. There are also some other issues like getting uniform locations, but those aren't blockers anyways. With Threaded GAL, we can continue sending graphics commands to the backend without stopping to wait for any draws (or shader compilation!). This means that we're also submitting new programs to be compiled for these future draws, which are allowed to jump the queue and start compilation while we're waiting for the first. Modern OpenGL drivers allow for multiple programs to be compiled at once so we can simply start compilation for these programs as we wait for others, poll them for completion to see if we can complete the draw, and continue when we can.

This is all very confusing, but I can simplify it. Basically, we're creating a time portal for the GAL thread that lets us see what programs are being submitted in the future, so that we can start compiling them early for the compilation (and therefore draws) to complete sooner in the future, making the future happen closer to the present. Easy enough.

This should be dropping rather soon, though likely as a Draft PR to start so that it can be thoroughly tested. It also greatly improves performance of AMD GPUs on Windows (as it has no driver multithreading of its own), and might interact positively with Vulkan, which apparently needs the developer to manage their own threading. As this change lies between the GPU and the backend, it should be extensible to any backend.

Are you working on any other projects outside of Ryujinx?

Yes.

Before I started working on Ryujinx, I was working on a fan re-implementation of The Sims Online, called FreeSO, which is still online as we speak. This game was mostly developed by me, but there have been a number of contributors to the code as it is open source. The goal there was not only to bring the game back from the dead, but technically improve it in a bunch of ways and do some crazy stuff like 3D reconstruction from the 2D sprites to make it more accessible and interesting. I'm also working on some personal projects, but I've nothing to show publicly yet.