Wonder what we've been up to since our reboot in 2022? Here's a wild mix of improvements, from compatibility fixes to major performance uplifts and more!
2023-07-18 11:17:32 +0000 UTC View PostSummer is here, and you probably noticed it: It's been fairly silent around Mikage on social media lately (other than my rants about hobbyist lawyers and actual lawyers). The truth is it's too nice outside to spend all day coding at home, so instead I'm taking some refreshing time off to power up energy levels and gather new ideas!
Of course Mikage will be back later. Partly I want to relieve some of the burden of keep people in the loop all the time, but also I felt compelled to say there won't be much active work for transparency's sake. So in case you were wondering why there's no news... well, this is why š
Thanks all for your steady support, and most importantly: Enjoy your summer!
2023-05-30 17:35:43 +0000 UTC View Post
Hey folks!
Just a quick heads up: In a few hours, a new gameplay video showing off VVVVVV will go live on Mikage's YouTube channel.
The twist: Audio support! (Yes, finally š)
Premiere will begin at 8:30 pm CET if you want to join and follow live chat, otherwise of course you can still catch up later.
That's all! Have a nice week and until next time š
2022-11-08 13:53:12 +0000 UTC View Post
Update August 2022: Alpha releases are no longer available. The text below is for historical reference, only.
You can now play Super Mario 3D Land on Android! I added lots of updates this month specifically for this game (most of which luckily will benefit other titles, too), as you'll have seen in the gameplay video earlier this month. As usual, the new alpha release is available for patrons in the $10 tier and above at the internal download area.
The list of changes includes these items over the March release:
Besides enabling Super Mario 3D Land to work, these changes also fix graphical glitches in other games, such as The Legend of Zelda: Ocarina of Time 3D, Cubic Ninja, Steel Diver, and more.
I'm also pleased to announce there will be another major performance boost next month - the code is almost complete, but didn't quite make it in time for this release. I'll keep you updated about further progress in May ;)
2020-04-23 12:24:43 +0000 UTC View Post
Yes! You're reading that right: Super Mario 3D Land is now (functionally) playable on Android via Mikage :)
I'll publish a write-up of this exciting development progress next week, and I'll also push the next alpha release soon so you can try the game out for yourself.
Until then, stay safe!
2020-04-18 16:23:01 +0000 UTC View Post
We have another update in store, this time with major optimization work going on and of course the obligatory compatibility improvements. Let's dive right in!
2020-03-26 12:46:45 +0000 UTC View Post
The big update is finally done - readily available from the download area for patrons!
You can read the full list of changes here. Notably this includes an early prototype of the new CPU JIT, but furthermore:
These changes lay the ground for fullspeed emulation: Some games already see a huge frame rate improvement (notably Retro City Rampage DX and yeti3DS). With further work in the future (especially on the CPU JIT), more games will start running at playable speed!
There are also some hugely exciting compatibility updates in the pipeline for the next release - suffice to say there's a big title you'll see covered a lot soon ;)
Took a while, but the second Progress Report is out now!
Note that if you've been following the Development Updates closely, most of it won't be new to you. But if you were looking for a more condensed and high-level overview of what has happened since August, that's it.
As usual, thanks for your support, and let's make 2020 a good one :)
2020-02-01 15:50:12 +0000 UTC View Post
Hey folks!
First of all - happy new year! Mikage has kicked off 2020 with some really cool progress, but more on that later this month ;)
As for today, I figured it's time for another video, and I already teased Cave Story 3D back in Development Update 7 so why not? Here you can see me playing the game (well, trying to) to show off what's now possible with December's emulation fixes.
There'll be more things to come in January - notably a new alpha release and, of course, another Development Update. Until then!
2020-01-03 17:04:45 +0000 UTC View PostHi folks!
I've got an announcement to make - Development Updates will be public from now on, telling you about all the latest progress and advances we make on 3DS emulation!
What's a Development Update? It's sort of a small Progress Report - not focusing on the big picture, but instead giving brief updates on what happened over the last couple of weeks: New games that started running, interesting bugs that were fixed, and other tasks involved with emulator development.
I've always wanted to give people on the edge of supporting Mikage a better idea of the development pace and challenges faced, but, alas, I frankly don't have enough time to write monthly Progress Reports. So instead, I decided to make the material that's already there accessible to a wider community. That does not mean patrons go away empty-handed: They'll be the first to read new Development Updates and Progress Reports!
The first two posts have been published today - the others will be opened up over the next couple of days, so keep an eye on this page!
Happy Holidays!
Another month, another update - let's see what December had in it for us :)
The big news comes first: Mikage runs another game, Cave Story 3D! Turns out after a couple of small additions and the fixes mentioned later, this game pretty much just worked out of the box. And for what it's worth, both the original Cave Story and Cave Story 3D are supported ;)
I've also given an old homebrew release a shot, smea's portal3DS. A couple of GPU features are missing to make it playable, but the menu screen is rendering mostly fine so far!
It's not always about getting new games to boot - often, fixing smaller bugs in games that already work uncovers subtle issues that would've been harder to debug in a game that outright refuses to render anything at all.
Cubic Ninja is a good example of this: It was one of the first games supported in Mikage, but its opening screen would instantly pop up rather than smoothly fading in. I decided to look into the issue, and found this:
Turns out I mixed up the GPU register definition for some of the alpha blend modes! Having fixed that issue, Cubic Ninja's intro now plays just fine:
A similarly subtle, but ultimately really simple issue to fix was the intro logo rendering in Nano Assault EX: I had misplaced some parentheses in the shader generated when rendering this effect, leading the computation of the AddThenMultiply texture combiner operation to return the wrong result. Voila, we finally know who created the game!
Meanwhile, remember the flame effect on the title screen of The Legend of Zelda OoT 3D that was working just fine in the software renderer? The Nano Assault fix made this flame effect work in Vulkan too! A great example of how fixing a bug in one game can resolve seemingly unrelated issues in other games in one go.
That's it, folks - plenty of good progress! I'm planning on having the next alpha release in January, but I'll upload another gameplay video soon to make the waiting a bit easier on you. Until then, enjoy your holidays and have a great start into the new year! :)
This post was originally posted on 11 December 2019 for patrons, and released publicly on 16 January 2020. Support us to get early access to Development Updates and more!
2019-12-11 19:45:08 +0000 UTC View Post
As a followup to yesterday's Development Update, I recorded some gameplay footage to show off in more detail how great Nano Assault already runs in Mikage.
Like I mentioned before, I'm super happy to have a game with such complex graphics work in the emulator already. Hope you enjoy :)
After the long-winded build up over the last months, we're finally seeing some work on the emulator core itself again. Lo and behold, there's progress on support for more games, and a new alpha release on the horizon!
I already teased Bravely Default back in July when it started booting. With the upcoming alpha release, it will get past the intro screens and let you create a new save game and even play the game! Thereās a small catch: None of the 3D graphics get rendered right nowā¦ Granted, maybe calling that āplayableā is a stretch even for emulation ;)
Thereās another title thatās fairing better though, which I havenāt mentioned so far even though itās been doing āsomethingā for a while now: Nano Assault!
Nano Assault works remarkably well already: Not only can you create save files and watch the gameās intro cutscene, but you can actually play the first two levels too! I'm very happy with this result, especially since it's one of the graphically more complex games. Not all effects are rendering perfectly due to missing features, so that's a good hint as to what to implement next.
With the news out of the way, letās see what work was involved to get Mikage up to the task.
By far the biggest work item was the RenderTargetCache: This is the part in Mikage that matches raw framebuffer data in emulated memory with fast render targets used on the phone GPU. This sort of cache is critical for getting any performance benefits out of hardware (Vulkan) rendering over software, and getting it right is essential to bug-free emulation without rendering glitches. In fact, Citra still struggles with this today, so it was important to me to address this early on.
Thanks to the work I put into the testing framework, it wasnāt too stressful to come up with a reliable rewrite of this component. As a result, not only is the RenderTargetCache itself much more readable, but I was also able to resolve various compatibility issues: Nano Assault picked up an old render target previously, causing it to crash shortly after due to an invalid viewport setup. The both_screens homebrew example now renders the bottom screen as youād expect.
Finally, the RenderTargetCache revamp removes the need for many annoying workarounds that harmed performance. Of course, the bottleneck is still largely CPU emulation, so don't expect this to do miracles to your frame rate yet. But at least the GPU wonāt hold us back anymore now either.
Link files are used to open a second "view" onto an already opened file, which is a feature that many games - notably Bravely Default and Nano Assault - use internally. Implementing this feature posed a bit of a challenge: I didnāt know what things games were allowed to do with them specifically, and so supporting link files couldāve required either a minor adjustment or a costly rewrite.
Once again, the testing framework came to the rescue though. Link files are only supported in the SelfNCCH archive, so writing tests for them was straightforward. In the process, I also ported the existing FS tests to the SelfNCCH archive (previously having only covered the save file archives).
Long story short: Having tested link files on actual hardware, I can now confidently say that a simple tweak is enough to support them properly. Basing this on a scientific research process feels much better than taking a leap of faith and crossing fingers nothing will break because of it!
Did you know you can use Mikage with USB keyboards? In fact, this feature had been present since the initial release. The four 3DS keys A, B, X, and Y are mapped to the keyboard keys A, S, Z, and X, respectively. This is a bit awkward though: If you look at the physical button layout used by the 3DS, you'll notice for example the keyboard's "S" is on the top row, whereas in the physical layout on the 3DS it's at the very bottom!
This was particularly noticeable in Nano Assault, where you fire bullets by pressing any of the four buttons, and each button will shoot in a different direction. Suffice to say it was rather confusing if you pressed a top key on your keyboard, yet you would fire bullets backwards!
One thing the old button layout got right is the button order in clockwise direction: On the 3DS this is ABYX, and on the keyboard it's ASXZ. If you hold the 3DS in your hands, the X and Y buttons require your fingers to reach out ever so slightly more though - hence so should their keyboard mappings! As a result, I chose Mikage's new default mapping to be ABYX->XZAS.
This may sound a little abstract - so just give it a try! You'll notice it feels much more natural than before.
It's been a while! But with the fruits of the hard work starting to show, it's finally time for a new alpha build. I hope to publish this on the weekend, but certainly before the end of November. Stay tuned :)
This post was originally posted on 19 November 2019 for patrons, and released publicly on 19 December 2019. Support us to get early access to Development Updates and more!
2019-11-19 15:51:21 +0000 UTC View Post
Back when we started development on Citra, all graphics emulation was done using a software renderer: All rendering was done purely on the host CPU and hence is incredibly slow, since it doesn't even attempt to use graphics acceleration. The reason for that was simple: Debugging rendering code on the GPU is notoriously difficult, so back when the 3DS hardware wasn't understood well, the software renderer was ideal for quick analysis and was a flexible code base that could easily be adapted for new findings as they popped up.
Mikage actually had initially reused Citra's software renderer to get graphics showing up before I wrote the Vulkan renderer earlier this year. But these days, the benefits of a software renderer aren't quite as clear-cut: The PICA200 GPU is researched well including even the 3DS-unique features, but, alas, the problem of debugging graphics issues persists. Sometimes you don't even know whether a graphical issue is truly a bug in the rendering code or just a CPU emulation bug manifesting on the user's screen.
That is why I decided to revive the old software rendering code and adapted it for Mikage's new display logic. Along the way, I fixed a big performance issue in the clipping code, and implemented a couple of previously missing features. With this work, I could finally confirm that the missing flame effect as well as the "radioactive water" in Kokiri Forest in TLoZ OoT3D are indeed graphics emulation bugs, since they are magically fixed in the software renderer.
What's left is actually fixing these issues in the Vulkan renderer now ;)
If you've followed my emulation work, you'll know I'm a strong believer in testing as a means for ensuring long-term success of an emulator project. Having tests that confirm your understanding of hardware not only is good to factually verify how the hardware works, but also ensures that any emulation fixes you make at a later point won't break already tested behavior.
I've seen this problem in Dolphin plenty of times: Fixing one bug would cause other bugs to appear, seemingly out of nowhere. You would run into circles chasing bug fix after bug fix, yet it appeared as if you weren't making any actual progress since new bugs would pop up. Sometimes we were just working sloppily and introduced bugs when writing new code, but other times we in fact uncovered bugs that were already there and just hidden by other code that never worked in the first place. Hardware testing is a good tool of ensuring thanks to hardware testing
Unfortunately, I haven't always been successful at convincing others that putting effort into testing is worthwhile: Citra in particular has always taken a rather lax approach here; some features of the 3DS GPU have had utility homebrew programs written to selectively confirm understanding about their behavior, whereas for other features this was deemed too difficult. This approach is better than nothing, but it's not useful in the long run: None of the tests are automated, and some won't even compile anymore with today's tools. And then most nontrivial features aren't backed by tests, so Citra doesn't actually verify the things they most likely got wrong. Being a bit tongue-in-cheek, I like to say Citra doesn't *really* have tests, for these reasons.
Long story short, Mikage's mission being to do things right, I've naturally invested heavily in testing. The CPU and OS HLE implementations already have a testing framework in place, and in the past weeks I've been setting up something similar for the graphics side of things. I've already uncovered a few small bugs even though only few actual tests have been written so far.
You might now ask yourself: What's in it for you? Well, making the rendering code faster has a high chance of breaking things; in fact, the Vulkan renderer already has a couple of bugs not present with the software renderer as mentioned above. Optimizing the rendering code to be faster has a high chance of introducing even more bugs. Having tests in place enables me to implement these optimizations while making sure the rendered picture is as flawless as seen when playing the game on your actual console at home.
Once written, we want our tests to be in a state of perpetuity: They verify the behavior of an emulator at one point in time, and to reliably verify this behavior for (possibly years into) the future they mustn't change. After all, if we find the emulator starts failing a test at some point, who would we blame otherwise? Did we introduce a bug in the emulator, or did the test change in a subtle way and now doesn't actually describe the hardware behavior anymore?
And here comes the catch with writing testing code for the 3DS: Tests must be written as 3DS homebrew programs, which are usually compiled using a set of tools collectively called devkitARM, plus a few other bits. These tools are written by people in the 3DS reverse-engineering community and, naturally, are changed very often. Often times you can notice this easily since your code won't compile anymore, but other times things just silently change under the hood. In fact, if you compile any 3DS homebrew title released, say, 2 years ago, you'll find it won't compile anymore.
You may already see how this poses a major threat to reliable long-term testability of an emulator: Of the handful of tests Citra has, most of them don't work anymore! I personally was hit by this once, when the library ctrulib decided to silently swap the order of width and height parameters in various places, hence breaking various tests of mine and wasting a couple of hours debugging until I found out about the library change.
On top of that, devkitARM and libraries such as ctrulib or citro3d can be quite a hassle to set up in an automated environment; it begins with devkitARM using a rather heavy custom package manager (that is only supported on newer versions), and ends with versioning problems (ever tried using an old version of ctrulib? Good luck figuring out which devkitARM version goes with that!).
And finally, ctrulib and citro3d make very specific assumptions about how they're going to be used. Some things frankly cannot be tested with the layers of abstractions they impose on the programmer. In particular, this has made writing custom system modules a rather headache-inducing task for me in the past.
Don't get me wrong though, these tools are certainly good options for homebrew game development. But they do severely lack as a base for emulator development. That's why I've been working towards building an alternative toolchain for the 3DS, based on modern tools such as the package manager Conan and the build system CMake. Earlier this month, I've reached a first mile stone of being able to compile one of the 3ds-examples from scratch - literally: My setup bootstrapped everything starting from the compiler, ctrulib, various 3ds-specific tools, and of course the program source code itself!
I've been at CppCon, one of the biggest C++ conferences in the US, and presented blobify, Mikage's serialization framework! I wrote a post about this earlier this month, so if you missed it check it out now.
This post was originally posted on 31 October 2019 for patrons, and released publicly on 18 December 2019. Support us to get early access to Development Updates and more!
2019-10-31 08:38:52 +0000 UTC View Post
... or well, on YouTube, but who still watches TV anyway :)
I went to the CppCon conference in Denver last month to give a public talk on blobify, one of Mikage's building blocks. Blobify solves the central emulation problem of data deserialization: Whenever a running game "talks" to the emulated hardware, what Mikage sees is a sequence of raw bytes (i.e. lots of 0s and 1s). This happens for example when decoding GPU command buffers, when reading files, or when emulating system calls. To make sense of the raw data we get, Mikage needs to convert it to C++ data structures, and that can sometimes be rather nontrivial. That's why blobify exists - to make repetitive and error-prone tasks as easy as they ought to be.
As I mentioned in Development Update 4, a lot of time went into developing this library (which is actually open-source and on GitHub, by the way). My talk goes into all the technical details, so if you're keen on programming you'll find some nice C++17 gems in there. Either way, considering the importance to Mikage and the time I spent on this, I figured it's worth a shout-out on this platform.
Meanwhile, there's of course more progress to share on Mikage. Some of the items I've been working on will be summed up in the next Development Update, to be released later this month. Until then, enjoy the video :)
2019-10-16 17:54:38 +0000 UTC View Posttl;dr: Patrons won't be charged in September. Mikage development continues, but no alpha build will be released next month.
Since Mikage's announcement on 1 July, a lot of progress has been made. About 70 source code changes resulted in many games starting to draw graphics, and those that didn't got much closer to it. As Alex (the author of the PS4 emulator Orbital) put it: "damn i wish i could commit that much time into getting stuff fixed"
Alas, despite the strong and steady progress being made, many patrons expressed it hasn't been as visible as they'd hoped. That's the unfortunate reality of emulation: You spend hundreds of hours fixing things, and then it's that one-line change that gets everything working - but only once the groundwork has been taken care of!
That said, I do want to give people a better feeling of progress, and that's hard to do without user-visible improvements. Also, I don't want to rely on the understanding generosity of patrons who keep pledging month after month either. Hence, I decided to pause the campaign for the time being.
This does not mean Mikage development itself is paused - far from it! Active progress is still made on an almost daily basis, as the Development Update released today shows. See you again in October for more updates and, as always, thanks for your support!
2019-08-31 09:56:47 +0000 UTC View PostIt's time for a quick update on development progress again! But first, let me announce I'll be moving away from a weekly update cycle from now on. I became clear that a weekly schedule pushes development too much towards short-term progress that don't pay off well in the long run and hence keeps me from doing the more fundamental work that's required for the larger picture. From now on, Development Updates will be released on an irregular schedule, i.e. whenever meaningful progress happens as part of bigger ongoing development efforts.
With that said, let's take a look at what happened since the last update!
After getting this game to boot last month, I implemented a number of features used by SM3DL later during the title screen:
Of these, the ExeFS change required a lot of refactoring in the FS module code, but the other changes were fairly minor. With the fixes, the title screen goes a fair bit further now - note in particular the 3D graphics in the background!
Unfortunately, as soon as you try to start the game, Mikage will freeze up due to the lack of the Mii selector applet (and applet support in general). Implementing this will require a lot of testing effort, but the prospect of getting SM3DL in-game is promising!
The other big item on this month's list is blobify, Mikage's internal serialization framework. Serialization is the task of turning blocks of binary data into meaningful objects such as C++ structures, and vice-versa. This comes up when reading 3DS-specific file formats (such as CXI or CIA), when decoding parameters for high-level emulation of services, when interpreting command buffers for GPU emulation, and generally whenever the emulator core needs to access data structures contained in the emulated 3DS's memory.
Considering how common this task is, I grew tired of reinventing the wheel for every component it came up in. To automate all of this redundant work, I created blobify as part of Mikage. blobify uses some very advanced C++ techniques to automate serialization tasks as much as possible, and it has proven to be a very effective tool at that. To allow others to benefit from this, I decided to spend a couple of weeks on extracting blobify from Mikage's source code and turning it into a proper library that can be used in other projects, too.
I released the fruits of this work earlier today on GitHub. This reiterates my dedication to open-source development, and is a first (if small) step towards open-sourcing Mikage as a whole.
Despite the progress, my todo list keeps growing! It's a good thing though - I'm getting a more accurate picture of what's required to get more games running, and hence it helps prioritizing development accordingly. Here are next "big items":
This post was originally posted on 31 August 2019 for patrons, and released publicly on 17 December 2019. Support us to get early access to Development Updates and more!
2019-08-31 09:12:26 +0000 UTC View Post
You might have come across an interesting piece of news earlier this week: An (unofficial) Android image for the Nintendo Switch has been released by the switchroot folks! All basic functionality works, and the graphics driver even supports Vulkan. So of course, I couldn't resist giving it a try and checking how Mikage fares on this exotic setup.
Lo and behold, it works without any modifications! Unsurprisingly performance isn't great - it's actually slower than my phone - but super cool regardless ;)
2019-08-02 14:39:25 +0000 UTC View Post
The first Progress Report is out! As usual, thanks for your support :)
Another game made it to the title screen: Super Mario 3D Land! As usual, achieving this was "just" a matter of fixing the right things. In this case, a corner case in DMAs wasn't implemented (transfer sizes not aligned to 4 bytes). Implementing this case and stubbing the MIC module was sufficient to get the title screen appear :)
I've also taken a day to improve the Android frontend UX. With the upcoming August alpha release, you'll finally be able to load games from any storage location rather than having to move your images to specific folders on your internal storage. Furthermore, Mikage will now properly display icons for games in CCI format.
This post was originally posted on 24 July 2019 for patrons, and released publicly on 16 December 2019. Support us to get early access to Development Updates and more!
2019-07-24 12:19:11 +0000 UTC View Post
Another week, another update - making steady progress, and I've got some new pictures to show!
With these new changes, I've got Bravely Default booting! It crashes after creating it's save file due to more unimplemented functionality, but progress is progress :)
This post was originally posted on 17 July 2019 for patrons, and released publicly on 15 December 2019. Support us to get early access to Development Updates and more!
2019-07-17 12:49:38 +0000 UTC View PostAs you imagine, it's been a busy week - besides closely watching and replying to the announcement thread on reddit and getting my hands dirty in video editing, I actually managed to spend some time on development too:
I don't have any new screenshots to show quite yet, but overall I have a better understanding what's blocking some titles from reaching the title screens - so hopefully it won't be too long until we see them run properly!
This post was originally posted on 9 July 2019 for patrons, and released publicly on 15 December 2019. Support us to get early access to Development Updates and more!
2019-07-09 16:48:43 +0000 UTC View Post
People have been asking to see more of Mikage in action, so I've recorded a gameplay session of my favorite testing title yesterday :)
You've probably already seen it, but just in case you haven't - here's our recent announcement!
2019-07-03 09:12:42 +0000 UTC View Post