What and Why is Unit Testing

A major feature of v0.40 will be the inclusion of a unit testing framework. This will reduce the number of bugs that make it out of development, and help prevent bugs from reappearing once fixed. Since most of these improvements and changes will be invisible to the end user I thought I'd detail the what unit testing is, and why it's a useful testing practice.

Let's start with the what. A unit test is a piece code written to test a singular bit of functionality in a more complex piece of code. This test code includes assertions that define what the expected outcome is. If you run your test code and the actual result is different from the expected result you know you have a problem. As an example: the LR2 Business class includes a function to hire a person you have handed it. The unit test for the hiring function would check that number of employees has increased by 1 after someone is hired, that the person is given the employee role, and that their schedule has them showing up to work at the correct time.

It's important to note that a unit test is written to test a small section of code all by itself. This is in contrast to an integration test, which tests how different functions, classes, ect. interact in more complex situations. Integration tests are harder to write, harder to maintain, and much harder to automate. An example of an integration test would be to create a person with specific stats, hire them to your business, and advance time to confirm they are producing the correct amount of research/production/whatever. v0.40 is focused on adding unit tests, not integration tests.

Python includes a unit testing framework inside of it's standard library, appropriately named "unittest". Unittest makes it simple to write, run, and organize test cases. It's slightly trickier to set it up to operate inside of a Ren'py operation, but I've got that working and have all of the automation working now.

That's what unit testing is, now let's talk about why unit testing is useful. Unit testing is useful because it's often much easier to describe the intended results of code than it is to actually write the functional code. It's also much easier to describe the intended results of code a little bit at a time, instead of trying to capture the full results all at once. Using the same example of hiring a new employee, it's easy to say that the business size should increase in one test. A different test can confirm that that person is given the correct role when hired. A third test might check that they are paid some amount of money at the end of the day.

Unit testing is useful for checking through existing code, but it's real strength is in preventing regression bugs and in providing a framework for adding new code. A regression bug is a bug that is introduced into a previously existing bit of functionality. It is important to write new unit test cases that fail because of that bug before fixing it. If a future change would reintroduce that bug, or a similar one, it will be immediately flagged by your unit tests. Over time the unit testing suite becomes more and more comprehensive, and fewer bugs are able to slip through.

This approach of writing test cases before fixing bugs can be extended even further by writing test cases before even coding new features. This is called Test Driven development. Each new feature addition begins by writing test cases that describe the results of that feature. As mentioned above, it's almost always easier to describe the results of a feature than it is to implement it. Once the test cases are written you can implement the feature, knowing that if it passes all of the written tests it is finished. This relies on having good requirements, but it makes the development process much more reliable. It also ensures the project has a solid suite of unit tests going forward, catching bugs that might otherwise break features in the future.

All of this begs the question of why it's taken 40 major versions of LR2 to begin implementing unit testing. The first reason was inexperience - this is the first python project I've had that has grown large enough to require it for good code maintenance and the first time I've made use of python's unit testing capabilities (which were much more well developed than I was expecting!). The second is a desire to "keep things simple", even when it made things more complex. Unit testing is another layer of code that needs to be tracked, organized in some way, and tests themselves need to be maintained over time as feature requirements change. It's easy to tell yourself that you don't need a testing framework because you'll just "do it right the first time". Finally, writing test code just isn't as fun as writing new code, so it's a hard mental hurdle to commit to working tests. I know I'll save time and effort in the long run, but it's difficult to convince my monkey brain of that.

I hope that gives a good overview of what's being added in v0.40 and why it's important, even if the effects won't be visible to the player!