Infrastructure as code

I've had this week off from work so I've had a little bit of time to work on various parts of SondeHub. Most of the work I'm doing has been around cleaning up our terraform configuration. Terraform is the tool we use to manage everything that's deployed in AWS. 

Our infrastructure has lots of moving parts, gateways, functions, CDNs, servers and everything in between. Each little aspect of our environment has many many many configuration options, like Python version, function name, memory usage, API path ect...

While I can happily click through the AWS console and build out a bunch of resources it hides how I configured it and makes it harder for anyone else to make changes. By using Terraform we can write all the configuration in a single spot, and start mapping the dependencies between the services we use.

Although not very practical here's a map of some of the SondeHub infrastructure as viewed by Terraform. It's a good example of how GUIs sometimes hide the complexity of the systems we are building.

So what's next in this space? There's a few resources that we don't have setup in terraform yet so I'll be working to add these in, followed by developing a build and deploy pipeline for this configuration. This will make it a lot easier for our team to make and test changes. The ideal place we want to get to is being able to deploy out your own SondeHub infrastructure for development.

Other updates

We are also working on a few other bugs and quirks such as the launch site predictions, including transmitted frequency (eg the frequency that the sonde is reporting vs the user) and some time formatting issues. 

Another focus we are planning is handling different time scales. The observant SondeHub users will note that some radiosonde data seems to be from the future, by about 18 seconds. Currently SondeHub stores time based on the reported time from the radiosonde, however this typically doesn't include any leapseconds - unlike UTC. We'll be exploring the best approach to this soon and is quite a complex problem. 

Once again, many thanks to Mark and Luke for their amazing work on SondeHub!

~ Michaela