Futureporn January 2024 Update

Thank you for your continued support! I have been fervently working towards delivering new features that have been funded. Presently, I'm working on adding multiple VTubers. 

Here's a sneak peek at the new Streams Archive page. 

This page is meant to display a summary of Futureporn's archive in a compact format. It shows all the livestreams Futureporn is aware of. This includes the date, the platform(s) the stream was originally on, and whether or not Futureporn has a VOD of that stream.

There are three possible values that display in the last column. good, missing, and issue. Good means we probably have the best VOD quality possible. Missing means we don't have a VOD of that stream at all. Issue means we have a VOD, but there is something wrong with it.

Clicking any of the entries brings us to the stream page.

This page shows us details of the stream, an archive status (good/issue/missing) and it gives us a list of all the associated VODs. 

This is a big addition which allows us to have multiple VODs per stream. The hope is that we collect more VODs over time from various sources, eventually arriving at the highest quality VOD to preserve for the future.

Streams Archive and Multi VTubers are slated to release in February. 

Development Challenges

Over time I've noticed that my normal development workflow has a lot of pain points. Because of Futureporn's complex system, and the rapid pace of change, sometimes it takes an hour or two just to get things set up at the start of the day. This issue has made testing and releasing new features difficult because my dev environment has been loosely defined and fragile.

I've spent some time modernizing Futureporn's dev environment to make it easier for me to get up and running. Previously this would entail opening up about five different terminal windows and typing a unique command from memory into each of them. It was error prone and took time to set up each time. Now I'm using docker compose, which requires only one command to spin up a dev instance of all the microservices that make up Futureporn.

IPFS Learnings

S3

One of the problems with running an IPFS kubo instance is disk space. Given enough time with an ever-increasing pinset, the disk will eventually fill up. Last month I ran an experiment to see if I could use S3 as a kubo datastore. S3 is a cloud storage API that many hosting providers use to offer storage that can scale without limit, so it seemed like a great idea.

The experiment started wonderfully as I saw hundreds of Gigabytes of VOD data being saved to the cloud. I left the instance alone for several days but then the complaints came in. Visitors weren't able to download certain vods or streaming was slower than usual.

I eventually tracked the problem down to S3 rate limiting. My kubo instance was making S3 API calls over 250 times per second, which exceeded the limit for the S3 hosting provider I was using.

My solution was to spin down the S3 kubo datastorage experiment, and migrate the kubo instance to a dedicated server with a 4TB HDD. For now, 4TB is enough storage for Futureporn's pinset, but this won't always be the case. Futureporn is rapidly growing and soon the need will arise to scale beyond 4TB. I'll be revisiting this one.

Stats

I recently improved Futureporn's observability using Prometheus and Grafana. This gives me a dashboard on my second monitor which shows me status of all Futureporn services.

With this insight, I learned that Futureporn IPFS cluster is tracking 562 Content IDs (CIDs), 100 of which are unreachable. I think they're unreachable because of past mistakes in handling the dataset. One mistake that comes to mind happened when I migrated the IPFS cluster to a new host and shut down the original before the new host was able to sync everything.

Another mistake I made was pinning the CID of the source VODs, but forgetting to pin the CID of the 240p encodes. I later realized the issue and came up with a script that re-pins all the CIDs listed on Futureporn's API, but that didn't restore everything because there were few cluster peers which still had the relevant content.

I'll be re-creating the missing CIDs from backup, and building in better checks to identify this type of VOD issue.

Over the years I've made a lot of mistakes with the IPFS cluster, but every time I fail I learn something, and the next time I make fewer mistakes. I plan on continuing the trend of improving and making services better.

Good Vibes?

On my radar is adding Lovense vibration detection and playback to Futureporn. This feature is a few months away, but I wanted to ask around to try and see if there is a popular format for sharing vibrator playback scripts. I'm admittedly a noob when it comes to teledildonics, so I want to ask and see if there is already a good format out there that I can integrate with Futureporn. Let me know in the comments below or over on Futureporn Discord.

Wrapping Up

We only get one shot in life and I'm shooting my shot building Futureporn. VTubers haven't been around for that long so we have a golden opportunity right now to build out the VTuber ecosystem. I'm grateful that I get to do this. Thank you to my patrons and supporters. What a time to be alive!

Have any ideas, feedback, or inspiration? Feel free to share them on Futureporn's Discord https://discord.gg/wrZQnK3M8z