November problems and solutions
Added 2020-12-02 22:42:05 +0000 UTCAs I'm sure many of you noticed, November was a turbulent time for nopy. Not only did we hit a new record with 9.3 petabytes transferred, in mid November a series of very large files were uploaded which also proved incredibly popular. This resulted in almost triple our normal traffic, although the big issue was the sheer size of the files, slowing nopy to a crawl and even crashing a few servers in both the EU and NA cache cluster.
The Problem
The way the cache cluster works is to offload very popular files to a dedicated cluster, thereby improving speeds for popular files whilst also reducing load on the normal file servers. The cache cluster featured servers with 1 or 2 SSDs and 32GB RAM, and until now this setup worked reasonably well, whilst we did see some occasional issues hitting the memory limit, these were rare and temporary. Additionally, many of the SSDs on these servers were older models, due to our budget limits.
Unfortunately the perfect storm in mid November completely overwhelmed these servers. We very quickly hit the memory limit, and the legacy SSD storage couldn't keep up with demand, resulting in incomplete transfers for users. Servers with 2 SSDs favoured much better, but still eventually had issues with lack of RAM and legacy SSD speeds.
The Solution
Thankfully, over "Black Friday" we have been able to secure some good deals with a new provider to replace the entire EU and NA cache clusters. All servers in both clusters now have double the RAM (64GB) and 2x Samsung 850/860 Pro SSDs. Additionally, we've been able to secure 10Gbps dedicated ports for these 10 new servers, bringing our total theoretical network capacity to well above 100Gbps. We're also in talks with another provider about the US file cluster, which we'll hopefully have news about soon.
As a non profit service with a finite amount of resources, it's unfortunately inevitable that there will be turbulent periods where our capacity is maxed. Thankfully due to the incredibly generous support of our community, we're able to stay ahead of most of these issues, and upgrade before any real problems hit. Sadly that wasn't the case this time, and we suffered severe impacts to our service. However, the changes and upgrades we've made over the past week should put nopy in a much more stable position to weather the next storm :)
As always, thank you for your support, without which nopy wouldn't exist <3
Comments
Thanks for the support :) You can disable the alert the next time it appears by ticking the "don't show this again" checkbox, then clicking "no thanks". This setting will then be remembered in your browser. There isn't a persistent way to hide it, as that would have to involve some way of linking nopy<->patreon, which would undermine our commitment to privacy. If you don't want to wait for the next alert, you can try pasting this into the address bar whilst you're on nopy.to (remove the spaces before and after the colon) javascript : window.localStorage.setItem('hideDonationAlert', 'true');
2021-08-09 22:08:33 +0000 UTCOkay, now that I'm donating, how do I get the "Hi we noticed you're a real fan of NOPY, please donate" popups to stop happening?
Shremedy
2021-08-07 12:03:28 +0000 UTCNopy seriously great hosting site...
MrKnobb
2021-07-29 18:09:00 +0000 UTCWhat's up with NOPY? Slow DL speed and bad connections.
Vrijgeest
2021-07-24 08:16:09 +0000 UTCWell worth the support.
Sim
2021-07-24 03:46:35 +0000 UTCYou guys do a great job thanks!
Sim
2021-07-24 03:46:16 +0000 UTCThanks for the transparency. Keep up the good work!
2021-06-29 02:26:33 +0000 UTCAwesome service, just pledged to help out. Thanks!
2021-06-26 23:54:15 +0000 UTCIn the page with the file to downloads. You find a heart symbol. Click on it to show the list of way to say tank you. Patreon is included.
Bulb
2021-03-07 05:26:15 +0000 UTC
