Nopy

The end?

Nopy started as a side project in June 2017, with the goal of creating a sustainable, ad free, privacy focused, full speed file hosting service. Despite some issues here and there, we're incredibly proud to say that we were successful for these past 4 years. We never imagined this weekend project would grow to serve 10s of millions of files and over 15 petabytes of bandwidth every month. Even more astounding is the level of support nopy received from the userbase, fuelling our growth and allowing us to maintain a high level 100% community driven service.

However, unfortunately we are now no longer able to continue operating, due to issues with payment processors. We use Patreon and Buymeacoffee for collecting donations, who in turn use Paypal, Stripe, and Payoneer for payouts. Stripe payouts are only available in a limited number of countries. Our PayPal account was abruptly banned last year with no explanation or option to appeal. Finally, we started having issues with Payoneer 3 months ago, with severe delays in reissuing an expired debit card, and suddenly a couple of weeks ago they completely blocked our account, again with no explanation other than "it's under review", and no further response.

This means, as of now, we have lost all ability to receive payouts from our fundraising platforms, so are unable to continue paying the $6000 server bills. We've looked at other options like other crowdfunding platforms (the majority use Stripe/Paypal/Payoneer), accepting credit card donations directly (ideally need a legal entity in US/EU for VISA/MasterCard), crypto (not widespread enough/sustainable), registering as an LLC in US or EU (having to deal with sales tax/VAT/international tax law/IRS/etc), and ultimately we've come to the conclusion that there is no realistically viable alternative at the moment. Nor do we want to sell or transfer ownership of nopy just to have it plastered in ads, completely undermining the very reason why nopy was created in the first place.

It's impossible for us to put into words how disappointed we are that nopy has come to an end, especially as it is due to third party payment processors, rather than issues with the service itself or a lack of support from our userbase. We will pause the Patreon campaign, and keep the code, database, and files backed up. If we are able to make a come back in the future, then we certainly will. Those with accounts are still able to login and delete files from the backup if you so wish. The code is unlikely to be open sourced, as things would need to be tidied up quite a bit, but we'll see.

We'd like to give a massive thanks to all the fantastic people who supported nopy throughout the years, nopy wouldn't have lasted more than 6 months without your generosity. Additionally, a final thanks to those who offered their help, expertise, advice and support out of the kindness of their hearts (especially PixelRepublic, Ryahn, and jimmy).

Cache situation resolved

The NA cache is now back online with 5 unmetered servers. I've also resolved an issue with the EU cache where some files could fail on certain servers.

We had to stick with the same provider we used before, whilst I had found some alternatives, they were ultimately still too expensive for our budget at the moment. I've also updated the current cost breakdown in the bio (currently $5700/m). As always, all donations are used for server bills, setup fees, new hardware costs, etc.

This unplanned upgrade has meant that we've had to postpone the planned NA file server upgrades, so that will now be next on the list once we have the budget for it, barring any other unforeseen situations that require us to upgrade our infrastructure.

Lastly, thank you very much for being patient whilst we resolved the situation, our Patreon and Buymeacoffee has seen really good support this month, despite the stability issues we saw. It's so incredible that the community is giving such support to this free service, I never imagined we would hit 15 petabytes transferred in a single month when we started nopy. I keep thinking that we're going to hit a ceiling, and we won't see anymore growth (either donations or downloads/traffic) but it keeps on going, so, thank you. ❤️

NA/EU Cache update

As you may have noticed, there has been some recent instability with the NA and EU cache servers over the past week. This is because we've had to temporarily redirect NA traffic to the EU cache, causing them to occasionally be overloaded at peak times.

TLDR: Hopefully the NA cache will be back online next week.

Detailed:

The NA cache server traffic wasn't "unmetered" from our provider, and instead had a capped traffic rate, with the option to upgrade to unmetered at a later date, as this is what would fit into our budget at the time. Over the past 6 months we've started hitting the cap and encountering bandwidth overages ($1/TB) on these servers, and in July we had a big leap and hit a $900 bill for excess traffic, making economic sense to pay for the unmetered upgrade on each of the servers. However, our provider refused to honor the previously quoted price and instead wanted over 3x more, which was way outside of our budget. So unfortunately, we had to cancel the servers to prevent further excess overages.

We've been talking with 3 providers (including our old provider but looking at a different datacenter), to get the NA cache servers replaced ASAP. Hopefully we can get them back online sometime next week.

Thanks very much for your support and patience whilst we deal with this ❤️

November problems and solutions

As I'm sure many of you noticed, November was a turbulent time for nopy. Not only did we hit a new record with 9.3 petabytes transferred, in mid November a series of very large files were uploaded which also proved incredibly popular. This resulted in almost triple our normal traffic, although the big issue was the sheer size of the files, slowing nopy to a crawl and even crashing a few servers in both the EU and NA cache cluster.

The Problem

The way the cache cluster works is to offload very popular files to a dedicated cluster, thereby improving speeds for popular files whilst also reducing load on the normal file servers. The cache cluster featured servers with 1 or 2 SSDs and 32GB RAM, and until now this setup worked reasonably well, whilst we did see some occasional issues hitting the memory limit, these were rare and temporary. Additionally, many of the SSDs on these servers were older models, due to our budget limits.

Unfortunately the perfect storm in mid November completely overwhelmed these servers. We very quickly hit the memory limit, and the legacy SSD storage couldn't keep up with demand, resulting in incomplete transfers for users. Servers with 2 SSDs favoured much better, but still eventually had issues with lack of RAM and legacy SSD speeds.

The Solution

Thankfully, over "Black Friday" we have been able to secure some good deals with a new provider to replace the entire EU and NA cache clusters. All servers in both clusters now have double the RAM (64GB) and 2x Samsung 850/860 Pro SSDs. Additionally, we've been able to secure 10Gbps dedicated ports for these 10 new servers, bringing our total theoretical network capacity to well above 100Gbps. We're also in talks with another provider about the US file cluster, which we'll hopefully have news about soon.

As a non profit service with a finite amount of resources, it's unfortunately inevitable that there will be turbulent periods where our capacity is maxed. Thankfully due to the incredibly generous support of our community, we're able to stay ahead of most of these issues, and upgrade before any real problems hit. Sadly that wasn't the case this time, and we suffered severe impacts to our service. However, the changes and upgrades we've made over the past week should put nopy in a much more stable position to weather the next storm :)

As always, thank you for your support, without which nopy wouldn't exist <3

Servers, servers, and more servers

With the amazing support we've seen over the past few months, we've been able to go on a bit of an upgrade spree. We now have a total of 49 servers online, in 9 locations, across 7 countries, and 3 continents. Our bandwidth capacity is now over 65Gbps!

Late July saw the new NA cache cluster go live, which added around 20Gbps of capacity for North and South American traffic.

Earlier today the SEA cache cluster went live too. Based in Singapore, and with approximately 4Gbps of capacity, this new cluster serves both SEA and Oceania.

Today also saw a couple of experimental EU cache servers go live (eu-cache1 and eu-cache2) which each have 4x the capacity of our other EU cache servers. This will give us better bursting capabilities at peak times. If this test is a success, we'll be looking to replace all of our EU cache servers once the next funding goal is reached.

Part of what sets nopy apart from other hosts is our cluster design. Our file clusters are comprised of traditional file servers, with each server storing a particular file. We have file clusters in both NA and EU. Once a file has been downloaded a certain number of times in a short period of time, future downloads for that file are served from our cache clusters, which are located in EU, NA, and now SEA. This allows popular files to be served from multiple servers at once, increasing speeds for that file, whilst also not slowing down the original file cluster. This is explained on our FAQ page, which also has some other information if you're interested in any of the technical aspects.

The past 5-6 months have seen us double our bandwidth capacity, whilst also serving record traffic (almost 7 petabytes in September!). Thank you again for all of your support. We'll continue to add servers and improve for as long as our awesome community is behind us :)

Finally, another shout out for the discord server! https://discord.gg/dzVC8qG

Goal reached, upgrades & Discord!

Thanks to reaching the goal, we're currently expanding with a new cluster in North America (all of our other servers are in Europe), adding approx ~10-15Gbps of capacity dedicated just for North American traffic, which will not only increase speeds for NA users, but move traffic from Europe making it faster for everyone. This additional cluster is quite a technical challenge, complicating our setup and infrastructure considerably, but we're making progress :).

Regarding interruptions and slow downs, especially at peak times, these issues are mostly localised to the file cluster, and is IO (HDD) related. To combat this, we're gradually replacing legacy file servers that have mechanical hard drives, with new, SSD based servers. This is providing more consistent speeds, preventing speed drop off, and should also solve the occasional download failures. So far, file servers: 2, 5, 8, 9 and 10 have been replaced. The remaining 10 servers in the file cluster will be gradually replaced over the coming month or so. Additionally, cache15 was replaced yesterday after a motherboard issue.

On top of the above changes, we've made further adjustments to better support download resuming. So far, these changes are only active on the new servers (server2, server5, server8, server9, server10, cache15) but will be rolled out across the rest of the servers soon.

April to May saw an increase in traffic of almost 1PB, with 5.6PB served in a single month. This is an absolutely insane number, I never dreamed nopy would reach this kind of traffic, and without the generosity of our users it wouldn't have been possible <3.

Finally, I've setup a discord. I'm not sure how much this is going to be used, but I think improving communication is always a good thing - https://discord.gg/dzVC8qG 

Upgrades, updates & more

The past few months have seen quite a few changes with nopy. In July, the new design was launched, along with our somewhat mesmerizing live status/statistics page. In September, we added an additional 5 cache servers, bringing the total to 15. November saw us replace our legacy file servers with servers on a better network giving us improved speeds and reliability. To reflect these changes, I've updated the patreon page with a breakdown of our exact costs per month.

All of these upgrades are only made possible due to the generous support of everyone here and on buymeacoffee, a massive thank you to you all. Nopy was started with a simple idea and a single server, and now, thanks to our donors, we have grown to over 30 servers.

Over the coming weeks we'll be experimenting with a new file caching configuration on a couple of our servers, this should lead to improved speeds, but there is a trade off with RAM and IO usage. If the tests prove to be a success, we'll be rolling out this change across all of our servers in the new year.

May growth, upgrades and news

May was a big month, thanks to the generosity of our supporters on Patreon and Buymeacoffee, our server costs are now covered! May was also the first month we transferred 2 petabytes of data (with over 1.8m downloads)!

With our current costs being covered, we can now start looking at upgrading our current infrastructure to improve performance and reliability. These upgrades will likely start manifesting late June.

We've also been working on a small redesign of the main site, refreshing the look and feel slightly, as well as revamping the statistics page to go into a bit more detail. Rest assured, the minimal feel isn't going anywhere, you can see the attached image for a preview. There isn't an ETA for the design going live yet, but will keep you updated.

The team at Jdownloader have recently added support for nopy, so if you're in need of a decent (and free!) download manager be sure to check them out.

Lastly, nopy couldn't exist without the kind support of all our donors, we're ecstatic that we've been able to cover our operating costs through donations alone, and this goes to show that free file hosts can exist without bombarding their users with ads or imposing limits, thanks to the generosity of others.