Storage
Added 2023-08-18 19:28:02 +0000 UTCLike pretty much everyone that's worked with video as long as we have, we've built up a huge amount of footage over the years. In the first image, you can see some of our earliest external hard drives which are bulky and weird shaped. They have to be close to 20 years old at this point!
For the majority of the time we've been doing Half in the Bag and beyond, we've stored all our raw footage and edits on these Western Digital drives and when one fills up, we just move on to a new one. We've got a ton of these drives now! Way more than what you see in these photos. They're still great for storing everything and we still have all our raw footage from pretty much everything we've ever shot on them, minus a couple of drives that have just stopped working over the years.
A year or two ago, we finally got a NAS server to have a secure and complete archive of every video we've ever made all in one place. If you're not familiar with a NAS, it stands for Network Attached Storage and it's basically just a bay of hard drives that all work together to create one big storage pool. All our raw footage is still just backed up on those Western Digital drives, but the final videos are stored on the NAS (in a RAID 6 configuration so if one or even two of the drives fails, all the data is still safe) and another storage unit is synced to it via USB to create a backup copy of the archive. With this set up, we should be in pretty good shape of never losing anything important from a hard drive failure.
Before the NAS archive, if we wanted to use a clip from an older video, we'd have to dig through our dozens of external drives to find the right one that has what we're looking for on it or we'd just download the youtube version and use that. But now we have every video and every short film and every feature we've ever done for the past 20+ years all saved in one place in full resolution. Organization is fun!
Comments
All good advice. The NAS is on a UPS and we do have off site backup!
Red Letter Media
2023-08-22 14:29:18 +0000 UTCJust in case you aren't.....Keep your QNAP up to date, Software wise QNAP is really bad about security. (Less of an issue as long as you don't have internet services enabled.) You guys might want to buy a few 20TB drives. Throw them in here https://www.amazon.com/GLOTRENDS-Protection-Resistant-Photography-B86/dp/B09CYZPR3F/ref=asc_df_B01LXO6HLG And store them offsite. Just a thought as god forbid you guys ever have a fire or a tornado or something and lose that archive. I have a QNAP NAS that up and died on me a few months back. I only had 60% of 83TB backed up. I'm was able to import the storage back on a new NAS and am in the process of backing up to 5x Ironwolf Pro 20TB drives for offsite backups at an aunts house a few miles away. :) If something nukes my house and her's well I have bigger issues to deal with. PS also put your NAS on a UPS if you haven't. The file systems on QNAP do NOT like going down hard.
Kellic
2023-08-22 00:11:36 +0000 UTCMan guys thanks for all you do! I know that is not easy. Hours and hours of staring at a screen, hoping it works with no mistakes.
Mr. Rivers
2023-08-21 00:44:36 +0000 UTCOoooh, that seagate in the middle right. I had one of those; ordered a 250gb one when I was in Iraq to "borrow" music and tv shows from the morale drive. That thing will probably still boot up, but it's a pain as it requires an outlet for power.
David
2023-08-20 21:17:19 +0000 UTCAlso consider backing up the data to a cloud service like Microsoft Azure or Amazon Web Services. Since transmitting through the internet isn't realistic, the cloud service can ship you a storage device to load your data into. For example, Microsoft offers Azure Data Box and Amazon has AWS Snowball. After shipping the device back, you'll be able to see your data online in a few days.
Carl Simon
2023-08-20 17:55:39 +0000 UTCSend it to SPACE!
Tiago Menem
2023-08-20 16:05:16 +0000 UTCThat's so cool. I'd always wondered what your backup/storage system was like.
Paul Hackett
2023-08-20 10:26:44 +0000 UTCForever is a relative term. I specifically mentioned the SanDisk issue which is due to bad firmware (something that has impacted both SSD and HDD alike, think the Seagate issue in the mid 00's). I actually had about 1/2 dozen of those things which all failed. Usually, SSD failures occur because the same storage cell is written to too many times and not because the whole drive went poof. This issue has been (mostly) solved on drives produced for the past few years unless they encounter the high write rates typically restricted to main OS drives. Even then, this just results in total capacity shrinking over tens of thousands, or even hundreds of thousands, of write cycles. HDD's, in contrast, have a disk spinning mechanism and another mechanism to move the read / write heads across the platter. neither part can touch, and the distance between them is at the nanometer scale. Those types of moving parts can seize if left without moving for decades. The moving parts are also pressure sealed, but that seal is usually protected by literal sticker. The pressure will equalize over time and the adhesive on the sticker will break down. Modern depend on a helium-filled low pressure environment which means too much leakage will render the entire drive useless. High capacity, drives also store data in a stacks, like roof tiles or reptile scales. This adds another potential complication for long term storage. If you are using an SSD as an archive. First, writing data to an SSD once and sticking it into an editing timeline so you can scrub to find the clip you need, an SSD will be far more reliable. After 10 years of zero use, an SSD will still have six sigma level failure rates vs at least 5% for an HDD. After a century any HDD would require some maintenance before spinning it up. In contrast, an SSD sealed in the tomb of Rich Evens's children, in the great pyramids, would most likely work immediately so Jay could retrieve precious footage of Cameron Mitchel's first appearance on BoTW.
The Real Zam
2023-08-20 02:28:38 +0000 UTCDisagree on ssds lasting forever. They can shit out like anything else, though obviously for different reasons than a spinner. WD’s San disk extreme ssds are in the middle of a huge controversy about losing data.
PACMKEInk
2023-08-20 01:28:38 +0000 UTCGreat write up! Always like to hear some of the bts stuff
PACMKEInk
2023-08-20 01:23:08 +0000 UTCBravo, as an IT person I applaud your data storage evolution!
Eric Radtke
2023-08-19 22:44:25 +0000 UTCIf you look up in earlier comments, they replied that all their data is stored in two different places.
Jessie Zimmer
2023-08-19 20:53:41 +0000 UTCGiven that he uses Plex, etc, my money's on Jay.
Jessie Zimmer
2023-08-19 20:49:27 +0000 UTCThere's actually a really good one in nearby WI that was able to salvage a good 99% of my iMac when it died. I'll find them and put a link here. (Edit: I was wrong. It's in Lake Forest, IL so they're might be a better choice closer to the guys, but I do recommend them. https://www.datarescuemds.com/)
Jessie Zimmer
2023-08-19 20:46:19 +0000 UTCSo organized so pretty. What's the over/under on whether Mike or Jay wrote this post purely so they could nerd out about organization? (I say as a fellow organization nerd ...)
Jessie Zimmer
2023-08-19 20:10:46 +0000 UTCThese are the Black Spines I'd actually wanna watch.
Hidde Jansen
2023-08-19 20:08:20 +0000 UTCThis is cool stuff to see. You guys probably know you have a heap of nerd fans. I'd be super curious just how big the NAS is. It must be absolutely massive. I'm going to guess in excess of 150TB? Just be sure to ALSO back that up. Things go wrong sometimes!
Scott
2023-08-19 06:56:21 +0000 UTCSo. Basically you still have a copy of Diamond Cobra vs White Fox Spotlight, right?
SKP
2023-08-19 06:16:56 +0000 UTCWant to join in and say you need an offsite backup as well—Francis Coppola lost the script and production note files of his early movies, originals and backups, when criminals broke into his house in Argentina and stole them from his office.
Timothy Liebe
2023-08-19 04:39:47 +0000 UTCThe clips are a source of unspeakable power and they have to be reused. By whom? Top Men.
Bonorum Malorum
2023-08-19 03:57:48 +0000 UTCIs the server on site or somewhere else? Was there a lot of hardware support needed for such a setup? Seems like something I should think about for my storage as well. I assume you would reccomend it?
Analog Pill
2023-08-19 03:47:42 +0000 UTCChrist! I did wonder how y’all stayed organized.
Adriana Lee Lopez
2023-08-19 03:29:28 +0000 UTCWow that is awesome!! You guys are so thorough
sammydowntheblock
2023-08-19 03:11:22 +0000 UTCOnce your NAS reaches its useful life span you need to invite Wendell https://www.youtube.com/watch?v=Jqg1G78cH2A to guest star a BoTW and have him make you guys a new server :)
Ross
2023-08-19 03:06:26 +0000 UTCThat’s good because we all know what happened after Rich installed the sprinkler system in that fireworks factory!
Eric Anderson
2023-08-19 02:48:23 +0000 UTCDitto to the offsite/cloud advice. Even if you don't want to pay somebody to host your data, at least archive them to another set of drives/another NAS, and have friends or family hold onto them. Given the level of insane bullshit you guys get into, it's not impossible to imagine the sprinklers getting tripped and the NAS ending up soaked or in a dangerously humid environment.
Warren McHenry
2023-08-19 01:48:56 +0000 UTCBut when do I get recognition for listening to the “Mr. Plinkett’s Bee Bustin’ Service” commentary track? You asked us to tell you, Jay! You asked us!
Joel Smith
2023-08-19 00:27:17 +0000 UTCThat is rad, but please tell me you have offsite cloud storage like Carbonite or an equivalent service. God forbid you guys have a fire or some such in the studio and these drives are destroyed, all your hard work would be safe and sound so you can redownload to physical media.
Jared Blando
2023-08-18 23:15:19 +0000 UTCSo... somewhere amongst all those ones and zeroes... is the last remaining evidence of the re:view of diamond fox vs the white cobra... Breathtaking.
Connor Martin
2023-08-18 23:03:09 +0000 UTC“When the light is green, the drive is clean. The footage is incarcerated here in our custom-made storage facility.” I can’t be the only one who sees the Ghostbusters containment unit in the last picture of the RLM archive NAS. It’s just asking for them build a replica around it.
Joshua Cubbin
2023-08-18 22:55:41 +0000 UTCWas going to post exactly this. I'm a former network admin, the idea of a bunch of external drives as a backup gave me chills. Also Backblaze is dirt cheap for unlimited storage.
Lou Enriquez
2023-08-18 22:36:29 +0000 UTCIf you still have the dead drives, there are services that can pull the platters and get your data off of them. I believe even Western Digital offers this.
Lou Enriquez
2023-08-18 22:35:48 +0000 UTCYou see you guys do more than entertain you also educate as far as I knew, they were devices for aficionados of porn. Or so I heard her. You learn something new every day.👍🎅🏿👍!
Kendel Fargo
2023-08-18 22:17:06 +0000 UTCThis post has been flagged for copyright infringement by Deuandra Brown
Neil Peart, Lord of Drums
2023-08-18 22:06:02 +0000 UTCThe raw footage of Rich watching Diamond Cobra Vs. The White Fox must be preserved!
Antone Bajor
2023-08-18 21:58:06 +0000 UTCThis is the power of math people!!!!?
Vix S.
2023-08-18 21:39:58 +0000 UTCThis, obviously. In 30 years, I expect The Emperor Palpatine Surgical Reconstruction Center (this should be the name) salt mine archives to be worth far more than Disney's, at least culturally, if not financially. Oh or they could sell the naming rights for charity! In which case we should all donate to name it "Endless Trash."
Neil Peart, Lord of Drums
2023-08-18 21:36:04 +0000 UTCCan you guesstimate how many terabytes you're storing? This is very interesting!
mike bouchard
2023-08-18 21:32:04 +0000 UTCPretty cool! As an IT guy I wouldn't be doing my due diligence if I didn't say please setup an off-site backup if you don't already have one! Companies like Backblaze offer cloud storage for dirt cheap and you can get software that will backup there directly from the NAS. Keep up the cool vids dudes 😎
capnchamomile
2023-08-18 21:11:28 +0000 UTCI get two free streaming services through my phone plan. Otherwise, I don’t pay anything to cable or anyone else. I get to choose to back RLM, and I’m damn happy to do it. I like an update like this. Makes me feel like I’m a small part of making things better for the crew.
Jason DeMaria
2023-08-18 21:03:08 +0000 UTCThis is like the epilogue of Raiders, when they store the Ark with all the other crates of presumed weird stuff. Except instead of face melting, this'd fart wetting. Still, very interesting share.
Bill Wilson
2023-08-18 20:56:14 +0000 UTCHey, I build NASes professionally! Always nice to see folks move from the hellscape of loose external hard drives and switch to more friendly storage solutions.
The dood
2023-08-18 20:51:01 +0000 UTCI would love to purchase a DVD of "RLM Archive Collection Volume Two".
John Kott
2023-08-18 20:47:54 +0000 UTCRLM Archive Vol 2 when?
Ross Nugent
2023-08-18 20:34:18 +0000 UTCTony, I'm saying it is unlikely these guys shared these images and info in order to get a bunch of folks who know about storage solutions, but not about their specific setup, to offer solutions.
Katy
2023-08-18 20:27:31 +0000 UTCI mean clearly all your work needs transferring to fine grain 35mm archival film prints and stored in a salt mine to survive for future generations after the apocalypse, but I guess this is a fine temporary solution.
David Sarginson
2023-08-18 20:24:30 +0000 UTCThose drives belong in a museum!
SphericalKitten
2023-08-18 20:15:36 +0000 UTCNever thought about all of the footage you guys have.
Daniel
2023-08-18 20:13:58 +0000 UTCNeat! This got my wheels spinning again on building a home media server.
Not like the other ghouls
2023-08-18 20:12:27 +0000 UTCA word of warning. Old school spinning HDD's don't like sitting around unused for an extended period of time. The official parlance is 3-5yr reliable shelf life, but that's very pessimistic. Expect about 3% of drives to fail within that period. Want them to last longer? Store them in a cool, dry place (desiccant is your friend). Don't throw them around when you don't need to. Spin them up once a year. When you do use a program that can read the reliability data on the drive. Not sure what platform you guys run, but you can look for programs that can read "SMART" data off of a USB drive. If the drive gives you warnings or is acting strange, move that data to a better storage format. Want better reliability? SSD's will last forever and you can get 2.5in drives for super cheap, just steer clear of SanDisk as their SSDs are currently subject to a class action suit. The 2.5in form factor is also more reliable for spinning HDD's. They can also run off of USB power so you don't need to worry about a power brick, or the potential of a failure of the internal power supply inside the drive enclosure. The simplest and cheapest fix would be to put each drive in an antistatic bag with some desiccant.
The Real Zam
2023-08-18 20:07:42 +0000 UTCOk time to call level1techs and do a collab vid upgrading the nas and network infra. Would be amazing
Erd
2023-08-18 20:07:22 +0000 UTCSomewhere in these drives is the review of The Batman.
Jazz Jackrabbit
2023-08-18 20:06:53 +0000 UTCGood... Can't lose that Rich droid pleasuring clip!
Pierce Johnson
2023-08-18 20:06:34 +0000 UTCI hope they have an offsite backup.
c
2023-08-18 20:06:26 +0000 UTCMost of the replies seem accurate to me? Are you implying having an off site backup as nearly every data specialist recommends if uninformed?
Tony James Morris
2023-08-18 20:05:27 +0000 UTCGood call, always remember the golden rule of hard drives: eventually all hard drives will fail. It's only a matter of time, glad to see you've worked on your archives!
JTruts
2023-08-18 20:03:38 +0000 UTCCurious, how much usable storage space do you have on your new NAS?
Eric Darancette
2023-08-18 20:03:34 +0000 UTCAs a librarian, this makes me happy
Ryan Cassidy
2023-08-18 19:56:59 +0000 UTCAren't you glad you shared these images and in return got a ton of unsolicited and uninformed advice on how to manage your data?
Katy
2023-08-18 19:55:57 +0000 UTCI mean as everyone already said, you also need an off-site backup to go along with your NAS. With your amount of storage maybe you should invest in a bigger server with something like TrueNAS that can also house the raw footage. Then you can off-site backup that one, at that scale cloud is not really something you should think about, better get another NAS. Maybe think about ingesting data directly to the server, and when filming video from static camera maybe bypass recording to sd-cards and the like all together.
Karkhon
2023-08-18 19:51:35 +0000 UTCthis is awesome behind the scenes stuff! have wondered before how so many years of content is stored and organized.
Tom McGraw
2023-08-18 19:51:18 +0000 UTCprobably a good idea to sync to a second nas and store it off site. re sync periodically. or do tape backups and store them offsite. Natural disasters and fires and crap happen!
plop
2023-08-18 19:48:13 +0000 UTCthank you for that incredible insight! just installed a NAS for my sister today and i keep telling everyone to use such a system for being safe with the tons of data nowadays
Rudolf Rohaczek
2023-08-18 19:45:32 +0000 UTCNice! NAS is a great solution to those external drives. Hopefully you have backups off site, yes? ... Yes?
Rainbow Randolph
2023-08-18 19:43:16 +0000 UTCGet a longer USB and keep them as far apart as you can. If a pipe burst above then they're still both toast.
Minimag
2023-08-18 19:43:05 +0000 UTCI got a NAS a couple years ago myself. I had about 10x external drives, between 2-4TB each. Labelled them for contents, had backups of backups, etc... Now I have 70TB, with redundancy.
AshesBoomstick
2023-08-18 19:41:17 +0000 UTCI had one of those Seagate external drives! Mine was 300GB, which felt absolutely massive in ~2003, lol. Also, hard agree with your choice to move to a Raid6 array in a NAS. I have a 96TB array currently and it's fantastic. However, make sure you are doing Snapshots and/or Snapshot backups! I had a ransomware attack about 2 years ago, and had I been using Snapshots, I wouldn't have lost about 10% of my data to the attack. Now I have everything Snapshotted once a day, and backed up once a week. It's very comforting to have!
Michael Cafarelli
2023-08-18 19:41:11 +0000 UTCIn 2001 I paid $3400.00 for a 36 gig 10k RPM SCSI drive. The 10k drive was fast enough for full res. Capture. Storage is soooo cheap now by comparison. Good times!
troutki
2023-08-18 19:38:51 +0000 UTCJust thinking about doing this the other day. My drawer is filled with 15+ drives ranging from 4-12TB each and I HATE digging through them to find old shit.
Videopocalypse
2023-08-18 19:38:33 +0000 UTCOh man, my dad still uses one of those bulky externals. Haha.
Kersten Peterson
2023-08-18 19:37:06 +0000 UTCSifting through old drives myself while I read this, so I feel your pain
Drew Van Weelden
2023-08-18 19:36:58 +0000 UTCThat is so cool! Makes my "sort it and store it!" brain so happy to see.
Kerri Lewis
2023-08-18 19:36:26 +0000 UTCIt is an investment though... Currently working with about 30 1tb loose drives, bit of a nightmare but, money is an object.
nebuloider
2023-08-18 19:35:58 +0000 UTCI never knew that’s what the NAS in NASCAR stood for.
c
2023-08-18 19:35:57 +0000 UTCI wonder how much storage the raws/sources for an average recent episode take up. I make time lapse videos and the raw data can often be 100+gb/video.
Mortrek
2023-08-18 19:35:47 +0000 UTCI never expected to geek out at a RLM post, but I'm geeking out! Is this Josh's magic at work?
Andre K
2023-08-18 19:33:39 +0000 UTCNeed to invest more time into figuring out how to install Debian on my Buffalo NAS, so that I can install Syncthing on it to make syncing with other devices more seamless
Super Space Hero
2023-08-18 19:33:02 +0000 UTCSick as hell.
Izzy Folsom
2023-08-18 19:32:58 +0000 UTCI always wanted to ask this exact question. Glad you guys have a somewhat robust backup solution. I lost my home NAS to a massive hard drive failure and had to pay $1600 to recover the data, learned that lesson the hard way.
DamienC
2023-08-18 19:32:44 +0000 UTCWe don't but everything is backed up in two separate locations
Red Letter Media
2023-08-18 19:32:34 +0000 UTC3-2-1 3 copies of your data 2 different types of storage media 1 of those copies off site You should be able to sync your archive to Amazon Glacier for dirt cheap off site storage. Backups are great, but if you lose both in a fire, it does you no good.
chase shirey
2023-08-18 19:32:07 +0000 UTC"Why did everyone think AIDS was so funny?"
Mortrek
2023-08-18 19:32:03 +0000 UTCThose WD drives are great $/terabyte most of my datahoarding is done on those lol i really need to set up a NAS but im scared if i shuck them it will corrupt whats inside.
roach
2023-08-18 19:31:48 +0000 UTCAnd to think Johnny Carson had to use a salt mine…
Dendo Star
2023-08-18 19:31:25 +0000 UTCThis server will be the document of our times that archaeologists 10,000 years hence unearth and use to piece together what early 21st century civilization was like.
CSL
2023-08-18 19:31:09 +0000 UTCDo you guys have any cloud backups? I'd be worried about fire/water wrecking all that important history.
slipwrench
2023-08-18 19:30:35 +0000 UTCI love my NAS, was a great investment. I moved all my movie and TV rips to it also and use Plex to stream the files. my plex instance runs on another machine but some NAS units can run it directly
dogboydog
2023-08-18 19:30:32 +0000 UTCThis gave me a little bit of a boner, thank you for sharing. organization is fun and erotic
Chance Sparks
2023-08-18 19:30:16 +0000 UTC
