Guys, finally after 3 years of career in video production I landed my biggest contract yet - a breakthrough in my career, but I'm nervous as my knowledge about RAID arrays is very limited.
I will be working with 4K - 5K RAW footage. A lot of it. The final video will be 25 minutes. There is no way I can do this job with regular hard drives or even SSDs as their size is limited. I need at least 40TB of FAST storage. So I know that I need to get a RAID array system. But have no idea what I should go for.
From I can tell, the connection is really important. USB 3 may not be fast enough. Should I buy a Thunderbolt 2 PCI card and a RAID system that has Thunderbolt 2 output?
how much are you willing to spend? i hope it's a lot.
Brandon Wood
Whats your budget? Do you want to backup your data locally as well? 40TB of HDD storage are not cheap and HDDs are slow you could double it and have double the speed + a backup but it's still too slow for 4k RAW you should split the data and get maybe only one TB of fast storage
Ayden Foster
Why do you need 40tb? How did you come up with that particular number?
Best solution is probably some form of dedicated network storage device (SAN), which will be expensive. I know Dell sells some ISCSI SAN solutions which might fit your insane needs but they are around 5k.
Dominic Bailey
They haven't told me how much they are going to pay me yet. They asked for me rate, and since this is a HUGE opportunity for me and my career, I said compensation won't be an issue. I will do my best regardless of the money. And they really liked my attitude.
I think I will be able to spend $2000-$2500
>you should split the data and get maybe only one TB of fast storage
This might be the solution. I buy one or two 1TB SSD, and put the data I'm working with in it, finish the shots, then move them to a regular HDD storage.
hmmm. Fuck.
Landon Ward
>$2000-$2500
dude, that isnt even enough to cover 40TB of drives, let alone any kind of redundancy OR the fucking machine itself
get the fuck out
Matthew Harris
company I work for just ordered expansion for current NAS. 10TB of usable space, enterprise grade HDDs and on site support. Just around $6000...
Christian Russell
Fuck me
Aaron Fisher
whatever you do, don't make a RAID5. When you know the number of drives picked your HDD, check the LSE rate and throughput and put that info in here, it checks how reliable your rate will be on paper.
>They haven't told me how much they are going to pay me yet. kek cuck
Easton Sullivan
If you really don't care about backing up this fuckhuge raid system, you can get 4tb sas drives for ~$130 each Get a decent size pci raid card and go to town
Jacob Sullivan
Never used RAID, I know only the theory behind it, Just curious as to why you advise against using RAID5.
just skimmed it as the first sensible written google hit
Lincoln Price
You'd have to get 5 8TB WD Reds, but even that would cost you close to $1500 based on average price of a WD Red of this Capacity, if you counter in an extra drive for RAID5 for parity checking which is required as a minimum, a $2500 should do you fine
Contact EMC. I'm sure they can put something together for you.
Of the top of my head, if it were me, I'd throw together a box of:
4x 1TB PCIE SSD 8x 2TB SAS SSD 12x 4TB SAS HDD
Enable flashcache or some other tiered storage model, and connect via 10GIG-E.
Austin Scott
make sure you end up with a redunant RAID array of independent or inexpensive disks
Xavier Hill
hahahaha you mean RAID 50
Justin Davis
you're a fucking idiot
Brandon Ortiz
NAS... are you serius? He need a S Storage Administrator here with 34PB raw on hand, RAID 5 is the best with put sacrify lot of space for double parity
Easton Jones
>40TB of FAST storage >I think I will be able to spend $2000-$2500
You're fucking boned. You can barely get 40TB raw for that price, no way can you get that with any kind of redundancy, backup, and high speed connection for that kind of money.
Levi Sullivan
You're short by 1 zero.
Wyatt King
>I think I will be able to spend $2000-$2500
Toss on 1 more digit and you'll be alright.
Noah Torres
Yeah realized that earlier in the thread. I'm fucked.
I think I should just email and ask them about the budget. I still have no idea what-so-ever how much I will get paid. Maybe they will pay me tons. In which case I can get better hardware to deal with the storage issue.
Luis James
To do a 40TB RAID 10 for redundancy you will be 10 8TB drives, along with a capable device. If you want a NAS for a more simple enclosure, then here's what you need.
>that isnt even enough to cover 40TB of drives Buy recertified WD RE4 2TB drives. You can easily get that much storage if you have 20 drive bays for about $1000
Jacob Rivera
40tb lol op you can't roll your own on this. Contact some storage vendors ASAP. Would recommend Netapp. I hope your clients have deep pockets.
Robert Bailey
Simplest sane route:
> 15-bay or so NAS with 10GbE NIC, 10GbE NIC for your workstation > multiple RAID5/6 or equivalent volumes > some sort of offsite backup even if it's lugging HDDs in your backpack to your friend's house
Between RAID, 10% difference between TB/TiB, offsite backup, FS overhead, etc., you'll really need at least 2x the naive space requirement, so something along the lines of 20 4TB drives, and that's if you're not running a live remote backup with its own RAID.
Consider getting a loan for all of this, from family or otherwise.
Logan Perry
> I will do my best regardless of the money. >And they really liked my attitude.
How to get paid the bare minimum 101
Noah Cook
You could bill them for labor and write into the contract materials are purchased by them / upon receipt.
Nolan Diaz
op bro this is going to cost $150k not $2k.
Eli Martinez
Why the fuck do you need 40TB?
Hunter White
I will ask them.
Lots of 4K RAW footage. For a 25 minute show, they will probably give me hours of material.
John Cox
Then tell them to nut up the cash for you to work on it or do it in 1080p.
Asher Brown
You clearly haven't worked with 5K RAW
Benjamin Russell
>I will ask them. Did you bill yourself as capable of handling this? Because going back to them with, "I don't actually have any of the required hardware to do the work," is probably going to get you dropped. It's not your first customer's job to pay your startup costs.
Henry Scott
He's working with 5K RAW footage, and his result is 25 minutes. If you've ever edited before in your life, especially for businesses, you'd know the amount of input footage you have is often several magnitudes more than the final result will be.
Hours of 5K RAW are going to add into a shitload of storage space, then add in saved cuts from your workflow and you're going to be measuring in TBs, not GBs.
Juan Myers
Yeah, well the fast harddrive idea is just to make things work fast.
They will send me HDDs with all the media files on them anyway. So it's not like I actually don't have storage. They are providing that.
I just need a solution to be able to work fast.
BTW I'm editing in Adobe After Effects.
Julian Long
You can't just stick HDDs in RAID. RAID populates blank disks.
They're just handing you a bigass USB drive through the mail or whatever.
Robert Sullivan
The Synology is $1400 and 10 4TB Seagate drives will cost $1200 which is a little over budget for OP ($2600). Don't worry about RAID, just buy an extra drive and hope for the best.
Brandon Campbell
And, for the purposes of a REAL solution, I recommend hobbling yourself together a storage cluster.
Assuming you live in Nova Scotia like me, these will come to ~$375 after tax and shipping. Also, in case you didn't know, you can rip these open to get a normal 3.5" HDD.
You said your budget is $2000-2500, that would get you 5 HDDs for ~$1870.
Now, take these HDDs and put them in your existing PC (if you run out of SATA ports pay $50 for a SATA expansion PCI-E card). Load up Windows or whatever you are using and set up RAID 0 (or use a RAID controller, whatever).
Congratulations, you now have ~40TB of storage with relatively fast read and write speeds.
"But what if one of the drives dies!!"
High risk high reward faggot. If this job works out you can buy more drives and do RAID 10 or whatever.
I was going to recommend this seeing it at $300CAD on pcpartpicker. Seeing it at $215 free shipping is... crazy.
Mason Ortiz
Oh, you already have SSD RAID. What size are your big Intels? I'm assuming you can scrap your (likely) twin 128GB RAID and your lone 128GB Crucial to make room for more?
Kevin Foster
This seems to me to be a workflow issue. You shouldn't need to access that much data that fast. Maybe 1-2Tib of RAID then regular HDDs for storage with good compression methods.
t. Someone who doesn't know shit about RAID or video editing
Hunter Walker
I think user summed up my actual problem:
>This seems to me to be a workflow issue. You shouldn't need to access that much data that fast.
I should split my work into chunks. Copy the media I need to work on to my SSD RAIDS, once I finish, render the file, move on to the next.
Would be ideal to have a RAID system but seems like it will cost way too much.
Benjamin Turner
>Maybe 1-2Tib then regular HDDs for storage
Hey retard, you don't take a clients footage and store it on anything but RAID. Holy fuck how retarded are you? Kiss your entire business goodbye when a drive fails and your clients time, money and footage are wasted.
Also 1-2TB is going to store a fraction of what he will have worth of 5K raw video.
Carson Morris
>t. Someone who doesn't know shit about RAID or video editing
Yeah it really shows, considering you think it's okay for a career video editor to store client video on non-redundant storage.
Carson James
>40TB You're honestly probably best off renting the necessary computational power and storage from Amazon or something. You're looking at putting together a server with 10 4TB drives in it for just the base storage, let alone backup and parity.
Anthony Scott
This gives room to make a RAID10 later when OP has the money to buy another 10 4TB Seagate drives.
Charles Collins
>shit goes bad >"it's apple's fault!!!" >sues Apple >even if wins will never get another job in the future because your productor doesn't give a shit wise
Adam Peterson
it isn't hard to guess having the runtime and knowing the bitrate i guess
Aaron Hall
I said I don't know anything about video editing. No need to be hostile. I actually meant SSDs/fast storage, not redundancy. Misspoke
Charles Lewis
Good luck rebuilding the array with fucking 8TB drives.
Get a e5-2670, c602 board, raid card(or some pike), eatx enclosure, 10gigE cards for nas and your desktop, sas drives or best deal you find for drives on ebay
