I'll get to the punchline. I've got a lot of data, intend to get a lot more, and have come into possession of a load of hard disks. I was hoping someone could advise me on how best to set this up, namely how many RAIDs, which disks, and what levels.
I have 14.5TB data in total - the most sensible way to split it up is 8.5 and 6, but the subfolders split something like (3.5, 1.5, 1, 3, 5.5).
My disk layout is: * Seagate Desktop: 4TB, 4TB, 4TB, 4TB, 5TB * WD Red (7200rpm): 6TB, 6TB * WD Gold (7200rpm): 6TB, 6TB * Seagate Enterprise (7200rpm): 6TB * Seagate Archive: 8TB (PMR, so probably not suitable)
The best idea I've had so far is to hook up the 4x4TB disks into a 12TB RAID-5, and one of the two parts I mentioned earlier could go there comfortable expansion room; but from there I'm not sure what would be best - i.e. how best to utilise the set of 6TB disks. Maybe I could hook the 5 of them together into a 24TB RAID-5, then maybe later add a final 6TB disk to up it to RAID-6?
Of course I could also use ZFS, but learning curve and RAM utilisation aside (I'm running on a pretty low powered board), I'm keen to not be too "locked down", i.e. it should be easy to set the same disks up again elsewhere if I need to.
raid 6 if you can't ZFS, raid 5 is just not safe with large amount of data you'd still be endangered by bit flipping/data corruption, but at least you're safe from drive failure during rebuilds
Robert Wood
If I were to set up a 5-disk RAID-5 with the 5 6TB disks (4x6=24TB usable space), would it be straightforward (using mdadm) to convert that to 6TB later on if I got another 6TB disk?
Is RAID striping resilient against, say, plugging the same set of disks into a different USB-SATA controller which either do or don't bother to report the disk's Advanced Format (4096 sector size) correctly? (I don't mean the kind of controller shenanigans where a normal single-disk partition isn't recognised because these properties are actually being spoofed/scaled - I had a controller in a USB enclosure which did this once, and I couldn't get rid of it until I'd finished using it to empty its" disk)
Xavier Green
I don't know enough about the specifics to tell you correct information about this unfortunately
Juan Robinson
dont even fuck with raid. You get drunk and make one dumb shit move and it all goes down the drain. dont ask me how I know.
use the drives, make mirror backups, store them in a dry place with a moderate temperature on the cool side.
Isaiah Hall
>muhh raid
bitch please its le current year
Colton Walker
>If I were to set up a 5-disk RAID-5 with the 5 6TB disks (4x6=24TB usable space), would it be straightforward (using mdadm) to convert that to 6TB later on if I got another 6TB disk?
I'm really not sure what you're asking here. I assume you either meant to say "convert that to a 6-disk array later on" or "convert that to RAID6 later on." Either way, don't do this. Don't use RAID5 for this much data. If you have a drive failure there is a very high probability that you will lose your whole array in a rebuild, which stresses all the remaining disks. Use RAID6. The usable capacity will not be 24TB. Striping takes up space. A 5x6TB array in RAID6 yields about 18TB of usable space. A bit less than that once you add a filesystem.
>Is RAID striping resilient against, say, plugging the same set of disks into a different USB-SATA controller which either do or don't bother to report the disk's Advanced Format (4096 sector size) correctly? (I don't mean the kind of controller shenanigans where a normal single-disk partition isn't recognised because these properties are actually being spoofed/scaled - I had a controller in a USB enclosure which did this once, and I couldn't get rid of it until I'd finished using it to empty its" disk)
Assuming you're talking about mdadm/software RAID, then as far as I know it won't matter what controller is being used as long as the disks are all visible to the OS - mdadm will just reassemble them back into an array. I've moved my 21TB array between two completely different systems just fine. That said, why on earth are you involving a USB-SATA controller? This type of array should either be connected directly to the motherboard or via a PCIe HBA. Either use SATA breakout cables or a real SAS backplane would be best.
I advise doing some more research on this before making the wrong decisions at the outset, leading to problems or data loss later.
Evan Thomas
Bit rot never happens on modern drives you fucking retard fuck off to /r/datahoarder
Jaxon Price
>I've got a lot of data >I have 14.5TB data in total get a load of this guy...
Jeremiah Gray
where can I get cheap drives
Aiden Scott
>Is RAID striping resilient against, say, plugging the same set of disks into a different USB-SATA controller which either do or don't bother to report the disk's Advanced Format (4096 sector size) correctly? (I don't mean the kind of controller shenanigans where a normal single-disk partition isn't recognised because these properties are actually being spoofed/scaled - I had a controller in a USB enclosure which did this once, and I couldn't get rid of it until I'd finished using it to empty its" disk)
I guess you talk about what is called 4K emulation. Some older USB housings and docking stations did that to be able to handle HDD's with more than 2TB. The turnside of that was that HDD's with more than 2TB filled with such USB docks and enclosures couldn't fully be used when directly connected to an internal SATA-Prort of a motherboard. The containing files and folders were then showed normally as with any other internal SATA-Port filled HDD too, but you are unable to read everything that is outside of the first two TB's that is written on the HDD. When you for example doubleclick to open a folder or file from a once via USB 4K emulation filled HDD now connected to an internal SATA-Port, simply nothing will happen because it's not readable for standard SATA-protocol that doesn't understand 4K emulation. Despite the fact, it's showing you the file as existing in the file structure. I'm not sure you talking about this issue??
Jack Mitchell
I've got nearly 20TB of data, which really doesn't feel like much. I've got: - a home server that has 5x8TB drives in RAIDZ2. - a dedicated server with 5x6TB drives in RAIDZ. - backups of important stuff in Google Drive (secured with EncFS)
Every night at midnight my local server and dedicated server sync up with Unison.
Always always always have an offsite backup.
Camden Stewart
>>I'm really not sure what you're asking here. I assume you either meant to say "convert that to a 6-disk array later on" or "convert that to RAID6 later on." Yes, I did, it was late and I typed the wrong thing.
It was 8TB a couple months ago. I have fibre internet now, so it's going to grow like crazy.
>guess you talk about what is called 4K emulation. Some older USB housings and docking stations did that to be able to handle HDD's with more than 2TB. The turnside of that was that HDD's with more than 2TB filled with such USB docks and enclosures couldn't fully be used when directly connected to an internal SATA-Prort of a motherboard. Yes, exactly that.
Nolan Hill
Look into enclosures and HDD Docking Stations from the brand "Inateck" Their support (from what i heard) seems to be good and they have good documentation on their products in regard of the 4K emulation. If you mail them, they will tell you wich models will be the way to go.
I happen to have only Sharkoon and one IcyBox Docking Station. The newer 3.0 and 3.1 USB docks are without 4K emulation and follow the SATA standard fully compliant.
>* Seagate Archive: 8TB (PMR, so probably not suitable)
Also back up
Ian White
dont waste terabytes of storage on raid. real backups to tape is what you should do.
Michael Brooks
PMR is used everywhere, you're thinking of SMR which limits you to Windows for that drive because Seagate is an enormous asshole. Everything else will fry the drive. I've had good experiences with BTRFS "RAID1" mode, which simply stores two copies of the same file on two different drives. It lets you easily combine drives of different sizes and in odd numbers. It recently survived a Seagate Ironwolf randomly fucking itself. I'd use it over RAID5 in most cases. BTRFS "RAID5/6" probably works again, but I haven't looked into it.
Nathan Harris
> subfolders split something like (3.5, 1.5, 1, 3, 5.5) Hardware RAID + LVM on one big partition.
Assume you have five 8TB drives, is that 32GB raw data and 8GB "data"? Why and how does that work? What's even on the fifth 8GB drive?
Carson Thompson
See the wiki. Also, I will recommend against RAID5 on large drives, in case of failing rebuild will take forever and may cause a failure of another drive. RAID10 is more straightforward and may withstand up to 50% of disks going down.
Jackson Brown
Magic meme parity bits to save you from one HDD failure. When that drive dies, you replace it, and the RAID has to rebuild that drive. While rebuilding that drive, if at any point you hit an URE, or another drive dies, you just lost your entire RAID.
William White
Ah, yes, that was a typo. I meant that the Seagate Archive disk is SMR, hence the well-understood issues.
Zachary Hughes
The basic idea is that you use parity.
For RAID-5, If I have two data disks and a parity disk, laying each disk in parallel vertically might look like: D D P 0 1 1 1 1 0 0 1 1 0 1 1 1 0 1 1 0 0 0 0 0 0 1 1 etc.
Notice that the third column in this case is the "sum" of the data disks (truncated to just the last digit, which for 2 data disks is simply 1/0 based on whether the data bits are identical, hence "parity").
If you lose one of the data disks, you can "reconstruct" the missing data by solving a simple substitution using the surviving disk against the parity data. Of course, if you lose more than one, you've lost the lot.
In practice, it's all "striped", which means something like: which of the disks is used for "parity" keeps alternating in chunks.
RAID-6 and higher use slightly more complicated linear algebra, resulting in 2 or more "parity" bits (number theory, Galois theory, go ask /sci/ or Wikipedia), but the underlying principle is the same.
Jason Harris
So yes, RAID-6 survives 2 disk failures because of 2 parity blocks being used. The prevailing mindset nowadays is that RAID-5 is insufficient, since if you have 1 disk failure, replace the dead disk, and re-generate the parity/data across it, that'll take a lot of disk thrashing; and disks are so large now that the expectation that /another/ disk will fail in that process is no longer negligible.
Nicholas Roberts
Also, I made a typo on one of the example binary lines, (1 0 0 should of course be 1 0 1).
To demonstrate the "striping", you'd get something like:
D1 D2 P D1 P D2 P D1 D2 repeat
Or maybe it does it by cyclic permutation.
D1 D2 P D2 P D1 P D1 D2 repeat
I'm not sure which.
Colton Cruz
Your ATM card has a PIN of four digits, 1, 2, 3 and 4. You write these down on a paper because fuck remembering all that. However, meme magic might destroy one of the digits you wrote down, so you create a checksum 1+2+3+4=10 and write it down next to them. As expected, one digit disappears, so you have 1, 2, ?, and 4, but you know their sum is 10.
Joshua Cooper
BTRFS is the best option if you don't want to fall for the ZFS meme.
The best thing about BTRFS over ZFS is that it's flexible and copes well with variably sized disks. If you want to add drives that's cool, just add them and rebalance. You can change RAID levels just as easily.
As previous user mentioned RAID1 is safe. RAID5 now works again (in theory) but I could understand wanting to wait a bit before relying on it.
The latest kernel now has zstd compression as an option which should be just as fast as lzo but smaller.
Jayden Cox
You don't need too much RAM unless you enable deduplication.
Juan Lee
RAID is for uptime, not backups.
I keep most of my data on Seagate Archives. Once copy on the server, one copy offline in a Pelican case.
Just a plain robocopy once in a while.
Hunter Hill
> HDDs on a carpet > robocopy As expected.
Nathan Lee
Deal with it. The circuit boards are all still within the bags.
Jacob Miller
It depends on wich HDD models of these 8TB archive SMR disks you have. I for exampe have the ST 8000A0002 and there the SMR recording is internally managed. This model does not depend on a special operating system and runs on both Windows, Mac OS and also under Linux To my knowledge, there aren't any issues but i might be wrong on that (correct me if that's the case). I use mine under both Windows (7 Ultimate 64bit) and Mac OSX El Capitan.
If you happen to have one of the others that need an external (means operating system drive managed SMR) you better stick to Windows. But in that regard you will need Windows Server anyways Because none of the client versions are able to handle External SMR management without patches and stuff (huge fricklin, i don't recommend this)
I do myself have 28 of these 8TB archive HDD's and are very satisfied. They are slower in recording but cheap and realiable. To this day, none of these HDD's has ever failed for me. But no matter what version you have of these archive disks, they are not RAIDable in any case.
I use mine with the docking stations i described in my previous comment.
Julian Bailey
Depending on how often you update the data, look into Snapraid.
Zachary Green
Hmm, I was using snapraid before (that's actually what's currently on the 4x4TB disks, with the 5TB disk as parity), but to be honest it's just clunky and inconvenient. Especially when using something like mergerfs. I figure a RAID setup would at least give me some much-missed performance, as long of course as I also had a backup strategy.
Josiah Sullivan
RAID 5/6 isn't great nowadays.
BtrFS is a bit new and non-traditional but would work decently in this situation. OP: read thoroughly and understand the risks.
Misc suggestions: Obtain an unlimited Google Drive account (EDU or Gapps) and Rclone your shit over. Best Buy 8TB EasyStores are easily shuckable and (usually) contain a WD Red drive. I've seen them at $140/8TB. Sales can vary so be patient.
Do your research. /r/datahoarder and the-eye.eu.
Dominic Hall
>Best Buy 8TB EasyStores are easily shuckable and (usually) contain a WD Red drive. I've seen them at $140/8TB. Sales can vary so be patient. Are these external drives? enclosures? If yes, the price is amazing for a red
Hudson Collins
>Best Buy 8TB EasyStores are easily shuckable and (usually) contain a WD Red drive. I've seen them at $140/8TB. Sales can vary so be patient. This isn't very helpful when one lives in the UK.
Leo Moore
>individual docks They do 4-bay docks, you know. I have a couple.
Anthony Diaz
>$140/8TB breh
Easton Reed
would ordering from Germany be a problem for you?
Joshua Howard
They're out of the EU, so I guess it would.
Jacob Lee
yes, i know, but i will soon get over to a HDD-tower with 10 slots. each one independently to turn on and of
Why RAID when you can just gsuite backup? Use a GFS schedule.
>
Landon Morgan
Amazon glacier
Hunter Hall
Why hasn't anyone educated OP that RAID was designed for low capacity disks? No RAID solution is going to save you from a >2TB disk failure. The probability of another drive failing on rebuild is just too high to even consider RAID.
RAID is for REDUNDANCY, aka 24/7 uptime. It's when your boss wants a certain .docx file in the middle of the night on a Sunday and one of your drives just decided to shit the bed that same morning.
Jaxson Mitchell
globalists holding you guys rocksolid in. but ok, thats a topic for Sup Forums
As i know, you guys are still not out and that means there will be no import taxes.
that's the shop, where i use to order my stuff. Good service, fast delivery (scarily fast) and in case of trouble with a produckt they don't fuck around with you like some others.
Logan Campbell
>trusting your personal data to amazon
Dominic Williams
goooooooooooooooogled them on bestbuy and they go for 199 USD right now.
Benjamin Gomez
goooooogled them on bestbuy and they go for 199 USD right now
Andrew Collins
ORICO 4-bay docks are only ~70$
Jacob Russell
i replied to the wrong user/comment. Yes you are right. i have seen them on some websites
Jeremiah Peterson
Don't like the cloud either but ...
>passworded .rar archives >in cryptocontainers >pretty good secured
Charles Parker
I've got probably 30 or so drives lying around 2+ TB MINIMUM, what's the best way to expose them to windows? I'm talking minimum price per disk, with scaleability. works out at $25 a disk where I live, I'm looking cheaper (And haven't succeeded with any raid chassis to date) Fucking IBM bullshit, can't download the software anymore, support won't help, not compatible with this windows version blah blah blah
Nolan Gray
you need them on system all at once? Otherwise you can go for the 4 bay dock or a single dock like i have a several of and swap the disks if needed.
Jace Lee
>static proof bags >places content on the carpet
Ryan Gutierrez
Not all at once, but recovering some is a pain without anywhere to recover to
James Brown
Not sure everybody cares who knows about their smug anime girls folder
Colton Nelson
>not using GPG
Elijah Gonzalez
I have 4x4tb wd reds in mdadm raid5 backed up onto 3x3tb spanned lvm volume. Am I retarded?
Landon Howard
Look in the mirror and laugh as a mad scientist. Can you pull it off? Then no. Are you embarassing? Then reconsider the raid5 setup, it's rather large. I'd use BTRFS RAID1 on the four drives as-is (well, on a partition on each), skipping mdraid.
Liam Rogers
I lied, the 3x3tb are using btrfs, I used lvm for it in the past. I just think it's overkill to use raid1 and a backup on top of that. I used raid5 because it was an okay mix of speed bump and not losing too much capacity. It used to be raid10 but I started running low on space.
Jaxson Russell
Do what you want, you are a pirate.
Nicholas Fisher
I also tried to find a cheap as possible solution to connect all my HDD's to the computer via USB. But its' at least on an average about 20 to 25 or even up to 30 Euros/USD for each HDD to calculate with. Didn't find a way to get that cheaper done.
Jack Anderson
i lost my raid configuration because mobo raid sucks.
recommend me a hardware raid card
Sebastian Fisher
Why not software RAID? inb4 windows
Justin Hernandez
windows also i've never heard a positive word about it
Jack Perry
So what's to be used for data hoarding? Not OP but I want to be educated as well.
Colton Thomas
You should just get a job or something to do with your time and DL stuff on demand from private trackers like any non retarded person
Noah Wood
I never hear a positive word about hardware RAID these days. Hardware RAID makes you dependent on a specific piece of hardware (and the firmware it runs, etc), software RAID makes you dependent on a particular piece of software. Software dependencies are a.) something you have to deal with anyway, and b.) much easier to recover from in the event of a fuckup. Sure the hardware RAID card will be able to expose its array to any kind of operating system, but that's a much less valuable feature than just being able to yank the drives, put them in another machine running the same operating system, and being able to read them.
Noah King
>online >non retarded Thanks but no. There is no guarantee the jews won't close them as well.
Hardware RAID was very relevant until multicore-CPUs became the norm. These days parity calculation isn't that taxing and Linux offloads it onto SSE/AVX. What may be more relevant today is a good controller which can talk to all drives at once witout getting bogged down, although I think newer mobo chipsets have seen to this aswell. You're likely to only ever "need" a controller for getting extra SATA ports.
Bentley Foster
Whatever you say, Shlomo. No need to worry, goy.
Blake Williams
Use RAID6 or ZFS for disks that large, RAID5 is dangerous because a rebuild takes a long time and the stress might cause another drive to fail, thus fucking up your data completely. ZFS (RAIDZ2) is nice because it can detect bit rot, which traditional RAID6 solutions will not, as in files can become corrupted without the system being able to tell they are corrupted and without being able to identify which drive is bad, despite having parity. The disadvantage to ZFS RAIDZ2 is that you cannot add more drives to the same RAIDZ2 array, so your future expansion is limited to adding entire new arrays, not just individual drives to an existing array. That's more expensive and potentially less efficient in terms of the ratio of effective storage to redundancy, but it is also safer.
If you do use RAID, use software RAID (like mdadm on Linux), that way in case of controller/motherboard failure you can just reassemble the array on another Linux PC without having to worry about finding an exact replacement for the dead controller so the array remains compatible.
I would also suggest you dedicate some of those drives to backup in any case. I don't know what data you have, how often it changes and how hard or easy it is to replace a recent loss, but for instance I back up all data on my online array once a month on average. Most of the content is video, so if I do lose 1 month worth of data due to a catastrophic failure, it's not that bad and can be reacquired without major difficulty since it's recent and the change is likely to be small.
Keep your backup drives offline and completely disconnected. That way they are not vulnerable to software damage (bugs, malicious programs), user damage (accidental deletion) or electrical damage. Connect them when you run a backup, then disconnect them again to limit the risk as much as possible. That's what I do for a modest home setup, without crazy enterprise gear or crazy expense, beyond the actual drives.
Gabriel Reyes
>bashing RAID 5 >suggests RAID 6 Neither are recommended at such drive sizes
RAID 10, everything. >4x4TB=8TB storage >4x6TB=12TB storage, added bonus of having a spare 6TB as hotswap
Andrew King
>RAID 10, everything. RAID 10 is where you make volumes using RAID 1, and stripe those volumes together, right?
(as opposed to RAID 01, which would make volumes using RAID 0, then mirror them against each other using RAID 1)
Just trying to get the ordering right.
Is there a reason to do RAID 10 instead of RAID 01, or than just having 2 independent RAID 0 arrays?
Benjamin Jackson
It's the 1st example you gave RAID 1 on 'lower' level, RAID 0 on the 'higher' level. RAID 01 performs mostly the same in terms of throughput. Until a disk fails
In RAID 10 if a disk fails, you still have 3 disks that are usable for all intents and purposes.
In RAID 01 if a disk fails, the disk it was paired with also stops, just like in an ordinary RAID 0
Evan Gutierrez
>In RAID 10 if a disk fails, you still have 3 disks that are usable for all intents and purposes. > >In RAID 01 if a disk fails, the disk it was paired with also stops, just like in an ordinary RAID 0 Can you maybe work that out in more detail, using an example with, say 2 sets of 3 disks, when one disk fails how that affects the rest? I'm struggling to keep my head around it.
Oliver Flores
To put it differently >What happens when a RAID 0 fails? The other disk also stops working.
So what happens when a disk fails in a RAID 01? Same shit, pic related
Levi Hall
>RAID 10, everything. >everything This isn't a very good idea. Backups are required since RAID doesn't protect against hardware failure (PSU burns out all your drives), user error or damage caused by malware. It protects only against drive failure and nothing else, even then it doesn't protect against some drive failure scenarios like random corruption.
Joseph Morris
Well yeah, but at that amount of data, backup to disk isn't much of an option anyway. OP needs to look at tapedrives if he wants backups. Ends up cheaper in the long run too
Jack Young
>OP needs to look at tapedrives if he wants backups. Ends up cheaper in the long run too I just did some quick Google searching and I don't seem to be finding what you're talking about. Most drives are several thousand dollars, and the tapes are still ~30 dollars per TB.
Is there some insider knowledge I'm missing, e.g. which specific types of drive/tape to look for?
Alexander Collins
That's the price of hoarding. Unless ofcourse you download loads of useless stuff that you don't care about too much. Notice how never said anything about the 5TB and 8TB drives. You could use those to backup actual important stuff
Nolan Rivera
What? 10TB drives are a thing and he has 14TB worth of data. He has an 8TB drive which is unsuitable for RAID and a 5TB drive which doesn't match anything else in terms of size, so probably won't end up in an array anyway. That's 13TB already, he can throw that lone 6TB Seagate in the backup set too and already has enough space for a full copy of his data. It's not optimal to use 8TB, 5TB and 6TB drives for backup, but it most certainly can be done and it would still be much safer to use them that way than throw everything in a machine and get fucked if the PSU dies or something.
He can add more drives as the amount of data grows, but he already has enough space for his current needs. It's not going to be the easiest and nicest thing to manage either, but once again it's much better than literally nothing.
Adam Edwards
I'm still wondering where the 'ZFS eats RAM' meme came from. If you're not using deduplication, it eats as much memory as regular file systems from Linux do.
Even then, OP could benefit from using huge record sizes for stuff he expects to write once, and enjoy awesome compression ratios.
Christopher Moore
Mirrored backups. 1:1. Fuck raid.
Colton Smith
To be honest, after all this, this seems like the best option. Occasional mirrored backup to disks which are otherwise switched off and kept somewhere.
However, there's still the question of combining disks for convenience. If you're doing backups, what would be the difference between using LVM and RAID-0 for this purpose?
Luke Cooper
To make things confusing, it seems that LVM now does offer striping, making it at least superficially very similar to RAID-0...
Parker Murphy
Since it's all retail and jr helpdesk nubs replying, I will give my professional opinion.
RAID5 is not safe, as some others have noted. RAID6 is acceptable for uptime and "last stand" only. You have a write hole with 5/6 and you gain no write speed. There are counter points but even with an understanding, your mindset should be that these are off limits.
RAID10 is costly but you should be building everything else around this RAID level. A 10 shall allow you a full drive-count write speed gain, full drive-count read speed gain and shall permit one or more drive failures.
Assuming the model of drives of their respective capacities are all matched, you should put the Seagate 4TBs in a RAID10 for 4x read and write speed gain, minimum single drive failure (possibly 2) tolerance for a total of 8TB usable space from 16TB of raw aggregate space.
Use deduplication on the above RAID10 VOLUME. BLOCK LEVEL (for deduplication savings to be retained) backup to Seagate 8TB archive. Alternative for the paranoid - WD Red 6TB in RAID1 and backup 8TB RAID 10 volume to this Red 6TB mirror. You'd be 2TB shy but dedupe may allow you to shink 14.5TB to less than 8TB. YMMV.
Configure both SETS of 6TB WD's in RAID1 for two separate arrays of RAID1 volumes. I've never mixed Reds with Golds for a 12TB RAID10, cannot advise but if you want to be a guinea pig, I would like to find out if you experience problems but anticipate it would be fineish.
This configuration of arrays would give you 8TB+6TB+6TB=20TB of fully-mirrored, usable space. You can fit your 14.5TB across this, without deduplication or compression. It's performance and rebuild characteristics are superior to RAID5/6.
You have 5TB+6TB+8TB of total single drive backup space left.
8TB RAID10 Volume can be deduped and backed up to Seagate 8TB SMR Archive drive. 6TB Gold RAID1 array can be deduped and backed up to 6TB Enterprise drive. 6TB Red RAID1 array can be deduped and backed up to 5TB, less 1TB. Shuck8TB
Hudson Lewis
con't
As other anons noted, shuck 8tb's for 150 as long as they are not the SMR/archive drives. Leverage dedupe but there are caveats so read up. Do your storage properly, stick to RAID10 and always have a single drive that matches the capacity of your volume for backup.
Don't be a scrub and fall for the memeFS unless it REALLY suits you. Remember that Windows for all of it's flaws is used worldwide successfully for majorly large and important things. Specific advice? Install Server 2016 with ReFS and dedupe for your 3 RAID arrays. They're relatively new to Windows Server but work well, but are not perfect.
I have 40TB of media, files, etc. Every file since my first Windows 95 PC. It fits on a 32TB RAID10 and is replicated to my 96TB Dell SAN in my personal colo space. I'm not married and haven't had a girlfriend for a long time.
Chase Cook
Don't forget a UPS for your server. Helps against spike/surge and can shutdown your server proper in event of power failure. Don't even try to go with the whole " keep on working for 1HR+ during outage" thing, Proper shutdown is key. Keeps your raid arrays from rebuilding which can lead to data corruption. Nothing wrong with Raid long as power does not fail during write/read. UPS device solves this problem. If a array is rebuilding, LEAVE it alone till it gets done. don't access the data, don't write new data. if it takes 24Hrs to do a rebuild, then wait 24 hrs. Keep your backup offline when not in use, thus you prolong the backup drive life span and guard against failure from corruption due to power outage (plus, an offline device can't be hacked either). This is my setup: Server attached to UPS along with all network gear & backup nas units. If power fails, server will shutdown. NAS units stay shutdown when not in use. Server data volumes are on a 9TB Raid 5 array and on a 2TB Raid 1 array. (Sometimes it's easier to just add more arrays than to break existing array, expand it, let it rebuild and restore data from backup) Server data + Client backups + Server System Image is backed up to a pair of Nas Units. The data is split between them. Core data + System image/Client backups is backed up again to a 3TB USB External drive. Basically the NAS units will only be used in event of total failure of both arrays. For small shit such as restoring a deleted file, I use Shadow Copies/Previous versions that is built into Windows server so that way the main backup will not need to be used. Next year I plan to get a 12TB drive and stuff it into a USB3 Enclosure so I can have a 2nd complete backup of everything on my server. Also transferring all 12TB of data over GB ethernet in event of total failure would take little over a day. With USB3 this time is cut down to 5hrs.
Julian Adams
You heard me. $140 for a WD Red 8TB inside of an EasyStore enclosure. Limit 1 per order but you can make as many orders as you like.
Check the SKUs on the bottom of the box to see if they're 128MB or 256MB cache. Ordering online is about 90% 256MB's.
Running the usual S.M.A.R.T. / badblocks tests before you shuck them is always a good idea.
They arrive in plastic USB 3.0 enclosures held together by four plastic clips. I opened mine by shimming with guitar picks. You're gonna break the clips on the 1st you open but you'll figure it out. Once they're open unplug the small PCB and undo the four stub screws and the drives are ready.
Save the enclosures if you ever want to do a warranty claim. WD only checks drive SN and there's no SN on the enclosure, so throw it in a random shell and ship it off.
I've personally shucked about 50 so far with no issues. Had one DOA but BB swapped no questions asked.
There are rumors that they sometimes contain a white label drive but I've never seen one. Even if you did get one, just pack it back up and return it.
Michael Flores
Should also mention they're probably going to $120 or so for Black Friday sales.
I'm spending my Thanksgiving night in a van in a Best Buy parking lot to buy as many drives as possible for work.
Dominic Mitchell
How do sync my main storage on my back up drive? It takes too much time to delete all of the content and copy again.
On Windows
Dominic Scott
Yes. The drives go on sale about monthly and stay on sale about 4-48 hours or until they sell out. If you go to the store in person they usually have a dozen in stock.
Jose Nguyen
Offline disks doesn't sound convenient to me. I'd set up a script to mirror your data as it's added. If a drive burns up, swap it out and copy data from its redundant drive.
Ayden Williams
LTO-7 drive from Tandberg about 2300 Euros LTO-7 kassettes with 6TB space about 95 Euros each
Every generation of the LTO drives is backwards compatible one generation in reading out the older tapes. So if a new Generation comes out exemplewise LTO-8, i can Copy over directly (if i happen to have the money to buy a newer generation drive before i sold the old one) or i just swap it beween old and new by selling off the old drive and then via these money partly refinance the newer drive. Their transfer speed is pretty high.
I am using LTO since revision 4 and have updated to the newer iterations ever since they hit the market Sure, not the cheapest way to backup. Certainly not in the beginning, where you have to fully finance the first drive and kassettes. But when updating to a newer generation, you sell off the old drive for like half the price and are able to partly cofinance it with additional money.
LTO-6 drives are sold new for about 1400 Euros and you can get them used for much less than that. Saw them for 500 to 800 Euros on Ebay. They're mostly not run much at all and their mechanical properties are made to be fucking rock solid. These Drives are used to last the fuck out of everything other in the computer branch. Previous generation kassettes also happen to be much cheaper with time and are largely available because especially bigger companies with huge ammounts of data don't update their LTO's with every generation that hits the market.
LTO-7 can hold 6.25TB uncomprimied data and the next generation when released estimates to hold 12 TB per cassette. For datahoarders like me (collecting TV-Series and Movies in HD) this is the way to go, because my collection grows with an average of 100 GB per day.
