Power goes out

>power goes out
>all data is corrupted
Wow, what a great file system.

Other urls found in this thread:

h-online.com/open/news/item/Kernel-developers-squabble-over-Ext3-and-Ext4-740787.html
twitter.com/AnonBabble

Werks in my machine

Absolute bullshit. Where did you hear that?

At most the current unfinished write is gone.

This desu. Lies and slander.

What's wrong with NTFS? Works on my machine

>2018
>lying on an anonymous imageboard

>being a poorfag
>not being connected to a UPS
Don't blame the filesystem for your own incompetence.

>small home server
>power goes out
why would YOU LET this happen? these sorts of things are easily preventable.

xfs has been deprecated since ext3's dir_index

Of course it werks, it's one of the main filesystems of big data hosting.

It's really really rock solid, performs well and -also an important thing- you can turn features off if you have them in the layer above.

Absolute nonsense.

So I was dicking around in Gparted the other day with a USB drive, and put XFS on it without really reading the wiki page. What are the advantages? I've been using it as a dedicated *chan related images drive and I haven't really seen any difference between it and ext4

This is why laptops are superior. They have their own UPS.

>he doesn't use btrfs

I've had problems with file data not being sequenced with metadata. It's a very common procedure to produce a new save-file under a name, and then rename that new file to the intended name, atomically overwriting what was there previously. XFS has a problem with data since filesystem metadata is journalled, but not the data in the files, so when power has gone down, I've had problems with those files being zero bytes since the data was lost in cache, but the previous file was still overwritten.

In order to prevent this, you apparently need to sync the file contents before overwriting the file, but that kills performance in my application since syncing takes time. I've instead had to take to first linking the old file to a backup name so that it's still left in case of a power outage, which works, but I arguably shouldn't have to.

Doesn't really help when the UPS itself goes bang.

I have two UPS in serial order to prevent just that because I'm not a fag.

>It's really really rock solid, performs well
So is/does ext4.

>in serial order
So when the one closest to the computer dies, you're still as bad off? Great idea.

>cow filesystems
>anything using random-access files
Choose one.

Two power supplies, one connected to each UPS.

BRING IT BITCH

>CPU dies

>hard disk dies

>never a problem because raid1

from a quick look at the wiki page it's mostly just specialized to get good performance for fuckhuge enterprise servers with dozens or hundreds of tb of space
also has some built-in features to get the maximum advantage out of raid striping, which is again more of a concern for big enterprise shit

There's a reason why red hat moved to XFS as their default FS in RHEL7. BTRFS a shit

>not raid0

>tfw not having saved that picture of the 24 tb raid0 array with ded drive

...

Hello, did you time travel from the year 2008?
XFS doesn't eat data (so much) anymore.

I'm skeptical about this since the metadata shouldn't be written to the disk before the actual data. At least that's the common sense (ext3/4) approach.
If you're feeling adventurous you can change the cache type of the disk to write-through so writes go right to the disk but reads still get cached.

>At least that's the common sense (ext3/4) approach.
Indeed, but it is not the XFS approach. I have been bitten by this personally (multiple times), it's not some retold story.

Excited for bcachefs continuing to develop. If I had more free time I'd love to contribute to it, the project is really shaping up.

raid0 does literally nothing to protect against hdd failure
it only increases read/write performance
however there is no reason to use raid1 over raid10 in any system with at least 4 drives

sounds like ext's data=writeback mode that would truncate any file open for writing on power loss until linus called them all a bunch of dumbfucks for prioritizing artificial benchmarks over real world scenarios and they changed it

>power goes out

No filesystem can strictly recover from all power cuts. Anyone who tries to sell one as such tries to sell snake oil. Even COW and log-structured filesystems are not safe. Only UPS and clean unmounting can preserve the filesystems in that case.

I thought data=ordered had always been the default.

That's not at all true. Some disks may have caching issues that breaks filesystems that use them, but that's the exception, and the disk's fault.

i can't remember but based on h-online.com/open/news/item/Kernel-developers-squabble-over-Ext3-and-Ext4-740787.html it sounds like writeback was going to be the default for ext4 until linus verbally harassed ted tso, but i definitely remember ext4 eating my dot and log files during that time

Ext4 is also a good filesystem. If you use that and see no problem, keep using it.

XFS is still great to even important to have in many cases. Ext4 is for example not necessarily the best choice when you put a distributed filesystem on top.

>Ext4 is for example not necessarily the best choice when you put a distributed filesystem on top.
Why not?

I do remember that, but I don't think that was related to the data option, but just delayed allocation doing something similar to what data=writeback was doing, originally.

Is that a Japanese Anderson Cooper (CIA Spook)

I've never had any problem with xfs or ext4 using them as big data/media drive. btrfs was the first linux FS I tried and yeah... it had problems.

>Ext4 is for example not necessarily the best choice when you put a distributed filesystem on top.
Could you elaborate a bit on this? Why? and maybe what mount options would make xfs better for DFS?

It is.

ZFS is shit too, EXT4 is superior.

XFS is better than EXT4 in general but especially when you have a lot of files or very big files (like >=1GB). It has bigger filesize limits, can handles larger volumes and handles them better, supports multiple cores/threads better (especially beyond like 8 cores/threads IIRC)

For smaller volumes and small files EXT4 is just fine though.

>ZFS is shit
No nigger, the bent-over GNU-compliant Linux version is shit, as everything that's bent over to comply with (((Stallman)))'s whims

I don't understand why nu-Sup Forums always suggest raid-0 like it's the first time they heard about it. Raid 0 is for increasing read/write and not for redundancy. It's not the first time i've seen this here when people talking about redundancy. If you mean raid 0+1 sure that's redundacy.

Maybe someone on Sup Forums knows shit about storage:

Why don't home data hoarders run lvm+mdadm (which is being replaced by lvm alone) to make RAID arrays and then run ZFS on top of that to protect against silent file corruption?
Keep in mind that mdadm could silently corrupt your data since it does not keep checksums and does not verify consistency when reading, it only protects against disk failures.

You would have all the data integrity guarantees ZFS offers and could freely expand your RAID array as needed. The only obvious downside I can see is that if ZFS does detect data corruption you would manually need to recalculate parity on the affected sectors, but that should be a very rare occurrence and the data would still be safe.

The whole reason you choose ZFS is to not micromanage.

Micromanaging your parity and shit is what led to ZFS becoming a thing in the first place. People don't know what the fuck they're doing and scramble data on bad mediums.
No one wants to deal with figuring out what sector hole corresponds to what file or what device is sporadically dropping out of the array when a specific SCSI command is fired and restoring manually from an ealier version if they don't have to.

If it all heals on the fly it's a lot less pucker factor when you see chcksum errors.

Plus the whole: Let ZFS do it's job, brainlet.
The code base is better equip than you are to deal with failure routes without shredding all the files contained within. The more you try to fuck with it the worse it gets.

I use btrfs for my servers, and then store the snapshots on an emulated freebsd server within one of the servers that has direct access to the disk hardware.

best of all worlds.

That's why you safely shutdown the server when it loses power and you buy a UPS that can handle such a load in the first place. Also don't talk to me about the battery - if you had ever owned or managed a UPS you'd know how insanely annoying they are once the battery is dying

I work for a NAS company. I've seen far worse than this. Harsh lesson to learn but at least it's not this harsh for most people.

> Why don't home data hoarders run lvm+mdadm (which is being replaced by lvm alone) to make RAID arrays and then run ZFS on top of that to protect against silent file corruption?
ZFS is slow and relatively resource hungry.

That one flipped bit in 2 Exabibyte or whatever isn't generally enough to upgrade computers for ZFS.

> Keep in mind that mdadm could silently corrupt your data since it does not keep checksums and does not verify consistency when reading, it only protects against disk failures.
A scrub on RAID6 should actually catch the error if it happened on one disk rather than, say, in RAM.

I agree that it is not a good choice for high-availability scenarios, but for the home user the overhead of running a full check when a checksum fails is minor compared to the cost of needing to buy all the storage you will need upfront.

When you receive a notification that a file has a failed checksum just scrub the array and try again, and that could very well be automated. Yes it is slow, but considering how rare it is, not a problem for the home user.

>A scrub on RAID6 should actually catch the error if it happened on one disk rather than, say, in RAM.
Only if you manually scrub the drive. If you are using, lets say, EXT4 on top of mdadm you will never know a bit flipped until you do scrub the array and then god knows how long there has been corrupt data going around, possibly even overwriting older valid backups.

>Only if you manually scrub the drive
Not really manually, just run the scrub at a convenient interval. Weekly? Bi-weekly? Set it up as systemd timed service or as fcron job or something so it runs regardless if the machine was up or not.

In some distros the job is even installed together with mdadm by default.

> you will never know a bit flipped until you do scrub the array and then god knows how long there has been corrupt data going around
Only if you never scrub. The machine does it at convenient intervals however, so you're good.

> possibly even overwriting older valid backups
I propose that you might use rolling backups like with borg or bup so you have an inexpensive version history. Or lvm snapshots or such if you prefer.

Yea, maybe at some point you got the bit flipped. The backup prior to it flipping and next backup after the scrub will fix it. Plus you got mitigation against all sorts of errors.

Much smoother in many ways than the constant higher access latencies and extra resource consumption, unless you got 24/7 steady load and can't just scrub anyhow.

>yet another anti-linux shill thread

Microsoft is having a party today...

>comes to thread late
>feels like he's being personally attacked for using xfs
>doesn't bother to read any of it
and what does MS have to with ZFS?

Still, sounds sub-optimal to me. You are potentially running for a full week with corrupted data which is not acceptable even for some home users (for example, subtly corrupted datasets during development).

What I think is a even larger problem than that is running a raid scrub with such a high frequency. On arrays with more than, lets say 24TB, that is a very stressing operation on the drives and could cause premature failure.

Using ZFS of top of mdadm still sounds like the best scenario for a home user. You do not risk using corrupted data, have lower stress on the drives since you don't need to scrub as often and if you are builing a RAID array the extra cost of some ECC ram and a decent server board is the same as two drives at most.

>You are potentially running for a full week with corrupted data which is not acceptable even for some home users
Then you probably should run something simpler than an x86 since the cpu can be buggy, code can be buggy, and so on

The lot of which is generally more likely than an already rare bit flip that additionally actually hurts you before a scrub happens.

If you have enough drives to make a bit corruption on one drive a likely event and it actually is THAT unacceptable, well, sure use checksums somewhere (could also be in the application where needed). But I figure you're already using a distributed filesystem at that point, which then is the better place to do "filesystem" checksums.

> On arrays with more than, lets say 24TB, that is a very stressing operation on the drives and could cause premature failure.
You just read every drive once (well, the allocated space - it'll omit the unallocated bits).

By and large, it's a pretty sequential low stress read (of course if you have other IO, it'll NCQ and jump around a bit, but that's nothing special for a HDD either).
This level of "stress" probably won't even easily show up on a statistic if you actually made one.

How good is ReFS in 2018?

Yeah, I guess it will work well in practice, my main gripe with it is how 'inelegant' it seems.
If the only downside of running ZFS is increased hardware cost I'd gladly pay the price since the difference is so low.
Even if the benefits of guaranteed data integrity and no need for periodic scrubs are not that big of a deal in practice, I would appreciate the peace of mind and lower maintenance overhead.
Of course you also get all the nice ZFS features like deduplication, snapshots, COW, etc which I would want to integrate in a large array for sure, even if I have no immediate use for them.

3/10 - Windows only, proprietary and undocumented. If something fails or bugs out -and it could-, you're presumably shit out of luck unless you're a major enterprise customer of Microsoft's.

You could say it "works" with some degree of data retention and not terribly predictable performance, and the management features are pretty good.

same but with reiserfs