Erasure coding, bcache2
Added 2020-07-07 16:15:59 +0000 UTCFinally been making some good progress on erasure coding: currently debugging the new code to update existing stripes with new blocks, which is the main piece that was missing - this is needed to deal with internal fragmentation across erasure coded stripes and avoid running out of space.
Also - I started working on a bcache -> bcachefs layer: it lets you attach existing bcache backing devices to a bcachefs filesystem and use that to store the cached data instead of the bcache cache device code. The bcachefs code has many, many performance improvements and refinements over bcache, so I expect there will be users out there happy to have a drop in replacement. You can detach from a bcache cache and attach to a bcachefs filesystem while the backing device is in use - it's as easy an upgrade as it gets.
However - this isn't my main interest area, and I know there's companies out there making good use of bcache and distributors with paying customers who've found it worthwhile to support bcache, so I'm looking for sponsorship - both cash, and just as importantly testing - for this project before I continue it any further - it's at proof of concept stage right now.
It looks like it'll be a bit better to manage than bcache - cached data is now in files, which can be deleted if a cached device is never coming back (bcache doesn't have an interface to drop cached data from a backing device that isn't around anymore).
The code, if anyone is interested, is in this branch right here: https://evilpiepirate.org/git/bcachefs.git/log/?h=bcache2
Comments
Oh I see. Thank you!
John-Gee
2020-07-07 16:51:23 +0000 UTCNot really. I do think this will be helpful for testing though as it's a somewhat more constrained workload than as a general purpose filesystem, and with bcache it's trivial to deal with a corrupted cache in writethrough mode - also, this way of doing it means users can easily roll back to bcache1.
Kent Overstreet
2020-07-07 16:26:58 +0000 UTCCould that new bcache layer be pushed to mainline as an evolution of the current bcache without the other new FS code? That would give you a lot of testing since there are plenty of bcache users, and it might be easier to push it to mainline as patches to something already there rather than something new (though I'm sure they're aware of that trick).
John-Gee
2020-07-07 16:21:28 +0000 UTC
