Further, this demented customer (there's always one) has decided to tar them up, 16 per file, just to hit the ZFS maximum file size limit. Imagine someone using their slice of Planet ZFS (formerly known as Earth) to store backups of maximally-sized ext4 disk images. That per-file maximum file size is 16 EiB under ZFS, which is 16× larger than the maximum volume size of ext4, which is considered ridiculously large today in its own right. (If it's bothering you that our storage array is still the size of a planet here in this example, remember that it had to be that big in order to hit the first limit above, so it is fair to continue to use it for this example here.) ![]() In other words, every person can store about a billion maximally-sized files in their tiny personal slice of our Earth-sized ZFS storage array. ![]() That's the size of the pool divided by the population of Earth² divided by the maximum file size, in round numbers. Let's give every person on the planet their own equally-sized slice of that pool: What can we say about the size of the files stored within it? So we've got a filesystem the size of a planet now. ![]() The approximation above is quite rough, and storage densities continue to climb, but keep things in perspective: in the future, to pull off this stunt of constructing maximally-sized ZFS pools, we'll still need to use the total crust-to-core resources of small planets. Only solid-state storage makes sense here. We have to assume that this Earth-sized ZFS pool will be composed of vdevs that never need replacing, and that they can transfer data fast enough that you can fill the pool in a reasonable time. If you think it's unfair that I'm using flash storage here instead of something denser like tape or disk, consider the data rates involved, as well as the fact that we haven't even tried to consider redundancy or device replacement. such that you don't end up with so much slag that you blow the estimate. That means that to a first approximation, to construct a single maximally-sized ZFS pool out of the current largest-available microSD cards, you might have to use an entire Earth-sized planet's worth of atoms, and then only if you start off with something close to the right mix of silicon, carbon, gold, etc. We also need some slop here to account for inter-chip interconnects, etc.Ī pico- anything is 10 -12, so our 47 pg and 2.5×10 -13 g/B numbers above are about an order of magnitude apart. The density of data in grams is 2.5×10 -13 g/byte for microSD storage, as of this writing: the largest available SD card is 1 TB, and it weighs about 0.25g.¹ A microSD card isn't made of pure silicon, but you can't ignore the packaging, because we'll need some of that in our Earth-computer, too we'll assume that the low density of the plastic and the higher density of the metal pins average out to about the same density as silicon. We can write that number instead as roughly 10 38 bytes, which means in order to hit that limit, you'd have to have a single Earth-sized ZFS pool where every one of its 10 50 atoms is used to store data, and each byte is stored by an element no larger than 10 12 atoms.ġ0 12 atoms sounds like a lot, but it's only about 47 picograms of silicon. volume sizeĢ 128 bytes is effectively infinite already. Now let's discuss the practical import of each of those limits: Max. A filesystem based on arbitrary sized integers would have to piece each number together from multiple blocks, requiring a lot of extra I/O from multiple disk hits relative to a filesystem that knows up front how big its metadata blocks are. The speed hit from arbitrary precision arithmetic is bad enough inside a computer's RAM, but when a filesystem doesn't know how many reads it needs to make in order to load all of the numbers it needs into RAM, that would be very costly. If you want fast arithmetic, you use fixed-size words, period. There are exceptions, but these are usually mathematics-oriented DSLs like bc or Wolfram Language. ![]() This is why arbitrary-precision arithmetic is an add-on library in most programming languages, not the default way of doing arithmetic. The alternative is called arbitrary-precision arithmetic, but it's inherently slow. ZFS's limits are based on fixed-size integers because that's the fastest way to do arithmetic in a computer. What internally limits these things? Long answer
0 Comments
Leave a Reply. |