![]() It also means we shouldn't assume the erasure code calculations happen on a machine with 60 drives attached. That allows them to lose a single machine without any data becoming unavailable. > They're running a fair bit of math (probably Reed Solomon matrix multiplications for error correction) over all the data accesses.ĭo those run on this machine? I imagine backblaze has redundancy at the cluster level rather than machine level. More I/O is needed if you add NVMe storage to cache hard drive reads/writes, tons of RAM is needed if you plan to dedup. We can leave Backblaze's workload and think about typical SAN or NAS workloads too. I don't think a cheap low-power CPU would be powerful enough to perform real-time error correction calculations over 9GBps of data. Maybe a cheap CPU with many I/O lanes going to a GPU / FPGA / ASIC for the error checking math, but. Server CPU seems to be a cheap and simple answer: get the large number of PCIe I/O lanes plus a beefy CPU to handle the calculations. Maybe if you had an FPGA or some kind of specialist DSP (lol GPUs maybe, since they're good at matrix multiplication), you can handle the bandwidth. Then the math comes in: matrix multiplications over every bit of data to verify checksums and reed-solomon error correction starts to get expensive. Throw down a NVMe-cache or other stuff and I'm not seeing any kind of small system working out here. We're looking at PCIe 3.0 x32 just for HDDs and Networking. ![]() And then another x16 for network connections (multiple PHY for Fiber in/out that can handle that 9GB/s to a variety of switches). At least PCIe 3.0 x16, just for the hard drives. Given the bandwidth of 60+ hard drives (150MB/s per hard drive x 60 = 9GB/s in/out), I'm pretty sure you need a decent CPU just to handle the PCIe traffic. They're running a fair bit of math (probably Reed Solomon matrix multiplications for error correction) over all the data accesses.
0 Comments
Leave a Reply. |
AuthorWrite something about yourself. No need to be fancy, just an overview. ArchivesCategories |