Replication

Facebook Twitter

XFS. Not to be confused with Xiafs.

XFS

XFS is a high-performance 64-bit journaling file system created by Silicon Graphics, Inc (SGI) in 1993.[1] It was the default file system in the SGI's IRIX operating system starting with its version 5.3; the file system was ported to the Linux kernel in 2001. As of June 2014[update], XFS is supported by most Linux distributions, some of which use it as the default file system.

HDFS and Erasure Codes (HDFS-RAID) The Hadoop Distributed File System has been great in providing a cloud-type file system.

HDFS and Erasure Codes (HDFS-RAID)

It is robust (when administered correctly :-)) and highly scalable. However, one of the main drawbacks of HDFS is that each piece of data is replicated in three places. This is acceptable because disk storage is cheap and is becoming cheaper by the day; this isn't a problem if you have a relatively small to medium size cluster. The price difference (in absolute terms) is not much whether you use 15 disks or whether you use 10 disks. HDFS slower than expected reading from localnode. Attaching v1 of a design document for this feature.

HDFS slower than expected reading from localnode

This does not include a test plan - that will follow once implementation has gone a bit further. Pasting the design doc below as well: Problem Definition.