Replication

Facebook Twitter

XFS. XFS is a 64-bit, high-performance journaling file system created by Silicon Graphics, Inc.

XFS

It was the default file system in IRIX releases 5.3 and onwards, and later it was ported to the Linux kernel. XFS is particularly proficient at parallel I/O due to its design based on allocation groups. HDFS and Erasure Codes (HDFS-RAID) The Hadoop Distributed File System has been great in providing a cloud-type file system.

HDFS and Erasure Codes (HDFS-RAID)

It is robust (when administered correctly :-)) and highly scalable. However, one of the main drawbacks of HDFS is that each piece of data is replicated in three places. This is acceptable because disk storage is cheap and is becoming cheaper by the day; this isn't a problem if you have a relatively small to medium size cluster. The price difference (in absolute terms) is not much whether you use 15 disks or whether you use 10 disks.

HDFS slower than expected reading from localnode. Attaching v1 of a design document for this feature.

HDFS slower than expected reading from localnode

This does not include a test plan - that will follow once implementation has gone a bit further. Pasting the design doc below as well: Problem Definition.