Analytics. Hadoop. Big Data. CloudBursting. VMware. Conferences. EBS. Disaster Recovery. Chunkd - Hail Cloud Computing Wiki. Chunkd: single-node data storage service Overview chunkd is a low-level, single node, key/value storage service for anonymous data objects. The intended use is as a low-level piece of large-scale distributed data storage infrastructure, for Project Hail. Keys are opaque binary data, from 1 byte to 1024 bytes in size. Objects are grouped into containers called "tables. " Currently the service is TCP-based. From chunkd's simple GET/PUT/DELETE API, you can build cluster-wide redundant storage services, exporting data storage capability from any device supported by your OS, via a single, standard, easy to use, device-agnostic interface.
The network protocol consists of TCP-based messages. Given that the protocol is message-based, it should be easy to add support for non-TCP network protocols such as RDMA, SCTP or AMQP. Design goals But it is also quite valid to have much larger chunkd nodes, perhaps tied to larger servers, 10gige, multiple disks and/or SAN networks. Status Early beta. Resources. Linux NFS faq. 2. Introduction. The Network File System (NFS) was developed to allow machines to mount a disk partition on a remote machine as if it were a local disk. It allows for fast, seamless sharing of files across a network.
It also gives the potential for unwanted people to access your hard drive over the network (and thereby possibly read your email and delete all your files as well as break into your system) if you set it up incorrectly. So please read Section 6, “Security and NFS” of this document carefully if you intend to implement an NFS setup. There are other systems that provide similar functionality to NFS. Samba ( provides file services to Windows clients. The Andrew File System, originally developed by IBM ( and now open-source, provides a file sharing mechanism with some additional security and performance features. 2.2.
This HOWTO is intended as a complete, step-by-step guide to setting up NFS correctly and effectively. 2.3. 2.4. 2.5.
Expanding Role Of Data Deduplication. Articles. Service Providers. Competition. Corporate. AWS. Backup Services. All Things Distributed. OpenStack. Research. ZFS.