For science, big data is the microscope of the 21st century. Johns Hopkins is taking a $1.2 million grant from the National Science Foundation to build a 100 gigabit per second network to shuttle data from the campus to other large computing centers at national labs and even Google. The network will be capable of transferring an amount of data equivalent to 80 million file cabinets filled with text each day. The head of the project, Dr. Alex Szalay, detailed the plans, which include gear from networking gear from Cisco, Arista and Solarflare; Nvidia GPUs; and 66,000 x86 cores. That’s on top of the actual fiber that will connect a new, 1-megawatt data center inside the physics building to regional Mid-Atlantic Crossroads research and engineering network at the University of Maryland. The new data center at Johns Hopkins, awaiting its 100 Gbps backbone. He ascribes this massive amount of data to the emergence of cheap compute, better imaging and more information, and calls it a new way of doing science.
Image courtesy of Flickr user RinzeWind. AWS Elastic Beanstalk. Amazon Web Services (AWS) comprises dozens of services, each of which exposes an area of functionality. While the variety of services offers flexibility for how you want to manage your AWS infrastructure, it can be challenging to figure out which services to use and how to provision them. With Elastic Beanstalk, you can quickly deploy and manage applications in the AWS cloud without worrying about the infrastructure that runs those applications.
AWS Elastic Beanstalk reduces management complexity without restricting choice or control. You simply upload your application, and Elastic Beanstalk automatically handles the details of capacity provisioning, load balancing, scaling, and application health monitoring. Elastic Beanstalk uses highly reliable and scalable services that are available in the AWS Free Usage Tier. Elastic Beanstalk provides developers and systems administrators an easy, fast way to deploy and manage their applications without having to worry about AWS infrastructure. NIST goes into detail on cloud's future, areas for improvement.
Jo Maitland, Senior Executive Editor Published: 04 Nov 2011 In a useful but long-winded set of documents aimed at furthering adoption of cloud computing , NIST has zeroed in on interoperability, security and portability as key areas for improvement. Kudos When you register, my team of editors will also send you alerts about public, private and hybrid cloud computing as well as other related technologies. to NIST for pushing the industry in the right direction, but did its "roadmap" report really require three volumes and over 200 pages? I feel like I just killed a tree printing it. If you can get through the verbose language, there is some helpful information for anyone buying or selling a cloud service. Volume I: High Priority Requirements to Further U.S.
Volume III: Technical Considerations for USG Cloud Computing Deployment Decisions , which is still being prepared, is intended for decision makers evaluating cloud services. Just kidding!
Hadoop. Cloud Education. Articles. MapReduce. Overview[edit] MapReduce is a framework for processing parallelizable problems across huge datasets using a large number of computers (nodes), collectively referred to as a cluster (if all nodes are on the same local network and use similar hardware) or a grid (if the nodes are shared across geographically and administratively distributed systems, and use more heterogenous hardware). Processing can occur on data stored either in a filesystem (unstructured) or in a database (structured). MapReduce can take advantage of locality of data, processing it on or near the storage assets in order to reduce the distance over which it must be transmitted. "Map" step: Each worker node applies the "map()" function to the local data, and writes the output to a temporary storage.
A master node orchestrates that for redundant copies of input data, only one is processed. " MapReduce allows for distributed processing of the map and reduction operations. Logical view[edit] Map(k1,v1) → list(k2,v2) Uses[edit] Welcome to Swift’s documentation! — Swift v1.4.4-dev documentation. SDSC Announces Scalable, High-Performance Data Storage Cloud. SDSC Announces Scalable, High-Performance Data Storage Cloud Web-based System Offers High Durability, Security, and Speed for Diverse User Base September 22, 2011 By Jan Zverina The San Diego Supercomputer Center (SDSC) at the University of California, San Diego, today announced the launch of what is believed to be the largest academic-based cloud storage system in the U.S., specifically designed for researchers, students, academics, and industry users who require stable, secure, and cost-effective storage and sharing of digital information, including extremely large data sets.
“We believe that the SDSC Cloud may well revolutionize how data is preserved and shared among researchers, especially massive datasets that are becoming more prevalent in this new era of data-intensive research and computing,” said Michael Norman, director of SDSC. “The SDSC Cloud marks a paradigm shift in how we think about long-term storage,” said Richard Moore, SDSC’s deputy director.