background preloader

Big Data challenge

Facebook Twitter

​DNA data storage landmark: Now it's 215 petabytes per gram or over 100 million movies. Researchers from Columbia University and the New York Genome Center have devised a new coding system, dubbed DNA Fountain, which is capable of stuffing 215 petabytes of data onto one gram of DNA.

​DNA data storage landmark: Now it's 215 petabytes per gram or over 100 million movies

That's about 100 times more than previous researchers have stored on DNA, and was achieved by customizing an algorithm for streaming video on a smartphone, Science Daily reports. DNA holds promise for data storage because of its superior density to tape, disk, and optical media. It can also store information for thousands of years if it's kept in the right conditions. While information in computers is written as ones and zeros, researchers have devised different algorithms for encoding data to conform with DNA's four base nucleotides: adenine, A, guanine, G, cytosine, C, and thymine, T.

Book: Python Data Science Handbook. Digital Transformation: The Age of Innocence, Inertia or Innovation? Data enrichment records for 200 million people up for sale on the Darknet. Full data enrichment profiles for more than 200 million people have been placed up for sale on the Darknet.

Data enrichment records for 200 million people up for sale on the Darknet

The person offering the files claims the data is from Experian, and is looking to get $600 for everything. Details of this incident came to Salted Hash via the secure drop at Peerlyst, where someone uploaded details surrounding the sale and the data. The State of Artificial Intelligence Technology, Per Nvidia's CEO. As industry embraces AI, computers aren’t the only ones that have to learn new tricks.

The State of Artificial Intelligence Technology, Per Nvidia's CEO

Fortune’s Andrew Nusca talks with Nvidia nvda CEO Jen-Hsun Huang. Fortune: What’s the current status of artificial intelligence? Jen-Hsun Huang: 2015 was a big year. Artificial intelligence is moving into the commercial world. Data Lakes, The Internet of Things and Other Assorted Mysteries. “Those who do not heed history are bound to repeat it.”

Data Lakes, The Internet of Things and Other Assorted Mysteries

Not so long ago the world of IT was surrounded by a phenomenon known as “”. We woke up one day and was everywhere. It was on TV. It was in Silicon Valley. Machine Learning. Many services that we use every day rely on machine learning - a field of science and a powerful technology that allows machines to learn from data and self-improve.

Machine Learning

Machine learning is used in internet search engines, email filters to sort out spam, websites to make personalised recommendations, banking software to detect unusual transactions, and lots of apps on our phones such as voice recognition. The technology has many more potential applications, some with higher stakes than others. Future developments could support the UK economy and will have a significant impact upon society. Why Every Manager Needs These Data Science Skills. The fact is, data is becoming an increasingly important business assets and more and more decisions are based on analysis of data.

Why Every Manager Needs These Data Science Skills

It’s infiltrating every department of every business, and that means that the employees and managers who have the skills to deal with data will be in a better position to help their company and move up. But don’t fret: you don’t need to go out and learn computer programming, database management, or advanced maths if you’re a manager in human resources or sales. Analytics & Automation Blog. Cisco Blog > Analytics & Automation Where We’ve Been When Judith Davis and I co-authored Data Virtualization: Going Beyond Traditional Data Integration to Achieve Business Agility in 2011, we identified the five most popular Data Virtualization usage patterns at that time.

Analytics & Automation Blog

NetAppVoice: How The Semantic Web Changes Everything. Again! The “semantic Web” is hugely important to tomorrow’s business.

NetAppVoice: How The Semantic Web Changes Everything. Again!

Data to Value Data Management Consultancy. Introduction to open data. Today the world generates vast quantities of data each day that can be used to enhance the quality of living of virtually anyone in the world.

Introduction to open data

Information is power but also a tool for supporting development, knowledge sharing and social initiatives. Data for Policy. Enterprise-Grade Machine Learning for Big Data. 72 Infographics about big data. Big Data, IoT, Wearables: A Connected World with Intelligence. At the CES 2015, I was fascinated by all sorts of possible applications of IoT – socks with sensors, mattresses with sensors, smart watches, smart everything – it seems like a scene in sci-fi movies has just come true.

Big Data, IoT, Wearables: A Connected World with Intelligence

People are eager to learn more about what’s happening around them and now they can. While I was at there I attended a talk given by David Pogue – he is awesome. He pointed out that the prevalence of smartphone is the key to the realization of the phenomenon called “Quantified Self.” I agreed with him. Smart phones play a vital role as a hub where all our personal data converge and present, seamlessly. It’s all relevant; Big Data, IoT, Wearable, Cloud Computing… While most data is uploaded to the cloud, the client devices are generally powerful enough that the computing can be decentralized.

Big Data, IoT, Wearables: A Connected World with Intelligence. A Visual Introduction to Machine Learning. Finding better boundaries Let's revisit the 240-ft elevation boundary proposed previously to see how we can improve upon our intuition. Clearly, this requires a different perspective. Data Scientist: The Sexiest Job of the 21st Century. When Jonathan Goldman arrived for work in June 2006 at LinkedIn, the business networking site, the place still felt like a start-up. The company had just under 8 million accounts, and the number was growing quickly as existing members invited their friends and colleagues to join.

But users weren’t seeking out connections with the people who were already on the site at the rate executives had expected. Something was apparently missing in the social experience. Small Pieces Loosely Joined: How smarter use of technology and data can deliver real reform of local government. Local authorities could save up to £10billion by 2020 through smarter and more collaborative use of technology and data. Small Pieces Loosely Joined highlights how every year councils lose more than £1 billion by failing to identify where fraud has taken place.

The paper also sheds light on how a lack of data sharing and collaboration between many local authorities, as well as the use of bespoke IT systems, keeps the cost of providing public services unsustainably high. The report sets out three ways in which local authorities could not only save billions of pounds, but also provide better, more coordinated public services: The Truth About Real-Time Analytics in the Era of Big Data - Mainframe Insights. Every day we see the impact of real-time analytics and personalization on modern business. Mobile has been the ultimate catalyst, driving new customer experiences that are tailored to the individual, as data fuels a race to provide more value than competitors do. Big data challenges? Look at your people, not your technology. As anyone pursuing a big data initiative knows, every big data strategy really has two components: the technology and the people. The technology part is actually very simple to solve, relative to the people.

As long as you're not trying to crack big data problems with relational database technology from 2004, this piece of the equation shouldn't be a big scary beast. The first thing you should do is capture all the structured and unstructured information you can, even if you don't know what's going to be useful.