background preloader

The Value of Big Data Isn't the Data - Kristian J. Hammond

The Value of Big Data Isn't the Data - Kristian J. Hammond
by Kristian J. Hammond | 11:00 AM May 1, 2013 It is clear that a new age is upon us. Evidence-based decision-making (aka Big Data) is not just the latest fad, it’s the future of how we are going to guide and grow business. The confusion here is understandable. I would argue that what you want and what you need is to turn that data into a story. It may well be the case that you already have someone who looks at the data, builds the queries, interprets the results and writes up the report. If we’re going to really capitalize on Big Data, we need get to human insight at machine scale. For the most part, we know what we want out of the data. In order to do this, your starting point has to be the story (or the communication) itself. The overall flow goes from data to fact to angle to story to language. Here’s an example. My favorite piece is the last element that our system (called Quill) generates: the advisory. Of course, each restaurant needs to get the advice that is relevant to it.

What Separates a Good Data Scientist from a Great One - Thomas C. Redman by Thomas C. Redman | 9:00 AM January 28, 2013 Companies that wish to take full advantage of their data must build strong, new, and different organizational capabilities. There is a lot to do, and data scientists are front and center. A good one can help you find relationships in vast quantities of disparate data — often important insights that you would not have gotten in any other way. Over the years I’ve had the privilege of working with dozens, maybe hundreds, of good statisticians, analysts, and data scientists. 1. But the great ones take this trait to an extreme. 2. What’s important here is that thousands of others did not see the error. Mathematics has turned out to provide a convenient, amazingly-effective language (Einstein used the phrase “unreasonably effective”) for describing the real world. 3. This is a really big deal. Great data scientists also persist in making themselves heard. 4. Great data scientists embrace uncertainty. Great data scientists are truly special.

Big Data, Social Media and the Long Tail of Public Policy Preamble You may have noticed that I didn't publish last week; as a result this week's article is both weightier and lengthier. Accordingly I decided to experiment a bit by providing a TL;DR version of the article up front, namely: Public Engagement via Social Media + Big Data Analytics = Future of Public Policy.How I got there ... To say that either Linked Data or Big Data are new would be a mischaracterization; to say that they are still new to government on the other hand is likely a fair assessment. Linked Data: A Primer For the unfamiliar, linked data is simply a way of structuring data so that it can be easily aligned with other data sets; linking data together increases its usefulness by providing richer strategic overviews or by facilitating a greater depth of analysis. If you are interested in seeing the quality of public policy analysis that properly linked data can inform Hans Rosling’s demonstration below is a prime example: Big Data: A Primer Abundance is the common denominator

To Work with Data, You Need a Lab and a Factory - Thomas C. Redman and Bill Sweeney by Thomas C. Redman and Bill Sweeney | 8:00 AM April 24, 2013 Companies that aim to score big over the long term with big data must do two very different things well. They must find interesting, novel, and useful insights about the real world in the data. And they must turn those insights into products and services, and deliver those products and services at a profit. While the two goals are mutually reinforcing, companies actually require two distinct departments. For the second, companies should set up and manage a “data factory,” staffed by process engineers and others with deep technical skills who “get the job done”; a tight structure that promotes consistency, scale, and decreasing unit cost; a shorter-term focus; and a culture that values quality and revenue above all else. Companies must not confuse the separate roles. The laboratory. We want to doubly emphasize these points because promises of just the opposite are so loud. The factory. There are two keys to success here.

The Rise of Big Data Everyone knows that the Internet has changed how businesses operate, governments function, and people live. But a new, less visible technological trend is just as transformative: “big data.” Big data starts with the fact that there is a lot more information floating around these days than ever before, and it is being put to extraordinary new uses. Big data is distinct from the Internet, although the Web makes it much easier to collect and share data. In the third century BC, the Library of Alexandria was believed to house the sum of human knowledge. This explosion of data is relatively new. Given this massive scale, it is tempting to understand big data solely in terms of size. To continue reading, please log in. Don't have an account? Register Register now to get three articles each month. As a subscriber, you get unrestricted access to Register for free to continue reading. Registered users get access to three free articles every month. Have an account?

Pundits: Stop Sounding Ignorant About Data - Andrew McAfee by Andrew McAfee | 11:00 AM April 23, 2013 The current surge of enthusiasm around big data has produced a predictable backlash. Some of it, like Gary Marcus’s New Yorker post “Steamrolled by Big Data,” is insightful and well-reasoned (even though I have my quibbles with some of his points). This is not surprising, since he’s a neuroscientist as well as a writer, and so quite comfortable with data. Unfortunately, some other prominent commentators clearly aren’t. So as a public service here’s a short list, written for non-quant-jock pundits, of things to keep in mind always when writing about data and its uses. Absolute Certainty is Not the Goal (Because It’s Impossible). Data geeks desperately want to make better predictions using the seas of digital information available today. People are Not Inherently Better at Making Decisions, Predictions, Judgments, and Diagnoses. And this is exactly the problem. How many of these glitches are there?

Program Ottawa, Canada – April 17, 2013 Ottawa Convention Centre Register for the conference now. We live in the data age. The smart use of this 21st Century resource offers an unprecedented opportunity to place Canada at the forefront of innovation, build our economy, catalyze research and development and improve the openness and responsiveness of government. The third edition of The Data Effect, in Ottawa in April 2013, will explore the emerging data revolution and how Canada should embrace it. The Data Effect: Tomorrow’s Canada Today will address how the prudent and innovative use of Big Data can improve outcomes in such areas as health care, drug and social policy research and public safety, to name a few. This day long event will offer leaders in government, the private sector and researchers a practical special briefing that surveys the best practices and opportunities being created by the Big Data revolution in Canada and around the world. NB: Agenda subject to change Mr.

Big Data's Promise and Limitations Five years ago, few people had heard the phrase “Big Data.” Now, it’s hard to go an hour without seeing it. In the past several months, the industry has been mentioned in dozens of New York Times stories, in every section from metro to business. Most of what’s written about Big Data is enthusiastic, like Kenneth Cukier and Viktor Mayer-Schonberger’s gushing ode “Big Data: A Revolution That Will Transform How We Live, Work, and Think,” which is currently selling briskly on Amazon, or the recent Times article on Mayor Bloomberg’s geek squad, and how “Big Data’s moment, especially in the management of cities, has powerfully and irreversibly arrived.” The reason scarcely anybody used to talk about Big Data is that, until very recently, it didn’t exist—most data had been, by current standards, small potatoes. As companies like Google have shown, more data often means newer and better solutions to old problems. Of course, Numenta is not the only company working with Big Data.

Models.Behaving.Badly.: Why Confusing Illusion with Reality Can Lead to Disaster, on Wall Street and in Life: Emanuel Derman: 9781439164990: Big Data: The Management Revolution Artwork: Tamar Cohen, Happy Motoring, 2010, silk screen on vintage road map, 26" x 18" “You can’t manage what you don’t measure.” There’s much wisdom in that saying, which has been attributed to both W. Edwards Deming and Peter Drucker, and it explains why the recent explosion of digital data is so important. Consider retailing. The familiarity of the Amazon story almost masks its power. As the tools and philosophies of big data spread, they will change long-standing ideas about the value of experience, the nature of expertise, and the practice of management. What’s New Here? Business executives sometimes ask us, “Isn’t ‘big data’ just another way of saying ‘analytics’?” Volume. Velocity.

Big Data Is Great, but Don’t Forget Intuition Andrew McAfee, principal research scientist at the M.I.T. Center for Digital Business, led off the conference by saying that Big Data would be “the next big chapter of our business history.” Next on stage was Erik Brynjolfsson, a professor and director of the M.I.T. center and a co-author of the article with Dr. McAfee. These drumroll claims rest on the premise that data like Web-browsing trails, sensor signals, GPS tracking, and social network messages will open the door to measuring and monitoring people and machines as never before. The results, according to technologists and business executives, will be a smarter world, with more efficient companies, better-served consumers and superior decisions guided by data and analysis. I’ve written about what is now being called Big Data a fair bit over the years, and I think it’s a powerful tool and an unstoppable trend. The quest to draw useful insights from business measurements is nothing new. The bubble that concerns Ms. Thomas H.