background preloader

Big Data and data mining

Facebook Twitter

Are you ready? This is all the data Facebook and Google have on you. Cyber Warfare Is Growing. We Need Rules to Protect Ourselves. In Brief Cybersecurity has become one of today's more pressing issues, and it's likely to get worse in the future.

Cyber Warfare Is Growing. We Need Rules to Protect Ourselves.

That's why the secretary-general of the U.N. is now calling for nations to come up with clearer rules to protect civilians from hacking. Corporate Surveillance in Everyday Life. Google and Facebook, followed by other large platforms such as Apple, Microsoft, Amazon and Alibaba have unprecedented access to data about the lives of billions of people.

Corporate Surveillance in Everyday Life

Although they have different business models and therefore play different roles in the personal data industry, they have the power to widely dictate the basic parameters of the overall digital markets. The large platforms mostly restrict how other firms can directly obtain their data; in this way, they force them to utilize the platform’s data on users within their own ecosystems and gather additional data from beyond the platforms’ reach. Although the large multinationals in different sectors that have frequent interactions with hundreds of millions of consumers are in a somewhat similar position, they not only acquire consumer data collected by others, but often also provide data. On Flipboard. Six ways (and counting) that big data systems are harming society.

There is growing consensus that with big data comes great opportunity, but also great risk.

Six ways (and counting) that big data systems are harming society

But these risks are not getting enough political and public attention. One way to better appreciate the risks that come with our big data future is to consider how people are already being negatively affected by uses of it. At Cardiff University’s Data Justice Lab, we decided to record the harms that big data uses have already caused, pulling together concrete examples of harm that have been referenced in previous work so that we might gain a better big picture appreciation of where we are heading. We did so in the hope that such a record will generate more debate and intervention from the public into the kind of big data society, and future we want. The following examples are a condensed version of our recently published Data Harm Record, a running record, to be updated as we learn about more cases. 1. Will Democracy Survive Big Data and Artificial Intelligence? Editor’s Note: This article first appeared in Spektrum der Wissenschaft, Scientific American’s sister publication, as “Digitale Demokratie statt Datendiktatur.”

Will Democracy Survive Big Data and Artificial Intelligence?

“Enlightenment is man’s emergence from his self-imposed immaturity. Artificial Intelligence Will Redesign Healthcare - The Medical Futurist. There are various thought leaders who believe that we are experiencing the Fourth Industrial Revolution, which is characterized by a range of new technologies that are fusing the physical, digital and biological worlds, impacting all disciplines, economies and industries, and even challenging ideas about what it means to be human.

Artificial Intelligence Will Redesign Healthcare - The Medical Futurist

I am certain that healthcare will be the lead industrial area of such a revolution and one of the major catalysts for change is going to be artificial intelligence. With the evolution of digital capacity, more and more data is produced and stored in the digital space. The amount of available digital data is growing at a mind-blowing speed, doubling every two years. In 2013, it encompassed 4.4 zettabytes, however by 2020 the digital universe – the data we create and copy annually – will reach 44 zettabytes, or 44 trillion gigabytes (!).

Executive Summary - Future HR Trends. The key findings from The Economist Intelligence Unit’s research into workforce analytics are as follows: Investment in workforce analytics is on the increase Research and supporting evidence detailed in this report strongly suggest that the overwhelming majority of organizations will either begin or increase their use of Big Data1 in HR over the coming years.

Executive Summary - Future HR Trends

There are several reasons for this reported increase. Book Review — The Traps of Big Data Revealed in “Weapons of Math Destruction” by Cathy O’Neil. “Weapons of Math Destruction,” by Cathy O’Neil Cathy O’Neil is a mathematician and data scientist who writes a blog at mathbabe.org.

Book Review — The Traps of Big Data Revealed in “Weapons of Math Destruction” by Cathy O’Neil

Her new book, “Weapons of Math Destruction,” is just out, and it’s a blistering journey through big data’s already dangerous effects on society, and a sobering preview of what could be coming if we’re not careful. Calling algorithms “an opinion formalized in code,” O’Neil starts by outlining her love of math, which led to a Ph.D. in mathematics from Harvard, a teaching stint at Barnard, then to a role as an analyst at the same hedge fund (D.E. Shaw) that spawned Jeff Bezos in the 1990s. Arriving at D.E. Is an algorithm any less racist than a human? We would all like to fancy ourselves as eminently capable of impartiality, able to make decisions without prejudices – especially at work.

Is an algorithm any less racist than a human?

Unfortunately, the reality is that human bias, both conscious and unconscious, can’t help but come into play when it comes to who gets jobs and how much money candidates get offered. Managers often gravitate to people most like themselves, make gender-based assumptions about skills or salaries, or reject candidates who have non-white names – to name just a few examples – even if they don’t mean to. There’s an increasingly popular solution to this problem: why not let an intelligent algorithm make hiring decisions for you?

Surely, the thinking goes, a computer is more able to be impartial than a person, and can simply look at the relevant data vectors to select the most qualified people from a heap of applications, removing human bias and making the process more efficient to boot. Forbes Welcome. This Guy Trains Computers to Find Future Criminals. When historians look back at the turmoil over prejudice and policing in the U.S. over the past few years, they’re unlikely to dwell on the case of Eric Loomis.

This Guy Trains Computers to Find Future Criminals

Police in La Crosse, Wis., arrested Loomis in February 2013 for driving a car that was used in a drive-by shooting. He had been arrested a dozen times before. Loomis took a plea, and was sentenced to six years in prison plus five years of probation. The episode was unremarkable compared with the deaths of Philando Castile and Alton Sterling at the hands of police, which were captured on camera and distributed widely online.

Archive

Why we need a methodology for data science. Those who work in the domain of data science solve problems and answer questions through data analysis every day.

Why we need a methodology for data science

They build models to predict outcomes or discover underlying patterns, all to gain insights leading to actions that will improve future outcomes. And the tools and technologies used in data analysis are evolving rapidly, enhancing data scientists’ abilities to reach their goal. But a chronic inhibitor to success somehow remains even in such rapid growth. Technology and the future of work. Mining public, private data loaded with large questions. CAMBRIDGE, Mass. — With the success of its free, open online-course system, called MITx, the Massachusetts Institute of Technology finds itself sitting on a wealth of student data that researchers might use to compare the efficacy of virtual teaching methods, and perhaps advance the field of Web-based instruction.

Since its inception several years ago, MITx has attracted more than 760,000 registered users from about 190 countries, university officials said. Those users have generated 700 million interactions with the school’s learning system and have contributed about 423,000 forum entries, many of them quite personal. As researchers contemplate mining the students’ details, however, the university is grappling with ethical issues raised by the collection and analysis of these huge data sets, known familiarly as Big Data, said L. Feds Grapple With Big Data Vs. Privacy.

Government study focuses on how privacy-enhancing technologies and large-scale analytics will shape the future of big data. Internet Of Things: 8 Cost-Cutting Ideas For Government (Click image for larger view and slideshow.) White House counselor John Podesta is leading a 90-day government study that explores the intersection of big data and privacy.

According to Podesta, now is the time to take a closer look at big data analytics and other comprehensive data-mining techniques that could shape future policies. "The study is fundamentally a scoping exercise. Big Data Ethics: How Does It Affect Your Privacy? Big data enables to check, control and know everything. How Big Data Is Changing Medicine. Here’s how science usually works: Come up with a question or a hypothesis. Develop an experiment to test it and create data. As any middle school student could tell you, it’s called the scientific method. Now, some researchers and entrepreneurs in the Bay Area say that method is being upended, especially when it comes to medicine. Big data, big business, Big Brother? With big data comes big responsibility. What Makes Big Data Projects Succeed - Tom Davenport.

By Tom Davenport | 12:00 PM March 26, 2014. Google Flu Trends' Failure Shows Good Data > Big Data - Kaiser Fung. By Kaiser Fung | 8:00 AM March 25, 2014. Quantiphobia and the turning of morals into facts. Innovation: How your search queries can predict the future - tech - 30 April 2009. Innovation is our new column that highlights the latest emerging technological ideas and where they may lead. Real-time web search – which scours only the latest updates to services like Twitter – is currently generating quite a buzz because it can provide a glimpse of what people around the world are thinking or doing at any given moment. Interest in this kind of search is so great that, according to recent leaks, Google is considering buying Twitter. The latest research from the internet search giant, though, suggests that real-time results could be even more powerful – they may reveal the future as well as the present. NSA Uses Google Ad-Tracking Cookies for Targeted Hacks: Report.

IBM's Next Big Thing: Psychic Twitter Bots. Using some of the same technologies that allowed the Watson natural language supercomputer to conquer Jeopardy, IBM's next step: Psychic artificial intelligences that read your Twitter feed and can tell when you're about to have a baby, get married, buy a house, or move across the country, and even tell you how these major life events make you feel, then approach you about them accordingly. Startup Trifacta Embracing the Data Scientist in All of Us. Organizations drowning in big and small data will soon have a new way to wrangle, munge or transform it – however you want to describe the process – thanks to software from startup Trifacta that's now in beta tests. The San Francisco company, whose staff has grown from its three computer scientist founders a year ago to a robust 22 employees, today announced a second round of venture funds totaling $12 million led by Greylock Partners and Accel Partners.

That brings overall funding to $16.3 million. The new funds will allow Trifacta to bulk up product design, engineering, sales and marketing, and should pave the way for the company to roll out its data management technology next year. To continue reading, register here to become an Insider. Beyond Data Mining. This article first appeared in IEEE Software magazine and is brought to you by InfoQ & IEEE Computer Society. The predictive modeling community applies data miners to artifacts from software projects. Big Data Security, Privacy Concerns Remain Unanswered. Is Data Complexity Blinding Your IT Decision-Making? Why Did Google Pay $400 Million for DeepMind? Big data blues: The dangers of data mining. Open data: Unlocking innovation and performance with liquid information.