background preloader

The 70 Online Databases that Define Our Planet

The 70 Online Databases that Define Our Planet
Back in April, we looked at an ambitious European plan to simulate the entire planet. The idea is to exploit the huge amounts of data generated by financial markets, health records, social media and climate monitoring to model the planet’s climate, societies and economy. The vision is that a system like this can help to understand and predict crises before they occur so that governments can take appropriate measures in advance. There are numerous challenges here. Nobody yet has the computing power necessary for such a task, neither are there models that will can accurately model even much smaller systems. But before any of that is possible, researchers must gather the economic, social and technological data needed to feed this machine. Today, we get a grand tour of this challenge from Dirk Helbing and Stefano Balietti at the Swiss Federal Institute of Technology in Zurich. These and other pursuits are now producing massive amounts of data, many of which are freely available on the web.

Related:  STEM

The Tehran Connection - By Kalev Leetaru Iran's nuclear program has been one of the hottest topics in foreign policy for years, and attention has only intensified over the past few days, as an interim agreement was reached in Geneva to limit enrichment activity in pursuit of a more comprehensive deal. The details of the deal itself are of course interesting, but in aggregate the news stories about Iran can tell us far more than we can learn simply by reading each story on its own. By using "big data" analytics of the world's media coverage, combined with network visualization techniques, we can study the web of relationships around any given storyline -- whether it focuses on an individual, a topic, or even an entire country. Using these powerful techniques, we can move beyond specifics to patterns -- and the patterns tell us that our understanding of Iran is both sharp and sharply limited. The fact that this network accurately distinguishes internal and external leaders is a critical finding.

How To Access THE HIDDEN WIKI (Deep web, secret Internet) (Some of this was copied from, I'm just bringing it to TBN) For those who don't know, the deep web represents a gargantuan part of the internet which is not accessible through regular searches via google or other search engines. Searching on the Internet today can be compared to dragging a net across the surface of the ocean. While a great deal may be caught in the net, there is still a wealth of information that is deep, and therefore, missed. The reason is simple: Most of the Web's information is buried far down on dynamically generated sites, and standard search engines never find it.

The Palestine Papers Attack on S Sudan UN base may be 'war crime'#link# of abducted girls freed in Nigeria#link# Korea divers see bodies inside sunken ferry#link# delays Keystone XL pipeline decision#link# leaders propose constitutional change#link# Francis leads Good Friday procession#link# bomb blast kills policeman#link# Korea arrests captain of sunken ferry#link# signs law blocking Iran's envoy to UN#link# urges Israel to reverse West Bank moves#link# ailing president wins fourth term#link# Sudan deploys army to guard UN base#link# earthquake rattles Mexico#link# Ukraine's political crisis#link# Korea ferry tragedy death toll rises#link#

Web science: an interdisciplinary approach to understanding the web Contributed articles By James Hendler, Nigel Shadbolt, Wendy Hall, Tim Berners-Lee, Daniel Weitzner Communications of the ACM, Vol. 51 No. 7, Pages 60-69 10.1145/1364782.1364798 Comments Despite the web's great success as a technology and the significant amount of computing infrastructure on which it is built, it remains, as an entity, surprisingly unstudied. Here, we look at some of the technical and social challenges that must be overcome to model the Web as a whole, keep it growing, and understand its continuing social impact. A systems approach, in the sense of "systems biology," is needed if we are to be able to understand and engineer the future Web. Despite the huge effect the Web has had on computing, as well as on the overall field of computer science, the best keyword indicator one can find in the ACM taxonomy, the one by which the field organizes many of its research papers and conferences, is "miscellaneous."

5 Ways to find, mix and mash your data One of the most popular trends in online journalism is taking publicly available data and translating it into visualizations or infographics that readers and viewers can quickly and easily understand. A large percentage of the visualizations you see on the web were built from scratch, which can take a considerable amount of time and effort. The following sites allow you to mash your data in record time. Swivel Eureqa Desktop Eureqa is a breakthrough technology that uncovers the intrinsic relationships hidden within complex data. Traditional machine learning techniques like neural networks and regression trees are capable tools for prediction, but become impractical when "solving the problem" involves understanding how you arrive at the answer. Eureqa uses a breakthrough machine learning technique called Symbolic Regression to unravel the intrinsic relationships in data and explain them as simple math. Using Symbolic Regression, Eureqa can create incredibly accurate predictions that are easily explained and shared with others. Over 35,000 people have relied on Eureqa to answer their most challenging questions, in industries ranging from Oil & Gas through Life Sciences and Big Box Retail.

All of our data journalism in one spreadsheet Data journalism and visualisation from the Guardian Illustration: Lisa Villani for the Guardian Today we have a list of all the data journalism produced here on the Guardian datablog. You can see what's been popular and get easy access to the data we have curated as this list shows each datablog post's associated spreadsheets, authors, the number of comments and retweets. Better Broadband: New Regulatory Rules Could Change the Way Americans Get Online At the turn of the millennium, the U.S. had some of the best broadband access in the world. It reached more homes, and at a lower price, than most every other industrial country. Ten years later the U.S. is a solid C-minus student, ranking slightly below average on nearly every metric. Just how the U.S. lost its edge and how it plans to get it back are the issues before the Federal Communications Commission as it prepares to launch the most significant overhaul of network policy since the birth of the Web. As part of last year’s stimulus package, Congress provided $7.2 billion to expand broadband access to every American. It also required the FCC to outline a plan for how to make that happen.

Tools to help bring data to your journalism « Michelle Minkoff NOTE: This entry was modified on the evening of 11/9/10 to deal with typos and missing words, resulting from posting this too late the previous night. Sleep deprivation isn’t always a good thing — although it allows one to do things more fun than sleep. Like play with data. The New York Times’ R&D Lab is building a quantified-self, semantic-analysis tool to track web browsing Let’s say you work in a modern digital newsroom. Your colleagues are looking at interesting stuff online all day long — reading stimulating news stories, searching down rabbit holes you’ve never thought of. There are probably connections between what the reporter five desks down from you is looking for and what you already know — or vice versa.

Not Neutrality?: U.S. Weighs Options for Turbo-Boosting Nation's Broadband into the 21st Century The Internet has ushered in an era of largely unfettered access to a wide variety of information. Yet, although so much of what the Internet has to offer is gratis, access to the Internet itself has never been free. This dichotomy lies at the heart of the prolonged and hairy "net neutrality" debate over what Internet service providers (ISPs) should charge for their services and what role those companies should play in managing the flow of information over their infrastructures. Flat-rate versus tiered-pricing structures, data equality versus prioritized content—the arguments rage on in the U.S. with no end in sight, even as other high-tech countries, South Korea in particular, claim to successfully moved beyond the issue. Net neutrality (flat-rate pricing and equal priority status given to all data) was not a successful business model for KT, Yoo said. The cost of KT's premium service?

Real-Time Data And A More Personalized Web - Smashing Magazine Advertisement As Web designers, we face a daily struggle to keep pace with advances in technology, new standards and new user expectations. We spend a large part of our working life dipping in and out of recent developments in an attempt to stay both relevant and competitive, and while this is what makes our industry so exciting to be a part of, it often becomes all too easy to get caught up in the finer details. Responsive Web design, improved semantics and rich Web typography have all seen their fair share of the limelight over the last year, but two developments in particular mark true milestones in the maturation of the Web: “real-time data” and a more “personalized Web.” Since the arrival of the new Web, we’ve been enraptured by social media.

scientists put free text-analysis tool on the web Now anyone can drag and drop text into a linguistic analysis tool powered by machine learning. By Andrew Myers and Tom Abate Ever wondered whether a certain TV show had a slant in favor of a political candidate? Stanford computer scientists have created a website that gives anyone who can cut and paste the ability to answer such questions, systematically and for free. The website is known as etcML, short for Easy Text Classification with Machine Learning.

Related:  Information Tecnology + Computer Science