ScraperWiki lets anyone scrape Twitter data without coding. The Obama administration’s open data mandate announced on Thursday was made all the better by the unveiling of the new ScraperWiki service on Friday. If you’re not familiar with ScraperWiki, it’s a web-scraping service that has been around for a while but has primarily focused on users with some coding chops or data journalists willing to pay to have someone scrape data sets for them. Its new service, though, currently in beta, also makes it possible for anyone to scrape Twitter to create a custom data set without having to write a single line of code.
Taken alone, ScraperWiki isn’t that big of a deal, but it’s part of a huge revolution that has been called the democratization of data. More data is becoming available all the time — whether from the government, corportations or even our own lives — only it’s not of much use unless you’re able to do something with it. First things first, I ran my query. Here’s what the data looks like viewed in a table in the ScraperWiki app. Police Dept. to Use Internet to Try to Stop Shootings. Predicting what topics will trend on Twitter - MIT Media Relations. Learning From Data - Online Course. A real Caltech course, not a watered-down version on YouTube & iTunes Free, introductory Machine Learning online course (MOOC) Taught by Caltech Professor Yaser Abu-Mostafa [article]Lectures recorded from a live broadcast, including Q&APrerequisites: Basic probability, matrices, and calculus8 homework sets and a final examDiscussion forum for participantsTopic-by-topic video library for easy review Outline This is an introductory course in machine learning (ML) that covers the basic theory, algorithms, and applications.
ML is a key technology in Big Data, and in many financial, medical, commercial, and scientific applications. What is learning? Live Lectures This course was broadcast live from the lecture hall at Caltech in April and May 2012. The Learning Problem - Introduction; supervised, unsupervised, and reinforcement learning. Is Learning Feasible? The Linear Model I - Linear classification and linear regression. Error and Noise - The principled choice of error measures. Stress. Marshall Kirkpatrick, Technology Journalist » How to Use Twitter Plus Needlebase to Discover Fabulous Things. My PR buddy Julie Wohlberg asked me tonight if I knew any good journalists she should invite to a social media conference in Florida called SheCon.
ReadWriteWeb’s Sarah Perez in Tampa was apparently inaccessible, so I took a few minutes to explore some possibilities…with web applications! I used the DIY data extraction and normalization service Needlebase, along with Twitter list search engine Tlists and of course Twitter itself to discover a list of journalists in the South of the US who have more than 2000 Twitter followers. (Mapped above) It wasn’t hard to do at all! Here they are in one Twitter list you can follow: Top South East Journalists What I did was use Tlists to find 3 Twitter lists of 500 journalists (this one, this one and this one), curated by people I was familiar with. This method isn’t complete until you say: hey everyone reading this post, if I missed anyone – let me know and we’ll add you to the list! How does one do such things at all though? Video 1Video 2Video 3. View From Above: The Startup Map. Featuring entrepreneurial communities outside Silicon Valley is a topic close to our hearts, particularly if they're outside the U.S.
Our "Never Mind The Valley" series challenges perceptions about locations for startups, and "entrepreneurial density" is an intriguing topic for VCs and entrepreneurs alike. Think you live in a startup hotspot? If you're reading this, you probably do - but you may not be visualizing just how hot (or not) your location is. For some entrepreneurs location is everything, while others might be intrigued to discover areas of activity in countries or cities they didn't expect.
Our Startups by Zip Codes post covered the United States, but a global overview is what we really want. It would be interesting to factor in "homeless" startups, as that would add a layer of more competitive locations (which tend to be expensive locations). The Startup Map (datalysed.com)
Sports. 16 Awesome Data Visualization Tools. From navigating the Web in entirely new ways to seeing where in the world twitters are coming from, data visualization tools are changing the way we view content. We found the following 16 apps both visually stunning and delightfully useful. Visualize Your Network with Fidg’tFidg’t is a desktop application that aims to let you visualize your network and its predisposition for different types of things like music and photos. Currently, the service has integrated with Flickr and last.fm, so for example, Fidg’t might show you if your network is attracted or repelled by Coldplay, or if it has a predisposition to taking photos of their weekend partying. As the service expands to support other networks (they suggest integrations with Facebook, digg, del.icio.us, and several others are in the works), this one could become very interesting.
BigSpy places stories at the top of the screen as they are dugg. Visualize Flickr Tags Over TimeTaglines from Yahoo! 7 Beautiful Data Visualizations (With Videos) Data visualizations can be gorgeous, but images often don't do them justice, especially if we're talking live, real-time visualizations (some of which require a lot of processing power and can't even be run on a regular desktop computer). We've dug out 7 videos of awesome data visualizations you simply must see. Skyrails Skyrails is a social network visualization system, but it can be used to visualize any graph or dataset. It has a built in programming language for processing, but it allows for the creation of menus so even non-experts can use it. As far as the actual visualizations go, they redefine awesomeness.
Radiohead - House of Cards After all the techy stunts they've pulled over the years, we're practically counting on Radiohead to be on the forefront of technology, and they didn't let us down with their last video, House of Cards. Fidg't Visualizer Animation of a Large Social Network There's not much info about the data used here, but boy does it look gorgeous! Project Palantir.