background preloader

Refine, reuse and request data

Refine, reuse and request data

https://blog.scraperwiki.com/

Related:  PDF ScrapingvisualisationCollecting and Scraping dataSites spécialisésOutils & Doc

Get Started With Scraping – Extracting Simple Tables from PDF Documents As anyone who has tried working with “real world” data releases will know, sometimes the only place you can find a particular dataset is as a table locked up in a PDF document, whether embedded in the flow of a document, included as an appendix, or representing a printout from a spreadsheet. Sometimes it can be possible to copy and paste the data out of the table by hand, although for multi-page documents this can be something of a chore. At other times, copy-and-pasting may result in something of a jumbled mess.

How to Make an Interactive Network Visualization Networks! They are all around us. The universe is filled with systems and structures that can be organized as networks. Recently, we have seen them used to convict criminals, visualize friendships, and even to describe cereal ingredient combinations. We can understand their power to describe our complex world from Manuel Lima's wonderful talk on organized complexity. Now let's learn how to create our own. Hub - Find, grab and organize all kinds of data and media from online sources. OutWit Hub Light is free and fully operational, but doesn’t include the automation features and limits the extraction to one or few hundred rows, depending on the extractor. When purchasing the Pro version, you will receive a key to remove these limitations and unlock all advanced features. Buy Now. The inline help function covers light and pro features. Check it out and get acquainted with OutWit Hub at no cost OutWit Hub breaks down Web pages into their different constituents.

Knight Foundation finance ScraperWiki for journalism ScraperWiki is the place to work together on data, and it is particularly useful for journalism. We are therefore very pleased to announce that ScraperWiki has won the Knight News Challenge! The Knight Foundation are spending $280,000 over 2 years for us to improve ScraperWiki as a platform for journalists, and to run events to bring together journalists and programmers across the United States.

This Simple Data-Scraping Tool Could Change How Apps Are Made The number of web pages on the internet is somewhere north of two billion, perhaps as many as double that. It’s a huge amount of raw information. By comparison, there are only roughly 10,000 web APIs–the virtual pipelines that let developers access, process, and repackage that data. In other words, to do anything new with the vast majority of the stuff on the web, you need to scrape it yourself. Even for the people who know how to do that, it’s tedious. Ryan Rowe and Pratap Ranade want to change that.

Chapter 3: Turning PDFs to Text Update (1/18/2011): We originally wrote that we had promising results with the commercial product deskUNPDF's trial mode. We have since ordered the full version of deskUNPDF and tried using it on some of the latest payments data. Adobe’s Portable Document Format is a great format for digital documents when it’s important to maintain the layout of the original format. However, it’s a document format and not a data format. Health InfoScape When you have heartburn, do you also feel nauseous? Or if you're experiencing insomnia, do you tend to put on a few pounds, or more? By combing through 7.2 million of our electronic medical records, we have created a disease network to help illustrate relationships between various conditions and how common those connections are. Take a look by condition or condition category and gender to uncover interesting associations. About this data The information used for this visualization is based on 7.2 million patient records from GE's proprietary database, and represents some of the conditions that commonly affect Americans today.

The Simple Way to Scrape an HTML Table: Google Docs Raw data is the best data, but a lot of public data can still only be found in tables rather than as directly machine-readable files. One example is the FDIC’s List of Failed Banks. Here is a simple trick to scrape such data from a website: Use Google Docs. The table on that page is even relatively nice because it includes some JavaScript to sort it. But a large table with close to 200 entries is still not exactly the best way to analyze that data.

Six ventures bring data to the public as winners of Knight News Challenge Watch the winners present their projects via web stream at 1 p.m. PDT/ 4 p.m. EDT Saturday Sept. 22 here. SAN FRANCISCO -- (Sept. 20, 2012) -— Six media innovation ventures that make it easier to access and use information on local communities, air quality, elections, demographics and more received a total of $2.22 million today as winners of the Knight News Challenge: Data. The data challenge, one of three launched by the John S. and James L. How to Scrape Google Search Results with Google Sheets Learn how to easily scrape Google search results pages and save the keyword ranking data inside Google Spreadsheets using the ImportXML formula. This tutorial explains how you can easily scrape Google Search results and save the listings in a Google Spreadsheet. It can be useful for monitoring the organic search rankings of your website in Google for particular search keywords vis-a-vis other competing websites. Or you can exporting search results in a spreadsheet for deeper analysis. There are powerful command-line tools, curl and wget for example, that you can use to download Google search result pages.

Xpdf: Home Xpdf: A PDF Viewer for X Current version: 3.03 (2011-aug-15) Xpdf 3.03 supports PDF 1.7. The Xpdf software and documentation are copyright 1996-2011 Glyph & Cog, LLC. Email: derekn@foolabs.com PGP key (also available from the usual keyservers) Xpdf is an open source viewer for Portable Document Format (PDF) files. (These are also sometimes also called 'Acrobat' files, from the name of Adobe's PDF software.)

Graphical visualization of text similarities in essays in a book The problem A collection of essays is collated for readers with visualizing graphics. The graphics should both serve as a thematic and structural overview of each text, and pose the essay in question in relation to the other essays in the book. Convert PDF to Excel, Word with the PDF Converter - Able2Extract Open Select Convert Advanced PDF Handling Need image (scanned) PDF conversion to Excel, Word, and PowerPoint? Able2Extract Professional combines leading edge technology with our proprietary PDF conversion algorithm to deliver high quality conversions every time.

The Best Data Visualization Projects of 2011 I almost didn't make a best-of list this year, but as I clicked through the year's post, it was hard not to. If last year (and maybe the year before) was the year of the gigantic graphic, this was the year of big data. Or maybe we've gotten better at filtering to the good stuff. (Fancy that.)

Related:  Web Automation & Scraping Apps/ServicesScrap Datathematic NetworksWeb ScrapingOpenDatapythoncrawlergrafologizacja wikipediidevTech StuffData Jurnalism tools