background preloader


Facebook Twitter

MOST POPULAR INFOGRAPHICS. Any set of figures needs adjusting before it can be usefully reported. Fox News was excited: "Unplanned children develop more slowly, study finds.

Any set of figures needs adjusting before it can be usefully reported

" The Telegraph was equally shrill in its headline ("IVF children have bigger vocabulary than unplanned children"). And the British Medical Journal press release drove it all: "Children born after an unwanted pregnancy are slower to develop. " The last two, at least, made a good effort to explain that this effect disappeared when the researchers accounted for social and demographic factors.

But was there ever any point in reporting the raw finding, from before this correction was made? I will now demonstrate, with a nerdy table illustration, how you correct for things such as social and demographic factors. Correcting for an extra factor is best understood by doing something called "stratification". But then some clever person comes along and says: wait, maybe this whole finding is confounded by the fact that drinkers also smoke cigarettes? Data journalism and data visualization from the Datablog.

My top ten datasets - a guest post by Simon Rogers. has become one of the finest national open data initiatives in the world - it now has more data than the mighty in the US, with 4,223 datasets, compared to 2,876 over the Atlantic.

My top ten datasets - a guest post by Simon Rogers

It's not perfect - far too many links take you to front pages on other sites, rather than the data itself. It could also do with more help for the less-experienced user, witness the multitude of downloads on the Treasury's Combined Online Information System (COINS) dataset ( But nevertheless, what a resource. And where it really comes into its own is in the publication of immense datasets previously kept within the confines of the civil service, many of which show highly local data.

So, if I had to pick my top ten datasets here is where I would start: 1) National Public Transport Data Repository (NPTDR) If you want a complete dataset, look no further. 2) Combined Online Information System 3) Youth cohort study 4) England in dog mess Really. 6) Regional Labour Market Statistics. Marathon 2010. A 24-hour student data visualization competition Click here to download Visualizing Marathon 2010 Poster.

Marathon 2010

Welcome is proud to have held the inaugural Visualizing Marathon: a 24-hour student data visualization competition. Inspired by robotics competitions and science fairs, the Marathon was created to give design students an opportunity to collaborate and use design to help tackle real-world issues. Journalism in the Age of Data: A Video Report on Data Visualization by Geoff McGhee. Let's Intersect! Conditional Risk. Doctor Who: Every single journey through time detailed detailed by Information is Beautiful. As a spreadsheet. Doctor Who time travels of the Doctor: Information is Beautiful gets the data - what can you do?

Doctor Who: Every single journey through time detailed detailed by Information is Beautiful. As a spreadsheet

Illustration: David McCandless for the Guardian Last year, I created a visualisation of Time travel in TV & Films. You know. Star Trek, Back To The Future, Planet Of The Apes etc. Escher-like internet map could speed online traffic - tech - 08 September 2010. A novel map of the internet created by Marián Boguñá and colleagues at the University of Barcelona, Spain, could help make network glitches a thing of the past.

Escher-like internet map could speed online traffic - tech - 08 September 2010

Statistical Analysis - Stack Exchange. Research tips. The latest issue of the IJF is a bumper issue with over 500 pages of forecasting insights.

Research tips

The GEFCom2014 papers are included in a special section on probabilistic energy forecasting, guest edited by Tao Hong and Pierre Pinson. This is a major milestone in energy forecasting research with the focus on probabilistic forecasting and forecast evaluation done using a quantile scoring method. Only a few years ago I was having to explain to energy professionals why you couldn’t use a MAPE to evaluate a percentile forecast.

With this special section, we now have a tutorial review on probabilistic electric load forecasting by Tao Hong and Shu Fan, which should help everyone get up to speed with current forecasting approaches, evaluation methods and common misunderstandings. The section also contains a large number of very high quality articles showing how to do state-of-the-art density forecasting for electricity load, electricity price, solar and wind power. Statistics How To.

Information is beautiful

Statistics blogs. Statistical modeling, causal inference, and social science: Blog of Andrew Gelman's research group, featuring Bayesian statistics, multilevel modeling, causal inference, political science, decision theory, public health, sociology, economics, and literatu. How to visualize data with cartoonish faces ala Chernoff. FlowingData reader Chris asks: I was wondering, have you ever considered doing a Chernoff faces tutorial for R?

How to visualize data with cartoonish faces ala Chernoff

I think Chernoff faces are pretty interesting and I haven't seen much about them on the web. This wasn't the first time someone's asked how to make Chernoff faces, so I did a quick search. Guess what. Why visualise data? Why visualise data? In the introduction to his classic text, The Visual Display of Quantitative Information, Edward Tufte answers this question in three words.

“Graphics reveal data”. Problem solving flowchart (slightly crass) Stochastic. Randomness. There was a query on the SAS mailing list today - someone got inconsistent results for confidence intervals between Excel and SAS.


In Excel, they were using the confidence() function, which I'd not come across before. And I'm glad about that. See, to calculate a confidence interval, you multiply the standard error of the distribution for the critical value from the t-distribution. You can find that value using (say) R, with the qt() function or Excel, with the tinv() function. The t-distribution approximates the normal distribution as the sample size increases - you need a sample size of infinity for them to be exactly the same, but if the same size is large enough, then it's close.

Skyrails Blog. Someone finally (thanks Christian) sent a mail in the skyrails-public mailing list, asking some questions, so I answered in the mailing list. It'll just become an FAQ here instead. > I am aware of that skyrails is a sprout of a PhD research > project and these details may change. Actually, I'm not doing a PhD *yet*.