background preloader


Migration Policy Institute | The Open Source Data Science Masters by datasciencemasters Machine Learning Data Science Cheat Sheet I will update this article regularly. An old version can be found here and has many interesting links. All the material presented here is not in the old version. This article is divided into 11 sections. 1. A laptop is the ideal device. Even if you work heavily on the cloud (AWS, or in my case, access to a few remote servers mostly to store data, receive data from clients and backups), your laptop is you core device to connect to all external services (via the Internet). 2. Once you installed Cygwin, you can type commands or execute programs in the Cygwin console. Figure 1: Cygwin (Linux) console on Windows laptop You can open multuple Cygwin windows on your screen(s). To connect to an external server for file transfers, I use the Windows FileZilla freeware rather than the command-line ftp offered by Cygwin. You can run commands in the background using the & operator. $ notepad VR3.txt & A few more things about files Other extensions include File management 3. Examples Miscellaneous 4. Exercise

Data Science Book Harvard Business Review calls it the sexiest tech job of the 21st century. Data scientists are in demand, and this unique book shows you exactly what employers want and the skill set that separates the quality data scientist from other talented IT professionals. Data science involves extracting, creating, and processing data to turn it into business value. This guide discusses the essential skills, such as statistics and visualization techniques, and covers everything from analytical recipes and data science tricks to common job interview questions, sample resumes, and source code. The applications are endless and varied: automatically detecting spam and plagiarism, optimizing bid prices in keyword advertising, identifying new molecules to fight cancer, assessing the risk of meteorite impact. Complete with case studies, this book is a must, whether you're looking to become a data scientist or to hire one. About the Author Dr. Introduction Table of Content Chapter 1 - What Is Data Science?

Togaware: Hands-On Data Science with R R Language Tutorials -- EndMemo R Tutorials R is an open source system widely used in statistics, bioinformatics and finance field etc. It's data structure and working environment are perfect for analysis of large sized data. » R Installation and Quick Start §§ Data Types §§ Functions Selected§§ Plotting, Graphics§§ Statistical Analysis

Multinomial Goodness of Fit A population is called multinomial if its data is categorical and belongs to a collection of discrete non-overlapping classes. The null hypothesis for goodness of fit test for multinomial distribution is that the observed frequency fi is equal to an expected count ei in each category. It is to be rejected if the p-value of the following Chi-squared test statistics is less than a given significance level α. Example In the built-in data set survey, the Smoke column records the survey response about the student’s smoking habit. > library(MASS) # load the MASS package > levels(survey$Smoke) [1] "Heavy" "Never" "Occas" "Regul" As discussed in the tutorial Frequency Distribution of Qualitative Data, we can find the frequency distribution with the table function. > smoke.freq = table(survey$Smoke) > smoke.freq Heavy Never Occas Regul 11 189 19 17 Problem Suppose the campus smoking statistics is as below. Heavy Never Occas Regul 4.5% 79.5% 8.5% 7.5% Solution Answer Exercise

hcistats:start [Koji Yatani's Course Webpage] Disclaimer (Please read this first!) This wiki was initially started as my personal note of statistical methods commonly used in HCI research, but I decided to make it public and put more content in it because I think this may be useful for some of you (particularly if you use R). I will also put some codes for R, so you can quickly apply the methods to your data. This wiki does not emphasize mathematical aspects of statistics much, and rather tries to provide some intuitions of them. Keep in mind that I am not an expert of statistics. I also strongly recommend you to get the second opinion on your analysis from other kinds of resource before you really run a test. In this website, I use R to show some examples of how you can run statistical tests. What is this page about? Another reason I decided to make this public is there isn't really a good training of statistics for HCI people and isn't a good place to collect the knowledge of statistics for HCI research. Why R? Experimental Design