background preloader

Surveys

Facebook Twitter

Forms Matter. This story was co-published with ProPublica. Forms. They’re the often-tedious tasks that stand in the way of an online purchase, seeing the doctor, or filing your taxes. They may be boring, but they have tremendous power. Whether you’re filling out a form or building it yourself, you should be aware that decisions about how to design a form have all kinds of hidden consequences.

How you ask a question, the order of questions, the wording and format of the questions, even whether a question is included at all—all affect the final result. Let’s take a look at how. How You Ask the Question: The Census The Census determines everything from how our congressional districts are drawn to how $400 billion in federal aid is distributed to enforcement of civil rights laws.

But even small changes in the ways Census forms ask questions can have surprising effects, and not just because of the inherent limitations of asking people to put their identity in a box. The Order of the Questions: The Ballot. Version 1.1 of the likert Package Released to CRAN. Categorized as: R, R-Bloggers After some delay, we are happy to finally get version 1.1 of the likert package on CRAN.

Although labeled 1.1, this is actually the first version of the package released to CRAN. After receiving some wonderful feedback from useR! This year, we held back releasing until we implemented many of the feature suggestions. The NEWS file details most of what is in this release, but here are some highlights: Simplify analyzing and visualizing Likert type items using R’s familiar print, summary, and plot functions.

Create LaTeX and HTML formatted tables using the xtable package. There are four demos available: likert - Shows most of the features of the package using data from the Programme of International Student Assessment (PISA). The useR! Install.packages('likert',repos=' require(likert) The package is hosted on Github at [ require(devtools) install_github('likert','jbryer') Oh Ordinal data, what do we do with you? <a href=" Our Poll</a> What can you do with ordinal data? Or more to the point, what shouldn’t you do with ordinal data? First of all, let’s look at what ordinal data is.

It is usual in statistics and other sciences to classify types of data in a number of ways. Nominal is pretty straight-forward. Ordinal data But then we come to ordinal level of measurement. A postgraduate degree is higher thana Bachelor’s degree,which is higher thana high-school qualification, which is higherthan no qualification. There are four steps on the scale, and it is clear that there is a logical sense of order. Another example of ordinal level of measurement is used extensively in psychological, educational and marketing research, known as a Likert scale. The question at the start of this post has an ordinal response, which could be perceived as indicating how quantitative the respondent believes ordinal data to be. Well! Here’s what I think: All ordinal data is not the same. Common Approaches for Analyzing Likert Scales and Other Categorical Data. Analyzing Likert scale responses really comes down to what you want to accomplish (e.g.

Are you trying to provide a formal report with probabilities or are you trying to simply understand the data better). Sometimes a couple of graphs are sufficient and a formalize statistical test isn’t even necessary. However, with how easy it is to conduct some of these statistical tests it is best to just formalize the analysis. There are several approaches that can be used. Here are just a few of them. The code to set up the data for some testing is as follows. Note that this is the same code used in Plotting Likert Scales: 01.set.seed(1234) 02.library(e1071) 03.probs < - cbind(c(.4,.2/3,.2/3,.2/3,.4),c(.1/4,.1/4,.9,.1/4,.1/4),c(.2,.2,.2,.2,.2)) 04.my.n <- 100 05.my.len <- ncol(probs)*my.n 06.raw <- matrix(NA,nrow=my.len,ncol=2) 07.raw <- NULL 08.for(i in 1:ncol(probs)){ 09.raw <- rbind(raw, cbind(i,rdiscrete(my.n,probs=probs[,i],values=1:5))) 11.raw <- data.frame(raw) 12.names(raw) <- c("group","value")

Which hypothesis test for Likert scale? Drinking, sex, eating: Why don't we tell the truth in surveys? 27 February 2013Last updated at 13:56 GMT By Brian Wheeler BBC News Magazine Many people are under-reporting how much alcohol they are drinking. But what else are we fibbing to researchers about and why do we do it? "I have the occasional sweet sherry. Purely medicinal. " It is a classic British sitcom scene. An inveterate boozer telling a little white lie about how much they drink to a doctor or other authority figure.

But the tendency to paint a less-than-honest picture about your unhealthy habits and lifestyle is not just restricted to alcohol. It is understandable that people want to present a positive image of themselves to friends, family and colleagues. After all, the man or woman from the Office for National Statistics or Ipsos Mori can't order you to go on a diet or lay off the wine. It is a question that has been puzzling social scientists for decades. They even have a name for it - The Social Desirability Bias. "People respond to surveys in the way they think they ought to. Video: Survey Package in R. Sebastián Duchêne presented a talk at Melbourne R Users on 20th February 2013 on the Survey Package in R. Talk Overview: Complex designs are common in survey data. In practice, collecting random samples from a populations is costly and impractical.

Therefore the data are often non-independent or disproportionately sampled, and violate the typical assumption of independent and identically distributed samples (IDD). The Survey package in R (written by Thomas Lumley) is a powerful tool that incorporates survey designs to the data. Standard statistics, from linear models to survival analysis, are implemented with the corresponding mathematical corrections. This talk will provide an introduction to survey statistics and the Survey package. About the presenter: Sebastián Duchêne is a Ph.D. candidate at The University of Sydney, based at the Molecular Phylogenetics, Ecology, and Evolution Lab.

See here for the full list of Melbourne R User Videos. Asdfree by anthony damico. How I can prevent and treat missing data for questionnaires. Sample Size: How Many Survey Participants Do I Need? Please ensure you have JavaScript enabled in your browser. If you leave JavaScript disabled, you will only access a portion of the content we are providing. <a href="/science-fair-projects/javascript_help.php">Here's how.

</a> In order to have confidence that your survey results are representative, it is critically important that you have a large number of randomly-selected participants in each group you survey. So what exactly is "a large number? " For a 95% confidence level (which means that there is only a 5% chance of your sample results differing from the true population average), a good estimate of the margin of error (or confidence interval) is given by 1/√N, where N is the number of participants or sample size (Niles, 2006). The table below shows this estimate of the margin of error for sample sizes ranging from 10 to 10,000. You can quickly see from the table that results from a survey with only 10 random participants are not reliable.

How Many Subjects Do I Need for a Statistically Valid Survey? By Daryle Gardner-Bonneau, Ph.D. Office of Research Michigan State University/Kalamazoo Center for Medical Studies Reprinted from Usability Interface, Vol 5, No. 1, July 1998 Beware of people who give quick, pat answers in response to the question - "I’m doing a survey. How many subjects do I need? " They probably haven’t a clue as to what they’re talking about.

There aren’t any valid quick answers to this question. I work in the medical domain and advise faculty/residents/medical students on sample size determination for survey research studies all the time because, in medicine, survey results are often discounted and are not publishable unless you can support/validate the decision you made regarding sample size. We do this through power analysis, and except for the simplest power analyses, it's good to have the advice and assistance of a statistician. Usually, surveys involve a number of hypotheses. 1. An intro to power and sample size estimation -- Jones et al. 20 (5): 453 -- Emergency Medicine Journal. + Author Affiliations Correspondence to: Dr S R Jones, Emergency Department, Manchester Royal Infirmary, Oxford Road, Manchester M13 9WL, UK; steve.r.jones@bigfoot.com Abstract The importance of power and sample size estimation for study design and analysis.

Understand power and sample size estimation. Understand why power is an important part of both study design and analysis. Understand the differences between sample size calculations in comparative and diagnostic studies. Learn how to perform a sample size calculation Power and sample size estimations are measures of how many patients are needed in a study. In previous articles in the series on statistics published in this journal, statistical inference has been used to determine if the results found are true or possibly due to chance alone. Power and sample size estimations are used by researchers to determine how many subjects are needed to answer the research question (or null hypothesis). Figure 1 Figure 2 Table 2 Table 3 Figure 3. What is a large enough random sample? With the well deserved popularity of A/B testing computer scientists are finally becoming practicing statisticians. One part of experiment design that has always been particularly hard to teach is how to pick the size of your sample.

The two points that are hard to communicate are that: The required sample size is essentially independent of the total population size.The required sample size depends strongly on the strength of the effect you are trying to measure. These things are only hard to explain because the literature is overly technical (too many buzzwords and too many irrelevant concerns) and these misapprehensions can’t be relieved unless you spend some time addressing the legitimate underlying concerns they are standing in for.

As usual explanation requires common ground (moving to shared assumptions) not mere technical bullying. We will try to work through these assumptions and then discuss proper sample size. The problem of population size. The problem of effect strength. From Power Calculations to P-Values: A/B Testing at Stack Overflow. Note: cross-posted with the Stack Overflow blog. If you hang out on Meta Stack Overflow, you may have noticed news from time to time about A/B tests of various features here at Stack Overflow. We use A/B testing to compare a new version to a baseline for a design, a machine learning model, or practically any feature of what we do here at Stack Overflow; these tests are part of our decision-making process.

Which version of a button, predictive model, or ad is better? We don’t have to guess blindly, but instead we can use tests as part of our decision-making toolkit. I get excited about A/B tests because tests like these harness the power of statistics and data to impact the day-to-day details of our business choices. At the same time, there can be confusion about how to approach an A/B test, what the statistical concepts involved in such a test are, and what you do before a test vs. after a test. How sure do we need to be that we are measuring a real change? How sure do you need to be? Binary sample size calculator. Binary outcomes Suppose you want to test whether more people respond to one drug versus another, or whether one advertising campaign is more effective than another. In either case, you have a binary outcome. Someone either responds to the drug or they don’t.

They either buy the product or they don’t. In either case you have a probability of something happening, p1 for one group and p2 for the other, and you would like to test whether the two probabilities are different enough to tell apart, i.e. that their difference is statistically significant. If you are designing an experiment, how many people should use in each group? The answer depends on many factors. Sample size calculator The calculator below will estimate n, the number of subjects you need to assign to each group, based on your initial guesses at p1 and p2 and some common assumptions. Example Suppose p1 = 0.1 and p2 = 0.3. Note that n is the number in each group, so the total needed is 2n. Assuptions and details where.