Daniel Lakens: How a p-value between 0.04-0.05 equals a p-value between 0.16-017. Question 1: Would you be inclined to interpret a p-value between 0.16- 0.17 as support for the presence of an effect, assuming the power of the study was 50%? Write down your answer – we will come back to this question later. Question 2: If you have 95% power, would you be inclined to interpret a p-value between 0.04 and 0.05 as support for the presence of an effect? Write down your answer – we will come back to this question later.
If you gave a different answer on question 1 than on question 2, you are over relying on p-values, and you’ll want to read this blog post. If you have been collecting larger sample sizes, and continue to rely on p < 0.05 to guide your statistical inferences, you’ll also want to read on. When we have collected data, we often try to infer whether the observed effect is random noise (the null hypothesis is true) or a signal (the alternative hypothesis is true). The latter is easy. If the power of the test is 50%, a p-value between 0.16-0.17 is 1.1% likely. The New Statistics: Confidence Intervals, NHST, and p Values (Workshop Part 1) The New Statistics: Estimation and Research Integrity. FAQ/effectSize - CBU statistics Wiki.
The scales of magnitude are taken from Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Hillsdale, NJ: Lawrence Erlbaum Associates (see also here). The scales of magnitude for partial are taken from Table 2.2 of Murphy and Myors (2004). Bakker et al. (2019) note that contextual effect sizes should be used wherever possible rather than 'canned' effects like Cohen's.
There is also a table of effect size magnitudes at the back of Kotrlik JW and Williams HA (2003) here. An overview of commonly used effect sizes in psychology is given by Vacha-Haase and Thompson (2004). Whitehead, Julious, Cooper and Campbell (2015) also suggest Cohen's rules of thumb for Cohen's d when comparing two independent groups with an additional suggestion of a d < 0.1 corresponding to a very small effect. Kraemer and Thiemann (1987, p.54 and 55) use the same effect size values (which they call delta) for both intra-class correlations and Pearson correlations.
Definitions References. Centre for Multilevel Modelling | Centre for Multilevel Modelling. The Centre for Multilevel Modelling (CMM) is a research centre based at the University of Bristol. Our researchers are drawn from the Graduate School of Education, Bristol Veterinary School and School of Geographical Sciences. Multilevel Modelling is a University Research Theme. We collaborate with a range of researchers working with multilevel models. Multilevel Modelling is one of the basic techniques used in quantitative social science research for modelling data with complex hierarchical structures.
The Multilevel Modelling research theme focuses on producing new statistical methods for tackling research questions, developing new software for implementing this methodology and disseminating these techniques to the national and international social science community. Lawrence Moon. Here's a wealth of information that I found useful for learning to implement mixed models in SPSS. My favourite book on statistics is the latest version of Andy Field's "Discovering statistics using SPSS" and his website ("Statistics Hell") is an excellent, albeit unorthodox, resource!
The chapter on mixed models has a couple of typos to navigate. Bristol's Centre_for_Multilevel_Modelling is outstanding. I attended one of their courses - it was excellent. The UCLA website has some great resources for SPSS: Repeated measures analysis with SPSS, Using SAS Proc Mixed to Fit Multilevel Models, Hierarchical Models, and Individual Growth Models, How to obtain pairwise comparisons of effects and interactions. David Garson's Statnotes are also very thorough. Here are some key resources: Singer, J. Singer, J. Here are some other important references: Rubin, L.
Landau, S. & Everitt, B. Heck, R. Krueger, C. & Tian, L. Gueorguieva, R. & Krystal, J. Chan, Y. Chan, Y. SPSS. Bickel, R. West, B. SPSS Tutorials and Statistical Guides | Laerd Statistics. SRME Home. Statistics Hell. Statistical Computing.