How much do you know about assessments? Quiz 2: Validity & Defensibility. Here is the second of our series of quizzes on assessment subjects, authored in conjunction with Neil Bachelor of Pure Questions.
You can see the first quiz on Cut Scores here. As always, we regard resources like this quiz as a way of contributing to the ongoing process of learning about assessment. In that spirit, please enjoy the quiz below and feel free to comment if you have any suggestions to improve the questions or the feedback. Is a longer test likely to be more defensible than a shorter one? Take the quiz and find out. eLearning Design: Start with the Assessment! Posted By Doug Peterson I would venture to guess that many elearning designers/developers start designing their new elearning course “at the beginning” – they start writing content and gathering illustrations, then maybe work in some delivery considerations, go through the review and sign-off procedures … and only at the very end do they remember, “Oh, yeah, I should probably have some sort of quiz or test.”
Dr. Jane Bozarth is an accomplished elearning designer and developer, and her latest article in Learning Solutions Magazine is called Nuts and Bolts: The 10-Minute Instructional Design Degree. In her article she recognizes that a lot of elearning designers and developers come from other disciplines and may not have much formal training when it comes to elearning, so she provides eight recommendations for designing and developing the best elearning possible. Her #1 recommendation? For example, I used to support the call center agent training for a large telecommunications company. Professor Roddy Roediger on applying the retrieval practice effect to creating and administering assessments. In the first part of this interview, I asked Professor Roediger to explain how quizzes and tests give retrieval practice that helps you learn.
Now, he moves on to giving some practical advice on how to use this effectively when creating and administering assessments. Roddy Roediger Is it better to have formal quizzes or self-retrieval practice? They both work great, but there is a puzzle inherent in using self-testing types of retrieval practice. In self-testing, students can base their studying on their self-knowledge. Jeff Karpicke at Purdue University has done experiments comparing cases where learning is put under students’ control compared to that of a computer algorithm that requires the same schedule of testing for all students.
Karpicke finds that when you put the ability to restudy and test under the students’ control, they usually do not test themselves enough. How many times should one get people to retrieve things, and how soon after learning? Yes, absolutely. Evidence from Medical Education that Quizzes Do Slow the Forgetting Curve. I’ve recently come across a really exciting set of results from a study of doctors-in-training that show quizzes really do slow down the forgetting curve.
To remind you, once we learn something, we start to follow a downwards “forgetting curve”. Psychology research says that taking quizzes and tests can reduce the forgetting curve by giving us retrieval practice, which helps retain information. But it’s rare to see this shown in action quite so vividly as in an experiment conducted with doctors-in-training and published in the respected Medical Education journal. (The citation is: Larsen, D. P., Butler, A. So what do we mean by assessments anyway? When one person talks about assessments, they might mean something different to someone else.
The best definition I’ve seen of assessment is from our CEO, Eric Shepherd: “any systematic method of obtaining evidence by posing questions to draw inferences about the knowledge, skills, attitudes, and other characteristics of people for a specific purpose.” You may have seen that Eric and I, along with our colleague Brian McNamara, have just published a white paper called Learning and Assessment on SharePoint.
As well as describing how assessments can be used in Microsoft SharePoint, it also describes how assessments fit into the learning process. Here is one of the diagrams from the white paper, showing how assessments are used at specific times during the learning process: For each type of the assessment in the diagram, here is what we mean by them: For more information, check out our white paper. Where’s the evidence for assessment? Posted by John Kleeman I’m always on the lookout for hard evidence that assessment improves learning outcomes, and I’m indebted to Denise Whitelock, Lester Gilbert and Veronica Gale for alerting me to some powerful evidence in their research report at the 2011 CAA Conference.
This Australian study looked at more than 1500 students taking part in an applied maths course and showed that taking formative quizzes during a course improved learning outcomes. The study was conducted by two economics lecturers, Dr Simon Angus and Judith Watson, and is titled Does regular online testing enhance student learning in the numerical sciences? Robust evidence from a large data set. It was published in the peer-reviewed British Journal of Educational Technology Vol 40 No 2, 255-272 in 2009. Angus and Watson introduced a series of 4 online, formative quizzes into the course, and wanted to determine whether students who took the quizzes learned more and did better on the final exam than those who didn’t.
E-Assessment Association.