For weeks now I have been presenting material to students using presentations that I would classify as well-designed in light of cognitive load theory. These presentations (as seen before) are light on words and heavy on context. They require students to listen, process, and synthesize the material presented.
The trouble is, I'm not sure they know how!
I developed an assessment by hand. It went through a process of review by a subject matter expert (another instructor of the same course) as well as was reviewed by a professor in Educational Research and Measurement. The assessment was refined more than once and many errors were fixed in the process. I ran a large amount of statistical analysis to get some objective data on the exam and here is what the results looked like..
The analysis only takes into account the four matching questions and twenty multiple choice. There were also six short answer and essay questions that I scored using a rubric. Out of the 30 students who took the test, the highest score was 23 and the lowest score was 11. There were a couple of items that had negative z scores (indicating the item did not differentiate between whether a high scoring student answered the question correctly) so that was disappointing. This means either I did not cover it properly, I covered it erroneously, or the low scoring students who got it right guessed it right. Either way, that's unacceptable.
The mean score was 18, and the average score (including all the short answer/essays) was an 81.
Was the problem my instruction or their lack of studying? Good question. The answer is likely not so cut and dry, but rather a mix of the two factors.
After the test was done, a student approached me and mentioned that the hardest part was applying the knowledge she had learned. She told me she knew how to memorize definitions but when it came to applying the knowledge, she found that quite difficult. I would imagine it is this way for many students. So again I ask, is it my instruction or their inability to apply knowledge?
Perhaps I did not sufficiently instruct on how to apply the knowledge we learned?
I have more questions than I do answers, and I am not sure I will know the answers at this young spot in my career. I can say this, however, I will be looking at these questions for years to come.
Chris
1 comment:
Interesting results, Chris. I know you are doing your best and it can be frustrating when students do not respond at the level you hope.
My belief is that when students do not achieve, there is a problem with the instruction. In other words, I should design instruction so that it reaches ALL students. This is the ideal, of course, but worth pursuing. It sounds as though you are making these adjustments and recognizing that some students come without the ability to APPLY.
Providing many examples of application and practicing it in class, when you are present, should result in better performance on the "test."
Post a Comment