Blog post #8- Reich article

The research article by Gabriel Reich explains a qualitative study about how students answer multiple-choice questions on a standardized test and whether those questions evoke thinking that is called for in the state standards.  This study was completed since little is known about the cognitive processes students use when answering multiple-choice questions.  Reich used a small sample of thirteen heterogeneous 10th grade students in two classes of world history taught by the same teacher.  Certain criteria were used to select a participating teacher.  This teacher’s world history classes were observed for one month in order to judge whether or not the students had a chance to learn the material that was consistent with the state standards.  Think-alouds and posttest interviews were then conducted with each participant in a one-on-one session.  The results showed that adult assumptions about what these exams measure may be incorrect.  Also, the crucial factors that separated the performances of high from low scorers did not appear to be those that the test designers anticipated.

Questions:

1.       Do you think you would have had similar results with a larger participant group of students and teachers?  How about if the same study was conducted using a different content area?

2.      What was the role of the observer while in the classroom?  Were they simply a passive observer?  Do you think there were any observer effects on either the teacher or the students?

3.      Was there only one researcher completing the classroom observations?  Do you think multiple researchers would make this study more reliable since field notes were taken (to limit interpretations)?

4.      Do you think this study would be more reliable if it was conducted over a longer period of time or if multiple think-alouds were conducted?

5.      I don’t really understand the criteria you used to select the teacher.  “Ambitious in their teaching practice” seems vague.  Was the teacher given any training on teaching to meet state standards?  Would you have moved to another teacher if Mr. Heche appeared to not be teaching to meet the standards?

6.      Do you think two people analyzing the data are enough for inter-rater reliability?