Final Blog Post

This class has completely changed my view of research and all that it entails.  I had previously conducted an action research project for my externship and therefore thought I had a pretty good understanding of educational research.  However I came into the class not knowing the definitions of quantitative and qualitative research.  I have learned that the research I previously conducted was only one of the vast amounts of research studies that can be performed.  I now understand the complexities of educational research and the focus one must have when choosing a topic to study.  I also understand the great difficulties one can have when trying to actually conduct research in a school.  Having never taken a statistics class, the statistical techniques pertaining to research were completely new to me.  I have not used them a lot since I am designing a qualitative study, but I now have a better grasp of their meanings and uses.  It has been interesting in discussing the ethical concerns of research because I am also taking a class right now on educational issues and policies in which we discuss ethical dilemmas.  I have been able to take some of the ideas from that class and think about them in terms of using human subjects in research.  My opinion of research has definitely changed as I now realize how time-consuming and in-depth it can really be.  I think one of the hardest parts for me was narrowing down a research topic, but I think if we were actually conducting this research that the data analysis would also prove to be a tremendous task.  I am interested in doing further research, but I think I would need to find a topic of great interest to me in order to stay focused throughout the entire process.

Blog Post #9- Field Notes

Last week I observed the students and environment at VCU from an overhead walkway.  I believe this was my first time observing people without having to "look" for something in particular.  At first it was difficult to just write down what I saw instead of my opinions on what I saw.  It was a warm sunny day and I first thing I noticed was the feel of being in a small city.  Public transportation was readily available and the streets were filled with restaurants, shops and street vendors.  Although there was a lot of traffic, most people seemed to be walking or biking.  I assumed these people were students since they were all wearing backpacks.  Most of the people I observed were walking quickly and not interacting or acknowledging others.  I noticed cell phones and Ipods being used by those who were walking.  It was not a quiet scene because of the traffic and loud music coming from several of the cars.  For the fifteen minutes I was observing I saw VCU students of multiple ethnicities and ages.  Mostly all of them were dressed casually on this warm day (shorts, jeans, sandals, shirtless).  I did notice multiple people wearing VCU t-shirts which might indicate school pride or simply be a result of the recent basketball games.  The student culture at VCU did seem considerably diverse from what I observed, however also had many commonalities such as the modes of transportation.

Blog post #8- Reich article

The research article by Gabriel Reich explains a qualitative study about how students answer multiple-choice questions on a standardized test and whether those questions evoke thinking that is called for in the state standards.  This study was completed since little is known about the cognitive processes students use when answering multiple-choice questions.  Reich used a small sample of thirteen heterogeneous 10th grade students in two classes of world history taught by the same teacher.  Certain criteria were used to select a participating teacher.  This teacher’s world history classes were observed for one month in order to judge whether or not the students had a chance to learn the material that was consistent with the state standards.  Think-alouds and posttest interviews were then conducted with each participant in a one-on-one session.  The results showed that adult assumptions about what these exams measure may be incorrect.  Also, the crucial factors that separated the performances of high from low scorers did not appear to be those that the test designers anticipated.

Questions:

1.       Do you think you would have had similar results with a larger participant group of students and teachers?  How about if the same study was conducted using a different content area?

2.      What was the role of the observer while in the classroom?  Were they simply a passive observer?  Do you think there were any observer effects on either the teacher or the students?

3.      Was there only one researcher completing the classroom observations?  Do you think multiple researchers would make this study more reliable since field notes were taken (to limit interpretations)?

4.      Do you think this study would be more reliable if it was conducted over a longer period of time or if multiple think-alouds were conducted?

5.      I don’t really understand the criteria you used to select the teacher.  “Ambitious in their teaching practice” seems vague.  Was the teacher given any training on teaching to meet state standards?  Would you have moved to another teacher if Mr. Heche appeared to not be teaching to meet the standards?

6.      Do you think two people analyzing the data are enough for inter-rater reliability?

Blog Post #7- Zumbrunn Research Proposal

The research proposal by Sharon Zumbrunn concerns the correlation between the feedback students receive on their writing and their writing grades.  Previous studies have shown that writing feedback can have a large influence on the quality of student writing, but less is known about students’ perception of the feedback and the effects of their perception.  Therefore this study seeks to examine the role of writing self-efficacy and writing feedback perceptions on elementary students’ writing grades.  Classroom teachers administered two surveys to 287 fourth grade students.  One survey tested self-efficacy and the other examined writing feedback attitudes.  The results showed that writing self-efficacy and writing feedback perceptions were significantly correlated with the self-reported writing grades.  Also suggested is that students’ perceptions of feedback may have a significant role in their experiences with writing.

Questions about the proposal:

1.       Which type of sampling technique was used?  Probability or nonprobability sampling?  Did the teachers volunteer to participate?

2.      Were students aware of whether or not their teachers would view the surveys?  (this may have skewed their answers in order to please their teacher)  Were students aware that their answers would be used for a research study?  How can you be sure of the accuracy of their self-reported grade in writing?

3.      How valid are the surveys used in this study?  Have they proven to be successful in the past?  With what type of participants were they previously used?

4.      To enhance reliability, were teachers given any guidance on how to administer these surveys and answer student questions other than simply reading the script?  Do you think the teachers could have unintentionally influenced student answers?

5.      Would this study have been more reliable if the surveys were administered more than once with this group of participants for stability over time (perhaps once at the beginning of the year and once at the end)? How about if the surveys were followed up by interviews for triangulation?

6.      Can this study be generalized to any larger group?

Blog #6- Cause and Effect

Cause and effect in social science research becomes far more complex than an experiment in the physical sciences.  In the physical sciences it is easier to directly correlate a result to the cause and to decrease other variables.  However, when working with human subjects many variables and extraneous situations come into play. 

I do not believe that developing causal relationships should be the primary goal of research.  Although it may be appropriate in certain situations, I think there are many other research designs that are just as, if not more, valuable.  Causal relationships are often difficult to prove when dealing with social sciences because of the nature of human subjects and the environments provided.

One of the main problems that arise when trying to develop cause is the idea that just because there is a relationship does not mean it is a causal one.  The relationship could be caused by many other factors.  The researcher tries to eliminate all factors except for the independent variable, such as the conditions and procedures.  However, in order to truly conclude that an intervention was causally related to the outcome, all extraneous variables must be controlled and this is often very difficult to do.  Some examples of factors that may influence a study include instrumentation, pretesting, experimenter effects and subject attrition.  For example, a longitudinal study concerning motivation may be swayed by students who moved or dropped out of school.  In another case, a researcher may assist students in receiving higher test scores or sway their answers on an interview.  A pretest may affect results if it changes attitudes or perceptions towards a topic.  Finally, a difference in results could be related to changes or unreliability in instruments or raters.  Cause and effect research can be valid, but direct control of the intervention and maximum control of extraneous variables should be closely monitored.  This is often difficult to regulate in an educational setting.

I definitely believe there is value to research that does not lead to causal inferences.  For example, even if a specific intervention was not conducted, simply observing naturally occurring phenomena can help build the knowledge base on teaching methods, student behavior or achievement.  This information could also be extremely valuable to pre-service teachers in learning classroom management techniques, strategies and teacher-student relationships.  Other types of research do not require such control and therefore may show more realistic and reliable findings.  You do not necessarily need to be looking for a desired outcome based on a cause to find valuable information and data.  For example, observing the book selection of elementary school boys in a library could help a teacher determine common interests which could lead to stronger reading achievement and motivation.

Blog post #5- Measurement

My current research problem is “How do the Directed Reading Thinking Activity and Story Mapping comprehension strategies utilized over a 20-week period retain their effectiveness in terms of usage and recall over a 2-year span with elementary students of varying reading ability levels?  Also, how is student motivation about comprehension tasks in the 2-year follow up?"

I have changed my design a bit to make my study more reliable.  The study will still be conducted at 5 elementary schools across 5 different school districts.  However, I will be at one school and four other reading specialists will be at the other schools.  They will be trained on how to conduct each of my measurements.

I have also decided to conduct the intervention over a 20-week period instead of an entire school year based off of many of the articles I have been reading.  The intervention will be done in the second half of the 2nd grade school year.  The teachers implementing the comprehension strategies will also be given significant training beforehand on how to teach each strategy using a specific framework.

Measurements to Collect Data

1.      Survey

a.   I will use a survey in the 2-year follow-up to measure whether students remember the strategies, how to use them and how often they still use them today.

b.   I will be measuring this with a short survey containing four open-opened questions.  The questions will ask students what the strategies were, how to use them, when and why they are useful and how often they still use them today.  The students will have the choice to either write their responses or respond orally. 

c.    This instrument will be valid because it is directly related to what I am trying to measure.  I will also be able to infer from this measurement whether students remember and use the strategies in the follow-up.  This one measurement tool will be used in conjunction with other measurements to confirm the validity of its findings (triangulation of data).  If similar results are found throughout all the measurements the validity will increase.  I will also confer with other literacy experts (reading specialists) about this survey to assure its validity. 

d.   This measurement will be reliable because it is a short survey which is more reliable than a long survey.  It will also be given to a heterogeneous group of students from five classes across five different school districts.  The students will have various reading levels which will make the results more comparable to other students.  In hopes of truthfulness, the students will be told that the survey is anonymous.  All of the reading specialists giving the survey will give students the same directions and the conditions will be similar.  The reading specialists will also all be trained on how to score this open-ended survey and we will compare results at the conclusion.

2.      Interviews

a.   The other reading specialists and I will anonymously interview both teachers and students throughout the study.  We will be trying to measure the teacher’s perspective on the effectiveness of the reading strategies immediately following the intervention and we will be trying to measure students’ attitude and motivation towards comprehension tasks two years after the intervention.

b.   In the interview with the classroom teacher we will ask them if they noticed a positive change in their students’ reading comprehension because of the strategies.  We will also ask if the students’ overall attitudes towards reading and comprehension tasks have changed at all as a result of the strategy intervention.  In the interview with the students we will ask questions such as:

-Did you enjoy using the comprehension strategies in 2nd grade?  Do you think they helped you?

-Do you now find comprehension tasks easy or difficult?

-Do you normally understand a story or book well after you have read it?

-Do you ever use some of those same strategies you learned in 2nd grade?

-Do you feel confident or unsure when asked to complete comprehension tasks?

-Would you rather complete a comprehension assignment or another reading task?

After completing the student interview we will gather their information and complete a short follow-up interview two weeks later to confirm their thoughts and ask additional questions based off of their first interview.

 c.  These interviews will be valid because we will have the perspectives of both the teachers and the students related to the main research question.  Some of the interview questions also overlap with the survey questions which will confirm validity if similar results are found.  Interviewing the teachers immediately following the intervention will allow me to see whether the strategies seemed to work well in order for the follow-up data in two years to be relevant and appropriate.  These interview questions have been specifically designed to help answer my research questions and I will also confer with the other literacy experts on whether to add or delete questions.

d.  These interviews will be reliable based on their stability over time.  We will be conducting two interviews with the students in order to assess whether their answers change or remain stable.  There is also inter-rater reliability as all the reading specialists will be trained on how to interview and interpret the answers.  The reading specialists will take detailed notes during the interviews.  At the conclusion we will compile and compare our data.  Also, the population of students is very heterogeneous which results in better reliability and generalizability. 

3.   Observations

a.   In the informal observations of teachers by the reading specialists we will be trying to measure whether the procedures are being followed and the strategies are being taught correctly in order to provide support where needed.  We would also be measuring student engagement, motivation and attitudes towards the strategies.

b.   In order to measure the above topics the observer will visit the classroom and take detailed field notes.  The observer will be cautious as to not interfere with the normal atmosphere of the class.

c.    These field notes will be valid because they will help ensure that the strategies in my research questions are being correctly taught in order to attain relevant data at the conclusion of the intervention.  Also, we will be able to compare field notes from these observations to follow-up data two years later.  Being literacy experts, the reading specialists will also know what to look for when they are in the classroom, making this measurement valid.  This is also only one source of data that will be combined with other measurement results.

d.   The observations will be reliable because there will be multiple reading specialists taking detailed field notes over an extended time period at various schools.  The schools will be different but the instruction and grade level will be the same.  Each reading specialist will also be trained in the observation procedure and what they should be looking for.  At the conclusion all of the reading specialists will compile and compare their data to look for similar results.  Since there will be multiple observations, the observer will be able to take extensive field notes and observe the class dynamics on different days.  The detailed field notes will eliminate the possible effect of the researcher’s subjective opinions.

4.       Records

a.   I will be trying to measure the characteristics of each school district in order to choose a school to work with that represents that district as a whole.

b.   I will measure the population of the school district by examining ethnicity, number of students using free and reduced lunch, district budget, dropout rates and test scores.  I will use information available to the public through district websites and articles.

c.   Using data collection will be a valid measurement because it provides crucial information to guide my research problem.  Analyzing multiple measures of population statistics will be more valid than simply basing my decisions off of one characteristic.

d.   These records are reliable because they have been tracked and recorded over the years.  The data has also been collected from multiple school officials representing their district.

Blog post #4- Ethical concerns

When conducting research with human subjects in educational settings there are many ethical concerns to keep in mind.  The first is the necessity of having consent from all participants before beginning any phase of the research interaction with students.  In a school setting this may often include parental consent since the students will be less than 18 years old.  This often presents obstacles as many parents are unsure of their children participating in research studies and may not want to consent.  Even if this does lead to less participants, it is simply one of the rules the researcher must follow.

Another ethical concern is privacy and confidentiality.  Especially in studies where achievement scores are going to be attained or personal interview questions will be asked, it is important for the researcher not to directly disclose this information to others.  This personal information could affect the student in many negative ways if it was released to others.  For example, a low achievement score or negativity during an interview could unfairly give that student’s teacher a different perspective of them.  A research proposal or article should not include the real names of students involved.  The subjects must feel as if they can trust the researcher and part of that trust is knowing that their answers will be kept private to a certain extent. 

Adequate interpretation and representation of data is another ethical concern as subjectivity can influence the research process.  For example, if a teacher was conducting research on their own class they may interpret the results more positively in order to reflect on their own teaching.  Also, if the researcher already knows the students participating in the study they may have a preconceived idea of how each student would do on an achievement test or how they would answer interview questions.  It is important to stay subjective when collecting and interpreting data from students.

Potentially harming students is another ethical concern involved with research in educational settings.  If students are put into a control group versus an intervention group, the circumstances of the control group cannot hinder students' education or ability to perform well in school.  The type of research being done must also be assessed for potential personal or psychological risk to the participants.  The students should not be put under any physical or mental stress while the research is being performed.

I personally think the most important thing for a researcher to keep in mind while conducting educational research is the aspect of potentially harming students.  The researcher must be sensitive to the students he or she is working with in order to obtain their results without significantly changing the lives of the participants.  The type of questions we ask during an interview must be assessed for appropriateness and subjectivity.  For example, certain topics may not be appropriate to bring up with first and second grade students.  The questions should not put students under any stress or make them feel negatively about themselves.  Many researchers want to compare a control group with an intervention group, but we must assess prior to the study if any harm will be done academically for the control group.  While educational research extends our knowledge base, it is also important to keep in mind that we are working with human subjects and their needs and wishes must be acknowledged first and foremost.  I think the results of the research study will be far less significant than any potential harm done to students.