Final Blog Post

This class has completely changed my view of research and all that it entails.  I had previously conducted an action research project for my externship and therefore thought I had a pretty good understanding of educational research.  However I came into the class not knowing the definitions of quantitative and qualitative research.  I have learned that the research I previously conducted was only one of the vast amounts of research studies that can be performed.  I now understand the complexities of educational research and the focus one must have when choosing a topic to study.  I also understand the great difficulties one can have when trying to actually conduct research in a school.  Having never taken a statistics class, the statistical techniques pertaining to research were completely new to me.  I have not used them a lot since I am designing a qualitative study, but I now have a better grasp of their meanings and uses.  It has been interesting in discussing the ethical concerns of research because I am also taking a class right now on educational issues and policies in which we discuss ethical dilemmas.  I have been able to take some of the ideas from that class and think about them in terms of using human subjects in research.  My opinion of research has definitely changed as I now realize how time-consuming and in-depth it can really be.  I think one of the hardest parts for me was narrowing down a research topic, but I think if we were actually conducting this research that the data analysis would also prove to be a tremendous task.  I am interested in doing further research, but I think I would need to find a topic of great interest to me in order to stay focused throughout the entire process.

Blog Post #9- Field Notes

Last week I observed the students and environment at VCU from an overhead walkway.  I believe this was my first time observing people without having to "look" for something in particular.  At first it was difficult to just write down what I saw instead of my opinions on what I saw.  It was a warm sunny day and I first thing I noticed was the feel of being in a small city.  Public transportation was readily available and the streets were filled with restaurants, shops and street vendors.  Although there was a lot of traffic, most people seemed to be walking or biking.  I assumed these people were students since they were all wearing backpacks.  Most of the people I observed were walking quickly and not interacting or acknowledging others.  I noticed cell phones and Ipods being used by those who were walking.  It was not a quiet scene because of the traffic and loud music coming from several of the cars.  For the fifteen minutes I was observing I saw VCU students of multiple ethnicities and ages.  Mostly all of them were dressed casually on this warm day (shorts, jeans, sandals, shirtless).  I did notice multiple people wearing VCU t-shirts which might indicate school pride or simply be a result of the recent basketball games.  The student culture at VCU did seem considerably diverse from what I observed, however also had many commonalities such as the modes of transportation.

Blog post #8- Reich article

The research article by Gabriel Reich explains a qualitative study about how students answer multiple-choice questions on a standardized test and whether those questions evoke thinking that is called for in the state standards.  This study was completed since little is known about the cognitive processes students use when answering multiple-choice questions.  Reich used a small sample of thirteen heterogeneous 10th grade students in two classes of world history taught by the same teacher.  Certain criteria were used to select a participating teacher.  This teacher’s world history classes were observed for one month in order to judge whether or not the students had a chance to learn the material that was consistent with the state standards.  Think-alouds and posttest interviews were then conducted with each participant in a one-on-one session.  The results showed that adult assumptions about what these exams measure may be incorrect.  Also, the crucial factors that separated the performances of high from low scorers did not appear to be those that the test designers anticipated.

Questions:

1.       Do you think you would have had similar results with a larger participant group of students and teachers?  How about if the same study was conducted using a different content area?

2.      What was the role of the observer while in the classroom?  Were they simply a passive observer?  Do you think there were any observer effects on either the teacher or the students?

3.      Was there only one researcher completing the classroom observations?  Do you think multiple researchers would make this study more reliable since field notes were taken (to limit interpretations)?

4.      Do you think this study would be more reliable if it was conducted over a longer period of time or if multiple think-alouds were conducted?

5.      I don’t really understand the criteria you used to select the teacher.  “Ambitious in their teaching practice” seems vague.  Was the teacher given any training on teaching to meet state standards?  Would you have moved to another teacher if Mr. Heche appeared to not be teaching to meet the standards?

6.      Do you think two people analyzing the data are enough for inter-rater reliability?

Blog Post #7- Zumbrunn Research Proposal

The research proposal by Sharon Zumbrunn concerns the correlation between the feedback students receive on their writing and their writing grades.  Previous studies have shown that writing feedback can have a large influence on the quality of student writing, but less is known about students’ perception of the feedback and the effects of their perception.  Therefore this study seeks to examine the role of writing self-efficacy and writing feedback perceptions on elementary students’ writing grades.  Classroom teachers administered two surveys to 287 fourth grade students.  One survey tested self-efficacy and the other examined writing feedback attitudes.  The results showed that writing self-efficacy and writing feedback perceptions were significantly correlated with the self-reported writing grades.  Also suggested is that students’ perceptions of feedback may have a significant role in their experiences with writing.

Questions about the proposal:

1.       Which type of sampling technique was used?  Probability or nonprobability sampling?  Did the teachers volunteer to participate?

2.      Were students aware of whether or not their teachers would view the surveys?  (this may have skewed their answers in order to please their teacher)  Were students aware that their answers would be used for a research study?  How can you be sure of the accuracy of their self-reported grade in writing?

3.      How valid are the surveys used in this study?  Have they proven to be successful in the past?  With what type of participants were they previously used?

4.      To enhance reliability, were teachers given any guidance on how to administer these surveys and answer student questions other than simply reading the script?  Do you think the teachers could have unintentionally influenced student answers?

5.      Would this study have been more reliable if the surveys were administered more than once with this group of participants for stability over time (perhaps once at the beginning of the year and once at the end)? How about if the surveys were followed up by interviews for triangulation?

6.      Can this study be generalized to any larger group?

Blog #6- Cause and Effect

Cause and effect in social science research becomes far more complex than an experiment in the physical sciences.  In the physical sciences it is easier to directly correlate a result to the cause and to decrease other variables.  However, when working with human subjects many variables and extraneous situations come into play. 

I do not believe that developing causal relationships should be the primary goal of research.  Although it may be appropriate in certain situations, I think there are many other research designs that are just as, if not more, valuable.  Causal relationships are often difficult to prove when dealing with social sciences because of the nature of human subjects and the environments provided.

One of the main problems that arise when trying to develop cause is the idea that just because there is a relationship does not mean it is a causal one.  The relationship could be caused by many other factors.  The researcher tries to eliminate all factors except for the independent variable, such as the conditions and procedures.  However, in order to truly conclude that an intervention was causally related to the outcome, all extraneous variables must be controlled and this is often very difficult to do.  Some examples of factors that may influence a study include instrumentation, pretesting, experimenter effects and subject attrition.  For example, a longitudinal study concerning motivation may be swayed by students who moved or dropped out of school.  In another case, a researcher may assist students in receiving higher test scores or sway their answers on an interview.  A pretest may affect results if it changes attitudes or perceptions towards a topic.  Finally, a difference in results could be related to changes or unreliability in instruments or raters.  Cause and effect research can be valid, but direct control of the intervention and maximum control of extraneous variables should be closely monitored.  This is often difficult to regulate in an educational setting.

I definitely believe there is value to research that does not lead to causal inferences.  For example, even if a specific intervention was not conducted, simply observing naturally occurring phenomena can help build the knowledge base on teaching methods, student behavior or achievement.  This information could also be extremely valuable to pre-service teachers in learning classroom management techniques, strategies and teacher-student relationships.  Other types of research do not require such control and therefore may show more realistic and reliable findings.  You do not necessarily need to be looking for a desired outcome based on a cause to find valuable information and data.  For example, observing the book selection of elementary school boys in a library could help a teacher determine common interests which could lead to stronger reading achievement and motivation.

Blog post #5- Measurement

My current research problem is “How do the Directed Reading Thinking Activity and Story Mapping comprehension strategies utilized over a 20-week period retain their effectiveness in terms of usage and recall over a 2-year span with elementary students of varying reading ability levels?  Also, how is student motivation about comprehension tasks in the 2-year follow up?"

I have changed my design a bit to make my study more reliable.  The study will still be conducted at 5 elementary schools across 5 different school districts.  However, I will be at one school and four other reading specialists will be at the other schools.  They will be trained on how to conduct each of my measurements.

I have also decided to conduct the intervention over a 20-week period instead of an entire school year based off of many of the articles I have been reading.  The intervention will be done in the second half of the 2nd grade school year.  The teachers implementing the comprehension strategies will also be given significant training beforehand on how to teach each strategy using a specific framework.

Measurements to Collect Data

1.      Survey

a.   I will use a survey in the 2-year follow-up to measure whether students remember the strategies, how to use them and how often they still use them today.

b.   I will be measuring this with a short survey containing four open-opened questions.  The questions will ask students what the strategies were, how to use them, when and why they are useful and how often they still use them today.  The students will have the choice to either write their responses or respond orally. 

c.    This instrument will be valid because it is directly related to what I am trying to measure.  I will also be able to infer from this measurement whether students remember and use the strategies in the follow-up.  This one measurement tool will be used in conjunction with other measurements to confirm the validity of its findings (triangulation of data).  If similar results are found throughout all the measurements the validity will increase.  I will also confer with other literacy experts (reading specialists) about this survey to assure its validity. 

d.   This measurement will be reliable because it is a short survey which is more reliable than a long survey.  It will also be given to a heterogeneous group of students from five classes across five different school districts.  The students will have various reading levels which will make the results more comparable to other students.  In hopes of truthfulness, the students will be told that the survey is anonymous.  All of the reading specialists giving the survey will give students the same directions and the conditions will be similar.  The reading specialists will also all be trained on how to score this open-ended survey and we will compare results at the conclusion.

2.      Interviews

a.   The other reading specialists and I will anonymously interview both teachers and students throughout the study.  We will be trying to measure the teacher’s perspective on the effectiveness of the reading strategies immediately following the intervention and we will be trying to measure students’ attitude and motivation towards comprehension tasks two years after the intervention.

b.   In the interview with the classroom teacher we will ask them if they noticed a positive change in their students’ reading comprehension because of the strategies.  We will also ask if the students’ overall attitudes towards reading and comprehension tasks have changed at all as a result of the strategy intervention.  In the interview with the students we will ask questions such as:

-Did you enjoy using the comprehension strategies in 2nd grade?  Do you think they helped you?

-Do you now find comprehension tasks easy or difficult?

-Do you normally understand a story or book well after you have read it?

-Do you ever use some of those same strategies you learned in 2nd grade?

-Do you feel confident or unsure when asked to complete comprehension tasks?

-Would you rather complete a comprehension assignment or another reading task?

After completing the student interview we will gather their information and complete a short follow-up interview two weeks later to confirm their thoughts and ask additional questions based off of their first interview.

 c.  These interviews will be valid because we will have the perspectives of both the teachers and the students related to the main research question.  Some of the interview questions also overlap with the survey questions which will confirm validity if similar results are found.  Interviewing the teachers immediately following the intervention will allow me to see whether the strategies seemed to work well in order for the follow-up data in two years to be relevant and appropriate.  These interview questions have been specifically designed to help answer my research questions and I will also confer with the other literacy experts on whether to add or delete questions.

d.  These interviews will be reliable based on their stability over time.  We will be conducting two interviews with the students in order to assess whether their answers change or remain stable.  There is also inter-rater reliability as all the reading specialists will be trained on how to interview and interpret the answers.  The reading specialists will take detailed notes during the interviews.  At the conclusion we will compile and compare our data.  Also, the population of students is very heterogeneous which results in better reliability and generalizability. 

3.   Observations

a.   In the informal observations of teachers by the reading specialists we will be trying to measure whether the procedures are being followed and the strategies are being taught correctly in order to provide support where needed.  We would also be measuring student engagement, motivation and attitudes towards the strategies.

b.   In order to measure the above topics the observer will visit the classroom and take detailed field notes.  The observer will be cautious as to not interfere with the normal atmosphere of the class.

c.    These field notes will be valid because they will help ensure that the strategies in my research questions are being correctly taught in order to attain relevant data at the conclusion of the intervention.  Also, we will be able to compare field notes from these observations to follow-up data two years later.  Being literacy experts, the reading specialists will also know what to look for when they are in the classroom, making this measurement valid.  This is also only one source of data that will be combined with other measurement results.

d.   The observations will be reliable because there will be multiple reading specialists taking detailed field notes over an extended time period at various schools.  The schools will be different but the instruction and grade level will be the same.  Each reading specialist will also be trained in the observation procedure and what they should be looking for.  At the conclusion all of the reading specialists will compile and compare their data to look for similar results.  Since there will be multiple observations, the observer will be able to take extensive field notes and observe the class dynamics on different days.  The detailed field notes will eliminate the possible effect of the researcher’s subjective opinions.

4.       Records

a.   I will be trying to measure the characteristics of each school district in order to choose a school to work with that represents that district as a whole.

b.   I will measure the population of the school district by examining ethnicity, number of students using free and reduced lunch, district budget, dropout rates and test scores.  I will use information available to the public through district websites and articles.

c.   Using data collection will be a valid measurement because it provides crucial information to guide my research problem.  Analyzing multiple measures of population statistics will be more valid than simply basing my decisions off of one characteristic.

d.   These records are reliable because they have been tracked and recorded over the years.  The data has also been collected from multiple school officials representing their district.

Blog post #4- Ethical concerns

When conducting research with human subjects in educational settings there are many ethical concerns to keep in mind.  The first is the necessity of having consent from all participants before beginning any phase of the research interaction with students.  In a school setting this may often include parental consent since the students will be less than 18 years old.  This often presents obstacles as many parents are unsure of their children participating in research studies and may not want to consent.  Even if this does lead to less participants, it is simply one of the rules the researcher must follow.

Another ethical concern is privacy and confidentiality.  Especially in studies where achievement scores are going to be attained or personal interview questions will be asked, it is important for the researcher not to directly disclose this information to others.  This personal information could affect the student in many negative ways if it was released to others.  For example, a low achievement score or negativity during an interview could unfairly give that student’s teacher a different perspective of them.  A research proposal or article should not include the real names of students involved.  The subjects must feel as if they can trust the researcher and part of that trust is knowing that their answers will be kept private to a certain extent. 

Adequate interpretation and representation of data is another ethical concern as subjectivity can influence the research process.  For example, if a teacher was conducting research on their own class they may interpret the results more positively in order to reflect on their own teaching.  Also, if the researcher already knows the students participating in the study they may have a preconceived idea of how each student would do on an achievement test or how they would answer interview questions.  It is important to stay subjective when collecting and interpreting data from students.

Potentially harming students is another ethical concern involved with research in educational settings.  If students are put into a control group versus an intervention group, the circumstances of the control group cannot hinder students' education or ability to perform well in school.  The type of research being done must also be assessed for potential personal or psychological risk to the participants.  The students should not be put under any physical or mental stress while the research is being performed.

I personally think the most important thing for a researcher to keep in mind while conducting educational research is the aspect of potentially harming students.  The researcher must be sensitive to the students he or she is working with in order to obtain their results without significantly changing the lives of the participants.  The type of questions we ask during an interview must be assessed for appropriateness and subjectivity.  For example, certain topics may not be appropriate to bring up with first and second grade students.  The questions should not put students under any stress or make them feel negatively about themselves.  Many researchers want to compare a control group with an intervention group, but we must assess prior to the study if any harm will be done academically for the control group.  While educational research extends our knowledge base, it is also important to keep in mind that we are working with human subjects and their needs and wishes must be acknowledged first and foremost.  I think the results of the research study will be far less significant than any potential harm done to students.

Blog post #3- Sampling

My current research problem is “How do the Directed Reading-Thinking Activity and Story Mapping comprehension strategies utilized throughout one school year retain their effectiveness in terms of usage over a 2-year span with elementary students of varying reading ability levels?  Also, how is students’ motivation about comprehension tasks in the 2-year follow up?"

My participants for this study will consist of 5 different school districts in the central Virginia area including Richmond City, Henrico, Hanover, Chesterfield and Goochland.  I will choose one elementary school in each district that I believe is representative of the district as a whole in terms of socioeconomic status.  From those 5 elementary schools I will then ask the principal to suggest one 2nd grade class that consists of students with varying reading ability levels.  Therefore, I will have five 2nd grade classes of students as participants in this research which could be generalizable to all 2nd graders in central Virginia.  The students will be participating in the study during their 2nd grade school year and a follow-up will be completed 2 years later when the participants are in 4th grade.

Below are the pros and cons of using different types of probability sampling methods in this research study:

Probability Sampling	Pros	Cons
Random Sampling	-Every 2nd grader has an equal chance of being selected. -A large participant group could be used	-Too many variables would be involved -It might be harder to follow up 2 years later with a random sample -Might not be representative of the population -Would require numbering each participant in the population
Systematic Sampling	-Simple to select participants -Every 2nd grader has an equal chance of being selected	-Students selected could all have similar characteristics, such as a high reading level -Might not be representative of the population -It might be harder to follow up 2 years later with these students
Stratified Sampling	-More representative of the population than random or systematic sampling -A smaller sample could be chosen -Able to narrow down which participants the researcher wants to study from the subgroups	-It may be difficult to prepare and gather data about the subgroups in the population -I would have to assign proper weighting to each subgroup in order to represent the population -Students would still be chosen randomly from each subgroup which could result in students with all similar characteristics
Cluster Sampling	-Would work well with a large population -Saves time and money -Would be easy to select a school as a naturally occuring group	-The results are less accurate than other  probability sampling methods -Involves all random selection which may result in a sample that does not represent the population -Would be difficult to follow students in a longitudinal study since everything is random

Below are the pros and cons of using different types of non-probability sampling methods in this research study:

Non-Probability Sampling	Pros	Cons
Convenience Sampling	-Easy access to participants -Follow-up with participants would be simple -Less time-consuming -High participation rate -Would work well if studying very specific topic on very specific participants	-I could not generalize the convenience sample to the population -May result in bias such as participants in only one socioeconomic status
Purposive Sampling	-Easy to answer research problem with useful subjects -Less time-consuming -Helpful in qualitative research in order to understand the problem in depth -Selected participants will provide the best information	-May be difficult to generalize to the population as a whole -Results depend on the characteristics of the subjects -May be difficult to find the exact subjects to participate.
Quota Sampling	-More representative of the entire population -Generalizable to other similar subjects -Researcher is able to set guidelines of who they choose as participants	-More time-consuming because the researcher must make decisions about selecting the sample -Results will depend on the characteristics of the participants

I believe the sampling procedure that fits best with my research problem is quota sampling.  My target population is central Virginia, but I would like to have some control over who my participants are in order for the study to be generalizable to all 2nd grade students in central Virginia.  I selected the five school districts and I will non-randomly decide on one school in each district that is representative of that population.  At each school I will be asking the principal to suggest a class that consists of 2nd grade students with varying reading abilities to participate in the study.  Therefore I am not handpicking each participant, but the sample is also non-random.  If I chose any of the probability sampling methods there would be too many variables involved such as SES, student achievement and my ability to follow these students 2 years later.  The convenience sampling method would not allow me to generalize my results to all 2nd grade students in central Virginia.  I think the purposive sampling technique would also be too specific in trying to generalize to all 2nd grade students.  I am trying to get a picture of whether these comprehension strategies are effective with students of varying ability levels and living in varying SES neighborhoods.  I think by narrowing my sample the way I have I will be able to achieve these results.  The quota sampling technique seems to fit best with setting guidelines for my participants but not being too specific.

Blog post #2- Research

After spending time researching I have come up with a new and refined research question.  My new question is "How do the Directed Reading-Thinking Activity and Story Mapping comprehension strategies retain their effectiveness in terms of usage over a 2-year span with elementary students?  Also, how is students' motivation about comprehension tasks in the 2-year follow up even if they are not still using the strategies?"

One of the articles I found is entitled "The Effects of Three Instructional Methods on the Reading Comprehension and Content Acquisition of Novice Readers."  This article explored the effects of three instructional comprehension methods including picture walks, the DR-TA and the KWL.  There were three intervention groups (one for each method) and a control group.  The results showed significant positive effects of the DR-TA on reading comprehension.  Also discussed is the consistency of this study with earlier studies concerning the DR-TA.

I believe this article relates to my research problem because the usage of the DR-TA strategy is at the heart of my problem.  I would like to begin my literature review with studies about the effectiveness of the DR-TA and story mapping strategies.  This will set a purpose for why my research question is important and should be studied further.  It needs to be shown that these strategies will be effective from the beginning in order to complete a longitudinal study.

Another article I found is entitled "Early Intervention in Reading:  Results of a Screening and Intervention Program for Kindergarten Students."  This study involved kindergarten students who were selected for a reading invention program on the basis of their ability to name letters of the alphabet.  The intervention included tutoring over a 20-week period using various strategies.  The results were that this group of students consistently outperformed a control group.  However, there was also a 2-year longitudinal follow-up which showed lower rates in special education or remedial reading programs for the students involved in the intervention.

This article relates to my research problem by showing that early intervention can have lasting effects.  I am trying to do something similar with comprehension strategies in 2nd grade.  Since there is no research on my specific topic, I plan to research other longitudinal reading intervention programs and see what their lasting effects were.  For the second part of my literature review I will relate previous studies to my own problem.  I will try to organize these studies by topic instead of by author.

I feel I am still working through many issues with my literature review.  My only experience is with searching for articles that are exactly linked to my topic, and therefore I am having trouble drawing the line of how relevant something is to my own research problem.  I want to relate other studies explicitly to my own since that is one of the criteria for literature reviews.  However, I am struggling with how related they need to be and how I can connect them.  Particularly in the second half of my literature review where I will be discussing other longitudinal studies, I am thinking about how to find relationships between the topics.
Another question I have is whether my research problem is now more focused as I have been trying to narrow it down.  Coming up with search words to find new articles has also been a struggle for me as I try to narrow my topic.
My last question is concerning whether we can also use books if we have the required 6 articles as well.  I am not finding many articles on the DR-TA strategy but I know I have read about it in multiple books.

References

Dougherty Stahl, K.A. (2008). The effects of three instructional methods on the

                 reading comprehension and content acquisition of novice readers.

                 Journal of Literacy Research, 40(3), 359-393.

Lennon, J.E., & Slesinski, C. (1999). Early intervention in reading:  Results of a

                 screening and intervention program for kindergarten students. School

                 Psychology Review, 28(3), 353-364.

Blog Post #1- Research Questions

The area of research I chose is related to literacy since I am on the reading specialist track.  I am especially interested in the area of comprehension because of my experiences working with children who struggle with comprehension.  When I first started out, I didn't know of any strategies to be able to help them.  As I learned different strategies and tried to implement them, I often wondered what their lasting impact really was.  I am thinking about basing my research problem specifically on three reading comprehension strategies that would be taught to students.  These include the Directed Reading-Thinking Activity (DR-TA), picture walks and graphic organizers (story maps).

My quantitative research question is:

What is the effect on standardized test scores by implementing three reading comprehension strategies in second grade classrooms over the course of one school year?

My qualitative research question is:

How do three research-based reading comprehension strategies retain their effectiveness is terms of usage and motivation over a two-year span with elementary students?

My second question is qualitative-based because the data collected would include interviews, observations, surveys and field notes.  The data would be more narrative in nature as opposed to the quantitative question in which the data would include numbers and statistics.  The role of the researcher in the qualitative question would be more involved and working directly with the teachers and students through interviews and observations.  This role is usually typical of qualitative research.  In the quantitative question the standardized tests could be administered without the research being involved and the researcher would not develop a close relationship with any of their subjects.  For my qualitative problem the participants would be several second grade classes in a middle-class neighborhood whose teachers had volunteered to participate.  The teachers would be trained on the specific comprehension strategies in order to implement them effectively and enthusiastically.  The following year the third grade teachers of these students would be advised about the study and would allow students to continue using their own comprehension strategies.  This same group of students would then be re-assessed two years later.  Therefore, this sample group is small and purposeful which coincides with qualitative research.  In my quantitative study the participant group could be much larger and more varied since I would not be following specific students.  Overall, the qualitative study would have the purpose of capturing and describing naturally occurring behavior while the quantitative study would be used to show a relationship and describe statistics.