Investigating the effects of teaching with Classroom Assessment Techniques (CATs) on the success of teacher candidates

Abstract

Classroom Assessment Techniques (CAT’s) provide substantial benefits for students and instructors in the process of teaching and learning. This study investigates the contribution of these techniques on the success of teacher candidates in the context of test and item statistics in measurement and evaluation courses. The study, utilized pretest-posttests control group experimental design, showed that the success levels of the teacher candidates in the experimental group who experienced CAT’s in classroom use were higher than those of the control group who had no experience of CAT’s.

Extended Summary

The study aims to investigate the effects of teaching with Classroom Assessment Techniques (CATs) on the success of teacher candidates regarding the test and item statistics of measurement and assessment courses. In context with this purpose the questions below were examined:
1.    Regarding the test and item statistics, is there a statistically significant  difference between pre-test and post-test means of success of the experimental  group of candidate teachers who were subjected to classes utilizing CATs and the control group of candidate teachers who did not experience CATs in the classroom?.
2.    Is there a statistically significant  difference between the post test success means related to competence in problem solving regarding the test and item statistics for the experimental  group of candidate teachers who were subjected to classes utilizing CATs and the control group of candidate teachers who did not experience CATs in the classroom?.
3.    Is there a statistically significant  difference between the post test success means related to interpretation of results of different problems regarding the test and item statistics for the experimental  group of candidate teachers who were subjected to classes utilizing CATs and the control group of candidate teachers who did not experience CATs in the classroom?.
CATs provide various important benefits in the teaching-learning process for both teachers and students. Some of these benefits are providing the competence to plan changes and make decisions that can develop the teachers and the students; achieving student participation in classes to motivate them for learning (Harwood, 1998); providing the students with useful feedback related to their learning (Soetaert, 1998; Steadman 1998) and ensuring that students express themselves more easily (Cuttic et.al, 1999).
This study is considered important in identifying the contribution of CATs to the success of learning and in teaching specific subjects that are regarded as difficult for the students to learn in some fields. 
Limitations of the study:
1.    The study is limited to the topic of test and item statistics in educational measurement and assessment classes (measures of central tendency and variation, dependent standard scores, the interpretation and type of distributions, correlations and item analysis).
2.    CATs is limited in the areas of application cards, problem recognition tasks, half sheet response, muddiest point, punctuated lectures, teacher-designed feedback forms.
3.    The study is limited to the third year day time and evening students of Abant İzzet Baysal University Department of Education, Faculty of Teaching Arts in the 2008-2009 educational period.
The study uses an experimental design model with pre-and post-test and control groups.
The study group consisted of a total of 68 teacher candidates studying at Abant İzzet Baysal University Department of Education, Faculty of Teaching Arts. 30 of these students (21 females and 9 males) attended day time classes whereas 38 of them attended evening classes (23 females and 15 males).Using randomization, the day time students formed the experimental group and the evening students took part in the control group.
A test with open-ended questions was utilized in order to determine the pre-application and post- application levels of students related to topics in measurement and assessment courses (measures of central tendency and variation, dependent standard scores, the interpretation and type of distributions, correlations and item analysis).
In the process of test preparation, attention was paid to include all the topics that were in the scope of the research. Expert views were sought to provide content validity and revisions were undertaken in line with the received feedback. Inter-rater reliability coefficient was found 0,97. Answer sheets were prepared first for the questions in the test in order to obtain reliable results in measurement and later a graded point was assigned to the question out of 100 in line with the ease and difficulty level of each question.
The results of the pre and post tests for both experimental and control groups were processed by the help of a statistical program and the solutions that required operations and interpretations were also taken into consideration.
'Two-way ANOVA for Mixed Measures" was used in the analysis of the first research question and independent samples t-test for the second and third research questions.
Following the identification of the subjects to be taught, development of the materials to be used in the process and finalizing the plans for teaching application that would be used both for the experimental and the control groups, the applications started in the spring term of 2008-2009 educational period. Applications lasted from the 4th week to the 9th week of the teaching process related to measurement and assessment courses.
A pre-test was given to the teacher candidates in the experimental and control groups in class hours in order to identify their competence levels in test and item statistics.
Before the teaching process, the teacher candidates in the experimental group were provided with information about the techniques and examples of different ways of utilizing these techniques in teaching test and item statistics.
The order of content and teaching of the topics were the same in both experimental and control groups. Explanations were given below related to the process undertaken in experimental and control groups about the teaching of the topics.
First of all, the concepts related to the week's topics were explained to the experimental group theoretically. This exercise was followed by examples related to the concepts and sample problems were solved. Later, students were asked to solve examples of similar processing situations. CATs were used in the process in order to identify whether meaningful learning took place and to support the process. The development and application of the tools related to the techniques in the format required in the teaching process was undertaken by the researcher along with the evaluation of the results. In line with the results obtained through utilizing the techniques, points that were not clear to the students or learned partially were reassessed and repeated in the first class of the following week. Revisions were done in context with the results in the teaching of the topics and in any other matter that was deemed important.
Similar to the experimental group, the control group was also given the concepts related to the week's topics in a theoretical manner. This exercise was followed by examples related to the concepts and sample problems were solved. However, in this group, CATs were not utilized in the teaching process. The teacher candidates were encouraged to ask for clarifications and ask questions in areas they felt unclear and additionally they were given reviews each week at the first class hour so as to reinforce learning and not to leave any topics half mastered.
At the end of the application process the pre-test that was administered previously was given as a post test to the teacher candidates.
The results of the study are summarized below:
The success levels of experimental and control group teacher candidates were rather low in test and item statistics at the beginning of the teaching process. There was no meaningful difference between the success grade means of the two groups.
The difference between the success levels of teacher candidates in both groups before and after the instruction was found to be statistically meaningful in favor of post test success levels.
The difference between the teacher candidates in regards to post test success levels was found to be meaningful in favor of the experimental group in the following areas and in the total success levels: to be able to solve different problems regarding the test and item statistics and to be able to interpret the results of the different problems.
Results show that teaching that utilizes CAT's has substantially increased the success of the teacher candidates in the related fields.
The results of the study support the research results stating that use of CAT's in the teaching process increases learning (Goldstein, 2007; Gaeddert, 2003; Juergen, 1999; Soetaert, 1998; Steadman, 1994; Walker, 1991).