Breedlovew@cofcAcademic Exchange Quarterly Fall 2004: Volume 8, Issue 3 Collaborative Testing andTest Performance William Breedlove, College of Charleston, SCTracy Burkett, College of Charleston, SCIdee Winfield, College of Charleston, SC William Breedlove, Ph.D isAssociate Professor of Sociology and Director of the Center for EffectiveTeaching and Learning. His researchinterests include the scholarship of teaching and learning and comparativesociology.Tracy Burkett, Ph.D isAssistant Professor of Sociology. Herresearch interests include social network analysis and political sociology. Idee Winfield, Ph.D is AssociateProfessor of Sociology and Associate Dean of the School of Humanities and Social Sciences. Her research interests include the scholarship of teaching and learningand economic sociology. AbstractDoescollaboration on a test, independent of prior collaborative learning, affecttest performance? In almost all thedescriptions, demonstrations, and studies of collaboration testing and academicachievement, the effect of collaborative learning on achievement is conflatedwith the effect of collaborative testing alone. The results of this study show that collaborative testing alone has a significantpositive association with test performance that varies by the level ofcognitive processing reflected in thetest question. KEYWORD:Assessment IntroductionCollaborativelearning techniques have been widely used at the primary and secondary levelsof education for some time now. Theeffects of collaborative learning are numerous and generalizableacross different populations and settings. Perhaps in reaction to this growing body of evidence and because moreemphasis is being placed on quality of teaching, increasing numbers of highereducation faculty have come to experiment with and adopt collaborativelearning. A review of the evidence forcollaborative learning reveals than in almost all the descriptions,demonstrations, and studies of collaboration, the effect of collaborativelearning on performance is conflated with the possible effect of collaborativetesting alone on student achievement. That is, most studies present cases where students either engaged inboth collaborative learning and collaborative testing, or more commonly,engaged collaborative learning but were evaluated individually. As a result, there currently is littleevidence or knowledge of the effect of collaboration on test performanceindependent of prior collaborative learning experience or training. This study seeks to answer the question ofwhether collaborative testing alone affects academic achievement, that is, whenstudents do not also engage in collaborative learning. Literature ReviewEvidenceof collaborative testing effects is sparse. By contrast, a large body of research has documented beneficial effectsof collaborative learning across diverse populations and disciplines. Studies have found positive effects amongelementary school children (Billington, 1994; Fuchs,Fuchs, Karns, Hamlet, Katzaroff,& Dutka, 1998), developmental students (Ley, Hodges & Young, 1995), and college students (Giraud & Enders, 2000; Gokhale,1995; Grzelkowski,1987; Guest & Murphy, 2000; Hanshaw,1982; Helmericks, 1993; Morgan, 2003; Muir &Tracy, 1999; Nowak, Miller, & Washburn, 1996; Rau & Heyl,1990; Reinhart, 1999; Russo & Warren, 1999; Sernau,1995; Zimbardo, Butler, & Wolfe, 2003). Significant effects have been found acrossdisciplines including business (Nowak et al, 1996), education (Morgan, 2003;Muir & Tracy, 1999), English composition (Russo & Warren, 1999),industrial technology (Gokhale, 1995), psychology(Guest & Murphy, 2000; Ley et al, 1995; Zimbardo et al, 2003), science (Hanshaw,1982), sociology (Grzelkowski, 1987; Helmericks, 1993; Rau & Heyl,1990; Reinhart, 1999; Sernau, 1995), and statistics (Giraud & Enders, 2000). Thesestudies and others find that collaborative learning leads to gains in bothacademic and nonacademic areas of student development. Among the many academicgains are improved performance on exams, greater retention of information,greater transfer of knowledge, and increased complexity of thought (Gamson, 1994; Johnson, Johnson, & Stanne,2000). Additionally, collaborativelearning fosters cooperation and connections with others (Muir & Tracy,1999; Rau & Heyl, 1990), develops skills criticalto workplace success such as team building and teamwork skills (Nowak et al,1996; Russo & Warren, 1999), humanizes the learning experience (Grzelkowski, 1987), eliminates cheating (Grzelkowski, 1987; Ley et al,1995), and is associated with higher levels of student satisfaction andmotivation to learn (Chickering & Gamson, 1991; Fuchs et al, 1998; Giraud& Enders, 2000; Sernau, 1995; Slavin,1980). Finally, collaboration lowerstest anxiety which may, in turn, improve test performance since high levels oftest anxiety have been found to negatively affect recall of learned information(Grzelkowski, 1987; Hanshaw,1982; Helmericks, 1993; Leyet al, 1995; Muir & Tracy, 1999; Russo & Warren, 1999). Howmight collaborative testing, independent of collaborative learning, affect testperformance? One means is throughreduced test anxiety and stress, both of which are associated with lower testperformance. Post-collaborative testsurveys show that students believe their levels of anxiety and stress werereduced by being allowed to work together (Morgan, 2003; Zimbardoet al, 2003). Another is that throughworking together, students can build upon each other’s knowledge with positiveperformance outcomes (Damon & Phelps, 1989). That building process can lead tounderstanding beyond what each individual could have accomplished working aloneand should, therefore, positively affect test performance. Further, anticipation of collaborativetesting may enhance test performance through a positive effect on motivation tolearn. Slavin(1980), for example, finds that collaboration creates a more favorable attitudetoward learning in general and this should result in better testpreparation. Students may also bemotivated by fears of looking ill-prepared, of failing their partners, or ofbeing seen as a “social loafer” (Morgan, 2003). Collaborativetesting, minus collaborative learning, has further implications for performanceon different kinds of cognitive tasks. Some cognitive tasks require the student to recall information. A test question may ask the student to choosefrom a list, fill in a blank, identify, match, or label. These types of questions emphasize recall, orthe ability to bring to mind the appropriate material. Bloom (1956) classifies this level ofthinking as “knowledge” and it represents the lowest level of cognitiveprocessing in his taxonomy. Alternatively, a test question may ask the student to apply, drawconclusions, make inferences, or form generalizations. These types of questions emphasize applyinggeneralizations, interpretation, and understanding patterns and relationshipsamong parts. This level of thinking is“application” and “explanation” and represents a higher level of cognitiveprocessing. Collaborativelearning facilitates the development of higher order thinking skills and shouldtherefore affect students’ performance on questions that emphasize applyinggeneralizations, interpretations, and making inferences. Collaborative testing without prior learningshould not affect performance on these kinds of questions. On the other hand, if “two brains are betterthan one”, students collaborating on a test without prior collaborativelearning may perform better on less complex questions that emphasize recallthan students working alone on similar questions.Teachersof introductory courses, and of many higher level courses too, are generallyconcerned that students learn the basic concepts and guiding theoreticalperspectives of their discipline. Although those concepts and theories are both abstractions, it isprobably true that theories represent a higher level of abstraction or generalizationthan do most concepts. To the extentthat is true, correctly answering questions about concepts and theoriesinvolves different levels of cognitive processing. The difference between questions aboutconcepts and questions about theories is not equivalent to the differencebetween levels of knowledge outlined by Bloom (1956), but the difference in complexity orabstraction does lead to plausiblehypotheses about the effect of collaborative testing on test performance. We anticipate no effect of collaborativetesting on correctly answering questions about theories. Students should gain from collaborativetesting when questions are about concepts. HypothesesH1:Collaborative testing is positively associated with test performance.H2:Collaborative testing is positively associated with performance on concept and“knowledge” questions.H3:Collaborative testing is not associated with performance on theory,application, and explanation type questions. Data and MethodsSubjectswere drawn from seven sections of an introductory sociology course taught at amedium-sized Southern liberal arts college: four sections from Fall 2001 andthree sections from Spring 2003. At thebeginning of the semester, students were told that they had the opportunity toparticipate in a study of learning. Onesection in each semester was chosen as a control group. In the experimental group, students agreeingto participate would be allowed take an exam with a randomly assigned same-partner. Collaborating pairs, spread outas much as space would allow, could quietly discuss questions, answers, andexplanations with their partners. The few students who chose to work alone weregiven the option of relocating to a quite room. Each student would have their own exam and would submit their own set ofexam answers. Those answers could differfrom their partner’s answers. Same-partnering was used to address any potential power imbalances or increasedanxiety that might arise from cross- pairings. In the Spring 2003 semester, one studentwithout a partner was added to an existing pair. All participation was voluntary and informedconsent was given. To examine whether collaboration, rather than a possibledifferential rate of learning or differential improvement in test takingmattered, we varied the number of partnered exams across semesters. The Fall 2001 semester took the second examwith a partner while the Spring 2003 semester took both the first and secondexams with a partner. In the results described below, collaboration is coded =1 and non-collaboration is coded = 0. Thecourse was divided into three approximately equal segments. Each instructorcovered the same general content for each segment with some allowance fordifferences in instructor interest. Because we allowed for variation in interest, different sets of testquestions were developed and those that represented content not covered by allinstructors were eliminated. Thisproduced the common sets of fifteen multiple choice questions for the two teststhat we examine here. These commonquestions were added to other questions that made up each instructor’s completetest. Test performance is measured asthe percentage correct of the fifteen common questions. Toexplore the association between collaborative testing and differences incognitive tasks, questions were classified according to whether they askedstudents about concepts or theories. Foreach test, seven of the fifteen questions were classified as concept questionsand seven were classified as theory questions. Examples of each type follow. ConceptQuestion: Marthahas a taste for fine clothing that she cannot afford on her small salary, soshe resorts to embezzling funds from her employer. This type of deviance is described by RobertK. Merton as:a.innovationb.rebellionc.retreatismd.ritualism TheoryQuestion: Inthe aftermath of the September 11th attack on the World Trade Center,American citizens have come together with a renewed sense of common identity,shared values, and greater unity. This outcome of an act of deviance would bemost emphasized by the ______ perspective.a.functionalistb.symbolic interactionistc.conflictd.social solidarity Othervariables that may have affected test performance were controlled for asfollows. More senior students shouldperform better due to their greater experience with test preparation andtesting. Further, poorly performingstudents are likely to have been selected out leaving mostly good studentsamong the junior and senior ranks. Female students may perform better under collaborative conditions givena preference for working together versus competitively. As reflected in SAT scores and otherindicators, minority students often come to higher education less prepared thanother students. Minority students arealso more likely to face distracting financial circumstances and perhaps tofeel more isolated and less supported than majority students. For these reasons, minority students’ testperformance was expected to be lower. Analysis and DiscussionAssociationsamong variables are estimated by zero-order and partial Pearson’s correlationcoefficients. In the Fall 2001 semester, on test 1 where all students workedalone, no significant association was found between experimental (n=91) orcontrol (n=40) group membership and overall test performance (r=.136, p >.05). The same is true for answeringconcept (r=.166, p > .05) and answering theory questions (r=.071, p >.05). At this point, there appears to beno significant difference in test performance between the experimental andcontrol groups. On test 2,collaboration was introduced and a positive zero-order association betweengroup membership and correctly answering concept questions (r=.245, p <.01)was detected, but no significant association for the theory questions (r=.025,p > .05) or overall test performance (r=138, p>.05). Collaborating students scored significantlyhigher than non-collaborators on the concept questions. Controlling for theeffects of class rank, gender, and minority status does not change thesubstantive results. The partialcorrelation coefficient for the association between collaboration and correctlyanswering concept questions is very modestly reduced but still significant(r=.239, p < .01). All otherassociations remain not significant. TheSpring 2003 semester had students in the experimental group collaborate on bothtest 1 and test 2. Again, a significantassociations between experimental (n=67) or control (n=14) group membership andperformance on concept questions was detected, but now for both test 1 (r=.344,p < .01) and test 2 (.423, p < .01). That the association is significant for both tests in Spring 2003, butonly for test 2 in the Fall 2001 semester, suggests that that earlier effectwas due to collaboration rather than a possible differential rate of learningor differential improvement test taking between test 1 and test 2. Consistent with expectations, there againis no significant association for performance on the theory questions on test 1(r=.071, p > .05) or test 2 (r=-.035, p > .05). The strength of the association betweencollaboration and performance on concept questions is sufficient to boost theassociation between collaboration and overall test performance to statisticalsignificance (r=.254, p < .05). Thepartial correlations show that introducing the control variables does notchange the substantive results. Conclusion Priorresearch reports a wide range beneficial effects of collaborative learningacross disciplines and diverse populations. Very few of these studies, however, carried collaboration into theprocess of evaluating student learning. Consequently, although much is known about how collaboration can affectlearning, little systematic evidence exists for the association betweencollaboration on tests and test performance. Thisstudy found that collaborative testing is significantly and positivelyassociated with performance on concept questions but not on theoryquestions. To the extent that theoryquestions represent a higher level of abstraction and answering theoryquestions requires higher cognitive processing, it appears that collaborativetesting, independent of collaborative learning, may not facilitate higherlevels of cognitive processing. Futureresearch might examine the effect of collaborative learning versuscollaborative learning combined with collaborative testing on performance onquestions requiring more complex reasoning. The significant association between effect of collaborative testing andconcept questions performance shows that collaborative testing may bebeneficial even if not combined with prior collaborative learning. Additional research should explore whetherthe gains on concept questions are retained or if it appears that students aresimply borrowing knowledge to answer a question without really learning why theanswer is correct. As some of thesubjects stated “two brains are betterthan one”. In what way are they better? Are they better simply because information missing in one can beborrowed from another without understanding, or are two brains better becausethey learn from each other and reinforce knowledge? Classroom teachers may find collaborativetesting useful for learning concepts, but they should examine whether thatlearning is retained, perhaps by retesting students individually at a laterdate. ReferencesBillington, R. (1994). Effects of collaborative test takingon retention in eight third-grade mathematics classes. The Elementary School Journal, 95, 23-32.Bloom,B. (1956). Taxonomy of educational objectives: The classification ofeducational goals: Handbook I, cognitive domain. New York:Longmans.Chickering, A. & Gamson, Z.(1991). Applying the seven principlesfor good practice in undergraduate education. San Francisco: Jossey Bass.Damon,W. & Phelps, E. (1989). Critical distinctions among three methods of peereducation. International Journal of Educational Research, 13, 9-19.Fuchs,L., Fuchs, D., Karns, K., Hamlett,C., Katzaroff, C, & Dutka,S. (1998). Comparisons among individual and cooperative performance assessmentsand other measures of mathematics competence. The Elementary School Journal,99, 23-52.Gamson, Z. (1994). Collaborative learning comes of age.Change, 26, 44-50.Giraud, G. & Enders, C. (2000). The effects of repeatedcooperative testing in an introductory statistics course. Paper presented at the Annual Meeting of theAmerican Educational Research Association.Gokhale, A. (1995). Collaborative learning enhancescritical thinking. Journal of Technology Education, 7, 1-2.Grzelkowski, K. (1987). A journey toward humanistictesting. Teaching Sociology, 15, 27-32.Guest,K. & Murphy, D. (2000). In support of memory retention: A cooperative oralfinal exam. Education, 121, 350-354.Hanshaw, L. (1982). Test anxiety, self-concept, and thetest performance of students paired with the same students working alone.Science Education, 66, 15-24.Helmericks, S. (1993). Collaborative testing in socialstatistics: Toward gemeinstat.Teaching Sociology, 21, 287-297.Johnson,D., Johnson, R. & Stanne, M. (2000).Cooperative learning methods: A meta-analysis. Retrieved from http://www.clcrc.com/pages/cl-methods.html.September 4 2002.Ley, K., Hodges, R., & Young, D. (1995). Partnertesting. Research and Teaching in Developmental Education, 12, 23-30.Morgan,B. (2003). Cooperative learning in higher education: Undergraduate studentreflections on group examinations for group grades. College Student Journal,37, 40-50.Muir,S. & Tracy, D. (1999). Collaborative essay testing. College Teaching, 47,33-36.Nowak,L., Miller, S., & Washburn, J. (1996). Team testing increasesperformance. Journal of Education for Business, 71, 253-256. Rau,W. & Heyl,B. (1990). Humanizing the college classroom: Collaborative learning and socialorganization among students. Teaching Sociology, 18, 141-155.Rinehart,J. (1999). Turning theory into theorizing: Collaborative learning in asociology theory course. Teaching Sociology, 27, 216-232.Russo,A., & Warren, S. 1999. Collaborativetest taking. College Teaching, 47, 18-20.Sernau, S. (1995). Using a collaborative problem-solvingapproach in teaching social stratification. Teaching Sociology, 23, 364-373.Slavin, R. (1980). Cooperative learning. Review of EducationalResearch, 50, 315-342.Zimbardo, P., Butler,L. & Wolfe, V. (2003). Cooperative college examinations: More gain, lesspain when students share information and grades. The Journal of ExperimentalEducation, 71, 101-126.geovisit(); |
|
