The effect of test instruction and risk taking tendency on psychometric properties of multiple choice tests<p>Test yönergesi ve risk alma eğiliminin çoktan seçmeli testlerin psikometrik özelliklerine etkisi
Keywords:
Correction formulas, test instruction, psychometric properties of test, validity and reliability, risk taking tendency, Düzeltme formülleri, test yönergesi, testin psikometrik özellikleri, geçerlik ve güvenirlik, risk alma eğilimiAbstract
The research is a quasi-experimental research which aims to investigate the effect of test instruction and risk taking tendency on the psychometric properties of multiple choice tests. The study group of the research comprised of 220 undergraduate students studying at different departments of Ankara University Faculty of Educational Sciences in 2014-2015 spring term. An achievement test of measurement and evaluation course consisting of 24 items was applied to students in the study group. The test was applied to four similar groups with different instructions. A risk taking scale was applied to determine the risk taking tendency of the students. The data were analyzed by using one way ANOVA, dependent sample t-test, independent sample t-test, Feldt test, Fisher’s z test, Pearson correlation coefficient, Spearman’s rank-order correlation coefficient and confirmatory factor analysis. The findings showed that corrected score means, uncorrected score means and reliability coefficients calculated from corrected scores of tests applied with different instructions were significantly different whereas reliability coefficients calculated from uncorrected scores were not significantly different. The validity of the test applied with the instruction stating that no correction for guessing would be made was the highest one. In general, risk taking tendency did not significantly change test scores.
Extended English abstract is in the end of PDF (TURKISH) file.
Özet
Bu araştırma, test yönergesi ve risk alma eğiliminin çoktan seçmeli testlerin psikometrik özellikleri üzerindeki etkisini incelemeyi amaçlayan yarı deneysel bir araştırmadır. Araştırmanın çalışma grubu, 2014-2015 bahar döneminde Ankara Üniversitesi Eğitim Bilimleri Fakültesinde öğrenim gören 220 öğrenciden oluşmaktadır. Çalışma grubundaki öğrencilere ölçme ve değerlendirme dersi başarısını ölçen 24 maddelik çoktan seçmeli test uygulanmıştır. Başarı testi birbirine benzer dört öğrenci grubuna farklı yönergelerle uygulanmıştır. Öğrencilerin risk alma eğilimini belirlemek için risk alma ölçeği uygulanmıştır. Veriler; tek faktörlü varyans analizi, ilişkili örneklemler için t-testi, ilişkisiz örneklemler için t-testi, Feldt testi, Fisher z testi, Pearson korelasyon katsayısı, Spearman sıra farkları korelasyon katsayısı ve doğrulayıcı faktör analizi teknikleriyle analiz edilmiştir. Araştırmadan elde edilen bulgular, farklı yönergelerle uygulanan testlerin ham puan ortalamaları, düzeltilmiş puan ortalamaları ve düzeltilmiş puana göre hesaplanan güvenirlik katsayıları arasında manidar fark olduğunu; buna karşın ham puanlara göre hesaplanan güvenirlik katsayıları arasında manidar fark olmadığını göstermektedir. Şans başarısı için herhangi bir düzeltme yapılmayacağını bildiren yönergeyle uygulanan testin geçerliği, diğer yönergelerle uygulanan testlerin geçerliğinden yüksek çıkmıştır. Risk alma eğilimi genel olarak test puanlarında manidar değişikliğe yol açmamıştır.
Downloads
Metrics
References
Abu-Sayf, F. K. (1975). Relative effectiveness of the conventional formula score. The Journal of Educational Research, 69(4), 160-162.
Albanese, M. A. (1988). The projected impact of the correction for guessing on individual scores. Journal of Educational Measurement, 25(2), 149-157.
Alnabhan, M. (2002). An emprical investigation of the effects of three methods of handling guessing and risk taking on the psychometric indices of a test. Social Behavior and Personality, 30(7), 645-652.
Alsawalmeh, Y. M., and Feldt, L. S. (1994). A modification of Feldt's test of the equality of two dependent alpha coefficients. Psychometrika, 59(1), 49-57.
Angoff, W. H., and Schrader, W. B. (1981). A study of alternative methods for equating rights scores to formula scores. ETS Research Report Series.
Bayar, N., and Sayıl, M. (2005). Brief report: risk taking behaviors in a non-western urban adolescent sample. Journal of Adolescence, 28, 671-676.
Baykul, Y. (2010). Eğitimde ve psikolojide ölçme: klasik test teorisi ve uygulaması (2. Baskı). Ankara: Pegem Akademi Yayınları.
Betts, L. R., Elder, T. J., Hartley, J., and Trueman, M. (2009). Does correction for guessing reduce students’ performance on multiple‐choice examinations? Yes? No? Sometimes?. Assessment &Evaluation in Higher Education, 34(1), 1-15.
Burton, R. F. (2002). Misinformation, partial knowledge and guessing in true/false tests. Medical Education, 36(9), 805-811.
Büyüköztürk, Ş., (2012). Sosyal bilimler için veri analizi el kitabı. (17. Baskı). Ankara: Pegem Akademi.
Can, A. (2013). SPSS ile Bilimsel Araştırma Sürecinde Nicel Veri Analizi. Ankara: Pegem Akademi Yayınları.
Creswell, J. W. (2012). Educational research: Planning, conducting and evaluating qualitative research (4th Edition). Boston: Pearson Education.
Crocker, L., and Algina, J. (1986). Introduction to classical ve modern test theory. Belmont: Wadsworth Group.
Cross, L. H., and Frary, R. B. (1977). AN empirical test of Lord's theoretical results regarding formula scoring of multiple‐choice tests. Journal of Educational Measurement, 14(4), 313-321.
Çelen, Ü., ve Demirtaşlı, N. Ç. (2006). Düzeltme yönergesinin testin psikometrik özelliklerine etkisi. Hacettepe Üniversitesi Eğitim Fakültesi Dergisi, 30(2006), 82-91.
Çokluk, Ö., Şekercioğlu, G., ve Büyüköztürk, Ş. (2012). Sosyal bilimler için çok değişkenli istatistik: SPSS ve LISREL uygulamaları. Ankara: Pegem Akademi Yayınları.
Ebel, R. L. (1965). Measuring educational achievement. New Jersey: Prentice-Hall.
Espinosa, M. P., and Gardeazabal, J. (2010). Optimal correction for guessing in multiple-choice tests. Journal of Mathematical Psychology, 54(5), 415-425.
Frary, R. B. (1982). A simulation study of reliability and validity of multiple-choice test scores under six response-scoring modes. Journal of Educational and Behavioral Statistics, 7(4), 333-351.
Glass, G. V., and Wiley, D. E. (1964). Formula scoring and test reliability. Journal of Educational Measurement, 1(1), 43-49.
Haladayna, T. M. (1997). Writing test İtems to evaluate higher order thinking. USA: Ally ve Bacon.
Jöreskog, K. G., and Sörbom, D. (1996). LISREL 8 user's reference guide. Scientific Software International.
Kubinger, K. D., and Wolfsbauer, C. (2010). On the risk of certain psychotechnological response options in multiple-choice tests: Does a particular personality handicap examinees?. European Journal of Psychological Assessment, 26(4), 302-308.
Kurz, T. B. (1999). A review of scoring algorithms for multiple-choice tests. San Antonio: Southwest Educational Research Association.
Lord, F. M. (1975). Formula scoring and number‐right scoring. Journal of Educational Measurement, 12(1), 7-11.
Morrengiello, B. A., and Rennie, H. (1998). Why do boys engage in more risk taking than girls? The role of attributions, beliefs, and risk appraisals. Journal of Pediatric Psychology, 23(1), 33-43.
Muijtjens, A. M. M, Mameren, H., Hoogenboom, R. J. I, Evers, J.L.H., and Vleuten, C. P. M. (1999), The effect of a ‘don't know’ option on test scores: number-right and formula scoring compared. Medical Education, 33, 267-275.
Popham, W. J. (2000). Modern educational measurement (3th Edition ). Boston: Ally and Bacon.
Prieto, G., and Delgado, A. R. (1999a). The effect of instructions on multiple-choice test scores. European Journal of Psychological Assessment, 15(2), 143–150.
Prihoda, T. J., Pinckard, R. N., McMahan, C. A., and Jones, A. C. (2006). Correcting for guessing increases validity in multiple-choice examinations in an oral and maxillofacial pathology course. Journal of Dental Education, 70(4), 378-386.
Reid, J. F. (1976). Scoring multiple- choice exams. The Journal of Economic Education, 8, 55-59.
Rowley, G. L., and Traub, R. E. (1977). Formula scoring, number right scoring, and test-taking strategy. Journal of Educational Measurement, 14(1), 15-22.
Rubio, V. J., Hernández, J. M., Zaldívar, F., Márquez, O., and Santacreu, J. (2010). Can we predict risk-taking behavior?: Two behavioral tests for predicting guessing tendencies in a multiple-choice test. European Journal of Psychological Assessment, 26(2), 87-94.
Sabers, D., and Feldt, L. (1968). An empirical study of the effect of the correction for chance success on the reliability and validity of an aptitude test. Journal of Educational Measurement, 5(3), 251-258.
Schermelleh-Engel, K., Moosbrugger, H., and Müller, H. (2003). Evaluating the fit of structural equation models: Tests of significance and descriptive goodness-of-fit measures. Methods of Psychological Research Online, 8(2), 23-74.
Scumacker, R. E., and Lomax, R. G. (2010). A beginner’s guide to structural equation modeling.(3rd Editon). New York: Taylor And Francis Group.
Socan, G. (2009).Scoring of multiple choice items by means of internal weighting. Review of Psychology, 16(2), 77-85.
Swineford, F., and Miller, P. M. (1953). Effects of directions regarding guessing on item statistics of a multiple-choice vocabulary test. Journal of Educational Psychology, 44(3), 129-139.
Thorndike, R. M. (2005). Measurement and evaluation in psycholgy and education (7th Edition). New Jersey: Pearson Education.
Ulutaş, S. (2003). Genel liselerdeki öğretmenlerin ölçme ve değerlendirme alanındaki yeterlikleri ile ölçme ve değerlendirme ilkelerini uygulama düzeylerinin araştırılması (Yayımlanmamış yüksek lisans tezi). Ankara Üniversitesi Eğitim Bilimleri Enstitüsü, Ankara.
Umay, A. (1998). Seçmeli derslerde yanıtlayıcı davranışları ve şans başarısının elimine edilmesi işlemlerine ilişkin bazı öneriler. Hacettepe Üniversitesi Eğitim Fakültesi Dergisi, 14, 54-61.
Zimmerman, D. W., and Williams, R. H. (1965). Effect of chance success due to guessing on error of measurement in multiple-choice tests. Psychological Reports, 16(3), 1193-1196.
Downloads
Additional Files
Published
How to Cite
Issue
Section
License
Authors can retain copyright, while granting the journal right of first publication. Alternatively, authors can transfer copyright to the journal, which then permits authors non-commercial use of the work, including the right to place it in an open access archive. In addition, Creative Commons can be consulted for flexible copyright licenses.
©1999 Creative Commons Attribution-ShareAlike 4.0 International License.
