CHALLENGES ASSOCIATED WITH MEASURING ATTITUDES USING THE SATS FAMILY OF INSTRUMENTS
DOI:
https://doi.org/10.52041/serj.v21i1.88Keywords:
Statistics education research, Attitudes towards statistics, Assessing attitudes, Validity evidenceAbstract
The Survey of Attitudes Toward Statistics (SATS) is a widely used family of instruments for measuring attitude constructs in statistics education. Since the development of the SATS instruments, there has been an evolution in the understanding of validity in the field of educational measurement emphasizing validation as an on-going process. While a 2012 review of statistics education attitude instruments noted that the SATS family had the most validity evidence, two types of challenges to the use of these instruments have emerged: challenges to the interpretations of scale scores and challenges using the SATS instruments in populations other than undergraduate students enrolled in introductory statistics courses. A synthesis of the literature and empirical results are used to document these challenges.
References
American Educational Research Association, American Psychological Association, & National Council on Measurement in Education. (2014). Standards for educational and psychological testing. American Educational Research Association. https://www.testingstandards.net/open-access-files.html
Bandalos, D. L. (2018). Measurement theory and applications for the social sciences. Guilford Press.
Bandura, A. (1977). Social learning theory. Prentice Hall.
Bandura, A. (1986). Social foundations of thought and action: A social cognitive theory. Prentice-Hall.
Batakci, L., Bolon, W., & Bond, M. E. (2018). A framework and survey for measuring instructors’ motivational attitudes toward statistics. In M. A. Sorto, A. White, & L. Guyot (Eds.), Looking back, looking forward. Proceedings of the Tenth International Conference on Teaching Statistics (ICOTS10, July, 2018), Kyoto, Japan. http://iase-web.org/icots/10/proceedings/pdfs/ICOTS10_4J3.pdf
Batanero, C., Estrada, A., Díaz, C., & Fortuny, J. M. (2005). A structural study of future teachers’ attitudes towards statistics. In M. Bosch (Ed.), Proceedings of the Fourth Congress of the European Society for Research in Mathematics Education (pp. 508–517). http://www.mathematik.uni-dortmund.de/~erme/CERME4/CERME4_WG5.pdf
Bateiha, S., Marchionda, H., & Autin, M. (2020). Teaching style and attitudes: A comparison of two collegiate introductory statistics classes. Journal of Statistics Education, 28(2), 154–164. https://doi.org/10.1080/10691898.2020.1765710
Bechrakis, T., Gialamas, V., & Barkatsas, A. N. (2011). Survey of Attitudes Toward Statistics (SATS): An investigation of its construct validity and its factor structure invariance by gender. International Journal of Theoretical Educational Practice, 1(1), 1–15.
Blair, R. M., Kirkman, E. E., & Maxwell, J. W. (2018). Statistical abstract undergraduate programs in the mathematical sciences in the united states: 2015 CBMS survey. American Mathematical Society. http://www.ams.org/cbms-survey/cbms2015-Report.pdf
Bond, M. E. (2008). Survey of Attitudes Toward Statistics: An exploratory look. 2008 JSM Proceedings, (pp. 3758-3760). http://www.statlit.org/pdf/2008BondASA.pdf
Bond, M. E., Perkins, S. N., & Ramirez, C. (2012). Students’ perceptions of statistics: An exploration of attitudes, conceptualizations, and content knowledge of statistics. Statistics Education Research Journal, 11(2), 6–25. https://iase-web.org/documents/SERJ/SERJ11(2)_Bond.pdf?1402525003
Carillo, K., Galy, N., Guthrie, C., & Vanhems, A. (2016).<< J' aime pas les stats!>> Mesure et analyse de l’attitude à l’égard du cours de statistique dans un école de management. Mesure et analyse de l’attitude à l’égard du cours de statistique dans un école de management [“I hate statistics!” — The measure and analysis of attitudes towards statistics in a management school]. Statistique et Enseignement, 7(1), 3–31. http://statistique-et-enseignement.fr/article/view/537
Carlson, K. A., & Winquist, J. R. (2011). Evaluating an active learning approach to teaching introductory statistics: A classroom workbook approach. Journal of Statistics Education, 19(1), 1–23. https://doi.org/10.1080/10691898.2011.11889596
Carmona Márquez, J. (2004). Una revisión de las evidencias de fiabilidad y validez de los cuestionarios de actitudes y ansiedad hacia la estadística. Statistics Education Research Journal, 3(1), 5–28. http://iase-web.org/documents/SERJ/SERJ3(1)_marquez.pdf
Carnell, L. J. (2008). The effect of a student-designed data collection project on attitudes toward statistics. Journal of Statistics Education, 16(1). http://jse.amstat.org/v16n1/carnell.html
Cashin, S. E., & Elmore, P. B. (2005). The survey of attitudes toward statistics acale: A construct validity study. Educational and Psychological Measurement, 65(3), 509–524. https://doi.org/10.1177/0013164404272488
Chance, B., Wong, J., & Tintle, N. (2016). Student performance in curricula centered on simulation-based inference: A preliminary report. Journal of Statistics Education, 24(3), 114–126. https://doi.org/10.1080/10691898.2016.1223529
Chiesi, F., & Primi, C. (2009). Assessing statistics attitudes among college students: Psychometric properties of the Italian version of the Survey of Attitudes Toward Statistics (SATS). Learning and Individual Differences, 19(2), 309–313. https://doi.org/10.1016/j.lindif.2008.10.008
Chiesi, F., & Primi, C. (2010). Cognitive and non-cognitive factors related to students’ statistics achievement. Statistics Education Research Journal, 9(1), 6–26. https://iase-web.org/documents/SERJ/SERJ9(1)_Chiesi_Primi.pdf
Chiesi, F., & Primi, C. (2018). What happens when attitudes toward statisitcs change (increases vs decrease) during the course? In M. A. Sorto, A. White, & L. Guyot (Eds.), Looking back, looking forward. Proceedings of the Tenth International Conference on Teaching Statistics (ICOTS10, July, 2018), Kyoto, Japan. https://iase-web.org/icots/10/proceedings/pdfs/ICOTS10_7D1.pdf
Cladera, M., Rejón-Guardia, F., Vich-i-Martorell, G. À., & Juaneda, C. (2019). Tourism students’ attitudes toward statistics. Journal of Hospitality, Leisure, Sport & Tourism Education, 24, 202–210. https://doi.org/10.1016/j.jhlste.2019.03.002
Coetzee, S., & Van der Merwe, P. (2010). Industrial psychology students’ attitudes towards statistics. SA Journal of Industrial Psychology, 36(1), 8 pages. https://doi.org/10.4102/sajip.v36i1.843
Crocker, L. M., & Algina, J. (1986). Introduction to classical and modern test theory. Holt, Rinehart, and Winston.
Cronbach, L. J. (1951). Coefficient alpha and the internal structure of tests. Psychometrika, 16(3), 297–334. https://link.springer.com/article/10.1007/BF02310555
Cronbach, L. J., & Meehl, P. E. (1955). Construct validity in psychological tests. Psychological Bulletin, 52, 281–302. https://doi.org/10.1037/h0040957
Dauphinee, T. L., Schau, C., & Stevens, J. J. (1997). Survey of attitudes toward statistics: Factor structure and factorial invariance for women and men. Structural Equation Modeling: A Multidisciplinary Journal, 4(2), 129–141. https://doi.org/10.1080/10705519709540066
Dillman, D. A., Smyth, J. D., & Christian, L. M. (2014). Internet, phone, mail, and mixed-mode surveys: The tailored design method (4th edition). Wiley.
Eccles, J. (1983). Expectancies, values, and academic behaviors. In J. T. Spence (Ed.), Achievement and achievement motives: Psychological and sociological approaches (pp. 75–145). W.H. Freeman.
Eccles, J. S. (2014). Expectancy-value theory. In R. Eklund & G. Tenenbaum (Eds.), Encyclopedia of Sport and Exercise Psychology. https://doi.org/10.4135/9781483332222.n110
Eccles, J. S., & Wigfield, A. (2002). Motivational beliefs, values, and goals. Annual Review of Psychology, 53, 109–132. https://www.annualreviews.org/doi/abs/10.1146/annurev.psych.53.100901.135153
Emmioglu Sarikaya, E., Ok, A., Capa Aydin, Y., & Schau, C. (2018). Turkish version of the Survey of Attitudes Toward Statistics: Factorial structure invariance by gender. International Journal of Higher Education, 7(2), 121–127. https://doi.org/10.5430/ijhe.v7n2p121
Estrada, A. (2002). Análisis de las actitudes y conocimientos estadísticos elementales en la formación del profesorado. [Analysis of attitudes and elementary statistical knowledge in training teachers] (Universidad Autónoma de Barcelona). https://www.tdx.cat/handle/10803/4697
Franklin, C., Kader, G., Mewborn, D., Moreno, J., Peck, R., Perry, M., & Scheaffer, R. (2007). Guidelines for Assessment and Instruction in Statistics Education (GAISE) Report: A Pre-K-12 curriculum framework. www.amstat.org/education/gaise
GAISE College Report ASA Revision Committee. (2016). Guidelines for Assessment and Instruction in Statistics Education College Report 2016. http://www.amstat.org/education/gaise
Gal, I., Ginsburg, L., & Schau, C. (1997). Monitoring attitudes and beliefs in statistics education. In I. Gal & J. B. Garfield (Eds.), The assessment challenge in statistics education (pp. 37–51). IOS Press. http://iase-web.org/Books.php?p=book1
García-Martínez, J. A., Fallas-Vargas, M. A., & Romero-Hernández, A. (2015). Las actitudes hacia la estadística del estudiantado de orientación. Revista Electrónica Educare, 19(1), 25–41. https://doi.org/10.15359/ree.19-1.2
Gundlach, E., Richards, K. A. R., Nelson, D., & Levesque-Bristol, C. (2015). A comparison of student attitudes, statistical reasoning, performance, and perceptions for web-augmented traditional, fully online, and flipped sections of a statistical literacy class. Journal of Statistics Education, 23(1). http://jse.amstat.org/v23n1/gundlach.pdf
Hannigan, A., Gill, O., & Leavy, A. M. (2013). An investigation of prospective secondary mathematics teachers’ conceptual knowledge of and attitudes towards statistics. Journal of Mathematics Teacher Education, 16(6), 427–449. https://doi.org/10.1007/s10857-013-9246-3
Harrell-Williams, L. M., Sorto, M. A., Pierce, R. L., Lesser, L. M., & Murphy, T. J. (2014). Validation of scores from a new measure of preservice teachers’ self-efficacy to teach statistics in the middle grades. Journal of Psychoeducational Assessment, 32(1), 40–50. https://doi.org/10.1177/0734282913486256
Harrell-Williams, L. M., Sorto, M. A., Pierce, R., Lesser, L. M., & Murphy, T. J. (2015). Identifying statistical concepts associated with high and low levels of self-efficacy to teach statistics in middle grades. Journal of Statistics Education, 23(1). http://jse.amstat.org/v23n1/harrell-williams.pdf
Harshe, D., & Abraham, D. (2017). A study of attitudes of teaching faculty and postgraduate residents at a tertiary care teaching hospital toward biostatistics. Muller Journal of Medical Sciences and Research, 8(1), 10–14. https://doi.org/10.4103/0975-9727.199362
Hilton, S. C., Schau, C., & Olsen, J. A. (2004). Survey of Attitudes Toward Statistics: Factor structure invariance by gender and by administration time. Structural Equation Modeling: A Multidisciplinary Journal, 11(1), 92–109. https://doi.org/10.1207/S15328007SEM1101_7
Hommik, C., & Luik, P. (2017). Adapting the Survey of Attitudes Towards Statistics (SATS-36) for Estonian secondary school students. Statistics Education Research Journal, 16(1), 228–239. https://doi.org/10.52041/serj.v16i1.229
Hood, M., Creed, P. A., & Neumann, D. L. (2012). Using the expectancy value model of motivation to understand the relationship between student attitudes and achievement in statistics. Statistics Education Research Journal, 11(2), 72–85. https://iase-web.org/documents/SERJ/SERJ11(2)_Hood.pdf
Kerby, A. T., & Wroughton, J. R. (2017). When do students’ attitudes change? Investigating student attitudes at midterm. Statistics Education Research Journal, 16(2), 476–486. https://doi.org/10.52041/serj.v16i2.202
Khavenson, T., Orel, E., & Tryakshina, M. (2012). Adaptation of Survey of Attitudes Towards Statistics (SATS 36) for Russian Sample. Procedia - Social and Behavioral Sciences, 46, 2126–2129. https://doi.org/10.1016/j.sbspro.2012.05.440
Krupa, E. E., Carney, M., & Bostic, J. (2019). Argument-based validation in practice: Examples from mathematics education. Applied Measurement in Education, 32(1), 1–9. https://doi.org/10.1080/08957347.2018.1544139
Lipka, O., & Hess, I. (2016). Attitudes toward statistics studies among students with learning disabilities. Numeracy, 9(2) Article 7. https://doi.org/10.5038/1936-4660.9.2.7
Luh, W. M., Guo, J. H., & Wisenbaker, J. M. (2004). Difference of attitudes toward statistics between cadets and college students. Paper Presented at the 10th International Congress on Mathematical Education, July 4–11, Denmark. http://iase-web.org/documents/papers/icme10/Luh.pdf
Messick, S. (1989). Validity. In R. L. Linn (Ed.), Educational Measurement (3rd ed., pp. 13–103). American Council on Education.
Messick, S. (1995). Validity of psychological assessment: Validation of inferences from persons’ responses and performances as scientific inquiry into score meaning. American Psychologist, 50(9), 741–749. https://doi.org/10.1037/0003-066X.50.9.741
Millar, A. M., & Schau, C. (2010). Assessing students’ attitudes: The good, the bad, and the ugly. 2010 JSM Proceedings (pp. 1133–1143). http://www.statlit.org/pdf/2010MillarSchauASA.pdf
Millar, A. M., & White, B. J. G. (2014). How do attitudes change from one stats course to the next? In K. Makar & R. Gould (Eds.), Sustainability in Statistics Education. Proceedings of the Ninth International Conference on Teaching Statistics (ICOTS9), Flagstaff, Arizona, July 13–18. https://iase-web.org/icots/9/proceedings/pdfs/ICOTS9_1F2_MILLAR.pdf
Millar, A. M., White, B. J., & Romo, R. (2013). Comparing apples with apples: Assessing student attitudes in the presence of regression to the mean. 2013 JSM Proceedings. Presented at the Joint Statistics Meetings, San Francisco, CA. http://ec.msvu.ca:8080/xmlui/handle/10587/1284
Moore, C. M. (1987). Group techniques for idea building. SAGE Publications.
Nasser, F. M. (2004). Structural model of the effects of cognitive and affective factors on the achievement of Arabic-speaking pre-service teachers in introductory statistics. Journal of Statistics Education, 12(1). http://ww2.amstat.org/publications/jse/v12n1/nasser.html
Nolan, M. M., Beran, T., & Hecker, K. G. (2012). Surveys assessing students’ attitudes toward statistics: A systematic review of validity and reliability. Statistics Education Research Journal, 11(2), 103–123. https://doi.org/10.52041/serj.v11i2.333
Patton, M. Q. (2002). Qualitative research and evaluation methods (3 ed). SAGE Publications.
Posner, M. A. (2011). The impact of a proficiency-based assessment and reassessment of learning outcomes system on student achievement and attitudes. Statistics Education Research Journal, 10(1), 3–14. https://doi.org/10.52041/serj.v10i1.352
Posner, M. A. (2014). A fallacy in student attitude research: The impact of the first class. In K. Makar & R. Gould (Eds.), Sustainability in Statistics Education. Proceedings of the Ninth International Conference on Teaching Statistics (ICOTS9), Flagstaff, Arizona, July 13–18. https://iase-web.org/icots/9/proceedings/pdfs/ICOTS9_1F3_POSNER.pdf
Ramirez, C., & Bond, M. (2014). Comparing attitudes toward statistics among students enrolled in project-based and hybrid statistics courses. In K. Makar & R. Gould (Eds.), Sustainability in Statistics Education. Proceedings of the Ninth International Conference on Teaching Statistics (ICOTS9), Flagstaff, Arizona, July 13–18. https://iase-web.org/icots/9/proceedings/pdfs/ICOTS9_1F4_RAMIREZ.pdf?1405041565
Ratanaolarn, T. (2016). The development of a structural equation model of graduate students’ statistics achievement. The Journal of Behavioral Science, 11(2), 153–168. https://so06.tci-thaijo.org/index.php/IJBS/article/view/63288
Rosli, M. K., & Maat, S. M. (2017). Attitude towards statistics and performance among post-graduate students. AIP Conference Proceedings, 1847(1) 030004. https://doi.org/10.1063/1.4983881
Schau, C. (1992). Survey of Attitudes Toward Statistics (SATS-28). http://evaluationandstatistics.com/
Schau, C. (2003a). Students’ attitudes: The “other” important outcome in statistics education. In proceedings of the 2003 Joint Statisitcal Meetings, 3 (pp. 3673–3683). http://www.statlit.org/pdf/2003SchauASA.pdf
Schau, C. (2003b). Survey of Attitudes Toward Statistics (SATS-36). http://evaluationandstatistics.com/
Schau, C. (2005a). Scoring the SATS-28©. https://irp-cdn.multiscreensite.com/281322c3/files/uploaded/SATS28Scoring.pdf
Schau, C. (2005b). Scoring the SATS-36©. https://irp-cdn.multiscreensite.com/281322c3/files/uploaded/Final36scoring.pdf
Schau, C. (2008). Common issues in SATS© research. 2008 JSM Proceedings (pp. 3761–3763). http://www.statlit.org/pdf/2008SchauASA.pdf
Please see the article for the complete list of references.
