• CHELSEY LEGACY University of Minnesota
  • ANDREW ZIEFFLER University of Minnesota
  • ELIZABETH BRONDOS FRY St. Catherine University
  • LAURA LE University of Minnesota



Statistics education research, computational thinking, assessment, instructional practices


The influx of data and the advances in computing have led to calls to update the introductory statistics curriculum to better meet the needs of the contemporary workforce. To this end, we developed the COMputational Practices in Undergraduate TEaching of Statistics  (COMPUTES) instrument, which can be used to measure the extent to which computation practices—specifically data, simulation, and coding practices—are included in the introductory statistics curriculum. Data from 236 instructors were used in a psychometric analysis to evaluate the latent structure underlying instructors’ response patterns and  understand the quality of the items. We also examined whether computational practices are being emphasized differently across institutional settings. Results suggest that the latent structure is best captured using a correlated multidimensional model and that most items were contributing information to the measurement process. Across institutional settings, curricular emphasis related to data and simulation practices seem quite similar, while emphasis on coding practices differs.


American Educational Research Association. (2014). Standards for educational and psychological testing. American Educational Research Association American Psychological Association National Council on Measurement in Education.

American Statistical Association. (2017). Curriculum guidelines for undergraduate programs in statistical science. American Statistical Association.

Baker, F. B. (2001). The basics of item response theory (2nd ed., ERIC Document Reproduction Service No. ED 458 219). Eric Clearing House on Assessment and Evaluation.

Barr, B. V., & Stephenson, C. (2011). Bringing computational thinking to K–12: What is involved and what is the role of the computer science education community? ACM Inroads, 2(1), 48–54.

Baumer, B. S., Cetinkaya-Rundel, M., Bray, A., Loi, L., & Horton, N. J. (2014). R Markdown: Integrating a reproducible analysis tool into introductory statistics. Technology Innovations in Statistics Education, 8(1).

Beckman, M. D., Çetinkaya-Rundel, M., Horton, N. J., Rundel, C. W., Sullivan, A. J, & Tackett, M. (2021). Implementing version control with Git and GitHub as a learning objective in statistics and data science courses. Journal of Statistics and Data Science Education, 29(sup1), S132-S144.

Benton, L., Hoyles, C., Kalas, I., & Noss, R. (2017). Bridging primary programming and mathematics: Some findings of design research in England. Digital Experiences in Mathematics Education, 3(2), 115–138.

Blair, R., Kirkman, E., & Maxwell, J. (2018). Statistical abstract of undergraduate programs in the mathematical sciences in the United States: Fall 2015 CBMS Survey. American Mathematical Society.

Bond, T. G., & Fox, C. M. (2001). Applying the Rasch model (2nd ed.). Lawrence Erlbaum.

Borsboom, D., Mellenbergh, G. J., & van Heerden, J. (2004). The concept of validity. Psychological Review, 111(4), 1061–1071.

Brennan, K. & Resnick, M. (2012). New frameworks for studying and assessing the development of computational thinking. In A. F. Ball & C. F. Tyson (Eds.), Non satis scire: To know is not enough. Proceedings of the 2012 Annual Meeting of the American Educational Research Association, April 13–17, Vancouver, Canada (Vol. 1, pp. 1–25). American Educational Research Association.

Broatch, J. E., Dietrich, S., & Goelman, D. (2019). Introducing data science techniques by connecting database concepts and dplyr. Journal of Statistics Education, 27(3), 147–153.

Business Higher Education Forum and PricewaterhouseCoopers. (2017), Investing in America’s data science and analytics talent: The case for action.

Carver, S. M., & Klahr, D. (1986). Assessing children's LOGO debugging skills with a formal model. Journal of Educational Computing Research, 2(4), 487–525.

Çetinkaya-Rundel, M., & Tackett, M. (2020). From drab to fab: Teaching visualization via incremental improvements. CHANCE, 33(2), 31–41.

Chalmers, R. P. (2016). Generating adaptive and non-adaptive test interfaces for multidimensional item response theory applications. Journal of Statistical Software, 71(5), 1–39.

Chalmers, R. P. (2012). mirt: A multidimensional item response theory package for the R environment. Journal of Statistical Software, 48(6), 1–29.

Clements, D. H. (1991). Enhancement of creativity in computer environments. American Educational Research Journal, 28(1), 173–187.

Cobb, G. (2015). Mere renovation is too little too late: We need to rethink our undergraduate curriculum from the ground up. The American Statistician, 69(4), 266–282.

College Board. (2020). AP Computer Science Principles: Course and exam description. Author.

de Ayala, R. J. (2009). The theory and practice of item response theory. The Guilford Press.

Dillman, D. A., Smyth, J. D., & Christian, L. M. (2008). Internet, mail, and mixed-mode surveys: The tailored design method. John Wiley.

Dogucu, M., & Çetinkaya-Rundel, M. (2021). Web scraping in the statistics and data science curriculum: Challenges and opportunities. Journal of Statistics Education, 29(sup1), S112–S122.

Engel, J. (2017). Statistical literacy for active citizenship: A call for data science education. Statistics Education Research Journal, 16(1), 44–49.

Engel, J., Biehler, R., Frischemeier, D [Daniel], Podworny, S., Schiller, A., & Martignon, L. (2019). Zivilstatistik: Konzept einer neuen Perspektive auf Data Literacy und Statistical Literacy. AStA Wirtschafts- und Sozialstatistisches Archiv, 13(3–4), 213–244.

Erickson, T., Wilkerson, M., Finzer, W., & Reichsman, F. (2019). Data moves. Technology Innovations in Statistics Education, 12(1).

Fiksel, J., Jager, L. R., Hardin, J. S., & Taub, M. A. (2019). Using gitHub classroom to teach statistics. Journal of Statistics Education, 27(2), 1–10.

Forbes, S. (2014). The Coming of Age of Statistics Education in New Zealand, and its Influence Internationally. Journal of Statistics Education, 22(2), 2.

Forbes, S., Chapman, J., Harraway, J., Stirling, D., & Wild, C. (2014). Use of data visualization in the teaching of statistics: A New Zealand perspecitve. Statistics Education Research Journal, 13(2), 187–201.

Futschek, G., & Moschitz, J. (2011). Learning algorithmic thinking with tangible objects eases transition to computer programming. In I. Kalaš & R.T. Mittermeir (Eds.), Situation, evolution, and perspectives. Proceedings of the 5th International Conference on Informatics in Schools, Bratislava, Slovakia, October 26–29, (pp. 155–164). Springer.

Gal, I., Ridgway, J., & Nicholson, J. (2016, December). Exploration of skills andconceptual knowledge needed for understanding statistics about society: A workshop. In J. Engel (Eds.), Promoting understanding of statistics about society. Proceedings of the Roundtable Conference of the International Association of Statistics Education (IASE), Berlin, Germany.

Gould, R. (2017). Data literacy is statistical literacy. Statistics Education Research Journal, 16(1), 22–25.

Hardin, J. (2018). Dynamic data in the statistics classroom. Technology Innovations in Statistics Education, 11(1).

Hardy, Q. (2012, January 30). BITS; The popularity of statistics. New York Times [online].

Holcomb, J. P., & Spalsbury, A. (2005). Teaching students to use summary statistics and graphics to clean and analyze data. Journal of Statistics Education, 13(3).

Holdren, J. P., & Lander, E. (2012). Engage to excel: Producing one million additional college graduates with degrees in science, technology, engineering, and mathematics. President’s Council of Advisors on Science and Technology.

Horton, N. J., Baumer, B. S., & Wickham, H. (2014). Teaching precursors to data science in introductory and second courses in statistics. arXiv:1401.3269 [stat.CO].

Horton, N. J., & Hardin, J. S. (2015). Teaching the next generation of statistics students to “think with data”. The American Statistician, 69(4), 259–265.

IDSSP Curriculum Team (2019). Curriculum Frameworks for Introductory Data Science.

Kahlenberg, R. D. (2015). How higher education funding shortchanges community colleges. The Century Foundation.

Kaplan, D. (2018). Teaching stats for data science. The American Statistician, 72(1), 89–96.

Marcelino, M. J., Pessoa, T., Vieira, C., Salvador, T., & Mendes, A. J. (2018). Learning computational thinking and scratch at distance. Computers in Human Behavior, 80, 470–477.

McNamara, A., & Horton, N. J. (2018). Wrangling categorical data in R. The American Statistician, 72(1), 97–104.

Melguizo, T., & Wolniak, G. C. (2012). The earnings benefits of majoring in STEM fields among high achieving minority students. Research in Higher Education, 53(4) 383–405.

National Academies of Sciences, Engineering, and Medicine. (2018). Data science for undergraduates: Opportunities and options. The National Academies Press.

Nolan, D., & Temple Lang, D. (2010). Computing in the statistics curricula. The American Statistician, 64(2), 97–107.

Rash, S. (Director). (1987). Can’t buy me love [Film]. Touchstone Pictures.

Rea, S (2017, October 16). Carnegie Mellon changes statistics department’s name to reflect its eminent position in data science research and education.,Department%20of%20Statistics%20%26%20Data%20Science

Reckase, M. D. (2009). Multidimensional item response theory. Springer.

Ridgway, J. (2016). Implications of the data revolution for statistics education: The data revolution and statistics education. International Statistical Review, 84(3), 528–549.

Ridgway, J., Ridgway, R., & Nicholson, J. (2018, March). Data science for all: A stroll in the foothills. In M. A. Sorto (Ed.), Looking back, looking forward. Proceedings of the 10th International Conference on the Teaching of Statistics (ICOTS10), Kyoto, Japan, July 8–14. International Statistical Institute.

Sáez-López, J. M., Román-González, M., & Vázquez-Cano, E. (2016). Visual programming languages integrated across the curriculum in elementary school: A two year case study using “Scratch” in five schools. Computers & Education, 97, 129–141.

Samejima, F. (1969). Estimation of latent ability using a response pattern of graded scores (Psychometric Monograph No. 17, Part 2). Psychometric Society.

Schiller, A., & Engel, J. (2016). Civic statistics and the preparation of future secondary school mathematics teachers. In J. Engel (Eds.), Promoting understanding of statistics about society: Proceedings of the Roundtable Conference of the International Association of Statistics Education, Berlin, Germany.

Thissen, D., Pommerich, M., Billeaud, K., & Williams, V. S. L. (1995). Item response theory for scores on tests including polytomous items with ordered responses. Applied Psychological Measurement, 19, 39-49.

U.S. Department of Commerce. (2017). Women in STEM: 2017 Update. ESA Issue Brief #06-17.

Weintrop, D., Beheshti, E., Horn, M., Orton, K., Jona, K., Trouille, L., & Wilensky, U. (2016). Defining computational thinking for mathematics and science classrooms. Journal of Science Education and Technology, 25(1), 127–147.

Zieffler, A., Park, J., Garfield, J., Delmas, R., & Bjornsdottir, A. (2012). The statistics teaching inventory: A survey on statistics teachers' classroom practices and beliefs. Journal of Statistics Education, 20(1).






Regular Articles