STUDENT PERFORMANCE IN CURRICULA CENTERED ON SIMULATION-BASED INFERENCE
DOI:
https://doi.org/10.52041/serj.v21i3.6Keywords:
Statistics education research, Randomization tests, Multi-level modelsAbstract
Using simulation-based inference (SBI), such as randomization tests, as the primary vehicle for introducing students to the logic and scope of statistical inference has been advocated with the potential of improving student understanding of statistical inference and the statistical investigative process as a whole. Moving beyond the individual class activity, entirely revised introductory statistics curricula centering on these ideas have been developed and tested. Preliminary assessment data have been largely positive. In this paper, we discuss three years of cross-institutional tertiary-level data from the United States comparing SBI-focused curricula and non-SBI curricula (86 distinct institutions). We examined several pre/post measures of conceptual understanding in the introductory algebra-based course using multi-level modelling to incorporate student-level, instructor-level, and institutional-level covariates. We found that pre-course student characteristics (e.g., prior knowledge) were the strongest predictors of student learning, but also that textbook choice can still have a meaningful impact on student understanding of key statistical concepts. In particular, textbook choice was the strongest “modifiable” predictor of student outcomes of those examined, with simulation-based inference texts yielding the largest changes in student learning outcomes. Further research is needed to elucidate the particular aspects of SBI curricula that contribute to observed student learning gains.
References
Aliaga, M., Cobb, G., Cuff, C., Garfield, J., Gould, R., Lock, R., Moore, T., Rossman, A., Stephenson, B., Utts, J., Velleman, P., & Witmer, J. (2005). Guidelines for Assessment and Instruction in Statistics Education College Report. American Statistical Association. https://www.amstat.org/docs/default-source/amstat-documents/2005gaisecollege_full.pdf
Beckman, M., delMas, R., & Garfield, J. (2017). Cognitive transfer outcomes for a simulation-based introductory statistics curriculum. Statistics Education Research Journal, 16(2), 419–440. https://doi.org/10.52041/serj.v16i2
Case, C., & Jacobbe, T. (2018). A framework to characterize student difficulties in learning information from a simulation-based approach. Statistics Education Research Journal, 17(2), 9–29. https://doi.org/10.52041/serj.v17i2
Chance, B., & McGaughey, K. (2014). Impact of a simulation/randomization-based curriculum on student understanding of p-values and confidence intervals. In K. Makar, B. de Sousa, & R. Gould (Eds.), Sustainability in Statistics Education. Proceedings of the Ninth International Conference on Teaching Statistics (ICOTS-9), Flagstaff, Arizona. International Statistical Institute. http://icots.info/icots/9/proceedings/pdfs/ICOTS9_6B1_CHANCE.pdf
Chance, B., & Rossman, A. (2006). Using simulation to teach and learn statistics. In A. Rossman & B. Chance (Eds.), Working Cooperatively in Statistics Education. Proceedings of the Seventh International Conference on Teaching Statistics (ICOTS-7), Salvador, Brazil. International Statistical Institute. www.ime.usp.br/~abe/ICOTS7/Proceedings/PDFs/InvitedPapers/7E1_CHAN.pdf
Chance, B., Wong, J., & Tintle, N. (2017). Student performance in curricula centered on simulation-based inference: A preliminary report. Journal of Statistics Education, 24(3), 114–126. https://doi.org/10.1080/10691898.2016.1223529
Cobb, G. W. (1992). Teaching statistics. In L. A. Steen (Ed.), Heeding the call for change: Suggestions for curriculum action (pp. 3–43). Mathematical Association of America.
Cobb, G. (2007). The introductory statistics course: A Ptolemaic curriculum? Technology Innovations in Statistics Education, 1(1). https://doi.org/10.5070/T511000028
Colt, G. C., Davoudi, M., Murgu, S., & Zamanian Rohani, N. (2011). Measuring learning gain during a one-day introductory bronchoscopy course. Surgical Endoscopy, 25, 207–216.
delMas, R., Garfield, J., Ooms, A., & Chance, B. (2007). Assessing students’ conceptual understanding after a first course in statistics. Statistics Education Research Journal, 6(2), 28–58. https://doi.org/10.52041/serj.v6i2.483
delMas, R., Garfield, J., Ooms, A., & Chance, B. (2007). Assessing students’ conceptual understanding after a first course in statistics. Statistics Education Research Journal, 6(2), 28–58. https://doi.org/10.52041/serj.v6i2.483
Diez, D., Barr, C., & Cetinkeya-Rundel, M. (2014). Introductory statistics with randomization and simulation. Openintro.org. https://openintro.org/book/isrs/
Doerr, H., & English, L. (2003). A modeling perspective on students’ mathematical reasoning about data. Journal for Research in Mathematics Education, 34(2), 110–136. https://doi.org/10.2307/30034902
Fry, E. B. (2014). Introductory statistics instructors’ practices and beliefs regarding technology and pedagogy. In K. Makar, B. de Sousa, & R. Gould (Eds.), Sustainability in Statistics Education. Proceedings of the Ninth International Conference on Teaching Statistics (ICOTS-9), Flagstaff, Arizona. International Statistical Institute. https://iase-icots/9/proceedings/pdfs/ICOTS9_C202_FRY.pdf?1405041868
Garfield, J. (1995). How students learn statistics. International Statistical Review, 63(1), 25–34.
Garfield, J. & Ben-Zvi, D. (2007). How students learn statistics revisited: A current review of research on teaching and learning statistics. International Statistical Review, 75(3), 372–396.
Garfield, J., delMas, R., & Zieffler, A. (2012). Developing statistical modelers and thinkers in an introductory, tertiary-level statistics course. ZDM Mathematics Education, 44, 883–898. https://doi.org/10.1007/s11858-012-0447-5
Garfield, J., Hogg, B., Schau, C., & Whittinghill, D. (2002). First courses in statistical science: The status of educational reform efforts. Journal of Statistics Education, 10(2). http://dx.doi.org/10.1080/10691898.2002.11910665
Hake, R. R. (1998). Interactive engagement versus traditional methods: A six-thousand student survey of mechanics test data for introductory physics courses, American Journal of Physics, 66, 64–74.
Hildreth, L., Robison-Cox, J., & Schmidt, J. (2018). Comparing student success and understanding in introductory statistics under consensus and simulation-based curricula. Statistics Education Research Journal, 17(1), 103–120. https://doi.org/10.52041/serj.v17i1.178
Konold, C., Harradine, A., & Kazak, S. (2007). Understanding distributions by modeling them. International Journal of Computers for Mathematical Learning, 12, 217–230. https://link.springer.com/article/10.1007/s10758-007-9123-1
Lane-Getaz, S. J. (2007). Toward the development and validation of the reasoning about p-values and statistical significance scale. In B. Philips & L. Weldon (Eds.), Proceedings of the ISI/IASE Satellite Conference on Assessing Student Learning in Statistics. International Statistical Institute. http://www.stat.auckland.ac.nz/~iase/publications/sat07/Lane-Getaz.pdf
Lane-Getaz, S. J. (2017). Is the p-value really dead? Assessing inference learning outcomes for social science students in an introductory statistics course. Statistics Education Research Journal, 16(1), 357–399. https://doi.org/10.52041/serj.v16i1.235
Lee, H. S., Doerr, H. M., Tran, D., & Lovett, J. N. (2015). The role of probability in developing learners’ models of simulation approaches to inference. Statistics Education Research Journal, 15(2), 216–238. https://doi.org/10.52041/serj.v15i2.249
Lock, R., Frazer Lock, P., Lock Morgan, K., Lock, E., & Lock, D. (2013). Statistics: Unlocking the power of data. Wiley.
Lock, R., Frazer Lock, P. F., Lock Morgan, K., Lock, E., & Lock, D. (2014). Intuitive introduction to the important ideas of inference. In K. Makar, B. de Sousa, & R. Gould (Eds.), Sustainability in Statistics Education. Proceedings of the Ninth International Conference on Teaching Statistics (ICOTS-9), Flagstaff, Arizona. International Statistical Institute. http://icots.info/icots/9/proceedings/pdfs/ICOTS9_4A3_LOCK.pdf
Malone, C., & Hooks, T. (2012, July 31). Finding an appropriate balance between simulation-based and traditional methods in the teaching of statistical inference [Paper presentation]. Statistics: Growing to Serve a Data-dependent Society, Joint Statistical Meetings, San Diego.
Maurer, K., & Lock, E. (2015). Bootstrapping in the introductory statistics curriculum. Technology Innovations in Statistics Education, 9(1). https://doi.org/10.5070/T591026161
Moore, D. M. (1997). New pedagogy and new content: The case of statistics. International Statistical Review, 65(2), 123–165.
Parker, N., Fry, E., Garfield, J., & Zieffler, A. (2014). Graduate teaching assistants’ beliefs, practices, and preparation for teaching introductory statistics. In K. Makar, B. de Sousa, & R. Gould (Eds.), Sustainability in Statistics Education. Proceedings of the Ninth International Conference on Teaching Statistics (ICOTS-9), Flagstaff, Arizona. International Statistical Institute. https://iase-web.org/icots/9/proceedings/pdfs/ICOTS9_C200_PARKER.pdf?1405041867
Pfannkuch, M., & Budgett, S. (2014). Constructing inferential concepts through bootstrap and randomization-test simulation: A case study. In K. Makar, B. de Sousa, & R. Gould (Eds.), Sustainability in Education. Proceedings of the Ninth International Conference on Teaching Statistics (ICOTS-9), Flagstaff, Arizona. International Statistical Institute. http://iase-web.org/icots/9/proceedings/pdfs/ICOTS9_8J1_PFANNKUCH.pdf
Posner, M. (2014). A fallacy in student attitudes research: The impact of the first class. In K. Makar, B. de Sousa, & R. Gould (Eds.), Sustainability in Education. Proceedings of the Ninth International Conference on Teaching Statistics (ICOTS-9), Flagstaff, Arizona. International Statistical Institute. http://icots.info/9/proceedings/pdfs/ICOTS9_1F3_POSNER.pdf
Reaburn, R. (2014). Introductory statistics course tertiary students’ understanding of p-values. Statistics Education Research Journal, 13(1), 53–65. https://doi.org/10.52041/serj.v13i1.298
Roberts, R., Scheaffer, R., & Watkins, A. (1999). Advanced Placement Statistics: Past, present, and future. The American Statistician, 53(4), 307–320.
Roy, S., & Mcdonnel, T. (2018). Assessing simulation-based inference in secondary schools. Unpublished manuscript. http://www.isi-stats.com/isi/presentations/ICOTS2018-5.pdf
Roy, S., Rossman, A., Chance, B., Cobb, G., VanderStoep, J., Tintle, T., & Swanson, T. (2014). Using simulation/randomization to introduce p-value in Week 1. In K. Makar, B. de Sousa, & R. Gould (Eds.), Sustainability in Education. Proceedings of the Ninth International Conference on Teaching Statistics (ICOTS-9), Flagstaff, Arizona. International Statistical Institute. https://icots.info/9/proceedings/pdfs/ICOTS9_4A2_ROY.pdf
Saldanha, L. A., & Thompson, P. W. (2002). Conceptions of sample and their relationship to statistical inference. Educational Studies in Mathematics, 51, 257–270.
Schau, C. (2003). Survey of Attitudes Toward Statistics (SATS-36). http://evaluationandstatistics.com/
Tabor, J., & Franklin, C. (2013). Statistical reasoning in sports. W.H. Freeman and Company.
Tintle, N. L., Chance, B., Cobb, G., Rossman, A., Roy, S., Swanson, T., & VanderStoep, J. (2015). Introduction to statistical investigations. Wiley.
Tintle, N., Clark, J., Fischer, K., Chance, B., Cobb, G., Roy, S., Swanson, T., & VanderStoep, J. (2018). Assessing the association between pre-course metrics of student preparation and student performance in introductory statistics: Results from early data on simulation-based inference vs. nonsimulation based inference. Journal of Statistics Education, 26(2), 103–109. https://www.tandfonline.com/doi/full/10.1080/10691898.2018.1473061
Tintle, N., Topliff, K., VanderStoep, J., Homes, V.-L., & Swanson, T. (2012). Retention of statistical concepts in a preliminary randomization-based introductory statistics curriculum, Statistics Education Research Journal, 11(1), 21–40. https://doi.org/10.52041/serj.v11i1.340
Tintle, N., VanderStoep, J., Holmes, V-L., Quisenberry, B., & Swanson, T. (2011). Development and assessment of a preliminary randomization-based introductory statistics curriculum, Journal of Statistics Education, 19(1). https://doi.org/10.1080/10691898.2011.11889599
Tobias-Lara, M. G., & Gomez-Blancarte, A. L. (2019). Assessment of informal and formal inferential reasoning: A critical research review. Statistics Education Research Journal, 18(1), 8–25. https://doi.org/10.52041/serj.v18i1.147
van Es, C., & Weaver, M. (2018). Race, sex, and their influences on introductory statistics education. Journal of Statistics Education, 26(1), 48–54. https://www.tandfonline.com/doi/full/10.1080/10691898.2018.1434426
Wild, C. J., Pfannkuch, M., Regan, M., & Horton, N. J. (2011). Towards more accessible conceptions of statistical inference. Journal of the Royal Statistical Society: Series A (Statistics in Society), 174(2), 247–295. https://doi.org/10.1111/j.1467-985X.2010.00678.x
Zieffler, A., & Catalysts for Change. (2015). Statistical thinking: A simulation approach to modeling uncertainty (3rd ed.). Catalyst Press.
Zieffler, A., Garfield, J., Alt, S., Dupuis, D., Holleque, K., & Chang, B. (2008). What does research suggest about the teaching and learning of introductory statistics at the college level? A review of the literature. Journal of Statistics Education, 16(2). http://dx.doi.org/10.1080/10691898.2008.11889566
Zieffler, A., Park, J., Garfield, J. delMas, R., & Bjornsdottier, A. (2012). The Statistics Teaching Inventory: A survey on statistics teachers’ classroom practices and beliefs. Journal of Statistics Education, 20(1). https://doi.org/10.1080/10691898.2012.11889632
Ziegler, L., & Garfield, J. (2018). Developing a statistical literacy assessment for the modern introductory statistics course. Statistics Education Research Journal, 17(2), 161–178. https://doi.org/10.52041/serj.v17i2.164
