Invited Talks

Topic 8 : Research in statistics education

Statistics education requires great and on-going commitment, and is vulnerable to stagnation, backsliding, reductionism and a multiplicity of miscellaneous pressures both intentional and inadvertent. Complexities arise because what works in one context, does not necessarily work sustainably in another. The precise differences associated with that contrast are not easily identified, or even are possibly unknowable. In the context of the explosion of published work it is important to describe and examine current research problems, directions for future areas of inquiry, and the use of theoretical models on which our research might be based. The invited sessions cover teaching and learning of both statistics and probability, the role of technology, and potential guidelines for publishing in education research journals.

Session 8A: Research on developing students’ reasoning using simulation methods for introductory statistical inference: Session I

8A1: Students’ visual reasoning and the randomization test

Stephanie Budgett   University of Auckland, New Zealand
Chris Wild   University of Auckland, New Zealand

The traditional approach to teaching statistical inference limits students’ reasoning to mathematical manipulations. Cobb (2007) advocated the use of simulation methods for introductory statistical inference. In this paper, the dynamic visual simulations we use for introductory statistics for experiment-to-causation inference are briefly described. Using data from recent research, two students’ reasoning is analysed; one in response to being asked to use visual inference tools to analyse data from a randomized experiment, the other being asked to visualize the simulation process. Our findings suggest that the dynamic visualizations are becoming part of these students’ cognitive processes for understanding experiment-to-causation inference. Remaining issues that arose in students’ reasoning will be discussed.


8A2: Designing and implementing an alternative teaching concept within a continuous professional development course for German secondary school teachers

Janina Oesterhaus   University of Paderborn, Germany
Rolf Biehler   University of Paderborn, Germany

New standards in mathematics at German high-school level laid a plethora of stress on teaching statistical inference in senior classes, a domain that enjoys little popularity among teachers. Their personal perception not being adequately trained in teaching statistics during their professional careers is considered as one possible explanation for this deplorable state of affairs. In 2013 the German Center for Mathematics Teacher Education (DZLM) implemented a specially designed continuous professional development (CPD) course for teaching statistical inference at high-school level using a new and illustrative teaching approach. This approach was designed by the authors of the paper by adopting main characteristics from new Anglo-American curricula and accommodating them to German syllabi. The paper will focus on presenting the design process of the teaching approach and its evaluation within the CPD course.


8A3: Quantitative evidence for the use of simulation and randomization in the introductory statistics course

Nathan Tintle   Dordt College, United States
Todd Swanson   Hope College, United States
Jill VanderStoep   Hope College, United States
Soma Roy   California Polytechnic State University, United States
Allan Rossman   California Polytechnic State University, United States
George Cobb   Mount Holyoke College, United States
Ally Rogers   Dordt College, United States
Beth Chance   California Polytechnic State University, United States

The use of simulation and randomization in the introductory statistics course is gaining popularity, but what evidence is there that these approaches are improving students’ conceptual understanding and attitudes as we hope? In this talk I will discuss evidence from early full-length versions of such a curriculum, covering issues such as (a) items and scales showing improved conceptual performance compared to traditional curriculum, (b) transferability of findings to different institutions, (c) retention of conceptual understanding post-course and (d) student attitudes. Along the way I will discuss a few areas in which students in both simulation/randomization courses and the traditional course still perform poorly on standardized assessments.


Session 8B: Research on developing students’ reasoning using simulation methods for introductory statistical inference: Session 2

8B1: The symbiotic, mutualistic relationship between modeling and simulation in developing students’ statistical reasoning about inference and uncertainty

Andrew Zieffler   University of Minnesota, United States
Joan Garfield   University of Minnesota, United States
Robert C delMas   University of Minnesota, United States
Ethan Brown   University of Minnesota, United States

We report preliminary results from an ongoing study of the development of tertiary students’ reasoning related to statistical inference and uncertainty during a one-semester modeling and simulation-based statistics course. Comparisons of students’ performance on assessments of statistical reasoning will be presented for both students enrolled in this course and those enrolled in courses that used conventional parametric methods of inference. Summaries of qualitative data from nine students who participated in problem-solving interviews will also be presented to illustrate the development of students' reasoning related to statistical inference and uncertainty. Analyses of the data indicate that students taking the modeling and simulation-based course demonstrate better understanding of the principles of study design and statistical inference and begin developing these understandings within the first few weeks of the course.


8B2: Bootstrapping for learning statistics

Tim Hesterberg   Google, United States

Collecting data, producing plots such as histograms and scatter plots, and calculating numerical statistics such as means, medians, and regression coefficients are relatively concrete operations. In contrast, ideas related to the random variability of those statistics—sampling distributions, standard error, confidence intervals, central limit theorems, hypothesis tests, P-values, and statistical significance—are relatively abstract, and more difficult for students to understand. Bootstrap methods and permutation tests take those concrete tools, that students are used to using with data, and apply them to sampling distributions. This promotes understanding. We demonstrate using two examples—one involving linear regression, the other comparing two sample means. We finish by discussing why the bootstrap works, and what to watch out for.


8B3: From data to decision-making: using simulation and resampling methods to teach inferential concepts

Mia Stephens   JMP Division of SAS, United States
Don McCormack   Stonehill College & Brandeis University, United States
Robert H Carver   Stonehill College & Brandeis University, United States

It is the year 2014. Despite advances in technology and the availability of interactive software and teaching tools, it is still the norm to use normality-based methods, and even look-up tables, to teach statistical inference. This puts a heavy burden on our students, who must struggle through difficult theory before they’re taught how to make decisions and draw inferences from data. Even then, do they understand what a p-value is or what a confidence interval represents? Is there a better way? Interactive computer simulations and resampling methods can help bridge the gap between graphs and summary statistics and inference, providing a gentler and more natural transition. Until recently, these methods required add-ins, specialized programs or custom code. Today, these techniques are available in mainstream statistical software. In this talk, we illustrate how to use simulations, bootstrapping, and randomization tests in JMP® to introduce sampling distributions and explore core inferential concepts.


Session 8C: Research on developing students’ informal statistical inferential reasoning

8C1: Informal statistical inference revisited

Katie Makar   University of Queensland, Australia
Andee Rubin   TERC, United States

Several years ago, we characterized “informal statistical inference” as a claim that went beyond the data, using the data as evidence and acknowledging uncertainty (Makar & Rubin, 2009). This characterization was intentionally ambiguous in order to provide a context for ongoing research on statistical reasoning (especially with younger learners and those without formal statistical training) and to provoke discussion among researchers whose uses of the term differed from one another. Since then, research about informal statistical inference has proliferated, from work with young children to tertiary settings. In this paper, we review recent research on informal statistical inference, investigating issues and new questions that have emerged around looking “beyond the data”, using “data as evidence” and “articulating uncertainty”.


8C2: Students’ reasoning about uncertainty while making informal statistical inferences in an “integrated pedagogic approach”

Dani Ben-Zvi   University of Haifa, Israel
Hana Manor Braham   University of Haifa, Israel
Keren Aridor-Berger   University of Haifa, Israel

Reasoning about uncertainty is a key and challenging element in informal statistical inferential reasoning. We designed and implemented an “Integrated Pedagogic Approach” to help students understand the relationship between sample and population in making informal statistical inferences. In this case study we analyze two sixth grade students’ reasoning about uncertainty during their first encounters with making informal statistical inferences based on random samples taken from a hidden TinkerPlots2 Sampler. We identified four main stages in the students’ reasoning about uncertainty: Account for, examine, control, and quantify uncertainty. In addition, two types of uncertainties–contextual and a statistical–shaped the students’ reasoning about uncertainty and played a major role in their transitions from stage to stage. Implications for research and practice are also discussed.


8C3: Exploring informal inferential reasoning through data games

Kosoom Kreetong   University of Massachusetts - Amherst, United States

This study explores the use of the computer game Ship Odyssey to facilitate learning of ideas underlying students’ informal inferential reasoning as they solve parameter estimation problems. I observed seventh graders as they played the game. Players send rats to locate sunken treasure. The rats return with “noisy” readings. Players determine the treasure location using these data. Thus the game requires parameter estimation in a repeated measurement context, a context that various researches (Konold & Pollatsek, 2002; Lehrer & Kim, 2009) have claimed to be particularly conducive for learning to conceive of data as signal and noise. This conception is precursory to understanding that larger samples give better estimates of a population or process signal.


Session 8D: Research on developing students’ statistical reasoning

8D1: Long-term impact on students’ informal inferential reasoning

Einat Gil   University of Haifa, Israel
Dani Ben-Zvi   University of Haifa, Israel

Long-term effects of learning are a desirable outcome of any educational program and are far from being an obvious result in education. Furthermore, statistical concepts tend to be ambiguous and “short lasting” in students’ reasoning, even among tertiary students. In this longitudinal study, long-term impact of teaching and learning was sought among ninth graders, three years after their participation in a three-year intervention (grades 4-6) of the Connections Program. In a mixed methods study, students from two groups – those who have / have not taken part in the program – were closely followed and compared throughout three extended data inquiry activities and took a statistical knowledge and thinking proficiency test. Results and implications are presented.


8D2: Statistical reasoning with the sampling distribution

Bridgette L Jacob   Onondaga Community College, United States
Helen Doerr   Syracuse University, United States

Statistical reasoning surrounding the sampling distribution is necessary for formal statistical inference. Our study of introductory statistics students aged 16-18 suggests that knowledge of the characteristics of a sampling distribution and experiences with generating sampling distributions do not provide a sufficient basis for this reasoning. Following instruction on the sampling distribution and its characteristics, the students in this study were given the opportunity to sample before drawing an informal conclusion based on a sampling distribution. The majority of the students took several samples and/or considered generating a second sampling distribution for comparison. These are not incorrect statistical notions; and they do provide insight into possible missing elements in instruction. The results are discussed in relation to the development of students’ statistical reasoning throughout their introductory statistics course.


8D3: Extending the curriculum with TinkerPlots: opportunities for early development of informal inference

Noleine Fitzallen   University of Tasmania, Australia
Jane Watson   University of Tasmania, Australia

Although increasing in recent years, the statistics content in the curriculum does not always align with students’ ability to develop an understanding of those key concepts. In this paper, two examples are presented that challenge the school curriculum by introducing students to activities that focus on decision making under uncertainty earlier than when first acknowledged in the curriculum. The first study investigated Grade 5 and 6 students’ understanding of covariation; the second investigated Grade 10 students’ understanding of resampling. Common to both studies were the emphases on opportunities for the development of informal inference and the application of the software package, TinkerPlots. In both cases, many students based their confidence in conclusions on visual aspects of the strength of an association or shape of a simulated distribution. Besides making suggestions for future curriculum improvement, this presentation draws attention to pointers for development of student understanding.


Session 8E: Research on developing students’ probabilistic reasoning

8E1: Empirical research on understanding probability and related concepts — a review of vital issues

Manfred Borovcnik   University of Klagenfurt, Austria

The 2007 ASA report renewed the debate on qualitative vs. quantitative statistical methods in educational research and promoted the randomized controlled experiment (RCT) as standard. While quantitative methods have their value, the target of the two approaches is completely different and RCTs will not provide reliable evidence in cases when the researcher is interested in how students understand concepts and why some fail, or which teaching methods are helpful for whom. This paper focuses on issues for improving qualitative educational research. Task analysis is shown to be one major factor. In probability and statistics such an analysis is more basic than in other branches of mathematics as tasks are often – unintentionally and unnoticed – ambiguous.


8E2: Reasoning development of a high school student about probability concept

Julio C Valdez   Center for Research and Advances Studies of IPN, Mexico
Ernesto Sánchez   Cinvestav-IPN, Mexico

In this article the development of reasoning of a high-school student on the concept of probability is described from the inferences formulated as he solved three problems. An adaptation of Jones et al.’s (1999) framework was used to indicate the important characteristics in his reasoning. As a result, the difficulties faced by the student at different moments were: overcoming the law of small numbers, managing the variation in a convenient way, giving meaning to the quantification of the propensity of occurrence of an event, articulating the uncertainty of the individual outcomes with the long-run regularity of the relative frequencies, and using probability as a premise to formulate inferences. Three important elements to overcome these difficulties were: an informal knowledge of the law of large numbers, the reasoning with relative numbers and the partial institutionalization of the classical interpretation of probability when it was already informally used by the student.


8E3: Characteristics of students’ probabalistic reasoning in a simulation-based statistics course

Aaron Weinberg   Ithaca College, United States

Although students in traditional introductory college statistics courses see a frequentist definition of probability, they are rarely asked to use this definition, instead relying on technology to conduct parametric tests. In contrast, in a simulation- and randomization-based statistics course, the relative frequency of simulated outcomes becomes the central focus of the process of drawing inferences. In this study, eight college students who were enrolled in an introductory simulation-based statistics course were interviewed and asked to describe sampling distributions and make inferences; the results and analysis describes the ways they used and appeared to think about empirical probabilities. Although the students appeared to be able to make connections between various aspects of inferential reasoning, they also encountered difficulties that may be related to their focus on empirical probabilities.


Session 8F: Research on professional development of teachers in statistics

8F1: Teachers’ professional development in a stochastics investigation community

Adair Mendes Nacarato   University São Francisco, Brazil
Regina Célia Grando   University São Francisco, Brazil

The aim of this work is to reveal how the formation processes adopted by this investigation community have enabled teacher’s learning and, consequently, the professional development of all of the involved ones; and how Stochastic favours investigations in basic school teachers’ pedagogical practice. It refers to a longitudinal study carried out by the investigation community composed by professors and basic school teachers who have Stochastic as their object of investigation. The group prepared sequences of teaching which were developed by the teachers at their classrooms, using audio and/or video recordings, besides students’ written notes. This material, along with teachers’ narratives, was taken as an object of discussion and analysis of the group. The results indicate how teachers have learned from their own experience and others’ by analysing videotapes of the classes.


8F2: A teacher development program in statistics within a community of practice

Lucía Zapata-Cardona   University of Antioquia, Colombia

Very often, teachers’ development programs unconsciously promote the dichotomy between theory and practice. Repeatedly, teachers are invited to programs where they are listeners instead of doers. In those programs, the teacher is conceived as a reproducer instead of a producer of knowledge. A teacher development program in statistics inspired by the social practice theory is an attempt to close the gap that teachers face every day trying to put together theory and practice. In this paper, I discuss the lessons learned from a teachers’ professional development program in statistics following the principles of a community of practice. I discuss the strengths, the weaknesses of the program and the implications for teacher education.


8F3: Professional development for teaching statistics: a collaborative action research project with middle-school mathematics teachers

Leandro de Oliveira Souza   Federal University of Amazonas, Brazil

Teachers need to develop many dimensions of their practice such as statistical knowledge, beliefs about statistics, designing their own lessons and adopting new teaching approaches. We developed a project named Teacher Professional Development Cycle in Statistics (TPDC) which involved sixteen teachers of Mathematics of the elementary school in a process of collaborative action research. The data were collected based on recording videos of meetings and on six questionnaires which were answered by the teachers during the project. The results showed that teachers worked collaboratively overcoming their feelings of insecurities about teaching statistics when an investigative and exploratory approach was used to improve their knowledge and conceptual content. This paper will present our description of how the TPDC project was designed, as well as the advantages in using it in teacher development.


8F4: Learning and teaching statistical investigations: a case study of a prospective teacher

Raquel Santos   Higher School of Education of Santarém, Portugal
João Pedro da Ponte   University of Lisbon, Portugal

More than a collection of tools to deal with problems, statistics provides a comprehensive framework to think about the world. One way of using it is doing statistical investigations (SI). In this communication, we present a case study of a prospective primary school teacher regarding her perspective on teaching and learning SI. To do so, we analyze her written report of a SI, a questionnaire, observation of a SI she carried out in a grade 3 class, and interviews. Results show that this prospective teacher has difficulty in planning a SI for her students, mainly because she sees this activity as a sequence of techniques to be applied. This suggests that in teacher education programs, the analysis of what is involved in teaching statistical concepts through SI must receive specific attention, instead of using SI only as a context to apply concepts and work with data.


Session 8G: Theoretical frameworks in statistics education research

8G1: Describing distributions

Pip Arnold   Cognition Education Limited, New Zealand
Maxine Pfannkuch   University of Auckland, New Zealand

Describing distributions require students to simultaneously consider many aspects of five overarching statistical concepts — contextual knowledge, distributional, signal and noise, variability and graph comprehension — making describing distributions challenging. In this paper two students descriptions of distributions are compared and contrasted through the lens of the distribution framework (Arnold & Pfannkuch, 2012) as an initial insight into the different features of distribution that are emphasised in teaching and learning within a year 10 class (ages 14-15).


8G3: Cultural diversity in statistics education: bridging uniqueness

Pieternella Verhoeven   University of Utrecht, The Netherlands
Dirk Tempelaar   Maastricht University, The Netherlands

Learning does not take place in a vacuum. Culture, individual and social background influence how we learn. Differences in learning styles have always existed, but in recent years the emphasis has shifted to the cultural background of these differences. College students with different cultural backgrounds taking introductory statistics face a special challenge: besides employing growing differences in learning styles they also have different entrance levels. This results from a diversity in mathematics preparation during high school; it puts an extra burden on teachers taking into account these differences. Besides giving an overview of the current state of literature on this topic, this paper describes the results of an analysis of a recent Dutch study aiming to provide an insight in cultural differences in learning styles, attitudes and goal setting.


8G4: About central issues of mental model theory in context of learning statistics

Markus Vogel   University of Education Heidelberg, Germany
Andreas Eichler   University of Kassel, Germany

Learning to draw right conclusions and to make well-founded decisions on base of data is a goal being at the core of statistics education. Because of its central characteristics and underlying concepts the theory of mental model is well suited for describing and reflecting on these processes of statistics learning in two respects: On the one hand the concepts of structure and function of mental model theory allow for adequately capturing data-based situations which require drawing conclusions. On the other hand the concept of mental model is an essential part of widely accepted information processing theories which are currently at the forefront of research. Thus, students’ processing of data-based problem situations and making decisions can be highlighted in terms of this theoretical approach. Beside a survey of fundamentals of mental model theory the presentation focuses on these aspects. Theoretical insights will be underlined by empirical research results.


Session 8H: Publishing in education research journals (panel discussion)

Robert C delMas   University of Minnesota, United States
E Jacquelin Dietz   Meredith College, Raleigh, United States
Katharine Richards   Plymouth University, United Kingdom
Robert Gould   University of California, Los Angeles, United States


Session 8I: Research on risk literacy

8I1: Getting alternative representations for risk into the school syllabus

David Spiegelhalter   Cambridge University, United Kingdom
Jenny Gage   University of Cambridge, United Kingdom

I shall extend the discussion of my plenary talk to give more details of work to teach a range of alternative representations for probabilities. I will also cover efforts to get better probability representations into the GCSE (ages 11-16) syllabus for England and Wales: after various machinations these have met with moderate success, although it remains to be seen what examining boards will make of the opportunities. I will also discuss the potential for the new “core maths” qualification that is being introduced as a post-GCSE course for 250,000 students a year who do not go on to study maths at A level (ages 17-18). I will also, if time permits, provide a critique of the latest PISA scores for mathematics, arguing that much if the recent change is simply regression-to-the-mean.


8I3: Risk literacy: first steps in primary school

Christoph Till   Ludwigsburg – University of Education, Germany

Data about risks in health care or investments are usually presented in terms of proportions, probabilities and percentages. Research shows that risk communication can be misleading as these representation formats are sometimes hard to understand. Thus a first level of “Risk Literacy” should be achieved as early as possible. With cognitive psychologists of the Harding Center for Risk Literacy I developed a school intervention on “Risk” for fourth graders. The intervention contains elements of proportional reasoning, loss/benefit trade-offs, conditional probabilities as well as dealing with relative and absolute risks. Prior studies have shown that hands-on materials in form of colored tinker-cubes in combination with “natural frequencies” and iconic representations constitute a good approach. The intervention was tested and evaluated by means of a pre-test, a post-test and a follow-up test. The results indicate that students benefited by the intervention as test performances increased significantly.


8I4: Comparing fast and frugal trees and Bayesian networks for risk assessment

Kathryn Laskey   George Mason University, United States
Laura Martignon   Ludwigsburg University of Education, Germany

Fast and frugal trees have been proposed as efficient heuristics for decision under risk. We describe the construction of fast and frugal trees and compare their robustness for prediction under risk with that of Bayesian networks. In particular we analyze situations of risky decisions in the medical domain. We show that the performance of fast and frugal trees does not fall too far behind that of the more complex Bayesian networks.


Session 8J: Research on technology in statistics education

8J1: Constructing inferential concepts through bootstrap and randomization-test simulations: a case study

Maxine Pfannkuch   University of Auckland, New Zealand
Stephanie Budgett   University of Auckland, New Zealand

Statisticians such as Cobb (2007) have promoted the use of computer intensive methods such as bootstrapping and the randomization test in introductory statistics courses. One of their arguments for using these simulations is that the logic of inference is conceptually more accessible to students than the traditional approach. In this paper we test indirectly the claim that simulations assist in the construction of inferential concepts using an analytical tool that is based on the versatile thinking framework for conceptual development. Using the tool, which identifies nine possible modes of student interaction with representations, we analyse two introductory statistics students’ interactions with the Visual Inference Tools (VIT) bootstrap confidence interval construction and randomization test modules. Our findings suggest that, for these students, the VIT simulations were facilitating the development of statistical inferential concepts.


8J2: Measuring the effectiveness of using computer assisted statistics textbooks in Kenya

Zachariah Mbasu   Maseno University, Kenya
Roger Stern   Maseno University, Kenya
David Stern   Maseno University, Kenya
Bernard Manyalla   Maseno University, Kenya

There is currently a big push to integrate technology into Kenyan education at all levels. Statistics is a subject that can benefit immensely from this improved access to technology. This study attempts to quantify the effect Computer Assisted Statistics Textbooks (CAST) has on student interest and performance. In Kenya CAST has now been used across all academic levels from schools to postgraduate, the implementations in a Diploma program is such that it allows for a quantitative analysis of performance against standardised grading, while the implementation in schools led to good data on student motivation. These implementations will be the focus of this paper.


8J3: Comparing groups by using TinkerPlots as part of a data analysis task — tertiary students’ strategies and difficulties

Daniel Frischemeier   University of Paderborn, Germany

The comparison of distributions of numerical variables is a fundamental idea in descriptive statistics. Preferably those processes are to be embedded in a data analysis-cycle. This emphasizes working with real and multivariate data and generating interesting statistical hypotheses. In the context of preservice teacher education, we designed an experimental course comprising 15 sessions on data analysis with TinkerPlots (Konold & Miller, 2011) in which Group comparisons played a fundamental role. After the course we have conducted a video study where we observed the participants while comparing groups with TinkerPlots. In the paper we will focus on several steps and the frequency of their occurrence, which can be identified when learners do group comparisons facilitated by software.