Future City Lab: Neighborhood Stats
ChiSquared Statistics
Keywords: null hypothesis, bias, statistical significance, chisquared, pvalue, social science
Objectives:
Students will:

Derive and use chisquared tests

Explain the use of this test in the context of New York City demographics

Compare and contrast social sciences with socalled “hard science”

Consider to what extent statistics are useful in the study of humans
Materials:

One bag of 30 beads (or Skittles) per pair of students. There should be five different colors.

Handouts (two student activity handouts, one data sheet, and one teacher key)

Calculators
Standards:

CCSS.ELALITERACY.RST.910.7: Translate quantitative or technical information expressed in words in a text into visual form (e.g., a table or chart) and translate information expressed visually or mathematically (e.g., in an equation) into words.

CCSS.MATH.CONTENT.HSS.MD.A.4: Develop a probability distribution for a random variable defined for a sample space in which probabilities are assigned empirically; find the expected value.
Guiding Questions:

How can you test whether a group is randomly distributed?

What are the major differences between the “hard sciences” and “social sciences” and what are the data limitations in each?

How can we use statistics from today to help us understand the past?
 Hook
 Practice
 Summary
 Apply
 Analyze the Data
 Discussion: the Nature of Social Sciences
 Discussion: Introduction to Historical Factors
Procedures
Chisquared tables are tools to help scientists understand whether data is significantly different from a randomized outcome – in other words, whether there is some external factor at play that needs to be explained. They are often illustrated in the classroom using inanimate objects or nonhuman actors (beads, genetic outcomes). This lesson considers whether statistical methods like chisquares can be useful to evaluate human behavior and whether it might be a tool to consider in the history classroom. A helpful tutorial for the nonscience classroom teacher can be found here: http://www.ling.upenn.edu/~clight/chisquared.htm .
In the first part of the lesson, students derive chisquared values by testing the distribution of beads. Then, students are presented with a more difficult problem: Are different ethnic groups in New York distributed randomly? Students can determine this by using calculators. (All New York City neighborhoods that they will be given do have significantly high chisquared values.) The teacher then will lead the students through a discussion of whether these types of tools can be meaningful in analyzing human behavior, and a discussion of the differences between social science and “hard science” more generally. Finally, the lesson ends with a discussion of whether these tools are useful in analyzing history and points towards resources that explore historical factors that influence why people in New York City are grouped nonrandomly.
Download the Neighborhood Stats Bead Activity Worksheet
Download the Neighborhood Stats Demographic Handout
Make an outlandish claim to students, something like “People who wear buttondown shirts can jump higher than those who wear tshirts.” Find a student to disagree with you. Ask the class: Who has the burden of proof? Which should we assume with no evidence? Why? Have students share responses.
Explain the goal of all experiments is to test the null hypothesis. Hypotheses are testable explanations for phenomena. The null hypothesis is the hypothesis that explains any differences as the result of randomness/chance. Until sufficient evidence is discovered, we cannot adopt new (alternate) hypotheses.
The specific alternative hypothesis the teacher proposed (H_{alt}) was: The differences between jumping heights of people can be predicted by the type of shirt they wear.
The null hypothesis (H_{0}) for the tshirt example would be: The differences between jumping heights of people is not related to the shirt they wear.
How can we figure out which is correct? This is where chisquare analysis comes in.
Explain to students that you’ve filled each bag with 30 beads of five different colors. Their job is to determine if you did so randomly or with a bias. Check students: Which explanation is the null hypothesis? (Answer: The null hypothesis is that they were distributed randomly, with no bias.)
Ask: if the null hypothesis (no bias) is true, what would you expect the distribution of the beads to be? (Answer: I’d expect with 30 beads and 5 colors that there would be 6 of each.)
How might a scientist/statistician determine how far away the actual (observed) numbers are from the expected? Allow students time to brainstorm potential mathematical solutions. They should be able to come up with subtraction as the way of measuring difference.
The equation for chisquared is:
x^{2} = [SUM] (o – e)^{2 }/ e
o = observed (this would be how many of each category there actually are)
e = expected (this is the amount predicted by random chance)
The difference represents how far each value is from the prediction of the H_{0}.
The differences are squared to eliminate negative numbers. Then they are divided by the expected value. This is like taking an average.
The resulting values are all added up (denoted by the sum, or sigma on handout). The larger the differences, the larger the chisquared value. Note that there is no consideration of sample size here, so a larger sample will in general mean a larger chisquared value.
Explain: Chisquared is a value that measures how far the observed is from the expected. If it is far enough away, we can reject the null hypothesis. In other words, we can say in this case that you, the teacher, did not act randomly. Instead, for some reason, you selected which beads to give to each group and therefore acted with bias. Naturally, this test cannot ascertain where this bias comes from. Ask students to come up with their own alternative hypotheses. Examples may include:
1.) The teacher prefers to use a certain color bead over other color beads.
2.) The teacher choice was random, but the original source of beads did not have equal numbers of each color.
Have students follow the directions on the handout to test their own bag. The will need to calculate degrees of freedom (number of colors – 1) and be taught to use the pvalue chart. All tables and charts are provided on the worksheet.
Students share out results as group, come to group decision as to whether there was bias in the distribution of beads.
Students are presented with real demographic data of neighborhoods in New York City from the Department of Education. The first task is to determine what the null hypothesis is: If the U.S. census tells us that 29% of New Yorkers identify as Hispanic, what would the null hypothesis state about the population of people in a given neighborhood of the city? (Answer: We’d expect 29% of the people in each neighborhood to be of Hispanic origin.)
Students choose a neighborhood from the provided data handout to test. Since chisquared only works well with smaller sample sizes, they shouldn’t use the raw data. They can use the percentages as the sample, as if each neighborhood had only 100 people in it. This work can be done on the handout with calculators.
Ask students to share their results. Why did every neighborhood have a significant chisquared value? What does that mean, statistically? (Every neighborhood has a significant chisquared value. That means we can reject the null hypothesis that people are distributed randomly.)
Reinforce: the data show that populations are not distributed randomly in these New York City neighborhoods – but the data also cannot tell us why. The Chisquare analysis simply lets us know that bias of some form is at play, and that the difference between the actual outcomes and the expected outcomes is statistically significant. In this scenario, determining why this might be the case is traditionally the realm of the social sciences – not statistics.
Use these questions to guide an indepth discussion and/or writing activity on the potential applications of the skills the students have just learned.
Explain to students that chisquares are traditionally used with populations that don’t exert choice (as in the case of the beads or skittles in the first activity). Analyzing human behavior has generally been the realm of the socalled “social sciences” (e.g. psychology, economics, sociology, political science, and history).
Ask: What do the “hard sciences” and “social sciences” have in common?
Ask: What challenges are unique to social science? Why is it more difficult to draw conclusions from a study of humans than one of plants or even model animals like mice? Is it useful to quantify human behavior? What potential use are demographics? What are the limitations of statistical studies like this one?
Ask: Why does it make sense that the we could reject the null hypothesis in all the neighborhoods we analyzed? (Expect answers along the lines of: people choose where they live, or people are limited by what they can afford.)
On the topic of the choices of immigrants: Why might groups of people choose to live near each other?
On the topic of other external factors: What other factors might limit the choice of immigrants in terms of where they can live?
Throughout New York City’s history, certain groups of people have been coerced or forced to live in certain areas – or, at the very least, have been kept from living in the most “desirable” parts of the city. In the middle of the 20th century, redlining – in which African Americans were denied mortgages within traditionally Black neighborhoods – prevented many African Americans from owning homes (and thus joining the middle class), and helped concentrate African Americans within certain sections of the city. On the other hand, many African Americans moved to traditionally Black neighborhoods like Harlem, which served as a safe space amidst the violence of Jim Crow America and allowed the freedom of creation, thought, and speech that fomented the Harlem Renaissance.
With the pushpull of history at play and multiple historical factors and contexts informing the choices (or lack of choice) of human actors, can we use statistical analysis as a meaningful tool in history? If you were writing a history paper, would you consider including chisquared analysis to support your argument?
Additional Resources
Use this website to teach about the history of redlining: how 1940s policies resulted in de facto segregation in cities: https://dsl.richmond.edu/panorama/redlining/
For a comprehensive history of redlining, see Richard Rothstein’s The Color of Law: A Forgotten History of How Our Government Segregated America (Liveright, 2017).
Fieldtrips: This content is inspired by the World City, 18982012 and Future City Lab galleries in the Museum’s flagship exhibition, New York at Its Core. If possible, consider bringing your students on a fieldtrip! Visit http://mcny.org/education/fieldtrips to find out more.
Acknowledgements
This series of lesson plans for New York at Its Core was developed in conjunction with a focus group of New York City public school teachers: Joy Canning, Max Chomet, Vassili Frantzis, Jessica Lam, Patty Ng, and Patricia Schultz.
This project was made possible in part by the Institute of Museum and Library Services.
The views, findings, conclusions or recommendations expressed in these lessons do not necessarily represent those of the Institute of Museum and Library Services.