Future City Lab: Neighborhood Stats

Make an outlandish claim to students, something like “People who wear button-down shirts can jump higher than those who wear t-shirts.” Find a student to disagree with you. Ask the class: Who has the burden of proof? Which should we assume with no evidence? Why? Have students share responses. 

Explain the goal of all experiments is to test the null hypothesis. Hypotheses are testable explanations for phenomena. The null hypothesis is the hypothesis that explains any differences as the result of randomness/chance. Until sufficient evidence is discovered, we cannot adopt new (alternate) hypotheses.  

The specific alternative hypothesis the teacher proposed (H_alt) was: The differences between jumping heights of people can be predicted by the type of shirt they wear. 

The null hypothesis (H₀) for the t-shirt example would be: The differences between jumping heights of people is not related to the shirt they wear. 

How can we figure out which is correct? This is where chi-square analysis comes in. 

Explain to students that you’ve filled each bag with 30 beads of five different colors. Their job is to determine if you did so randomly or with a bias. Check students: Which explanation is the null hypothesis? (Answer: The null hypothesis is that they were distributed randomly, with no bias.) 

Ask: if the null hypothesis (no bias) is true, what would you expect the distribution of the beads to be? (Answer: I’d expect with 30 beads and 5 colors that there would be 6 of each.)  

How might a scientist/statistician determine how far away the actual (observed) numbers are from the expected? Allow students time to brainstorm potential mathematical solutions. They should be able to come up with subtraction as the way of measuring difference. 

The equation for chi-squared is: 

x² = [SUM] (o – e)² / e

o = observed (this would be how many of each category there actually are) 

e = expected (this is the amount predicted by random chance) 

The difference represents how far each value is from the prediction of the H₀. 

The differences are squared to eliminate negative numbers. Then they are divided by the expected value. This is like taking an average. 

The resulting values are all added up (denoted by the sum, or sigma on handout). The larger the differences, the larger the chi-squared value. Note that there is no consideration of sample size here, so a larger sample will in general mean a larger chi-squared value. 

Explain: Chi-squared is a value that measures how far the observed is from the expected. If it is far enough away, we can reject the null hypothesis. In other words, we can say in this case that you, the teacher, did not act randomly. Instead, for some reason, you selected which beads to give to each group and therefore acted with bias. Naturally, this test cannot ascertain where this bias comes from. Ask students to come up with their own alternative hypotheses. Examples may include: 

1.) The teacher prefers to use a certain color bead over other color beads. 

2.) The teacher choice was random, but the original source of beads did not have equal numbers of each color.  

Have students follow the directions on the handout to test their own bag. The will need to calculate degrees of freedom (number of colors – 1) and be taught to use the p-value chart. All tables and charts are provided on the worksheet.  

Students share out results as group, come to group decision as to whether there was bias in the distribution of beads. 

Students are presented with real demographic data of neighborhoods in New York City from the Department of Education. The first task is to determine what the null hypothesis is: If the U.S. census tells us that 29% of New Yorkers identify as Hispanic, what would the null hypothesis state about the population of people in a given neighborhood of the city? (Answer: We’d expect 29% of the people in each neighborhood to be of Hispanic origin.) 

Students choose a neighborhood from the provided data handout to test. Since chi-squared only works well with smaller sample sizes, they shouldn’t use the raw data. They can use the percentages as the sample, as if each neighborhood had only 100 people in it. This work can be done on the handout with calculators. 

Ask students to share their results. Why did every neighborhood have a significant chi-squared value? What does that mean, statistically? (Every neighborhood has a significant chi-squared value. That means we can reject the null hypothesis that people are distributed randomly.) 

Reinforce: the data show that populations are not distributed randomly in these New York City neighborhoods – but the data also cannot tell us why. The Chi-square analysis simply lets us know that bias of some form is at play, and that the difference between the actual outcomes and the expected outcomes is statistically significant. In this scenario, determining why this might be the case is traditionally the realm of the social sciences – not statistics. 

Use these questions to guide an in-depth discussion and/or writing activity on the potential applications of the skills the students have just learned. 

Explain to students that chi-squares are traditionally used with populations that don’t exert choice (as in the case of the beads or skittles in the first activity). Analyzing human behavior has generally been the realm of the so-called “social sciences” (e.g. psychology, economics, sociology, political science, and history).  

Ask: What do the “hard sciences” and “social sciences” have in common?  

Ask: What challenges are unique to social science? Why is it more difficult to draw conclusions from a study of humans than one of plants or even model animals like mice? Is it useful to quantify human behavior? What potential use are demographics? What are the limitations of statistical studies like this one? 

Ask: Why does it make sense that the we could reject the null hypothesis in all the neighborhoods we analyzed? (Expect answers along the lines of: people choose where they live, or people are limited by what they can afford.)  

On the topic of the choices of immigrants: Why might groups of people choose to live near each other? 

On the topic of other external factors: What other factors might limit the choice of immigrants in terms of where they can live? 

Throughout New York City’s history, certain groups of people have been coerced or forced to live in certain areas – or, at the very least, have been kept from living in the most “desirable” parts of the city. In the middle of the 20th century, redlining – in which African Americans were denied mortgages within traditionally Black neighborhoods – prevented many African Americans from owning homes (and thus joining the middle class), and helped concentrate African Americans within certain sections of the city. On the other hand, many African Americans moved to traditionally Black neighborhoods like Harlem, which served as a safe space amidst the violence of Jim Crow America and allowed the freedom of creation, thought, and speech that fomented the Harlem Renaissance.  

With the push-pull of history at play and multiple historical factors and contexts informing the choices (or lack of choice) of human actors, can we use statistical analysis as a meaningful tool in history? If you were writing a history paper, would you consider including chi-squared analysis to support your argument?