Racial segregation in St. Louis (blue is African American, red is white). Map: Eric Fischer

Polls show that Americans’ views of the state of race relations have soured recently. It is not that everyday race relations have turned for the worse; real trends, such as increasing cross-racial marriages, are positive. But current events, including the killing of Michael Brown in Ferguson, Missouri, and subsequent protests, have revealed problems many Americans never saw before. They have provided a dose of realism. Contributing to this realistic view are new tools, in particular, videos of police activity.

New tools also allow us to more realistically see the racial past, such as the rigidity of residential segregation over time. For many years scholarly consensus held that severe black-white residential segregation was a northern urban development of the mid-twentieth century, produced by a combination of heavy black migration from the rural South and redlining—real estate interests colluding with the federal government to enclose black neighborhoods. But studies now show how profound racial segregation was in the nineteenth century and how it is being reshaped in the twenty-first. We are finding that segregation is a matter of scale.

Historical segregation turns out to be greater than we thought.

Historically, researchers measured black-white segregation mainly by comparing the distribution of blacks and whites across geographical units, typically census tracts, which generally contain thousands of residents. An even sprinkling of blacks and whites across the census tracts of a city registered as low segregation, while concentrations of blacks in some and whites in others indicated high segregation. We have learned a good deal about this district-level segregation, but, it turns out, this doesn’t describe how Americans personally experienced segregation.

Thanks to the Minnesota Population Center, which has collected and processed original handwritten census forms going back to the nineteenth century, researchers can now look much more closely at who lived in which neighborhoods and even in which buildings. They can assess the degree to which blacks and whites were next-door neighbors who probably interacted, rather than just people sharing a census tract. At that finer scale, historical segregation turns out to be greater than we thought.

John Logan, Weiwei Zhang, and Miao David Chunyu, in the American Journal of Sociology, investigated segregation in New York City and Chicago from 1880 to 1940. (For now, original census manuscripts are only available through 1940, because privacy rules keep individual information under wraps for seventy-two years.) The group identified who was living in which buildings on which streets. Many 1880 neighborhoods that seemed, by previous methods, integrated were really much less so. Many blacks residing in largely white neighborhoods were live-in or nearby servants to white employers and not members of independent households. Also, adjacent apartment buildings were often segregated by race. In an era when apartment dwellers typically shared bathrooms, kitchens, and stoops and when almost everybody walked to work, segregation at this smaller scale mattered.

Census officials instructed their agents to approach every household in a neighborhood by going unit to unit in a specific pattern, which means that families listed after one another on the forms lived next to each other on the street. Angelina Grigoyeva and Martin Ruef, in the American Sociological Review, exploit this pattern to reverse our understanding of regional differences in 1880. Southern cities were indeed less segregated than northern ones—but at the district level. Grigoyeva and Ruef found that a type of “street-front segregation” was much more pervasive in the South: whites lived in large homes facing main streets, while blacks occupied huts in back alleys. Southern blacks and whites often shared districts or even blocks, but their social distance was huge. The big house/small shack pattern is, of course, reminiscent of owner/slave housing arrangements. It also came with caste cultural codes, often enforced by law, that left southern whites comfortable with having blacks in the alleys behind them. Blacks knew their (social) place. In the North, where race codes were much weaker, whites used physical distance, maintaining distinct neighborhoods, to divide themselves from blacks.

Trevor Logan and John Parman, reporting in a National Bureau of Economic Research paper, used similar lists to calculate how often the heads of neighboring households were of different races. They came up with a neighbor-level segregation index, which also overturns the old picture of nineteenth-century southern integration. At the scale of adjoining doors, southerners were considerably more segregated than northerners were. Logan and Parman also found a substantial increase in neighbor-level segregation across America, northern and southern, urban and rural, from 1880 to 1940. “The likelihood,” they write, “that an African American household had a non-African American neighbor declined by more than 15 percentage points . . . through the mid-twentieth century.”

To understand what happened after 1940, we must return to looking at district-level segregation, however limited that technique is, because the more intimate data are still sequestered. Most research suggests that district-level racial segregation peaked around 1970 and has been declining since. By that measure, race relations have been improving. But attention to the scale of segregation reveals a more complex story—one that gets us back to Ferguson.

Daniel T. Lichter, Domenico Parisi, and Michael C. Taquino, also in the American Sociological Review, track black-white segregation in the United States from 1990 to 2010. They extend work that colleagues and I published in 2004, which showed that segregation operates differently at different spatial levels. Races can be mixed or separated at the level of neighborhoods (compare, say, 125th Street in Harlem to Gramercy Park), urban regions (center cities versus outer suburbs), or municipalities (say, suburbs of St. Louis: 67 percent black Ferguson versus 12 percent black Maryland Heights in 2010). Lichter and colleagues show that overall segregation declined modestly in American metropolitan areas from 1990 to 2010 at the census-tract level. This means that blacks and whites increasingly shared city districts. (It is likely that if the research team had access to finer-grained data, they would have found more street- and neighbor-level integration, too.) But the researchers also found some increase in segregation based on municipal lines. That is, racial segregation in America today functions more according to official borders; it is less a question of who one’s neighbor is than who else lives in one’s city or town.

In an era when most people drive many miles to work or shop or play, this larger scale matters. It matters, especially, because cities—New York and Chicago, but also Ferguson and Maryland Heights—determine important aspects of people’s lives, such as which public schools they can access, how much tax they pay, and dramatically, how they are policed. Local governments also screen who can move in by using housing codes of various kinds, such as zoning that rules out multi-family buildings or requires large, and therefore expensive, residential lots. The old style—micro-segregation of the nineteenth- and mid-twentieth century—has declined. Indeed, overall segregation has declined, but it is still here, taking on new forms at a larger geographic and political scale.