This story was originally published by CalMatters.
The system California uses to classify neighborhoods at risk of environmental harm is highly subjective and flawed, potentially leaving communities missing out on billions of dollars in funding. according to new research.
The study, by researchers who started the project at Stanford University, examined a tool that the California Environmental Protection Agency developed in 2013 as the nation’s “first comprehensive statewide environmental health screening tool” to identify communities disproportionately burdened by pollution.
Communities designated as “disadvantaged” by the system, named CalEnviroScreen, can qualify for substantial government and private funding. The tool was used to target large parts of the Central Valley, communities around the ports of Long Beach and Los Angeles, and neighborhoods in the Bay Area cities of Richmond and Oakland, among others.
The researchers found that the screening tool used a small number of health problems that could bias the communities. About 16 percent of census tracts in the state could be ranked differently with changes in EnviroScreen’s model, according to the study.
The system raises equity issues because it is biased in favor of certain groups over others, and has the potential to pit groups against each other for funding in what is essentially a winner-take-all, or loser-take-all, system. according to the research.
For example, “we found that the existing model potentially underrepresents foreign-born populations,” the researchers wrote.
Loren Elliott for CalMatters
Community groups and environmental justice advocates have said for years that the tool communities look out which must be designated as disadvantaged.
At stake is a huge amount of funding — about $2.08 billion over just a recent four-year period, the researchers reported.
The findings come as scientists increasingly show that algorithms can be as biased as the people who create them, and that many disproportionately harm marginalized populations.
“The big takeaway is that if you asked 10 different experts in California to come up with their own screening algorithm to determine which neighborhoods are ‘disadvantaged,’ you’d probably get 10 very different algorithms,” said lead author Benjamin Huynh, who is a doctoral student at Stanford and is now a researcher at Johns Hopkins University. “These things can come across as very technical, but when you look at the numbers and you see the billions of dollars flowing … these very seemingly technical details actually matter a lot.”
Amy Gilson, a spokeswoman for CalEPA’s environmental health office, said the study’s recommendations are being reviewed. Any potential changes to CalEnviroScreen must “go through a robust scientific evaluation” as well as “extensive public process,” she said.
“CalEnviroScreen’s methods are transparent to allow for these types of external evaluations, and we welcome discussion about the merits of different approaches,” Gilson said in an emailed statement to CalMatters.
CalEnviroScreen identifies neighborhoods by census tracts—localized regions that typically include between 1,000 and 8,000 residents, as defined by the U.S. Census Bureau. California released its fourth iteration of CalEnviroScreen in October 2021.
CalEnviroScreen evaluates 21 environmental, public health and demographic factors to identify which neighborhoods are most susceptible to environmental damage. Among the factors considered: air and drinking water pollution, pesticide use, toxic emissions, low birth weight babies, poverty and unemployment rates. The tool then ranks the 25 percent most disadvantaged communities in California — which determines which neighborhoods get billions of dollars in state and private funds.
Under state law, at least a quarter of funds from the California Climate Investments fund must be spent on these communities. That money comes from California’s cap-and-trade market program, which allows polluters to buy credits to offset their emissions.
In 2022 the fund paying for nearly 19,500 new projects with $1.3 billion, according to the state Air Resources Board. Of that, $933 million was directed to disadvantaged or low-income communities, the air board said.
Huynh said he became interested in CalEnviroScreen’s classification of neighborhoods after reading a 2021 article in The San Francisco Chronicle that found some of San Francisco’s poorest neighborhoods ineligible for funding, largely due to their position in CalEnviroScreen.
“Under such a high-uncertainty model, every subjective model decision is implicitly a value judgment,” the study authors wrote. “Any variation of a model can benefit one subpopulation or harm another.”
The tool includes only three health factors – low birth weight babies, cardiovascular disease, and emergency room visits for asthma. That leaves out other serious health conditions, such as chronic obstructive pulmonary disease, which the authors say could mean leaving out communities with many foreign-born residents. Asthma may be less common among immigrants or they may be less likely to seek emergency room care, but they still have other serious respiratory problems, the study said.
Other common health problems, such as cancer and kidney disease, are also left out, which can skew which neighborhoods are designated as disadvantaged. The authors said that changing the instrument to include these diseases could mean that fewer black communities are designated as disadvantaged. This is because it would dilute the importance of low birth weight babies, which disproportionately affect black people.
Race is not a factor in the screening system. But the researchers found that tweaking the model could make big differences for communities of color: For example, they found that changes in the metrics would mean more non-white communities with high levels of poverty would be classified as disadvantaged.
The research team suggested some possible solutions “to reduce equity concerns,” such as using multiple models. Doing so will increase the number of designated communities by 10 percent.
“Because there is no single ‘best’ model, we suggest assessing robustness through sensitivity analysis and including additional models accordingly,” the researchers wrote.
Additionally, a safeguard such as an external advisory committee comprised of domain experts and leaders of local community groups can also help reduce harm by identifying ethical issues that may have been missed internally.
