California Air Resources Board

Scope of Work

Objective

The objective of this study is to improve CARB’s health analysis method to better account for impacts in overburdened communities. This study would utilize high spatial resolution exposure assessment methods and health estimates (e.g., baseline disease rates). The results of this study will allow for the quantitative analysis of the health impacts of criteria pollutant control scenarios at a more granular spatial scale than CARB’s current state and regional regulatory health assessments. In addition, this study should aim to develop methodologies to incorporate consideration of socio-economic (SE) and community factors, thus providing demonstrably improved estimates of the health impacts faced by communities, especially for vulnerable communities such as those identified as disadvantaged (e.g., communities identified in SB 535).

Background

Many air pollution regulations undergo health benefit assessments to determine the estimated changes in air pollution-related health outcomes, as conducted by U.S. EPA and others. These quantitative assessments utilize multiple components, including concentration-response relationships, population demographics and projections, baseline disease rates, and health valuations.CARB routinely evaluates the health benefits of its regulations and programs to reduce air pollution exposure using a streamlined health analysis model and state emission, mortality and disease incidence, and population data sources.

CARB’s current methodology (incidence-per-ton) for assessing health impacts of PM2.5^‎1 is conducted at a regional level (e.g., air basin), which does not account for the disproportionate impacts experienced by communities or subpopulations. CARB uses a California specific Incidence Per Ton method‎ for assessing the health impact of PM2.5, which is developed based on BenMAP assessment at 12-km resolution. Background on CARB’s health analysis methodology is available on the CARB website. Previous research has shown that the vulnerability of communities to air pollution health effects is influenced by multiple factors, including exposure, socio-economic, and community characteristics³. Moreover, the spatial resolution of health benefits analysis often lacks the granularity needed to capture the heterogeneity of air pollution exposure accurately⁴. For health benefits assessment at the local scale, inputs such as air quality, baseline disease rate, and community data are important to provide a high level of accuracy and minimize biases and uncertainties. Such biases could limit the ability to identify and address the areas and populations most at risk, particularly within cities.

There is a growing need to employ high-resolution, local-scale data to more accurately assess health impacts and better understand differential impacts between communities. To address this challenge, analytical approaches have been suggested, such as using pooling techniques or meta-analyses‎³. Additionally, finer resolution and more demographically resolved (e.g., by race/ethnicity where possible) baseline health data, in combination with more accurate exposure data, could improve health outcome estimates. Finer-resolution health data could be developed by using prediction or interpolation methods (e.g., small area estimation‎⁵). Developing methodological improvements to improve health outcome estimates will help CARB to evaluate the health impacts of air pollution on a finer scale, focusing on community and subpopulation levels, and identifying disproportionate health risks in overburdened communities.

The key focus for this contract is to improve CARB’s health analysis method to better account for impacts in overburdened communities. This would involve the development of approaches for integrating advanced air quality modeling and finer-scale health information.

The main objectives for this project will be to:

• Develop an advanced exposure assessment with high-resolution air quality models, such as CMAQ, to evaluate the exposure of overburdened communities to criteria pollutants. The primary pollutant of interest for this study is PM2.5, but other criteria pollutants can be proposed.
• Establish a methodology for estimating fine-scale spatial baseline health data^‎6,‎7, accounting for limitations in health data availability.
• Create an approach to take into account the impacts of socio-economic and community characteristics on health outcomes, enabling a more nuanced understanding of how different populations experience the effects of air pollution.
• Define criteria for selecting communities for in-depth exposure analysis, prioritizing those with high pollution burdens and vulnerability, such as those in the top percentiles for cumulative impacts, using CalEnviroScreen scores. The communities should be selected to ensure the scalability of results to other comparable communities elsewhere in the state. The selected communities will be used as key sites for ground truthing of pollution (see next bullet).
• Conduct ground truthing, which may include consultation with a technical advisory group, use of portable monitoring equipment and saturation monitoring^‎8, and validation of air quality modeling data.

The results of this study will allow CARB to better estimate and inform the public of the health impacts of the agency’s air pollution reduction programs in overburdened communities. Due to the broad scope of this project, teams involving multiple disciplines and universities are encouraged to apply.

Scope of Work

The proposed study should address the following objectives.

Task 1: Literature review of community health impact assessments

• The contractor should conduct a thorough literature search on
  • Current knowledge, including gaps in health data, particularly in terms of geographical resolution of baseline disease rates and specific concentration-response functions.
  • Existing methodologies for incorporating socio-economic datasets in health analysis, as well as results of previous epidemiological studies about developing socio-economic and community adjustment factors that highlight the sensitivities of overburdened communities to exposure to air quality.
• Deliverables:
  • The literature review findings will be included in the draft final report.

Task 2: Technical advisory group (TAG) including community experts 

• The researchers should assemble a Technical Advisory Group (TAG) to guide the study design and execution, and to review interim deliverables as well as the final report. The TAG should consist of at least 5 representatives who have extensive knowledge related to air quality and health and/or health policies. The TAG should include community experts and other members from state and local agencies or academia. Inclusion of community experts in the TAG is required, and all TAG members must be compensated as sub-contractors with contract funds. The TAG roles in this study include 1) reviewing the study design, 2) assuring the quality of the exposure data through ground-truthing and reviewing the selection of communities for this purpose (see task 3), 3) providing feedback on the methodology used to estimate baseline disease rates and develop community adjustment factors (see tasks 4 & 5). The researchers should submit a draft list of TAG members to CARB staff for approval.

Task 3: Exposure assessment and ground-truthing of air quality data

• Use high spatial resolution air quality models to compare exposures in overburdened communities with those in other communities in California and develop metrics that can be used to estimate exposure discrepancies from regional analyses. The exposure assessment should be statewide and performed using state-of-the-art air quality models (such as CMAQ) and the latest quality-assured air quality data. The air quality modeling should compare a baseline exposure scenario and a future exposure scenario associated with emission changes from dominant source categories (to be determined in consultation with CARB staff). The air quality modeling should be simulated for a period of at least one year (to represent long-term exposure). The simulation time frame of the future scenario can be decided by the research team. The assumptions for spatial distribution of emissions to grid-cells and future projections of emission and population data should be reviewed by CARB staff before proceeding to modeling. The spatial resolution of the air quality modeling should be fine enough to make meaningful comparisons between communities (e.g., census tract level or any finer spatial resolution that the researchers think is appropriate to this project). The main pollutant of interest for this study is PM2.5, but if feasible, investigators can also include ozone.
• Conduct quality control of air quality data through ground truthing in coordination with TAG members. The ground-truthing will consist of validation and quality assurance of the results of air quality modeling with the use of saturation monitoring by mobile measurements, deploying low-cost sensors within study communities, using existing networks of low-cost sensors, or a combination of these approaches. The ground truthing methods must be reviewed by TAG members. Consultation with a community partner as a subcontractor is required in the case of performing ground truthing in the form of mobile measurements or deploying new sensors within communities.
• Identify viable communities for ground truthing and quality assurance of air quality data. The number of communities selected for ground truthing can be decided based on the ground truthing method, but at least two communities should be selected. The communities should be selected to ensure the scalability of the results to other similar communities across the state, based on information such as land-cover, meteorology, and their proximity to sources of emissions. The research team will consult with TAG members on the selection of communities for ground truthing.
• Deliverables:
  • Summary of the ground truthing methodology that is reviewed by the TAG members to be approved by CARB staff. Consultation with the TAG and the approval of the process by CARB is required to proceed to ground truthing.
  • All the intermediate data, including the results of the air quality modeling and data collected in identified communities. Census tract specific adjustment factors that capture the increased air pollution exposure in overburdened communities compared to the average population across California. These adjustment factors will be used in CARB's health benefits assessment methodology to assess the health effects in overburdened communities compared to the average population. A summary of statistical comparisons of air quality data of communities across California, the results of the ground truthing of air quality data, and the summary of community engagement effort.

Task 4: Refining the spatial resolution of health data

• Develop a methodology to overcome current spatial limitations for constructing baseline health profiles for communities, preferably at the scale of census tracts, utilizing statistical models (e.g., small-area estimation) to estimate disease rates^‎4. The study should investigate the baseline disease rates for all-cause mortality, but other health effects can also be proposed. Investigators should consider the availability of the health data from previous studies and available datasets to estimate baseline disease rate data at a finer resolution (as fine as the availability of the data and current methodology allows). The baseline disease rates should be stratified or adjusted by race/ethnicity, income level, and educational status where possible.
• Deliverables:
  • Providing baseline disease rate data, stratified or adjusted by race/ethnicity, income level, and educational status where possible at a spatially appropriate resolution for this study in the form of spreadsheets or databases. Documentation of all methods, including the modeling tools, codes, and the methodology used to estimate health data. Adjustment factors are to be incorporated in CARB health benefits assessment methodology at the census tract level, which reflect the increased health effect in overburdened communities compared to other communities across California based on differences in baseline disease rates. Uncertainty analysis in the form of confidence interval estimates for the adjustment factors or the spatially refined baseline disease rates.

Task 5: Developing community adjustment factors

• Investigate the role of non-chemical stressors, such as socio-economic and community characteristics, in modifying the health impacts of air pollution. This includes developing a framework to incorporate variables such as income, access to healthcare, and existing health disparities in order to quantify the vulnerability of overburdened communities to air pollution health impacts. Predictors relevant to task 5 may include predictors for the outcome of task 4, so investigators should demonstrate a clear plan for identifying the interference of the predictors included in task 4 on community-level effects.
• Deliverables:
  • Adjustment factors that can be used in CARB health analysis methodology to quantify the vulnerability of overburdened communities to air pollution compared to other communities across California based on community characteristics, analyzed at the finest spatial resolution possible. Definition of vulnerability, estimation of community adjustment factors using socio-economic and environmental justice data, uncertainty analysis of the community adjustment factors, and statistical summaries of comparison of communities across California.

Task 6: Project Review and Completion

• Submit a draft final report to CARB summarizing methodologies developed to evaluate health outcomes and any potential limitations. The outcome of the study should include adjustment factors quantifying the differences in the health impact of overburdened communities compared to the average population based on the exposure, baseline disease rates, and community characteristics (specific adjustment factors for each component) and methodology on how to integrate these adjustment factors and implement them in the health impact assessment of CARB’s regulations to reflect the combined increased health risks in overburdened communities. The report should summarize how this project satisfies and addresses the following requirements and tasks in the pre-proposal:
  • Completion and delivery date(s) for each task.
  • Submission of quarterly progress reports, a draft final report, and a final report to CARB, which also includes the preparation of a lay summary of these reports for public dissemination.
  • Participation in progress update meetings and a seminar at the conclusion of the project.
  • Preparation and provision of all raw data, modeled data, and all data analysis results generated through the course of the project to CARB in electronic format.

Proposals shall detail the methods, data sources, and proposed spatial resolution intended to be used for each component of this project. The results of this study could be used to estimate the relative health impacts in overburdened communities across California resulting from air pollution control programs.

Deliverables

The project proposal must include but is not limited to the following deliverables:

At the Beginning of the Contract

• All researchers must undergo cultural competency training (examples include implicit bias training, racial equity training, etc.). Training should be completed or scheduled within 30 days of contract execution.

• Work with CARB staff at the beginning of the project to create a 1-page plain-language outreach deliverable for the public describing the project’s goals, process, and planned deliverables (available in multiple languages; template will be provided).
• A public kickoff meeting with the methodology of the project explained in lay-oriented terms.

During the Active Contract Period

• Quarterly Progress Reports and conference calls. The progress reports will include plain-language summaries that can be posted publicly. A progress report template will be provided.
• The investigators will work closely and collaboratively with CARB staff for all tasks.
• The investigators will have meetings with CARB staff (frequency of biweekly to monthly as agreed to by contractors and CARB).
• Provide interim milestones and dates for accomplishing tasks, submit quarterly progress reports, and participate in quarterly progress update meetings.

Interim Deliverables at the End of the First Year

• Results of the literature review including all the used keywords, journals, and the findings.
• Selected Communities for ground truthing, methodology for ground truthing, and results of community expert(s) consultations.
• Summary of data acquisition efforts for improving the spatial resolution of health data.
• Results of the emission modeling including all the input data and the spatial distribution of emissions.

Prior to Contract Close

• Submit a report outlining a step-by-step process, computer programs, all data (e.g., in Excel spreadsheets), analyses, and analytical tools generated through the course of this project. This includes all the results, codes, and intermediate data of air quality modeling, baseline disease rate modeling, and community factors modeling.
• Provide the adjustment factors quantifying the increased health risks in overburdened communities based on exposure to air pollution, baseline disease rates, and community characteristics compared to other communities across California (specific for each component), as well as the adjustment factors aggregated from all the aforementioned components and the methodology on how they can be incorporated into CARB’s methodology for health benefits assessment.
• Produce plain-language fact sheets and these will be translated into Spanish.
• Draft final report.
  • 6 months prior to contract close, provide a draft final report reviewed and approved by the Principal Investigator.
  • Include a plain language summary in draft final report.
  • Include an equity implications section in draft final report.
  • Must be copy-edited.
• Work with CARB to create plain-language outreach deliverables for the public summarizing results and the impact of the project (available in multiple languages).
• Submit the final report and conduct a plain-language public webinar or in-person seminar.
• Peer-reviewed publications should be publicly available (please budget for this expense; submission-ready publications shall be reviewed by CARB staff).
• An end-of-contract meeting with the results of the project explained in lay-oriented terms.
• Additional deliverables are to be determined in consultation with CARB staff.

Timeline

It is anticipated this project will be completed in 24 months from the start date. Cost shall not exceed $800,000.

References

1. CARB’s Methodology for Estimating the Health Effects of Air Pollution.Retrieved February 9, 20231, from https://ww2.arb.ca.gov/resources/documents/carbs-methodology-estimating-health-effects-air-pollution.
2. Carb’s Incidence Per Ton (IPT) methodology. Retrieved February 29, 2024, from https://ww3.arb.ca.gov/regact/2010/truckbus10/correctedappj.pdf.
3. Fann N, Roman HA, Fulcher CM, Gentile MA, Hubbell BJ, Wesson K, Levy JI. Maximizing health benefits and minimizing inequality: incorporating local-scale data in the design and evaluation of air quality policies. Risk Anal. 2011 Jun;31(6):908-22. doi: 10.1111/j.1539-6924.2011.01629.x. Epub 2011 May 26. PMID: 21615761.
4. Hubbell, B., Fann, N. & Levy, J.I. Methodological considerations in developing local-scale health impact assessments: balancing national, regional, and local data. Air Qual Atmos Health 2, 99–110 (2009). https://doi.org/10.1007/s11869-009-0037-z
5. Mercer L, Wakefield J, Pantazis A, Lutambi A, Mosanja H, and Clark S (2015). Small area estimation of childhood mortality in the absence of vital registration. Annals of Applied Statistics, 9, 1889–1905.
6. Coker, E., et al. Modeling spatial effects of PM2.5 on term low birth weight in Los Angeles County. Environmental Research,142 (2015). https://doi.org/10.1016/j.envres.2015.06.044.
7. Quiñones, S., Goyal, A. & Ahmed, Z.U. Geographically weighted machine learning model for untangling spatial heterogeneity of type 2 diabetes mellitus (T2D) prevalence in the USA. Sci Rep 11, 6955 (2021). https://doi.org/10.1038/s41598-021-85381-5
8. Shatas, A., & Hubbell, B. (2022). Using Low-Cost Sensor Networks: Considerations to Help Reveal Neighborhood-Level Exposure Disparities. American journal of public health, 112(12), 1693–1695. https://doi.org/10.2105/AJPH.2022.307128

Review Criteria

Responsiveness to the goals and objectives outlined in the proposal solicitation (15 points)

Proposers should demonstrate a clear understanding of the policy objectives and research needs that CARB seeks to address with this project and should convey their knowledge of the subject. The proposal should spell out, in adequate detail, exactly what the Proposer proposes to do to satisfy the requirements of the Solicitation. The draft proposal must propose work that would satisfy the objective(s) stated in the Research Solicitation: The objective of this study is to improve CARB’s health benefits analysis method by integrating high spatial resolution exposure assessment, finer-resolution health information, and community characteristics to provide an improved estimate of the health impacts faced by overburdened communities beyond current regional assessments.

Policy relevance/benefits to the state  (10 points)

Does the proposal describe how the project will provide data, information, and/or products to help CARB accomplish its mission? The results of this project will allow CARB to better estimate and inform the public of the health impacts of the agency’s air pollution control programs in communities across California, particularly in overburdened communities.

Previous work (15 points)

Do the researchers have relevant experience in this area? Is the team composed of a multidisciplinary team of experts? Do they discuss how they will build upon previous relevant work that was funded by CARB, other state agencies, and federal agencies such as US EPA, NSF, or NIH? If including community engagement, the relevant research partner should describe previous experience in community engagement and provide letters of support, references, or a community impact statement, describing how previous work impacted communities. Does the team bring in new talent that has not worked with CARB previously? Researchers new to CARB are encouraged to apply. Multidisciplinary teams and/or researchers from smaller universities are also encouraged to apply. 5 points will be reserved for project teams that meet at least one of the following criteria:

• The project team is multi-disciplinary
• The project team members come from various universities or include non-academic institutions or community-based organizations
• The project team includes one or more members, contributing significantly to the project (i.e., a principal investigator, co-principal investigator, or co-investigator, contributing 25% or more of their time to the project) who have not worked with CARB in the past 5 years.

Technical merit (25 points)

Describe the submission’s technical strengths and/or weaknesses. Proposers should demonstrate the logic and feasibility of the methodology and technical approach to the project, spell out the sequence and relationships of major tasks, and explain methods for performing the actual work. Please factor in how well the draft proposal describes these areas:

• Quantifying the community health effects of air pollution control programs using high spatial resolution exposure assessment methods and finer resolution health data (e.g., baseline disease rates).
• Multiple components that should be considered are:
  • An advanced exposure assessment framework with a high spatial resolution that is appropriate to investigate the health impact of air pollution for communities.
  • Methodology to refine the spatial resolution of health (baseline disease rates) data using statistical models (e.g., small area estimation).
  • Estimating community adjustment factors by using SE and EJ information. The community adjustment factors will be used by CARB to quantify the increased health risk in communities.
• Does the proposal include a detailed and feasible plan for ground-truthing exposure data?
• Rigorous and scientifically defensible study designs and methods
• Does the proposed work address all the deliverables required in the section “DELIVERABLES”? If not, the proposal should not be considered for funding.

Level and quality of effort to be provided (15 points)

Does the proposal allocate time and resources in such a way that the objectives of the study will be met? Is supervision and oversight adequate for ensuring that the project will remain on schedule? Is the distribution of workload appropriate for activities such as research, evaluation and analysis, data reduction, computer simulation, report preparation, meetings, and travel?

Cost effectiveness (20 points)

Does the cost seem appropriate for the proposed work? Does the proposed work seem feasible within the requested budget? Projects that provide co-funding should be evaluated more favorably.