California Air Resources Board

Scope of Work

Background

The Bayview-Hunters Point (BVHP) neighborhood is located in the southeast area of San Francisco and is home to the largest Black community in San Francisco. BVHP struggles with multiple sources of pollution exposure, a high poverty rate, high unemployment and housing cost burdens, and some of the lowest educational attainment.  Most of the land is zoned for industrial use and is also the location of numerous hazardous material facilities and waste sites. The BVHP area also encompasses the Hunters Point Naval Shipyard, the only federal Superfund site located in San Francisco. In comparison to other neighborhoods in San Francisco, the community suffers from significant health disparities that many residents, advocates, and medical experts attribute to toxic exposure. Efforts to address health and racial disparities in Bayview-Hunters Point include community organizing, advocacy for affordable housing and economic development, initiatives to improve educational outcomes, healthcare access, and environmental justice advocacy. However, systemic challenges rooted in historical inequities require sustained and comprehensive efforts from various stakeholders to achieve meaningful change and promote equity in the neighborhood. Residents of this community have lower life expectancies and higher mortality rates from lung diseases, which can be partially attributed to constant exposure to air pollution. This community has a higher rate of asthma-related emergency room visits and cardiovascular disease than most of California. These health disparities are detailed in the Community Health Needs Assessment (SF CHNA) developed by San Francisco Health Improvement Partnership https://sfhip.org/chna/sf-chna/.

Bayview Hunters Point community groups have been especially concerned about legacy pollution from the Naval Shipyard, dust and asbestos from on-going large-scale redevelopment, odors and emissions from a wastewater treatment facility, diesel truck idling, and from industrial rendering plants. The community is bisected by the commercial-oriented Third Street corridor and straddles two busy freeways (Interstate- 280 and Interstate-101) which bring freight trucks and high-volume commuter traffic adding to the mobile source pollution burdens.  These pollutants also contribute to poor indoor air quality in BVHP, and community leaders have expressed their concerns about the need to improve IAQ for area residents.   

Objectives

Commercially available, consumer-friendly monitoring devices and portable air cleaners provide the technical potential to improve air quality in California homes. The objective of this proposal is to evaluate the impact of air cleaning devices and air quality monitoring on indoor air quality for residents.

Residents will be given high efficiency, portable air cleaners (HEPA) and low-cost portable indoor air sensors to detect and record indoor PM2.5 levels and possibly other pollutants. Residents will be trained in using the equipment to improve their indoor air quality and they will be surveyed throughout study to qualitatively assess whether they felt if there was an improvement in their respiratory health.  Quantitative data will also be gathered to compare changes in indoor air quality in homes using a low-cost sensor. Both the sensors and air cleaners will remain with residents after the conclusion of the project.   A part of this project will also include community engagement with the goal of incorporating the knowledge of the BVHP residents into current and future research. Community members have information regarding sources of air pollution, circumstances that lead to elevated air pollution in the area, gaps in air pollution research, and health burdens related to air pollution exposure that are unique to their community.

Scope of Work

Accurate estimation of exposures to air pollutants in impacted communities like BVHP, and determining what methods are effective in reducing those exposures, is essential to achieving the goals of AB 617. It is known that when used properly, portable air cleaners are effective in lowering indoor PM levels; however, less is known about how to maximize efficacy.  To address the study objectives, the investigator will complete the following tasks and include milestones and deliverables for the tasks in the project proposal:

Task 1: Community Outreach and Engagement Plan

An important component of this project is community engagement and input during all phases of the project. This includes aspects of study design, participate recruitment, and dissemination of results.  Task 1 should incorporate the following:

• Develop initial plans for reaching out to community planning groups, local or regional government representatives including representatives from health and air pollution agencies, and other stakeholders to establish relationships with stakeholders.
• Develop the process for forming an advisory group comprised of local residents and representatives of community groups in the BV-Hunters Point area. This advisory group will be charged with identifying a community partner(s), reviewing the proposed research and to provide meaningful contributions on study methods, goals and the identification of community-supported research objectives.
• Provide a plan to engage community partners, study participants, and stakeholders throughout the study. This process may include the development of an interactive website and/or the use of social media, as well as the use of email distribution lists and newsletters to maintain relationships and get feedback from key stakeholders. Community partners and study participants are to be compensated for their time, and this must be included in the budget. Community partner(s) will be subcontractors on the project and they will be required to provide a detailed budget and justifications.
• Note:For community engagement efforts the contractor must co-create meeting materials, including presentation slides, flyers, prompts and speaking notes with CARB staff. The contractor will work with CARB to understand policies and agree to accurately represent those policies or defer for follow-up. CARB will participate in community meetings unless mutually agreed upon with CARB and the PI in cases where it could impact community engagement efforts negatively.

Task 2: Literature Review

The literature review should provide information which is currently known regarding pollution levels and sources in the BVHP community, results from similar intervention studies or model studies, and the impact of expected indoor sources. This includes but is not limited to the following:

• A review of the pertinent literature about the pollution issues in the BVHP community, primarily focused on particulate matter (PM) from commuter traffic and diesel PM, construction/demolition dust, and potential PM sources from the port.
• A summary of previous studies focusing on interventions to reduce ambient PM concentrations, and interventions to reduce indoor exposures and the relative strengths and weaknesses of the interventions.
  • This can include studies conducted in other communities and also those conducted under controlled conditions such as a laboratory or a model residence,
• A summary of information on the primary sources of indoor PM (e.g. cooking, smoking, heating) and the historical context of those sources should also be included.      

Task 3: Questionnaire Development for collection of personal data and home characteristics that may impact indoor air quality

Information from these questionnaires will be used to characterize demographic and health information from the participants in the study and to identify confounding factors that could affect the health benefits of the air cleaner intervention and the health self-assessments.

For this task in the proposal, the investigator should:

• Develop a questionnaire for the collection of personal data which may include but not limited to:
  • Demographic information such as age, income, zip code, ethnicity, education, number and age of children in home, pets, years at current residence etc.
  • Health information such as smoking status, asthma symptoms, COPD, allergies, medication use, hospital visits, school days missed, etc.
  • Self-assessment of health effects such as respiratory symptom severity pre and post intervention. Participants will qualitatively assess whether they felt an improvement in their respiratory health when using air cleaning technology to reduce PM exposure.

In addition, the investigator should identify and address any issues that may arise via the collection of such data including the need for possible institutional review board (IRB) approvals.

• Include collection of factors which may impact indoor air quality by designing surveys for gathering data on home characteristics (type, size, age), HVAC type and age, window usage patterns, appliances (gas/electric), air cleaner use, flooring, use of cleaning products, cosmetics, and pesticides, mold or other potential confounding factors for the study.
• Determine how the investigator plans to involve the advisory group in this task

Task 4: Participant selection and recruitment

This task is to determine the method for recruitment and may be refined based upon consultation with the community advisory group.  Participants will be selected from within the community and the number of participants must be no fewer than 100. The study must be adequately powered so that significant differences can be detected between the baseline and the intervention conditions of the study, but also the budget maximized to allow the greatest number of air cleaners and low-cost sensors given to participants. For this task, the investigator will work in consultation with the community advisory group to:

• Develop plans for identifying and engaging with specific community partners to recruit study participants and residences for indoor air quality monitoring, intervention set up,
• Assess and discuss necessary participant sample size to provide statistically relevant results
• Discuss and decide participant selection criteria or how such criteria will be determined and outline the scope of recruitment with the help of community partners
• Development of recruitment strategies with community partners and how recruitment will be implemented

Task 5: Air Cleaner Intervention and Field Monitoring

This task will determine the methodology for the design and implementation of the field study including setup of monitoring equipment and air cleaning devices as well as data and survey collection. Each participant in the study will retain the air cleaner and low-cost sensor. For this task the investigator should:

• Identify and justify the methodology for determining an “average” indoor and outdoor air quality level over a selected time period (30 or 60 days for example) that can be used to compare changes in indoor and outdoor air quality before and after the air filtration interventions.
• Identify factors which will be used to determine which air monitoring devices will be considered for measurement of indoor air quality.  If known, identify possible air monitoring devices that may be used for this study and the reason for their selection.
• Discuss and develop possible general protocols for field monitoring by the investigator’s team and/or community partner staff including but not limited to training, equipment set up, data monitoring, survey collection. 

Task 6: Modeling and Analysis

Data obtained from the previous tasks will be analyzed to assess possible levels of exposure reduction and health improvements.  This task should include:

• Analyzing results from field monitoring and exposure modeling to estimate the PM2.5 exposure reduction.
• Conducting statistical analysis to determine whether there is a significant difference in PM2.5 reduction when participants are provided with air cleaners and indoor air quality sensors versus just air cleaners alone.  
• Estimate the health impacts based on self-reported respiratory health symptoms and estimate health impacts due to cumulative exposure reductions to indoor PM2.5.

Task 7: Reporting

The contractors will develop a final report incorporating the results of the study. The results of this study will be used to assess the exposure patterns for the study participants living in vulnerable communities and identify the primary sources or impacting factors responsible for their disparate pollutant exposures experienced by those in vulnerable communities. Information from this study may also be used to provide recommendations for mitigation of exposures, and the most cost-effective ways to improve IAQ. To ensure that information for this study is disseminated widely to stakeholders of this project, the proposal should include for this task a timeline and brief statement on the preparation of the deliverables listed below in the “deliverables section”

Possible Additional Research Objectives

Depending on the budget, other components that could potentially be part of the study include:

• Collecting quantitative health data beyond surveys for respiratory disease and cardiovascular disease.
• Investigating other factors such as location relative to major air pollution sources and building information on indoor air quality
• Impact of interventions such as active messaging that could alert residents about poor air quality using texts or mobile app notifications.
• Comparisons of indoor and outdoor air quality data from existing sensor networks to estimate exposure differences between BVHP and neighboring communities that were not identified under AB 617.
• Including assessment and health impacts of additional indoor pollutants
• Any additional components suggested by the researcher that would investigate strategies to improve IAQ in vulnerable neighborhoods.

Deliverables

The project proposal must include but is not limited to the following deliverables:

At Pre-Proposal Stage

• Provide a cultural competency statement in the pre-proposal.
• Provide a community engagement plan in the pre-proposal.

At the Beginning of the Contract

• All researchers must undergo cultural competency training (examples include implicit bias training, racial equity training, etc.). Training should be completed or scheduled within 30 days of contract execution.
• Work with CARB staff at the beginning of the project to create a 1-page plain-language outreach deliverable for the public describing the project’s goals, process, and planned deliverables (available in multiple languages; template will be provided).
• A public kickoff meeting with the methodology of the project explained in lay-oriented terms.

During the Active Contract Period

• Quarterly Progress Reports and conference calls. The progress reports will include plain-language summaries that can be posted publicly. A progress report template will be provided.
• The investigators will work closely and collaboratively with CARB staff for all tasks.
• Community engagement materials: flyers, presentation slides, prompts and speaking notes shall be co-created with CARB input.
• The investigators will have meetings with CARB staff (frequency of biweekly to monthly as agreed to by contractors and CARB).
• Provide interim milestones and dates for accomplishing tasks, submit quarterly progress reports, and participate in quarterly progress update meetings.

Interim Deliverables at the End of the First Year

• Community engagement and outreach plan
• Contracts with community advisory group and study participantsResults of the literature review including all the used keywords, journals, and the findings
• Summary of recruitment progress questionnaire development and preliminary results from questionnaires

Additional Interim Deliverables should be included in the proposal

Prior to Contract Close

• All measurement and modeling results, including any potential health effects and potential health benefits of reduced exposure.
• Indoor and outdoor pollution measurements, I/O ratios, infiltration rates (if applicable), differences in filtration efficacy for participants with and without IAQ sensors, accounting of confounding variables
• Final report to include discussion of indoor air quality, best strategies to improve IAQ, implications for impacted communities.
• Recommendations for cost-effective means to improve IAQ and health, if indicated.
• Produce plain-language fact sheets which will also be translated into Spanish
• Draft final report
  • 6 months prior to contract close, provide a draft final report reviewed and approved by the Principal Investigator
  • Include a plain language summary in draft final report
  • Include an equity implications section in draft final report
  • Must be copy-edited
• Work with CARB to create plain-language outreach deliverables for the public summarizing results and the impact of the project (available in multiple languages).
• Submit the final report and conduct a plain-language public webinar or in-person seminar.
• Peer-reviewed publications should be publicly available (please budget for this expense; submission-ready publications shall be reviewed by CARB staff).
• An end-of-contract meeting with the results of the project explained in lay-oriented terms.
• CARB research seminar to disseminate findings and report to community partners.
• Additional deliverables are to be determined in consultation with CARB staff.

Timeline

It is anticipated this project will be completed in 24 months from the start date. Cost shall not exceed $850,000.

Scoring Criteria

Responsiveness to the Goals and Objectives Outlined in the Proposal Solicitation (15 points).

Proposers should demonstrate a clear understanding of the policy objectives and research needs that CARB seeks to address with this project and should convey their knowledge of the subject. The proposal should provide adequate details of exactly what the Proposer will do to satisfy the requirements of the Solicitation. The proposal must include work that satisfies the research objectives stated in the Research Solicitation. It must leverage on-the-ground knowledge of BVHP’s air quality issues to be responsive to people’s voices (i.e., the community) by achieving active participation and receiving meaningful guidance from local community members and community organizations.

Policy Relevance/Benefits to the State (10 points)

Does the proposal describe how the project will provide data, information, and/or products to help CARB accomplish its mission? Through this project, CARB seeks to better understand the long-standing air quality concerns and unique perspectives and ideas of the community to develop meaningful solutions and actions that improves the lives of people living in the BVHP community. How comprehensive is the conceptual community engagement plan? How are equity requirements highlighted and leveraged in the proposal? How synergistic will the community-centric strategic research plan be with CARB’s Triennial Strategic Research Plan? How effective will the proposal interconnect CARB’s existing air pollution control and mitigation strategies with the project’s objectives?

Previous Work (15 points)

1. Policy expertise (5 points): Do the researchers have relevant experience in this area? Is the team composed of a multidisciplinary team of experts? Do they discuss how they will build upon previous relevant work that was funded by CARB and other state agencies? Researchers should clearly describe the team’s level of experience with community engagement and coordination. The relevant research partner(s) should describe previous experience in community engagement and provide letters of support, references, or a community impact statement, describing how previous work impacted communities. The researchers should provide any completed projects that can evaluated as examples of community engagement.
2. Racial equity expertise (5 points): Does the team commit to cultural competency training (e.g. implicit bias training, racial equity training) as a deliverable and provide a cultural competency statement in the proposal? Does the research team have experience creating, using, or summarizing community recommendations to inform strategic research and/or policies? The researchers should have at least one completed project that can be provided as examples of community engagement.
3. Technical Expertise (5 points): The PI should have demonstrated technical experience with community air monitoring and indoor air quality research.  This should include experience in filtration interventions to improve indoor air quality and expertise in the use of low-cost sensors. 

Expanding Expertise (10 points)

Is the team composed of a multidisciplinary team of experts? Researchers are required to show intent to partner with non-academic partners, such as non-profits or community advocates in BVHP who have relevant experience in this area.

Technical Merit (20 points)

Describe the submission's technical strengths and/or weaknesses. Proposers should demonstrate the logic and feasibility of the methodology and technical approach to the project, spell out the sequence and relationships of major tasks, and explain methods for performing the actual work. Please factor in how well the draft proposal describes these areas.

1. How comprehensive are the approaches used to address each of the tasks?
   1. Community engagement planning
   2. Information gathering and interpretation (literature review)
   3. The use or development of validated research questionnaires and effective participant recruitment from previous studies 
   4. A well-developed study design and clear objectives for the air cleaner intervention and field monitoring components.  This should include a modeling and data analysis plan.
   5. Reporting results to CARB and communicating results to diverse stakeholders.
2. How well does the proposal reflect community engagement needs? How well does the proposal interconnect community engagement and the greater objective of the project?

Level and Quality of Effort to be Provided (15 points)

Does the proposal allocate time and resources in such a way that the objectives of the study will be met? Is supervision and oversight adequate for ensuring that the project will remain on schedule?  Do community partners have the capabilities to support the needs of the PI, and do they have a demonstrated record of success with previous studies? Is the distribution of workload appropriate for activities such as community engagement, research, evaluation and analysis, report preparation, meetings, and travel? The proposal should include a resource and work plan with sufficient detail to understand the work to be completed for this proposal.

Cost Effectiveness (20 points)

Does the cost seem appropriate for the proposed work?  Does the proposed work seem feasible within the requested budget?  Projects that provide co-funding should be evaluated more favorably.