Return to Home Page

Epidemiological Concerns

The EPA has indicated a pattern of inconsistent results in epidemiological time-series studies that is troubling. The epidemiological evidence has played a disproportionately large role in the policymaking process. Time-series findings indicate associations of mortality with not only PM and ozone, but with all of the criteria pollutants. Because results of time-series studies implicate all of the criteria pollutants, findings of mortality time-series studies do not seem to allow one to confidently attribute observed effects specifically to individual pollutants. This raises concern about the utility of these types of studies in the current NAAQS-setting process.

In November 2005, the Technical Team of scientists associated with A.S.L. & Associates reviewed the second draft of the Air Quality Criteria for Ozone and Related Photochemical Oxidants Document and submitted its comments to CASAC. In December 2005, Dr. Lefohn presented his conclusions to CASAC in Durham, North Carolina. Over 125 pages of technical comments were provided to the U.S. EPA with specific focus on policy-relevant background, epidemiological methodological shortcomings, evidence for nonlinearity in human health studies, and the importance of quantitatively characterizing peak exposures for vegetation exposure indices. In addition to focusing on ozone, the ASL Technical Team reviewed the August 2004 draft of Chapter 9 of the PM Criteria Document and submitted its findings to U.S. EPA. The comments can be read by accessing Dr. Lefohn's and Dr. Switzer's reports. In the past, the Team has pointed out several important technical issues and the ramifications associated with the Agency embracing a proportional concentration-response relationship in the epidemiological methodology. Many of the statistical caveats raised throughout the epidemiology section (Chapter 8) and Chapter 9 of the PM Criteria Document indicate a pattern of inconsistent results that is troubling. Examples of the growing pattern of inconsistent and inconclusive findings include the following:

  • Instability of PM mortality effect estimates resulting from different model specifications of weather effects and time trends.
  • Instability of PM effect estimates resulting from different selections of monitoring sites within cities.
  • Increased heterogeneity of PM effect estimates across cities.
  • Greater diversity of findings among studies and across study areas.
  • Contradictory results from mortality displacement studies.
  • PM effect lags that are inconsistent across cities and across studies.
  • Exposure-response relations that are inconsistent across cities and across studies.
  • Inconsistencies between short-term and long-term effect studies, such as respiratory effects of fine particles.
  • Contradictory findings among long-term studies.

Recently Smith et al. (2009) discussed some of the concerns about the use of time-series data. The authors investigated intercity variability, as well as the sensitivity of the ozone-mortality associations to modeling assumptions and choice of daily ozone metric, based on reanalysis of NMMAPS data. Smith et al. (2009) examined the sensitivity of city-specific ozone-mortality estimates to adjustments for confounders and effect modifiers, showing substantial sensitivity. They examined ozone-mortality associations in different concentration ranges, finding a larger incremental effect in higher ranges, but also larger uncertainty. Alternative ozone exposure metrics defined by maximum 8-h averages or 1-h maxima show different ozone-mortality associations that the authors believed could not be explained by simple scaling relationships. The authors' view is that ozone-mortality associations, based on time-series epidemiologic analyses of daily data from multiple cities, reveal still-unexplained inconsistencies and show sensitivity to modeling choices and data selection that contribute to serious uncertainties when epidemiological results are used to discern the nature and magnitude of possible ozone-mortality relationships or are applied to risk assessment.

Personal exposure is not reflected adequately, and sometimes not at all, by concentrations measured at central outdoor monitoring sites. Typically, personal exposures are much lower than the ambient concentrations, and can be dramatically lower depending on time-activity patterns, housing characteristics and season. In addition, and of particular importance for the time-series studies, there can be no correlation between personal concentrations measured over time and concentrations measured at central outdoor sites.

Previous review comments made by the Team were associated with spatial variability and the statistical shortcomings associated with epidemiological analyses. Dr. Paul Switzer's earlier comments on some of the shortcomings associated with the epidemiological findings can be read by clicking here.

On July 18, 2002, Dr. Lefohn, President of A.S.L. & Associates, summarized the group's findings on the third external review at the EPA's Clean Air Scientific Advisory Committee (CASAC) meeting in North Carolina. Dr. Paul Switzer, a member of the technical team, provided written comments on his concerns about the shortcomings of the current statistical methodology utilized in the time-series epidemiological analyses. His July 8, 2002 comments can be reviewed by clicking here. In the review of the first and second drafts of the Particulate Matter Criteria Document, the technical team noted several key limitations associated with the methodologies employed in the epidemiological studies.

On May 30, 2002 the EPA was informed by the Health Effects Institute (HEI) of a generally unappreciated aspect in the use of S-Plus statistical software often employed to fit generalized additive models (GAM) to data in time-series analyses. Additional concerns have been identified since that time. More detailed information concerning the possible changes in relative risk estimates previously published in the peer-review literature is discussed in an Adobe Acrobat PDF file. As a result of these concerns, the National Center For Environmental Assessment - RTP Division (NCEA/RTP), within US EPA's Office of Research and Development ORD), hosted a Workshop on GAM-Related Statistical Issues in PM Epidemiology that was held on November 4-6, 2002. The workshop was held to provide a forum for discussion of: (a) newly identified issues related to the conduct of General Additive Model (GAM) analyses, using commercially-available software packages (e.g., S-plus or SAS), in time-series studies of relationships between ambient air particulate matter (PM) and mortality/morbidity endpoints (e.g., daily deaths, hospital admissions, etc); (b) progress made to date in the conduct of reanalyses of a group of GAM-related studies identified by EPA as being of high priority for policy considerations; and (c) additional considerations for future directions for new PM epidemiologic analyses. The workshop agenda can be downloaded.

In November 2001, Dr. Lefohn was invited by the National Research Council to participate in a panel discussion at the Workshop of the National Research Council Committee on Research Priorities for Airborne Particulate Matter. The purpose of the workshop was to discuss research progress in exposure assessment. In July 2001, Dr. Lefohn outlined a series of concerns to EPA's Clean Air Scientific Advisory Committee (CASAC) during his oral testimony. A peer-review paper is being prepared that describes the limitations of the statistical methodology. In March 2001, several scientists, including Dr. Lefohn, commented on specific concerns associated with a paper authored by Wilson et al. (2000). The set of comments and responses can be found in the J. Air & Waste Manage. Assoc., 2001, 51:322-338. It is interesting reading.

Previous to the review of the external review draft documents for particulate matter, Professor Paul Switzer (Stanford University) and Dr. Allen S. Lefohn (A.S.L. & Associates) provided input to the EPA prior to the publication of the final version of the Carbon Monoxide Criteria Document (CD). A summary of our input is available in an Adobe Acrobat PDF file.

An alternative to the cause-and-effect explanation provided in Chapter 9 of the PM CD is that the results that are cited may be mostly associated with modelling artifacts. One is left with answering a serious question: If the time-series data are the most important information available for establishing Federal PM standards, are the data good enough to use in the decision-making process? Based on the evidence presented in Chapters 8 and 9 in the PM CD, one simply cannot draw comfortable conclusions regarding the circumstances and magnitudes of ambient PM health effects, or whether reported PM health effects are causative. There is still much uncertainty remaining in the epidemiological time-series results and many of the concerns expressed by the EPA in the Carbon Monoxide CD about the strengths and limitations of the extensive body of epidemiologic evidence of associations between health effects and air pollutants have not been adequately addressed in either the Ozone or PM Criteria Documents. The growing pattern of inconsistent and inconclusive findings is troublesome and presents both scientists and policymakers with a very difficult decision. Simply stated, the science based on epidemiological results is not substantial enough at the moment to provide the foundation upon which a clear path can be built that leads directly from the science to the policymaking decision arena.

It its 2013-2015 review by EPA of the national ambient ozone standard, the science based on epidemiological results did not provide strong support for reducing the current level of the federal ozone standard in the US (FR Vol. 79, No. 242). In November 2015, the EPA Administrator concluded that the epidemiological risk analyses showed that small net benefits resulted from changing the ozone standard from its current level of 75 ppb to lower values (70, 65, or 60 ppb). EPA (2014) provides the details supporting this observation by the Administrator. The Administrator's conclusion is based on the observation that because the short-term epidemiological risk analyses integrated from the maximum concentration all the way down to zero concentration, minimum benefits occurred. As emission reduction occur to meet lower proposed ozone standards, the lower concentrations begin to rise (due to lack of NOx scavenging) and the epidemiological models predict that additional morality and morbidity may occur. Although benefits occur as the peak ozone concentrations are reduced, this benefit is greatly neutralized by the rise in predicted mortality and morbitidy due to the low end of the concentration distribution rising. The epidemiologists ignored the scientific observations reported in the literature (e.g., Hazucha and Lefohn, 2007; Lefohn et al., 2010) that the higher concentrations should be provided greater weight than the lower concentrations of which many are in the background range (i.e., 25-55 ppb) (see Lefohn et al., 2014). Additional discussion of the lack of NOx scavenging affecting the movement of the lower hourly average ozone concentrations toward the mid-levels can be found in Lefohn et al. (2016).


Federal Register / Vol. 79, No. 242 / Wednesday, December 17, 2014 / Proposed Rules, page 75234-75411.

Hazucha, M. J.; Lefohn, A. S. (2007) Nonlinearity in Human Health Response to Ozone: Experimental Laboratory Considerations. Atmospheric Environment. 41:4559-4570.

Lefohn, A.S., Hazucha, M.J., Shadwick, D., Adams, W.C. (2010). An Alternative Form and Level of the Human Health Ozone Standard. Inhalation Toxicology. 22: 999-1011.

Lefohn, A.S., Emery, C., Shadwick, D., Wernli, H., Jung, J., Oltmans, S.J. (2014). Estimates of Background Surface Ozone Concentrations in the United States Based on Model-Derived Source Apportionment. Atmospheric Environment, 84: 275-288.

Lefohn, A.S., Malley, C.S., Simon, H., Wells. B., Xu, X., Zhang, L., Wang, T., 2016. Responses of human health and vegetation exposure metrics to changes in ozone concentration distributions in the European Union, United States, and China. Atmospheric Environment 152: 123-145. doi:10.1016/j.atmosenv.2016.12.025.

Smith, R.L., Xu, B., and Switzer, P. (2009). Reassessing the relationship between ozone and short-term mortality in U.S. urban communities. Inhalation Toxicology, 29(S2): 37–61.

US Environmental Protection Agency, US EPA. (2014). Health Risk and Exposure Assessment for Ozone. EPA/452/R-14-004a. Research Triangle Park, NC: Office of Air Quality Planning and Standards. August.

Home Page | News | Corporation | Maps | Publications | Table of Contents | Multimedia Center

Copyright © 1995-2017 A.S.L. & Associates. All rights reserved.