The EPA has indicated a
pattern of inconsistent results in epidemiological time-series
studies that is troubling. The epidemiological evidence has played
a disproportionately large role in the policymaking process.
Time-series findings indicate associations of mortality with
not only PM and ozone, but with all of the criteria pollutants.
Because results of time-series studies implicate all of the criteria
pollutants, findings of mortality time-series studies do not
seem to allow one to confidently attribute observed effects specifically
to individual pollutants. This raises concern about the utility
of these types of studies in the current NAAQS-setting process.
In November 2005, the Technical
Team of scientists associated with A.S.L. & Associates reviewed
the second
draft of the Air Quality Criteria for Ozone and Related Photochemical
Oxidants Document and submitted its comments to CASAC. In December 2005, Dr. Lefohn presented
his conclusions to CASAC in Durham, North Carolina. Over 125 pages
of technical comments were provided to the U.S. EPA with specific
focus on policy-relevant background, epidemiological methodological
shortcomings, evidence for nonlinearity in human health studies,
and the importance of quantitatively characterizing peak exposures
for vegetation exposure indices. In addition to focusing on ozone,
the ASL Technical Team reviewed the August 2004 draft of Chapter 9 of the PM Criteria
Document and submitted its findings to U.S. EPA. The comments
can be read by accessing Dr.
Lefohn's and Dr. Switzer's reports. In the past, the Team has
pointed out several important technical issues and the ramifications
associated with the Agency embracing a proportional concentration-response
relationship in the epidemiological methodology. Many of the
statistical caveats raised throughout the epidemiology section
(Chapter 8) and Chapter 9 of the PM Criteria Document indicate
a pattern of inconsistent results that is troubling. Examples
of the growing pattern of inconsistent and inconclusive findings
include the following:
- Instability of PM mortality
effect estimates resulting from different model specifications
of weather effects and time trends.
- Instability of PM effect
estimates resulting from different selections of monitoring sites
within cities.
- Increased heterogeneity
of PM effect estimates across cities.
- Greater diversity of findings
among studies and across study areas.
- Contradictory results
from mortality displacement studies.
- PM effect lags that are
inconsistent across cities and across studies.
- Exposure-response relations
that are inconsistent across cities and across studies.
- Inconsistencies between
short-term and long-term effect studies, such as respiratory
effects of fine particles.
- Contradictory findings
among long-term studies.
Recently Smith et al. (2009)
discussed some of the concerns about the use of time-series data.
The authors investigated intercity variability, as well as the
sensitivity of the ozone-mortality associations to modeling assumptions
and choice of daily ozone metric, based on reanalysis of NMMAPS
data. Smith et al. (2009) examined the sensitivity of city-specific
ozone-mortality estimates to adjustments for confounders and
effect modifiers, showing substantial sensitivity. They examined
ozone-mortality associations in different concentration ranges,
finding a larger incremental effect in higher ranges, but also
larger uncertainty. Alternative ozone exposure metrics defined
by maximum 8-h averages or 1-h maxima show different ozone-mortality
associations that the authors believed could not be explained
by simple scaling relationships. The authors' view is that ozone-mortality
associations, based on time-series epidemiologic analyses of
daily data from multiple cities, reveal still-unexplained inconsistencies
and show sensitivity to modeling choices and data selection that
contribute to serious uncertainties when epidemiological results
are used to discern the nature and magnitude of possible ozone-mortality
relationships or are applied to risk assessment.
Personal exposure is not
reflected adequately, and sometimes not at all, by concentrations
measured at central outdoor monitoring sites. Typically, personal
exposures are much lower than the ambient concentrations, and
can be dramatically lower depending on time-activity patterns,
housing characteristics and season. In addition, and of particular
importance for the time-series studies, there can be no correlation
between personal concentrations measured over time and concentrations
measured at central outdoor sites.
Previous review comments
made by the Team were associated with spatial variability and
the statistical shortcomings associated with epidemiological
analyses. Dr. Paul Switzer's earlier comments on some of the
shortcomings associated with the epidemiological findings can
be read by clicking here.
On July 18, 2002, Dr. Lefohn,
President of A.S.L. & Associates, summarized the group's
findings on the third external review at the EPA's Clean Air Scientific
Advisory Committee (CASAC) meeting in North Carolina. Dr. Paul Switzer, a member of
the technical team, provided written comments on his concerns
about the shortcomings of the current statistical methodology
utilized in the time-series epidemiological analyses. His July
8, 2002 comments can be reviewed by clicking here. In the review of the first and second
drafts of the Particulate Matter Criteria Document, the technical
team noted several key limitations associated with the methodologies
employed in the epidemiological studies.
On May 30, 2002 the EPA
was informed by the Health Effects Institute (HEI) of a generally
unappreciated aspect in the use of S-Plus statistical software
often employed to fit generalized additive models (GAM) to data
in time-series analyses. Additional concerns have been identified
since that time. More detailed information concerning the possible
changes in relative risk estimates previously published in the
peer-review literature is discussed in an Adobe Acrobat PDF file. As a result of these concerns, the
National Center For Environmental Assessment - RTP Division (NCEA/RTP),
within US EPA's Office of Research and Development ORD), hosted
a Workshop on GAM-Related Statistical Issues in PM Epidemiology
that was held on November 4-6, 2002. The workshop was held to
provide a forum for discussion of: (a) newly identified issues
related to the conduct of General Additive Model (GAM) analyses,
using commercially-available software packages (e.g., S-plus
or SAS), in time-series studies of relationships between ambient
air particulate matter (PM) and mortality/morbidity endpoints
(e.g., daily deaths, hospital admissions, etc); (b) progress
made to date in the conduct of reanalyses of a group of GAM-related
studies identified by EPA as being of high priority for policy
considerations; and (c) additional considerations for future
directions for new PM epidemiologic analyses. The workshop agenda can be downloaded.
In November 2001, Dr. Lefohn
was invited by the National Research Council to participate in
a panel discussion at the Workshop of the National Research Council Committee
on Research Priorities for Airborne Particulate Matter. The purpose of the workshop was
to discuss research progress in exposure assessment. In July
2001, Dr. Lefohn outlined a series of concerns to EPA's Clean
Air Scientific Advisory Committee (CASAC) during his oral testimony.
A peer-review paper is being prepared that describes the limitations
of the statistical methodology. In March 2001, several scientists,
including Dr. Lefohn, commented on specific concerns associated
with a paper authored by Wilson et al. (2000). The set
of comments and responses can be found in the J. Air &
Waste Manage. Assoc., 2001, 51:322-338. It is interesting
reading.
Previous to the review
of the external review draft documents for particulate matter,
Professor Paul Switzer (Stanford University) and Dr. Allen S.
Lefohn (A.S.L. & Associates) provided input to the EPA prior
to the publication of the final version of the Carbon Monoxide
Criteria Document (CD). A summary of our input is available in
an Adobe Acrobat
PDF file.
An alternative to the cause-and-effect
explanation provided in Chapter 9 of the PM CD is that the results
that are cited may be mostly associated with modelling artifacts.
One is left with answering a serious question: If the time-series
data are the most important information available for establishing
Federal PM standards, are the data good enough to use in the
decision-making process? Based on the evidence presented in Chapters
8 and 9 in the PM CD, one simply cannot draw comfortable conclusions
regarding the circumstances and magnitudes of ambient PM health
effects, or whether reported PM health effects are causative.
There is still much uncertainty remaining in the epidemiological
time-series results and many of the concerns expressed by the
EPA in the Carbon Monoxide CD about the strengths and limitations
of the extensive body of epidemiologic evidence of associations
between health effects and air pollutants have not been adequately
addressed in either the Ozone or PM Criteria Documents. The growing
pattern of inconsistent and inconclusive findings is troublesome
and presents both scientists and policymakers with a very difficult
decision. Simply stated, the science based on epidemiological
results is not substantial enough at the moment to provide the
foundation upon which a clear path can be built that leads directly
from the science to the policymaking decision arena.
Reference
Smith, R.L., Xu, B., and Switzer,
P. (2009). Reassessing the relationship between ozone and short-term
mortality in U.S. urban communities. Inhalation Toxicology, 29(S2):
37–61.
