interest. Your task as reviewer is to decide whether the
level of uncertainty, imposed by sample size and by the
inherent variability of the data, is low enough to make
point estimates credible.
● Look for a 95% CI. All main results deemed important by
the authors should have a 95% CI in addition to the point
estimate. The “confidence” that can be placed in results
that lack a 95% CI is difficult to determine unless you
calculate it yourself using a statistical program or one of
many electronic calculators readily available on the In-
ternet.
● Interpret a “positive result” with the 95% CI lower limit.
When the authors conclude that an outcome or group
difference is statistically significant or clinically impor-
tant, scrutinize the lower limit of the 95% CI. If the
magnitude of effect at the low end is consistent with a
trivial or nonimportant outcome, not enough subjects
were studied to create credible confidence that the point
estimate is meaningful.
● Interpret a “negative result” with the 95% CI upper limit.
When the authors conclude that an outcome or group
difference is not significant or important, scrutinize the
upper limit of the 95% CI. If the magnitude of effect at
the high end is consistent with a nontrivial or important
outcome, not enough subjects were studied to ensure that
a potentially important effect was overlooked (low statis-
tical power).
Understanding confidence intervals makes clear why de-
scribing study results as simply statistically “significant” or
“nonsignificant” is unacceptable.
21
A P value measures
strength of evidence against the null hypothesis but offers
no information on effect size. A P value approaching 0.05
often has precision that is too low to exclude a trivial effect,
and a “nonsignificant” P value often has an associated 95%
CI that contains clinically important sample means. Studies
with narrow CIs have high precision and are most mean-
ingful, regardless of the P values. Conversely, studies with
broad CIs require careful scrutiny.
22
Level of Evidence
Science is a cumulative process, and new research should
improve the knowledge base beyond what has already been
published. Viewed in this context, there is no absolute level
of “best” evidence, only a continuing effort for quality
improvement. If current knowledge of treatment effects
were limited to case series (Table 8), then a case-control or
cohort study would be a welcome addition. Conversely, if
high-quality randomized trials had already been published,
then observational studies (level 2, 3, 4) would be unlikely
to add any new insights. Additional studies within an exist-
ing level of best evidence are worth publishing if they equal,
or exceed, the quality of existing work.
The level of evidence generally increases as we progress
from uncontrolled observational studies (case reports, case
series) to controlled observational studies (cross sectional,
retrospective, prospective) to controlled experiments (ran-
domized trials). Levels of research evidence are most often
applied to studies of therapy or prevention, but can also be
defined for prognosis, symptom prevalence, and diagnostic
test assessment (Table 8).
23
Reviewers should determine the
level of evidence for the manuscript under review and how
it compares with other related published work before offer-
ing a recommendation to reject, revise, or accept the sub-
mission.
As the lowest level of clinical evidence, case reports are
suitable for publication (assuming the journal accepts this
type of submission) if they provide important new informa-
tion that offers understanding or management of a disorder.
Examples include a unique or nearly unique case, an unex-
pected association of two or more diseases or disorders, an
important variation from the expected pattern (outlier case),
or a case that reports unexpected outcomes or adverse
events.
24
Conversely, minor case reports not worth publish-
ing include variations of a well known theme, the “high
index of suspicion” or “everyone should remember” case,
grand rounds presentations with literature reviews, and bi-
zarre events that fail to impact future management.
Case reports worth publishing should not be assessed
with the usual criteria of internal validity and generalizabil-
ity, but instead should be checked for structure and clarity of
presentation. A recommended structure for case reports is as
follows:
24
1. Introduction: one paragraph stating how the case came to
the author’s attention, why the case is worth reading
about, and what the main features are;
2. Case description: a full account of the case, usually
chronologic, with only truly relevant data;
3. Discussion: details of the literature search, why the case
is unique or unexpected, potential explanations for the
findings, and implications for clinicians.
Ethical Conduct
Ethical behavior in the conduct and reporting of research is
an essential part of biomedical publishing. Although the
editor and editorial staff have primary responsibility for
assessing manuscripts in this regard, peer reviewers should
also consider the issues in Table 9 when performing a
review. Reviewers may have special insight into industry
relationships, conflicts of interest, or investigator relation-
ships that impact validity of findings or conclusions. Even if
only a hunch, these can still be reported confidentially in the
comments to the editor section, allowing the editorial staff
to further investigate and reach appropriate conclusions
prior to manuscript acceptance.
Authorship credit should be based on criteria established
by the International Committee of Medical Journal Edi-
tors:
25
1) substantial contributions to conception and design,
acquisition of data, or analysis and interpretation of data; 2)
drafting the article or revising it critically for important
intellectual content; and 3) final approval of the version to
be published. Authors must meet all three criteria, which are
typically verified in writing in the manuscript submission
480 Otolaryngology–Head and Neck Surgery, Vol 142, No 4, April 2010