Please use this identifier to cite or link to this item:
Title: Misleading medical literature: An observational study.
Austin Authors: Olaussen, Alexander;Abetz, Jeremy;Qin, Kirby R ;Mitra, Biswadev;O'Reilly, Gerard
Affiliation: Centre for Research and Evaluation, Ambulance Victoria, Melbourne, Victoria, Australia
Emergency and Trauma Centre, The Alfred Hospital, Melbourne, Victoria, Australia
National Trauma Research Institute, The Alfred Hospital, Melbourne, Victoria, Australia
Department of Paramedicine, Monash University, Melbourne, Victoria, Australia
School of Public Health and Preventive Medicine, Monash University, Melbourne, Victoria, Australia
Department of Paediatrics, School of Clinical Sciences at Monash Health, Monash University, Melbourne, Victoria, Australia
Department of Surgery, Ballarat Health Services, Ballarat, Victoria, Australia
Issue Date: 2022
Date: 2021-08-05
Publication information: Emergency medicine Australasia : EMA 2022-02; 34(1): 39-45
Abstract: Language that implies a conclusion not supported by the evidence is common in the medical literature. The hypothesis of the present study was that medical journal publications are more likely to use misleading language for the interpretation of a demonstrated null (i.e. chance or not statistically significant) effect than a demonstrated real (i.e. statistically significant) effect. This was an observational study of the medical literature with a systematic sampling method. Articles published in The Journal of the American Medical Association, The Lancet and The New England Journal of Medicine over the last two decades were eligible. The language used around the P-value was assessed for misleadingness (i.e. either suggesting an effect existed when a real effect did not exist or vice versa). There were 228 unique manuscripts examined, containing 400 statements interpreting a P-value proximate to 0.05. The P-value was between 0.036 and 0.050 for 303 (75.8%) statements and between 0.050 and 0.064 for 97 (24.3%) statements. Forty-four (11%) of the statements were misleading. There were 40 (41.2%) false-positive sentences, implying statistical significance when the P-value was >0.05, and four (1.3%) false-negative sentences, implying no statistical significance when the P-value <0.05 (relative risk 31.2; 95% confidence interval 11.5-85.1; P < 0.0001). The proportion of included manuscripts containing at least one misleading sentence was 16.2% (95% confidence interval 12.0-21.6). Among a random selection of sentences in prestigious journals describing P-values close to 0.05, 1 in 10 are misleading (n = 44, 11%) and this is more prevalent when the P-values are above 0.05 compared to below 0.05. Caution is advised for researchers, clinicians and editors to align with the context and purpose of P-values.
DOI: 10.1111/1742-6723.13831
ORCID: 0000-0001-8799-077X
Journal: Emergency Medicine Australasia : EMA
PubMed URL: 34355494
Type: Journal Article
Subjects: P-value
ethics in publishing
statistical significance
Appears in Collections:Journal articles

Show full item record

Page view(s)

checked on Apr 17, 2024

Google ScholarTM


Items in AHRO are protected by copyright, with all rights reserved, unless otherwise indicated.