Review of Numbers Data and Statistics for Non-specialist by Shvidko

• Tutorial
• Open up Access
• Published: 02 November 2015

How to spot a statistical problem: advice for a non-statistical reviewer

BMC Medicine volume 13, Article number:270 (2015) Cite this article

• 16k Accesses

• 14 Citations

• 208 Altmetric

• Metrics details

Abstract

Statistical analyses presented in general medical journals are becoming increasingly sophisticated. BMC Medicine relies on subject reviewers to indicate when a statistical review is required. Nosotros consider this policy and provide guidance on when to recommend a manuscript for statistical evaluation. Indicators for statistical review include insufficient detail in methods or results, some common statistical issues and estimation not based on the presented bear witness. Reviewers are required to ensure that the manuscript is methodologically audio and clearly written. Within that context, they are expected to provide constructive feedback and stance on the statistical design, analysis, presentation and estimation. If reviewers lack the advisable background to positively confirm the appropriateness of whatsoever of the manuscript'southward statistical aspects, they are encouraged to recommend it for skillful statistical review.

Peer Review reports

Introduction

About papers published in general medical journals, including BMC Medicine, incorporate some chemical element of statistical methods, analysis and interpretation. There is evidence that statistical analyses are becoming increasingly sophisticated [1]. Expert statistical review has therefore become an integral part of the editorial process. Some journals ship all manuscripts for statistical review. Other journals merely send a manuscript for statistical review if it is considered necessary; for case, if the methods are peculiarly complex or if the Editor or subject reviewer has concerns. The arroyo taken by BMC Medicine is to ask discipline reviewers if they are able to appraise all the statistical aspects of the manuscript themselves or whether they recommend an additional statistical review.

1 potential weakness of this approach is that it is a system that relies heavily upon the statistical expertise of subject reviewers, who may not have a formal qualification or professional accreditation in statistics. As such, the field of study reviewer may be competent in a specific range of statistical methods applicative to their expanse of expertise, only may not necessarily exist enlightened of more general statistical problems or more recent methodological developments and all-time practices. The discipline reviewer may be able to spot the most egregious errors just is likely to miss the subtlety of inappropriate statistics that might exist picked up by an accordingly qualified statistical expert. The aim of this paper is to provide subject reviewers with some help in deciding when a manuscript might do good from undergoing a proper statistical review. Our comments mainly refer to review of primary research, rather than to systematic reviews and meta-analysis, for which a divide tutorial is available [2].

Statistical review is an important element of the peer-review process that has been shown to substantially amend the quality of manuscripts [3–v]. This relates not only to the statistical assay, just too to other relevant areas, such every bit data sources, study design, presentation of results and interpretation of results [1, 6].

We argue that sending a newspaper for statistical review should not be limited to studies where the bailiwick reviewer considers the methods to be potentially wrong, or across their expertise. Rather, the bailiwick reviewer should generally recommend good statistical review unless they can positively confirm that in that location are no problems with the report design, statistical assay, presentation and interpretation of results.

Although some statistical irregularities are subtle and only likely to be detected by a statistical good, subject field reviewers should consider some of the following indicators of the more than mutual problems encountered in master enquiry:

Is there sufficient detail to review the statistical aspects?

• Have the relevant reporting guidelines been followed (for example, Consort for randomized controlled trials [7] or STROBE for observational studies [8])?

• Have the authors justified their sample size and made reasonable assumptions near the effect size they consider of import to find? Have they presented plenty information to verify their calculations [nine]?

• Have the methods been provided in sufficient detail to replicate the results if the data were available [1, x, 11]?

• Is information technology articulate how all the results were derived, such as the test or model used, including whatsoever covariates, and were the assumptions made in implementing the model reasonable?

Are there whatever common statistical bug?

• Are in that location lots of P values, or subgroup analyses, particularly unplanned subgroup analyses that were not pre-specified, indicating multiple testing [12]?

• Are the covariates adjusted for in models appropriate, without remaining confounding, or over-adjustment for covariates on the causal pathway (for case, longitudinal studies where a covariate is measured later the exposure)?

• Are at that place any hierarchical data structures (for case, cluster randomized trials, repeated measures or matching of cases and controls), and if so has the analysis taken this into account?

• Should the analysis address understanding rather than clan [13]?

• Has the intention-to-treat principle been appropriately applied in pragmatic effectiveness trials [14, 15]?

• Take continuous variables been categorized? Take trends been ignored? This may not necessarily mean an inappropriate analysis, but may point that a total statistical review would be beneficial.

Is the presentation of results appropriate?

• Is there any prove of selective reporting? Do the main results focus on the main research question, or do they deviate to a secondary question or subgroup? This is particularly problematic if the subgroup analysis was not specified prior to undertaking the assay [12].

• Are results presented without estimates, but P values [16]?

• Are estimates presented with no confidence intervals? Standard errors alone are rarely adequate for presenting the doubtfulness in estimates, either in the text or graphically [sixteen].

Is the interpretation of results appropriate?

• Are limitations of observational studies correctly acknowledged, with no implication of causality in the wording of results and conclusions?

• Are results over-extrapolated, beyond the range of the information, or to populations non represented past the study sample?

• Is in that location an advisable consideration of the impact of any incomplete or missing data?

Although there might be culling approaches to statistical analysis or presentation, this does not necessarily imply the authors' methods are invalid. What is important is that the methods chosen are appropriate for the research question and have been done correctly [17]. BMC Medicine allows comments under "discretionary revisions" where such observations can be fabricated.

The same caution we recommend to non-statistical reviewers also applies to statistical experts. Statistical methods are many and varied, specially in a full general medical journal such as BMC Medicine. Some of the more specialist methods may be exterior of the feel of a general statistical reviewer. Consequently they should be encouraged to recommend that the editorial office approach an boosted specialist in those particular methods for further scrutiny of the article.

Conclusions

In advising the Editor on publication, reviewers are required to comment on whether a manuscript is methodologically sound and clearly written. Inside that context, they are expected to provide clear, constructive feedback and opinion on study blueprint, statistical analysis, presentation and interpretation of results. We have provided a number of indicators to assist the non-statistical reviewer in this task. If reviewers lack the appropriate background to positively ostend the ceremoniousness of any of the manuscript's statistical aspects, they are encouraged to recommend it for skillful statistical review.

Abbreviations

CONSORT:

Consolidated Standards of Reporting Trials

STROBE:

Strengthening the Reporting of Observational Studies in Epidemiology

References

1. Altman DG. Statistical reviewing for medical journals. Stat Med. 1998;17:2661–74.

   CAS  Article  PubMed  Google Scholar

2. Moher D. Optimal strategies to consider when peer reviewing a systematic review and meta-analysis. BMC Medicine 2015. doi: 10.1186/s12916-015-0509-y.

3. Chauvin A, Ravaud P, Businesswoman Thousand, Barnes C, Boutron I. The almost important tasks for peer reviewers evaluating a randomized controlled trial are not congruent with the tasks nearly often requested by periodical editors. BMC Med. 2015;13:158.

   PubMed Primal  Commodity  PubMed  Google Scholar

4. Gardner MJ, Bond J. An exploratory study of statistical assessment of papers published in the British Medical Journal. J Am Med Assoc. 1990;263:1355–7.

   CAS  Commodity  Google Scholar

5. Schor S, Karten I. Statistical evaluation of medical journal manuscripts. J Am Med Assoc. 1966;195:1123–8.

   CAS  Article  Google Scholar

6. Murray GD. The chore of a statistical referee. Br J Surg. 1988;75:664–vii.

   CAS  Article  PubMed  Google Scholar

7. Schulz KF, Altman DG, Moher D. CONSORT 2010 Statement: updated guidelines for reporting parallel group randomised trials. BMC Med. 2010;8:18.

   PubMed Central  Commodity  PubMed  Google Scholar

8. von Elm E, Altman DG, Egger M, Pocock SJ, Gotzsche PC, Vandenbroucke JP. The Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) statement: guidelines for reporting observational studies. J Clin Epidemiol. 2008;61:344–9.

   Article  Google Scholar

9. Altman DG. Statistics and ethics in medical research: 3 How large a sample? Br Med J. 1980;281:1336–viii.

   PubMed Central  CAS  Commodity  PubMed  Google Scholar

10. International Commission of Medical Journal Editors. Uniform requirements for manuscripts submitted to biomedical journals. International Committee of Medical Journal Editors. Med Educ. 1999;33:66–78.

    Article  Google Scholar

11. Altman DG. Making research articles fit for purpose: structured reporting of key methods and findings. Trials. 2015;16:53.

    PubMed Primal  Article  PubMed  Google Scholar

12. Assmann SF, Pocock SJ, Enos LE, Kasten LE. Subgroup assay and other (mis)uses of baseline data in clinical trials. Lancet. 2000;355:1064–9.

    CAS  Article  PubMed  Google Scholar

13. Bland JM, Altman DG. Statistical methods for assessing agreement between two methods of clinical measurement. Lancet. 1986;i:307–ten.

    Article  Google Scholar

14. Hollis Due south, Campbell MJ. What is meant by intention to treat analysis? Survey of published randomised controlled trials. BMJ. 1999;319:670–4.

    PubMed Fundamental  CAS  Article  PubMed  Google Scholar

15. White IR, Carpenter J, Horton NJ. Including all individuals is not enough: lessons for intention-to-care for assay. Clinical Trials. 2012;9:396–407.

    PubMed Central  Commodity  PubMed  Google Scholar

16. Gardner MJ, Altman DG. Confidence intervals rather than P values: estimation rather than hypothesis testing. Br Med J (Clin Res Ed). 1986;292:746–50.

    CAS  Article  Google Scholar

17. Pyke DA. Writing and speaking in medicine. How I referee. Br Med J. 1976;2:1117–8.

    PubMed Central  CAS  Commodity  PubMed  Google Scholar

Download references

Acknowledgements

We are grateful for the detailed and constructive comments of the Editor, Sabina Alam, and the reviewers of this commodity, Andrea Tricco and Jaime Peters.

Author information

Affiliations

1. Sectionalisation of Epidemiology and Biostatistics, University of Leeds, Leeds, LS2 9JT, UK

   Darren C. Greenwood & Jennifer V. Freeman

Corresponding author

Correspondence to Darren C. Greenwood.

Boosted information

Competing interests

The authors declare that they accept no competing interests.

Authors' contributions

DCG wrote the commencement draft of the manuscript. Both authors contributed to further drafts. Both authors read and canonical the concluding manuscript.

Authors' information

DCG is a Senior Lecturer in Biostatistics at the University of Leeds, and a member of the Editorial Board of BMC Medicine and British Periodical of Nutrition. JVF is a Senior Lecturer in Biostatistics, a Majestic Statistical Society Chartered Statistician and current Vice-President of the Imperial Statistical Society. Both authors have acted as subject field reviewer or statistical reviewer of numerous manuscripts for a wide range of medical journals.

Rights and permissions

Open Admission This article is distributed nether the terms of the Creative Eatables Attribution 4.0 International License (http://creativecommons.org/licenses/past/four.0/), which permits unrestricted use, distribution, and reproduction in whatsoever medium, provided you give appropriate credit to the original author(due south) and the source, provide a link to the Artistic Commons license, and signal if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data fabricated bachelor in this article, unless otherwise stated.

Reprints and Permissions

Virtually this article

Cite this article

Greenwood, D.C., Freeman, J.V. How to spot a statistical problem: communication for a not-statistical reviewer. BMC Med 13, 270 (2015). https://doi.org/10.1186/s12916-015-0510-5

Download citation

• Received: 07 July 2015

• Accepted: 18 September 2015

• Published: 02 November 2015

• DOI : https://doi.org/10.1186/s12916-015-0510-5

Keywords

• Completeness of reporting
• Editorial policy
• Peer review
• Presentation
• Professional competence
• Publishing
• Recommendations to reviewers
• Reporting guidelines
• Statistics

postunintork.blogspot.com

Source: https://bmcmedicine.biomedcentral.com/articles/10.1186/s12916-015-0510-5

Share this post