Accuracy of PubMed Classification of Orthodontic Randomized Controlled Trials

Background:  PubMed allows users to search over 47,000 orthodontic related citations from the National Library of Medicine’s Medline Database.   The citations are indexed in multiple ways, including “Publication Type” to allow efficient searches for the highest levels of evidence.

Objectives: To identify the percentage of articles that are true randomized controlled trials that appear in the PubMed search for orthodontic Randomized Controlled Trials.

Methods: A PubMed search with the MeSH term “orthodontics” and limits of “randomized controlled trials” and articles published within the 10 year period from 2001 to 2011 resulted in 818 articles.  It was assumed all 818 articles were categorized correctly with the MeSH term “orthodontics.”  Using the random.org random number generator, a simple random sample of 110 articles were selected and read to verify that each was correctly classified as randomized controlled trial according to the MeSH Descriptor Data, Scope Note, and criteria for RCTs.  Studies using extracted teeth and split mouth designs were included as true randomized controlled trials if two or more treatments were randomly assigned.  Five articles from the sample were not available and were excluded from the analysis.

Results: 88.6% [95% CI 82.5 to 94.7] of the 105 articles sampled met the criteria for an RCT and were thus correctly classified as RCTs in PubMed.

Conclusions: PubMed’s classification of orthodontic RCTs is quite accurate with less than 12% error, suggesting that users can have reasonable confidence in the search results for orthodontic RCTs.  This evaluation did not address the possible errors of omission, i.e. the number of orthodontic RCTs that may have been missed by a PubMed search.