Bibliometric analysis of the top 50 most cited publications of the Journal of Clinical Orthopaedics and Trauma

BackgroundThe Journal of Clinical Orthopaedics and Trauma (JCOT) is one of the top three orthopaedic journals from India. We set out to analyse the top 50 cited articles from JCOT since indexing in PubMed and Scopus.MethodsWe looked into the bibliometrics of the top 50 cited articles and compared citations from PubMed and Scopus, and depicted outputs from VOS viewer analysis on co-authorship and keywords.ResultsTotal citations for top-cited articles were 1076 in numbers, with a maximum of 103.2016 and 2018 were the most productive years. The major contribution was from India with 74%, followed by the USA. New Delhi published maximally at 72%. Clinical topics and narrative reviews were the most common types of studies. Trauma and Adult reconstruction was the most common sub-specialities, and Level 4 was the most frequent level of study. The basic science and COVID-19 related articles received the maximum citations. The authors from Indraprastha Apollo Hospitals published the maximum number of top-50 cited articles in the JCOT.ConclusionsThere is a steady increase in the number of publications in the JCOT, with an increasing number of citation counts. Both the Indian and foreign authors have been publishing in this journal at a comparative rate. Although the citation counts in Scopus are more than those in PubMed for given articles, more than 80% of articles are listed in both databases as top 50 cited articles. The majority of top-cited articles belonged to trauma and adult reconstruction, level III studies, and narrative reviews.     

An analysis of citations of publications in anaesthesia journals

Which journals cite work published in anaesthetic journals is of potential interest to authors, editors and publishers. We analysed citations made in 2017–2018 for articles, reviews, editorials and letters published by 12 anaesthetic journals in 2016, using the Web of Science™ citation index platform. We analysed 12,544 citations made for 3518 items. Citations were most often made by specialist anaesthesia journals and critical care journals, and occurred most commonly in articles, followed by reviews, editorials and letters. The median (IQR [range]) number of citations made per item was 3.3 (2.6–4.1 [1.6–5.1]). The median (IQR [range]) number of journals that cited the 12 source journals was 302 (236–449 [139–671]). The median (IQR [range]) proportion of citations made by the same journal that published the items (i.e. ‘self-citations’) was 15% (11–17% [5–32%]). There were 1305/1932 (68%) citations made by North American journals for items published in North American journals and 1712/2063 (83%) citations made by European journals for items published in European journals, p < 0.0001. Our analysis may inform authors, editors and publishers where to submit work, what editorial policy to pursue and what journal strategy to follow, respectively.     

The 50 Most-cited Articles in Orthopaedic Surgery From Mainland China

Background

Citation analysis has been widely used to evaluate the impact of articles in medical and surgical specialties. Although China is the most populous country in the world, and although more than 50,000 orthopaedic surgeons practice there, to our knowledge no formal citation analysis of Chinese orthopaedic articles has been performed.

Questions/purposes

We identified the 50 most-cited orthopaedic articles from mainland China and evaluated these articles in terms of their language of publication, source journals, and topics.

Methods

Science Citation Index Expanded was searched in July 31, 2014 for citations of articles published in 70 selected journals since the inception of the database. The 50 most-cited orthopaedic articles originating in mainland China were identified. Basic information, including title, authors, year of publication, article type, journal in which the work was published, city, institution, number of citations, decade published, and topic or subspecialty of the research were recorded.

Results

The number of citations for the top 50 papers ranged from 181 to 31 (mean, 52). These articles were published between 1981 and 2010. The decade of 2000 to 2009 was the most prolific, with 36 of the top 50 articles published during this time. All articles were written in English and they were published in a total of 16 journals. The journal Spine published the largest number of articles (12), followed by Clinical Orthopaedics and Related Research^® (seven). The journal Lancet had the highest impact factor (39.207 for 2013) among any of the journals that published articles we identified. The top 50 articles originated mainly from Beijing (16) and Shanghai (12), with basic research being the focus of the majority (27 of 50; 54%); the remaining were clinical studies. Bone was the most-investigated topic in basic research; the spine was the most-common topic among the identified clinical studies.

Conclusions

The 50 most-cited articles that we identified should be considered influential, although a large gap remains between mainland China and the global orthopaedic community in terms of citations per article. Nevertheless, insofar as the most-recent decade of our survey generated the most articles in this top-50 list, we would characterize mainland China’s effect on musculoskeletal research as increasing, and as funding increases to programs in mainland China, we anticipate this trend will continue in the future.     

The fifty most cited Latin-American articles in the orthopaedic literature

Purpose

The number of citations of an article is a marker of its academic influence. Several medical specialties, including orthopaedics, have ranked the articles with more citations. We identified the 50 most cited orthopaedic articles from Latin-America and analyzed the characteristics that made them citable.

Methods

Science Citation Index Expanded was searched for citations of articles originated in Latin-America, published in any of the 63 journals in the category “Orthopaedics” from 1988 to 2013. We created a list ranking the 50 most commonly cited articles and determined the citation density (Citations/years since publication). Information noted for each article included authors, year of publication, country of origin, source journal, article type, and field of research.

Results

Latin-American countries were the origin of 1 % of orthopaedic articles. The top 50 most cited articles had between 29 and 150 citations (mean, 44.48); the citation density ranged from 1.43 to 15.5 citations/years (mean, 5.25). The articles were published in 19 of the 63 journals (11 general and eight sub-specialty journals), and all were published in English. Most articles (n = 29) were published in 2000 or later. The majority were clinical articles (n=40), and the most common fields were arthroscopy (n = 15) and hip surgery (n = 13). The top 50 articles originated mainly from Brazil (n = 20) and Argentina (n = 15).

Conclusions

This top 50 list displays articles that have become important references for the orthopaedic scientific community. Researchers may use this work to make their future publications more influential on future investigators.     

Risk of bias and methodological appraisal practices in systematic reviews published in anaesthetic journals: a meta‐epidemiological study

The validity of primary study results included in systematic reviews plays an important role in drawing conclusions about intervention effectiveness and carries implications for clinical decision‐making. We evaluated the prevalence of methodological quality and risk of bias assessments in systematic reviews published in the five highest‐ranked anaesthesia journals since 2007. The initial PubMed search yielded 315 citations, and our final sample after screening consisted of 207 systematic reviews. One hundred and seventy‐four reviews conducted methodological quality/risk of bias analyses. The Jadad scale was most frequently used. Forty‐four of the 83 reviews that included high risk of bias studies re‐analysed their data omitting these trials: 20 showed differences in pooled effect estimates. Reviews containing a greater number of primary studies evaluated quality less frequently than smaller reviews. Overall, the majority of reviews evaluated bias; however, many applied questionable methods. Given the potential effects of bias on summary outcomes, greater attention is warranted.     

Misrepresentation of research criteria by orthopaedic residency applicants

BACKGROUND: Previous studies have shown that applicants for postgraduate training may misrepresent research citations. We evaluated the research citations that were identified in a review of the Publications and Work and Research sections from the Electronic Residency Application Service (ERAS) data for all applicants to our orthopaedic residency program for the 1998 to 1999 academic year. METHODS: The citations were searched for on Medline. We initially used the name of the first author, then the name of the applicant, the name of the journal, the volume number, the issue number, and the page numbers. When a journal was not listed in Medline, an interlibrary search was instituted with use of the same format. When no match was made for any category, the citation was defined as misrepresented. Point estimates are reported as percentages. RESULTS: Publications were listed on sixty-four (30.0 percent) of 213 applications. One hundred and thirty-eight publications were cited; there were fifteen citations (10.9 percent) to book chapters, twenty-six (18.8 percent) to journals not listed in Ulrich's International Periodicals Directory, and twenty-one (15.2 percent) to articles listed as in press, in print, or submitted for publication. Seventy-six articles that had been cited as appearing in journals listed in Ulrich's Directory were checked and verified. Fourteen (18 percent) of these seventy-six publications were misrepresented. Misrepresentations included citations of nonexistent articles in actual journals and nonauthorship of existing articles. CONCLUSIONS: We concluded that publications listed on postgraduate applications should be scrutinized carefully. Copies of cited publications should be required by residency programs before applications are considered complete. The importance of professionalism needs to be emphasized in the curricula of medical schools. Residency training programs should develop guidelines regarding misrepresentation.     

Follow-up on misrepresentation of research activity by orthopaedic residency applicants: has anything changed?

BACKGROUND: In our previous study, published in 1999, we showed that 18% of research citations listed as published by orthopaedic residency applicants were misrepresented. Since our last report, we sought to determine whether there had been any change in the behavior of applicants wishing to pursue the field of orthopaedic surgery. METHODS: We evaluated the research citations that were identified after a review of the Publications section of the Common Application Form from the Electronic Residency Application Service for all applicants to our orthopaedic residency program for 2005 and 2006. Inclusion and exclusion criteria were established for citations listed on candidate applications. Citations were required to be from journals listed in Ulrich's Periodicals Directory. The PubMed-MEDLINE database engine was used to search for citations. If searching failed to yield the cited publication, a review of the journal of alleged publication was undertaken and an interlibrary search was conducted with the use of several research databases. When no match was found, the citation was labeled as misrepresented. Misrepresentation was defined as either (1) nonauthorship of an existing article or (2) claimed authorship of a nonexistent article. RESULTS: One hundred and forty-two (35.9%) of 396 applicants during the 2005 and 2006 application periods listed publications. A total of 304 citations were claimed from these 142 applicants. Listings included articles that were in press or in print (thirty-four citations), articles in journals not found in Ulrich's Periodicals Directory (twenty-eight citations), book chapters (twenty-three citations), and articles recorded as having been submitted (eighty-eight citations). These 173 works were excluded from our analysis. One hundred and thirty-one citations were referenced as appearing in journals per our search criteria, and all were verified. Twenty-seven or 20.6% (95% confidence interval, 14.2% to 28.7%) of 131 citations were misrepresented. CONCLUSIONS: The prevalence of misrepresented research publications from orthopaedic surgery residency applicants increased modestly to 20.6% compared with that found in our original report (18%). As we recommended in our last report, we strongly urge residency programs to require applicants to submit reprints of their publications with their residency applications. Perhaps standardized guidelines should be developed to help to prevent misrepresentation through the Electronic Residency Application Service.     

Impact of COVID-19 on Orthopaedic Care and Practice: A Rapid Review

IntroductionThe COVID-19 pandemic has led to a large body of literature regarding the impact of COVID-19 on orthopaedic care and practice. This rapid review aims to synthesize this published literature to give the orthopaedic fraternity an overview about the best practices that need to be followed during this period.MethodologyA rapid review was conducted following the preferred reporting items for systematic reviews and meta-analyses (PRISMA) statement for rapid reviews on the impact of COVID-19 on orthopaedic care and practice. A Pubmed search was done to identify all literature related to the impact of COVID-19 on orthopaedic care and practice, published between December 2019 and October 2020 using a predefined search strategy. The final review included 375 peer-reviewed articles addressing the objectives.ResultsThe majority of articles were expert opinions ( 37.1%) and narrative reviews (13.1%). There were 17.3% retrospective studies and 2.1% prospective studies with only one randomized control trial and ten systematic reviews.  83.8% of articles had levels of evidence IV and V, 79.5% of the articles were published in core-orthopaedic journals. Maximum publications were from the United States of America (31.7%), followed by India (11.5%). European countries together contributed to  32.0% of all publications.ConclusionCOVID-19 has had a significant impact on all aspects of orthopaedic care and practice. The pandemic has affected outpatient clinics, emergency and elective surgery, rehabilitation, resident training, personnel management, use of personal protective equipment, telemedicine and all sub-specialities of orthopaedics. Orthopaedic practice will require the incorporation of new technologies, restructuring of health systems and reorganizing of training programs for optimal patient care. There would also be a need for frequent review of emerging literature to provide evidence-based guidelines to the orthopaedic fraternity which will not only help in mitigation of transmission of disease but also ensure continuity of optimum patient care.Supplementary InformationThe online version contains supplementary material available at 10.1007/s43465-021-00354-0.     

Impact of Spanish publications on foreign anesthesiology journals

OBJECTIVE: To analyze the number of articles by Spanish authors in three international anesthesiology journals and assess the impact of Spanish anesthesiology literature during the same period by way of citations. MATERIAL AND METHODS: Articles published in Anesthesiology, British Journal of Anesthesiology (BJA) and Anesthesia & Analgesia (Anesth Analg) during 1997 were reviewed. We collected the following data: type of document, country of origin of the authors, total number of references, number of citations of Spanish articles and the journals from which they were cited. Later, the Spanish contribution was compared to that of other countries of the European Union, with data considered in function of Gross National Product per capita (GNPpc) of each country. RESULTS: The incidence of Spanish articles published in Anesthesiology, BJA and Anesth Analg was 0.37%, 0.71% and 0.9%, respectively. The production of Spanish documents in function of GNPpc ranked Spain in the tenth position in Anesthesiology, eleventh in BJA and sixth in Anesth Analg. Of the total number of citations, 143 (0.4%) were of Spanish publications. In Spanish articles in Anesthesiology the percentage of Spanish citations was 11%, in BJA the figure was 44% and in Anesth Analg it was 13%. Finally, over 90% of the Spanish articles cited in Anesthesiology and in Anesth Analg were published in English in foreign journals. CONCLUSIONS: The number of Spanish articles published in Anesthesiology, BJA, and Anesth Analg is low, although the comparison of our productivity with that of other EU countries in function of GNPpc places us in an intermediate position. The impact of Spanish literature on international studies is low, particularly research published in Spanish national journals, a circumstance that is reflected even in articles published by Spanish authors.     

Factors associated with citation rates in the orthopedic literature

INTRODUCTION: Investigators aim to publish their work in top journals in an effort to achieve the greatest possible impact. One measure of impact is the number of times a paper is cited after its publication in a journal. We conducted a review of the highest impact clinical orthopedic journal (Journal of Bone and Joint Surgery, American volume [J Bone Joint Surg Am]) to determine factors associated with subsequent citations within 3 years of publication. METHODS: We conducted citation counts for all original articles published in J Bone Joint Surg Am 2000 (12 issues). We used regression analysis to identify factors associated with citation counts. RESULTS: We identified 137 original articles in the J Bone Joint Surg Am. There were 749 subsequent citations within 3 years of publication of these articles. Study design was the only variable associated with subsequent citation rate. Meta-analyses, randomized trials and basic science papers received significantly more citations (mean 15.5, 9.3 and 7.6, respectively) than did observational studies (mean retrospective 5.3, prospective 4.2) and case reports (mean 1.5) (p = 0.01). These study designs were also significantly more likely to be cited in the general medical literature (p = 0.02). CONCLUSION: Our results suggest that basic science articles and clinical articles with greater methodological safeguards against bias (randomized controlled trials and meta-analyses) are cited more frequently than are clinical studies with less rigorous study designs (observational studies and case reports).     

《中华创伤骨科杂志》2007、2008年所载文献统计分析与评价

目的 统计分析<中华创伤骨科杂志>2007、2008年载文、著者及引文的特点.方法 用文献计量学的方法对<中华创伤骨科杂志>2007、2008年的载文、著者和引文情况进行统计,并分析其特点.结果 <中华创伤骨科杂志>两年内共载文718篇(2007、2008年分别载文为375和343篇),每期平均载文30篇,篇密度0.30;平均发表时滞204 d(4～545 d);基金论文比为25.5%;著者来自国外、我国港、澳、台地区和大陆29个省、自.冶区和直辖市,合著率90.8%,合作度4.69;88.0%(87.2%和88.9%)的论文有引文,93.1%的引文来自期刊,引文语种主要为英文,普莱斯指数为51.6%.结论 <中华创伤骨科杂志>所载文献质量较高,是我国骨科研究领域主要的信息源之一,也是我国陕学领域重要的核心期刊.     

A comprehensive appraisal of meta-analyses focusing on nonsurgical treatments aimed at decreasing perioperative mortality or major cardiac complications

Purpose

Millions of patients worldwide who undergo surgical procedures face significant morbidity and mortality risks. Several systematic reviews have been performed on ancillary treatments aimed at improving surgical outcomes, but their features and scholarly impact are unclear. We describe characteristics of meta-analyses on ancillary treatments aimed at improving surgical outcomes and explore factors associated with scholarly citations.

Methods

Systematic reviews published up to 2008 were searched without language restrictions in MEDLINE/PubMed. Reviews focusing on nonsurgical treatments aimed at decreasing mortality or major cardiac complications were included. Associations between content, quality, and bibliometric details and scholarly citations in several indexes were systematically appraised.

Results

From 2,239 citations, 84 systematic reviews were identified. Patients most commonly underwent cardiovascular surgery (40.2?%), and were tested for cardiovascular drugs (25.8?%), with placebo acting as control (38.1?%). Internal validity appeared largely robust, as most (50.5?%) reviews were at low risk of bias. Normalized yearly citations for the included reviews ranged between 5.6 in Google Scholar and 4.3 in Web of Science. Multivariable analysis showed that citations were significantly and positively associated with number of authors, North American corresponding author, number of studies included, number of patients included, noncardiothoracic surgical scope, explicit funding, and lack of competing interests (all p?Conclusions Systematic reviews currently represent a key element in defining state of the art ancillary treatments of patients undergoing surgery. However, the citation success of available meta-analyses is not significantly associated with prognostically relevant findings or quality features.     

Citation classics in anaesthesia and pain journals: a literature review in the era of the internet

BACKGROUND: The number of citations an article receives reflects its scientific impact. The introduction of internet-based resources to search and access articles has made it possible to review even whole scientific areas. This study identifies the top 50 most-cited articles over the last 25 years in specialty journals dedicated to anaesthesia and pain, respectively. METHODS: Twenty-two journals listed in The Science Citation Index Expanded in the field of anaesthesiology and nine major medical journals were screened using the cited reference option to identify articles cited more than 100 times between 1986 and 2002. The top 50 articles in specialty journals and the top 10 articles in major medical journals concerning anaesthesia and pain were retrieved for further analysis. RESULTS: The most-cited articles in specialty journals were published from 1986 to 1997 and received a mean of 222 (anaesthesia) and 279 (pain) citations. Sixty-seven institutions produced the most-cited articles and of those 28 were located outside North America. The articles were published in six journals led by the Journal of Pain (50 articles) and Journal of Anaesthesiology (39 articles). Forty-seven articles were classified as related to clinical experience, 33 were review articles, and 20 discussed basic science. CONCLUSION: To make an article a possible 'citation classic' in anaesthesiology, it should be published in one of six leading journals and originate from an established institute in North America. Internet resources to publish and cite the literature have to date not advanced any article published in the last 5 years to the top 100 list.     

Meta-analyses in orthopaedic surgery. A systematic review of their methodologies

BACKGROUND: The number and quality of well-designed scientific studies in the orthopaedic literature are limited. The purpose of this review was to determine the methodological qualities of published meta-analyses on orthopaedic-surgery-related topics. METHODS: A systematic review of meta-analyses was conducted. A search of the Medline database provided lists of meta-analyses in orthopaedics published from 1969 to 1999. Extensive manual searches of major orthopaedic journals, bibliographies of major orthopaedic texts, and personal files identified additional studies. Of 601 studies identified, forty met the criteria for eligibility. Two investigators each assessed the quality of the studies under blinded conditions, and they abstracted relevant data. RESULTS: More than 50% of the meta-analyses included in this review were published after 1994. We found that 88% had methodological flaws that could limit their validity. The main deficiency was a lack of information on the methods used to retrieve and assess the validity of the primary studies. Regression analysis revealed that meta-analyses authored in affiliation with an epidemiology department and those published in nonsurgical journals were associated with higher scores for quality. Meta-analyses with lower scores for quality tended to report positive findings. The meta-analyses that focused upon fracture treatment and degenerative disease (hip, knee, or spine) had significantly lower mean quality scores than did meta-analyses that examined thrombosis prevention and diagnostic tests (p < 0.05). CONCLUSIONS: The majority of meta-analyses on orthopaedic-surgery-related topics have methodological limitations. Limitation of bias and improvement in the validity of the meta-analyses can be achieved by adherence to strict scientific methodology. However, the ultimate quality of a meta-analysis depends on the quality of the primary studies on which it is based. A meta-analysis is most persuasive when data from high-quality randomized trials are pooled.     

Analysis of bibliographic citations received by Spanish anesthesiologists for work indexed by Science Citation Index, 1988-2002

OBJECTIVE: To analyze the international impact of articles published by authors in Spanish anesthesiology departments. METHOD: Citable articles indexed by Science Citation Index between 1988 and 2002 and authored by members of Spanish departments of anesthesiology were considered. Citations were counted 2 years and 5 years after publication. Authors and institutions were ranked according to number of citations received. We also determined the journals Spanish anesthesiologists most often chose for publishing their work. RESULTS: Of the 322 citable articles identified, 61.8% were cited in the 2 years following publication (total 587 citations), and 79.5% were cited within 5 years (total 1472 citations). The most frequently cited articles received 17 citations in 2 years and 45 in 5 years after publication. Articles from the Department of Anesthesiology of Hospital Clinic i Provincial of Barcelona received the largest number of citations (333 citations in 5 years). The author with the highest rate of citations received 11.57 per article. The author with the largest number of citations received 86. Anesthesia & Analgesia was the journal publishing the largest number of articles by Spanish anesthesiologists (35 articles). CONCLUSION: This citation analysis shows the international impact of publications by Spanish anesthesiologists.     

Citation classics in main pain research journals

The number of citations of an article in scientific journals reflects its impact on a specific biomedical field and its recognition in the scientific community. In the present study, we identified and analyzed the characteristics of the 100 most frequently cited articles published between 1970 and 2010 in journals pertaining to pain research and related fields. These articles were identified using the database of the Science Citation Index (1970 to present). The most cited article received 3,017 citations and the least cited article received 302 citations, with a mean of 585 citations per article. These citation classics were published in six high-impact journals, led by Pain (84 articles). Of the 100 articles, 39 were observational studies, 25 were review articles, and 20 concerned basic science. The articles originated from 14 countries, with the United States contributing 47 articles; 67 institutions produced these 100 top-cited articles, led by National Institutes of Health of the United States (8 articles) and University College London (6 articles); 18 persons authored 2 or more of the top-cited articles. This analysis of the top citation classics allows for the recognition of major advances in pain research and gives a historical perspective on the scientific progress of this specialty.     

Basic science research publications in general surgical journals: are they obsolete?

Introduction: Basic science research (BSR) publications in general surgical journals are an important ‘translational bridge’ for practicing surgeons and surgical trainees. However, these articles are considered by some as left-over from higher impact basic science journals, and perhaps as obsolete. The purpose of this study is to characterize BSR articles published in general surgical journals and to analyze their citation frequencies. Methods: All BSR articles published in the five highest rated US general surgical journals (by Impact Factor, ISI Journal Citation reports) in the year 1996 were reviewed, characterized and their citation frequencies analyzed (total citations, citations within a year of publication, and citations in journals with impact factor greater than 5). The Wilcoxon rank-sum test was used to compare citation rates in different groups. Results: In 1996, 226 BSR articles were published in the five general surgical journals studied (10-43% of the total articles in each journal). Three fourths of BSR articles were from the USA (12% of total BSR articles by US surgeons in 1996), 12% from Europe and 8% from Japan. In 58% of the BSR articles, one of the authors was a basic scientist (PhD). Fifty-six percent of the publications were presented at a society conference. One in two US BSR publications was funded by the government (14% by private non-profit, 3% industry). On average a BSR article from these journals is cited 32 times (range 1-141, median 11). Half of the articles were cited more than ten times in six years (1-5: 24%; 6-10: 23%; >10: 52%) and 22% were cited twice or more within a year of publication. One in four publications were cited twice or more in journals with an impact factor greater than five (0: 35%; 1: 23%; 2: 17%; >2: 25%). Citation frequencies of BSR publications in general surgical journals were related to the journal impact factor (p = 0.07), to having a basic scientist (i.e. PhD) as one of the authors (p < 0.01) and to the research having been presented at a conference. Conclusions: Basic science research articles in US general surgical journals in the year studied have significant citation frequencies and were not obsolete.     

Systematic reviews of nonrandomized clinical studies in the orthopaedic literature

We systematically reviewed systematic reviews of surgical orthopaedic interventions published between 1996 and 2001 to document when and how nonrandomized studies were included. From more than 10,000 citations examined in various electronic databases, 58 orthopaedic systematic reviews were eligible for inclusion based on specific criteria. Thirty of these (52%) included nonrandomized studies, 15 of which found no randomized controlled trials. Systematic reviews were more likely to include randomized controlled trials if nondistinguishable operations were compared (if participants could be blinded). Only six of the systematic reviews that included nonrandomized studies (20%) assessed the quality of primary studies. Heterogeneity of studies was a major concern. In 21 of the systematic reviews that included nonrandomized studies (70%), data for groups treated similarly were pooled across studies, and outcomes for pooled groups were compared. The conclusions of systematic reviews that included nonrandomized studies are weakened by the limitations of nonrandomized study designs. The absence of established methods for including nonrandomized studies in systematic reviews, and consequently variability in the methods adopted, also limits the comparability of such reviews. Therefore the findings of systematic reviews that include nonrandomized studies should be interpreted with caution.