Evaluation of methodology and quality characteristics of systematic reviews in orthodontics

Systematic reviews (SRs) are published with an increasing rate in many fields of biomedical literature, including orthodontics. Although SRs should consolidate the evidence-based characteristics of contemporary orthodontic practice, doubts on the validity of their conclusions have been frequently expressed. The aim of this study was to evaluate the methodology and quality characteristics of orthodontic SRs as well as to assess their quality of reporting during the last years. Electronic databases were searched for SRs (without any meta-analytical data synthesis) in the field of orthodontics, indexed up to the start of 2010. The Assessment of Multiple Systematic Reviews (AMSTAR) tool was used for quality assessment of the included articles. Data were analyzed with Student's t-test, one-way ANOVA, and linear regression. Risk ratios (RR) with 95% confidence intervals were calculated to represent changes during the years in reporting of key items associated with quality. A total of 110 SRs were included in this evaluation. About half of the SRs (46.4%) were published in orthodontic journals, while few (5.5%) were updates of previously published reviews. Using the AMSTAR tool, thirty (27.3%) of the SRs were found to be of low quality, 63 (57.3%) of medium quality, and 17 (15.5%) of high quality. No significant trend for quality improvement was observed during the last years. The overall quality of orthodontic SRs may be considered as medium. Although the number of orthodontic SRs has increased over the last decade, their quality characteristics can be characterized as moderate.     

Methodological and Reporting Quality of Systematic Reviews and Meta-analyses in Endodontics

Introduction

The aim of this systematic review (SR) was to evaluate the quality of SRs and meta-analyses (MAs) in endodontics.

Methodological Quality Assessment of Meta-analyses and Systematic Reviews of the Relationship Between Periodontal and Systemic Diseases

ObjectivesThe aims of this article are to identify all the published systematic reviews (SRs) and meta-analyses (MAs) that studied the relationship between periodontal and systemic diseases and to assess their quality using 2 scales (the Overview Quality Assessment Questionnaire [OQAQ] and A Measurement Tool to Assess Systematic Reviews [AMSTAR] checklist).MethodsFor SRs and MAs to be included, they should have investigated one of the following systemic diseases: pulmonary conditions, cardiac conditions, endocrine conditions, cancer, blood disorders, psychological conditions, anxiety, depression, mood disorders, and several other diseases. Two investigators screened MEDLINE via PubMed, Embase, Scopus, Web of Science, and the Cochrane Database of Systematic Reviews. The tools used to evaluate quality were the AMSTAR scale and OQAQ. The protocol was prospectively registered in PROSPERO (CRD42018102208).ResultsThe search strategy found 691 unique articles, 42 of which met the eligibility criteria and were included in this review. Diabetes mellitus was the most investigated disease (14 out of 42 studies), followed by obesity (11 studies) and cardiovascular diseases (5 studies). A total of 40 reviews reported on the characteristics of included studies, and, as per the AMSTAR scale, 39 reviews had an a priori design. The number of reviews that fulfilled the status of publication criterion was the lowest (7 reviews only), followed by the number used in the assessment of publication bias (11 reviews). The number of high-quality reviews was higher with the OQAQ than with the AMSTAR checklist (33 vs 25 studies), but the AMSTAR showed a higher number of medium-quality reviews than the OQAQ (14 vs 6 studies). Both showed the same number of low-quality reviews.ConclusionsHigh-quality SRs and MAs are crucial to understanding the relationship between systemic and periodontal diseases. Medical practitioners must be able to inform patients about oral health and specific periodontal health concerns.     

Quality Assessment of Systematic Reviews on Periodontal Regeneration in Humans

Background: Systematic reviews represent the highest form of evidence in the current hierarchy of evidence‐based dentistry. Critical analysis of published systematic reviews may help to analyze their strengths and weaknesses and to identify areas that need future improvement. The aim of this overview is to determine and compare the quality of systematic reviews published in the field of periodontal regeneration using established checklists, such as the Assessment of Multiple Systematic Reviews (AMSTAR) guidelines. Methods: A systematic search was conducted to retrieve reviews on periodontal regeneration in humans. A total of 14 systematic reviews were selected using a set of inclusion and exclusion criteria. Two independent reviewers appraised the quality of the selected reviews using AMSTAR guidelines. Each article was given an AMSTAR total score, based on the number of AMSTAR criteria that were fulfilled. The quality of included reviews was further assessed using a checklist proposed in 2003. Results: Only one of the selected systematic reviews satisfied all the AMSTAR guidelines, whereas two reviews satisfied just two of the 11 items. This study shows that published systematic reviews on periodontal regeneration exhibit significant structural and methodologic variability. Quality assessment using the additional checklist further confirmed the variability in the way systematic reviews were conducted and/or reported. Conclusion: Consideration of guidelines for quality assessment, such as AMSTAR, when designing and conducting systematic reviews may increase the validity and clinical applicability of future reviews.     

Effectiveness of adjunctive interventions for accelerating orthodontic tooth movement: a systematic review of systematic reviews

This study was aimed to summarise published systematic reviews (SRs) that assess the effects of adjunctive interventions on the acceleration of orthodontic tooth movement (OTM). Electronic and manual searches were performed up to August 2016. Systematic reviews investigating the impact of adjunctive techniques on the promotion of OTM were included. The methodological quality of the included reviews was evaluated using the A Measurement Tool to Assess Systematic Reviews (AMSTAR) scale. The quality of evidence for each intervention was assessed using GRADE. The Jadad decision algorithm was used to select a study to provide body evidence from discordant reviews on the same intervention. A total of 11 SRs were included in this study. AMSTAR scores ranged from 4 to 10 of 11. The quality of evidence ranged from very low to low. The short‐term (1–3 months) effects of low‐level laser therapy (LLLT, 5 and 8 J cm^?2) and corticotomy were supported by low‐quality evidence. The evidence regarding the efficacy of photobiomodulation, pulsed electromagnetic field, interseptal bone reduction, two vibrational devices (Tooth Masseuse and Orthoaccel) and electrical current was of very low quality. Relaxin injections and extracorporeal shock waves were reported to have no impact on OTM according to low‐ and very low‐quality evidence, respectively. Based on currently available information, we conclude that low‐quality evidence indicates that LLLT (5 and 8 J cm^?2) and corticotomy are effective to promote OTM in the short term. Future high‐quality trials are required to determine the optimal protocols, as well as the long‐term effects of LLLT and corticotomy, before warranting recommendations for orthodontics clinics.     

Current status of the reporting quality of abstracts in systematic reviews related to implant dentistry: a literature survey

The aim of this study was to assess the reporting quality of abstracts in systematic reviews (SRs) related to implant dentistry and to assess the possible factors associated with the reporting quality. Abstracts of SRs in the field of implant dentistry, published in the last 5 years, were searched. The reporting quality was assessed and scored using the PRISMA for Abstracts checklist (PRISMA-A). The overall PRISMA-A score (OPS) and relative score (OPS%) per review were calculated according to adherence to the criteria presented in the checklist. Multivariable linear regression was performed to identify possible factors associated with reporting quality. Overall, 310 SRs were eligible for this study. Based on the maximum PRISMA-A score (score of 12), the mean OPS was 6.5 and OPS% was 54.2%. The items ‘title’, ‘objectives’, and ‘number of included studies’ were those most frequently reported in the abstracts, while the items ‘registration’ and ‘funding’ were the least reported. According to multivariable linear regression, the geographical origin of the articles was the only factor associated with better quality of abstract reporting, with higher OPS for SRs from Europe when compared to North America (coefficient 0.73; P = 0.049). The reporting quality of abstracts in SRs related to implant dentistry is suboptimal and needs to be improved. Journals should encourage adherence to reporting checklists in SRs.     

ACCURACY OF INTRAORAL SCANNERS VERSUS TRADITIONAL IMPRESSIONS: A RAPID UMBRELLA REVIEW

PurposeThis study aimed to (1) report the trueness and precision of intraoral scanning (IOS) in dentistry based on recent secondary sources and to (2) appraise the reporting quality of the titles and abstracts of the included literature.Materials and methodsThis rapid overview searched the PubMed/Medline and Cochrane Database of Systematic Reviews in March 2021 to identify reviews reporting on the accuracy of IOS. The reference list from the eligible studies was also screened for identification of other potentially eligible studies. The inclusion criteria consisted of English language systematic reviews or meta-analyses published between 2019 and 2021. The exclusion criteria were primary studies, narrative review, and extraoral scanners. The assessment of reporting quality of abstracts of systematic reviews was performed using the reporting checklist PRISMA extension for Abstracts (PRISMA-A). This was a self-funded research project.ResultsOut of the full text screened 25 records, 11 reviews were included. Most studies supported the IOS approach being as precise and accurate as the conventional one. Only one study significantly favored the conventional approach over the IOS, and two studies abstained from making a recommendation. The IOS was significantly superior to the traditional technique in terms of patient preference and time efficiency. After applying PRISMA-A, recommendations for improvements on titles and abstracts of future reviews of IOS and conventional impressions are provided.ConclusionLaboratory data indicated similar accuracy between IOS and conventional impressions, whereas clinical data found the same in less than 4-unit fixed dental prostheses. For more extensive definitive fixed solutions or removable prostheses, the conventional approach is recommended. IOS was superior in terms of patient preference and time reduction. More clinical trials are required to determine the clinical effectiveness of incorporating IOS in broader scenarios. Better quality of reporting secondary sources abstract is advised.     

Methodological quality and outcome of systematic reviews reporting on orthopaedic treatment for class III malocclusion: Overview of systematic reviews

Objectives: The aim of this overview of systematic reviews was to investigate methodological quality and outcome of current systematic reviews (SRs) reporting on orthopaedic treatment for class III malocclusion.

Materials and methods: Computerized and manual searches were performed in Medline, Google Scholar, Cochrane Library, Embase, LILACS, SciELO, American Journal of Orthodontics and Dentofacial Orthopaedics, Angle Orthodontist, European Journal of Orthodontics, Journal of Orthodontics, Conference abstracts and Grey literature. No restrictions were set on language or date of publication. The search covered the starting date of the relevant databases until 30 April 2015. MeSH terms and free-text terms included ‘malocclusion’, ‘Angle class III’, ‘orthodontic appliances’, ‘functional’, facemask, review and meta-analysis. Screening of eligible studies, assessment of the methodological quality of the SRs and data extraction were conducted in duplicate and independently by two reviewers. Methodological quality was assessed using AMSTAR (assessment of multiple systematic reviews).

Results: A total of 222 studies were retrieved and after removal of duplicates, irrelevant studies, literature reviews and surgical approach treatments, 14 SRs and/or meta-analyses were included for qualitative synthesis. Mean AMSTAR score was 7.7/11 with a range of 3–10. There was evidence to demonstrate that face mask therapy can move the maxilla forward whilst causing a backward rotation of the mandible and increased facial height. There was also some evidence of mandibular growth retardation with chin cup therapy.

Conclusions: Orthopaedic appliances can improve a class III malocclusion in growing patients over the short-term; however, each appliance has a characteristic effect on the underlying skeletal pattern.     

THE REPORTING QUALITY OF SYSTEMATIC REVIEW ABSTRACTS IN LEADING GENERAL DENTAL JOURNALS: A METHODOLOGICAL STUDY

ObjectiveTo assess the reporting quality of systematic review (SR) abstracts published in leading general dental journals according to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses for Abstracts (PRISMA-A) guidelines, and to identify factors associated with overall reporting quality.MethodsWe identified SR abstracts published in 10 leading general dental journals and assessed their reporting quality. For each abstract, an overall reporting score (ORS, range: 0-13) was calculated. Risk ratio (RR) was calculated to compare the reporting quality of abstracts in Pre-PRISMA (2011-2012) and Post-PRISMA (2017-2018) periods. Univariable and multivariable linear regression analyses were performed to identify factors associated with reporting quality.ResultsA total of 104 eligible abstracts were included. The mean ORS was 5.59 (SD = 1.48) and 6.97 (1.74) respectively in the Pre- and Post-PRISMA abstracts, with statistically significant difference (mean difference = 1.38; 95% CI: 0.70, 2.05). Reporting of the exact P-value (B = 1.22; 95% CI: 0.45, 1.99) was a significant predictor of higher reporting quality.ConclusionThe reporting quality of SR abstracts published in leading general dental journals improved after the release of PRISMA-A guidelines, but is still suboptimal. Relevant stakeholders need to work together to enhance the reporting quality of SR abstracts in dentistry.     

Tooth autotransplantation: An umbrella review

Tooth autotransplantation is a versatile procedure with several clinical applications among patients across different age groups. The success of this procedure depends on multiple factors. Despite the wealth of studies available, no single primary study or systematic review is able to report on every factor affecting the outcomes of autotransplantation. The aims of this umbrella review were to evaluate treatment-related and patient-related outcomes of autotransplantation and to assess the pre-, peri- or post-operative factors that could affect these. An umbrella review was conducted according to the PRISMA statement. A literature search of five databases was performed up to 25 September 2022. Systematic Reviews (SR) with and without meta-analysis evaluating autotransplantation were included. Calibration among reviewers was carried out prior to study selection, data extraction and Risk of Bias (RoB) assessment. Study overlap was calculated using corrected covered area. Meta-meta-analysis (MMA) was performed for suitable SRs. The AMSTAR 2 critical appraisal tool was used to evaluate the quality of evidence. Seventeen SRs met the inclusion criteria. Only two SRs were suitable for conduct of MMA on autotransplantation of open apex teeth. The 5-year and 10-year survival rates were >95%. A narrative summary on factors that could affect autotransplantation outcomes and comparisons of autotransplantation to other treatment options were reported. Five SRs were rated as ‘low quality’ and 12 SRs were rated as ‘critically low quality’ in the AMSTAR 2 RoB assessment. In order to facilitate a more homogenous pool of data for subsequent meta-analysis, an Autotransplantation Outcome Index was also proposed to standardise the definition of outcomes. Autotransplantation of teeth with open apices have a high survival rate. Future studies should standardise the reporting of clinical and radiographic findings, as well as the definition of outcomes.     

Reporting quality of systematic review abstracts in leading oral implantology journals

Objectives

Abstracts of systematic reviews are of critical importance, as consumers of research often do not access the full text. This study aimed to assess the reporting quality of systematic review (SR) abstracts in leading oral implantology journals.

Methods

Six specialty journals were screened for SRs between 2008 and 2012. A 16-item checklist, based on the PRISMA statement, was used to examine the completeness of abstract reporting.

Results

Ninety-three SR abstracts were included in this study. The majority were published in Clinical Oral Implants Research (43%). The mean overall reporting quality score was 72.5% (95% CI: 70.8–74.2). Most abstracts were structured (97.9%), adequately reporting objectives (97.9%) and conclusions (93.6%). Conversely, inadequate reporting of methods of the study, background (79.6%), appraisal (65.6%), and data synthesis (65.6%) were observed. Registration of reviews was not reported in any of the included abstracts. Multivariate analysis revealed no difference in reporting quality with respect to continent, number of authors, or meta-analysis conduct.

Conclusions

The results of this study suggest that the reporting quality of systematic review abstracts in implantology journals requires further improvement.

Clinical significance

Better reporting of SR abstracts is particularly important in ensuring the reliability of research findings, ultimately promoting the practice of evidence-based dentistry. Optimal reporting of SR abstracts should be encouraged, preferably by endorsing the PRISMA for abstracts guidelines.     

Class II functional orthopaedic treatment: a systematic review of systematic reviews

This Systematic Review (SR) aims to assess the quality of SRs and Meta‐Analyses (MAs) on functional orthopaedic treatment of Class II malocclusion and to summarise and rate the reported effects. Electronic and manual searches were conducted until June 2014. SRs and MAs focusing on the effects of functional orthopaedic treatment of Class II malocclusion in growing patients were included. The methodological quality of the included papers was assessed using the AMSTAR (Assessment of Multiple Systematic Reviews). The design of the primary studies included in each SR was assessed with Level of Research Design scoring. The evidence of the main outcomes was summarised and rated according to a scale of statements. 14 SRs fulfilled the inclusion criteria. The appliances evaluated were as follows: Activator (2 studies), Twin Block (4 studies), headgear (3 studies), Herbst (2 studies), Jasper Jumper (1 study), Bionator (1 study) and Fränkel‐2 (1 study). Four studies reviewed several functional appliances, as a group. The mean AMSTAR score was 6 (ranged 2–10). Six SRs included only controlled clinical trials (CCTs), three SRs included only randomised controlled trials (RCTs), four SRs included both CCTs and RCTs and one SR included also expert opinions. There was some evidence of reduction of the overjet, with different appliances except from headgear; there was some evidence of small maxillary growth restrain with Twin Block and headgear; there was some evidence of elongation of mandibular length, but the clinical relevance of this results is still questionable; there was insufficient evidence to determine an effect on soft tissues.     

Management of TMD: evidence from systematic reviews and meta‐analyses

Summary This systematic review (SR) synthesises recent evidence and assesses the methodological quality of published SRs in the management of temporomandibular disorders (TMD). A systematic literature search was conducted in the PubMed, Cochrane Library, and Bandolier databases for 1987 to September 2009. Two investigators evaluated the methodological quality of each identified SR using two measurement tools: the assessment of multiple systematic reviews (AMSTAR) and level of research design scoring. Thirty‐eight SRs met inclusion criteria and 30 were analysed: 23 qualitative SRs and seven meta‐analyses. Ten SRs were related to occlusal appliances, occlusal adjustment or bruxism; eight to physical therapy; seven to pharmacologic treatment; four to TMJ and maxillofacial surgery; and six to behavioural therapy and multimodal treatment. The median AMSTAR score was 6 (range 2–11). Eighteen of the SRs were based on randomised clinical trials (RCTs), three were based on case–control studies, and nine were a mix of RCTs and case series. Most SRs had pain and clinical measures as primary outcome variables, while few SRs reported psychological status, daily activities, or quality of life. There is some evidence that the following can be effective in alleviating TMD pain: occlusal appliances, acupuncture, behavioural therapy, jaw exercises, postural training, and some pharmacological treatments. Evidence for the effect of electrophysical modalities and surgery is insufficient, and occlusal adjustment seems to have no effect. One limitation of most of the reviewed SRs was that the considerable variation in methodology between the primary studies made definitive conclusions impossible.     

Methodological quality assessment of paper-based systematic reviews published in oral health

Objectives

This study aimed to conduct a methodological assessment of paper-based systematic reviews (SR) published in oral health using a validated checklist. A secondary objective was to explore temporal trends on methodological quality.

Material and methods

Two electronic databases (OVID Medline and OVID EMBASE) were searched for paper-based SR of interventions published in oral health from inception to October 2014. Manual searches of the reference lists of paper-based SR were also conducted. Methodological quality of included paper-based SR was assessed using an 11-item questionnaire, Assessment of Multiple Systematic Reviews (AMSTAR) checklist. Methodological quality was summarized using the median and inter-quartile range (IQR) of the AMSTAR score over different categories and time periods.

Results

A total of 643 paper-based SR were included. The overall median AMSTAR score was 4 (IQR 2–6). The highest median score (5) was found in the pain dentistry and periodontology fields, while the lowest median score (3) was found in implant dentistry, restorative dentistry, oral medicine, and prosthodontics. The number of paper-based SR per year and the median AMSTAR score increased over time (median score in 1990s was 2 (IQR 2–3), 2000s was 4 (IQR 2–5), and 2010 onwards was 5 (IQR 3–6)).

Conclusion

Although the methodological quality of paper-based SR published in oral health has improved in the last few years, there is still scope for improving quality in most evaluated dental specialties.

Clinical relevance

Large-scale assessment of methodological quality of dental SR highlights areas of methodological strengths and weaknesses that can be targeted in future publications to encourage better quality review methodology.

     

Dental and skeletal effects of palatal expansion techniques: a systematic review of the current evidence from systematic reviews and meta‐analyses

The aim was to assess the quality and to summarise the findings of the Systematic Reviews (SRs) and Meta‐Analyses (MAs) on the dental and skeletal effects of maxillary expansion. Electronic and manual searches have been independently conducted by two investigators, up to February 2015. SRs and MAs on the dentoalveolar and skeletal effects of fixed expanders were included. The methodological quality was assessed using the AMSTAR (A Measurement Tool to Assess Systematic Reviews). The design of the primary studies included in each SR/MA was assessed with the LRD (Level of Research Design scoring). The evidence for each outcome was rated applying a pre‐determined scale. Twelve SRs/MAs were included. The AMSTAR scores ranged from 4 to 10. Two SRs/MAs included only RCTs. The current findings from SRs/MAs support with high evidence a significant increase in the short‐term of maxillary dentoalveolar transversal dimensions after Rapid Maxillary Expansion (RME). The same effect is reported with moderate evidence after Slow Maxillary Expansion (SME). However, there is moderate evidence of a non‐significant difference between the two expansion modalities concerning the short‐term dentoalveolar effects. With both RME and SME, significant increase of skeletal transversal dimension in the short‐term is reported, and the skeletal expansion is always smaller than the dentoalveolar. Even though dental relapse to some extent is present, long‐term results of the dentoalveolar effects show an increase of the transversal dimension, supported by moderate evidence for RME and low evidence for SME. Skeletal long‐term effects are reported only with RME, supported by very low evidence.     

An AMSTAR assessment of the methodological quality of systematic reviews of oral healthcare interventions published in the Journal of Applied Oral Science (JAOS)

Background

Systematic reviews are not an assembly of anecdotes but a distillation of current best available evidence on a particular topic and as such have an important role to play in evidence-based healthcare. A substantial proportion of these systematic reviews focus on interventions, and are able to provide clinicians with the opportunity to understand and translate the best available evidence on the effects of these healthcare interventions into clinical practice. The importance of systematic reviews in summarising and identifying the gaps in evidence which might inform new research initiatives is also widely acknowledged. Their potential impact on practice and research makes their methodological quality especially important as it may directly influence their utility for clinicians, patients and policy makers. The objectives of this study were to identify systematic reviews of oral healthcare interventions published in the Journal of Applied Oral Science (JAOS) and to evaluate their methodological quality using the evaluation tool, AMSTAR.

Methods

Potentially eligible systematic reviews in JAOS were identified through an electronic search of the Scientific Electronic Library Online (SciELO). Details of the relevant aspects of methodology as reported in these systematic reviews were extracted from the full text publications. Methodological quality was assessed independently by two reviewers using the AMSTAR questionnaire.

Results

Five systematic reviews were identified, one of which was subsequently excluded as it was a review of a diagnostic test. Summary AMSTAR scores for the four included reviews were: 1, 5, 2 and 4 out of a maximum score of 11 (range 1-5, mean 3) with only one of the reviews scoring 5.

Conclusion

AMSTAR evaluation of the methodological quality of the relatively small number of systematic reviews published in JAOS illustrated that there was room for improvement. Pre-publication and editorial appraisal of future systematic reviews might benefit from the application of tools such as AMSTAR and is to be recommended.     

Does non-surgical periodontal treatment influence on rheumatoid arthritis? A systematic review and meta-analysis

ObjectiveThe objective of this systematic review was to evaluate the efficacy of non-surgical periodontal therapy on rheumatoid arthritis activity.Material and methodsArticles published until April 2019 were electronically searched and screened using PubMed / MEDLINE, Scopus, and Cochrane databases according to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines (PRISMA). This study was recorded in the international PROSPERO database (CRD42019132205). The PICO question (population, intervention, comparison, results) was: in adult patients with rheumatoid arthritis and periodontitis (P), does non-surgical periodontal treatment (I), as compared to no treatment (C), provides better outcomes in rheumatoid arthritis activity (O).ResultsAfter searching the databases, seven articles were selected for qualitative and five for quantitative analysis. The total number of participants included was 292, with an average age of 50.5 years. All patients had rheumatoid arthritis and periodontal disease. Non-surgical periodontal treatment significantly reduced Disease Activity Score 28 (P = 0.004; I² = 92%) and erythrocyte sedimentation rate (P = 0.01; I² = 78%), but with no significant effect on C-reactive protein (P = 0.34; I² = 92%).ConclusionsIt can be concluded that non-surgical periodontal treatment can benefit patients with rheumatoid arthritis.