Methodological and Reporting Quality of Systematic Reviews and Meta-analyses in Endodontics

Introduction

The aim of this systematic review (SR) was to evaluate the quality of SRs and meta-analyses (MAs) in endodontics.

Effectiveness of adjunctive interventions for accelerating orthodontic tooth movement: a systematic review of systematic reviews

This study was aimed to summarise published systematic reviews (SRs) that assess the effects of adjunctive interventions on the acceleration of orthodontic tooth movement (OTM). Electronic and manual searches were performed up to August 2016. Systematic reviews investigating the impact of adjunctive techniques on the promotion of OTM were included. The methodological quality of the included reviews was evaluated using the A Measurement Tool to Assess Systematic Reviews (AMSTAR) scale. The quality of evidence for each intervention was assessed using GRADE. The Jadad decision algorithm was used to select a study to provide body evidence from discordant reviews on the same intervention. A total of 11 SRs were included in this study. AMSTAR scores ranged from 4 to 10 of 11. The quality of evidence ranged from very low to low. The short‐term (1–3 months) effects of low‐level laser therapy (LLLT, 5 and 8 J cm^?2) and corticotomy were supported by low‐quality evidence. The evidence regarding the efficacy of photobiomodulation, pulsed electromagnetic field, interseptal bone reduction, two vibrational devices (Tooth Masseuse and Orthoaccel) and electrical current was of very low quality. Relaxin injections and extracorporeal shock waves were reported to have no impact on OTM according to low‐ and very low‐quality evidence, respectively. Based on currently available information, we conclude that low‐quality evidence indicates that LLLT (5 and 8 J cm^?2) and corticotomy are effective to promote OTM in the short term. Future high‐quality trials are required to determine the optimal protocols, as well as the long‐term effects of LLLT and corticotomy, before warranting recommendations for orthodontics clinics.     

Methodological Quality Assessment of Meta-analyses and Systematic Reviews of the Relationship Between Periodontal and Systemic Diseases

ObjectivesThe aims of this article are to identify all the published systematic reviews (SRs) and meta-analyses (MAs) that studied the relationship between periodontal and systemic diseases and to assess their quality using 2 scales (the Overview Quality Assessment Questionnaire [OQAQ] and A Measurement Tool to Assess Systematic Reviews [AMSTAR] checklist).MethodsFor SRs and MAs to be included, they should have investigated one of the following systemic diseases: pulmonary conditions, cardiac conditions, endocrine conditions, cancer, blood disorders, psychological conditions, anxiety, depression, mood disorders, and several other diseases. Two investigators screened MEDLINE via PubMed, Embase, Scopus, Web of Science, and the Cochrane Database of Systematic Reviews. The tools used to evaluate quality were the AMSTAR scale and OQAQ. The protocol was prospectively registered in PROSPERO (CRD42018102208).ResultsThe search strategy found 691 unique articles, 42 of which met the eligibility criteria and were included in this review. Diabetes mellitus was the most investigated disease (14 out of 42 studies), followed by obesity (11 studies) and cardiovascular diseases (5 studies). A total of 40 reviews reported on the characteristics of included studies, and, as per the AMSTAR scale, 39 reviews had an a priori design. The number of reviews that fulfilled the status of publication criterion was the lowest (7 reviews only), followed by the number used in the assessment of publication bias (11 reviews). The number of high-quality reviews was higher with the OQAQ than with the AMSTAR checklist (33 vs 25 studies), but the AMSTAR showed a higher number of medium-quality reviews than the OQAQ (14 vs 6 studies). Both showed the same number of low-quality reviews.ConclusionsHigh-quality SRs and MAs are crucial to understanding the relationship between systemic and periodontal diseases. Medical practitioners must be able to inform patients about oral health and specific periodontal health concerns.     

Extent and quality of systematic review evidence related to minimum intervention in dentistry: essential oils, powered toothbrushes, triclosan, xylitol

Objective: To investigate extent and quality of current systematic review evidence regarding: powered toothbrushes, triclosan toothpaste, essential oil mouthwashes, xylitol chewing gum. Methods: Five databases were searched for systematic reviews until 13 November 2010. Inclusion criteria: relevant to topic, systematic review according to title and/or abstract, published in English. Article exclusion criteria were based on QUOROM recommendations for the reporting of systematic review methods. Systematic review quality was judged using the AMSTAR tool. All trials included by reviews were assessed for selection bias. Results: 119 articles were found, of which 11 systematic reviews were included. Of these, six were excluded and five accepted: one for triclosan toothpaste; one for xylitol chewing gum; two for powered toothbrushes; one for essential oil mouthwashes. AMSTAR scores: triclosan toothpaste 7; powered toothbrushes 9 and 11; xylitol chewing gum 9; essential oil mouthwashes 8. In total, 75 (out of 76) reviewed trials were identified. In‐depth assessment showed a high risk of selection bias for all trials. Conclusions: The extent of available systematic review evidence is low. Although the few identified systematic reviews could be rated as of medium and high quality, the validity of their conclusions needs to be treated with caution, owing to high risk of selection bias in the reviewed trials. High quality randomised control trials are needed in order to provide convincing evidence regarding true clinical efficacy.     

Quality Assessment of Systematic Reviews on Periodontal Regeneration in Humans

Background: Systematic reviews represent the highest form of evidence in the current hierarchy of evidence‐based dentistry. Critical analysis of published systematic reviews may help to analyze their strengths and weaknesses and to identify areas that need future improvement. The aim of this overview is to determine and compare the quality of systematic reviews published in the field of periodontal regeneration using established checklists, such as the Assessment of Multiple Systematic Reviews (AMSTAR) guidelines. Methods: A systematic search was conducted to retrieve reviews on periodontal regeneration in humans. A total of 14 systematic reviews were selected using a set of inclusion and exclusion criteria. Two independent reviewers appraised the quality of the selected reviews using AMSTAR guidelines. Each article was given an AMSTAR total score, based on the number of AMSTAR criteria that were fulfilled. The quality of included reviews was further assessed using a checklist proposed in 2003. Results: Only one of the selected systematic reviews satisfied all the AMSTAR guidelines, whereas two reviews satisfied just two of the 11 items. This study shows that published systematic reviews on periodontal regeneration exhibit significant structural and methodologic variability. Quality assessment using the additional checklist further confirmed the variability in the way systematic reviews were conducted and/or reported. Conclusion: Consideration of guidelines for quality assessment, such as AMSTAR, when designing and conducting systematic reviews may increase the validity and clinical applicability of future reviews.     

Management of TMD: evidence from systematic reviews and meta‐analyses

Summary This systematic review (SR) synthesises recent evidence and assesses the methodological quality of published SRs in the management of temporomandibular disorders (TMD). A systematic literature search was conducted in the PubMed, Cochrane Library, and Bandolier databases for 1987 to September 2009. Two investigators evaluated the methodological quality of each identified SR using two measurement tools: the assessment of multiple systematic reviews (AMSTAR) and level of research design scoring. Thirty‐eight SRs met inclusion criteria and 30 were analysed: 23 qualitative SRs and seven meta‐analyses. Ten SRs were related to occlusal appliances, occlusal adjustment or bruxism; eight to physical therapy; seven to pharmacologic treatment; four to TMJ and maxillofacial surgery; and six to behavioural therapy and multimodal treatment. The median AMSTAR score was 6 (range 2–11). Eighteen of the SRs were based on randomised clinical trials (RCTs), three were based on case–control studies, and nine were a mix of RCTs and case series. Most SRs had pain and clinical measures as primary outcome variables, while few SRs reported psychological status, daily activities, or quality of life. There is some evidence that the following can be effective in alleviating TMD pain: occlusal appliances, acupuncture, behavioural therapy, jaw exercises, postural training, and some pharmacological treatments. Evidence for the effect of electrophysical modalities and surgery is insufficient, and occlusal adjustment seems to have no effect. One limitation of most of the reviewed SRs was that the considerable variation in methodology between the primary studies made definitive conclusions impossible.     

Tooth autotransplantation: An umbrella review

Tooth autotransplantation is a versatile procedure with several clinical applications among patients across different age groups. The success of this procedure depends on multiple factors. Despite the wealth of studies available, no single primary study or systematic review is able to report on every factor affecting the outcomes of autotransplantation. The aims of this umbrella review were to evaluate treatment-related and patient-related outcomes of autotransplantation and to assess the pre-, peri- or post-operative factors that could affect these. An umbrella review was conducted according to the PRISMA statement. A literature search of five databases was performed up to 25 September 2022. Systematic Reviews (SR) with and without meta-analysis evaluating autotransplantation were included. Calibration among reviewers was carried out prior to study selection, data extraction and Risk of Bias (RoB) assessment. Study overlap was calculated using corrected covered area. Meta-meta-analysis (MMA) was performed for suitable SRs. The AMSTAR 2 critical appraisal tool was used to evaluate the quality of evidence. Seventeen SRs met the inclusion criteria. Only two SRs were suitable for conduct of MMA on autotransplantation of open apex teeth. The 5-year and 10-year survival rates were >95%. A narrative summary on factors that could affect autotransplantation outcomes and comparisons of autotransplantation to other treatment options were reported. Five SRs were rated as ‘low quality’ and 12 SRs were rated as ‘critically low quality’ in the AMSTAR 2 RoB assessment. In order to facilitate a more homogenous pool of data for subsequent meta-analysis, an Autotransplantation Outcome Index was also proposed to standardise the definition of outcomes. Autotransplantation of teeth with open apices have a high survival rate. Future studies should standardise the reporting of clinical and radiographic findings, as well as the definition of outcomes.     

Methodological Quality Assessment of Meta-analyses in Endodontics

Introduction

The objectives of this review were to assess the methodological quality of published meta-analyses related to endodontics using the assessment of multiple systematic reviews (AMSTAR) tool and to provide a follow-up to previously published reviews.

Current status of the reporting quality of abstracts in systematic reviews related to implant dentistry: a literature survey

The aim of this study was to assess the reporting quality of abstracts in systematic reviews (SRs) related to implant dentistry and to assess the possible factors associated with the reporting quality. Abstracts of SRs in the field of implant dentistry, published in the last 5 years, were searched. The reporting quality was assessed and scored using the PRISMA for Abstracts checklist (PRISMA-A). The overall PRISMA-A score (OPS) and relative score (OPS%) per review were calculated according to adherence to the criteria presented in the checklist. Multivariable linear regression was performed to identify possible factors associated with reporting quality. Overall, 310 SRs were eligible for this study. Based on the maximum PRISMA-A score (score of 12), the mean OPS was 6.5 and OPS% was 54.2%. The items ‘title’, ‘objectives’, and ‘number of included studies’ were those most frequently reported in the abstracts, while the items ‘registration’ and ‘funding’ were the least reported. According to multivariable linear regression, the geographical origin of the articles was the only factor associated with better quality of abstract reporting, with higher OPS for SRs from Europe when compared to North America (coefficient 0.73; P = 0.049). The reporting quality of abstracts in SRs related to implant dentistry is suboptimal and needs to be improved. Journals should encourage adherence to reporting checklists in SRs.     

Evidence-based orthodontics, still a long way to go

Clinical performance can be kept up to date by learning how to practice evidence-based orthodontics, by seeking and appraising evidence-based summaries from the literature and by applying evidence-based strategies to change clinical behaviour. A MEDLINE search over the period 1990-2000 identified 8345 publications on clinical orthodontic subjects. Of these articles 49.5% was published in five specific orthodontic journals, while the others were published in about seventy other journals making it difficult for the clinician to stay current easily. Systematic reviews are an efficient and reliable source of information, but due to a lack of well-designed randomised clinical trials systematic reviews in orthodontics are still rare.     

An AMSTAR assessment of the methodological quality of systematic reviews of oral healthcare interventions published in the Journal of Applied Oral Science (JAOS)

Background

Systematic reviews are not an assembly of anecdotes but a distillation of current best available evidence on a particular topic and as such have an important role to play in evidence-based healthcare. A substantial proportion of these systematic reviews focus on interventions, and are able to provide clinicians with the opportunity to understand and translate the best available evidence on the effects of these healthcare interventions into clinical practice. The importance of systematic reviews in summarising and identifying the gaps in evidence which might inform new research initiatives is also widely acknowledged. Their potential impact on practice and research makes their methodological quality especially important as it may directly influence their utility for clinicians, patients and policy makers. The objectives of this study were to identify systematic reviews of oral healthcare interventions published in the Journal of Applied Oral Science (JAOS) and to evaluate their methodological quality using the evaluation tool, AMSTAR.

Methods

Potentially eligible systematic reviews in JAOS were identified through an electronic search of the Scientific Electronic Library Online (SciELO). Details of the relevant aspects of methodology as reported in these systematic reviews were extracted from the full text publications. Methodological quality was assessed independently by two reviewers using the AMSTAR questionnaire.

Results

Five systematic reviews were identified, one of which was subsequently excluded as it was a review of a diagnostic test. Summary AMSTAR scores for the four included reviews were: 1, 5, 2 and 4 out of a maximum score of 11 (range 1-5, mean 3) with only one of the reviews scoring 5.

Conclusion

AMSTAR evaluation of the methodological quality of the relatively small number of systematic reviews published in JAOS illustrated that there was room for improvement. Pre-publication and editorial appraisal of future systematic reviews might benefit from the application of tools such as AMSTAR and is to be recommended.     

Meta-analysis for orthodontists: Part II – Is all that glitters gold?

Abstract

Systematic reviews with quantitative synthesis of included studies (meta-analysis) are being produced at increased rates in orthodontics. However, their reporting quality is often suboptimal, and methodological limitations can affect their conclusions. This article is designed to help orthodontists critically appraise the validity of a meta-analysis by providing 11 questions that address the planning, conduct and interpretation of these investigations.     

Systematic reviews in orthodontics: what have we learned?

The literature is becoming replete with systematic reviews of orthodontic-related topics. However, their findings have not been appraised and summarised collectively. METHODS: Systematic reviews related to orthodontics published between 1 January 2000, and 31 January 2007 were identified and reviewed. RESULTS: Fifty-one orthodontic systematic reviews were identified in the defined period. A highly significant percentage of reviews (29%) failed to reach conclusions. Conclusions were made in the remaining reviews although many relied on secondary levels of evidence to do so. In this narrative review the findings from these studies are summarised. CONCLUSIONS: Systematic reviews in orthodontic-related areas have produced some interesting findings. However an unacceptably high number continue to be inconclusive, reflecting methodological inadequacies of the review process and exposing inadequacies in our evidence base.     

Do fixed orthodontic appliances adversely affect the periodontium? A systematic review of systematic reviews

This overview aimed to summarize the available systematic reviews that assess the effects of treatment with fixed orthodontic appliances (FOAs) on the periodontium. Unrestricted electronic search of nine databases and additional manual searches were performed up to January 2019. Systematic reviews and meta analyses that assessed the effect of FOAs on the periodontal parameters were included. The methodological quality of the included reviews was evaluated using the A Measurement Tool to Assess Systematic Reviews2 (AMSTAR 2). The initial search yielded 2529 articles from which 19 were included in the current study. AMSTAR 2 scores ranged from critically low to high quality. The quality of evidence ranged from very low to low. The superiority of the periodontal outcomes of self-ligating brackets over conventional brackets could not be proven. The available evidence regarding the effects of FOAs on the periodontium is controversial and of very low quality. The short-term effects of FOAs were temporary worsening the periodontal parameters. Some conclusions regarding the periodontal outcomes of self-ligating brackets could be withdrawn. Future high-quality trials are required. The review was registered at the International prospective register of systematic reviews with registration number CRD42018106662.     

Class II functional orthopaedic treatment: a systematic review of systematic reviews

This Systematic Review (SR) aims to assess the quality of SRs and Meta‐Analyses (MAs) on functional orthopaedic treatment of Class II malocclusion and to summarise and rate the reported effects. Electronic and manual searches were conducted until June 2014. SRs and MAs focusing on the effects of functional orthopaedic treatment of Class II malocclusion in growing patients were included. The methodological quality of the included papers was assessed using the AMSTAR (Assessment of Multiple Systematic Reviews). The design of the primary studies included in each SR was assessed with Level of Research Design scoring. The evidence of the main outcomes was summarised and rated according to a scale of statements. 14 SRs fulfilled the inclusion criteria. The appliances evaluated were as follows: Activator (2 studies), Twin Block (4 studies), headgear (3 studies), Herbst (2 studies), Jasper Jumper (1 study), Bionator (1 study) and Fränkel‐2 (1 study). Four studies reviewed several functional appliances, as a group. The mean AMSTAR score was 6 (ranged 2–10). Six SRs included only controlled clinical trials (CCTs), three SRs included only randomised controlled trials (RCTs), four SRs included both CCTs and RCTs and one SR included also expert opinions. There was some evidence of reduction of the overjet, with different appliances except from headgear; there was some evidence of small maxillary growth restrain with Twin Block and headgear; there was some evidence of elongation of mandibular length, but the clinical relevance of this results is still questionable; there was insufficient evidence to determine an effect on soft tissues.     

多系统评价评估问卷量表在口腔医学领域中文系统评价应用中的信度和效度评价

目的 评价多系统评价评估问卷（AMSTAR）量表应用于口腔医学领域中文系统评价中的一致性、信度和效度。方法 计算机检索中国生物医学文献数据库、维普中文科技期刊数据库和中国期刊全文数据库,截止日期为 2011年3月1日。手工检索19种中文口腔医学杂志,检索已发表的口腔医学类系统评价。2名评价者分别用总体质量 评估问卷（OQAQ）量表和AMSTAR量表对系统评价进行评价,计算观察者间使用AMSTAR量表的Kappa值,AMSTAR 量表重测信度的级内相关系数（ICC）以及AMSTAR和OQAQ量表得分的最大得分百分比的级内相关系数（结构效度）。 结果 纳入52篇系统评价文献。评价者使用AMSTAR量表的Kappa值为0.81[95%C（I 0.73,0.89）],使用OQAQ量表的 Kappa 值为0.74 [95% CI（0.66,0.83）] 。 重测信度的ICC 为 0.98 [95% CI（0.97,0.99）,P =0.000]。 内部一致性信度Cronbach’α为0.69[95%CI（0.56,0.80）,P=0.000]。AMSTAR和OQAQ量表最大得分百分比的ICC为0.94[95%CI（0.90, 0.97）,P=0.000]。结论 AMSTAR量表在应用于口腔医学领域系统评价时有很好的一致性、信度和效度。AMSTAR 量表可很好的推广至口腔医学领域进行系统评价的方法学质量评价,为医务工作者进行系统评价方法学质量评价时带来了较大的便利。     

Dental and skeletal effects of palatal expansion techniques: a systematic review of the current evidence from systematic reviews and meta‐analyses

The aim was to assess the quality and to summarise the findings of the Systematic Reviews (SRs) and Meta‐Analyses (MAs) on the dental and skeletal effects of maxillary expansion. Electronic and manual searches have been independently conducted by two investigators, up to February 2015. SRs and MAs on the dentoalveolar and skeletal effects of fixed expanders were included. The methodological quality was assessed using the AMSTAR (A Measurement Tool to Assess Systematic Reviews). The design of the primary studies included in each SR/MA was assessed with the LRD (Level of Research Design scoring). The evidence for each outcome was rated applying a pre‐determined scale. Twelve SRs/MAs were included. The AMSTAR scores ranged from 4 to 10. Two SRs/MAs included only RCTs. The current findings from SRs/MAs support with high evidence a significant increase in the short‐term of maxillary dentoalveolar transversal dimensions after Rapid Maxillary Expansion (RME). The same effect is reported with moderate evidence after Slow Maxillary Expansion (SME). However, there is moderate evidence of a non‐significant difference between the two expansion modalities concerning the short‐term dentoalveolar effects. With both RME and SME, significant increase of skeletal transversal dimension in the short‐term is reported, and the skeletal expansion is always smaller than the dentoalveolar. Even though dental relapse to some extent is present, long‐term results of the dentoalveolar effects show an increase of the transversal dimension, supported by moderate evidence for RME and low evidence for SME. Skeletal long‐term effects are reported only with RME, supported by very low evidence.