Potential Implications of Recent and Proposed Changes in the Regulatory Oversight of Solid Organ Transplantation in the United States

Every 6 months, the Scientific Registry of Transplant Recipients (SRTR) publishes evaluations of every solid organ transplant program in the United States, including evaluations of 1‐year patient and graft survival. The Centers for Medicare & Medicaid Services (CMS) and the Organ Procurement and Transplantation Network (OPTN) Membership and Professional Standards Committee (MPSC) use SRTR's 1‐year evaluations for regulatory review of transplant programs. Concern has been growing that the regulatory scrutiny of transplant programs with lower‐than‐expected outcomes is harmful, causing programs to undertake fewer high‐risk transplants and leading to unnecessary organ discards. As a result, CMS raised its threshold for a “Condition‐Level Deficiency” designation of observed relative to expected 1‐year graft or patient survival from 1.50 to 1.85. Exceeding this threshold in the current SRTR outcomes report and in one of the four previous reports leads to scrutiny that may result in loss of Medicare funding. For its part, OPTN is reviewing a proposal from the MPSC to also change its performance criteria thresholds for program review, to review programs with “substantive clinical differences.” We review the details and implications of these changes in transplant program oversight.     

SRTR Center-Specific Reporting Tools: Posttransplant Outcomes

Measuring and monitoring performance—be it waiting list and posttransplant outcomes by a transplant center, or organ donation success by an organ procurement organization and its partnering hospitals—is an important component of ensuring good care for people with end-stage organ failure. Many parties have an interest in examining these outcomes, from patients and their families to payers such as insurance companies or the Centers for Medicare and Medicaid Services; from primary caregivers providing patient counseling to government agencies charged with protecting patients.
The Scientific Registry of Transplant Recipients produces regular, public reports on the performance of transplant centers and organ procurement organizations. This article explains the statistical tools used to prepare these reports, with a focus on graft survival and patient survival rates of transplant centers—especially the methods used to fairly and usefully compare outcomes of centers that serve different populations. The article concludes with a practical application of these statistics—their use in screening transplant center performance to identify centers that may need remedial action by the OPTN/UNOS Membership and Professional Standards Committee.     

CMS oversight,OPOs and transplant centers and the law of unintended consequences

Abstract: The Health Resources and Services Administration launched collaboratives with the goals of increasing donation rates, increasing the number of organs transplanted, eliminating deaths on the waiting list and improving outcomes. The Center for Medicare and Medicaid Services (CMS) recently published requirements for organ procurement organizations (OPOs) and transplant centers. Failure to meet CMS performance measures could result in OPOs losing their service area or transplant centers losing their CMS certification. CMS uses analyses by the Scientific Registry of Transplant Recipients (SRTR) to evaluate a transplant center’s performance based on risk‐adjusted outcomes. However, CMS also uses a more liberal (one‐sided) statistical test rendering more centers likely to qualify as low performing. Furthermore, the SRTR model does not incorporate some important patient variables in its statistical model which may result in biased determinations of quality of care. Cumulatively, there is much unexplained variation for transplant outcomes as suggested by the low predictive ability of survival models compared to other disease contexts. OPOs and transplant centers are unlikely to quietly accept their elimination. They may take certain steps that can result in exclusion of candidates who might otherwise benefit from transplantation and/or result in fewer transplants through restricted use of organs thought to carry higher risk of failure. CMS should join with transplant organizations to ensure that the goals of the collaborative are not inhibited by their performance measures.     

Scientific Registry of Transplant Recipients: Collecting,analyzing, and reporting data on transplantation in the United States

Founded in 1987, the Scientific Registry of Transplant Recipients (SRTR) operates under a contract from the US government administered by the Health Resources and Services Administration (HRSA). SRTR maintains a database of comprehensive information on all solid organ transplantation in the US. The registry supports the ongoing evaluation of the clinical status of solid organ transplantation, including kidney, heart, liver, lung, intestine, pancreas, and multi-organ transplants. Data in the registry are from multiple sources, but most are collected by the Organ Procurement and Transplantation Network (OPTN) from hospitals, organ procurement organizations, and immunology laboratories. The data include information on current and past organ donors, transplant candidates, transplant recipients, transplant outcomes, and outcomes of living donors. SRTR uses these data to create reports and analyses for HRSA, OPTN committees that make organ allocation policy, and the Centers for Medicare & Medicaid Services to carry out quality assurance surveillance activities; SRTR also creates standard analysis files for scientific investigators. In addition, SRTR and OPTN produce an Annual Data Report and provide information upon request for the general public. Thus, SRTR supports the transplant community with information services and statistical analyses to improve patient access to and outcomes of organ transplant.     

The Challenging Triangle: Balancing Outcomes, Transplant Numbers and Costs

Transplant centers juggle at least three objectives important to success: transplant volume, outcomes and costs. Improving one frequently occurs at the expense of the others. It may be difficult to have a high-volume program with excellent outcomes and low costs. There are some programs that can do well in all these areas, but even they might be able to improve one of the outcome measures if they would compromise on the one of the others (i.e. a transplant center with good outcomes might be able to achieve even better outcomes if it restricted the patients it was willing to transplant or used only the best organs). It is up to transplant surgeons and physicians to balance all these objectives.     

Pediatric transplantation, 1994–2003

This article uses OPTN/SRTR data to review trends in pediatric transplantation over the last decade. In 2003, children younger than 18 made up 3% of the 82,885 candidates for organ transplantation and 7% of the 25,469 organ transplant recipients. Children accounted for 14% of the 6,455 deceased organ donors. Pediatric organ transplant recipients differ from their adult counterparts in several important aspects, including the underlying etiology of organ failure, the complexity of the surgical procedures, the pharmacokinetic properties of common immunosuppressants, the immune response following transplantation, the number and degree of comorbid conditions, and the susceptibility to post-transplant complications, especially infectious diseases. Specialized pediatric organ transplant programs have been developed to address these special problems, The transplant community has responded to the particular needs of children and has provided them special consideration in the allocation of deceased donor organs. As a result of these programs and protocols, children are now frequently the most successful recipients of organ transplantation; their outcomes following kidney, liver, and heart transplantation rank among the best. This article demonstrates that substantial improvement is needed in several areas: adolescent outcomes, outcomes following intestine transplants, and waiting list mortality among pediatric heart and lung candidates,     

Seeking new answers to old questions about public reporting of transplant program performance in the United States

The Scientific Registry of Transplant Recipients (SRTR) is mandated by the National Organ Transplant Act, the Final Rule, and the SRTR contract with the Health Resources and Services Administration to report program‐specific information on the performance of transplant programs. Following a consensus conference in 2012, SRTR developed a new version of the public website to improve public reporting of often complex metrics, including changing from a 3‐tier to a 5‐tier summary metric for first‐year posttransplant survival. After its release in December 2016, the new presentation was moved to a “beta” website to allow collection of additional feedback. SRTR made further improvements and released a new beta website in May 2018. In response to feedback, SRTR added 5‐tier summaries for standardized waitlist mortality and deceased donor transplant rate ratios, along with an indicator of which metric most affects survival after listing. Presentation of results was made more understandable with input from patients and families from surveys and focus groups. Room for improvement remains, including continuing to make the data more useful to patients, deciding what additional data elements should be collected to improve risk adjustment, and developing new metrics that better reflect outcomes most relevant to patients.     

Using root cause analysis to improve survival in a liver transplant program

BACKGROUND: With the advent of programs such as the American College of Surgeons-National Surgical Quality Improvement Program, surgical services will be compared with their peers across the United States. At times, many programs will experience lower-than-expected outcomes. During July 1, 1998, to June 30, 2000 our 1-year graft (76.86%, P = 0.23) and patient (80.61%, P = 0.016) survivals after liver transplantation were lower than our expected rates (graft 81.89% and patient 88.3%), according to the U.S. Scientific Registry of Transplant Recipients (SRTR). METHODS: We used aggregate root cause analysis to determine underlying reasons for our patient deaths. Two of our surgeons performed a systematic review of all our center's liver transplant patient deaths from January 1, 1995, to December 31, 2000. Each phase of the transplant process was reviewed. RESULTS: Of 355 patients receiving their first transplant, there were 90 deaths, with 188 root causes identified. The apportionment according to phase of the transplant process was patient selection, 50%; transplant procedure, 17%; donor selection, 15%; post-transplant care, 8%, and psychosocial issues, 10%. Risk reduction action plans were developed, and several important changes made in our care protocol. In April 2004, SRTR data revealed that for patients transplanted between January 1, 2001 and June 30, 2003, our 1-year liver graft survival of 90.73% (P = 0.018) was significantly higher than the national expected rate of 84.48%. Our 1-year patient survival rate of 92.66% (P = 0.285) was higher than the expected rate of 89.29%. CONCLUSIONS: Lower-than-expected outcomes can provide an impetus for improving patient care and raising the quality of a surgical service. Aggregate root cause analysis of adverse events is a valuable method for program improvement.     

Report of a Consensus Conference on Transplant Program Quality and Surveillance

Public reports of organ transplant program outcomes by the US Scientific Registry of Transplant Recipients have been both groundbreaking and controversial. The reports are used by regulatory agencies, private insurance providers, transplant centers and patients. Failure to adequately adjust outcomes for risk may cause programs to avoid performing transplants involving suitable but high‐risk candidates and donors. At a consensus conference of stakeholders held February 13–15, 2012, the participants recommended that program‐specific reports be better designed to address the needs of all users. Additional comorbidity variables should be collected, but innovation should also be protected by excluding patients who are in approved protocols from statistical models that identify underperforming centers. The potential benefits of hierarchical and mixed‐effects statistical methods should be studied. Transplant centers should be provided with tools to facilitate quality assessment and performance improvement. Additional statistical methods to assess outcomes at small‐volume transplant programs should be developed. More data on waiting list risk and outcomes should be provided. Monitoring and reporting of short‐term living donor outcomes should be enhanced. Overall, there was broad consensus that substantial improvement in reporting outcomes of transplant programs in the United States could and should be made in a cost‐effective manner.     

Reported effects of the Scientific Registry of Transplant Recipients 5‐tier rating system on US transplant centers: results of a national survey

In the United States, the Scientific Registry of Transplant Recipients (SRTR) provides publicly available quality report cards. These reports have historically rated transplant programs using a 3‐tier system. In 2016, the SRTR temporarily transitioned to a 5‐tier system, which classified more programs as under‐performing. As part of a larger survey about transplant quality metrics, we surveyed members of the American Society of Transplant Surgeons and American Society of Transplantation (N = 280 respondents) on transplant center experiences with patient and payer responses to the 5‐tier SRTR ratings. Over half of respondents (n = 137, 52.1%) reported ≥1 negative effect of the new 5‐tier ranking system, including losing patients, losing insurers, increased concern among patients, and increased concern among referring providers. Few respondents (n = 35, 13.7%) reported any positive effects of the 5‐tier ranking system. Lower SRTR‐reported scores on the 5‐tier scale were associated with increased risk of reporting at least one negative effect in a logistic model (P < 0.01). The change to a more granular rating system provoked an immediate response in the transplant community that may have long‐term implications for transplant hospital finances and patient options for transplantation.     

Racial differences in outcomes of the evaluation of potential live kidney donors: a retrospective cohort study

Background: In the USA, the lower rate of live donor kidney transplant among Black transplant candidates may stem from lower rates of donation among potential live donors who are Black. We determined whether outcomes of the evaluation of potential live kidney donors varied according to the potential donors' demographic characteristics. Methods: We performed a single-center, retrospective observational cohort study of 1,179 potential live kidney donors, who came forward between 2000 and 2007. Potential donors' intended recipients were first-time transplant recipients who were evaluated between 2000 and 2005. Results: There were 268 (22.7%) potential live kidney donors who were Black, of whom 93.7% were recruited by Black transplant candidates. Donor outcomes included actual donation (38.3%), exclusion due to blood group or crossmatch incompatibility (20.4%), exclusion due to medical contraindication to donation (13.7%), and lack of further donor interest (11.2%). Black (vs. non-Black) potential donors were less likely to actually donate (27.2 vs. 41.6%, p < 0.001). Black potential donors were more likely to stop pursuing live donation (p = 0.047) or be excluded from donation for medical reasons (p = 0.008) or blood group or crossmatch incompatibility (p = 0.01). These racial differences persisted in a multivariable multinomial logistic regression model of factors associated with outcomes of the donor evaluation. Conclusions: Potential live kidney donors who are Black are less likely to actually donate. Future studies should determine whether paired exchange and desensitization programs decrease these racial differences and why Black potential donors appear more likely to stop pursuing live donation.     

Comparison of Cancer Diagnoses Between the US Solid Organ Transplant Registry and Linked Central Cancer Registries

US transplant centers are required to report cancers in transplant recipients to the transplant network. The accuracy and completeness of these data, collected in the Scientific Registry of Transplant Recipients (SRTR), are unknown. We compared diagnoses in the SRTR and 15 linked cancer registries for colorectal, liver, lung, breast, prostate and kidney cancers; melanoma; and non‐Hodgkin lymphoma (NHL). Among 187 384 transplants, 9323 cancers were documented in the SRTR or cancer registries. Only 36.8% of cancers were in both, with 47.5% and 15.7% of cases additionally documented solely in cancer registries or the SRTR, respectively. Agreement between the SRTR and cancer registries varied (kappa = 0.28 for liver cancer and kappa = 0.52–0.66 for lung, prostate, kidney, colorectum, and breast cancers). Upon evaluation, some NHLs documented only in cancer registries were identified in the SRTR as another type of posttransplant lymphoproliferative disorder. Some SRTR‐only cases were explained by miscoding (colorectal cancer instead of anal cancer, metastases as lung or liver cancers) or missed matches with cancer registries, partly due to recipients' outmigration from catchment areas. Estimated sensitivity for identifying cancer was 52.5% for the SRTR and 84.3% for cancer registries. In conclusion, SRTR cancer data are substantially incomplete, limiting their usefulness for surveillance and research.     

Kidney offer acceptance at programs undergoing a Systems Improvement Agreement

In the United States, the Centers for Medicare and Medicaid Services (CMS) use Systems Improvement Agreements (SIAs) to require transplant programs repeatedly flagged for poor‐performance to improve performance or lose CMS funding for transplants. We identified 14 kidney transplant (KT) programs with SIAs and 28 KT programs without SIAs matched on waitlist volume and characterized kidney acceptance using SRTR data from 12/2006‐3/2015. We used difference‐in‐differences linear regression models to identify changes in acceptance associated with an SIA independent of program variation and trends prior to the SIA. SIA programs accepted 26.9% and 22.1% of offers pre‐ and post‐SIA, while non‐SIA programs accepted 33.9% and 44.4% of offers in matched time periods. After adjustment for donor characteristics, time‐varying waitlist volume, and secular trends, SIAs were associated with a 5.9 percentage‐point (22%) decrease in kidney acceptance (95% CI: ?10.9 to ?0.8, P = .03). The decrease in acceptance post‐SIA was more pronounced for KDPI 0‐40 kidneys (12.3 percentage‐point decrease, P = .007); reductions in acceptance of higher KDPI kidneys occurred pre‐SIA. Programs undergoing SIAs substantially reduced acceptance of kidney offers for waitlisted candidates. Attempts to improve posttransplant outcomes might have the unintended consequence of reducing access to transplantation as programs adopt more restrictive organ selection practices.     

Time for reform in transplant program–specific reporting: AST/ASTS transplant metrics taskforce

In accordance with the National Organ Transplant Act and Department of Health and Human Services’ Final Rule, the Scientific Registry of Transplant Recipients (SRTR) publicly releases biannual program‐specific reports that include analyses of transplant centers’ risk‐adjusted waitlist mortality, organ acceptance ratios, transplant rates, and graft and patient survival . Since the inception of these center metrics, 1‐year posttransplant graft and patient survival have improved, and center variation has decreased, casting uncertainty on their clinical relevance. The SRTR has recently modified center evaluations by ranking centers into 5 tiers rather than 3 tiers in an attempt to discriminate between programs performing within a tight range, further exacerbating this uncertainty. The American Society of Transplantation/American Society of Transplant Surgeons convened an expert taskforce to examine both the utility and unintended consequences of transplant center metrics. Estimates of center variation in outcomes in adjacent tiers are imprecise and fleeting, but can result in consequential changes in clinician and center behavior. The taskforce has concerns that current metrics, based principally on 1‐year graft and patient survival, provide minimal if any benefit in informing patient choice and access to transplantation, with the untoward effect of decreased utilization of organs and restriction of research and innovation.     

Reliability of voluntary and compulsory databases and registries in the United States

In the United States, there are several transplant registries and databases. Most are voluntary and may be organ specific, disease or condition specific, organ and disease specific, or specific for a certain demographic patient group. There is one compulsory and comprehensive registry, the Scientific Registry of Transplant Recipients (SRTR). These databases all have strengths and weaknesses. There is no doubt, however, that they contribute significantly to the scientific analyses of transplant outcomes and complications and provide important data through publications and presentations that can improve patient care and influence local, regional, and national transplant policies. Audits can and should be performed to guarantee the reliability of the data and accuracy of the conclusions drawn from this data as was done by the European Liver Transplant Registry.     

Database Comparison of the Adult‐to‐Adult Living Donor Liver Transplantation Cohort Study (A2ALL) and the SRTR U.S. Transplant Registry†,‡

Data submitted by transplant programs to the Organ Procurement and Transplantation Network (OPTN) are used by the Scientific Registry of Transplant Recipients (SRTR) for policy development, performance evaluation and research. This study compared OPTN/SRTR data with data extracted from medical records by research coordinators from the nine‐center A2ALL study. A2ALL data were collected independently of OPTN data submission (48 data elements among 785 liver transplant candidates/recipients; 12 data elements among 386 donors). At least 90% agreement occurred between OPTN/SRTR and A2ALL for 11/29 baseline recipient elements, 4/19 recipient transplant or follow‐up elements and 6/12 donor elements. For the remaining recipient and donor elements, >10% of values were missing in OPTN/SRTR but present in A2ALL, confirming that missing data were largely avoidable. Other than variables required for allocation, the percentage missing varied widely by center. These findings support an expanded focus on data quality control by OPTN/SRTR for a broader variable set than those used for allocation. Center‐specific monitoring of missing values could substantially improve the data.     

Program‐specific transplant rate ratios: Association with allocation priority at listing and posttransplant outcomes

The Scientific Registry of Transplant Recipients (SRTR) is considering more prominent reporting of program‐specific adjusted transplant rate ratios (TRRs). To enable more useful reporting of TRRs, SRTR updated the transplant rate models to adjust explicitly for components of allocation priority. We evaluated potential associations between TRRs and components of allocation priority that could indicate programs' ability to manipulate TRRs by denying or delaying access to low‐priority candidates. Despite a strong association with unadjusted TRRs, we found no candidate‐level association between the components of allocation priority and adjusted TRRs. We found a strong program‐level association between median laboratory Model for End‐stage Liver Disease (MELD) score at listing and program‐specific adjusted TRRs (r = .37; P < .001). The program‐level association was likely confounded by regional differences in donor supply/demand and listing practices. In kidney transplantation, higher program‐specific adjusted TRRs were weakly associated with better adjusted posttransplant outcomes (r = ?.14; P = .035) and lower adjusted waitlist mortality rate ratios (r = ?.15; P = .022), but these associations were absent in liver, lung, and heart transplantation. Program‐specific adjusted TRRs were unlikely to be improved by listing candidates with high allocation priority and can provide useful information for transplant candidates and programs.     

Transplant data: sources, collection and research considerations, 2004

The process of collecting and analyzing transplant data is complex. Familiarity with how these data are collected is crucial to a thorough understanding of the information. This article focuses on available OPTN-SRTR data and the continuing evolution of data collection mechanisms; how that data collection system is improving the data quality and reducing the data collection burden; how additional ascertainment of outcomes both completes and validates existing data; and caveats that remain for researchers. This year's article focuses further on research considerations related to cohort choice, timing of data submission, and potential biases in follow-up data. Ongoing improvements in data collection timeliness and scope are covered. The impact of extra ascertainment of outcomes, particularly for post-transplant kidney graft failure from Medicare data, are also examined. A section on graft failure reporting among different sources traces the steps by which the SRTR reconciles different data sources in its analyses. It is important that those reading and conducting transplant research understand the origin, structure, and scope of the available data. All of these issues should be carefully considered when choosing cohorts and data sources for analysis.