Ensemble learning for poor prognosis predictions: A case study on SARS-CoV-2

ObjectiveRisk prediction models are widely used to inform evidence-based clinical decision making. However, few models developed from single cohorts can perform consistently well at population level where diverse prognoses exist (such as the SARS-CoV-2 [severe acute respiratory syndrome coronavirus 2] pandemic). This study aims at tackling this challenge by synergizing prediction models from the literature using ensemble learning.Materials and MethodsIn this study, we selected and reimplemented 7 prediction models for COVID-19 (coronavirus disease 2019) that were derived from diverse cohorts and used different implementation techniques. A novel ensemble learning framework was proposed to synergize them for realizing personalized predictions for individual patients. Four diverse international cohorts (2 from the United Kingdom and 2 from China; N = 5394) were used to validate all 8 models on discrimination, calibration, and clinical usefulness.ResultsResults showed that individual prediction models could perform well on some cohorts while poorly on others. Conversely, the ensemble model achieved the best performances consistently on all metrics quantifying discrimination, calibration, and clinical usefulness. Performance disparities were observed in cohorts from the 2 countries: all models achieved better performances on the China cohorts.DiscussionWhen individual models were learned from complementary cohorts, the synergized model had the potential to achieve better performances than any individual model. Results indicate that blood parameters and physiological measurements might have better predictive powers when collected early, which remains to be confirmed by further studies.ConclusionsCombining a diverse set of individual prediction models, the ensemble method can synergize a robust and well-performing model by choosing the most competent ones for individual patients.     

Development of a “meta-model” to address missing data,predict patient-specific cancer survival and provide a foundation for clinical decision support

ObjectiveLike most real-world data, electronic health record (EHR)–derived data from oncology patients typically exhibits wide interpatient variability in terms of available data elements. This interpatient variability leads to missing data and can present critical challenges in developing and implementing predictive models to underlie clinical decision support for patient-specific oncology care. Here, we sought to develop a novel ensemble approach to addressing missing data that we term the “meta-model” and apply the meta-model to patient-specific cancer prognosis.Materials and MethodsUsing real-world data, we developed a suite of individual random survival forest models to predict survival in patients with advanced lung cancer, colorectal cancer, and breast cancer. Individual models varied by the predictor data used. We combined models for each cancer type into a meta-model that predicted survival for each patient using a weighted mean of the individual models for which the patient had all requisite predictors.ResultsThe meta-model significantly outperformed many of the individual models and performed similarly to the best performing individual models. Comparisons of the meta-model to a more traditional imputation-based method of addressing missing data supported the meta-model’s utility.ConclusionsWe developed a novel machine learning–based strategy to underlie clinical decision support and predict survival in cancer patients, despite missing data. The meta-model may more generally provide a tool for addressing missing data across a variety of clinical prediction problems. Moreover, the meta-model may address other challenges in clinical predictive modeling including model extensibility and integration of predictive algorithms trained across different institutions and datasets.     

Importance-aware personalized learning for early risk prediction using static and dynamic health data

ObjectiveAccurate risk prediction is important for evaluating early medical treatment effects and improving health care quality. Existing methods are usually designed for dynamic medical data, which require long-term observations. Meanwhile, important personalized static information is ignored due to the underlying uncertainty and unquantifiable ambiguity. It is urgent to develop an early risk prediction method that can adaptively integrate both static and dynamic health data.Materials and MethodsData were from 6367 patients with Peptic Ulcer Bleeding between 2007 and 2016. This article develops a novel End-to-end Importance-Aware Personalized Deep Learning Approach (eiPDLA) to achieve accurate early clinical risk prediction. Specifically, eiPDLA introduces a long short-term memory with temporal attention to learn sequential dependencies from time-stamped records and simultaneously incorporating a residual network with correlation attention to capture their influencing relationship with static medical data. Furthermore, a new multi-residual multi-scale network with the importance-aware mechanism is designed to adaptively fuse the learned multisource features, automatically assigning larger weights to important features while weakening the influence of less important features.ResultsExtensive experimental results on a real-world dataset illustrate that our method significantly outperforms the state-of-the-arts for early risk prediction under various settings (eg, achieving an AUC score of 0.944 at 1 year ahead of risk prediction). Case studies indicate that the achieved prediction results are highly interpretable.ConclusionThese results reflect the importance of combining static and dynamic health data, mining their influencing relationship, and incorporating the importance-aware mechanism to automatically identify important features. The achieved accurate early risk prediction results save precious time for doctors to timely design effective treatments and improve clinical outcomes.     

儿童再发性泌尿道感染的危险因素及预测模型的构建和验证

目的 探讨儿童再发性泌尿道感染的危险因素，并建立其感染风险预测模型，为临床提供便捷有效的预测方法及预防措施。方法 选取2020年8月至2021年7月在河南省儿童医院泌尿外科确诊为泌尿道感染患儿237例，依据随访结果，未再发生泌尿道感染患儿205例为非再发组，复发或再发泌尿道感染患儿32例为再发组。收集两组患儿临床资料，通过单因素和多因素logistic回归分析患儿再发性泌尿道感染的影响因素，并构建其再发性泌尿道感染风险预测模型，通过受试者工作特征（ROC）曲线和校准曲线进行模型评价，采用Hosmer-Lemeshow拟合优度检验。结果 237例泌尿道感染患儿经随访发现，再发性泌尿道感染的发生率为13.50%；再发组的贫血、过敏体质、便秘、膀胱输尿管反流和其他类型泌尿系统畸形患儿数占比高于非再发组（P均<0.05），再发组的IgA和IgG水平降低及病原菌为大肠埃希菌患儿数占比明显高于非再发组（P<0.05）。多因素logistic回归分析结果显示，女性、膀胱输尿管反流、其他类型泌尿系统畸形、过敏体质、便秘、IgA水平降低、IgG水平降低、贫血、病原菌（大肠埃希菌）是儿童再发性...     

临床医学专业课程学习者画像模型构建

目的 以“超声在麻醉、疼痛和重症医学中的应用”课程为例，构建临床医学专业课程学习者画像模型。方法 梳理临床医学专业课程学习者画像模型构建框架及流程，对多来源学习者数据进行收集及预处理。利用Python 3.9编程语言进行统计分析、自然语言处理、聚类分析，并以可视化技术呈现，构建机器学习预测模型并评估模型预测效能。结果 从学习背景、兴趣偏好和行为效果3个维度构建临床医学专业课程学习者画像模型。学习背景画像揭示了学习者基本信息、认知基础和学习动机。兴趣偏好画像分析了学习目的与其他选课信息，根据内容关注程度、学习效果影响因素识别出3类不同的学习者群体。行为效果画像通过构建4种机器学习算法预测模型实现了对课程考试成绩的分类预测，结果显示朴素贝叶斯算法效果最佳，准确率为0.80、F1分数为0.79，受试者工作特征（receiver operating characteristic，ROC）曲线下面积（area under the curve，AUC）为0.79（P=0.035），与其他算法差异有统计学意义。结论 本研究构建了临床医学专业课程学习者画像模型并开展实证研究，画像结果对教学内容、教学方式与教学团队、学习效果预测提供了指导建议。     

Predicting pressure injury using nursing assessment phenotypes and machine learning methods

ObjectivePressure injuries are common and serious complications for hospitalized patients. The pressure injury rate is an important patient safety metric and an indicator of the quality of nursing care. Timely and accurate prediction of pressure injury risk can significantly facilitate early prevention and treatment and avoid adverse outcomes. While many pressure injury risk assessment tools exist, most were developed before there was access to large clinical datasets and advanced statistical methods, limiting their accuracy. In this paper, we describe the development of machine learning-based predictive models, using phenotypes derived from nurse-entered direct patient assessment data.MethodsWe utilized rich electronic health record data, including full assessment records entered by nurses, from 5 different hospitals affiliated with a large integrated healthcare organization to develop machine learning-based prediction models for pressure injury. Five-fold cross-validation was conducted to evaluate model performance.ResultsTwo pressure injury phenotypes were defined for model development: nonhospital acquired pressure injury (N = 4398) and hospital acquired pressure injury (N = 1767), representing 2 distinct clinical scenarios. A total of 28 clinical features were extracted and multiple machine learning predictive models were developed for both pressure injury phenotypes. The random forest model performed best and achieved an AUC of 0.92 and 0.94 in 2 test sets, respectively. The Glasgow coma scale, a nurse-entered level of consciousness measurement, was the most important feature for both groups.ConclusionsThis model accurately predicts pressure injury development and, if validated externally, may be helpful in widespread pressure injury prevention.     

Expected clinical utility of automatable prediction models for improving palliative and end-of-life care outcomes: Toward routine decision analysis before implementation

ObjectiveThe study sought to evaluate the expected clinical utility of automatable prediction models for increasing goals-of-care discussions (GOCDs) among hospitalized patients at the end of life (EOL).Materials and MethodsWe built a decision model from the perspective of clinicians who aim to increase GOCDs at the EOL using an automated alert system. The alternative strategies were 4 prediction models—3 random forest models and the Modified Hospital One-year Mortality Risk model—to generate alerts for patients at a high risk of 1-year mortality. They were trained on admissions from 2011 to 2016 (70 788 patients) and tested with admissions from 2017-2018 (16 490 patients). GOCDs occurring in usual care were measured with code status orders. We calculated the expected risk difference (beneficial outcomes with alerts minus beneficial outcomes without alerts among those at the EOL), the number needed to benefit (number of alerts needed to increase benefit over usual care by 1 outcome), and the net benefit (benefit minus cost) of each strategy.ResultsModels had a C-statistic between 0.79 and 0.86. A code status order occurred during 2599 of 3773 (69%) hospitalizations at the EOL. At a risk threshold corresponding to an alert prevalence of 10%, the expected risk difference ranged from 5.4% to 10.7% and the number needed to benefit ranged from 5.4 to 10.9 alerts. Using revealed preferences, only 2 models improved net benefit over usual care. A random forest model with diagnostic predictors had the highest expected value, including in sensitivity analyses.DiscussionPrediction models with acceptable predictive validity differed meaningfully in their ability to improve over usual decision making.ConclusionsAn evaluation of clinical utility, such as by using decision curve analysis, is recommended after validating a prediction model because metrics of model predictiveness, such as the C-statistic, are not informative of clinical value.     

Predicting outcomes in central venous catheter salvage in pediatric central line–associated bloodstream infection

ObjectiveCentral line–associated bloodstream infections (CLABSIs) are a common, costly, and hazardous healthcare-associated infection in children. In children in whom continued access is critical, salvage of infected central venous catheters (CVCs) with antimicrobial lock therapy is an alternative to removal and replacement of the CVC. However, the success of CVC salvage is uncertain, and when it fails the catheter has to be removed and replaced. We describe a machine learning approach to predict individual outcomes in CVC salvage that can aid the clinician in the decision to attempt salvage.Materials and MethodsOver a 14-year period, 969 pediatric CLABSIs were identified in electronic health records. We used 164 potential predictors to derive 4 types of machine learning models to predict 2 failed salvage outcomes, infection recurrence and CVC removal, at 10 time points between 7 days and 1 year from infection onset.ResultsThe area under the receiver-operating characteristic curve varied from 0.56 to 0.83, and key predictors varied over time. The infection recurrence model performed better than the CVC removal model did.ConclusionsMachine learning–based outcome prediction can inform clinical decision making for children. We developed and evaluated several models to predict clinically relevant outcomes in the context of CVC salvage in pediatric CLABSI and illustrate the variability of predictors over time.     

Pregnancy outcomes of the first thawing cycle in “freeze-all” strategy of infertility patients with fever during oocyte recruitment: a matched-pair study

Background:It is currently unknown whether patients with a fever after controlled ovulation during egg retrieval could increase the risk of pelvic infection or not, and fever itself may affect endometrial receptivity or embryo quality with poor pregnancy outcomes. The aim of this study was to analyze the outcomes of patients with fever during oocyte retrieval after the first frozen-thawed embryo transfer (FET) cycle.Methods:This was a 1:3 retrospective paired study matched for age. In this study, 58 infertility patients (Group 1) had a fever during the control ovulation, and the time of the oocyte retrieval was within 72 hours, they underwent ovum pick up and whole embryo freezing (“freeze-all” strategy). The control subjects (Group 2) are 174 patients matched for age who underwent whole embryo freezing for other reasons. The baseline characteristics, clinical data of ovarian stimulation, and outcomes, such as the clinical pregnancy rate, ongoing clinical pregnancy rate were compared between the two groups in the subsequent FET cycle.Results:All patients had no pelvic inflammatory disease after oocyte retrieval. Anti-Mullerian hormone (AMH) levels (4.2 vs. 2.2, P <0.001) were higher in group 2, and the number of oocytes retrieved, and fertilization rate were lower in group 1 (P < 0.001), but the endometrial thickness, the number of embryo transfers, and the type of luteal support supplementation were similar between the two groups. Regarding pregnancy outcomes in the subsequent FET cycle, the implantation rate, clinical pregnancy rate, early spontaneous rate, ectopic pregnancy rate, and ongoing pregnancy rate were all not significantly different. Further regression analyses showed that the clinical pregnancy rate and ongoing pregnancy rate were also not significantly different.Conclusions:Transvaginal ultrasound-guided follicular puncture for oocyte retrieval is a safe and minimally invasive method for patients with fever. Moreover, the fever had almost no effect on embryo quality.     

Effect of an educational programme on critical care nurses' competence at two tertiary hospitals in Malawi

BackgroundCritical care specialty deals with the complex needs of critically ill patients. Nurses who provide critical care are expected to possess the appropriate knowledge and skills required for the care of critically ill patients. The aim of this study was to assess the effect of an educational programme on the competence of critical care nurses at two tertiary hospitals in Lilongwe and Blantyre, Malawi.MethodsA quantitative pre- and post-test design was applied. The training programme was delivered to nurses (n = 41) who worked in intensive care and adult high dependency units at two tertiary hospitals. The effect of the training was assessed through participants'' self-assessment of competence on the Intensive and Critical Care Nursing Competence Scale and a list of 10 additional competencies before and after the training.ResultsThe participants'' scores on the Intensive and Critical Care Nursing Competence Scale before the training, M = 608.2, SD = 59.6 increased significantly after the training, M = 684.7, SD = 29.7, p <.0001 (two-tailed). Similarly, there was a significant increase in the participants'' scores on the additional competencies after the training, p <.0001 (two-tailed).ConclusionThe programme could be used for upskilling nurses in critical care settings in Malawi and other developing countries with a similar context.     

User-centered design of a longitudinal care plan for children with medical complexity

ObjectiveTo determine the content priorities and design preferences for a longitudinal care plan (LCP) among caregivers and healthcare providers who care for children with medical complexity (CMC) in acute care settings.Materials and MethodsWe conducted iterative one-on-one design sessions with CMC caregivers (ie, parents/legal guardians) and providers from 5 groups: complex care, primary care, subspecialists, emergency care, and care coordinators. Audio-recorded sessions included content categorization activities, drawing exercises, and scenario-based testing of an electronic LCP prototype. We applied inductive content analysis of session materials to elicit content priorities and design preferences between sessions. Analysis informed iterative prototype revisions.ResultsWe conducted 30 design sessions (10 with caregivers, 20 with providers). Caregivers expressed high within-group variability in their content priorities compared to provider groups. Emergency providers had the most unique content priorities among clinicians. We identified 6 key design preferences: a familiar yet customizable layout, a problem-based organization schema, linked content between sections, a table layout for most sections, a balance between unstructured and structured data fields, and use of family-centered terminology.DiscussionFindings from this study will inform enhancements of electronic health record-embedded LCPs and the development of new LCP tools and applications. The design preferences we identified provide a framework for optimizing integration of family and provider content priorities while maintaining a user-tailored experience.ConclusionHealth information platforms that incorporate these design preferences into electronic LCPs will help meet the information needs of caregivers and providers caring for CMC in acute care settings.     

An implementation model for managing cloud-based longitudinal care plans for children with medical complexity

ObjectiveWe aimed to iteratively refine an implementation model for managing cloud-based longitudinal care plans (LCPs) for children with medical complexity (CMC).Materials and MethodsWe conducted iterative 1-on-1 design sessions with CMC caregivers (ie, parents/legal guardians) and providers between August 2017 and March 2019. During audio-recorded sessions, we asked participants to walk through role-specific scenarios of how they would create, review, and edit an LCP using a cloud-based prototype, which we concurrently developed. Between sessions, we reviewed audio recordings to identify strategies that would mitigate barriers that participants reported relating to 4 processes for managing LCPs: (1) taking ownership, (2) sharing, (3) reviewing, and (4) editing. Analysis informed iterative implementation model revisions.ResultsWe conducted 30 design sessions, with 10 caregivers and 20 providers. Participants emphasized that cloud-based LCPs required a team of owners: the caregiver(s), a caregiver-designated clinician, and a care coordinator. Permission settings would need to include universal accessibility for emergency providers, team-level permission options, and some editing restrictions for caregivers. Notifications to review and edit the LCP should be sent to team members before and after clinic visits and after hospital encounters. Mitigating double documentation barriers would require alignment of data fields between the LCP and electronic health record to maximize interoperability.DiscussionThese findings provide a model for how we may leverage emerging Health Insurance Portability and Accountability Act–compliant cloud computing technologies to support families and providers in comanaging health information for CMC.ConclusionsUtilizing these management strategies when implementing cloud-based LCPs has the potential to improve team-based care across settings.     

居家老年患者家庭支持问题探讨

目的探讨居家老年患者家庭支持的常见问题,创造良好的养病治病家庭环境。方法用访谈法对在建家庭病床的老年患者及家庭成员、照顾者进行非结构性访谈。结果老年患者家庭支持问题主要存在三个方面：①家庭关怀不足,未能满足老年患者的情感需要;②健康照顾功能有限,家庭护理困难,需求专业指导;③家庭经济支持乏力。结论社区护士要善于鉴别有问题的家庭及其患病成员,采取必要的护理干预措施,帮助家庭发挥最大功能。     

Deep significance clustering: a novel approach for identifying risk-stratified and predictive patient subgroups

ObjectiveDeep significance clustering (DICE) is a self-supervised learning framework. DICE identifies clinically similar and risk-stratified subgroups that neither unsupervised clustering algorithms nor supervised risk prediction algorithms alone are guaranteed to generate.Materials and MethodsEnabled by an optimization process that enforces statistical significance between the outcome and subgroup membership, DICE jointly trains 3 components, representation learning, clustering, and outcome prediction while providing interpretability to the deep representations. DICE also allows unseen patients to be predicted into trained subgroups for population-level risk stratification. We evaluated DICE using electronic health record datasets derived from 2 urban hospitals. Outcomes and patient cohorts used include discharge disposition to home among heart failure (HF) patients and acute kidney injury among COVID-19 (Cov-AKI) patients, respectively.ResultsCompared to baseline approaches including principal component analysis, DICE demonstrated superior performance in the cluster purity metrics: Silhouette score (0.48 for HF, 0.51 for Cov-AKI), Calinski-Harabasz index (212 for HF, 254 for Cov-AKI), and Davies-Bouldin index (0.86 for HF, 0.66 for Cov-AKI), and prediction metric: area under the Receiver operating characteristic (ROC) curve (0.83 for HF, 0.78 for Cov-AKI). Clinical evaluation of DICE-generated subgroups revealed more meaningful distributions of member characteristics across subgroups, and higher risk ratios between subgroups. Furthermore, DICE-generated subgroup membership alone was moderately predictive of outcomes.DiscussionDICE addresses a gap in current machine learning approaches where predicted risk may not lead directly to actionable clinical steps.ConclusionDICE demonstrated the potential to apply in heterogeneous populations, where having the same quantitative risk does not equate with having a similar clinical profile.     

Closing the translation gap for justice requirements in international research

Bioethicists have long debated the content of sponsors and researchers' obligations of justice in international clinical research. However, there has been little empirical investigation as to whether and how obligations of responsiveness, ancillary care, post-trial benefits and research capacity strengthening are upheld in low- and middle-income country settings. In this paper, the authors argue that research ethics guidelines need to be more informed by international research practice. Practical guidance on how to fulfil these obligations is needed if research groups and other actors are to successfully translate them into practice because doing so is often a complicated, context-specific process. Case study research methods offer one avenue for collecting data to develop this guidance. The authors describe how such methods have been used in relation to the Shoklo Malaria Research Unit's vivax malaria treatment (VHX) trial (NCT01074905). Relying on the VHX trial example, the paper shows how information can be gathered from not only international clinical researchers but also trial participants, community advisory board members and research funder representatives in order to: (1) measure evidence of responsiveness, provision of ancillary care, access to post-trial benefits and research capacity strengthening in international clinical research; and (2) identify the contextual factors and roles and responsibilities that were instrumental in the fulfilment of these ethical obligations. Such empirical work is necessary to inform the articulation of obligations of justice in international research and to develop guidance on how to fulfil them in order to facilitate better adherence to guidelines' requirements.     

糖尿病足感染病原微生物分布及风险预测模型的建立

目的 分析糖尿病足（DF）感染病原微生物分布特征，并建立DF感染风险预测模型。方法 选取2019年1月—2022年1月文昌市人民医收治的82例DF患者为研究对象，对其足部创面分泌物进行细菌培养，总结DF感染病原菌分布特点，多因素Logistic回归分析确定DF感染独立危险因素并建立风险预测模型，绘制受试者工作特征（ROC）曲线判断各独立指标与风险预警模型诊断的准确性，交叉验证法验证风险预警模型效能。结果 82例DF患者中有50例足部分泌物中分离出病原菌，共检出病原菌79株，其中革兰阳性菌37株（46.84%），革兰阴性菌38株（48.10%），真菌4株（5.06%）。多因素Logistic回归分析结果提示，DF病程［O^R=2.201（95% CI：1.754，2.763）］、周围神经病变［O^R=3.177（95% CI：1.518，6.652）］、白细胞计数（WBC） ［O^R=2.425（95% CI：1.512，3.890）］、低密度脂蛋白胆固醇（LDL-C） ［O^R=1.976（95% CI：1.481，2.636）］是DF感染的独立危险因素（P <0.05）。ROC曲线结果表明，DF病程、周围神经病变、WBC、LDL-C预测DF感染的曲线下面积（AUC）分别为0.665、0.659、0.685和0.645，敏感性分别为68.0%（0.538，0.841）、60.0%（0.563，0.794）、76.0%（0.550，0.882）和62.0%（0.512，0.803，特异性分别为68.7%（0.548，0.853）、71.9%（0.615，0.899）、59.4%（0.440，0.603）和62.5%（0.536，0.815）。将上述进入多因素Logistic回归模型变量的b作为系数，建立风险预警模型，Logit（P）=e^y/（1+e^y），y =0.789×DF病程+1.156×周围神经病变+0.886×WBC+0.681×LDL-C-3.157。ROC曲线分析结果表明，风险预测模型AUC为0.908，截断值为0.513，敏感性及特异性分别为84.10%（0.651，0.917）和75.61%（0.579，0.855）。结论 DF感染率较高，但多为单一病原菌感染，DF病程、周围神经病变、WBC、LDL-C是DF感染的独立危险因素，以此为基础建立风险预测模型，便于临床快速准确筛查DF感染高危人群。     

中心静脉导管相关性血流感染的临床及病原学的回顾性调查分析

目的 了解重症监护室（ICU）中心静脉导管相关性血流感染（CRBSI）的感染率、危险因素、病原菌种类、耐药性及临床结局,对有效抗感染治疗及预防CRBSI的发生提供指导。方法 回顾性分析2010年4月~2014年4月共483例留置过中心静脉导管（CVC）的患者。计算感染率、感染相关性因素及筛选危险因素分析并应用SPSS20.0统计软件进行多因素Logistic回归分析。结果 共有17例患者发生CRBSI,感染率为3.5%;不同置管者、置管地点、置管部位构成的比较,差异有统计学意义（P<0.05）。经多因素Logistic回归分析,发现影响CRBSI的独立危险因素是抗生素大量使用和导管日,OR值分别为7.898和1.044。革兰阳性菌（gram-positive bacteria,G⁺）占35.3%,革兰阴性菌（gram-negative bacteria,G^-）占47.1%,真菌占17.6%;G⁺菌感染患者,经治疗后病情100% 好转;G^-菌感染患者,87.5% 好转,12.5% 恶化;真菌感染患者,66.7% 好转,33.3% 恶化。结论 CRBSI的发生率随抗生素大量使用以及导管日延长而增加。患者经治疗后,大部分好转,但真菌感染者病死率相对较高,应引起临床关注。     

A bias evaluation checklist for predictive models and its pilot application for 30-day hospital readmission models

ObjectiveHealth care providers increasingly rely upon predictive algorithms when making important treatment decisions, however, evidence indicates that these tools can lead to inequitable outcomes across racial and socio-economic groups. In this study, we introduce a bias evaluation checklist that allows model developers and health care providers a means to systematically appraise a model’s potential to introduce bias.Materials and MethodsOur methods include developing a bias evaluation checklist, a scoping literature review to identify 30-day hospital readmission prediction models, and assessing the selected models using the checklist.ResultsWe selected 4 models for evaluation: LACE, HOSPITAL, Johns Hopkins ACG, and HATRIX. Our assessment identified critical ways in which these algorithms can perpetuate health care inequalities. We found that LACE and HOSPITAL have the greatest potential for introducing bias, Johns Hopkins ACG has the most areas of uncertainty, and HATRIX has the fewest causes for concern.DiscussionOur approach gives model developers and health care providers a practical and systematic method for evaluating bias in predictive models. Traditional bias identification methods do not elucidate sources of bias and are thus insufficient for mitigation efforts. With our checklist, bias can be addressed and eliminated before a model is fully developed or deployed.ConclusionThe potential for algorithms to perpetuate biased outcomes is not isolated to readmission prediction models; rather, we believe our results have implications for predictive models across health care. We offer a systematic method for evaluating potential bias with sufficient flexibility to be utilized across models and applications.     

A novel application of SMART on FHIR architecture for interoperable and scalable integration of patient-reported outcome data with electronic health records

ObjectiveDespite a proliferation of applications (apps) to conveniently collect patient-reported outcomes (PROs) from patients, PRO data are yet to be seamlessly integrated with electronic health records (EHRs) in a way that improves interoperability and scalability. We applied the newly created PRO standards from the Office of the National Coordinator for Health Information Technology to facilitate the collection and integration of standardized PRO data. A novel multitiered architecture was created to enable seamless integration of PRO data via Substitutable Medical Apps and Reusable Technologies on Fast Healthcare Interoperability Resources apps and scaled to different EHR platforms in multiple ambulatory settings.Materials and MethodsWe used a standards-based approach to deploy 2 apps that source and surface PRO data in real-time for provider use within the EHR and which rely on PRO assessments from an external center to streamline app and EHR integration.ResultsThe apps were developed to enable patients to answer validated assessments (eg, a Patient-Reported Outcomes Measurement Information System including using a Computer Adaptive Test format). Both apps were developed to populate the EHR in real time using the Health Level Seven FHIR standard allowing providers to view patients’ data during the clinical encounter. The process of implementing this architecture with 2 different apps across 18 ambulatory care sites and 3 different EHR platforms is described.ConclusionOur approach and solution proved feasible, secure, and time- and resource-efficient. We offer actionable guidance for this technology to be scaled and adapted to promote adoption in diverse ambulatory care settings and across different EHRs.