Digital video analysis of health professionals’ interactions with an electronic whiteboard: A longitudinal,naturalistic study of changes to user interactions

As hospital departments continue to introduce electronic whiteboards in real clinical settings a range of human factor issues have emerged and it has become clear that there is a need for improved methods for designing and testing these systems. In this study, we employed a longitudinal and naturalistic method in the usability evaluation of an electronic whiteboard system. The goal of the evaluation was to explore the extent to which usability issues experienced by users change as they gain more experience with the system. In addition, the paper explores the use of a new approach to collection and analysis of continuous digital video recordings of naturalistic “live” user interactions. The method developed and employed in the study included recording the users’ interactions with system during actual use using screen-capturing software and analyzing these recordings for usability issues. In this paper we describe and discuss both the method and the results of the evaluation. We found that the electronic whiteboard system contains system-related usability issues that did not change over time as the clinicians collectively gained more experience with the system. Furthermore, we also found user-related issues that seemed to change as the users gained more experience and we discuss the underlying reasons for these changes. We also found that the method used in the study has certain advantages over traditional usability evaluation methods, including the ability to collect analyze live user data over time. However, challenges and drawbacks to using the method (including the time taken for analysis and logistical issues in doing live recordings) should be considered before utilizing a similar approach. In conclusion we summarize our findings and call for an increased focus on longitudinal and naturalistic evaluations of health information systems and encourage others to apply and refine the method utilized in this study.     

Combining efficacy and completion rates with no data imputation: A composite approach with greater sensitivity for the statistical evaluation of active comparisons in antipsychotic trials

Outcomes in RCT's of antipsychotic medications are often examined using last observation carried forward (LOCF) and mixed effect models (MMRM), these ignore meaning of non-completion and thus rely on questionable assumptions. We tested an approach that combines into a single statistic, the drug effect in those who complete trial and proportion of patients in each treatment group who complete trial. This approach offers a conceptually and clinically meaningful endpoint. Composite approach was compared to LOCF (ANCOVA) and MMRM in 59 industry sponsored RCT's. For within study comparisons we computed effect size (z-score) and p values for (a) rates of completion, (b) symptom change for complete cases, which were combined into composite statistic, and (c) symptom change for all cases using last observation forward (LOCF). In the 30 active comparator studies, composite approach detected larger differences in effect size than LOCF (ES=.05) and MMRM (ES=.076). In 10 of the 49 comparisons composite lead to significant differences (p≤.05) where LOCF and MMRM did not. In 3 comparisons LOCF was significant, in 2 MMRM lead to significant differences whereas composite did not. In placebo controlled trials, there was no meaningful difference in effect size between composite and LOCF and MMRM when comparing placebo to active treatment, however composite detected greater differences than other approaches when comparing between active treatments. Composite was more sensitive to effects of experimental treatment vs. active controls (but not placebo) than LOCF and MMRM thereby increasing study power while answering a more relevant question.     

Methodological considerations when assessing restricted and repetitive behaviors and aggression

Methodological issues impacting the relationship between aggression and restricted, repetitive, and stereotyped behaviors and interests (RRSBI) were examined in 2648 children and adolescents with autism spectrum disorders (ASD) using a multi-method, multi-informant analysis model to assess the effects of informant, assessment method, and aggression phenotype. Overall, a significant, but small relationship was found between RRSBI and aggression (p < .05). There was significant heterogeneity of estimates with large effect sizes observed when utilizing teacher report and a broad phenotype of aggression. Variance in estimates was attributed to differences in informant and assessment method with two times greater effect attributed to informant. Results suggest strategies to optimize future investigations of the relationship between RRSBI and aggression. Findings also provide the opportunity for the development of targeted interventions for aggression in youth with ASD.     

Surrogate end points for overall survival in breast cancer trials: A review

Our aim was to review the studies which assessed potential surrogate endpoints for overall survival (OS) in breast cancer trials. A Literature search in PubMed database of studies which assessed potential surrogate endpoints for OS in breast cancer trials was conducted. The surrogacy was assessed with the German institute of Quality and efficiency in Health Care’s (IWQiG) framework and the Fleming hierarchy. Thirteen studies were identified. At the neoadjuvant setting, two individual patient data (IPD) meta-analyses and one aggregate data meta-analysis assessing surrogacy of pathological complete response (PCR) were identified. Trial-level association was calculated in one study and the squared correlation was 0.24. Therefore PCR was not judged to be valid surrogate for OS at the neoadjuvant setting according to the IWQiG framework and Fleming hierarchy. At the adjuvant setting, one meta-analysis on aggregate data was identified. 2-year DFS was not judged to be valid surrogate for OS at the neoadjuvant setting according to the IWQiG framework and Fleming hierarchy. At the metastatic setting, six meta-analyses based on aggregate data, three IPD meta-analyses and one retrospective study were identified. Within the IPD meta-analyses, at the trial-level association the squared correlation between the potential surrogates and OS ranged from 0.10 to 0.57 and no endpoint was judged to be valid surrogate for OS at the metastatic setting. The level of evidence available supporting a relationship between OS and potential surrogate endpoints in breast cancer trials is low.     

Methods for development of a core outcome set for clinical trials integrating traditional Chinese medicine and Western medicine

Clinical trial outcome reporting differs between studies integrating traditional Chinese medicine (TCM) and Western medicine, so that some clinical trials are not eligible for inclusion in a systematic review. The excluded studies are therefore less widely disseminated, and even valid studies are less likely to yield impact. This problem may be addressed by developing core outcome sets (COSs) for integrative medicine in specific healthcare areas. The first stage of development is to define the scope of the COS for integrative medicine, the second stage is to establish the need for such a COS, and the third stage is to develop a protocol and register the COS. The final stage involves three steps: (i) development of a comprehensive list of outcomes (including efficacy outcomes and safety outcomes and TCM syndromes) using systematic review, qualitative or cross-sectional research, and reviews of package inserts and medical records; (ii) merging and grouping of outcomes within domains; (iii) conducting two rounds of Delphi survey and consensus meetings with a range of stakeholders. The final COS will include a general COS and core TCM syndrome- set. Development of COSs for clinical trials of integrative medicine may help to standardize outcome reporting and reduce publication bias in the future.     

How can we evaluate an interrelation of symptoms?

A pandemic of 2019 novel coronavirus (COVID-19) is an international problem and factors associated with increased risk of mortality have been reported. However, there exists limited statistical method to estimate a comprehensive risk for a case in which a patient has several characteristics and symptoms concurrently. We applied Boolean Monte Carlo method (BMCM) to the Novel Corona Virus 2019 Dataset to determine interrelation of patient’s characteristics and symptoms. In the analyses, age, fever as an onset symptom, and sex were used as explanatory variables, and death as the objective variable. Among 265 patients included in the analysis, the interrelations for estimating death were determined as age “and” fever “and” sex (p < 0.0001 for both operators). This result indicates that satisfying the three conditions of age, fever, and sex concurrently may be associated with an increased risk of mortality.     

Who and how are children selected for early autism intervention

Early Intensive Behavioral Interventions (EIBI) has become a well recognized treatment for autism spectrum disorders (ASD). Nonetheless, many questions remain about how to best administer these interventions and tailor treatments to given children. For researchers to make the best decisions regarding treatment, client populations must be precisely defined and described. Thus, the correct methods for client selection need to be described and used. The purpose of this paper is to take stock of methods of client selection for EIBI research. This paper reviews 34 studies which focus on this treatment model. Methods and procedures used in these studies are discussed. At present, they fall short of optimal standards.     

Assessing the economic benefits of vaccines based on the health investment life course framework: A review of a broader approach to evaluate malaria vaccination

BackgroundEconomic evaluations have routinely understated the net benefits of vaccination by not including the full range of economic benefits that accrue over the lifetime of a vaccinated person. Broader approaches for evaluating benefits of vaccination can be used to more accurately calculate the value of vaccination.MethodologyThis paper reflects on the methodology of one such approach – the health investment life course approach – that looks at the impact of vaccine investment on lifetime returns. The role of this approach on vaccine decision-making will be assessed using the malaria health investment life course model example.ResultsWe describe a framework that measures the impact of a health policy decision on government accounts over many generations. The methodological issues emerging from this approach are illustrated with an example from a recently completed health investment life course analysis of malaria vaccination in Ghana. Beyond the results, various conceptual and practical challenges of applying this framework to Ghana are discussed in this paper.Discussion and conclusionsThe current framework seeks to understand how disease and available technologies can impact a range of economic parameters such as labour force participation, education, healthcare consumption, productivity, wages or economic growth, and taxation following their introduction. The framework is unique amongst previous economic models in malaria because it considers future tax revenue for governments. The framework is complementary to cost-effectiveness and budget impact analysis. The intent of this paper is to stimulate discussion on how existing and new methodology can add to knowledge regarding the benefits from investing in new and underutilized vaccines.