“Yes,and …” Exploring the Future of Learning Analytics in Medical Education

This Conversations Starter article presents a selected research abstract from the 2017 Association of American Medical Colleges Northeastern Region Group on Educational Affairs annual spring meeting. The abstract is paired with the integrative commentary of three experts who shared their thoughts stimulated by the study. Commentators brainstormed “what's next” with learning analytics in medical education, including advancements in interaction metrics and the use of interactivity analysis to deepen understanding of perceptual, cognitive, and social learning and transfer processes.     

Perception or reality: Can thermal perceptions inform management of firefighters in the heat?

Accurately assessing the physiological status of firefighters during work in the heat is critical to ensuring their safety. Evaluating core temperatures (T_c) in the field is problematic due to cost and limitations in technology and accuracy. As such, fire services rely on individual perceptions of wellbeing. The present study aimed to establish whether perceptual responses measured using the perceptual strain index (PeSI), calculated from rate of perceived exertion (RPE) and thermal sensation (TS), could reliably predict the physiological strain (PSI) encountered by experienced firefighters working in a hot environment. We conducted two firefighting simulations (set-pace and self-paced) in a purposefully built heat chamber (100 ± 5°C) comprised of two 20-min periods separated by a 10-min recovery outside the chamber. Physiological strain was measured via heart rate (HR) and gastrointestinal temperature (T_gi) and compared with PeSI at 5-min intervals. To evaluate the predictive ability of the PeSI for PSI, mean differences and the 95% limits of agreement (LOA) were established, along with correlation coefficients at each 5-min interval. Moderately significant correlations occurred in the second work bout of the self-paced trial only (10 min: r = 0.335, 15 min: r = 0.498, 20 min r = 0.439) with no other correlations observed at any other time during either trial or during the rest periods. Bland-Altman analysis revealed mean differences of ?0.74 ± 2.70 (self-paced) and +0.04 ± 2.04 (set-paced) between PeSI and PSI with the 95% LOA being ?4.77 to 3.28 (self-paced) and ?4.01 to 2.01 (set-paced). The wide LOA and lack of correlations observed between perceptual and physiological strain in both self-paced and set-paced work trials indicate that PeSI is not sufficiently reliable as a sole measure of wellbeing for firefighters working in the heat. Hence, we recommend that fire services prioritise the development of reliable and effective monitoring tools for use in the field.     

Crime and Violence among MDMA Users in the United States

The question of whether MDMA use is associated with increased crime and violence has not been adequately explored especially in nationally representative samples. This study used data from the National Epidemiologic Survey on Alcohol and Related Conditions (NESARC) to assess the association between MDMA use and violent and non-violent antisocial behavior while controlling for sociodemographic variables, lifetime psychiatric, alcohol and drug use disorders, and family history of antisocial behavior. MDMA users, both male and female, were involved in a number of crimes in acts of violence including drunk driving, shoplifting, theft, intimate partner violence, and fighting. Notably, female MDMA users were more antisocial than male non-MDMA users. Although adjusting the results for numerous confounds attenuated the relationships, MDMA users were still at significantly greater odds of engaging in violent and nonviolent crime than non-MDMA users. Although MDMA has been considered a facilitator of empathy and closeness, the current study suggests a dark side as MDMA is associated with a broad array of crimes and transgressions. Additional tests of the MDMA-crime link are needed to properly inform policy.     

Quality target negotiation in health care: evidence from the English NHS

We examine how public sector third-party purchasers and hospitals negotiate quality targets when a fixed proportion of hospital revenue is required to be linked to quality. We develop a bargaining model linking the number of quality targets to purchaser and hospital characteristics. Using data extracted from 153 contracts for acute hospital services in England in 2010/2011, we find that the number of quality targets is associated with the purchaser’s population health and its budget, the hospital type, whether the purchaser delegated negotiation to an agency, and the quality targets imposed by the supervising regional health authority.     

Use of retail data in the assessment of natural experiments: the case of Reducing the Strength,an intervention to reduce alcohol availability

BackgroundRetailers routinely collect data about people's purchasing behaviours and access to consumer products associated with health and wellbeing. Here we discuss how retail data can be used in public health research and consider potential strengths and limitations to such research. To illustrate the discussion we refer to an evaluation of an intervention called Reducing the Strength, whereby off-licence shops and supermarkets voluntarily stopped selling inexpensive superstrength (≥6·5% alcohol by volume) beers and ciders.MethodsMonthly data from a large retail chain (East of England Co-operative Society) were obtained for three UK counties (141 stores). In one county the intervention started 12 months earlier than the others, allowing for a pre–post study design with a delayed implementation comparator. Difference-in-differences analysis of unit alcohol sales controlled for socioenvironmental confounders and shop-level characteristics including shop size, parking facilities, cash machines, opening hours, and other factors.FindingsThe retail data detailed shop-level characteristics and sales data such as prices, quantities, product brands, alcohol content, sales, and factors affecting sales. The wide geographical coverage, shop-level data, including data for potential confounding factors, and frequent timepoints made the retail data well-suited for a quasi-experimental evaluation capitalising on temporal and spatial variations in intervention exposure. Limitations of this study include a lack of longitudinal data for individual customers, and shops that are not covered by the data. Qualitative interviews with shop workers and customers, and triangulation using alternative data sources can help to address limitations. Alternative sources of retail data such as private sector consultants who specialise in collecting shop-level and sales data for a range of companies might also address some limitations; however, there are potential barriers of expense, accessibility, and coverage associated with the use of such consultants.InterpretationIncreasingly, researchers recognise the potential of retail data for evaluating interventions affecting social determinants of health and inequalities, such as local access to alcohol. However, shop-level data have frequently proved difficult for researchers to obtain. By obtaining such data we have been able to assess, using a quasi-experimental design, the effects of removing strong, cheap beers and ciders from shops. We have also been able to explore in more detail how to optimise the strengths and address some limitations of the data in ways that could potentially assist others planning to use this important data source in their research.FundingThe study is funded as part of the School of Public Health Research by NHS National Institute of Health Research. AJ, SA, and JW contributed as employees of Public Health Suffolk, Suffolk County Council.     

The effect of information technology on hospital performance

While healthcare entities have integrated various forms of health information technology (HIT) into their systems due to claims of increased quality and decreased costs, as well as various incentives, there is little available information about which applications of HIT are actually the most beneficial and efficient. In this study, we aim to assist administrators in understanding the characteristics of top performing hospitals. We utilized data from the Health Information and Management Systems Society and the Center for Medicare and Medicaid to assess 1039 hospitals. Inputs considered were full time equivalents, hospital size, and technology inputs. Technology inputs included personal health records (PHR), electronic medical records (EMRs), computerized physician order entry systems (CPOEs), and electronic access to diagnostic results. Output variables were measures of quality, hospital readmission and mortality rate. The analysis was conducted in a two-stage methodology: Data Envelopment Analysis (DEA) and Automatic Interaction Detector Analysis (AID), decision tree regression (DTreg). Overall, we found that electronic access to diagnostic results systems was the most influential technological characteristics; however organizational characteristics were more important than technological inputs. Hospitals that had the highest levels of quality indicated no excess in the use of technology input, averaging one use of a technology component. This study indicates that prudent consideration of organizational characteristics and technology is needed before investing in innovative programs.