Student perceptions of the progress test in two settings and the implications for test deployment

Background The Progress Test (PT) was developed to assess student learning within integrated curricula. Whilst it is effective in promoting and rewarding deep approaches to learning in some settings, we hypothesised that implementation of the curriculum (design and assessment) may impact on students’ preparation for the PT and their learning. Aim To compare students’ perceptions of and preparations for the PT at two medical schools. Method Focus groups were used to generate items for a questionnaire. This was piloted, refined, and then delivered at both schools. Exploratory factor analysis identified the main factors underpinning response patterns. ANOVA was used to compare differences in response by school, year group and gender. Results Response rates were 640 (57%) and 414 (47%) at Schools A and B, respectively. Three major factors were identified: the PT’s ability to (1) assess academic learning (2) support clinical learning; (3) the PT’s impact on exam preparation. Significant differences were found between settings. In the school with early clinical contact, more frequent PTs and no end of unit tests, students were more likely to appreciate the PT as a support for learning, perceive it as fair and valid, and use a deeper approach to learning—but they also spent longer preparing for the test. Conclusion Different approaches to the delivery of the PT can impact significantly on student study patterns. The learning environment has an important impact on student perceptions of assessment and approach to learning. Careful decisions about PT deployment must be taken to ensure its optimal impact.     

Learning styles and approaches: implications for medical education

This paper reviews the recent literature on learning styles and approaches to learning. It identifies two separate streams of research, one originating from mainstream cognitive and psychometric psychology and one from research undertaken within the everyday learning environment. The latter is dealt with in greater detail as it seems to have more immediate practical relevance. A simple model of the teaching-learning process is presented showing how students learn in different ways which are partly attributable to their preferred learning style and partly to the context in which the learning takes place. Three basic approaches have been identified: surface, deep and strategic, each resulting in a different learning outcome. The most desirable and successful is the deep approach. The way in which the teaching and the policies of the department and school influence the students' approach to learning are reviewed in some detail. A consideration of these characteristics in medical schools suggested that many may hinder rather than assist in the development of the desired approach. The work reviewed here suggests that the remedy will require not only substantial changes in the teaching, curriculum and, particularly, assessment, but also a new strategy based on identifying and assisting individual students whose approaches to study are not those expected of a competent university-educated doctor.     

The Critical Role of Working Memory in Academic Achievement

Far too many culinary arts educators are unaware of the critical role that working memory plays in student’s ability to perform cognitive and academic tasks, such as reading, note taking, and studying for tests (memorization). In this article, we provide research that suggests working memory may be uniquely responsible for the difficulties faced by culinary students in the kitchen and classroom. This literature review not only defines working memory but describes its functionality based on the work of key researchers in the field and gives practical recommendations to apply this knowledge in culinary arts education. The vital role that working memory plays in teaching and learning should give teachers great pause and make them re-examine their approach to instructional design to enhance the academic achievement potential of culinary students.     

Regulated competition in health care: Switching and barriers to switching in the Dutch health insurance system

Background

Patients have the potential to provide a rich source of information on both organisational aspects of safety and patient safety incidents. This project aims to develop two patient safety interventions to promote organisational learning about safety - a patient measure of organisational safety (PMOS), and a patient incident reporting tool (PIRT) - to help the NHS prevent patient safety incidents by learning more about when and why they occur.

Methods

To develop the PMOS 1) literature will be reviewed to identify similar measures and key contributory factors to error; 2) four patient focus groups will ascertain practicality and feasibility; 3) 25 patient interviews will elicit approximately 60 items across 10 domains; 4) 10 patient and clinician interviews will test acceptability and understanding. Qualitative data will be analysed using thematic content analysis. To develop the PIRT 1) individual and then combined patient and clinician focus groups will provide guidance for the development of three potential reporting tools; 2) nine wards across three hospital directorates will pilot each of the tools for three months. The best performing tool will be identified from the frequency, volume and quality of reports. The validity of both measures will be tested. 300 patients will be asked to complete the PMOS and PIRT during their stay in hospital. A sub-sample (N = 50) will complete the PMOS again one week later. Health professionals in participating wards will also be asked to complete the AHRQ safety culture questionnaire. Case notes for all patients will be reviewed. The psychometric properties of the PMOS will be assessed and a final valid and reliable version developed. Concurrent validity for the PIRT will be assessed by comparing reported incidents with those identified from case note review and the existing staff reporting scheme. In a subsequent study these tools will be used to provide information to wards/units about their priorities for patient safety. A patient panel will provide steering to the research.

Discussion

The PMOS and PIRT aim to provide a reliable means of eliciting patient views about patient safety. Both interventions are likely to have relevance and practical utility for all NHS hospital trusts.     

Deep and surface learning in problem-based learning: a review of the literature

In problem-based learning (PBL), implemented worldwide, students learn by discussing professionally relevant problems enhancing application and integration of knowledge, which is assumed to encourage students towards a deep learning approach in which students are intrinsically interested and try to understand what is being studied. This review investigates: (1) the effects of PBL on students’ deep and surface approaches to learning, (2) whether and why these effects do differ across (a) the context of the learning environment (single vs. curriculum wide implementation), and (b) study quality. Studies were searched dealing with PBL and students’ approaches to learning. Twenty-one studies were included. The results indicate that PBL does enhance deep learning with a small positive average effect size of .11 and a positive effect in eleven of the 21 studies. Four studies show a decrease in deep learning and six studies show no effect. PBL does not seem to have an effect on surface learning as indicated by a very small average effect size (.08) and eleven studies showing no increase in the surface approach. Six studies demonstrate a decrease and four an increase in surface learning. It is concluded that PBL does seem to enhance deep learning and has little effect on surface learning, although more longitudinal research using high quality measurement instruments is needed to support this conclusion with stronger evidence. Differences cannot be explained by the study quality but a curriculum wide implementation of PBL has a more positive impact on the deep approach (effect size .18) compared to an implementation within a single course (effect size of ?.05). PBL is assumed to enhance active learning and students’ intrinsic motivation, which enhances deep learning. A high perceived workload and assessment that is perceived as not rewarding deep learning are assumed to enhance surface learning.     

Genome-wide association study-based deep learning for survival prediction

Informative and accurate survival prediction with individualized dynamic risk profiles over time is critical for personalized disease prevention and clinical management. The massive genetic data, such as SNPs from genome-wide association studies (GWAS), together with well-characterized time-to-event phenotypes provide unprecedented opportunities for developing effective survival prediction models. Recent advances in deep learning have made extraordinary achievements in establishing powerful prediction models in the biomedical field. However, the applications of deep learning approaches in survival prediction are limited, especially with utilizing the wealthy GWAS data. Motivated by developing powerful prediction models for the progression of an eye disease, age-related macular degeneration (AMD), we develop and implement a multilayer deep neural network (DNN) survival model to effectively extract features and make accurate and interpretable predictions. Various simulation studies are performed to compare the prediction performance of the DNN survival model with several other machine learning-based survival models. Finally, using the GWAS data from two large-scale randomized clinical trials in AMD with over 7800 observations, we show that the DNN survival model not only outperforms several existing survival prediction models in terms of prediction accuracy (eg, c-index =0.76 ), but also successfully detects clinically meaningful risk subgroups by effectively learning the complex structures among genetic variants. Moreover, we obtain a subject-specific importance measure for each predictor from the DNN survival model, which provides valuable insights into the personalized early prevention and clinical management for this disease.     

Students’ reflections on the relationships between safe learning environments,learning challenge and positive experiences of learning in a simulated GP clinic

Learning environments are a significant determinant of student behaviour, achievement and satisfaction. In this article we use students’ reflective essays to identify key features of the learning environment that contributed to positive and transformative learning experiences. We explore the relationships between these features, the students’ sense of safety in the learning environment (LE), the resulting learning challenge with which they could cope and their positive reports of the experience itself. Our students worked in a unique simulation of General Practice, the Safe and Effective Clinical Outcomes clinic, where they consistently reported positive experiences of learning. We analysed 77 essays from 2011 and 2012 using an immersion/crystallisation framework. Half of the students referred to the safety of the learning environment spontaneously. Students described deep learning experiences in their simulated consultations. Students valued features of the LE which contributed to a psychologically safe environment. Together with the provision of constructive support and immediate, individualised feedback this feeling of safety assisted students to find their own way through clinical dilemmas. These factors combine to make students feel relaxed and able to take on challenges that otherwise would have been overwhelming. Errors became learning opportunities and students could practice purposefully. We draw on literature from medical education, educational psychology and sociology to interpret our findings. Our results demonstrate relationships between safe learning environments, learning challenge and powerful learning experiences, justifying close attention to the construction of learning environments to promote student learning, confidence and motivation.     

Culture, Self-Rated Health and Resource Allocation Decision-Making

It has been observed that some groups in society tend to report their health to be better than would be expected through more objective measures. The available evidence suggests that while variations in self-assessed measures of health may act as good proxies of mortality and morbidity in homogeneous populations, in some groups, such as the Aboriginal and Torres Strait Islander communities of Australia, these subjective measures may provide a misleading picture. Useful insights into the formation of health perceptions can be drawn from a range of disciplines, in particular, from social comparison theories, models of illness behaviour, survey literature and linguistics. These theories and models help to provide an understanding of the different ways in which illness may be perceived, evaluated and acted upon by different kinds of people. Such considerations can have very direct implications for those planning and evaluating public health programs as well as those responsible for funding such programs. This revised version was published online in August 2006 with corrections to the Cover Date.     

Do childhood vaccines have non-specific effects on mortality?

A recent article by Kristensen et al. suggested that measles vaccine and bacille Calmette-Gu rin (BCG) vaccine might reduce mortality beyond what is expected simply from protection against measles and tuberculosis. Previous reviews of the potential effects of childhood vaccines on mortality have not considered methodological features of reviewed studies. Methodological considerations play an especially important role in observational assessments, in which selection factors for vaccination may be difficult to ascertain. We reviewed 782 English language articles on vaccines and childhood mortality and found only a few whose design met the criteria for methodological rigor. The data reviewed suggest that measles vaccine delivers its promised reduction in mortality, but there is insufficient evidence to suggest a mortality benefit above that caused by its effect on measles disease and its sequelae. Our review of the available data in the literature reinforces how difficult answering these considerations has been and how important study design will be in determining the effect of specific vaccines on all-cause mortality.     

Introducing a problem-based unit into a lifespan nutrition class using a randomized design produces equivocal outcomes

Problem-based learning (PBL) is a tutorial, student-centered, problem-driven educational strategy adopted by medical and allied health educators to positively influence self-directed learning, critical thinking, and learning behavior. PBL was examined in dietetics education through random assignment of 32 undergraduate dietetics students for two weeks to either a problem-based or a lecture-based case format for the infant and elderly units of a 16-week lifespan nutrition course. Random assignment followed stratification for gender and earlier course examination performance. Expert validation of PBL curricular components and noted differences in discussion structure and information resources verified curriculum distinctiveness. Main outcome measures were pre- and post-Cognitive Behavior Survey scores for memorization, reflection, and positive learning experience scales, unit and course evaluations and unit examination scores. Students in problem-based modules demonstrated greater gains in reflective thinking with stable memorization, suggesting improved critical thinking skills. Tenets that problem-based learning promotes knowledge retention and provides a more positive learning experience were not upheld. Knowledge acquisition was not hindered by a problem-based approach. We conclude that gains in reflective thinking and evidence of increased self-directed learning argue for inclusion of PBL in dietetics curricula and that a problem-based education will help dietitians successfully respond to professional development needs.     

Reporting bayesian analyses of clinical trials

Many clinicians wrongly interpret p-values as probabilities that treatment has an adverse effect and confidence intervals as probability intervals. Such inferences can be validly drawn from Bayesian analyses of trial results. These analyses use the data to update the prior )or pre-trial( beliefs to give posterior )or post-trial( beliefs about the magnitude of a treatment effect. However, for these methods to gain acceptance in the medical literature, understanding between statisticians and clinicians of the issues involved in choosing appropriate prior distributions for trial reporting needs to be reached. I focus on two types of prior that deserve consideration. The first is the non-informative prior giving standardized likelihood distributions as post-trial probability distributions. Their use is unlikely to be controversial among statisticians whilst being intuitively appealing to clinicians. The second type of prior has a spike of probability mass at the point of no treatment effect. Varying the magnitude of the spike illustrates the sensitivity of the conclusions drawn to the degree of prior scepticism in a treatment effect. With both, graphical displays provide clinical readers with the opportunity to explore the results more fully. An example of how a clinical trial might be reported in the medical literature using these methods is given.     

A conceptual framework for international service-learning course planning: promoting a foundation for ethical practice in the physical therapy and occupational therapy professions

As physical therapy (PT) and occupational therapy (OT) educational programs endeavor to foster core values of social responsibility, justice, and altruism in an increasingly global community, the incorporation of local and international service-learning (ISL) into the curriculum is growing. Much of the research has focused on the measurement of student learning, with little written about the impact on the host community. Proponents of global health initiatives are calling for consideration of all stakeholders to ensure ethical practice. This paper explores the current literature related to PT and OT ISL and builds a conceptual framework for ISL course planning. The essential phases in the framework include: 1) pre-experience planning/preparation stage, 2) field immersion experience stage, and 3) postexperience stage. The essential elements are: 1) cultural competency training, 2) communication and coordination with community, 3) comprehensive assessment, and 4) strategic planning. The authors suggest this framework as a practical tool to structure ISL courses with an explicit emphasis on ethical concerns. Additionally, they seek to foster more dialogue and action related to the promotion of ethical practices in ISL in PT and OT education programs.     

What every teacher needs to know about clinical reasoning

CONTEXT: One of the core tasks assigned to clinical teachers is to enable students to sort through a cluster of features presented by a patient and accurately assign a diagnostic label, with the development of an appropriate treatment strategy being the end goal. Over the last 30 years there has been considerable debate within the health sciences education literature regarding the model that best describes how expert clinicians generate diagnostic decisions. PURPOSE: The purpose of this essay is to provide a review of the research literature on clinical reasoning for frontline clinical teachers. The strengths and weaknesses of different approaches to clinical reasoning will be examined using one of the core divides between various models (that of analytic (i.e. conscious/controlled) versus non-analytic (i.e. unconscious/automatic) reasoning strategies) as an orienting framework. DISCUSSION: Recent work suggests that clinical teachers should stress the importance of both forms of reasoning, thereby enabling students to marshal reasoning processes in a flexible and context-specific manner. Specific implications are drawn from this overview for clinical teachers.     

Problem-based learning: a case study in providing e-health education using the Internet

The current trend among many universities is to increase the number of courses available online. However, there are fundamental problems in transferring traditional education courses to virtual formats. Delivering current curricula in an online format does not assist in overcoming the negative effects on student motivation which are inherent in providing information passively. Using problem-based learning (PBL) online is a method by which computers can become a tool to encourage active learning among students. The delivery of curricula via goal-based scenarios allows students to learn at different rates and can successfully shift online learning from memorization to discovery. This paper reports on a Web-based e-health course that has been delivered via PBL for the past 12 months. Thirty distance-learning students undertook postgraduate courses in e-health delivered via the Internet (asynchronous communication). Data collected via online student surveys indicated that the PBL format was both flexible and interesting. PBL has the potential to increase the quality of the educational experience of students in online environments.     

Problem-based learning: a case study in providing e-health education using the Internet

The current trend among many universities is to increase the number of courses available online. However, there are fundamental problems in transferring traditional education courses to virtual formats. Delivering current curricula in an online format does not assist in overcoming the negative effects on student motivation which are inherent in providing information passively. Using problem-based learning (PBL) online is a method by which computers can become a tool to encourage active learning among students. The delivery of curricula via goal-based scenarios allows students to learn at different rates and can successfully shift online learning from memorization to discovery. This paper reports on a Web-based e-health course that has been delivered via PBL for the past 12 months. Thirty distance-learning students undertook postgraduate courses in e-health delivered via the Internet (asynchronous communication). Data collected via online student surveys indicated that the PBL format was both flexible and interesting. PBL has the potential to increase the quality of the educational experience of students in online environments.     

Deep learning for survival outcomes

Deep learning is a class of machine learning algorithms that are popular for building risk prediction models. When observations are censored, the outcomes are only partially observed and standard deep learning algorithms cannot be directly applied. We develop a new class of deep learning algorithms for outcomes that are potentially censored. To account for censoring, the unobservable loss function used in the absence of censoring is replaced by a censoring unbiased transformation. The resulting class of algorithms can be used to estimate both survival probabilities and restricted mean survival. We show how the deep learning algorithms can be implemented by adapting software for uncensored data by using a form of response transformation. We provide comparisons of the proposed deep learning algorithms to existing risk prediction algorithms for predicting survival probabilities and restricted mean survival through both simulated datasets and analysis of data from breast cancer patients.     

Actor-network theory and the OSCE: formulating a new research agenda for a post-psychometric era

The Objective Structured Clinical Examination (OSCE) is a ubiquitous part of medical education, although there is some debate about its value, particularly around possible impact on learning. Literature and research regarding the OSCE is most often situated within the psychometric or competency discourses of assessment. This paper describes an alternative approach: Actor-network-theory (ANT), a sociomaterial approach to understanding practice and learning. ANT provides a means to productively examine tensions and limitations of the OSCE, in part through extending research to include social relationships and physical objects. Using a narrative example, the paper suggests three ANT-informed insights into the OSCE. We describe: (1) exploring the OSCE as a holistic combination of people and objects; (2) thinking about the influences a checklist can exert over the OSCE; and (3) the implications of ANT educational research for standardisation within the OSCE. We draw from this discussion to provide a practical agenda for ANT research into the OSCE. This agenda promotes new areas for exploration in an often taken-for-granted assessment format.     

Methods for Developing Patient-Reported Outcome-Based Performance Measures (PRO-PMs)

ObjectiveTo recommend methods for assessing quality of care via patient-reported outcome-based performance measures (PRO-PMs) of symptoms, functional status, and quality of life.MethodsA Technical Expert Panel was assembled by the American Medical Association–convened Physician Consortium for Performance Improvement. An environmental scan and structured literature review were conducted to identify quality programs that integrate PRO-PMs. Key methodological considerations in the design, implementation, and analysis of these PRO-PM data were systematically identified. Recommended methods for addressing each identified consideration were developed on the basis of published patient-reported outcome (PRO) standards and refined through public comment. Literature review focused on programs using PROs to assess performance and on PRO guidance documents.ResultsThirteen PRO programs and 10 guidance documents were identified. Nine best practices were developed, including the following: provide a rationale for measuring the outcome and for using a PRO-PM; describe the context of use; select a measure that is meaningful to patients with adequate psychometric properties; provide evidence of the measure’s sensitivity to differences in care; address missing data and risk adjustment; and provide a framework for implementation, interpretation, dissemination, and continuous refinement.ConclusionMethods for integrating PROs into performance measurement are available.     

Teacher feedback to support oral language learning for young dual language learners

With the increasing numbers of dual language learners (DLLs) in early childhood programmes, the use of research-based and effective pedagogical strategies to support oral language learning is important. Early childhood classrooms can provide many opportunities to support language learning. Teacher feedback is an intentional teaching strategy to support oral language skills for both English and children's home language. We first present an overview of contexts for language learning in early childhood including a discussion of intentional teaching. Next, we discuss considerations for early childhood teachers’ provision of feedback to children regarding their oral language use. Finally, we present six teacher feedback strategies to foster oral English and/or home language skills for DLLs.     

Examining the presence of computer-assisted instruction in physical therapy education

The purpose of this study was to compare computer-assisted instruction (CAI) use in physical therapist (PT) and physical therapist assistant (PTA) education programs. Secondary purposes included examining faculty attitudes toward CAI, selection criteria, and evaluation criteria. Questionnaires were mailed to 389 PT and PTA programs. The total return rate was 52% (N = 201). Frequencies, percentages, and modes of responses were calculated for each questionnaire item. Eight percent of PT and 35% of PTA programs indicated that all faculty are using CAI, whereas 3% of PT and 6% of PTA reported that no faculty are using CAI. PT program respondents indicated using nine different types of CAI, whereas PTA programs indicated using eight different types. Fifty-seven percent of PT and 33% of PTA programs provide formal computer instruction. Forty-six percent of PT and 47% of PTA programs have a computer prerequisite. Nine percent of PT and 0% of PTA programs have mandatory computer ownership or lease policies. Positive aspects reported included improved knowledge of technology and independence with information gathering. Negative aspects reported included cost and time. Respondents were unsure whether different students benefit more from CAI (high or low aptitude, learning style) Results indicated that criteria for selecting CAI material are based on cost, ease of use, and compatibility with current systems rather than instructional design, and evaluation procedures included student outcomes and course evaluations. Contextual differences between PT and PTA probably account for differences in types and uses of CAI for PT and PTA programs. Instructor feedback is important when using CAI to teach psychomotor skills. More research is needed to determine if one type of student benefits more than others. Programs should consider the instructional design of CAI materials, in addition to practical aspects such as cost, and ease of use. We need to be aware of the instructional goals, content, and learners when selecting, designing, and using CAI.