Application of MCAT questions as a testing tool and evaluation metric for knowledge graph–based reasoning systems

“Knowledge graphs” (KGs) have become a common approach for representing biomedical knowledge. In a KG, multiple biomedical data sets can be linked together as a graph representation, with nodes representing entities, such as “chemical substance” or “genes,” and edges representing predicates, such as “causes” or “treats.” Reasoning and inference algorithms can then be applied to the KG and used to generate new knowledge. We developed three KG‐based question‐answering systems as part of the Biomedical Data Translator program. These systems are typically tested and evaluated using traditional software engineering tools and approaches. In this study, we explored a team‐based approach to test and evaluate the prototype “Translator Reasoners” through the application of Medical College Admission Test (MCAT) questions. Specifically, we describe three “hackathons,” in which the developers of each of the three systems worked together with a moderator to determine whether the applications could be used to solve MCAT questions. The results demonstrate progressive improvement in system performance, with 0% (0/5) correct answers during the first hackathon, 75% (3/4) correct during the second hackathon, and 100% (5/5) correct during the final hackathon. We discuss the technical and sociologic lessons learned and conclude that MCAT questions can be applied successfully in the context of moderated hackathons to test and evaluate prototype KG‐based question‐answering systems, identify gaps in current capabilities, and improve performance. Finally, we highlight several published clinical and translational science applications of the Translator Reasoners.

Study Highlights

• WHAT IS THE CURRENT KNOWLEDGE ON THE TOPIC?

Knowledge graphs (KGs) are common approaches for representing biomedical knowledge. KGs are typically tested and evaluated from a software engineering perspective.

• WHAT QUESTION DID THIS STUDY ADDRESS?

We explored a team‐based approach to evaluate three prototype KG‐based question‐answering systems through the application of Medical College Admission Test (MCAT) questions within a moderated “hackathon” setting.

• WHAT DOES THIS STUDY ADD TO OUR KNOWLEDGE?

We demonstrate that MCAT questions can be applied successfully in the context of moderated hackathons to test and evaluate KG‐based question‐answering systems, identify gaps, and improve performance.

• HOW MIGHT THIS CHANGE CLINICAL PHARMACOLOGY OR TRANSLATIONAL SCIENCE?

We expect that our approach will have broad application in biomedical software development efforts such as ours.     

Experiences of acupuncturists in a placebo-controlled, randomized clinical trial

BACKGROUND: This paper describes the experiences of 8 licensed acupuncturists in a placebo-controlled randomized clinical trial (RCT). This information is important to the design and conduct of high-quality trials. METHODS: We conducted a RCT (N=135) with a 2-week placebo run-in followed by 4 weeks of twice-weekly treatments comparing genuine to sham acupuncture (using the Streitberger placebo needle) in the treatment of arm pain caused by repetitive use. At the end of this study, we conducted written structured interviews with 8 participating acupuncturists. The acupuncturists were not aware of the study's results at the time of these interviews. The questions focused on their experiences in the study, adherence to study protocols, their thoughts about the technical and ethical issues involved in using a sham needling device, and their expectations of trial outcomes. The questions were motivated by expressions of concerns the acupuncturists raised in feedback groups during the course of the study, and our desire to improve further trials. RESULTS: The acupuncturists differed widely in their comfort levels with the research methods used, their adherence to the study protocol, and their expectations of trial outcomes. CONCLUSIONS: We conclude that careful monitoring of acupuncturists, including observation of treatments and frequent meetings to support them throughout the trial, is necessary to maintain a high degree of quality control.     

Graduate Education for the Future: New Models and Methods for the Clinical and Translational Workforce

This paper is the third in a five‐part series on the clinical and translational science educational pipeline, and it focuses on strategies for enhancing graduate research education to improve skills for interdisciplinary team science. Although some of the most cutting edge science takes place at the borders between disciplines, it is widely perceived that advancements in clinical and translational science are hindered by the “siloed” efforts of researchers who are comfortable working in their separate domains, and reluctant to stray from their own discipline when conducting research. Without appropriate preparation for career success as members and leaders of interdisciplinary teams, talented scientists may choose to remain siloed or to leave careers in clinical and translational science all together, weakening the pipeline and depleting the future biomedical research workforce. To address this threat, it is critical to begin at what is perhaps the most formative moment for academics: graduate training. This paper focuses on designs for graduate education, and contrasts the methods and outcomes from traditional educational approaches with those skills perceived as essential for the workforce of the future, including the capacity for research collaboration that crosses disciplinary boundaries.     

Statistical Methodology

How many underlying characteristics (or factors) does a set of survey questions measure? When subjects answer a set of self-report questions, is it more appropriate to analyze the questions individually, to pool responses to all of the questions to form one global score, or to combine subsets of related questions to define multiple underlying factors? Factor analysis is the statistical method of choice for answering such questions. When researchers have no idea beforehand about what factors may underlie a set of questions, they use exploratory factor analysis to infer the best explanatory model from observed data “after the fact.” If, on the other hand, researchers have a hypothesis beforehand about the underlying factors, then they can use confirmatory factor analysis (CFA) to evaluate how well this model explains the observed data and to compare the model's goodness-of-fit with that of other competing models. This article describes the basic rules and building blocks of CFA: what it is, how it works, and how researchers can use it. The authors begin by placing CFA in the context of a common research application—namely, assessing quality of medical outcome using a patient satisfaction survey. They then explain, within this research context, how CFA is used to evaluate the explanatory power of a factor model and to decide which model or models best represent the data. The information that must be specified in the analysis to estimate a CFA model is highlighted, and the statistical assumptions and limitations of this analysis are noted. Analyzing the responses of 1,614 emergency medical patients to a commonly-used “patient satisfaction” questionnaire, the authors demonstrate how to: 1) compare competing factor-models to find the best-fitting model; 2) modify models to improve their goodness-of-fit; 3) test hypotheses about relationships among the underlying factors; 4) examine mean differences in “factor scores”; and 5) refine an existing instrument into a more streamlined form that has fewer questions and better conceptual and statistical precision than the original instrument. Finally, the role of CFA in developing new instruments is discussed.     

Research Fundamentals: III. Elements of a Research Protocol for Clinical Trials

Abstract. A clinical trial is a powerful technique for evaluating the effectiveness of an experimental intervention. The initial stages of planning a clinical trial involve choosing and refining a research question, selecting a study design, and deciding on appropriate statistical tests and sample sizes. The success of the study depends upon how well these issues are thought out in advance, and how they can be put into practice. The protocol is the written document that allows the investigator to communicate details of how the research question will be answered. In the following article, the basic components of the research protocol are described. Issues related to quality control, data entry, and pilot testing are discussed. This is the third in a series of research fundamental concept papers, written by members of the SAEM Research Committee.     

PCOR,CER, and CBPR: Alphabet Soup or Complementary Fields of Health Research?

Comparative effectiveness research (CER) and community‐ based participatory research (CBPR) are two fields of research that do not have a history of strong collaboration. However, CER and CBPR researchers could benefit from interdisciplinary collaboration to design and implement relevant, timely, action‐oriented research. This commentary explores field‐specific definitions of stakeholders and then outlines various roles stakeholders might play within grant‐funded research. Questions such as “What stakeholders should be involved?” and “How are stakeholders involved?” are addressed. The goal of this commentary is to highlight how the expertise and experiences of CBPR investigators can enhance the field of CER and to describe strategies for encouraging stakeholder involvement in CER research through the lens of CBPR. It is recommended that a team‐based approach to conducting stakeholder‐engaged CER encourages multiple stakeholders and “end users” to contribute their diverse expertise to the research process and contributes to the development of research with an increased likelihood of improving patient health and healthcare.     

Clinimetrics corner: choosing appropriate study designs for particular questions about treatment subgroups

Background

Many clinicians and researchers believe that there are subgroups of people with spinal pain who respond differently to treatment and have different prognoses. There has been considerable interest in this topic recently. However, problems occur when conclusions about subgroups are made that are inappropriate given the randomized controlled trial design used. The research design to choose, when developing a study protocol that investigates the effect of treatment subgroups, depends on the particular research question. Similarly, the inferences that can be drawn from an existing study will vary, depending on the design of the trial.

Objectives

This paper discusses the randomized controlled trial designs that are suitable to answer particular questions about treatment subgroups. It focuses on trial designs that are suitable to answer four questions: (1) ‘Is the treatment effective in a pre-specified group of patients?’; (2) ‘Are outcomes of treatment applied using a subgrouping clinical reasoning process, better than a control treatment?’; (3) ‘Are the outcomes for a patient subgroup receiving a particular treatment (compared to a control treatment) better than for patients not in the subgroup who receive the same treatment?’; and (4) ‘Are outcomes for a number of treatments better if those treatments are matched to patients in specific subgroups, than if the SAME treatments are randomly given to patients?’. Illustrative examples of these studies are provided.

Conclusion

If the clinical usefulness of targeting treatments to subgroups of people is to be determined, an important step is a shared understanding of what different RCT designs can tell us about subgroups.     

Secondary analysis: theoretical, methodological, and practical considerations.

Secondary analysis, which involves the use of existing data sets to answer new research questions, is an increasingly popular methodological choice among researchers who wish to investigate particular research questions but lack the resources to undertake primary data collections. Much time loss and considerable frustration may result, however, if researchers begin secondary analyses without an awareness of the distinctive methodological and practical challenges involved. This article highlights difficulties that may arise when researchers use data from previous clinical research projects, including theoretical issues and problems involving sampling, measurement, and external and ecological validity. It also offers practical suggestions for undertaking a secondary analysis and criteria for evaluating secondary analyses.     

Reported challenges in nurse-led randomised controlled trials: an integrative review of the literature

Scand J Caring Sci; 2011; 25; 194–200
Reported challenges in nurse‐led randomised controlled trials: an integrative review of the literature Aims: The purpose of this integrative literature review was to explore and discuss the methodological challenges nurse researchers report after conducting nurse‐led randomised controlled trials in clinical hospital settings. Our research questions were (i) what are the most commonly experienced methodological barriers and challenges and (ii) which lessons can be learned from these reports? Method: A database search of Medline, Cinahl, The Cochrane Library and Embase was carried out. Variants of the following search terms were used: randomised controlled trial, controlled clinical trial, clinical trial, methods, nursing research, nursing, research, challenges, barriers, nurse’s role, nurse attitude, attitude of health personnel. Findings: The literature on reported challenges and barriers between 1999 and 2009 showed that the most often experienced problems were (i) sufficient patient recruitment, (ii) nonadherence to research protocols and (iii) economic and organisational obstacles. These three challenges and barriers were inter‐related and all were affected by time and timing. Conclusion: Randomised controlled trials are complex, expensive, time‐consuming and highly demanding for researchers and the clinical nursing staff. Two lessons learned from this integrative review can be highlighted. First, we recommend researchers openly to share their experiences of barriers and challenges. They should describe factors that may have inhibited the desired outcome. Second, efforts to improve the collaboration between nurse researchers and clinicians, including education, training and support may increase the success rate and quality of nurse‐led studies using the randomised controlled trial.     

Incidental findings: A practical protocol for reporting elevated depressive symptoms in behavioral health research

Intervention studies conducted in caregivers often focus on improving mental health. Consequently, researchers may discover incidental findings like elevated depressive symptoms. Researchers have an ethical obligation to report incidental findings to participants, but no protocols exist for reporting behavioral health symptoms. The purpose of this paper was to describe a protocol for reporting elevated depressive symptoms to participants, based on the protocol used in a national randomized clinical trial of stress-reduction methods for 348 grandmothers raising grandchildren. Each questionnaire included the CES-D scale, and was scored immediately after completion. We established a cut-off score of 30 based on previous research. A registered nurse on the research team called participants with scores over 30 and ascertained whether the participant 1) was aware of the problem and 2) had sought help, and then offered additional resources. Overall, 94 (27%) participants had a CES-D score > 30. The majority (91%) were aware of the problem. About a third of the participants were on medication for their symptoms, and a third were seeing a therapist. Nine participants were not aware they had depressive symptoms. This paper outlines the ethical premise for developing our protocol, details of protocol development, and discussion for how research teams can apply this protocol to their work.     

Clinical trials of homeopathy and placebo: analysis of a scientific debate

OBJECTIVE: Homeopathy is controversial, primarily because of the use of medicines diluted beyond the Avogadro limit. This article examines the scientific debate on whether homeopathy can have effects greater than placebo in humans. METHODS: Five rigorous English-language clinical studies published in high-impact journals that favored homeopathy were identified. Letters and other articles written in response to these articles were then retrieved and analyzed. RESULTS: Much of the content of responses to positive homeopathic research was rhetorical in nature and antagonistic in tone: homeopathic researchers were accused of bias and of being in the pocket of homeopathic manufacturers; arguments were raised that are demonstrably inaccurate or irrelevant to the interpretation of research; words such as "magic" or "potion" were used to paint homeopathic research as inherently unscientific. Other commentators argued that based on theoretical grounds homeopathy could not possibly be effective, any positive trial result must be due to bias. Surprisingly few responses raised substantive methodologic or statistical issues. Many of these are clearly inaccurate or irrelevant. The possibility of publication bias-selective publication of positive results-was raised by some commentators and remains an important criticism of the apparently positive nature of the clinical trial evidence. The most persuasive argument against accepting differences between homeopathy and placebo on the basis of current evidence is that homeopathy is scientifically implausible and so requires more evidence than normal. CONCLUSIONS: Investigators undertaking clinical research in homeopathy need to be responsive both to the dangers of publication bias and to the requirement for stronger than usual levels of evidence.     

Treatments to enhance recovery from the vegetative and minimally conscious states: ethical issues surrounding efficacy studies

Randomized double-blind placebo-controlled trials have been argued to provide the strongest test of efficacy and, as such, are important tools for advancing the evidence base supporting rehabilitation treatment. However, such trials present difficult ethical issues, because one group, by definition, receives no treatment for the condition being studied. In the case of an experimental treatment that is available only within a research protocol, a 50% chance of receiving the desired treatment may be sufficient to motivate enrollment. However, many rehabilitation treatments that need further study are available outside of research protocols and are perceived as low risk, making the advantages of research participation less clear and the task of weighing the pros and cons of research participation more difficult. In this article, we discuss a placebo-controlled trial currently underway in which this issue is combined with a number of other complicating factors, such as the inability of study participants to provide their own informed consent, and the catastrophic nature of the disability under study. We examine whether other research designs could successfully answer efficacy questions in this area, and we discuss the ethical and psychosocial issues involved in planning the trial and seeking enrollment.     

Variation in administration of cyclophosphamide and mesna in the treatment of childhood malignancies

The objective of this study was to describe the variation in preparation and administration of cyclophosphamide, mesna, and hydration for the treatment of childhood malignancies within clinical trial protocol documents. All cyclophosphamide-containing cooperative group (Pediatric Oncology Group) protocols that were open at Dana-Farber Cancer Institute in April 1998 were evaluated. Among the 14 active protocols, there were 23 unique cyclophosphamide regimens. Marked variation existed in infusion rate, fluid type, and volume used for admixing cyclophosphamide and mesna, as defined in the “Treatment” section of the protocols that we evaluated. Further variation was found in the type, amount, and rate of infusion of prehydration and posthydration fluid. Internal inconsistency existed within the protocols pertaining to the administration methods described in the “Agent Information,” “Treatment,” and “Consent” sections of the written documents. Clinical trial protocol documents serve as reference material for health care providers who prescribe, dispense, and administer protocol chemotherapy. Misinterpretation of protocol documents and clinician orders are contributing factors in serious and deadly medication errors. Internal inconsistency within protocol documents and variation in drug administration across protocols is a potential source of error. We recommend improved accuracy, clarity, and internal consistency of protocol documents to improve patient safety and compliance with protocol specifications. In addition, the use of standard concentrations, volumes, and methods of administration of chemotherapeutic agents and accompanying fluids is recommended.     

Chiropractic program changes facilitated by the European Council on Chiropractic Education Accreditation reports:: A mixed methods audit and thematic analysis

ObjectiveThe European Council on Chiropractic Education (ECCE) currently accredits 10 programs throughout Europe and South Africa. It is assumed that ECCE evaluation activities lead to changes to the chiropractic programs but no systematic evaluation as to whether this is true, and the extent of changes has previously been done. The purpose of this study was to obtain feedback from program heads as to whether ECCE evaluation reports facilitated changes/improvements to their programs and to identify their reported changes.MethodsThis was a mixed methods audit study using questionnaires with 2 sections. Closed statements requesting the degree of change to each section of the “Standards” based on ECCE evaluation reports (substantial, some, none) were analyzed using frequencies. Written responses identifying the specific changes made based on previous evaluation reports were evaluated independently by 3 researchers using a modified “thematic analysis” approach.ResultsAll 10 accredited programs responded. Seven of the 10 programs (70%) reported “some” or “substantial” changes to ≥ 6 sections of the ECCE Standards. The most common section with reported changes was “Educational Program” (8 of 10). “Educational Resources” had the largest number of programs reporting “substantial changes” (4) and was the second most common section to have reported changes. The main themes identified emphasized changes in “infrastructure, equipment and faculty,” “increasing evidence-based practice,” and “instilling a research culture in faculty and students.”ConclusionECCE accreditation processes facilitate changes to the chiropractic programs, particularly in the areas of improved infrastructure and faculty, research, and evidence-based practice.     

The Evolution of Master Protocol Clinical Trial Designs: A Systematic Literature Review

PurposeRecent years have seen a change in the way that clinical trials are being conducted. There has been a rise of designs more flexible than traditional adaptive and group sequential trials which allow the investigation of multiple substudies with possibly different objectives, interventions, and subgroups conducted within an overall trial structure, summarized by the term master protocol. This review aims to identify existing master protocol studies and summarize their characteristics. The review also identifies articles relevant to the design of master protocol trials, such as proposed trial designs and related methods.MethodsWe conducted a comprehensive systematic search to review current literature on master protocol trials from a design and analysis perspective, focusing on platform trials and considering basket and umbrella trials. Articles were included regardless of statistical complexity and classified as reviews related to planned or conducted trials, trial designs, or statistical methods. The results of the literature search are reported, and some features of the identified articles are summarized.FindingsMost of the trials using master protocols were designed as single-arm (n = 29/50), Phase II trials (n = 32/50) in oncology (n = 42/50) using a binary endpoint (n = 26/50) and frequentist decision rules (n = 37/50). We observed an exponential increase in publications in this domain during the last few years in both planned and conducted trials, as well as relevant methods, which we assume has not yet reached its peak. Although many operational and statistical challenges associated with such trials remain, the general consensus seems to be that master protocols provide potentially enormous advantages in efficiency and flexibility of clinical drug development.ImplicationsMaster protocol trials and especially platform trials have the potential to revolutionize clinical drug development if the methodologic and operational challenges can be overcome.     

Is biomedical research a good investment?

As the US addresses its budget dilemma, the easiest items to cut are those with the longest-term payoff. Research stands out among this group. Biomedical research has already been markedly reduced, and further reductions appear to be in store. As a frequent witness in Congressional hearings on such matters, here I discuss the challenge of assessing the value of investments in biomedical research.Providing mathematical proof of the positive overall cost-benefit relationship of investments in research, particularly biomedical research, is at best a formidable task; nonetheless, the body of evidence in support of the proposition is substantial.Several years ago, a bipartisan group of the Senate and House of Representatives asked the National Academies to assess trends in America’s global competitiveness, and it was my honor to chair that effort. The twenty participants, including university presidents, CEOs, Nobel laureates, and former and future presidential appointees, unanimously concluded that the two most important actions to assure a high-quality life for future Americans were to significantly increase the nation’s investment in scientific research and to repair the US K–12 education system (1).Prior to the above effort, the Congressionally established bipartisan Hart-Rudman Commission on National Security, of which I was also a member, unanimously concluded that “. . . second only to a weapon of mass destruction detonating on an American city, we can think of nothing more dangerous than a failure to manage properly science, technology, and education for the common good” (2).In the UK, Prime Minister Margaret Thatcher said, “. . . although basic science can have colossal economic rewards, they are totally unpredictable. . . . Nevertheless, the value of Faraday’s work today must be higher than the capitalization of all shares on the stock exchange. . .” (3).In China, Wen Jiabao, then Premier of the State Council of China, had this to say on the subject: “The history of modernization is in essence a history of scientific and technological progress. Scientific discovery and technological inventions have brought about new civilizations, modern industries, and the rise and fall of nations. . . . I firmly believe that science is the ultimate revolution” (4).But with such broad emphasis the question arises as to why the US spends more on potato chips than on energy research or why, over the past decade, it has reduced its investment in NIH research in real terms by 22 percent and dropped from first to seventh place among OECD (Organisation for Economic Co-operation and Development) nations in the fraction of GDP devoted to basic research (5). Or why our states have disinvested in their support of higher education per student in real terms by a median 27 percent in the past five years alone (6).Having testified before Congress dozens of times in support of increased investment in research, I have personally experienced the difficulty of making a succinct, compelling, analytical case as to the cost effectiveness of research, certainly including biomedical research. In the latter instance, personal anecdotes are powerful, but the specific does not prove the general.Similarly, I have reviewed dozens of studies that examine the cost-benefit relationship of biomedical research, and, while many of these investigations offer valuable insights to the professional engaged in the field, none seems to offer what might be termed compelling “proof” — especially to a skeptic.In the case of specific curiosity-driven research projects, there never can be an a priori cost-benefit defense. Yet in 1945, Vannevar Bush, writing in “Science the Endless Frontier,” pointed out that “discoveries pertinent to medical progress have often come from remote and unexpected sources, and it is certain that this will be true in the future” (7). Among the canonical cases have been studies of butterflies, seals, and, serendipitously, molds, that have led to cancer treatments; advances in surgical procedures; and the discovery of penicillin, respectively.Assessments of the benefits of biomedical research suffer a variety of afflictions. Many seek to place a monetary value on human life (the Environmental Protection Agency’s value of statistical life is $7.4 M; ref. 8), but few citizens, certainly including members of Congress, will agree upon any such esoteric metric. Other analytical studies in effect simply stipulate the answer, say, by assuming that half of the gain in quality-adjusted life expectancy is attributable to biomedical research. Similarly, few taxpayers are motivated by tortured counts of research papers, citations, patents, or even Nobel Prizes. Although many assessments are based on purely economic considerations, even overlooking such complications as those above, one would seem to be poorly advised to argue that the justification of biomedical research is to make money, save money, or produce jobs.Then there is the matter of attribution. For example, what fraction of the investment in robotics and computational research that played such an important role in deciphering the human genome should be charged to the cost of the medical benefits that derive from the latter research? And should not the cost of implementing the results of basic research be included when assessing the cost-benefit ratio of that research? How much of the extension of statistical life expectancy due to the decline in smoking should be credited to biomedical research and how much merely considered a cultural change, albeit triggered by research? Should the health benefits accrued by future generations or the citizens of other nations be credited to advancements made by specific research entities, such as the NIH? And how does one treat so-called research “failures” — that years later not infrequently prove to be of value?In such regards, biological systems are extraordinarily complex entities that interact in complex ways, both internally and with their environments, the latter including social factors not readily quantified. The challenge when assessing cost-benefit relationships of such systems is that, if all possible interactions are included, the task becomes insurmountable and, if all important interactions are not included, the results become irrelevant. Professor Friedrich Wiekhorst of the Max Planck Institute derived the equation that defines the number of possible states that exist for a system having a specified number of elements, each of which interacts with every other element in a binary manner, the simplest of all manners. He called his equation, “The Monster” — and with good reason. In the case of a two-element system there are only four possible states. But with even ten elements the number exceeds the count of stars in our galaxy. So what of the human brain with its 10¹¹ cells and 10³ connections per cell?While in no way disparaging the informative value of mathematical assessments of cost-benefit relationships in biomedical research, particularly when taking the form of parametric analyses, it would appear that evaluations that depend on the judgment of experts may have to play a far greater role in the assessment process than has been generally accepted.Another possibility, of course, is simply to ask the citizens who pay the bills and reap the benefits of biomedical research to offer their views. It is likely that few would prefer to live in the environment that existed in, say, 1900, with its polio, tuberculosis, rubella, yellow fever, measles, and tetanus and absence of antibiotics, insulin, artificial joints, laser eye surgery, and stents — and a life expectancy of 47 years. The cost portion of that assessment hinges upon what a person would be willing to pay to have the benefit of the advancements — medical and nonmedical — that have taken place since 1900. In my own youth, few argued for reducing the investment in research on potential polio vaccines in order to fund means of improving the efficiency of producing iron lungs.In the case of the basic research supported by the NIH, the cost per US citizen is about 25 cents per day (9). In polls conducted by Research!America, over half of the respondents indicated a willingness to have their taxes increased by an amount that, assuming the same willingness to pay on behalf of their children, would permit the NIH research budget to be increased by more than fifty percent (10).The nation’s citizens can presumably afford such an investment, given that they currently spend an average each day of 56 cents on snack foods, 95 cents on illegal drugs, 98 cents on store-bought alcoholic beverages, 18 cents on spectator sports, and 93 cents on legal tobacco products (11–13).The question nonetheless arises with regard to investment in biomedical research, “How much is enough?” Many people today would probably agree that having invested more in research to counter Ebola would have been a sound decision. And with 18 percent of America’s GDP being devoted to healthcare, there is a great deal of leverage to be realized from research that reduces the purely financial burden of providing health care. But, with a mere 0.2 percent of GDP currently being devoted to federally funded basic research of all kinds and about 0.1 percent to biomedical research (5), it would seem that we are far from any danger of overinvesting.It has not infrequently been pointed out to me by members of Congress during Congressional hearings that “America has a budget problem.” Indeed, it does . . . a very serious budget problem. But, as a businessperson from an industry that once lost 40 percent of its employees and three-fourths of its companies in a five-year market downturn, it is very clear that even when drastically cutting back overall there are some areas in which one must simultaneously increase funding. The key, if one wishes to survive the long term, is to understand the difference between spending for investment and spending for consumption.Unfortunately, an air-tight mathematical proof of the full cost and full benefit of America’s investment in biomedical research seems beyond our grasp. Even so, research in the biomedical sciences appears to be among the soundest investments the nation can make on behalf of its citizenry.     

Rehabilitation After Anterior Cruciate Ligament Injury: Review of Current Literature and Recommendations

Purpose of ReviewAnterior cruciate ligament reconstruction (ACLR) is a common surgical procedure with an estimated 120,000 cases performed in the USA each year. Physical therapy plays a critical role in the successful recovery of both surgically and non-surgically managed patients. Interestingly, ACL rehabilitation protocols vary greatly with little consensus among practitioners. Nonetheless, there has been agreement over the last decade to shift from conservative, standardized length protocols to more accelerated, individualized protocols that vary in length and modalities based on patient-specific findings and preferences. This review summarizes the most recent trends, opinions, and modalities in ACL rehabilitation research, with a specific focus on novel methods to treat the specific psychosocial needs of ACL deficient patients.Recent FindingsWe found that new protocols emphasize early weight bearing, open kinetic chain (OKC) exercises, and other alternative modalities such as neuromuscular electrical stimulation and blood flow restriction. We also found a recent trend toward the use of clinical milestones to determine when a patient is ready for the next phase of a “step-up” rehabilitation program. One particularly nascent topic of research is the inclusion of methods to treat the psychosocial impacts of ACL injury, recovery, and the anxiety around return to sport.SummaryRehabilitation strategy has become increasingly patient-dependent, and the new modalities being utilized are accelerating patient recovery. Return to sport is a particularly important factor for many ACLR patients, and recovery has an important psychological component that has only recently been addressed in the literature, with positive preliminary findings.     

International research project to devise a protocol to test the effectiveness of acupuncture on painful shoulder

OBJECTIVES: To describe in detail the questions and deliberations leading to the development of a methodologically rigorous protocol to test the effectiveness of acupuncture on painful shoulder. DESIGN: Randomized controlled trial using three groups, including one control group. SETTINGS/LOCATION: A hospital in the north of Italy treating at least 8-10 painful shoulders a day, with physician/acupuncturists, physiotherapists, and assessors available to participate in the study. SUBJECTS: Sixty patients with monolateral painful shoulder. A list of exclusion criteria is given. INTERVENTIONS: Acupuncture + mobilization; mobilization alone (control); ear acupuncture + mobilization. OUTCOME MEASURES: Severity of pain, scored on a visual analogue scale (VAS) and on the McGill Pain Questionnaire and restriction of range of motion (ROM) of the shoulder joint. Voluntary use of pain medication, Tenoxicam 20 mg, was added to the protocol. RESULTS: The research team developed a protocol and methodology that avoids the common flaws and difficulties of previous clinical trials on acupuncture. CONCLUSIONS: Physicians and acupuncture specialists will benefit from the advice and support of a project group consisting of experienced clinicians, researchers, and statisticians when designing and preparing clinical trials on acupuncture and other complementary and alternative therapies.     

Decision making in family medicine: Randomized trial of the effects of the InfoClinique and Trip database search engines

Objective

To compare the ability of users of 2 medical search engines, InfoClinique and the Trip database, to provide correct answers to clinical questions and to explore the perceived effects of the tools on the clinical decision-making process.

Design

Randomized trial.

Setting

Three family medicine units of the family medicine program of the Faculty of Medicine at Laval University in Quebec city, Que.

Participants

Fifteen second-year family medicine residents.

Intervention

Residents generated 30 structured questions about therapy or preventive treatment (2 questions per resident) based on clinical encounters. Using an Internet platform designed for the trial, each resident answered 20 of these questions (their own 2, plus 18 of the questions formulated by other residents, selected randomly) before and after searching for information with 1 of the 2 search engines. For each question, 5 residents were randomly assigned to begin their search with InfoClinique and 5 with the Trip database.

Main outcome measures

The ability of residents to provide correct answers to clinical questions using the search engines, as determined by third-party evaluation. After answering each question, participants completed a questionnaire to assess their perception of the engine’s effect on the decision-making process in clinical practice.

Results

Of 300 possible pairs of answers (1 answer before and 1 after the initial search), 254 (85%) were produced by 14 residents. Of these, 132 (52%) and 122 (48%) pairs of answers concerned questions that had been assigned an initial search with InfoClinique and the Trip database, respectively. Both engines produced an important and similar absolute increase in the proportion of correct answers after searching (26% to 62% for InfoClinique, for an increase of 36%; 24% to 63% for the Trip database, for an increase of 39%; P = .68). For all 30 clinical questions, at least 1 resident produced the correct answer after searching with either search engine. The mean (SD) time of the initial search for each question was 23.5 (7.6) minutes with InfoClinique and 22.3 (7.8) minutes with the Trip database (P = .30). Participants’ perceptions of each engine’s effect on the decision-making process were very positive and similar for both search engines.

Conclusion

Family medicine residents’ ability to provide correct answers to clinical questions increased dramatically and similarly with the use of both InfoClinique and the Trip database. These tools have strong potential to increase the quality of medical care.