Planning an evaluation and estimating its cost

Scientific evaluations can provide funders and program administrators with useful input concerning where to allocate scarce service dollars. But evaluation itself is legitimately the subject of cost concerns. What will it cost to get the various potential benefits of a scientific evaluation? What are the evaluation options of a given program, given its budget and the size and expertise of its staff? Where can a program administrator go for consulting help? This article provides a helpful framework for answering these important questions. First, it describes two types of evaluations which vary in both the questions they can answer and in their consequent cost. Second, it delineates and briefly describes the technical elements or steps required by each evaluation type. Third, it describes the nature and potential variability of costs associated with each technical step. Finally, it steers the reader to available sources of expert information and help.     

Leadership in turbulent times is spiritual

In the past decade, two formerly estranged domains, spirituality and work, have been increasingly linked. Why has this happened? This articles posits that this strange union is an unavoidable consequence of living in a turbulent time. As our world grows more chaotic and unpredictable, leaders must respond to questions that have historically only been answered through spiritual traditions: How do I cope with incertainty? How do I help others find meaning in their lives? What are my values? How can I act with courage and integrity? Leaders also play a key role in helping people recognize that life is cyclical, unpredictable, and unstable and that they must engage it as such. This is well-explored in spiritual traditions, but not in traditional management literature. Several organizational and personal practices are offered as simple ways to develop leadership capacity for leading in turbulence.     

Wissen umsetzen

It is indisputable that preventive and health-promoting interventions are also effective for older people. In order to put this knowledge into practice, several important questions have to be answered first. Which preventive goals have priority? Which contents have to be conveyed in the context of “healthy ageing”? Which strategies of intervention have proven effective so far? How can older people be addressed in their surroundings? Working Group 3 on “Healthy Ageing” of the German Forum/Platform for Prevention and Health Promotion is currently discussing these questions. The following article presents the proceedings as well as the results of Working Group 3 up to now.     

A BETR way to track toxic pollutants

Why are chemicals that have been banned for over a decade still being detected in the environment? How is it that chemicals and other pollutants such as particulate matter turn up hundreds or even thousands of miles from their sources? These are just two of the questions that a team of American and Canadian scientists hope will be answered using a model they have developed to track pollutants across North America. In the short term, the model's developers are concentrating on using it to assess rates of exposure from food sources, but they hope to expand the model to include global data, a step they hope will help policy makers better understand the consequences of pollution.     

The COMPAQH project: researches on quality indicators in hospitals

After three years, the coordination de la mesure de la performance pour l'amélioration de la qualité hospitalière (COMPAQH) project can deliver its first findings and consider new perspectives of development. Which indicators are diffused? Under which criteria are they assessed? Which interhospital variability is observed? How to consider their application into hospitals? Which balance can we define between internal and external use? And finally, which consideration can we give to this program of quality measurement? This article addresses these different questions, giving a state of the development of this program.     

Cardiovascular epidemiology in a changing world--challenges to investigators and the National Heart, Lung, and Blood Institute

Over the past 60 years, revolutionary discoveries made by epidemiologists have contributed to marked declines in cardiovascular disease morbidity and mortality. Now, in an era of increasingly constrained resources, researchers in cardiovascular epidemiology face a number of challenges that call for novel, paradigm-shifting approaches. In this paper, the authors pose to the community 4 critical questions: 1) How can we avoid wasting resources on studies that provide little incremental knowledge? 2) How can we assure that we direct our resources as economically as possible towards innovative science? 3) How can we be nimble, responding quickly to new opportunities? 4) How can we identify prospectively the most meritorious research questions? Senior program staff at the National Heart, Lung, and Blood Institute invite the epidemiology community to join them in an ongoing Web-based blog conversation so that together we might develop novel approaches that will facilitate the next generation of high-impact discoveries.     

Using Evaluation Techniques To Select Health Education Programs

Health educators often are faced with the job of selecting a new program to meet particular objectives. When one or more options is available, the selection process may involve the application of rather sophisticated evaluation skills whether or not the health educator is aware of the fact. By consciously applying evaluation techniques, health educators can increase the likelihood that a new program will meet their needs. The article provides guidelines for choosing an effective program by posing questions that normally are raised at the end of a program during the summative evaluation but are equally relevant at its beginning during the program selection stage. Seeking the answers to these questions before a program is implemented can contribute much toward ensuring the desired outcomes will be achieved.     

The public cost of kidney disease

As part of the general concern with the cost of medical care, attention is being focused on the End-Stage Renal Disease program. However, much confusion surrounds the issue of the cost of end-stage renal care. This article seeks to clarify some of the points inherent in this issue by addressing the following questions: Who is covered by the program? How expensive is the program? Why have costs increased? Can costs be reduced without sacrificing quality or access? And, how can one object to a program that extends life?     

Managed care: quality management differentiates the product

Comprehensive quality management is a goal that requires major corporate commitment to implement and maintain. The best results in a quality management program will be achieved when all components are tied together in a comprehensive program. To do that, a significant investment in personnel and equipment is required. As the benefits of having a program such as the one as described above become more widely known and accepted, more and more managed care insurers and providers will adopt these or similar standards. The question, "How do you know that patients are getting high-quality and sufficient care?" cannot be answered today by insurers who do not have such a program.     

Successful strategic planning: creating clarity

Most healthcare organizations have a strategic plan of some kind. Many of these organizations also have difficulty translating their strategic plan into specific actions that result in successful performance. In the worst cases, this can jeopardize the viability of the organization. The trouble lies in a lack of clarity in what a strategic plan is and what it should do for the organization. This article will answer key questions such as: What is strategy and how does it fit with other commonly used constructs such as mission, vision, and goals? What criteria can be used to determine if something is truly strategic to the organization? What are the phases of the strategy lifecycle? How do approaches for dealing with uncertainty, such as scenario planning, fit with organizational strategic planning? How can a meaningful IT strategy be developed if the organization strategy is lacking? What principles should guide a good IT planning process?     

Patient safety: mindful, meaningful, and fulfilling

Five years after the landmark report of the Institute of Medicine To Err Is Human (Kohn, Corrigan, and Donaldson 2000), many are asking, "Is U.S. healthcare safer?" A number of articles addressing this question have been written, interviews with nationally recognized patient safety leaders have been published, and governing boards of many healthcare organizations are examining reports of care provided by their institutions. Robert M. Wachter, writing in the November 2004 issue of Health Affairs, concludes that, "At this point, I would give our efforts an overall grade of C+, with striking areas of progress tempered by clear opportunities for improvement." We describe in this article the pursuit of a culture of safety at William Beaumont Hospital in Royal Oak, Michigan. Our experience has offered us the opportunity to ponder a number of key questions: How does leadership guide an organization toward a culture of safety? Does culture truly drive behavior, or is it really the reverse? How can a culture of safety be measured or observed? What levels of resources and commitment are required for success? Is safety all about systems and processes, or are core values also involved? What role does the patient play in ensuring safe care? We attempt to offer guidance, and share lessons learned, for each of these important questions.     

Comprehensive evaluation model for nursing education

The proposed model provides structure for determining decisions to be made, questions to be answered, and sources of information about evaluation. A sample of the kinds of decisions that can be made for the various matrix elements of the evaluation model is reflected. Examples of questions that could be asked (why) in order to evaluate various aspects of a program (what) and of whom these questions could be asked (who) are provided. The model as described is useful in comprehensively delineating all aspects of the system. The flexibility of the system allows for the unique needs and aspects of the individual school to emerge. The model also provides a framework for organizing and implementing a plan for total program evaluation.     

Evaluation issues in the development of expert systems in medicine.

Evaluation of medical decision support software (MDSS)--computer programs to assist health professionals with diagnostic and/or therapeutic decisions--has not kept pace with the development of such programs. This article describes the following formative evaluation issues that must be addressed by developers of MDSS to evaluate these programs properly: (1) How can systematic feedback be obtained about an evolving program? (2) How can enough data to evaluate the program be obtained? (3) How much instruction is necessary? (4) What are the most important aspects for users to evaluate? and (5) How can the appropriate use of a developing MDSS be assured? Data from an ongoing evaluation of an existing MDSS, Quick Medical Reference, are used to illustrate the issues and to suggest recommendations for addressing them.     

Accumulating evidence from independent studies: what we can win and what we can lose

When asking ‘what is known’ about a drug or therapy or program at any time, both researchers and practitioners often confront more than a single study. Facing a variety of findings, where conflicts may outweigh agreement, how can a reviewer constructively approach the task? In this discussion, I will outline some questions that can only be answered by examining a group of independent studies. I will also discuss some pitfalls that sometimes swamp the benefits we can gain from synthesis. Most of these pitfalls are avoidable if anticipated early in a review. The benefits of a quantitative review include information about how to match a treatment with the most promising recipients; increasing the statistical power to detect a significant new treatment; telling us when ‘contextual effects’ are important; helping us to assess the stability and robustness of treatment effectivenes; and informing us when research finds are especially sensitive to investigators' research design. The pitfalls include aggregating data from studies on different populations; aggregating when there is more than one underlying measure of central tendency; and emphasizing an average outcome when partitioning variance gives far more useful information.     

Measurement of progression in Alzheimer's disease: a clinician's perspective

Patients with Alzheimer's disease (AD) and their families must confront two fundamental truths. First, AD is a uniformly progressive disease that ultimately results in debilitating cognitive impairment. Second, although there is now evidence that some medications may produce transient improvement or possibly even slowing of disease progression, there is currently no way to halt the progression of AD. Consequently, patients and their families consistently ask the following questions: 1. What new management issues can be anticipated, and when? 2. What clinical developments are atypical and merit evaluation for a superimposed problem? 3. Is the current treatment working? These questions can only be answered by referring to the natural course of AD, and specifically, information regarding measures of functional impairment and how they change over time. The information that is currently available on this topic is limited and often embodies implicit assumptions that have not been adequately tested. This information will be reviewed, and directions for future research will be outlined.     

The future of health planning

What is the future of health planning? Will the profession survive? What will health planners be doing thirty years from now? The answers to these questions depend in part on prognostication and in part on the collective will of the profession to shape its own destiny. You ought to be doing something about the future in addition to waiting around for it. If by nature you are impatient for the good life, don't wait for it--invent it. The science of design is the sovereign science of the 21st century. A design for health requires us to answer three questions: (1) What should be done? (values) (2) How can it be done? (knowledge) (3) Who can do it? (power) To the degree health planners help society in its search for answers to these three questions, it will secure its own future as a profession. To the degree it fails in this quest, it will continue in its current mechanistic preoccupations and will eventually pass into irrevelance and obscurity. The impact of twenty-four societal trends are projected, as are the effects of the trends on the health planning profession and the images of its practitioners.     

The amino acid requirements of disease

It is now clear that accidental injury, surgery, infection, cancer and psychosocial stress activate new metabolic pathways that consume amino acids. For example, immune activation appears to alter glutamine and arginine metabolism, acute phase protein synthesis demands the increased provision of the aromatic and sulfur amino acids, while an increased rate of glutathione turnover increases the cysteine requirement of the body. Critically, under many stressful conditions, the accompanying anorexia means that these new demands have to be met from the patient's protein stores. This sets up what is, in effect, an internal amino acid balance in which the supply (muscle protein) does not match the demand. The result is tissue loss and eventual malnutrition. In our view, decisions as to whether specific amino acid supplementation of the stressed patient will be beneficial should consider three crucial questions. What should the supplement contain? How should it be administered? At what stage should it be administered? Although we argue that the first question may be close to being answered, the answers to the second and third still remain uncertain. Even so, there is a good case for the serious consideration of the use of targeted amino acid supplements appropriate to the nature of different chronic conditions.     

What questions do patients undergoing lower extremity joint replacement surgery have?

Background

The value of the Internet to deliver preoperative education would increase if there was variability in questions patients want answered. This study's goal was to have patients consulting an orthopedic surgeon about undergoing either a total hip arthroplasty (THA) or a total knee arthroplasty (TKA) rate the importance of different questions concerning their care.

Methods

We assembled questions patients might have about joint replacement surgery by analyzing the literature and querying a pilot group of patients and surgeons. Twenty-nine patients considering undergoing THA and 19 patients considering TKR completed a written survey asking them to rate 30 different questions, with a 5 point Likert scale from 1 (least important) – 5 (most important).

Results

For patients considering THA or TKR, the 4 highest rated questions were: Will the surgery affect my abilities to care for myself?, Am I going to need physical therapy?, How mobile will I be after my surgery?, When will I be able to walk normally again? The mean percentage disagreement was 42% for questions answered by TKR patients and 47% for the THA group. Some patients gave a high rating to questions lowly rated by the rest of the group.

Conclusions

Although there was enough agreement to define a core set of questions that should be addressed with most patients considering THA or TKA, some of the remaining questions were also highly important to some patients. The Web may offer a flexible medium for accommodating this large variety of information needs.

     

Nurse practitioners in prevention and health education

The Where? How? Who will pay? and Who will provide? of preventive medicine and health education can be answered. And a promising answer is the family nurse practitioner, who may be able to handle 60 to 80 percent of routine tasks normally done by physicians and to provide health education as a part of routine care.