Visions and strategies to improve evaluation of health information systems. Reflections and lessons based on the HIS-EVAL workshop in Innsbruck

BACKGROUND: Health care is entering the Information Society. It is evident that the use of modern information and communication technology offers tremendous opportunities to improve health care. However, there are also hazards associated with information technology in health care. Evaluation is a means to assess the quality, value, effects and impacts of information technology and applications in the health care environment, to improve health information applications and to enable the emergence of an evidence-based health informatics profession and practice. OBJECTIVE: In order to identify and address the frequent problems of getting evaluation understood and recognised, to promote transdisciplinary exchange within evaluation research, and to promote European cooperation, the Exploratory Workshop on "New Approaches to the Systematic Evaluation of Health Information Systems" (HIS-EVAL) was organized by the University for Health Sciences, Medical Informatics and Technology (UMIT), Innsbruck, Austria, in April 2003 with sponsorship from the European Science Foundation (ESF). Methods: The overall program was structured in three main parts: (a). discussion of problems and barriers to evaluation; (b). defining our visions and strategies with regard to evaluation of health information systems; and (c). organizing short-term and long-term activities to reach those visions and strategies. RESULTS: The workshop participants agreed on the Declaration of Innsbruck (see ), comprising four observations and 12 recommendations with regard to evaluation of health information systems. Future activities comprise European networking as well as the development of guidelines and standards for evaluation studies. CONCLUSION: The HIS-EVAL workshop was intended to be the starting point for setting up a network of European scientists working on evaluation of health information systems, to obtain synergy effects by combining the research traditions from different evaluation fields, leading to a new dimension and collaboration on further research on information systems' evaluation.     

'CHEATS': a generic information communication technology (ICT) evaluation framework

This paper describes a generic framework for the evaluation of information communication technologies. This framework, CHEATS, utilises both qualitative and quantitative research methods and has proved appropriate in multiple clinical settings including telepsychiatry, teledermatology and teleeducation. The paper demonstrates how a multidisciplinary approach is essential when evaluating new and emerging technologies, particularly when such systems are implemented in real service as opposed to a research setting.