A New Look at Information Retrieval Evaluation: Proposal for Solutions

This is the fourth and the final in the series of the papers on an application of the systems analytic approach to evaluation of information retrieval (IR). In the previous papers terminological and evaluation problems associated with IR were identified and discussed, and it was proposed that the systems analytic approach could provide solutions to these problems. A review of existing systems analytic approaches in IR was given, and a conceptual framework and two models of IR were presented. This paper discusses how the proposed conceptual framework and the models can be used to improve terminology of IR, to analyze the relevance-based method of recall and precision, and to propose new approaches to evaluate relevance relationships and search engines. The proposed evaluation approaches are compared amongst each other and with the existing methods. The paper is concluded by a summary of the contributions of the research and an identification of the directions for a future research in IR evaluation.     

Computer-based information systems models

A synthesis of theories from the management information systems (MIS) field provided the conceptual framework to describe how to design and develop effective computer-based information systems. An analysis of activities and interviews with nurse managers in a tertiary care medical center provided the basis for describing five major nursing department functions and the information requirements arising from those functions. The MIS framework was applied to the nursing department information requirements in order to derive MNIS, a normative model of a nursing information system, which was then used to “bench mark” the current status of commercially available nursing information systems. The researchers compared the MNIS with 28 nursing information systems that vendors are currently developing or marketing. The comparison revealed large gaps between nursing department needs and vendor-provided solutions.     

Reference Standards, Judges, and Comparison Subjects: Roles for Experts in Evaluating System Performance

Medical informatics systems are often designed to perform at the level of human experts. Evaluation of the performance of these systems is often constrained by lack of reference standards, either because the appropriate response is not known or because no simple appropriate response exists. Even when performance can be assessed, it is not always clear whether the performance is sufficient or reasonable. These challenges can be addressed if an evaluator enlists the help of clinical domain experts. 1) The experts can carry out the same tasks as the system, and then their responses can be combined to generate a reference standard. 2)The experts can judge the appropriateness of system output directly. 3) The experts can serve as comparison subjects with which the system can be compared. These are separate roles that have different implications for study design, metrics, and issues of reliability and validity. Diagrams help delineate the roles of experts in complex study designs.Medical informatics systems are often designed to carry out complex tasks and to perform at the level of human experts. For example, diagnostic systems use clinical evidence, such as admission history, clinical signs, and diagnostic results, to produce probabilities of disease or lists of diagnoses. Therapeutic systems suggest interventions tailored to patients. Information retrieval systems produce lists of documents that are relevant to some topic. Image processing systems detect features in a digital image.Evaluating the function of these systems can be difficult.¹ Determining the appropriate responses that a system should have produced, deciding whether the system output matches an appropriate response, and even deciding whether a given level of performance is good enough are all challenges. Clinical domain experts have frequently been enlisted to address these challenges. In this paper, we review the many designs that have incorporated human experts into system evaluation, enumerate the roles that experts may play in evaluation, and provide a framework for describing designs. We draw largely on examples from clinical informatics and from information retrieval, but the framework and issues are more broadly applicable across medical informatics.     

HIS/BUI: A Conceptual Model for Bottom-Up Integration of Hospital Information Systems

Many successful applications of information systems have been introduced and implemented in hospitals. However, the integration of these applications into a cohesive hospital-wide information system has proved to be more complicated to develop and difficult to accomplish than expected. This paper introduces HIS/BUI, a framework for bottom-up integration of hospital information systems, and demonstrates its application through a real-life case scenario. The scope of the proposed framework is the integration of heterogeneous clinical, administrative, and financial information elements of a hospital into a unified system environment. Under the integrated architecture, all existing local applications are preserved and interconnected to an information hub that serves as a central medical and administrative data warehouse.     

基于实物期权理论的企业信息系统建设项目投资决策

在实物期权理论的框架下,根据信息系统建设项目投资的特点,分析得出可以将系统建设的投资过程看成是含有多种实物期权的序列决策。在此基础上,利用二叉树方法建立单个实物期权单阶段的定价模型,以及多阶段复合实物期权的定价模型,以期为企业在投资建设信息系统时,做出更加科学合理的投资决策提供理论依据。     

Healthcare Process Modeling to Phenotype Clinician Behaviors for Exploiting the Signal Gain of Clinical Expertise (HPM-ExpertSignals): Development and evaluation of a conceptual framework

ObjectiveThere are signals of clinicians’ expert and knowledge-driven behaviors within clinical information systems (CIS) that can be exploited to support clinical prediction. Describe development of the Healthcare Process Modeling Framework to Phenotype Clinician Behaviors for Exploiting the Signal Gain of Clinical Expertise (HPM-ExpertSignals).Materials and MethodsWe employed an iterative framework development approach that combined data-driven modeling and simulation testing to define and refine a process for phenotyping clinician behaviors. Our framework was developed and evaluated based on the Communicating Narrative Concerns Entered by Registered Nurses (CONCERN) predictive model to detect and leverage signals of clinician expertise for prediction of patient trajectories.ResultsSeven themes—identified during development and simulation testing of the CONCERN model—informed framework development. The HPM-ExpertSignals conceptual framework includes a 3-step modeling technique: (1) identify patterns of clinical behaviors from user interaction with CIS; (2) interpret patterns as proxies of an individual’s decisions, knowledge, and expertise; and (3) use patterns in predictive models for associations with outcomes. The CONCERN model differentiated at risk patients earlier than other early warning scores, lending confidence to the HPM-ExpertSignals framework.DiscussionThe HPM-ExpertSignals framework moves beyond transactional data analytics to model clinical knowledge, decision making, and CIS interactions, which can support predictive modeling with a focus on the rapid and frequent patient surveillance cycle.ConclusionsWe propose this framework as an approach to embed clinicians’ knowledge-driven behaviors in predictions and inferences to facilitate capture of healthcare processes that are activated independently, and sometimes well before, physiological changes are apparent.     

A conceptual framework for measuring clinical problem-solving.

Most attempts to measure clinical competence start by modeling the clinician's problem-solving process. The conflicting data from studies built around this approach suggest the need for rethinking the underlying concepts. Presented is a conceptual framework for clinical competence which is a natural expansion of earlier approaches. The framework is based upon defining the domain in which the clinician functions as the starting point for measuring clinical competence. There are three dimensions to the framework-problem-solving process, clinical discipline, and context of care. The intersection of the dimensions defines the clinical practice domain to be measured. For each domain specific problems can be identified and clinicians asked to demonstrate competence in resolving them.     

An Interdisciplinary Computer-based Information Tool for Palliative Severe Pain Management

Objectives

As patient care becomes more collaborative in nature, there is a need for information technology that supports interdisciplinary practices of care. This study developed and performed usability testing of a standalone computer-based information tool to support the interdisciplinary practice of palliative severe pain management (SPM).

Design

A grounded theory-participatory design (GT-PD) approach was used with three distinct palliative data sources to obtain and understand user requirements for SPM practice and how a computer-based information tool could be designed to support those requirements.

Results

The GT-PD concepts and categories provided a rich perspective of palliative SPM and the process and information support required for different SPM tasks. A conceptual framework consisting of an ontology and a set of three problem-solving methods was developed to reconcile the requirements of different interdisciplinary team members. The conceptual framework was then implemented as a prototype computer-based information tool that has different modes of use to support both day-to-day case management and education of palliative SPM. Usability testing of the computer tool was performed, and the tool tested favorably in a laboratory setting.

Conclusion

An interdisciplinary computer-based information tool can be developed to support the different work practices and information needs of interdisciplinary team members, but explicit requirements must be sought from all prospective users of such a tool. Qualitative methods such as the hybrid GT-PD approach used in this research are particularly helpful for articulating computer tool design requirements.     

The BioIntelligence Framework: a new computational platform for biomedical knowledge computing

Breakthroughs in molecular profiling technologies are enabling a new data-intensive approach to biomedical research, with the potential to revolutionize how we study, manage, and treat complex diseases. The next great challenge for clinical applications of these innovations will be to create scalable computational solutions for intelligently linking complex biomedical patient data to clinically actionable knowledge. Traditional database management systems (DBMS) are not well suited to representing complex syntactic and semantic relationships in unstructured biomedical information, introducing barriers to realizing such solutions. We propose a scalable computational framework for addressing this need, which leverages a hypergraph-based data model and query language that may be better suited for representing complex multi-lateral, multi-scalar, and multi-dimensional relationships. We also discuss how this framework can be used to create rapid learning knowledge base systems to intelligently capture and relate complex patient data to biomedical knowledge in order to automate the recovery of clinically actionable information.     

Regulation of animal experimentation

This article examines the nature of the controversy between animal activists and the scientific community over the need for and desirability of additional regulation of animal experimentation. We critically review the existing federal and state regulatory environment and discuss a number of proposals for new federal legislation for regulating the use of animals in scientific experimentation. We identify and discuss three factors that encompass the issues that are at the core of the controversy and use these to suggest a conceptual framework in which to analyze schemes for regulating animal experimentation. Finally, we discuss the implications that such regulation will have for animal experimentation and conclude that it is neither necessary nor desirable.     

场景分析在公共卫生信息概念模型构建中的应用研究

目的:探索场景分析方法在公共卫生概念模型构建中的应用。方法:应用场景分析概念和功能,对公共卫生业务活动各要素进行归类、特化与泛化,探索公共卫生概念模型框架,并应用公共卫生业务实例验证其有效性。结果构建了公共卫生领域信息活动场景,初步形成了公共卫生概念模型框架。结论:场景分析是公共卫生概念建模过程中抽象业务活动、提取模型类及类关联的有效方法。     

Benchmarking Electronic Medical Records Initiatives in the US: a Conceptual Model

This article provides a conceptual model for benchmarking the use of clinical information systems within healthcare organizations. Additionally, it addresses the benefits of clinical information systems which include the reduction of errors, improvement in clinical decision-making and real time access to patient information. The literature suggests that clinical information systems provide financial benefits due to cost-savings from improved efficiency and reduction of errors. As a result, healthcare organizations should adopt such clinical information systems to improve quality of care and stay competitive in the marketplace. Our research clearly documents the increased adoption of electronic medical records in U.S. hospitals from 2005 to 2007. This is important because the electronic medical record provides an opportunity for integration of patient information and improvements in efficiency and quality of care across a wide range of patient populations. This was supported by recent federal initiatives such as the establishment of the Office of the National Coordinator of Health Information Technology (ONCHIT) to create an interoperable health information infrastructure. Potential barriers to the implementation of health information technology include cost, a lack of financial incentives for providers, and a need for interoperable systems. As a result, future government involvement and leadership may serve to accelerate widespread adoption of interoperable clinical information systems.     

面向区域医疗的图像互操作系统设计

区域医疗信息网络提供了区域下多种医学信息平台数据的有效互通。随着医疗机构每年产生愈加庞大的医疗信息。互操作将在区域医疗信息网络中发挥越来越重要的作用。主要介绍了面向区域医疗的图像互操作系统框架,由区域医疗中心提供用户认证服务和存放用户的诊疗信息,用户通过Internet登陆区域医疗中心进行访问,若想要查看详细的诊疗图像信息,则由诊疗医院建立临时图像数据库将相应的内容向患者开放。     

The anatomy of online information for physicians.

Online medical networked information (OMNI) is one of the newest and fastest growing types of information sources for physicians. The authors present an organizational framework for understanding the range of available OMNI sources and discuss the practical applications, strengths and limitations of online resources. Physicians can now gain access on line to a wealth of information relating to many aspects of clinical medicine and can consult interactively with colleagues on clinical and research questions. The limitations of networked online resources include lack of access, difficulty in navigating online systems and the potential for fraudulent use. Net-worked online systems are growing in popularity and may become integral to medical practice as barriers to efficient use are overcome.     

循证医学主题综述知识系统UpToDate研究分析

利用信息技术和数据库技术提供的知识管理服务是医学知识服务的重要模式,构建专业的医学知识服务系统、搭建知识服务平台,是有效实施医学知识服务的重要手段。循证医学专题综述知识系统,已经成为国外提供医学知识服务的重要形式之一,出现了诸如UpToDate、DynaMed、BMJC1inicalEvidence、MDConsult等一批高质量的医学知识系统。以UpToDate知识系统为例,从知识系统的框架结构、系统功能、系统特色和评价研究等方面详细介绍了国外循证医学主题综述知识系统的发展现状,为国内医学知识服务平台和知识系统的开发提供参考和借鉴。     

面向服务的医院信息集成方法研究

当前医院业务系统林立,医疗系统之间互操作较为困难,难以适应医疗信息化建设的发展趋势,需要研究医院信息集成的新模式,以满足临床路径、电子病历、区域医疗的新需求。为此,研究了面向服务的医院信息集成模式,给出了系统互操作基本框架,阐述了病人主索引、数据综合浏览视图等实现技术。     

Strategic management of technostress

The article proposes the concept of technostress and makes a strong recommendation for conducting research based on key researchable hypotheses. A conceptual framework of technostress is suggested to provide some focus to future research. A number of technostress management strategies are put forward, including strategic technological planning, organization culture development, technostress monitoring systems, and technouser self-development programs. The management of technostress is compared to the chaining of Prometheus, which, left uncontrolled, can create havoc in an organization. The authors believe that organizations have a responsibility to introduce, diffuse, and manage computer technology in such a way that it is congruent with the principles of sound, supportive, and humanistic management.     

Use of Information Systems in Air Force Medical Treatment Facilities in Strategic Planning and Decision-Making

An exploratory study used Ansoff’s strategic planning model as a framework to assess perceived effectiveness of information systems in supporting strategic business plan development at Air Force medical treatment facilities (MTFs). Results showed information systems were most effective in supporting historical trend analysis, strategic business plans appeared to be a balance of operational and strategic plans, and facilities perceived a greater need for new clinical, vice administrative, information systems to support strategic planning processes. Administrators believed information systems should not be developed at the local level and perceived information systems have the greatest impact on improving clinical quality outcomes, followed by ability to deliver cost effective care and finally, ability to increase market share.     

医院信息系统软件架构的比较与选择

医院信息系统架构决定了系统功能的实现和性能及软硬件的开发成本、实施成本及难度。介绍目前医院HIS系统各种软件结构和各自优缺点,阐述医院信息系统对软件的10大功能要求,并对使用各种架构软件开发出的系统所能达到的功能进行量化比较,得出混合结构是目前软件架构最优选择的结论。