Evaluation of teleconsultation systems

The aim of this study was to identify criteria used for evaluation of teleconsulatation systems in evaluation studies, and to estimate trend of their changes. Evaluation of teleconsultation (TC) systems was made on the basis of 82 evaluation studies from the inventory of evaluation studies in medical informatics consisting of all the information and communication technology (ICT) evaluation studies published in 1982-2002. In order to estimate trend of changes in using evaluation criteria, the whole period with at least one evaluation study, it means the period of 1995-2002 was divided to 2 of 4 years, 1995-1998 and 1999-2002. Proportions of evaluation studies according to any single criterion were calculated. Index of changes I(ci) for ith criterion was defined as difference of proportion of studies evaluating teleconsultation by using a certain criterion in the second period and the same proportion in the first period divided by the number of all studies evaluating teleconsultation systems. This research showed that hardware and technical aspects were the leading evaluation criterion, but it has been decreasing (from 74.2% in the first period to 43.1% in the second one). Satisfactions of both health care provider and patient achieved the highest increase in the second period (from 3.2 to 19.6% and from 0 to 15.7%, respectively). Appropriateness and cost of patient care were on the second place (from 22.6 to 33.3% and from 16.1 to 23.5%, respectively). Considering defined index of changes it can be concluded that there are four criteria for evaluation of teleconsultation systems, having a positive trend (in frequency of evaluation studies): user satisfaction (Ic=0.20), satisfaction of patient with patient care (Ic=0.19), appropriateness of patient care (Ic=0.13) and cost of patient care (Ic=0.09). Hardware and technical aspects shows negative trend in the number of evaluation studies (Ic=-0.38), as well as cost of IT (Ic=-0.10).