Doc'UMVF: two search tools to provide quality-controlled teaching resources in French to students and teachers

The UMVF project is a federation of medical teaching resources covering 32 medical schools in France. Today, the indexing of these resources is carried out manually by the CISMeF team at the University Hospital of Rouen. This indexing is based on MeSH thesaurus. We use a subset of SCORM metadata standard. This choice was defined in collaboration with the French Medical Virtual University consortium (French acronym: UMVF). Currently, with the UMVF searching tool (called Doc'UMVF), medical students can reach more than 3300 resources useful in their curriculum. Doc'UMVF is developed in close collaboration between the medical informatics laboratories of Rennes and Rouen. In this paper we present two complementary searching tools based on different methods and which are integrated and used to improve both the relevance and the coverage rate of the answers. A specific searching module has been built to retrieve specific resources concerning the National Medical Exam ENC ("Examen National Classant") is also available. Nevertheless, due to lack of time, numerous resources are not yet indexed. Therefore we have decided to use also automatic indexing method (Nomindex). This approach will be improved by further research works, resulting from Rouen and Geneva teams. After having built a searching meta-motor, our objective is to develop a meta-tool intended to index the whole set of digital pedagogical resources produced by the UMVF framework. This manual re-indexing will be carried out only for the most important resources (national references), with a more or less fine granularity.     

Collaborative environment for clinical reasoning and distance learning sessions

Background: The medical curriculum has changed with the adoption of the student-centered learning paradigm. Clinical reasoning learning (CRL) is used in order to develop and improve students’ clinical reasoning and problem-solving skills. Purpose: We have observed that, in complement to traditional CRL sessions, students commonly consult resources available on the internet. Based on this observation, our objective is to create computer tools to coordinate CRL sessions at distance, integrating these electronic resources at every step of the reasoning process. Material and methods: In order to create the system, we elaborated an object-oriented model of a computer-supported collaborative learning environment. The proposed system includes a local web-server to store electronic resources and a relational database to store their electronic addresses (urls). was used as the programming language. Results: We developed a set of cooperative platform-independent tools. This environment includes a communication tool. Multimedia data exchange is possible. Information is shared thanks to an electronic notepad and whiteboard tools. Perspectives: This learning environment will be integrated in the French Virtual Medical University project, and is intended to be used for undergraduate, internships, residency or continuing medical education.     

Factors affecting the effectiveness of biomedical document indexing and retrieval based on terminologies

ObjectiveThe aim of this work is to evaluate a set of indexing and retrieval strategies based on the integration of several biomedical terminologies on the available TREC Genomics collections for an ad hoc information retrieval (IR) task.Materials and methodsWe propose a multi-terminology based concept extraction approach to selecting best concepts from free text by means of voting techniques. We instantiate this general approach on four terminologies (MeSH, SNOMED, ICD-10 and GO). We particularly focus on the effect of integrating terminologies into a biomedical IR process, and the utility of using voting techniques for combining the extracted concepts from each document in order to provide a list of unique concepts.ResultsExperimental studies conducted on the TREC Genomics collections show that our multi-terminology IR approach based on voting techniques are statistically significant compared to the baseline. For example, tested on the 2005 TREC Genomics collection, our multi-terminology based IR approach provides an improvement rate of +6.98% in terms of MAP (mean average precision) (p < 0.05) compared to the baseline. In addition, our experimental results show that document expansion using preferred terms in combination with query expansion using terms from top ranked expanded documents improve the biomedical IR effectiveness.ConclusionWe have evaluated several voting models for combining concepts issued from multiple terminologies. Through this study, we presented many factors affecting the effectiveness of biomedical IR system including term weighting, query expansion, and document expansion models. The appropriate combination of those factors could be useful to improve the IR performance.     

A recent advance in the automatic indexing of the biomedical literature

The volume of biomedical literature has experienced explosive growth in recent years. This is reflected in the corresponding increase in the size of MEDLINE^®, the largest bibliographic database of biomedical citations. Indexers at the US National Library of Medicine (NLM) need efficient tools to help them accommodate the ensuing workload. After reviewing issues in the automatic assignment of Medical Subject Headings (MeSH^® terms) to biomedical text, we focus more specifically on the new subheading attachment feature for NLM’s Medical Text Indexer (MTI). Natural Language Processing, statistical, and machine learning methods of producing automatic MeSH main heading/subheading pair recommendations were assessed independently and combined. The best combination achieves 48% precision and 30% recall. After validation by NLM indexers, a suitable combination of the methods presented in this paper was integrated into MTI as a subheading attachment feature producing MeSH indexing recommendations compliant with current state-of-the-art indexing practice.     

Using n-gram method in the decomposition of compound medical diagnoses

Objective: Our goal in this study was to find an easy to implement method to detect compound medical diagnosis in Hungarian medical language and decompose them into expressions referring to a single disease. Methods: A corpus of clinical diagnoses extracted form discharge reports (3079 expressions, each of them referring to only one disease) was represented in an n-gram tree (a series of n consecutive word). A matching algorithm was implemented in a software, which is able to identify sensible n-grams existing both in test expressions and in the n-gram tree. A test sample of another 92 diagnoses was decomposed by two independent humans and by the software. The decompositions were compared with measure the recall and the precision of the method. Results: There was not full agreement between the decompositions of the humans, (which underlines the relevance of the problem). A consensus was arrived in all disagreed point by a third opinion and open discussion. The resulting decomposition was used as a gold standard and compared with the decomposition produced by the computer. The recall was 82.6% the precision 37.2%. After correction of spelling errors in the test sample the recall increased to 88.6% while the precision slightly decreased to 36.7%. Conclusion: The proposed method seems to be useful in decomposition of compound diagnostic expressions and can improve quality of diagnostic coding of clinical cases. Other statistical methods (like vector space methods or neural networks) usually offer a ranked list of candidate codes either for single or compound expressions, and do not warn the user how many codes should be chosen. We propose our method especially in a situation where formal NLP techniques are not available, as it is the case with scarcely spoken languages like Hungarian.     

The frequency distribution of age at natural menopause among Saudi Arabian women

Objective: Hormonal changes at the menopause are associated with the onset of a number of medical conditions. The distribution of age-at-menopause (AAM) within a given population can, therefore, indicate how the disease prevalence changes with age. The objective of this study was to estimate the distribution of AAM among Saudi Arabian women, in order to predict local trends in the prevalence of osteoporosis. Methods: Patient age, AAM, medical history and associated information for 858 Saudi Arabian women were extracted from a Dual Energy Absorptiometry database at King Faisal Specialist Hospital and Research Centre, resulting in an AAM distribution for 391 postmenopausal women with natural menopause. This was preprocessed using a Fast Fourier Transform 0.15 cycles/year low-pass filter, eliminating last-digit-preference errors and high frequency noise, and facilitating quantitative comparison with other published results. Results: Mean AAM was 48.94 years (S.E. 0.290 years) with a median of 50 years (25th/75th percentiles: 45 and 53 years, respectively). The AAM distribution was described by a quadruple-Gaussian curve with a major peak at almost 51 years and minor peaks at approximately 36, 44 and 59 years. Although both the central peaks were similar to that observed in other populations (UK, USA and Finland), the early menopause peak at 36 years was larger. The peak over 55 years may be unique to the Kingdom of Saudi Arabia. It may reflect local cultural and childbearing practices. Conclusions: Although the median menopause age and general shape of the AAM distribution in Saudi Arabia appear similar to that observed in the West, the parameters governing the distribution are different, and there is evidence that it may have a unique fourth peak.     

Improving the utility of MeSH® terms using the TopicalMeSH representation

ObjectiveTo evaluate whether vector representations encoding latent topic proportions that capture similarities to MeSH terms can improve performance on biomedical document retrieval and classification tasks, compared to using MeSH terms.Materials and methodsWe developed the TopicalMeSH representation, which exploits the ‘correspondence’ between topics generated using latent Dirichlet allocation (LDA) and MeSH terms to create new document representations that combine MeSH terms and latent topic vectors. We used 15 systematic drug review corpora to evaluate performance on information retrieval and classification tasks using this TopicalMeSH representation, compared to using standard encodings that rely on either (1) the original MeSH terms, (2) the text, or (3) their combination. For the document retrieval task, we compared the precision and recall achieved by ranking citations using MeSH and TopicalMeSH representations, respectively. For the classification task, we considered three supervised machine learning approaches, Support Vector Machines (SVMs), logistic regression, and decision trees. We used these to classify documents as relevant or irrelevant using (independently) MeSH, TopicalMeSH, Words (i.e., n-grams extracted from citation titles and abstracts, encoded via bag-of-words representation), a combination of MeSH and Words, and a combination of TopicalMeSH and Words. We also used SVM to compare the classification performance of tf-idf weighted MeSH terms, LDA Topics, a combination of Topics and MeSH, and TopicalMeSH to supervised LDA’s classification performance.ResultsFor the document retrieval task, using the TopicalMeSH representation resulted in higher precision than MeSH in 11 of 15 corpora while achieving the same recall. For the classification task, use of TopicalMeSH features realized a higher F1 score in 14 of 15 corpora when used by SVMs, 12 of 15 corpora using logistic regression, and 12 of 15 corpora using decision trees. TopicalMeSH also had better document classification performance on 12 of 15 corpora when compared to Topics, tf-idf weighted MeSH terms, and a combination of Topics and MeSH using SVMs. Supervised LDA achieved the worst performance in most of the corpora.ConclusionThe proposed TopicalMeSH representation (which combines MeSH terms with latent topics) consistently improved performance on document retrieval and classification tasks, compared to using alternative standard representations using MeSH terms alone, as well as, several standard alternative approaches.     

Disambiguation in the biomedical domain: The role of ambiguity type

Word Sense Disambiguation (WSD), the automatic identification of the meanings of ambiguous terms in a document, is an important stage in text processing. We describe a WSD system that has been developed specifically for the types of ambiguities found in biomedical documents. This system uses a range of knowledge sources. It employs both linguistic features, such as local collocations, and features derived from domain-specific knowledge sources, the Unified Medical Language System (UMLS) and Medical Subject Headings (MeSH). This system is applied to three types of ambiguities found in Medline abstracts: ambiguous terms, abbreviations with multiple expansions and names that are ambiguous between genes. The WSD system is applied to the standard NLM-WSD data set, which consists of ambiguous terms from Medline abstracts, and was found to perform well in comparison with previously reported results. The system’s performance and the contribution of each knowledge source depends upon the type of lexical ambiguity. 87.9% of the ambiguous terms are correctly disambiguated using a combination of linguistic features and MeSH terms, 99% of abbreviations are disambiguated by combining all knowledge sources, while 97.2% of ambiguous gene names are disambiguated using the MeSH terms alone. Analysis reveals that these differences are caused by the nature of each ambiguity type. These results should be taken into account when deciding which information to use for WSD and the level of performance that can be expected.     

Translating medical terminologies through word alignment in parallel text corpora

Developing international multilingual terminologies is a time-consuming process. We present a methodology which aims to ease this process by automatically acquiring new translations of medical terms based on word alignment in parallel text corpora, and test it on English and French. After collecting a parallel, English–French corpus, we detected French translations of English terms from three terminologies—MeSH, SNOMED CT and the MedlinePlus Health Topics. We obtained respectively for each terminology 74.8%, 77.8% and 76.3% of linguistically correct new translations. A sample of the MeSH translations was submitted to expert review and 61.5% were deemed desirable additions to the French MeSH. In conclusion, we successfully obtained good quality new translations, which underlines the suitability of using alignment in text corpora to help translating terminologies. Our method may be applied to different European languages and provides a methodological framework that may be used with different processing tools.     

Research summary of a scheme to ascertain the availability of speech information in the myoelectric signals of neck and head muscles using surface electrodes

The availability of speech information in the myoelectric signals (MES) of neck and head muscles was observed during five experiments conducted on two subjects. The MES of four channels, obtained using surface electrodes, was analog amplified, filtered and enhanced prior to digitization. Information was extracted at the rate of 20 points per second per channel using an average magnitude algorithm. The presence of speech related information was statistically verified with a pattern recognition algorithm based on the maximum likelihood algorithm. The results indicated that statistically significant (p = 0.01) information was present and that this scheme might be valuable in the future development of a vocal prosthesis.     

Improving MeSH classification of biomedical articles using citation contexts

Medical Subject Headings (MeSH) are used to index the majority of databases generated by the National Library of Medicine. Essentially, MeSH terms are designed to make information, such as scientific articles, more retrievable and assessable to users of systems such as PubMed. This paper proposes a novel method for automating the assignment of biomedical publications with MeSH terms that takes advantage of citation references to these publications. Our findings show that analysing the citation references that point to a document can provide a useful source of terms that are not present in the document. The use of these citation contexts, as they are known, can thus help to provide a richer document feature representation, which in turn can help improve text mining and information retrieval applications, in our case MeSH term classification. In this paper, we also explore new methods of selecting and utilising citation contexts. In particular, we assess the effect of weighting the importance of citation terms (found in the citation contexts) according to two aspects: (i) the section of the paper they appear in and (ii) their distance to the citation marker.We conduct intrinsic and extrinsic evaluations of citation term quality. For the intrinsic evaluation, we rely on the UMLS Metathesaurus conceptual database to explore the semantic characteristics of the mined citation terms. We also analyse the “informativeness” of these terms using a class-entropy measure. For the extrinsic evaluation, we run a series of automatic document classification experiments over MeSH terms. Our experimental evaluation shows that citation contexts contain terms that are related to the original document, and that the integration of this knowledge results in better classification performance compared to two state-of-the-art MeSH classification systems: MeSHUP and MTI. Our experiments also demonstrate that the consideration of Section and Distance factors can lead to statistically significant improvements in citation feature quality, thus opening the way for better document feature representation in other biomedical text processing applications.     

Systematic clinical methodology for validating bipolar-II disorder: data in mid-stream from a French national multi-site study (EPIDEP)

Background: This paper presents the methodology and clinical data in mid-stream from a French multi-center study (EPIDEP) in progress on a national sample of patients with DSM-IV major depressive episode (MDE). The aim of EPIDEP is to show the feasibility of validating the spectrum of soft bipolar disorders by practising clinicians. In this report, we focus on bipolar II (BP-II). Method: EPIDEP involves training 48 French psychiatrists in 15 sites; construction of a common protocol based on the criteria of DSM-IV and Akiskal (Soft Bipolarity), as well as criteria modified from the work of Angst (Hypomania Checklist), the Ahearn-Carroll Bipolarity Scale, HAM-D and Rosenthal Atypical Depression Scale; Semi-Structured Interview for Evaluation of Affective Temperaments (based on Akiskal-Mallya), self-rated Cyclothymia Scale (Akiskal), family history (Research Diagnostic Criteria); and prospective follow-up. Results: Results are presented on 250 (of the 537) MDE patients studied thus far during the acute phase. The rate of BP-II disorder which was 22% at initial evaluation, nearly doubled (40%) by systematic evaluation. As expected from the selection of MDE by uniform criteria, inter-group comparison between BP-II vs unipolar showed no differences on the majority of socio-demographic parameters, clinical presentation and global intensity of depression. Despite such uniformity, key characteristics significantly differentiated BP-II from unipolar: younger age at onset of first depression, higher frequency of suicidal thoughts and hypersomnia during index episode, higher scores on Hypomania Checklist and cyclothymic and irritable temperaments, and higher switching rate under current treatment. Eighty-eight percent of cases assigned to cyclothymic temperament by clinicians (with a cut-off of 10/21 items on self-rated cyclothymia) were recognized as BP-II. Evaluation of this temperament by clinician and patient correlated at a highly significant level (r=0.73; p<0.0001). Cyclothymia and hypomania were also correlated significantly (r=0.51; p<0.001). Limitation: In a study conducted in diverse clinical settings, it was not possible to assure that clinicians making affective diagnoses were blind to the various temperamental measures. However, bias was minimized by the systematic and/or semi-structured nature of all evaluations. Conclusion: With a systematic search for hypomania, 40% of major depressive episodes were classified as BP-II, of which only half were known to the clinicians at study entry. Cyclothymic temperamental dysregulation emerged as a robust clinical marker of BP-II disorder. These data indicate that clinicians in diverse practice settings can be trained to recognize soft bipolarity, leading to changes in diagnostic practice at a national level.     

Initial evaluation of a continuous speech recognition program for radiology

The aims of this work were to measure the accuracy of one continuous speech recognition product and dependence on the speaker's gender and status as a native or nonnative English speaker, and evaluate the product's potential for routine use in transcribing radiology reports. IBM MedSpeak/Radiology software, version 1.1 was evaluated by 6 speakers. Two were nonnative English speakers, and 3 were men. Each speaker dictated a set of 12 reports. The reports included neurologic and body imaging examinations performed with 6 different modalities. The dictated and original report texts were compared, and error rates for overall, significant, and subtle significant errors were computed. Error rate dependence on modality, native English speaker status, and gender were evaluated by performing ttests. The overall error rate was 10.3 +/- 3.3%. No difference in accuracy between men and women was found; however, significant differences were seen for overall and significant errors when comparing native and nonnative English speakers (P = .009 and P = .008, respectively). The speech recognition software is approximately 90% accurate, and while practical implementation issues (rather than accuracy) currently limit routine use of this product throughout a radiology practice, application in niche areas such as the emergency room currently is being pursued. This methodology provides a convenient way to compare the initial accuracy of different speech recognition products, and changes in accuracy over time, in a detailed and sensitive manner.     

Initial evaluation of a continuous speech recognition program for radiology

The aims of this work were to measure the accuracy of one continuous speech recognition product and dependence on the speaker's gender and status as a native or nonnative English speaker, and evaluate the product's potential for routine use in transcribing radiology reports. IBM MedSpeak/Radiology software, version 1.1 was evaluated by 6 speakers. Two were nonnative English speakers, and 3 were men. Each speaker dictated a set of 12 reports. The reports included neurologic and body imaging examinations performed with 6 different modalities. The dictated and original report texts were compared, and error rates for overall, significant, and subtle significant errors were computed. Error rate dependence on modality, native English speaker status, and gender were evaluated by performing ttests. The overall error rate was 10.3 +/- 3.3%. No difference in accuracy between men and women was found; however, significant differences were seen for overall and significant errors when comparing native and nonnative English speakers (P = .009 and P = .008, respectively). The speech recognition software is approximately 90% accurate, and while practical implementation issues (rather than accuracy) currently limit routine use of this product throughout a radiology practice, application in niche areas such as the emergency room currently is being pursued. This methodology provides a convenient way to compare the initial accuracy of different speech recognition products, and changes in accuracy over time, in a detailed and sensitive manner.     

Inductive creation of an annotation schema for manually indexing clinical conditions from emergency department reports

Evaluating automated indexing applications requires comparing automatically indexed terms against manual reference standard annotations. However, there are no standard guidelines for determining which words from a textual document to include in manual annotations, and the vague task can result in substantial variation among manual indexers. We applied grounded theory to emergency department reports to create an annotation schema representing syntactic and semantic variables that could be annotated when indexing clinical conditions. We describe the annotation schema, which includes variables representing medical concepts (e.g., symptom, demographics), linguistic form (e.g., noun, adjective), and modifier types (e.g., anatomic location, severity). We measured the schema's quality and found: (1) the schema was comprehensive enough to be applied to 20 unseen reports without changes to the schema; (2) agreement between author annotators applying the schema was high, with an F measure of 93%; and (3) the authors made complementary errors when applying the schema, demonstrating that the schema incorporates both linguistic and medical expertise.     

Selective automated indexing of findings and diagnoses in radiology reports

The recent improvements in capabilities of desktop computers and communications networks give impetus for the development of clinical image repositories that can be used for patient care and medical education. A challenge in the use of these systems is the accurate indexing of images for retrieval performance acceptable to users. This paper describes a series of experiments aiming to adapt the SAPHIRE system, which matches text to concepts in the UMLS Metathesaurus, for the automated indexing of image reports. A series of enhancements to the baseline system resulted in a recall of 63% but a precision of only 30% in detecting concepts. At this level of performance, such a system might be problematic for users in a purely automated indexing environment. However, if the ability to retrieve images in repositories based on content in their reports is desired by clinical users, and no other current systems offer this functionality, then follow-up research questions include whether these imperfect results would be useful in a completely or partially automated indexing environment and/or whether other approaches can improve upon them.