Retrieving Clinical Evidence: A Comparison of PubMed and Google Scholar for Quick Clinical Searches

Background

Physicians frequently search PubMed for information to guide patient care. More recently, Google Scholar has gained popularity as another freely accessible bibliographic database.

Objective

To compare the performance of searches in PubMed and Google Scholar.

Methods

We surveyed nephrologists (kidney specialists) and provided each with a unique clinical question derived from 100 renal therapy systematic reviews. Each physician provided the search terms they would type into a bibliographic database to locate evidence to answer the clinical question. We executed each of these searches in PubMed and Google Scholar and compared results for the first 40 records retrieved (equivalent to 2 default search pages in PubMed). We evaluated the recall (proportion of relevant articles found) and precision (ratio of relevant to nonrelevant articles) of the searches performed in PubMed and Google Scholar. Primary studies included in the systematic reviews served as the reference standard for relevant articles. We further documented whether relevant articles were available as free full-texts.

Results

Compared with PubMed, the average search in Google Scholar retrieved twice as many relevant articles (PubMed: 11%; Google Scholar: 22%; P<.001). Precision was similar in both databases (PubMed: 6%; Google Scholar: 8%; P=.07). Google Scholar provided significantly greater access to free full-text publications (PubMed: 5%; Google Scholar: 14%; P<.001).

Conclusions

For quick clinical searches, Google Scholar returns twice as many relevant articles as PubMed and provides greater access to free full-text articles.     

A Search Engine to Access PubMed Monolingual Subsets: Proof of Concept and Evaluation in French

BackgroundPubMed contains numerous articles in languages other than English. However, existing solutions to access these articles in the language in which they were written remain unconvincing.ObjectiveThe aim of this study was to propose a practical search engine, called Multilingual PubMed, which will permit access to a PubMed subset in 1 language and to evaluate the precision and coverage for the French version (Multilingual PubMed-French).MethodsTo create this tool, translations of MeSH were enriched (eg, adding synonyms and translations in French) and integrated into a terminology portal. PubMed subsets in several European languages were also added to our database using a dedicated parser. The response time for the generic semantic search engine was evaluated for simple queries. BabelMeSH, Multilingual PubMed-French, and 3 different PubMed strategies were compared by searching for literature in French. Precision and coverage were measured for 20 randomly selected queries. The results were evaluated as relevant to title and abstract, the evaluator being blind to search strategy.ResultsMore than 650,000 PubMed citations in French were integrated into the Multilingual PubMed-French information system. The response times were all below the threshold defined for usability (2 seconds). Two search strategies (Multilingual PubMed-French and 1 PubMed strategy) showed high precision (0.93 and 0.97, respectively), but coverage was 4 times higher for Multilingual PubMed-French.ConclusionsIt is now possible to freely access biomedical literature using a practical search tool in French. This tool will be of particular interest for health professionals and other end users who do not read or query sufficiently in English. The information system is theoretically well suited to expand the approach to other European languages, such as German, Spanish, Norwegian, and Portuguese.     

Answers to questions posed during daily patient care are more likely to be answered by UpToDate than PubMed

Background

UpToDate and PubMed are popular sources for medical information. Data regarding the efficiency of PubMed and UpToDate in daily medical care are lacking.

Objective

The purpose of this observational study was to describe the percentage of answers retrieved by these information sources, comparing search results with regard to different medical topics and the time spent searching for an answer.

Methods

A total of 40 residents and 30 internists in internal medicine working in an academic medical center searched PubMed and UpToDate using an observation portal during daily medical care. The information source used for searching and the time needed to find an answer to the question were recorded by the portal. Information was provided by searchers regarding the topic of the question, the situation that triggered the question, and whether an answer was found.

Results

We analyzed 1305 patient-related questions sent to PubMed and/or UpToDate between October 1, 2005 and March 31, 2007 using our portal. A complete answer was found in 594/1125 (53%) questions sent to PubMed or UpToDate. A partial or full answer was obtained in 729/883 (83%) UpToDate searches and 152/242 (63%) PubMed searches (P < .001). UpToDate answered more questions than PubMed on all major medical topics, but a significant difference was detected only when the question was related to etiology (P < .001) or therapy (P = .002). Time to answer was 241 seconds (SD 24) for UpToDate and 291 seconds (SD 7) for PubMed.

Conclusions

Specialists and residents in internal medicine generally use less than 5 minutes to answer patient-related questions in daily care. More questions are answered using UpToDate than PubMed on all major medical topics.     

Net Improvement of Correct Answers to Therapy Questions After PubMed Searches: Pre/Post Comparison

Background

Clinicians search PubMed for answers to clinical questions although it is time consuming and not always successful.

Objective

To determine if PubMed used with its Clinical Queries feature to filter results based on study quality would improve search success (more correct answers to clinical questions related to therapy).

Methods

We invited 528 primary care physicians to participate, 143 (27.1%) consented, and 111 (21.0% of the total and 77.6% of those who consented) completed the study. Participants answered 14 yes/no therapy questions and were given 4 of these (2 originally answered correctly and 2 originally answered incorrectly) to search using either the PubMed main screen or PubMed Clinical Queries narrow therapy filter via a purpose-built system with identical search screens. Participants also picked 3 of the first 20 retrieved citations that best addressed each question. They were then asked to re-answer the original 14 questions.

Results

We found no statistically significant differences in the rates of correct or incorrect answers using the PubMed main screen or PubMed Clinical Queries. The rate of correct answers increased from 50.0% to 61.4% (95% CI 55.0%-67.8%) for the PubMed main screen searches and from 50.0% to 59.1% (95% CI 52.6%-65.6%) for Clinical Queries searches. These net absolute increases of 11.4% and 9.1%, respectively, included previously correct answers changing to incorrect at a rate of 9.5% (95% CI 5.6%-13.4%) for PubMed main screen searches and 9.1% (95% CI 5.3%-12.9%) for Clinical Queries searches, combined with increases in the rate of being correct of 20.5% (95% CI 15.2%-25.8%) for PubMed main screen searches and 17.7% (95% CI 12.7%-22.7%) for Clinical Queries searches.

Conclusions

PubMed can assist clinicians answering clinical questions with an approximately 10% absolute rate of improvement in correct answers. This small increase includes more correct answers partially offset by a decrease in previously correct answers.     

Effective Metadata Discovery for Dynamic Filtering of Queries to a Radiology Image Search Engine

We sought to demonstrate the effectiveness of techniques to index radiology images using metadata discovered in their free-text figure captions. The ARRS GoldMiner™ image library incorporated 94,256 figures from 11,712 articles published in peer-reviewed online radiology journals. Algorithms were developed to discover metadata—age, sex, and imaging modality—from the figures’ free-text captions. Age was recorded in years, and was classified as infant (less than 2 years), child (2 to 17 years), or adult (18+ years). Each figure was assigned to one of eight imaging modalities. A random sample of 1,000 images was examined to measure accuracy of the metadata. The patient’s age was identified in 58,994 cases (63%), and the patient’s sex was identified in 58,427 cases (62%). An imaging modality was assigned to 80,402 (85%) of the figures. Based on the 1,000 sampled cases, recall values for age, sex, and imaging modality were 97.2%, 99.7%, and 86.4%, respectively. Precision values for age, sex, and imaging modality were 100%, 100%, and 97.2%, respectively. Automated techniques can accurately discover age, sex, and imaging modality metadata from captions of figures published in radiology journals. The metadata can be used to dynamically filter queries for an image search engine.     

New developments in the molecular epidemiology of adenovirus 8 keratoconjunctivitis.

Four consecutive epidemics of keratoconjunctivitis caused by adenovirus 8 (Ad8) occurred over a 5-year period in Brest, France. A selection of 30 strains isolated during this period was studied by DNA restriction enzyme analysis using nine restriction enzymes. BglI and SacI were the most discriminative enzymes and allowed the recognition of four DNA variants, all different from the prototype strain Trim. Within each of the epidemics, the strains tested could not be distinguished in this analysis. Between strains from different epidemics differences in DNA structure could be detected however. Thus, the Ad8 epidemics of 1983/1984, 1984, 1987, and 1988 appear to have been due to DNA variants Ad8/D7, D8, D9, and D10, respectively. These results demonstrate that the DNA of Ad8 seems to display a considerable variability, comparable to that observed with Ad7 and Ad21. As has been described for Ad7, Ad21 and Ad41, successive DNA variants of Ad8 prevail during one or more years, and are then replaced by other, newly emerging variants sometimes associated with epidemics.     

Differential findings regarding molecular epidemiology of tuberculosis between two consecutive periods in the context of steady increase of immigration

The demographic characteristics of the population of Madrid, with a steady increase in immigrants, from 4.7% in 1998 to 17.4% in 2007, provide an opportunity to study in depth the transmission of TB. Our aim was to compare two 3-year longitudinal molecular studies of TB to define transmission patterns and predictors of clustering. Two prospective population-based molecular and epidemiological studies (2002–2004 and 2005–2007) of TB patients were conducted in nine urban districts in Madrid using the same methodology. During the period 2002–2007, 2248 cases of TB were reported, and the incidence decreased from 23.5 per 100 000 in 2002 to 20.8 in 2007 (p <0.001). A total of 1269 isolates were molecularly characterized and included in the study. The comparison between the two periods showed that the percentage of foreign-born patients among TB cases increased from 36.2% to 45.7% (p <0.001). Furthermore, the percentage of clustered cases decreased (36.6% vs. 30.6%; p 0.028), and this decline was associated with a decrease of clustered cases among men and people under 35 years. We also observed a decrease in cases belonging to clusters containing >6 people (14.2% vs. 8.2%; p <0.001), and in cases belonging to mixed clusters containing Spanish-born and foreign-born patients (18.5% vs. 11.1%, p <0.001). Our molecular epidemiology study provides clues to interpret the decrease in the incidence of TB in a context of steady increase of immigration. In our region, the decrease in the incidence of TB can be explained predominantly as a result of a decline in recent transmission.     

A study of the influence of task familiarity on user behaviors and performance with a MeSH term suggestion interface for PubMed bibliographic search

PurposePrevious research has shown that information seekers in biomedical domain need more support in formulating their queries. A user study was conducted to evaluate the effectiveness of a metadata based query suggestion interface for PubMed bibliographic search. The study also investigated the impact of search task familiarity on search behaviors and the effectiveness of the interface.MethodsA real user, user search request and real system approach was used for the study. Unlike tradition IR evaluation, where assigned tasks were used, the participants were asked to search requests of their own. Forty-four researchers in Health Sciences participated in the evaluation – each conducted two research requests of their own, alternately with the proposed interface and the PubMed baseline. Several performance criteria were measured to assess the potential benefits of the experimental interface, including users’ assessment of their original and eventual queries, the perceived usefulness of the interfaces, satisfaction with the search results, and the average relevance score of the saved records.ResultsThe results show that, when searching for an unfamiliar topic, users were more likely to change their queries, indicating the effect of familiarity on search behaviors. The results also show that the interface scored higher on several of the performance criteria, such as the “goodness” of the queries, perceived usefulness, and user satisfaction. Furthermore, in line with our hypothesis, the proposed interface was relatively more effective when less familiar search requests were attempted.ConclusionsResults indicate that there is a selective compatibility between search familiarity and search interface. One implication of the research for system evaluation is the importance of taking into consideration task familiarity when assessing the effectiveness of interactive IR systems.     

eTACTS: A method for dynamically filtering clinical trial search results

ObjectiveInformation overload is a significant problem facing online clinical trial searchers. We present eTACTS, a novel interactive retrieval framework using common eligibility tags to dynamically filter clinical trial search results.Materials and methodseTACTS mines frequent eligibility tags from free-text clinical trial eligibility criteria and uses these tags for trial indexing. After an initial search, eTACTS presents to the user a tag cloud representing the current results. When the user selects a tag, eTACTS retains only those trials containing that tag in their eligibility criteria and generates a new cloud based on tag frequency and co-occurrences in the remaining trials. The user can then select a new tag or unselect a previous tag. The process iterates until a manageable number of trials is returned. We evaluated eTACTS in terms of filtering efficiency, diversity of the search results, and user eligibility to the filtered trials using both qualitative and quantitative methods.ResultseTACTS (1) rapidly reduced search results from over a thousand trials to ten; (2) highlighted trials that are generally not top-ranked by conventional search engines; and (3) retrieved a greater number of suitable trials than existing search engines.DiscussioneTACTS enables intuitive clinical trial searches by indexing eligibility criteria with effective tags. User evaluation was limited to one case study and a small group of evaluators due to the long duration of the experiment. Although a larger-scale evaluation could be conducted, this feasibility study demonstrated significant advantages of eTACTS over existing clinical trial search engines.ConclusionA dynamic eligibility tag cloud can potentially enhance state-of-the-art clinical trial search engines by allowing intuitive and efficient filtering of the search result space.     

POSTGRESQL-IE: An Image-handling Extension for PostgreSQL

The last decade witnessed a growing interest in research on content-based image retrieval (CBIR) and related areas. Several systems for managing and retrieving images have been proposed, each one tailored to a specific application. Functionalities commonly available in CBIR systems include: storage and management of complex data, development of feature extractors to support similarity queries, development of index structures to speed up image retrieval, and design and implementation of an intuitive graphical user interface tailored to each application. To facilitate the development of new CBIR systems, we propose an image-handling extension to the relational database management system (RDBMS) PostgreSQL. This extension, called PostgreSQL-IE, is independent of the application and provides the advantage of being open source and portable. The proposed system extends the functionalities of the structured query language SQL with new functions that are able to create new feature extraction procedures, new feature vectors as combinations of previously defined features, and new access methods, as well as to compose similarity queries. PostgreSQL-IE makes available a new image data type, which permits the association of various images with a given unique image attribute. This resource makes it possible to combine visual features of different images in the same feature vector. To validate the concepts and resources available in the proposed extended RDBMS, we propose a CBIR system applied to the analysis of mammograms using PostgreSQL-IE.     

Rabies virus in insectivorous bats: Implications of the diversity of the nucleoprotein and glycoprotein genes for molecular epidemiology

Insectivorous bats are the main reservoirs of rabies virus (RABV) in various regions of the world. The aims of this study were to (a) establish genealogies for RABV strains from different species of Brazilian insectivorous bats based on the nucleoprotein (N) and glycoprotein (G) genes, (b) investigate specific RABV lineages associated with certain genera of bats and (c) identify molecular markers that can distinguish between these lineages. The genealogic analysis of N and G from 57 RABV strains revealed seven genus-specific clusters related to the insectivorous bats Myotis, Eptesicus, Nyctinomops, Molossus, Tadarida, Histiotus and Lasiurus. Molecular markers in the amino acid sequences were identified which were specific to the seven clusters. These results, which constitute a novel finding for this pathogen, show that there are at least seven independent epidemiological rabies cycles maintained by seven genera of insectivorous bats in Brazil.     

Infodemiology and Infoveillance: Framework for an Emerging Set of Public Health Informatics Methods to Analyze Search,Communication and Publication Behavior on the Internet

Infodemiology can be defined as the science of distribution and determinants of information in an electronic medium, specifically the Internet, or in a population, with the ultimate aim to inform public health and public policy. Infodemiology data can be collected and analyzed in near real time. Examples for infodemiology applications include: the analysis of queries from Internet search engines to predict disease outbreaks (eg. influenza); monitoring peoples'' status updates on microblogs such as Twitter for syndromic surveillance; detecting and quantifying disparities in health information availability; identifying and monitoring of public health relevant publications on the Internet (eg. anti-vaccination sites, but also news articles or expert-curated outbreak reports); automated tools to measure information diffusion and knowledge translation, and tracking the effectiveness of health marketing campaigns. Moreover, analyzing how people search and navigate the Internet for health-related information, as well as how they communicate and share this information, can provide valuable insights into health-related behavior of populations. Seven years after the infodemiology concept was first introduced, this paper revisits the emerging fields of infodemiology and infoveillance and proposes an expanded framework, introducing some basic metrics such as information prevalence, concept occurrence ratios, and information incidence. The framework distinguishes supply-based applications (analyzing what is being published on the Internet, eg. on Web sites, newsgroups, blogs, microblogs and social media) from demand-based methods (search and navigation behavior), and further distinguishes passive from active infoveillance methods. Infodemiology metrics follow population health relevant events or predict them. Thus, these metrics and methods are potentially useful for public health practice and research, and should be further developed and standardized.     

Phylogenetic and molecular analysis of HTLV-1 isolates from a medium sized town in northern of Brazil: tracing a common origin of the virus from the most endemic city in the country

Salvador‐Bahia has the highest prevalence of HTLV‐1 infection in Brazil; about 2% of the population is infected. In this city, the prevalence of HTLV in pregnant women is 1%. There is no data of the HTLV‐1 prevalence in others cities of the Bahia's Recôncavo, where the population has similar social and demography characteristics to those from Salvador. Our aim was to evaluate the seroprevalence of HTLV in pregnant women in Cruz das Almas‐Bahia, a medium‐sized city from the Bahia's Recôncavo. All individuals were tested for HTLV (ELISA) and the positive samples were confirmed by Western Blot. Phylogenetic analyses of the total LTR region were performed in all positive samples. We tested 408 samples (45.4% of the estimate pregnant women population) between June 1st and October 31, 2005. The prevalence of HTLV‐1 infection was 0.98%. In addition, all isolated virus were grouped in the subtype HTLV‐1a, in the Latin American group. Our results suggest that the introduction of HTLV‐1 occurred after the slave trade into Salvador. In addition, HTLV‐1‐infection should be screened during the pregnancy in women originating from HTLV‐1 endemic areas. J. Med. Virol. 80:2040–2045, 2008. © 2008 Wiley‐Liss, Inc.     

Applicability of consumer activity monitor data in marathon events: an exploratory study

Emerging opportunities to measure individual and population-level health data with activity monitors during recreational running activities may set the stage for new research possibilities in mass participation running events and marathon medicine. This study explores the applicability of consumer activity monitor data in a preliminary study for future marathon health research with a cohort of 12 (n?=?12) participants completing a 3.379?km walking or running course. This study explored the feasibility of collecting pace and distance data from Fitbit brand consumer activity monitors, from access to user data to reporting of data characteristics and data analysis. We show that a large percentage of participant data can be successfully retrieved from Fitbit consumer activity monitor devices for analysis in marathon health research, and that identifying variations in pace across participants is a practical possibility. We note a mean absolute percentage error of 13% over the true distance of 3.379?km, a higher error than that reported by other studies. We also observe a Pearson correlation coefficient between participant variation in pace and absolute distance error of 0.61. This study provides preliminary evidence to support the applicability of consumer activity monitor data in marathon health research.     

HealthTrust: a social network approach for retrieving online health videos

Background

Social media are becoming mainstream in the health domain. Despite the large volume of accurate and trustworthy health information available on social media platforms, finding good-quality health information can be difficult. Misleading health information can often be popular (eg, antivaccination videos) and therefore highly rated by general search engines. We believe that community wisdom about the quality of health information can be harnessed to help create tools for retrieving good-quality social media content.

Objectives

To explore approaches for extracting metrics about authoritativeness in online health communities and how these metrics positively correlate with the quality of the content.

Methods

We designed a metric, called HealthTrust, that estimates the trustworthiness of social media content (eg, blog posts or videos) in a health community. The HealthTrust metric calculates reputation in an online health community based on link analysis. We used the metric to retrieve YouTube videos and channels about diabetes. In two different experiments, health consumers provided 427 ratings of 17 videos and professionals gave 162 ratings of 23 videos. In addition, two professionals reviewed 30 diabetes channels.

Results

HealthTrust may be used for retrieving online videos on diabetes, since it performed better than YouTube Search in most cases. Overall, of 20 potential channels, HealthTrust’s filtering allowed only 3 bad channels (15%) versus 8 (40%) on the YouTube list. Misleading and graphic videos (eg, featuring amputations) were more commonly found by YouTube Search than by searches based on HealthTrust. However, some videos from trusted sources had low HealthTrust scores, mostly from general health content providers, and therefore not highly connected in the diabetes community. When comparing video ratings from our reviewers, we found that HealthTrust achieved a positive and statistically significant correlation with professionals (Pearson r ₁₀ = .65, P = .02) and a trend toward significance with health consumers (r ₇ = .65, P = .06) with videos on hemoglobinA_{1 c}, but it did not perform as well with diabetic foot videos.

Conclusions

The trust-based metric HealthTrust showed promising results when used to retrieve diabetes content from YouTube. Our research indicates that social network analysis may be used to identify trustworthy social media in health communities.     

Molecular epidemiology of measles virus in Taiwan in 2010–2011: The common genotype changed from H1 to D9 and the first appearance of D4

Measles has been controlled effectively in some countries because of high coverage rates with an effective vaccine. However, measles outbreaks still occasionally occur in areas with high vaccine coverage as a result of imported transmission. To identify the sources of measles infection and to determine whether measles cases are part of a single outbreak or due to multiple importations, measles virus (MV) genotyping is required and plays an important role in MV elimination. In Taiwan, genotype H1 of MV was detected most frequently before 2009. From 2006 to 2011, 47 of 48 genotype H1 cases were associated with the imported cases, indicating that genotype H1 was not an endemic genotype in Taiwan after 2006. The distribution of the other genotypes (D3, D4, D5, D8, D9, and G3) detected during 2006–2011 varied by year. Taiwan has a pattern of measles genotypes that is consistent with the elimination of MV and with the absence of endemic genotypes. In this study, the genotypes of 40 cases of MV detected during 2010–2011 were investigated and analyzed. In 2010, the most common genotype changed from H1 (3/40) to D9 (35/40). In 2011, genotype H1 was not detected, and genotype D4 first appeared and was imported from Europe. The dynamic change of detected genotypes of MV in Taiwan is influenced by the activity of a measles control program in WHO regions. This study emphasizes that global synchronous elimination is important for an individual country or area to maintain free from MV. J. Med. Virol. 85: 1095–1099, 2013. © 2013 Wiley Periodicals, Inc.