Readability of Consumer Health Information on the Internet: A Comparison of U.S. Government–Funded and Commercially Funded Websites

The Internet has become an extremely prevalent means of communicating health information to consumers. Guidelines for selecting reliable health information websites give preference to U.S. government sites over commercially funded sites. However, these websites are not useful to consumers unless they are able to read and understand them. The authors’ objective was to compare the readability of Internet health information intended for consumers found on U.S. government–funded websites versus that found on commercially funded websites. Consumer health websites were identified through a systematic Internet search. Webpages for 10 common health topics were extracted from each website. Readability of webpages was determined by 3 validated measures: Flesch Reading Ease, Flesch-Kincaid Reading Level, and SMOG Formula. Mean readability of government-funded and commercially funded websites was compared using the Mann-Whitney U test. Commercially funded websites were significantly more difficult to read as measured by Flesch Reading Ease (49.7 vs. 55.6 for government-funded sites, p = .002) and Flesch-Kincaid Reading Level (10.1 vs. 9.3, p = .012). There was no significant difference according to SMOG Formula (12.8 vs. 13.2, p = .150). The overall readability of Internet health information intended for consumers was poor. Efforts should be made to ensure that health information communicated via the Internet is easy for consumers to read and understand.     

Health literacy and the Internet: a study on the readability of Australian online health information

Objective: Almost 80% of Australian Internet users seek out health information online so the readability of this information is important. This study aimed to evaluate the readability of Australian online health information and determine if it matches the average reading level of Australians. Methods: Two hundred and fifty‐one web pages with information on 12 common health conditions were identified across sectors. Readability was assessed by the Flesch‐Kincaid (F‐K), Simple Measure of Gobbledygook (SMOG) and Flesch Reading Ease (FRE) formulas, with grade 8 adopted as the average Australian reading level. Results: The average reading grade measured by F‐K and SMOG was 10.54 and 12.12 respectively. The mean FRE was 47.54, a ‘difficult‐to‐read’ score. Only 0.4% of web pages were written at or below grade 8 according to SMOG. Information on dementia was the most difficult to read overall, while obesity was the most difficult among government websites. Conclusions and implications: The findings suggest that the readability of Australian health websites is above the average Australian levels of reading. A quantifiable guideline is needed to ensure online health information accommodates the reading needs of the general public to effectively use the Internet as an enabler of health literacy.     

Readability of online health information: implications for health literacy

Accessibility is one of six quality criteria articulated by the European Commission in its code of conduct for health websites. Readability plays an integral part in determining a website's accessibility. Health information that is hard to read may remain inaccessible to people with low health literacy. This study aimed to calculate the readability of websites on various causes of disease. The names of 22 health conditions were entered into five search engines, and the readability of the first 10 results for each search were evaluated using Gunning FOG, SMOG, Flesch-Kincaid and Flesch Reading Ease tests (n=352). Readability was stratified and assessed by search term, search term complexity, top-level domain and paragraph position. The mean reading grade was 12.30, and the mean FRE was 46.08, scores considered 'difficult'. Websites on certain topics were found to be even harder to read than average. Where conditions had multiple names, searching for the simplest one led to the most readable results. Websites with .gov and .nhs TLDs were the most readable while .edu sites were the least. Within texts, a trend of increasing difficulty was found with concluding paragraphs being the hardest to read. It was also found that some of the most frequent search results (such as Wikipedia pages) were amongst the hardest to read. Health professionals, with the help of public and specialised libraries, need to create and direct patients towards high-quality, plain language health information in multiple languages.     

A cross-section of readability of Health Information Portability and Accountability Act authorizations required with health care research

The Health Information Portability and Accountability Act (HIPAA) regulations require that agencies provide authorizations that can be read and understood. Agencies are responsible to comply with this regulation. It is estimated that half of Americans read at an eighth-grade reading level. All institutional review boards (IRBs) listed on the website of the U.S. Department of Health and Human Services were contacted by investigators in this study. Fifty-one IRBs returned HIPAA templates on which readability tests could be completed. Three readability tests were used on all templates, including the Flesch Reading Ease formula, the Dale-Chall formula, and the Fry formula. For the returned templates, the mean of the Dale-Chall readability formula gave a ninth-grade level; the mean scores for the Flesch Reading Ease and Fry formulas were at a high-school reading level or above. The readability scores ranged from sixth-grade to college graduate. This study demonstrates that HIPAA authorization forms are written at too high a level for most of the population. Based on the results of this study and other similar studies, it can be concluded that consumers/patients are currently being asked to read important health-related documents that may be beyond their reading level.     

An Exploratory Study of Older Adults' Comprehension of Printed Cancer Information: Is Readability a Key Factor?

Printed cancer information often is written at or beyond high school reading levels, despite lower average literacy abilities of the public. The objectives of this exploratory study were twofold: (1) to evaluate older adults' comprehension of breast (BC), prostate (PC), and colorectal (CC) cancer information; and (2) to determine if comprehension of BC, PC, and CC information varies according to text readability. Comprehension of printed cancer resources was evaluated with 44 community-dwelling older adults using the Cloze procedure and recall questions. Participants' comprehension scores were compared with Simple Measure of Gobbledegook (SMOG) readability scores (<grade 13 vs. grade 13+). Overall, older adults had satisfactory comprehension of cancer information as measured by Cloze (.86 ± .01) and recall (.71 ± .02). For CC information written at grade 13, however, a significant negative correlation between readability and Cloze comprehension was found (r _s  = ?.44, SE = .17, p = .019), indicating poorer participant comprehension at higher readability levels. Comprehension of BC or PC information did not vary by readability level. Though readability plays a role in older adults' understanding of cancer information, cancer type and content are also important factors that influence comprehension. Use of plain language is recommended for CC resources.     

The SOL Formulas for Converting SMOG Readability Scores Between Health Education Materials Written in Spanish,English, and French

The SMOG formula is widely used to determine how easy written health education materials are to read and comprehend. This study was conducted to assess whether the SMOG formula, originally developed and tested in English, was also valid for texts written in Spanish and French. Readability scores from a sample of passages written in Spanish, English, and French were analyzed. Study results showed that the SMOG formula seemed to be consistent in measuring readability in the three languages. However, SMOG scores varied from language to language. Two hypotheses are being laid out to explain these results. First, the SMOG equation is system atically biased for texts in Spanish or French. Second, English is more readable than French, and French is more readable than Spanish. Under the assumption of a systematic bias in the SMOG formula, the so-called SOL formulas are presented here to convert SMOG scores between Spanish, English, and French. In addition, a new scale for grading reading difficulty is proposed based on SMOG scores obtained from classical literature. This is important to the extent that it will provide Spanish speaking and French speaking health communicators with a readability formula for preparing written materials appropriate to the level of comprehension of specific target audiences.     

The SOL formulas for converting SMOG readability scores between health education materials written in Spanish, English, and French

The SMOG formula is widely used to determine how easy written health education materials are to read and comprehend. This study was conducted to assess whether the SMOG formula, originally developed and tested in English, was also valid for texts written in Spanish and French. Readability scores from a sample of passages written in Spanish, English, and French were analyzed. Study results showed that the SMOG formula seemed to be consistent in measuring readability in the three languages. However, SMOG scores varied from language to language. Two hypotheses are being laid out to explain these results. First, the SMOG equation is systematically biased for texts in Spanish or French. Second, English is more readable than French, and French is more readable than Spanish. Under the assumption of a systematic bias in the SMOG formula, the so-called SOL formulas are presented here to convert SMOG scores between Spanish, English, and French. In addition, a new scale for grading reading difficulty is proposed based on SMOG scores obtained from classical literature. This is important to the extent that it will provide Spanish speaking and French speaking health communicators with a readability formula for preparing written materials appropriate to the level of comprehension of specific target audiences.     

Readability of informed consent documents used in the hospitals of Asturias

OBJECTIVE: To evaluate the readability of the informed consent documents (ICD) used in the health centers of Asturias (Spain) with the aim of verifying whether the ICD are understandable and sufficient as a vehicle of information to empower patients to participate in decisions related to their diagnosis and treatment. METHOD: A total of 1114 ICD were gathered from the health centers in Asturias. A representative random sample from each hospital was selected and the following readability indexes were obtained: the Flesch index, the sentence complexity index, and the integrated legibility index. RESULTS: Of the ICD in the sample studied, legibility was acceptable in 77.3% (221 out of 286) according to the integrated legibility index, in 75.2% (215 out of 286) according to the sentence complexity index and in 3.5% (10 out of 286) according to the Flesch index. The mean values of the indexes differed among hospitals (p < 0.001, ANOVA test). CONCLUSIONS: ICD written in Spanish achieved low readability scores in the Flesch index and require corrective measures. Three quarters of the ICD were acceptable due to the use of simple sentences in the text. We believe that readability indexes should be used when drafting or improving informed consent documents.     

Evaluating the health literacy demand and cultural appropriateness of online immunisation information available to refugee and migrant communities in Australia

BackgroundThe internet is a source of health information for many consumers. However, little is known about the availability of online resources about immunisation (for children and adults) directed at refugee and migrant populations. The aim of this study was to evaluate the health literacy demand (understandability, actionability & readability) and cultural appropriateness of immunisation resources in Australia for these communities.MethodsOur study involved two search approaches. Firstly, Google trends was used to identify the most common search terms used in Australia. Search terms used included ‘immunisation’, ‘vaccine’ and ‘refugee immunisation’ amongst others. These search terms were entered into: Google, Bing, Duck Duck Go and Yahoo and the first 5 pages of results for each search were examined. Searches were conducted from November 2018 to June 2019. Secondly, requests were sent out directly to key stakeholders in local health districts and state/territory health departments. Understandability and actionability were assessed using the Patient Education Materials Assessment Tool (PEMAT). The Simple Measure of Gobbledygook (SMOG) and the Flesch readability ease tools were used to assess readability. Cultural appropriateness was assessed using additional criteria developed by the Centre for Culture, Ethnicity and Health (CEH), Victoria, Australia.Results27/33 resources were included in the analysis. The overall mean understandability score was 70% (range: 50–100%). The overall mean actionability score of the resources was 47% (range: 0–83%). In terms of readability, the mean score was a 10.5, which was indicative of an eleventh grade reading level. The average Flesch-Kincaid readability score was 47.7. The average score for cultural appropriateness was 79% (range: 29–100%).ConclusionsOur findings suggest that there are currently not many refugee-specific resources on immunisation. Future work needs to better improve the health literacy demand of online immunisation information. Engaging with members from migrant communities is also recommended so that appropriately tailored resources are co-developed.     

Quality of the patient-oriented information on thyroid cancer in the internet

ObjectivesThis study was conducted to analyse the readability and quality of patient education websites about thyroid cancer.MethodsYahoo!, Google and Bing were searched with their default settings using the keywords thyroid cancer, thyroid tumour, thyroid tumor, and thyroid malignancy for relevant websites. The first 50 websites obtained for each keyword and search engine were evaluated using validated Flesch Reading Ease Score (FRES), LIDA and DISCERN scores to assess readability, usability and reliability, and quality of information. Statistical analysis were done using non-parametric tests.ResultsThe analysis included 49 eligible websites selected out of 600 websites obtained from the search engines. Twenty-two (44.9%) web sites had obtained Health on the net foundation code of conduct (HON-code) certification. The included websites had a median FRES score of 55.3(range 25.3–85.3) and the LIDA reliability and usability scores were 22.0(range 14–30) and 39.0(range 28–50), respectively. The median DISCERN score was 48.0(range 33–63). A total DISCERN score of 50% or less (low score) was found in 28.5%(n = 14) of the websites. A significant association was found between LIDA usability and, reliability scores and DISCERN score (p<0.001, p=0.001, respectively).ConclusionThe quality of the websites, readability and usability scores were moderate or low based on the criteria used. Access to quality information may help with better informed decisions on treatment, follow up and prognosis. As the internet use by the public is ever increasing, implementing measures to ensure the quality information without a bias should be considered a priority.     

Age-related macular degeneration: what do patients find on the internet?

OBJECTIVE: To assess the quality of information and readability of the top internet sites for age-related macular degeneration (AMD). DESIGN: An examination of the technical information provision, quality and readability of websites found during an internet search for 'age-related macular degeneration'. Setting Six internet search engines were used to find 26 unique sites on AMD. MAIN OUTCOME MEASURES: Technical information and quality were assessed using a simple grading system. Readability was assessed using a Simple Measure Of Gobbledygook (SMOG) rating. RESULTS: Twelve organizational, seven academic and seven commercial sites were identified. The average technical scores were 82.3%, 67.9% and 65.2% for each type of site, respectively (P=0.097, one way ANOVA). The average quality scores were 62.2%, 62.6%, and 49.5% for each type of site, respectively (P=0.356, one-way ANOVA). The average SMOG ratings were 16.3, 16.1, and 16.2 for each type of site, respectively (P=0.983, one-way ANOVA). Fifteen of the sites provided details of new and emerging treatments, with seven providing a detailed discussion. CONCLUSIONS: Many websites are now meeting the challenge of providing comprehensive information about AMD and its new treatments. Quality scores were disappointing, with sites needing to provide more evidence of authorship and attribution of information. The majority of sites had SMOG scores above 10, making them difficult for the average person to understand. As physicians we need to help design and direct patients to sites that provide high quality, current information.     

Assessment of Selected Patient Educational Materials of Various Chain Pharmacies

Pharmacy literature commonly used to inform patients about medication side-effects and complications was examined for readability. Forty-five (45) informational sheets were obtained from nine national and regional pharmacies. According to the McLaughlin's SMOG (Simple Measure of Gobbledegook) formula, the reading level of the informational sheets ranged from grade 8–14 (8 = 8th grade level and 14 = collegiate level), with a mean reading level of 12. The sampled pharmacy educational materials may be too difficult for most Americans to read and comprehend. The pharmacist may assist in increasing patient compliance by offering education in a form that is understandable, which increases the likelihood of adherence to desired health behaviors.     

Readability of commercial versus generic health instructions for condoms

Despite the fact that condoms are increasingly recommended for the prevention of acquired immune deficiency syndrome (AIDS) and other sexually transmitted diseases (STDs) and have long been used as a barrier method of contraception, little is known about the readability of the range of materials that have been written exclusively for consumers on how to use a condom. The authors described and compared the reading comprehension levels of condom patient package inserts (PPIs) prepared commercially and those generic instructions (GIs) prepared by health care providers. Fifteen sets of PPIs and 30 sets of GIs were analyzed using six standard readability formulas (Fry, Fog, Dale‐Chall, Flesch, Flesch‐Kincaid, SMOG). Readability levels of the instructions ranged from grade 6.3 to 13.7. The PPIs required a significantly higher grade level for comprehension (mean 10.32) than the GIs (mean 8.69). This study suggests that instructions prepared by both sources may require a reading level above that of many consumers who need information on how to use a condom.     

The numeracy demands of health education information: an examination of numerical concepts in written diabetes materials

The comprehension of numerical concepts that appear in written health educational materials is an important aspect of health literacy. Health materials that include many advanced numerical concepts may place high demands on readers, especially those with low health literacy levels. The purpose of this study was to examine and document the types of numerical concepts that appear in passages selected from written diabetes educational materials that varied in their type of content. Readability of the passages was also analyzed. One hundred and fifty passages of 100 words each were selected from 50 diabetes-related educational documents, produced by major health organizations. Passages were coded for type of numerical concept, SMOG Reading Grade Level, Flesch Reading Ease, and features that elaborated the numerical concepts, such as pictures or explanations. Findings indicate that passages containing general information about diabetes and its prevention contained significantly more advanced numerical concepts, the highest reading grade level, the lowest reading ease scores, and the lowest frequency of elaborative features, relative to many other types of content. Recommendations are made for enhancing the presentation of numerical concepts in written diabetes educational materials.     

The Numeracy Demands of Health Education Information: An Examination of Numerical Concepts in Written Diabetes Materials

The comprehension of numerical concepts that appear in written health educational materials is an important aspect of health literacy. Health materials that include many advanced numerical concepts may place high demands on readers, especially those with low health literacy levels. The purpose of this study was to examine and document the types of numerical concepts that appear in passages selected from written diabetes educational materials that varied in their type of content. Readability of the passages was also analyzed. One hundred and fifty passages of 100 words each were selected from 50 diabetes-related educational documents, produced by major health organizations. Passages were coded for type of numerical concept, SMOG Reading Grade Level, Flesch Reading Ease, and features that elaborated the numerical concepts, such as pictures or explanations. Findings indicate that passages containing general information about diabetes and its prevention contained significantly more advanced numerical concepts, the highest reading grade level, the lowest reading ease scores, and the lowest frequency of elaborative features, relative to many other types of content. Recommendations are made for enhancing the presentation of numerical concepts in written diabetes educational materials.     

Online Educational Tool to Promote Bone Health in Cancer Survivors

Osteoporosis burden is significant in cancer survivors. Websites providing health information abound, but their development, quality, and source of information remain unclear. Our aim was to use a systematic and transparent approach to create an educational website on bone health, and to evaluate its potential to improve knowledge, self-management, and awareness in prostate cancer (PCa) and breast cancer (BCa) survivors. Guided by the Health Belief Model, we created a website using international standards and evaluated it in 10 PCa and 10 BCa survivors with self-administered questionnaire before, after, and 1 month after navigating the website. The mean scores on the knowledge questionnaire at baseline, postintervention and 1 month were, respectively, 5.1 (±2.0), 6.9 (±2.5), and 6.7 (±2.4), p < .008, in PCa and 3.4 (±2.7), 7.6 (±3.0), and 6.5 (±3.8), p = .016, in BCa survivors. Acceptability ratings ranged from 60% to 100%. Participants found the website useful, helpful, and able to raise bone health awareness. Our website improved bone health knowledge in both PCa and BCa survivors. A systematic and transparent approach to the development of online educational websites could result in a tool capable of meeting the educational needs of targeted consumers. Cancer survivors could benefit from proven online educational tools.     

Readability of Educational Materials to Support Parent Sexual Communication With Their Children and Adolescents

Sexual communication is a principal means of transmitting sexual values, expectations, and knowledge from parents to their children and adolescents. Many parents seek information and guidance to support talking with their children about sex and sexuality. Parent education materials can deliver this guidance but must use appropriate readability levels to facilitate comprehension and motivation. This study appraised the readability of educational materials to support parent sexual communication with their children. Fifty brochures, pamphlets, and booklets were analyzed using the Flesch-Kincaid, Gunning Fog, and Simple Measure of Gobbledygook (SMOG) index methods. Mean readability grade-level scores were 8.3 (range = 4.5–12.8), 9.7 (range = 5.5–14.9), and 10.1 (range = 6.7–13.9), respectively. Informed by National Institutes of Health–recommended 6th to 7th grade levels and American Medical Association–recommended 5th to 6th grade levels, percentages falling at or below the 7.0 grade level were calculated as 38%, 12%, and 2% and those falling at or below the 6.0 grade level were calculated as 12%, 2%, and 0% based on the Flesch-Kincaid, Gunning Fog, and SMOG methods, respectively. These analyses indicate that the majority of educational materials available online to support parents’ communication with their children about sex and sexuality do not meet the needs of many or most parents. Efforts to improve the accessibility of these materials are warranted.     

Peer Reviewed: A Comprehensive Assessment of the Difficulty Level and Cultural Sensitivity of Online Cancer Prevention Resources for Older Minority Men

Introduction

Older men are at increased risk for prostate cancer. As seniors turn to the Internet for cancer information, it is important that the resources they locate about lifestyle behaviors and screening are culturally appropriate and easy to understand. This study was a comprehensive analysis of prostate cancer risk as portrayed on the Internet with assessment of content readability and cultural sensitivity.

Methods

We selected Web sites about prostate cancer risk and prevention by comparing common sites across three top-rated search engines (Google, Yahoo!, and MSN). A total of 70 Web sites on prostate cancer containing a Web page on risk factors or prevention or both for racial and ethnic populations were included. We assessed readability of one page per Web site using Simple Measure of Gobbledygook (SMOG), Flesch-Kincaid (FK), and Flesch Reading Ease (FRE) measures. Cultural sensitivity of the Web page was evaluated using the Cultural Sensitivity Assessment Tool (CSAT) and questions from a cultural sensitivity checklist.

Results

Mean readability of Web pages was Grade 12.90 (high school graduate level) using SMOG and Grade 11.20 according to FK. Mean FRE was 45.04 (fairly difficult to read). The mean CSAT score was 2.78 and classified as culturally sensitive. Of the 36 Web pages considered culturally sensitive (CSAT >2.50), 75% did not portray images of representative racial or ethnic individuals as intended readers or as being at high risk for prostate cancer. Older adults and seniors were identified as intended readers on 73% of Web pages.

Conclusion

Online cancer resources are targeting appropriate age groups (high-risk older adults). However, the pages required fairly high-level reading skills and had limited cultural sensitivity. These factors make the pages unsuitable for diverse Internet users.     

Reading Difficulty Level of Medical Resources on Television Web Sites

ABSTRACT

Mass media plays a critical role in communicating health messages to the public. Local television stations are posting more health content on the Web; however, the difficulty level of this information has not been evaluated. Study objectives were to assess the content of health stories on local television Web sites, and to evaluate the reading level of these health messages. The top three health articles from 79 local television news Web pages were selected for analysis. Readability was assessed using SMOG, Flesch-Kincaid (FK), and Flesch Reading Ease (FRE). A total of 278 articles were located over a two-day search. The majority of stories were national (47.0%) and educational (48.0%) pieces. Fewer articles were breaking news (19.8%). More articles focused on both prevention and treatment (30.6%) rather than on prevention (21.6%) or treatment (21.2%) alone. Mean readability was at a senior high school level according to SMOG (Grade 11.9) and FK (Grade 10.3). Overall FRE was 51.6, implying a “fairly difficult” reading style. As broadcast news media mobilizes viewers toward online health content, they must consider the difficulty of the information and literacy abilities of their audience.     

Optimizing the Presentation of Mental Health Information in Social Media: The Effects of Health Testimonials and Platform on Source Perceptions,Message Processing,and Health Outcomes

Using social media for the purpose of disseminating mental health information is a critical area of scientific inquiry for health communication professionals. The purpose of this study was to investigate whether the presence of a first-person testimonial in educational mental health information placed in Facebook and Twitter messages influenced college students’ (N = 257) source perceptions, information processing, cognitive elaboration, health information recall, beliefs, and behavioral intentions. Results show that exposure to social media messages that featured mental health information embedded with a testimonial predicted less source homophily and more critical thoughts about the social media source, less systematic message processing, and less cognitive elaboration. Health information recall was significantly impacted by both the social media platform and message content such that participants in the testimonial condition on Facebook were more likely to recall the health facts in those messages whereas participants who viewed the testimonial in Twitter were less likely to recall the facts in those tweets. Compared to those who read Facebook messages, participants who read Twitter messages reported higher levels of systematic message processing. These findings suggest that the integration of health testimonials into social media messages might inadvertently provoke psychological resistance to mental health information, thereby reducing the persuasive impact of those messages.