Comparison of Open and Closed Questionnaire Formats in Obtaining Demographic Information From Canadian General Internists

The objective of this study was to compare the impact of closed- versus open-ended question formats on the completeness and accuracy of demographic data collected in a mailed survey questionnaire. We surveyed general internists in five Canadian provinces to determine their career satisfaction. We randomized respondents to receive versions of the questionnaire in which 16 demographic questions were presented in a closed-ended or open-ended format. Two questions required respondents to make a relatively simple computation (ensuring that three or four categories of response added to 100%). The response rate was 1007/1192 physicians (80.0%). The proportion of respondents with no missing data for all 16 questions was 44.7% for open-ended and 67.0% for closed-ended formats (P < 0.001). The odds of having missing items remained higher for open-ended response options after adjusting for a number of respondent characteristics (2.67, 95% confidence interval 2.01 to 3.55). For the two questions requiring computations focused on professional activity and income, there were more missing data (P = 0.02, 0.02, respectively) but fewer inaccurate responses (P = 0.009, 0.20, respectively) for the open-ended compared to the closed-ended format. Investigators can achieve higher response rates for demographic items using closed format response options, but at the risk of increasing inaccuracy in response to questions requiring computation.     

Students' perception of good and bad teaching: report of a critical incident study

Evaluation of teaching by the use of questionnaires to students is now commonplace. If it is to be useful the data obtained must be reliable and valid. One criterion of reliability is the response rate, and a low response rate may indicate low validity, i.e. that the questions asked do not reflect the students' real concerns. In order to inform questionnaire design a critical incident study was undertaken. A 20% random sample of students in each of the 5 years of the course were asked to describe one piece of good teaching and one piece of bad teaching, and say why they were good or bad. There was a 65% response rate, and replies were independently categorized by three people. The factors identified fell into three 'domains': interpersonal behaviour of teachers; planning and preparation; and the ability to run the session well. There was no evidence that teaching which 'played to the gallery' or was very examination-oriented would earn high ratings from students.     

The reliability of self-reported drinking in adolescence

AIMS: To study the reliability of adolescents' self-reported drinking and perceived drunkenness in surveys. METHODS: The data from two cross-sectional school-based questionnaire surveys with representative cluster samples (the European School Survey Project on Alcohol and Other Drugs, ESPAD) in Finland were used; there were 2161 (1995) and 3109 (1999) 15-year-old respondents. The response rates were 94% and 90% respectively. The measurements analysed were an open-ended and a set of closed-category questions concerning the latest drinking occasion. RESULTS: The set of three closed questions used in 1995 yielded mean amounts of 6.6 (girls) and 8.7 (boys) centilitres of pure alcohol whereas the figures obtained from the open question were 8.5 (girls) and 11.8 (boys) centilitres. With the closed set extended in 1999 into five questions, the two figures among girls were 7.7 (closed) and 7.7 (open) centilitres; the corresponding figures among boys were 11.3 (closed) and 11.7 (open) centilitres. Individual level correlations between the two measures among girls were 0.69 in 1995 and 0.69 in 1999; and 0.69 (1995) and 0.65 (1999) among boys. The numbers of students reporting specific beverage type use were higher when using closed questions compared with an open question. Drunkenness self-reports related logically to amounts of alcohol drunk. CONCLUSIONS: The adolescent drinking amount self-reports seem reasonably reliable and valid both on a population and individual level. A set of closed questions may capture the amount drunk even better than an open question.     

Factors underlying performance on written tests of knowledge

The study compares two popular forms of written tests; the multiple choice test (MCQ) and the Modified Essay Question (MEQ). Two factors were varied in the experiment: the format of the questions (multiple choice, directed free response, or open-ended free response) and the context of the questions (in a patient problem or in random sequence). Six problems were developed in each version, and administered to a total of 36 medical students at three educational levels using a Latin-square design. The results showed a significant effect of each factor in the design, amounting to a difference of 8.7% between MCQ and directed free response, 4.2% between directed and open-ended free response and 4.3% between problem and random context. However, the correlation of scores based on content across the formats approached unity after correction for attenuation. A process score, based on the style and presentation in the undirected format, correlated more strongly with the free-response questions. The results suggest that, although the MCQ and MEQ may assess different skills, there is a very strong relationship between content scores derived from the two formats. The free response formats may present the opportunity for assessment of other factors related to presentation if scoring procedures are modified. Finally, the effect of randomizing questions is a deterioration of performance when compared to placing questions in the problem context.     

The influence of ethnicity on patient satisfaction

Methods

Results

Conclusion

To investigate the influence of ethnicity on patient satisfaction with hospitalization care.

We conducted a random selection, cross-sectional study. Data were collected by telephone interviews over a three-year period utilizing a 16-question survey. Patients were excluded from the study if they were admitted for an obstetric visit, physical rehabilitation, or psychiatric illness or if we were unable to reach them by telephone. We used logistic regression to compare ethnicity with the responses for each of the 16 questions while controlling for three confounders (age, gender, and insurance status). For each question, patient responses of excellent and very good were considered satisfied. Patient responses of good, fair, and poor were considered not satisfied.

We surveyed 7,795 patients. Compared to African-Americans, non-Hispanic white Americans were significantly older, included more males, and were insured by Medicaid less often (p?

African-Americans reported significantly lower rates of satisfaction compared to non-Hispanic white Americans for six of 16 questions regarding satisfaction during hospitalization care.     

Outpatient satisfaction: validation of a French-language questionnaire: data quality and identification of associated factors

OBJECTIVES: Following 1996 legislation requiring French hospitals to assess patient satisfaction, this study developed and validated a brief French-language multidimensional questionnaire designed to measure outpatient satisfaction with hospital visits and compared data quality for two patient-satisfaction survey methods. DESIGN: Authors developed a 19-item questionnaire following a strict procedure (identification of dimensions to explore, formulation, and selection of items). SETTING: Validation data were obtained from patients of six outpatient clinics in a teaching hospital. PARTICIPANTS: 586 consenting eligible patients were randomized to receive the questionnaire 2 weeks after their visit with one of two survey methods: a mailed self-administered questionnaire or a telephone interview. RESULTS: The response rate (79%) was not significantly different between the two survey methods. The risk of having one or more missing values was higher in the mail survey group (odds ratio, 1.65; 95% confidence interval, 1.03-2.63), but mail respondents were less likely to use the "extremely positive" response category. Principal component analysis identified four factors that accounted for 56% of the variance: interpersonal skills and information transfer, physical surroundings, convenience, and appointment delay. Patients' comments on open-ended questions validated the semantic content of the factorial construct. The internal consistency coefficient was greater than 0.70 for three of four subscales. Patient background characteristics accounted for less than 10% of the factorial score variance. Patient satisfaction was correlated with age, type of visit, and, to a lesser extent, gender and education level. CONCLUSION: This easily administered, multidimensional out-patient-satisfaction questionnaire provided encouraging preliminary psychometric characteristics.     

Computerized long-menu questions as an alternative to open-ended questions in computerized assessment

To optimally avoid cueing effects and computer scoring problems in computerized examinations a computerized long-menu question (CLM) was developed. This question type was compared to open-ended questions in one treatment group and to multiple-choice questions in another treatment group. Also, scores were compared to self-perceived computer anxiety of the participants. CLMs yield comparable scores to open-ended questions, but the scores differ significantly from those on multiple-choice tests. Correlations in the first comparison (CLMs with multiple-choice) were higher than those in the second camparison (CLMs with open-ended questions). The amount of positive and negative cueing was considerably higher in the first than in the second comparison. Response times of CLMs were higher than those of multiple-choice questions and open-ended questions, differing significantly from both. Computer anxiety did not influence the mean scores in either comparison. Therefore, in computerized testing CLMs seem to offer an acceptable replacement of open-ended questions.     

Ethnicity questions and antenatal screening for sickle cell/thalassaemia [EQUANS] in England: a randomised controlled trial of two questionnaires

Concepts allied to ethnicity are increasingly coming under question as legitimate variables for use in health research. A randomised controlled trial of two ethnicity screening questions for ascertaining risk of carrying genes associated with sickle cell and thalassaemia illustrates the challenges and limitations of assessing an association of social constructs and genetic statuses.

Objectives . To evaluate two candidate ethnicity screening questions in antenatal screening programmes in low, mixed and high sickle cell prevalence areas, and to identify time taken in administration of the questions by use of the following measures:

1. Proportions of respondents with missing ethnicity data and/or significant changes in ethnic/family origins upon re-interview.

2. Numbers of carriers of clinically significant haemoglobin disorders missed by ethnicity screening questions.

3. Time taken to explain screening question for sickle cell disease (SCD)/thalassaemia and obtain ethnic/family origins.

4. Proportion of clients providing usable ethnic/family origins data.

5. Reported ethnic/family origins in pregnant women at first booking with midwife.

Design . Ten-month (September 2002–June 2003) questionnaire study with random allocation to two self-administered ethnicity questions, comparison with laboratory results and results from re-interview. The settings were antenatal booking clinics in four geographical areas of England of varying expected foetal prevalence of SCD: very high (29.75 per 10,000 pregnancies); high (8.2); mixed high and low (1.29); and low (0.18). The subjects were 4,559 pregnant women at first booking with midwife.

Results . Proportions of respondents with missing ethnicity data and/or significant changes in ethnic/family origins upon re-interview were 4.33% (CI 2.63–6.68%) for a category-based question and 9.45% (CI 6.86–12.61%) for a binary plus open-ended question. Proportions of carriers missed were 5.74% (CI 2.34–11.46%) and 9.71% (CI 4.75–17.13%) by category-based and binary plus open-ended questions, respectively. Average time taken to ascertain ethnic/family origins for screening was between 2.17 and 5.12 minutes in different areas, and up to 15 minutes at the 95th centile. Usable ethnicity screening data was missing in 2.94% of instances. Errors in interpretation or missing data were 3.2% for a category-based question and 4.71% for a binary plus open-ended ethnicity question. Ethnicity Question A produces fewer cases of missing or misinterpreted data (p < 0.001).

Conclusions . A category-based ethnicity screening question was more effective than a binary plus open-ended question. Using the more effective question, 5.74% (CI 2.34–11.46%) of significant haemoglobinopathies will be missed in a selective screening programme, and 4.33% (CI 2.63–6.68%) of replies to an ethnicity screening question will be unreliable when compared to information given upon re-interview. In specific carefully circumscribed situations, namely, in antenatal screening for sickle cell and thalassaemia, it is possible to measure the degree of association between social constructs of ethnicity and health status in a manner that may help in effecting policy decisions.     

Ethnicity/race in the Pró-Saúde Study: comparative results of two methods of self-classification in Rio de Janeiro, Brazil

Variations in classification systems related to color/race and the contextual dependence of their utilization are two challenges for researchers conducting health studies within an ethnic/racial framework. A comparison was made between the results obtained by two distinct approaches to racial self-classification -- a closed question (using categories from the IBGE, or National Census Bureau) and an open-ended question -- in a cohort study of employees at a university in Rio de Janeiro. According to the closed question, 54.0% of the 3,717 respondents classified themselves as white, 30.0% as brown (or mixed-race), and 16.0% as black. According to answers to the open-ended question, the proportions were 53.0%, 25.0%, and 22.0%, respectively, when the terms "moreno", "mesti?o", and "mulato" were grouped under the "brown" category. Despite the high level of agreement (kappa = 0.80; 95%CI: 0.78-0.82), a sizeable numbers of black and brown respondents (open-ended question) chose "whitening" categories available in the list supplied by the IBGE. The use of ethnic/racial categories in health studies can both reveal information on health inequalities in Brazil and also contribute to the formulation of public health policies adequately informed by the specificities of Brazilian society.     

Communicative competence in the delivery of bad news

Grounded in the Cegala and Waldron (Communication Studies 43 (1992) 105) model of communicative competence, the present study applied the McNeilis (Health Communication 13 (2001) 5) provider-patient coding scheme to video tapes of 3rd year medical students delivering bad news to a standardized patient. The goal of the study was to understand the specific communicative moves that are associated with perceptions of competence during bad news delivery. The coding scheme assesses Content, Acknowledgment Tokens, Interruptions, Alignment, and Function of the message. Na?ve observers also evaluated the tapes on several items, assessing empathy and communicative effectiveness. Nonmedical talk was the most common type of content, followed by discussion of the current health problem. Neither acknowledgment tokens nor interruptions were frequent. The most common function of a message was a closed question, followed by explanations, assertions, and open questions. Summing across the functions indicated that information giving was the nost common behavior. The perceivers' data showed fairly neutral assessments of the medical students--they were generally not evaluated very positively, although they were not disliked. Regression analyses indicated numerous specific communicative behaviors that were associated with judgments of competence. Statements falling into the Nonspecific Content category were associated with more positive perceptions, while relational statements, moderately closed questions, solicited answers, expansions, restatements, assertions, explanations, open questions, bracketing, and small talk as well as information verifying, seeking, and giving (summed functions) led to more negative perceptions. The results indicate that the delivery of bad news requires communicative moves that differ from other kinds of medical communication. Depending on the results of future analyses of this topic health are providers may be well advised to focus little of their communication information seeking, giving, or verifying during the initial lab news delivery consultation, but rather to save most communication information for a follow-up scheduled shortly afterwards.     

Patient satisfaction with obstetric care

Patient satisfaction with obstetric care was studied in a cohort of postpartum women from a rural midwestern county. Birth certificate data defined the population, and satisfaction data were acquired through a mailed questionnaire. An indirect measure (satisfaction scale) was derived with acceptable construct validity and internal consistency. A direct measure (open-ended questions) elicited specific comments about each woman's recent experience with obstetric care. Satisfied women, as described by the scale, were more likely to have had good physician continuity and to have attended childbirth classes. The open-ended responses most frequently described problems relating to the physician-patient relationship. In comparing the indirect and direct measures, women with high satisfaction scores were more likely to make no critical comments about their obstetric care (chi 2 = 9.16, P less than .003). The patient's perception of the physician's attitude of concern emerged as an important issue in both measures. The data demonstrate that perceived physician concern is an important component of patient satisfaction with obstetric care.     

Consulting With a Surgeon Before Breast Cancer Surgery: Patient Question Asking and Satisfaction

Research has found a negative association between patient question asking and aspects of their satisfaction. In the context of surgical oncology, the aim of this exploratory study was to test the association between patient question asking and 3 indices of their satisfaction. Participants included 51 women who were newly diagnosed with breast cancer engaged in presurgical consultations with a surgical oncologist from a National Cancer Institute–designated cancer center in the Northeastern United States. Outcomes were patients' postconsultation reports of their satisfaction with the treatment plan, intentions to adhere to the treatment plan, and satisfaction with the surgeon. The main predictor was the frequency of patients' self-initiated questions coded from videotapes of consultations. The frequency of patients' self-initiated questions was negatively associated with their satisfaction with the treatment plan (p = .02), intentions to adhere to the treatment plan (p = .02), and satisfaction with the surgeon (p = .07). Results can be explained in terms of patients' perceptions that the surgeon's information was insufficient or inadequate. Future research needs to identify the specific content of patients' questions and how such content might be associated with satisfaction.     

Association between patient education and health-related quality of life in patients with Parkinson's disease

BACKGROUND: Providing patients with disease- and treatment-related information is an important role of medical staff and is now reimbursed in Japan by the national health insurance system under the rubric 'patient education'. Evaluation of the effectiveness of patient education programs is necessary to ensure that limited health care resources are used efficiently. OBJECTIVE: The objective is to determine whether educating patients with Parkinson's disease (PD) is related to better health-related quality of life (HRQOL). DESIGN: A cross-sectional study was conducted. SETTING: Members of the Japan Association of Patients with Parkinson's disease were randomly selected. PARTICIPANTS: A total of 1200 patients with PD were asked to fill in written questionnaires and replies from 762 (63.5%) were analyzed. MEASUREMENTS: The questionnaire inquired about clinical characteristics, comorbidity, symptoms of PD, complications of therapy, HRQOL, and patient education. SF-36 was used to assess HRQOL. The section on patient education comprised one question each on patient-perceived satisfaction with information provided on (1) disease condition and pathophysiology, (2) effectiveness of drug therapy, (3) adverse drug reactions, (4) publicly available financial and social resources, and (5) rehabilitation and daily activities. Patient education score was defined as the sum of the individual scores for these five questions. The relationships between scores on the SF-36 subscales and the patient education score were examined. RESULTS: More satisfaction with patient education was associated with higher scores in all SF-36 subscales except physical functioning and bodily pain. The difference in score between the most satisfied and the least satisfied patients ranged from 8.4 points on the subscales of general health and 16.7 points on the subscale of role limitation due to emotional problems. CONCLUSION: The conclusion that patient education is associated with better HRQOL in patients with PD is drawn.     

Implications for health education and intervention strategies arising from children's caregivers concerns following successful malaria control

Malaria is still responsible for 10% of the total disease burden in Africa. This study was an empirical investigation addressing the extent to which, and the ways in which, sociocultural, family-based, demographic and economic circumstances in a poor rural African environment influence levels of active malaria infection risk awareness. Face-to-face interviews were conducted and the questions included both open-ended questions allowing for unprompted replies and closed fixed-answer alternative (prompted) items. The adult female responsible for the day-to-day care of the children in 156 homesteads (60 in each of two malaria villages and 36 in a reference village) was interviewed. The families received financial assistance but little physical support from the fathers. Unemployment, poverty, crime and clean water were perceived as the main, unprompted threats, and everyday awareness appeared to exclude the potentially serious disease threats of malaria. Only when malaria was prompted did the concerns rise to 52% and 38% in the sprayed villages. The apparent discrepancy between actual daily and potential future threats significantly increases the difficulty of mobilising communities for preventive action regarding potential threats. The lack of community involvement in an existing community-passive malaria control system may become a problem when promoting new/additional measures to reduce exposure to indoor residual spraying chemicals.     

Convergent validity between a discrete choice experiment and a direct,open-ended method: Comparison of preferred attribute levels and willingness to pay estimates

The Discrete Choice Experiment (DCE) has become increasingly popular as a method for eliciting patient or population preferences. If DCE estimates are to inform health policy, it is crucial that the answers they provide are valid. Convergent validity is tested in this paper by comparing the results of a DCE exercise with the answers obtained from direct, open-ended questions. The two methods are compared in terms of preferred attribute levels and willingness to pay (WTP) values. Face-to-face interviews were held with 292 women in Calgary, Canada. Similar values were found between the two methods with respect to preferred levels for two out of three of the attributes examined. The DCE predicted less well for levels outside the range than for levels inside the range reaffirming the importance of extensive piloting to ensure appropriate level range in DCEs. The mean WTP derived from the open-ended question was substantially lower than the mean derived from the DCE. However, the two sets of willingness to pay estimates were consistent with each other in that individuals who were willing to pay more in the open-ended question were also willing to pay more in the DCE. The difference in mean WTP values between the two approaches (direct versus DCE) demonstrates the importance of continuing research into the different biases present across elicitation methods.     

Mexican patient satisfaction in a rural Minnesota primary care clinic

The Latino population in rural Minnesota has grown significantly in recent years. Despite the increase, few studies have considered whether these newcomers are satisfied with the care they receive from local medical clinics. This article describes the results of a pilot study that assessed 20 Mexican patients' satisfaction with care they received in a primary care clinic in rural central Minnesota. Participants were interviewed using questions from Stewart's Interpersonal Care Survey and open-ended questions. Results showed the patients were generally satisfied with their health care. However, they suggested improvements in the areas of communication and involvement in decisions. Answers to the open-ended questions suggested that even though patients seemed satisfied with their care, they still strongly desired a bilingual physician.     

Convergent validity between a discrete choice experiment and a direct, open-ended method: comparison of preferred attribute levels and willingness to pay estimates

The Discrete Choice Experiment (DCE) has become increasingly popular as a method for eliciting patient or population preferences. If DCE estimates are to inform health policy, it is crucial that the answers they provide are valid. Convergent validity is tested in this paper by comparing the results of a DCE exercise with the answers obtained from direct, open-ended questions. The two methods are compared in terms of preferred attribute levels and willingness to pay (WTP) values. Face-to-face interviews were held with 292 women in Calgary, Canada. Similar values were found between the two methods with respect to preferred levels for two out of three of the attributes examined. The DCE predicted less well for levels outside the range than for levels inside the range reaffirming the importance of extensive piloting to ensure appropriate level range in DCEs. The mean WTP derived from the open-ended question was substantially lower than the mean derived from the DCE. However, the two sets of willingness to pay estimates were consistent with each other in that individuals who were willing to pay more in the open-ended question were also willing to pay more in the DCE. The difference in mean WTP values between the two approaches (direct versus DCE) demonstrates the importance of continuing research into the different biases present across elicitation methods.     

Investigation of validity of closed questions in a survey of British South Asian and white populations

Objectives. The paper concerns the validity of closed-format questions in an interview-based population survey and focuses on a comparison of South Asian and white respondents. Method. A two-part interview consisting of open, respondent-centred questions followed by closed questions taken from a large-scale interview-based health survey was carried out with 15 persons from white and 14 from South Asian communities resident in the UK. Interviewees’ views of their stays in hospital was the focus. Results. The two-part interview was found to provide a broadly satisfactory method for the investigation of validity although a limitation was recognised. Twenty-nine per cent of the variation in response to a closed question on ‘overall satisfaction’ was predictable from views expressed in the open interview. The views of both groups, as expressed in the open interview, were inadequately represented by the closed questions. This appeared to be especially true for the South Asian sample. The validity of the closed question appeared greater for white than for South Asian interviewees. Conclusions. It is concluded that when designing and piloting health surveys consideration should be given to members of all communities to be included in the survey.     

Measuring Patient Satisfaction in Primary Care: a Joint Project of Community Representatives, Clinic Staff Members and a Social Scientist

We describe the process of planning and developing a questionnaireand conducting a patient satisfaction survey in a neighbourhoodclinic in Beer-Sheva, Israel. The project was conducted by theclinic staff members, patient representatives and a medicalsociologist. The satisfaction survey was conducted in patients'homes, with a 67% response rate. General satisfaction and satisfactionwith specific components of service are described. Patient satisfactionwas higher among men than among women, and negatively correlatedwith family size. The strongest predictor of general satisfactionwas satisfaction with physicians' services. Implications ofthe survey results were decided upon by active collaborationbetween the clinic staff and the patient representatives. Theinferences drawn from the patients' replies and the changesintroduced as a result of them, are discussed. Health care consumersshould be active participants in carrying out surveys of satisfactionon a regular basis.