Measuring treatment outcomes and client satisfaction in a partial hospitalization program

Mental health practitioners are increasingly being called on to evaluate the effectiveness of the treatment they provide. The partial hospitalization component of the mental health industry also has felt this call for outcome research. This article presents the results of one program's attempts to answer that call through an assessment of treatment outcome and client satisfaction at a partial hospitalization program. Participants (N=287) were evaluated at admission, discharge, and 3-month follow-up. The article outlines the procedures used for assessment and uses the results as an example of the type of data that can be obtained through outcomes measurement. The article is intended to provide an example of program evaluation that is easy and inexpensive to administer.     

Using client satisfaction surveys to evaluate and improve services in locked and unlocked adult inpatient facilities

This article describes the implementation, in five inpatient subacute treatment facilities, of a satisfaction survey designed especially for adults with serious and persistent mental illnesses. The survey measures not only global satisfaction but also client perceptions about different treatment modalities and services, important treatment goals, and the philosophy of treatment. Data are presented from 770 completed surveys, illustrating patterns of satisfaction across facilities and services and patterns over time of stability and change in satisfaction. Data are also presented showing how the surveys were used to facilitate and measure improvements in clinical services. Finally, the implications for mental health services delivery are summarized.     

Drug misuse treatment services in Scotland: predicting outcomes.

OBJECTIVE: To investigate which aspects of treatment satisfaction are the best predictors of improved health, improved mental health and achievement of abstinence in drug misuse treatment services. DESIGN: Data were collected as part of the Drug Outcome Research in Scotland study, a prospective cohort study designed to evaluate drug misuse treatment provided in Scotland. Data were collected using a structured interview. Participants were recruited between 1 October 2001 and 30 June 2002. Follow-up interviews were carried out approximately 8 months later. Logistic regression analysis is used to explore client satisfaction with treatment on outcomes, using the Treatment Perceptions Questionnaire (discussed in Marsden et al., Assessing client satisfaction with treatment for substance use problems and the development of the Treatment Perceptions Questionnaire (TPQ). Addict Res 2000;8:455-70). SETTING: Prison, residential and community facilities. PARTICIPANTS: A total of 841 drug users starting a new episode of drug treatment in Scotland in 2000-01. INTERVENTIONS: Methadone, substitute drugs other than methadone, residential rehabilitation, residential detoxification and non-clinical. MAIN OUTCOME MEASURES: Reported improvements in physical health, mental health and abstinence. RESULTS: Client satisfaction predicted positive outcomes, independent of treatment setting. Predicting abstinence and improved physical and mental health were the items: 'I have received the help that I was looking for' and 'The staff have helped to motivate me to sort out my problems'. CONCLUSIONS: Feeling that treatment is appropriate, finding staff motivating, and having enough time to sort out problems are important aspects of satisfaction with treatment among users of drug treatment services who achieved positive treatment outcomes. Services should seek to provide more individualized services based on understanding of individual client needs. This may require longer treatment periods and greater client involvement.     

Factors related to client satisfaction with community-based respite services

The goal of this study was to identify factors related to client satisfaction with respite services. Data were analyzed from interviews with 1183 family caregivers participating in the AoA Demonstration Grants to States program. Two groups of potential explanatory variables were examined: (1) characteristics of client families, and (2) caregivers' perceptions of service delivery. Separate regression models were estimated for users of in-home respite and adult day care. Factors related to satisfaction with in-home respite included elder's ADL, ethnicity, caregivers' expectations for what the respite worker would and would not do, access to services, and the amount of red tape. Factors related to satisfaction with adult day care included the caregiver's age and health, ethnicity, caregivers' expectations for what the respite worker would and would not do, access to services, and the amount of red tape. Differences be- tween the models are discussed in the context of how providers can improve client satisfaction with respite services.     

Integrating behavioral health services into a university health center: patient and provider satisfaction

The goals of this study were to (a) describe an Integrated Behavioral Health Care (IBHC) program within a university health center and (b) assess provider and patient acceptability and satisfaction with the IBHC program, including behavioral health screening and clinical services of integrated behavioral health providers (BHPs). Fifteen providers (nine primary care providers and six nurses) and 79 patients (75% female, 65% Caucasian) completed program ratings in 2010. Providers completed an anonymous web-based questionnaire that assessed satisfaction with and acceptability of behavioral health screening and the IBHC program featuring integrated BHPs. Patients completed an anonymous web-based questionnaire that assessed program satisfaction and comfort with BHPs. Providers reported that behavioral health screening stimulated new conversations about behavioral health concerns, the BHPs provided clinically useful services, and patients benefited from the IBHC program. Patients reported satisfaction with behavioral health services and reported a willingness to meet again with BHPs. Providers and patients found the IBHC program beneficial to clinical care. Use of integrated BHPs can help university health centers support regular screening for mental and behavioral health issues. Care integration increases access to needed mental health treatment.     

Maximizing Satisfaction and Managing Dissatisfaction in Mental Health and Human Services:

The production satisfaction is central to the delivery of mental health and human service. The most critical fator in service delivery is providing quality care. However, the ways in which services are delivered impacts the perception of quality. This paper provides a model for maximizing satisfaction among client and other importance publics of mental health and human service organizations.     

Maximizing satisfaction and managing dissatisfaction in mental health and human services: a model for administrative practice

The production satisfaction is central to the delivery of mental health and human service. The most critical factor in service delivery is providing quality care. However, the ways in which services are delivered impacts the perception of quality. This paper provides a model for maximizing satisfaction among client and other important public of mental health and human service organizations.     

'Shared Care - Shared Dream': Model of shared care in rural Australia between mental health services and general practitioners

OBJECTIVE: To identify a working model between rural and remote mental health services and the local GPs in Australia. DESIGN: Postal questionnaire to assess the GPs' satisfaction level with the involvement of the mental health services in their ongoing management of mentally ill in the community. RESULT: There was a greater sense of satisfaction with the mental health services over the five years this program was implemented. CONCLUSION: This study offers a model on how a rural mental health service could enter into a shared care program with the local GP practice and achieve a greater level of satisfaction in serving the rural communities in Australia.     

宁波市江北区社区卫生服务激励机制与工作满意度探讨

社区卫生服务工作人员是否满意、是否能够得到有效的激励是社区卫生服务改革中一个十分关键的问题。宁波市江北区从分配激励、发展激励和责任激励三个方面进行了社区卫生服务员工激励机制的改革。本文构建了由公平指数、发展指数、和谐指数、快乐指数、满意指数构成的社区卫生服务工作人员满意度综合评价体系,来评估江北社区卫生机构人员的满意度。结果显示,员工的发展、快乐与和谐指数得分较高,而公平指数与满意指数得分较低。因此,需要进一步深化对社区卫生服务的改革,通过物质激励和精神激励并重、责任和发展并重、公平和效率并重的激励机制来提升工作满意度。     

Perceptions by patients and families towards treatment outcomes in mental health services

The need to evaluate treatment outcomes in mental health services from the user's perspective has been highlighted in the literature. The objective of this study was to compare the perceptions of patients and their families regarding treatment outcomes, in terms of perceived changes in the patient's life. The Perception of Change Scale was applied to 100 psychiatric patients that used public mental health services and 100 family members. The results showed a high percentage of agreement between patients' and families' perceptions of change on most items in the scale and on the open-ended questions. The results confirmed previous studies comparing patients' and families' satisfaction with other aspects of services. They also attest to the validity of patients' responses in the evaluation of treatment outcomes in mental health services.     

Benefit–Cost in the California Treatment Outcome Project: Does Substance Abuse Treatment “Pay for Itself”?

OBJECTIVE: To examine costs and monetary benefits associated with substance abuse treatment. DATA SOURCES: Primary and administrative data on client outcomes and agency costs from 43 substance abuse treatment providers in 13 counties in California during 2000-2001. STUDY DESIGN: Using a social planner perspective, the estimated direct cost of treatment was compared with the associated monetary benefits, including the client's costs of medical care, mental health services, criminal activity, earnings, and (from the government's perspective) transfer program payments. The cost of the client's substance abuse treatment episode was estimated by multiplying the number of days that the client spent in each treatment modality by the estimated average per diem cost of that modality. Monetary benefits associated with treatment were estimated using a pre-posttreatment admission study design, i.e., each client served as his or her own control. DATA COLLECTION: Treatment cost data were collected from providers using the Drug Abuse Treatment Cost Analysis Program instrument. For the main sample of 2,567 clients, information on medical hospitalizations, emergency room visits, earnings, and transfer payments was obtained from baseline and 9-month follow-up interviews, and linked to information on inpatient and outpatient mental health services use and criminal activity from administrative databases. Sensitivity analyses examined administrative data outcomes for a larger cohort (N=6,545) and longer time period (1 year). PRINCIPAL FINDINGS: On average, substance abuse treatment costs $1,583 and is associated with a monetary benefit to society of $11,487, representing a greater than 7:1 ratio of benefits to costs. These benefits were primarily because of reduced costs of crime and increased employment earnings. CONCLUSIONS: Even without considering the direct value to clients of improved health and quality of life, allocating taxpayer dollars to substance abuse treatment may be a wise investment.     

State agency policy and program coordination in response to the co-occurrence of HIV,chemical dependency,and mental illness

The co-occurrence of HIV infection, chemical dependency, and mental illness challenges federal and state governments to develop flexible and coordinated health policy and financing for public health services. State agencies play a critical role in the organization and support of these services at the local level. With emerging stress upon state government budgets and concomitant increasing need for publicly funded services, state agency coordination may be an important policy safety net to assure services for populations at the margins of health systems. Despite this important potential role, nothing is known about the degree to which state HIV, substance abuse, and mental health agencies coordinate policies and/or programs in response to these co-morbid conditions. OBJECTIVE: This study sought to establish a conservative and initial understanding of state HIV, substance abuse, and mental health agency coordination of policy and program in response to the co-occurrence of HIV, chemical dependency, and mental illness. METHOD: Estimation of coordination was accomplished through the comparison of three surveys conducted among state substance abuse directors (1998), state AIDS directors (1999), and state mental health directors (2000). Data from 38 states were reviewed. RESULTS: The most frequently reported state agency activities included coordinating funding, engaging in integrative planning activities, and conducting staff cross-training. When compared for association with state characteristics, coordination among state agencies was found to be associated with Early Intervention Services (EIS) designation, higher rates of AIDS generally, higher rates of AIDS among African Americans, and higher rates of AIDS among Hispanic populations. Given the limitations of comparing three disparate surveys, we determined the estimate of interagency coordination to be conservative and preliminary. CONCLUSION: While this study was useful as an initial step toward identifying state interagency policy and program coordination in response to the co-occurrence of HIV, chemical dependency, and mental illness, there were methodological challenges that should be addressed in future studies of state agency coordination. Several recommendations were advanced.     

Issues in assessing satisfaction: findings of the arkansas medicaid family planning demonstration waiver evaluation.

Arkansas was one of the first states to expand family planning services under its state Medicaid plan through a Section 1115 Demonstration Waiver. In addition to an assessment of other outcome measures, evaluators used exit interviews, which had been designed to reduce courtesy bias, to assess client satisfaction with family planning services available under the waiver in public health units. Client satisfaction with family planning services can serve as a simple measure of one dimension of perceived quality of care. Findings indicate overall client satisfaction with family planning services. However, noted methodological issues may limit the generalizability of these findings. Potential corrective measures are suggested for improving client satisfaction with family planning services and for increasing the utility of survey instrument utilized in this effort.     

Migrant health revisited: a model for statewide health planning and services.

Since the migrant farmworker family is a marginal issue among competing priorities for public health services, the logical strategy ought to be the pooling of limited resources at the state and local level to provide maximum benefit for the dollar and the client. A program planning model in inter-agency migrant health services delivery has been developed in Colorado. The model includes tangible evidence of cooperation by front-line service agencies. A task force approach for joint agency programming was initiated at state and local levels, and a structure for accountability was established which was carried out with performance contracts.     

MHSPY: A Children's Health Initiative for Maintaining At-Risk Youth in the Community

The Massachusetts Mental Health Services Program for Youth (MHSPY) is a home-based clinical intervention that seeks to maintain youth with severe functional impairment in the community via delivery of integrated primary care, mental health, substance abuse, and social services. Using blended public agency funding, traditional and nontraditional services are provided within a private, not-for-profit, managed care organization. Individualized, comprehensive care plans are developed by an MHSPY care manager, who works intensively with the family and the Care Planning Team to identify needs and resources. Data on clinical functioning are collected at baseline and every six months during the program. Service utilization and cost are measured on a quarterly basis. Family, youth, and agency satisfaction ratings are collected at disenrollment. Aggregate analyses based on four years of data show that MHSPY participants have improved clinical functioning, including significant reduction in risk to self and others. They also experience reduced service utilization and cost and high rates of family satisfaction.Brian Mullin, BA, is a Senior Data Analyst at Neighborhood Health Plan, 253 Summer St., Boston, MA 02210, USA.     

Clients with substance abuse and mental health concerns: A guide for conducing intake interviews

Although comorbidity (co-occurrence of a psychiatric and substance use disorder) is a common phenomenon at both mental health and substance abuse treatment agencies, rarely do such agencies thoroughly assess for both types of diagnoses during their standard intake interview. This article describes the development of an intake form designed to guide a comprehensive assessment of both mental health and substance abuse concerns. The form guides intake interviewers toward documenting administrative and demographic information, substance use and mental health concerns, and variables needed for compliance with grant funding sources. Use of the protocol can provide a clinical foundation for treatment planning and continuity of care for clients, while also providing error-free agency data that can be used for administrative, program planning, outcome assessment, and research purposes.     

Case management for the mentally ill: a comparative evaluation of client satisfaction

This paper reports on client satisfaction with community mental health services. A Client Satisfaction Interview was used as one outcome measure in a randomized controlled study of clinical case management vs. standard community care. The Interview had high internal consistency reliability (Cronbach's alpha = 0.89) with very high inter-rater reliability for overall scores (r= 0.99) and good inter-rater reliability for individual items (kappa over 0.9 for 18 of 22 items). Case management clients had significantly higher total satisfaction scores than control group clients who received standard community care at 9 months (t= 2.02 P= 0.05) but not at baseline (t= 0.59, P= 0.55) or 18 months (t= 1.76, P= 0.09). In addition to a quantitative measure of satisfaction, clients were asked to provide more qualitative accounts of their experience with services. The findings are discussed in relation to the team's model of case management and the differences between case management and control services. The limitations of this study are considered in the context of methodological issues surrounding measurement of client satisfaction and the study's own methodological shortcomings.     

Comparing alternative risk-adjustment models

The use of mental health indicators to compare provider performance requires that comparisons be fair. Fair provider comparisons mean that scores are risk adjusted for client characteristics that influence scores and that are beyond provider control. Data for the study are collected from 336 outpatients receiving publicly funded mental health services in Washington State. The study compares alternative specifications of multiple regression-based risk-adjustment models to argue that the particular form of the model will lead to different conclusions about comparative treatment agency performance. In order to evaluate performance fairly it is necessary to not only incorporate risk adjustment, but also identify the most correct form that the risk-adjustment model should take. Future research is needed to specify, test, and validate the mental health risk-adjustment models best suited to particular treatment populations and performance indicators.     

Trends in acute mental health care: Comparing psychiatric and substance abuse treatment programs

This study compared psychiatric and substance abuse acute care programs, within both inpatient and residential modalities of care, on organization and staffing, clinical management practices and policies, and services and activities. A total of 412 (95% of those eligible) Department of Veterans Affairs' programs were surveyed nationwide. Some 40% to 50% of patients in psychiatric and substance abuse programs, in both inpatient and residential venues of care, had dual diagnoses. Even though psychiatric programs had a sicker patient population, they provided fewer services, including basic components of integrated programs, than substance abuse programs did. Findings also showed that there is a strong emphasis on the use of clinical practice guidelines, performance monitoring, and obtaining client satisfaction and outcome data in mental health programs. The author's suggest how psychiatric programs might better meet the needs of acutely ill and dually diagnosed patients (eg, by incorporating former patients as role models and mutual help groups, as substance abuse programs do; and by having policies that balance patient choice with program demand).