Measurement-Based Care

• Grant Title: Development and Validation of a Feedback System for Tracking and Improving Trust/Respect in an Economically Disadvantaged Behavioral Health Population
• Sponsor/Funder: Robert Wood Johnson Foundation
• Principal Investigator: Paul Crits-Christoph

Abstract

This study sought to (1) develop and validate a measure of patient trust/respect for use in a feedback system for providers for use in community mental health center (CMHC) populations and (2) test the hypothesis that receiving feedback on a suggestions for improving trust/respect will lead to improvements in patient outcomes. For the randomized effectiveness trial, we planned on recruiting approximately 40 providers and 180 patients who have at least one post-baseline assessment completed. Patients were randomly assigned in a 1:1 ration to the two treatment conditions: symptom only feedback and symptom plus trust/respect feedback. 

Trust and Respect in the Patient-Clinician Relationship: Preliminary Development of a New Scale

Background: Trust and respect may be an important component of client-provider relationships. This study aimed to develop and report preliminary psychometric analyses of a new brief measure to evaluate a patient's level of trust and respect for their clinician. The scale was designed to be applicable in multiple healthcare contexts, with a particular focus on mental healthcare. Methods: Adult patients completed the study survey in an academic outpatient psychiatric clinic waiting room. Classical and Item Response Theory (IRT) analyses were utilized to examine the adequacy of scale items. Validity was examined in relation to the patient-therapist alliance and to willingness to share private information (social media content) with one's clinician. Results: Beginning with 10 items, a final 8-item version of the measure was created with an internal consistency reliability of .91. Principal components analysis indicated that the scale was best viewed as capturing one overall dimension. A Graded Response Model IRT model indicated that all items contributed information on the latent dimension, and all item curves were not flat at any region. The correlation of the trust/respect total score with the alliance was .53 when respect-related items were deleted from the alliance score. The trust/respect scale was significantly associated with patient willingness to share social media posts with their clinician but the alliance was not. Conclusions: The brief measure of patient trust and respect towards their clinician was unidimensional, showed good internal consistency, and was not redundant with existing measures of the alliance. The scale has the potential to be used in a wide variety of healthcare settings. Click on the paper above to learn more!

• Grant Title: Development of a Tool to Measure Consumer Preferences in MDD Treatment
• Grant Number: 5R34MH085817-03
• Principal Investigator: Paul Crits-Christoph

Abstract

Evidence-based practice is defined as the integration of best research evidence with clinical expertise and consumer preferences. However, little attention has been devoted to how to integrate consumer preferences into evidence-based practice in the treatment of major depressive disorder. No practical clinical methodology is available that provides real-time, consumer-weighting of preferences that would permit empirical findings to be used to individualize treatment choice for each mental health consumer with major depressive disorder. The overall goal of this R34 application is to develop and pilot a multi-attribute decision modeling approach in which clinical treatment decisions for people seeking treatment for major depressive disorder in a community mental health setting are guided by evidence-based practice data that has been customized to the preferences of individual consumers. We will apply multi-attribute decision modeling to match up consumers' ratings of their preferences regarding specific treatment attributes (i.e., efficacy, safety, tolerability) to the performance of available treatments as measured by meta-analytic data on each of the attributes (e.g., response rate; incidence of adverse events). Three development steps are proposed here: (1) compile information from existing meta-analyses, or conduct meta-analyses as needed, on the performance of existing evidence-based pharmacotherapies and psychotherapies for major depressive disorder in regard to a list of salient treatment attributes (efficacy; adverse events; tolerability; time commitment), (2) conduct a survey of 80 consumers and 40 clinicians from a community mental health center to evaluate the importance of various specified treatment attributes, and solicit additional treatment attributes deemed to be important, in the treatment of major depressive disorder, and (3) conduct a study examining the feasibility, ease of use, and predictive validity of 3 measures for assessing consumer preferences in regard to a final list of treatment attributes. This final study will be conducted using 72 consumers seeking treatment for major depressive disorder in a community mental health center, with preferences being used to predict duration of time each consumer stays on the treatment recommended to them at the agency. Results of these studies will be used (in future work) to develop a software product that provides real-time assessment of consumer preferences together with a matching of the preferences to attributes of evidence-based treatments for major depressive disorder so that an individualized treatment recommendation is produced to guide the clinician in decision making. Public Health Relevance: Major depressive disorder is one of the most common psychiatric disorders and is associated with considerable social and occupational disability. Incorporating consumer preferences into treatment will facilitate the tailoring of evidence-based practice to the individual and potentially increase consumer satisfaction and improve outcomes.

• Grant Title: Software to Measure Consumer Preferences in the Treatment of Depression
• Grant Number: 1R21MH108996-01A1
• Principal Investigator: Paul Crits-Christoph

Abstract

The goal of this proposal is to develop a novel web-based software program to evaluate consumer preferences in the treatment of depression and potentially other disorders. During year 1 we will develop the requisite software, and during year 2 we will pilot the software with 20 clinicians and 75 consumers in a community mental health setting to determine feasibility and clarity. We will also assess the stability of preference scores over a two month period. The method for assessing preferences, developed and n validated with a previous NIMH R34 award, involves first asking consumers to provide input on their preferences regarding the attributes of different treatments for depression. Attributes of treatments are primarily the side effects of antidepressants, the time commitment for and nature of psychotherapy. Based on our initial R34 work that involved clinicians and consumer input, we arrived at a final list of 18 attributes. The software will use a MAXDIFF (maximum difference scaling, also known as “best worst scaling”) method that presents 4 (of the 18) attribute choices to consumers at a time interactively, and the consumer chooses the most and least preferred of the 4. Scores are then output that rank the attributes from most preferred to least preferred. For each consumer, these preference scores are then compared to actual attributes of treatments for depression using multiattribute decision modeling to arrive at a final ranking of treatments from most preferred to least preferred. Our previous research has shown that receiving a non-preferred treatment leads to considerably longer durations of treatment and significantly greater likelihood of changing treatments. To move this program of research forward, software is needed so that preferred treatments can be measured in real time so consumer preferences can be integrated into actual treatment decisions made at the initial visits to the clinic. Incorporating consumer preferences into the decision-making process for the treatment of depression and other disorders can help expand the use of available options, better meet consumers' needs, and potentially yield better outcomes. Programming the software will involve integrating a web interface for MAXDIFF assessment programming for scoring MAXDIFF output, conducting MADM analyses, and outputting a report that will inform clinicians about consumer treatment preferences.

Methods for Incorporating Patient Preferences for Treatments of Depression in Community Mental Health Settings

We developed three methods (rating, ranking, and discrete choice) for identifying patients' preferred depression treatments based on their prioritization of specific treatment attributes (e.g., medication side effects, psychotherapy characteristics) at treatment intake. Community mental health patients with depressive symptoms participated in separate studies of predictive validity and short-term (1-week) stability. Click the paper above to view our findings!

• Grant Title: The Development of a Therapist Feedback System for MDD in Community Mental Health
• Grant Number: 5R34MH085841-03
• Principal Investigator: Mary Beth Connolly Gibbons

Abstract

The goal of the proposed research is to develop and evaluate the feasibility of a cost-efficient and easily disseminated intervention for improving psychotherapy outcomes for patients with major depressive disorder treated in the community mental health system. A number of studies have demonstrated promising results for feedback systems that identify potential treatment failures and provide outcome information that enables psychotherapists to alter the treatment process to maximize outcomes. Yet no investigations have evaluated the effects of feedback interventions in community mental health centers that are in dire need of cost-efficient and effective interventions to improve outcomes. Our goal is to develop a feedback system that provides information to therapists on the patient's early progress in treatment based on the BASIS-24 as well as important clinical information to guide treatment based on the Personality Assessment Inventory (PAI). We propose a two phase intervention development program in which we will 1) evaluate the feasibility of computerized assessment of the BASIS-24 and PAI, 2) conduct therapist focus groups to develop community friendly feedback reports, and 3) conduct a pilot randomized controlled trial of the feedback system in the community mental health center for the purpose of evaluating the feasibility of the research protocol and the acceptability of the feedback intervention to the community mental health system.. We will randomize therapists delivering services at two community mental health centers to either receive feedback or not receive feedback on the early progress of new patients entering treatment for major depressive disorder. All patients diagnosed with major depressive disorder at the mental health agency will complete the BASIS-24 at baseline and at each treatment session as part of regular clinic procedures. Therapists of patients in the feedback group will receive a brief report based on the BASIS-24 prior to the early sessions of treatment that indicates the patient's progress to date. The report will also include a color code system that indicates to the therapist whether the patient is on track to improve in treatment based on the estimated recovery curves. For patients who are predicted to do poorly, the report will indicate to the therapist that the patient should complete the PAI following the session. The patient will complete the PAI on a computer following his/her therapy session and a clinical report that includes useful clinical recommendations will be given to the therapist prior to the next session. This pilot trial will provide the necessary feasibility and acceptability data to support a future fully-powered trial and to justify the potential success of this feedback system clinically.

PUBLIC HEALTH RELEVANCE

Major depressive disorder is a severe and disabling disorder afflicting 7% of individuals in the United States annually and approximately 17% of individuals across their lifetime (Kessler et al., 2005). Depression has been ranked as the fourth greatest public health problem by the World Health Organization (Murray & Lopez, 1996) and is considered the most likely illness to result in disability (Murray & Lopez, 1996). Despite multiple investigations demonstrating that both medications as well as psychotherapeutic interventions are effective in the treatment of major depressive disorder (APA, 2000), response rates in well-done efficacy trials still reach only 40 to 60% (DeRubeis et al., 2005; Bielski, Ventura, & Chang, 2004; Keller et al., 2000), and response rates for public sector clients are less than 30% (Rush, Trivedi, Carmody, et al., 2004). While many have suggested that outcomes in community-based settings could be improved through the dissemination of empirically-supported psychotherapies (Stirman, Crits-Christoph, & DeRubeis, 2004; Barlow et al., 1999; Chorpita et al., 2002; Henggler et al., 1995), such efforts have a variety of hurdles, including the cost of training therapists in new methods, and resistances of therapists to adopting new approaches that are discrepant from their own preferred style of therapy. The current research paradigm represents an alternative way to improve outpatient mental health outcomes, through performance feedback to the therapist that has the potential to improve mental health outcomes in community clinics in a feasible and sustainable way.

The Effectiveness of Clinician Feedback in the Treatment of Depression in the Community Mental Health System

We describe the development and evaluation of a clinician feedback intervention for use in community mental health settings. The Community Clinician Feedback System (CCFS) was developed in collaboration with a community partner to meet the needs of providers working in such community settings. The CCFS consists of weekly performance feedback to clinicians, as well as a clinical feedback report that assists clinicians with patients who are not progressing as expected. Click the paper above to view our findings!

• Grant Title: Patient Feedback Effectiveness Stud
• Grant Number: 1R01DA020799-01
• Principal Investigator: Paul Crits-Christoph

Abstract

Quality improvement (Ql) methods are a cornerstone of business and healthcare management throughout the United States yet there have been few studies of Ql interventions in addiction treatment settings. The proposed study tests the effectiveness of one Ql system - Patient Feedback (PF) - at increasing outpatient group therapy attendance and self-reported abstinence. The feasibility and acceptability of PF was established in a six-site study conducted within the National Drug Abuse Clinical Trials Network. In the proposed study, 32 community-based outpatient treatment programs with approximately 250 clinicians will be randomly assigned to PF, or usual clinic practices. In the PF condition, every other week clinic patients are invited to complete a 12-item, self-administered survey in which they rate therapeutic alliance and treatment satisfaction, and report past week substance use. These anonymous surveys are faxed by clinic staff to a University of Pennsylvania data center where a custom software application converts the surveys into feedback reports and posts them to a password protected website. Clinicians can access their caseload feedback reports and aggregated reports for the whole clinic; supervisors can only access the aggregated clinic reports. On a monthly basis staff meet as a team to review the feedback reports and develop Ql plans intended to yield improvements in select Ql indicators. The PF website and the monthly PF e-newsletter provide social recognition, clinical resources, and a virtual community for participating clinicians. After 20- weeks, participants in both conditions complete follow-up measures and then both groups are given open access to PF for 12 additional months. During "sustainability phase" staff usage of the PF website is monitored. Alternate versions of the PF Survey are introduced during the sustainability study, including one that monitors HIV risk behavior and one developed collaboratively by staff from the participating sites. The rapid processing of surveys enables near real time feedback to clinic staff. Organizations may share their feedback reports with funding sources, regulatory agencies, policy makers, and other stakeholders. This centralized, semi-automated feedback system eases fulfillment of accreditation requirements and as such, reduces the cost of clinic operations. A collaborating clinical trial application for this test of PF is being submitted by John Rotrosen, M.D., from New York University, School of Medicine.

A Randomized Controlled Study of a Web-Based Performance Improvement System for Substance Abuse Treatment Providers

We report here on the results of a randomized, controlled trial evaluating the efficacy of a semi-automated performance improvement system (“Patient Feedback”) that enables real-time monitoring of patient outcomes in outpatient substance abuse treatment clinics. The study involved 118 clinicians working at 20 community-based outpatient substance abuse treatment clinics in the northeast United States. Ten clinics received 12 weeks of the Patient Feedback performance improvement intervention and ten clinics received no intervention during the 12 weeks. Over 1500 patients provided anonymous ratings of therapeutic alliance, treatment satisfaction, and drug/alcohol use. Click the paper above to view our findings!

A Preliminary Study of the Effects of Individual Patient-Level Feedback in Outpatient Substance Abuse Treatment Programs

The purpose of this study was to examine the effects of feedback provided to counselors on the outcomes of patients treated at community-based substance abuse treatment programs. A version of the Outcome Questionnaire (OQ-45), adapted to include drug and alcohol use, was administered to patients in three substance abuse treatment clinics. Phase I of the study consisted only of administration of the assessment instruments. Phase II consisted of providing feedback reports to counselors based on the adapted OQ-45 at ever treatment session up to Session 12. Patients who were found to not be progressing at an expectable rate (i.e., "offtrack") were administered a questionnaire that was used a second feedback report for counselors. Click the paper above to view our findings!