Active Studies

Please check out all of the active studies at the Penn Collaborative below:

Each year, millions of Americans receive evidence-based psychotherapies (EBPs) like cognitive behavioral therapy (CBT) for the treatment of mental and behavioral health problems. Yet, at present, there is no scalable method for evaluating the quality of psychotherapy services, leaving EBP quality and effectiveness largely unmeasured and unknown. AFFECT will develop and evaluate an AI-based software system to automatically estimate CBT fidelity from a recording of a CBT session.  

AFFECT is an NIMH-funded research partnership between the Penn Collaborative and, Inc., (“Lyssn”, pronounced “listen”), a start-up company developing AI-based technologies to support training, supervision, and quality assurance of EBPs. Lyssn’s goal is to develop innovative health technology solutions that are objective, scalable, and cost efficient. Lyssn offers a HIPAA-compliant, cloud-based platform for secure recording, sharing, and reviewing of therapy sessions, which includes AI-generated metrics for MI. The proposed tool will build from and be integrated into this core platform.  

Phase I will work from an existing AI-CBT prototype to develop LyssnCBT. Core activities include user-centered design focus groups and interviews with community mental health therapists, supervisors, and administrators, which will inform the design and development of LyssnCBT. LyssnCBT will be evaluated for usability and implementation readiness in a final stage of Phase I.  

Phase II will conduct a field-based usability trial and a stepped-wedge, hybrid implementation-effectiveness randomized trial (N = 1,850 clients) to evaluate the effectiveness of LyssnCBT to improve therapist CBT skills and client outcomes, and to reduce client drop-out. Analyses will also examine the hypothesized mechanism of action underlying LyssnCBT.  

Successful execution will provide automated, scalable CBT fidelity feedback for the first time ever, supporting high-quality training, supervision, and quality assurance, and providing a core technology foundation that could support a range of EBPs in the future. 

For more information about this study, please contact Torrey Creed PhD at

Despite known benefits to clients and many resources dedicated to implementation, cognitive behavioral therapy (CBT) use in the community remains low. There are many factors that can make it challenging to deliver CBT effectively. However, we do not know how and where to best intervene to support clinicians to deliver CBT.

Project ACTIVE is a NIMH funded study conducted as a partnership between the Center for Mental Health and the Penn Collaborative with the goal to determine the best targets for implementation strategies to increase the use of CBT. This research studies the opinions and experiences of clinicians trained through the Penn Collaborative.

The primary aim of Project ACTIVE is to identify which clinic, clinician, and client factors best support CBT delivery. To achieve these aims, we are recruiting community mental health clinicians from the city of Philadelphia and Texas who have completed the Penn Collaborative CBT training. Participants complete batteries of self-reports related to CBT, submit audio-recordings of their therapy sessions, and answer questions about their audio-recorded sessions.

Successful completion of the proposed study will help us design better strategies for supporting clinicians and agencies to deliver CBT, with the long-term goal to alleviate the suffering of those with mental illness.

Interested in learning more about this project? Contact Temma Schaecter, the Project ACTIVE study coordinator, at

Limited funding is often cited as the biggest impediment to improving the quality of community behavioral health services through the implementation of evidence-based practices (EBPs). Nevertheless, it is rare for publicly funded behavioral health systems serving predominantly low-income and racially/ethnically minoritized populations to have the resources to support EBP use through financial strategies. Despite the inherent challenges, the Philadelphia Department of Behavioral Health and Intellectual Disability Services (DBHIDS) has expended significant resources to improve the quality of behavioral health services through EBPs by utilizing a rarely-used financial strategy. DBHIDS’ use of a financial strategy offers an unprecedented opportunity to empirically examine the real-time experiences of key community stakeholders and target beneficiaries of DBHIDS’ financial strategy. Thus, in partnership with DBHIDS, we are conducting qualitative and quantitative investigations to explore key stakeholder perceptions of the financial strategy and its impact on implementation and service outcomes.

Findings from this investigation will reveal real-time outcomes of the financial strategy as reported by various key community stakeholders. The stakeholder feedback data we obtain will also inform future plans for the financial strategy, and may have implications for the efforts of other similar under-resourced publicly funded behavioral health systems working toward increasing the use of EBPs.

For more information about this study, please contact Vanesa Mora Ringle PhD at

Purpose: The goal of the study is to develop and evaluate methods to efficiently understand what occurs in CBT interventions, using materials that are generated during routine clinical care. Once we have more efficient, scalable and acceptable tools, we can use them for purposes such as training, supervision, and supporting the delivery of cognitive behavioral therapies in routine care settings. We seek to identify good alternatives to "gold standard" of reviewing and coding recordings of therapy sessions, which can be time- and labor-intensive, and not always acceptable to clinicians and their clients. We are asking therapists and their clients to help us evaluate and compare different strategies, and will be asking for their feedback on these strategies.

Who Can Participate: We welcome participation of any treatment provider (clinicians, including interns, postdocs, and practicum students) who uses cognitive behavioral strategies and who use CBT/CPT worksheets as a part of their practice. It could be a good opportunity for trainees to receive feedback based on session recordings and work samples. Decisions about whether, how, and when to use CBT or CPT strategies are entirely up to the clinicians and their patients—we are simply observing what occurs in session. Both patients and therapists will be consented to participate. Clients will be compensated for the time spent participating in the study, and clinicians will receive supportive feedback and continuing education credits, and, depending on their employers' regulations, compensation for their participation. 

For more information, please contact the Study Coordinator at

Advancements in the effort to implement evidence-based practices (EBPs) are being made rapidly, but a corresponding understanding of their sustainment remains quite underdeveloped (Proctor et al., 2015). In part, this shortfall may be related to unique methodological challenges including the lack of a consensus operational definition of the construct or methodology for measurement (Proctor et al., 2015; Stirman et al., 2012). Further, while many early process models of implementation have described sequenced stages with sustainability addressed primarily in later stages (e.g., PRISM, Feldstein & Glasgow, 2008; RE-AIM, Glasgow, Vogt, & Boles, 1999), strategies to support sustained practice of an innovation may best initiated in the earliest steps of implementation (Pluye, Potvin, & Denis, 2004; Pluye et al., 2005).     Despite definitional and methodological challenges, sustainability tools are beginning to emerge; however, the majority of these tools assess sustainment, rather than facilitate sustainment. Therefore, this study aims to develop and test a tool that would allow mental health care organizations to proactively identify areas of strength and places for growth related to their ability to sustain an EBP, then create and follow a tailored plan to improve sustainment.

Efforts are being made around the world to broaden access to evidence-based mental health practices, but a rate-limiting factor is the need to provide feedback at scale for supervision, training, and evaluation of therapist skill. Gold-standard measures like the Cognitive Therapy Rating Scale (CTRS; Young & Beck, 1980) require hours of rating time by experts, and are therefore not feasible to measure implementation at the scale of organizations or systems. Artificial intelligence (AI) and machine learning can provide a set of tools that analyze large quantities of data to quickly identify patterns and relationships including those that humans, on their own, cannot perceive (Bickman, 2020). In mental health care, AI-based tools can provide specific, accurate feedback on aspects of psychotherapy sessions without the limitations of depending on expensive, finite human experts. In this study, we have partnered with experts at the University of Washington, University of Southern California, University of Utah, and Northeastern University to develop a tool that can quickly and automatically rate CBT competence from session audio. The machine learning model can predict the CTRS total score, and 9 of the 11 individual items, at a level of performance that is indistinguishable from human reliability. As the tool is refined, it offers an alternative for measuring therapists' CBT skills efficiently and accurately, which has significant implications for training, implementation, and equitable access to EBPs.

Beyond tool development, we are also examining stakeholder perceptions of the use of machine learning tools in clinical practice. Feedback is being gathered from community mental health clinicians, supervisors, and agency leadership to better understand the acceptability, fit, and appropriateness of a machine-learning tool, as well as ways in which it might be integrated into typical supervision and clinical practices.

Back to Top