Evaluation Research Frameworks

Phases of Research

Intervention studies can be categorized into different phases of research as described in the tables below. This classification can assist researchers in understanding what is needed to take a possible intervention from design through to dissemination. It is also useful to understand how funders (including the National Cancer Institute, for example) conceive of the research process.

Phase Health Behavior Examples:
Hypothesis Development Researchers in the field of alcohol abuse prevention conduct prevalence studies of problems arising from alcohol use, and conduct research to develop potential explanatory models for these behaviors
Methods Development Dr. Leanne Mauriello and colleagues developed and pilot tested a school-based intervention designed to prevent obesity in adolescents that was based on the Transtheoretical Model. The evaluation measures used in the pilot test focused on whether or not participants were satisfied with the program (acceptability) and what could be improved prior to running an efficacy trial.

Phase Health Behavior Examples:
Controlled Intervention Trials (Efficacy Studies) To test the efficacy of an HBM and TTM-based intervention, 96 researcher-recruited participants were randomized to intervention and control groups. While a small sample size was used, the intervention produced significant effects on self-efficacy, intentions and reported stage of change (all p<0.05).
Defined Population Studies (Effectiveness) Authors Robin Wood and Mary Duffy assessed the effectiveness of a video-based health kit designed to encourage women to obtain breast cancer screening via breast self-exam (BSE) and mammography. The program was widely disseminated and the evaluation was conducted among 439 women in two states. Kit users were significantly more proficient at BSE as compared to non-users (p<0.05).

Phase Health Behavior Examples:
Demonstration(Dissemination) Dr. Ross Brownson and colleagues evaluated the effectiveness of The Guide to Community Preventive Services (The Guide) when disseminated to 8 study states as compared to 42 other states. The researchers assessed the extent to which The Guide improved their knowledge of effective programs and also how The Guide improved their use of effective programs. Researchers selected sites but did not oversee or manage the dissemination of The Guide.

References:

Hypothesis Development: Holder, H, Flay, B et al. (1999). Phases of Alcohol Problem Prevention Research. Alcoholism: Clinical and Experimental Research, 23(1): 183-194.

Methods Development: Mauriello, LM, Driskell, MM et al. (2006). Acceptability of a school-based intevention for the prevention of adolescent obesity. Journal of School Nursing, 22(5): 269-277.

Efficacy Trial: Park, S, Chang, S and C Chung. (2005). Effects of a cognition-emotion focused program to increase public participation in Papanicolaou Smear screening. Public Health Nursing, 22(4): 289-298.

Effectiveness Trial: Wood, RY and M Duffy. (2004). Video breast health kits: testing a cancer education innovation in older high-risk populations. Journal of Cancer Education, 19(2): 98-104.

Demonstration: Brownson, RC et al (2007). The effect of disseminating evidence-based interventions that promote physical activity to health departments. Am J Public Health, 97: 1900-1907


Evaluation Research Frameworks

The RE-AIM Framework

As described in the chapter, the RE-AIM Framework can be used by evaluators and other interested in evaluation to frame evaluation research questions. RE-AIM specifically addresses the ways in which the program was received by sites and participants, and the ways in which the program continued beyond the end of the study. Future dissemination of the intervention is often one goal of a RE-AIM-based evaluation team.

The RE-AIM Framework was used to evaluate the Well Integrated Screening and Evaluation for Women Across the Nation (WISEWOMAN) Program, a CDC-funded initiative designed to increase rates of screening for cardiovascular disease-related risk factors among low-income women. Primary outcome measures included changes in risk factors (blood pressure, cholesterol, body weight and smoking) as well as screening rates for these risk factors.

Besculides M, Zaveri H, Farris R, Will J. (2006). Identifying best practices for WISEWOMAN programs using a mixed-methods evaluation. Prev Chronic Dis. Accessed 11/25/2007 from: http://www.cdc.gov/pcd/issues/2006/jan/05_0133.htm.


RE-AIM Dimension WISEWOMAN Evaluation
Reach Number of women participating in program

Number of screenings, first-time screenings

Demographic information about participants
Efficacy/Effectiveness Improvements in blood pressure, cholesterol, body weight and smoking assessed at 1-year follow-up
Adoption Racial and ethnic diversity of participants as compared to entire eligible population
Implementation Participation rate (number of sessions attended)

Re-screening rate among participants
Maintenance Changes in screening rates at each intervention site