Guidelines for Operationalizing the Administration for Community Living (ACL) Criteria for Evidence-Based Programs

Criteria 1: Demonstrated through evaluation to be effective for improving the health and well-being or reducing disease, disability and/or injury among older adults.

1a. Intervention targets at least one primary behavioral, psychosocial, physical and/or physiological outcome(s) relevant to improving the health and well-being, or reducing disease, disability or injury among older adults (age 60+) and/or adults with disabilities.

Being “relevant” depends on what intervention is being studied, e.g. for physical activity, may include strength or function; for depression, includes depressive symptoms. Changes to knowledge or attitudes is not sufficient.

1b. Meaningful improvement is demonstrated in at least one relevant primary outcome at least 6 months following the end of the intervention. “Meaningful improvement” is indicated by effect size or other clinically or statistically significant change in outcome using a valid and reliable measure.

“Clinically significant change” may be demonstrated using effect sizes, comparison to an established intervention (e.g., a new falls prevention intervention delivered using lay leaders provides similar positive health outcomes and is more cost effective than a well-established falls prevention intervention delivered by physical therapists), or a public health criterion (e.g. exercising 150 minutes or more per week per recommended CDC physical activity guidelines).

1c. Outcomes are reported as effect sizes or provide data to be able to calculate effect sizes (e.g. mean, SD, N).

1d. Study provides eligibility criteria and descriptive statistics (demographics, representatives) on study participants to describe the study population (at least half of which are older adults or adults with disabilities).*

*We recommend that descriptive statistics include age; gender; education or income; other chronic conditions; disease severity; recruitment source/setting; enrollment rate.

While not a minimum threshold, ACL supports the development of evidence-based programs that a) are broadly applicable and b) reduce or at a minimum do not exacerbate health disparities experienced by under-served populations (e.g. tribal communities, people who speak languages other than English) and fill gaps in health areas (e.g. oral health, nutrition, hypertension).

1e. Evidence is provided for the safety and tolerability of the intervention as indicated by: (a) minimal/no adverse events directly associated with intervention delivery; and (b) dropout rate is reported for the intervention group and is comparable (or better) than the study’s control group or for similar interventions with similar populations.


Criteria #2: Proven effective with older adult population*, using Experimental or Quasi-Experimental Design.**

*Title III-D criteria specifically addresses older adults; however, programs submitted through this review process may address older adults and/or adults with disabilities.
**Experimental designs use random assignment and a control group. Quasi-experimental designs do not use random assignment. See appropriate designs from the US Preventive Services Task Force here on page 74 of the article (page 31 of the PDF).

2a. Intervention is evaluated using an appropriate experimental or quasi-experimental design that includes an appropriate control group.

An “appropriate control group” is one in which the intervention (treatment) and control (comparison) groups are equivalent OR statistically control for confounding differences between groups if such differences are identified. Furthermore, allocation to the intervention and control group is conducted using a standard/systematic process that minimizes bias (randomization could be utilized, but is not required).

Pilot studies are acceptable if the study meets other criteria.

2b. The sample size provides sufficient power to determine an effect.
2c. If more than one study is published, there are consistent trends in study findings (direction and magnitude).
2d. Information is provided on the implementation of the intervention during the study (e.g., planned and actual frequency, intensity and duration; participation rates).
2e. Methods are reported in sufficient detail for replication and are appropriate given study design.


Criteria #3. Research results published in a peer-review journal.

3a. The published study article(s) has gone through a journal’s independent, external peer-review.
3b. Journal has a published Impact Factor or other published measure of quality.
3c. Journal is indexed in a national scientific indexing database such as PubMed or Web of Science.


Criteria #4: Fully translated** in one or more community site(s).

4a. The program has been delivered with fidelity AND achieved positive outcomes in at least one community site that was not part of the original research study.
4b. The program developer and/or replication sites can be contacted to learn about program implementation and maintenance.
4c. The program’s forms can be adapted for local context using appropriate standards (e.g. changes to program setting, population or modality) without removing or significantly altering core functions.

Appropriate standards include RTIP and HHS/ACF.

Forms are “modes of delivery, who delivers, materials/tools, dose, frequency/intensity,” that can be tailored to local literacy, language, culture and learning styles.

Core functions are “the intended purpose or goals of the intervention” that are done across delivery settings and populations.

**For purposes of the Title III-D definitions, being “fully translated in one or more community sites” means that the evidence-based program in question has been carried out at the community level (with fidelity to the published research) at least once before. Sites should only consider programs that have been shown to be effective within a real-world community setting.


Criteria #5. Developed dissemination products that are available to the public.

5a: The program training is standardized and available on a regular basis, so sites that adopt the program can be trained within 6 months of selecting the program.
5b: There is a reliable way to contact the program developer or national office to obtain training, manuals, and dissemination materials; to discuss implementation; and to receive timely technical assistance regarding implementation on an ongoing basis.
5c: Supports and guidelines for implementing the program are readily available, including implementation manual, quality assurance/fidelity guidelines, data collection protocol, anticipated costs for implementing the program, and overall technical assistance.
5d: Supports for implementing the program are updated on a regular basis.