Abstract
Established in 1996 with funding from CDC and NIH, the HIV/AIDS Prevention Program Archive (HAPPA) is now the biggest private sector collection of HIV-related evidence-based behavioral interventions (EBIs). Each EBI in HAPPA has been determined by a distinguished Scientist Expert Panel to have demonstrated efficacy in preventing HIV or its risk-related behaviors in the United States. The multimedia replications kits contain everything that a new site would need to implement an EBI such as a user guide that gives an overview of the program and the evidence of its effectiveness; a facilitator’s manual that gives step-by-step implementation protocols for each session; and session implementation materials referenced in the facilitator's manual such as slides, video clips, participant handouts, activity masters, checklists, and homework assignments for the next session. The program packages also contain evaluation materials such as surveys and questionnaires that were used in the original demonstration of effectiveness and that may be used to re-evaluate the program as implemented in a new setting. Recently, we have expanded HAPPA’s scope to include HIV EBIs developed globally and to include evidence-based structural interventions (effective in modifying the physical, social, cultural, political, economic, legal, and/or policy aspects of the HIV risk environment). This paper describes HAPPA’s procedures for identifying, selecting, acquiring and packaging HIV EBIs. It also provides comprehensive lists of evidence-based HIV behavioral and structural interventions and gives information on how to access EBI program packages for implementation in new settings.
Author Contributions
Academic Editor: Duy Quang Trinh, Nihon University School of Medicine. Division of Microbiology, Department of Pathology and Microbiology.
Checked for plagiarism: Yes
Review by: Single-blind
Copyright © 2013 Josefina J. Card, et al
This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Competing interests
The authors have declared that no competing interests exist.
Citation:
Introduction
To make the best use of limited resources, researchers, practitioners, and funding agencies have increasingly emphasized the importance of disseminating and implementing evidence-based behavioral interventions (EBIs) for HIV prevention 1, 2, 3. Sociometrics’ HIV/AIDS Prevention Program Archive (HAPPA) is a collection of program packages containing everything needed to implement and re-evaluate behavioral programs that have demonstrated efficacy in preventing HIV or its risk-related behaviors among adults in the United States (US) 4. A complementary resource to the Centers for Disease Control and Prevention’s (CDC) Replicating Effective Programs (REP) and Diffusion of Effective Behavioral Interventions (DEBI) projects 5, 6, HAPPA has likewise been shown to promote organizations’ adoption, implementation, and evaluation of evidence-based interventions. Since its inception, more than 4,400 HAPPA products have been disseminated to over 600 unique customers nationwide. In the past three years alone, HAPPA programs were implemented with a diverse pool of more than 11,000 individuals 7.
The original HAPPA collection included 11 EBIs for which evaluation studies had been published through 2001 4. In the past decade, many new HIV prevention programs have been developed and rigorously evaluated. In 2003, for example, CDC, in partnership with other government and nongovernment agencies, launched the Advancing HIV Prevention (AHP) initiative, which included prevention for persons living with HIV (PLH) as one of its key strategies 8. To this end, AHP funded multiple large-scale demonstration projects to test behavioral intervention models to help PLH reduce their risk of transmitting HIV 8, 9. Development of new prevention programs has also focused on populations disproportionately affected by HIV including African Americans, Hispanics, and men who have sex with men (MSM), for whom there had been few EBIs available 10.
Historically, EBIs for HIV prevention have been delivered by health professionals working in schools, clinics, and community-based organizations. Recent advances in information technology have enabled the development of a new generation of computer-delivered programs that incorporate a variety of interactive media elements such as websites, multimedia presentations, animations, audio and video clips, games, and/or social networking applications 11, 12. Computer-based interventions have been shown to have demonstrated efficacy in reducing sexual risk behaviors 11, 12, 13, 14. Moreover, they can often be implemented at a reduced cost and with greater fidelity to program content while allowing for greater flexibility of administration in terms of timing and location 14, 15.
In this article, we describe the methods used and results of our efforts to update and expand HAPPA with new EBIs that have had evaluation studies published since our original article in 2001. We examine trends regarding key characteristics of EBIs over the last decade. We conduct a brief tour of the 24 EBIs available from HAPPA. We look at recent developments that are updating and expanding HAPPA to include computer-delivered interventions, global interventions, and effective structural interventions aimed at changing the contexts or environments that shape individual behavior.
Methods and Findings
HAPPA Collection Development
In 1996 Sociometrics Corporation, under terms of Small Business Innovation Research (SBIR) Grants from the Centers for Disease Control and Prevention and the National Institute of Allergy and Infectious Diseases, began development of the HIV/AIDS Prevention Program Archive (HAPPA). HAPPA is a collection of HIV prevention program packages (also known as replication kits) containing all the materials needed to replicate and re-evaluate EBIs that have been shown to be efficacious in preventing HIV infection or its risk-related behavioral antecedents among adults 4, 16. A related collection, the Program Archive on Sexuality, Health, & Adolescence (PASHA) is developing similar prevention program packages for adolescent target audiences 17. Training on appropriate use of both HAPPA and PASHA program packages is available, but not typically required, to implement the programs. Free technical assistance is provided to all users upon request as needed.
In 2006, a new set of SBIR grants from the National Center on Minority Health and Health Disparities was received to update and augment the set of program packages comprising HAPPA. A similar, if slightly updated, 7-step procedure was used to select programs 4:
1. Establish an independent Scientist Expert Panel to select programs. The HAPPA Panel currently consists of four nationally recognized experts in HIV/AIDS prevention research: Don Des Jarlais (Beth Israel Medical Center), Ralph DiClemente (Emory University), Don Morisky (University of California Los Angeles), and Gina Wingood (Emory University).
2. Articulate the selection criteria to assess program efficacy. HAPPA staff worked with the Scientist Expert Panel to update the objective criteria used for assessing the evidence of effectiveness of diverse HIV/AIDS prevention programs. In keeping with the evolving state of the science regarding evaluation of HIV prevention programs, including the shift toward the development of programs focused on secondary-in addition to primary-prevention, two updates were made to the selection criteria from those used to select programs evaluated through 2001. First, the required follow-up time period for evaluation assessment was increased from three months to six months. Programs had to demonstrate that their positive outcomes persisted for at least six months after the end of the intervention in order to be included in HAPPA. Second, additional criteria were added that were relevant for prevention programs targeting PLH, such as improved antiretroviral therapy (ART) adherence and decreased viral load. In addition to these two updated criteria, programs were evaluated according to: quality of program implementation; scientific rigor of evaluation; and positive effects on HIV risk behavior (sexual, drug injection, or prenatal/perinatal), antiretroviral therapy adherence, or biological markers (STI/HIV rates or viral load). For a complete list of HAPPA EBI-selection criteria, please refer to the table titled HAPPA EBI-Selection Criteria provided on the Sociometrics website (http://www.socio.com/pdf/happasupplementary.pdf).
3. Identify candidate programs. HAPPA staff identified candidate programs via a comprehensive search of the peer-reviewed literature using databases such as Academic Search Complete, JSTOR, and PubMed. To ensure that important programs were not missed, nominations and literature pointers were solicited from funders, researchers, program directors, and evaluators working in the field.
4. Prepare briefing materials for candidate programs. To assist HAPPA’s Scientist Expert Panel in making selection decisions, HAPPA staff prepared a 4-6 page summary for each candidate program consisting of the name(s) of the original developer(s) and evaluator(s), a description of the program intervention and program materials, a description of the evaluation methods and findings, and a list of references. Each summary was accompanied by one or more key journal articles providing further detail on the program and its evaluation.
5. Select programs. Panel members were asked to review the briefing materials and, based on the selection criteria described in Step 2 above, assign each candidate program an overall priority score ranging from 1 (weakest evidence of effectiveness, lowest priority for HAPPA) to 10 (strongest evidence of effectiveness; highest priority). Panel members were informed that HAPPA staff would interpret an average priority score of 7 or higher as their consensus judgment that the program was an EBI approved for inclusion in the HAPPA collection. In order to avoid any conflicts of interest, Panel members were asked to abstain from voting on their own programs. For a complete list of the 20 EBIs approved for inclusion in HAPPA in Round 1 (1997-2001) and 35 EBIs approved for inclusion in Round 2 (2007-2011) please refer to the table titled EBIs Selected by Scientist Expert Panel for Inclusion in HAPPA provided on the Sociometrics website (http://www.socio.com/pdf/happasupplementary.pdf).
6. Acquire selected programs. HAPPA staff worked with developers or current holders of selected programs to acquire their programs and evaluation materials for archiving and public dissemination. The table referenced above notes the 24 (of the 55 approved EBIs) we were able to acquire for HAPPA. We gave acquisition priority to programs selected by the HAPPA Panel that were not yet publicly available from other sources such as the CDC DEBI program.
7. Prepare HAPPA program packages in both print and digitized formats. Each HAPPA program package includes a comprehensive User’s Guide containing detailed instructions and tips for how to implement the intervention; all relevant implementation materials (e.g., facilitator’s manuals, videos, posters, handouts), reformatted as needed to increase their visual appeal and usability; and evaluation instruments that were used in the original investigation to demonstrate the program’s efficacy in changing HIV-related risk behavior. In addition, each program package contains a research-based program adaptation toolkit, a new tool to help users tailor EBIs for new target populations and contexts, while maintaining fidelity to the intervention’s theory of change and core components. Preparation of the HAPPA program packages was done in collaboration with the original developer(s), who received full authorship credit for the program package, with HAPPA staff receiving secondary credit for work on documentation and production.
Characteristics of Programs Selected by HAPPA’s Scientist Expert Panel as EBIs
Since HAPPA’s inception, 88 programs have been submitted to the Scientist Expert Panel as candidates for inclusion in the collection. Of these, 55 have been selected by the Panel as EBIs for inclusion in HAPPA, based on the strength of their evidence for effectiveness per the objective criteria previously described. Sample sizes for the outcome studies documenting the efficaciousness of these 55 programs ranged from 43 to 38,635 participants. The outcome studies were done in all parts of the U.S., often in multiple locations, on diverse samples of young, middle-age, and older adults.
The 55 EBIs approved for inclusion in HAPPA target many different populations such as gay, bisexual, and non-gay identified MSM (N=10), women (N=13), substance users (N=12), sexually transmitted infection (STI) clinic patients (N=6), and PLH (N=10), among others (Table 2). Additional information portraying the diversity of the EBIs is shown in Table 1. The majority are primary prevention programs (82%); intended to be conducted with small groups (64%); addressing sexual risk behaviors (91%); and originally implemented in clinics, hospitals, or treatment facilities (60%).
Significant differences emerged between the 20 programs selected as EBIs in Round 1 versus the 35 programs selected a decade later, providing some data on trends in the field. As summarized in Table 1, the total number of EBIs designed specifically for African Americans, Hispanics, and other minority populations more than tripled by 2011. The total number of EBIs targeting gay, bisexual, and/or non-gay identified MSM likewise nearly doubled, accounting for 20% of all programs deemed EBIs. A significantly greater proportion of EBIs targeting PLHs was selected in Round 2 compared to Round 1 (p=0.008). A significantly greater proportion of Round 2 EBIs targeted individuals and couples as opposed to small groups and communities (p=0.003). There was also a significant change in the distribution of selected programs’ original intervention settings: a greater proportion of Round 2 EBIs were implemented in clinics, hospitals, and treatment facilities and fewer were implemented on a community-wide scale (p=.016).
Table 1. Summary of Key Characteristics of 55 Programs Selected as EBIs by the HAPPA Panel| Characteristic | Selected Programs, N (%) | ||
|---|---|---|---|
| Round 1 EBIs, Selected form 1997-2001 (N=20) | Round 2 EBIs, Selected from 2007-2011 (N=35) | Total (N=55) | |
| Type * | |||
| Primary prevention program | 20 (100) | 25 (71) | 45 (82) |
| Prevention with positives | 0 | 10 (29) | 10 (18) |
| Level * | |||
| Individual | 1 (5) | 13 (37) | 14 (25) |
| Couples | 0 (0) | 2 (6) | 2 (4) |
| Small group | 15 (75) | 20 (57) | 35 (64) |
| Community | 4 (20) | 0 (0) | 4 (7) |
| Behavior Targeted | |||
| Injection drug use | 2 (10) | 2 (6) | 4 (7) |
| Sexual risk behaviors | 14 (70) | 25 (71) | 39 (71) |
| Injection drug use and sexual risk behaviors | 4 (20) | 7 (20) | 11 (20) |
| Antiretroviral Therapy (ART) Adherence | 0 (0) | 1 (3) | 1 (2) |
| Original Implementation Setting * | |||
| Clinic/hospital/treatment facility | 9 (45) | 24 (69) | 33 (60) |
| Community-based organization | 7 (35) | 11 (31) | 18 (33) |
| Community-wide (e.g., neighborhood/city/county) | 4 (20) | 0 (0) | 4 (7) |
| Sex Targeted | |||
| Males | 6 (30) | 9 (26) | 15 (27) |
| Females | 4 (20) | 9 (26) | 13 (24) |
| Both males and females | 10 (50) | 17 (49) | 27 (49) |
| Race/Ethnicity Targeted a | |||
| Primarily Black or African American | 5 (25) | 12 (34) | 17 (31) |
| Primarily Hispanic/Latino (a) | 1 (5) | 2 (6) | 3 (5) |
| Primarily Asian-American or Pacific Islander | 1 (5) | 0 (0) | 1 (2) |
| Primarily White | 7 (35) | 4 (11) | 11 (20) |
| No predominant race/ethnicity targeted | 6 (30) | 17 (49) | 23 (42) |
| Sexual Orientation Targeted | |||
| Men who have sex with men (MSM)/gay/bisexual | 6 (30) | 5 (14) | 10 (20) |
| Heterosexual or unknown/not reported | 14 (70) | 30 (86) | 45 (80) |
EBIs Publicly Available from HAPPA
The public availability of the 55 programs selected as EBIs by HAPPA’s Scientist Expert Panel is also provided in the EBIs Selected by Scientist Expert Panel for Inclusion in HAPPA table provided on the Sociometrics website (http://www.socio.com/pdf/happasupplementary.pdf). Of the 55 programs deemed EBIs by the Expert Panel, 24 (44%) are currently publicly available from HAPPA (http://www.socio.com/happa.php). Another 20 (36%) can be obtained from other sources (e.g., CDC or the original program developers). This means that a full 80% of programs deemed EBIs by HAPPA’s Scientist Expert Panel are publicly available, a testament to the level of cooperation existing in the field.
Only 11 (20%) of HAPPA Panel-approved EBIs are not publicly available. Some developers deemed their EBI “out of date” or had concerns that their evaluation results were not sufficiently positive overall, despite the positive priority score assigned to their program by the HAPPA Scientist Expert Panel. Other developers, for their own reasons, preferred to keep their EBI private.
Table 2 shows characteristics of the 24 EBIs available from HAPPA, in terms of approaches and specific pedagogical techniques employed. The majority of EBIs included in the HAPPA collection incorporate behavioral skills development (92%), HIV/STI education (88%), promotion of condom education and access (79%), and/or self-efficacy/self-esteem building (79%). Approximately one-fourth additionally include some form of community outreach (29%) and/or education regarding safe injection drug use practices (25%). Pedagogical techniques most commonly employed as part of these approaches include group discussion (88%), role play (88%), behavioral skills practice (83%), and safer sex communication/negotiation activities (79%). More than half (58%) include video and/or slide presentations; one-third (33%) include a peer counseling/instructional component; and one-fourth (25%) incorporate HIV antibody testing and counseling.
Table 2. Approaches and Pedagogical Techniques of 24 EBIs Available from HAPPA| N (%) | |
| Approach | |
| Behavioral Skills Development | 22 (92) |
| Community Outreach | 7 (29) |
| Condom Education/Access | 19 (79) |
| HIV/STI Education | 21 (88) |
| Needle Cleaning/Exchange | 6 (25) |
| Self-Efficacy/Self-Esteem | 19 (79) |
| Pedagogical Techniques | |
| Behavioral Skills Practice | 20 (83) |
| Group Discussion | 21 (88) |
| HIV Antibody Testing/Counseling | 6 (25) |
| Lectures | 19 (79) |
| Peer Counseling/Instruction | 8 (33) |
| Role Play | 21 (88) |
| Safer Sex Communication/Negotiation | 19 (79) |
| Video/Slide Presentation | 14 (58) |
Dissemination and Implementation of EBIs
HAPPA has contributed significantly to the dissemination and implementation of EBIs. Since HAPPA’s inception, 4,488 HAPPA-related materials have been obtained by over 600 different organizations. A related study 7 found that among organizations that had obtained an HIV prevention EBI from Sociometrics in the last three years intending to implement such EBI, 53% had implemented the EBI at least once or were in the process of implementing the EBI for the first time; another 22% were preparing for implementation. Minority populations were well reached: 87% of implemented programs included African American participants and 67% included Hispanic participants. These HIV-related EBIs, obtained in the last three years from Sociometrics, had been implemented with approximately 11,381 individuals.
Augmenting the HAPPA Collection
Electronic Access and Delivery Platforms
All HAPPA program packages have now been converted to digitized PDF format, for 24/7 download from the Internet (http://www.socio.com/happa.php). A full half of HAPPA customers now obtain the HAPPA EBIs in digitized, as opposed to printed, format. The digitized versions have proven to be especially popular among international HIV researchers and practitioners.
In addition, multimedia, computer-delivered versions of two HAPPA programs, SISTA for African American women ages 18-29 and WiLLOW for HIV+ African American women, have been developed and evaluated. The computer-delivered version of SISTA is known as SAHARA: Sistas Accessing HIV/AIDS Resources At-a-click (http://www.socio.com/sahara.php). A multimedia version of the efficacious SiHLE program for African American female teens ages 14-18 is also available. Evaluations of SAHARA and Multimedia SiHLE have shown the programs to be as efficacious-in terms of effect size-as their health educator-delivered forebears 14, 15. Data collection for the evaluation of Multimedia WiLLOW’s efficacy has just been completed. Positive results about the continued efficacy of this new computer-delivered program have also been attained 18.
Online EBI Search Assistant
An online HIV EBI Search Assistant has been created. The Search Assistant, freely available from the HAPPA website (http://www.socio.com/pdf/happasupplementary.pdf), helps guide practitioners in selecting the HIV EBIs that are the best match for their local contexts. Users of the Search Assistant can search for relevant EBIs by one or more key criteria, including: intervention level (individual level, group level, community level); behavior targeted (injection drug use, sexual risk behaviors, both injection drug use and sexual risk behaviors); original implementation setting (clinic/hospital/treatment facility, community-based organization, community-wide neighborhood/city/county); age (adolescent and young adult, adults of all ages); sex (all male, all female, both male and female); race/ethnicity (all or primarily African American, all or primarily Hispanic/Latino, all or primarily Asian or Pacific Islander, all or primarily White, no predominant race/ethnicity); sexual orientation (lesbian/gay/bisexual including MSM, heterosexual, unknown); and HIV status (negative/mixed/unknown/not reported, positive). The search results display a list of HIV prevention EBIs exactly and partially matching the search criteria inputted by the user, with links to additional information about these EBIs and the opportunity to purchase/download them immediately, if desired. Figure 1 gives an illustrative HAPPA search (input and results).
Figure 1.Illustrative HAPPA Search Input and Results.
Global EBIs
Global EBIs, developed for and evaluated in international settings, are currently being added to HAPPA. Global EBIs were selected for public dissemination by the Global HIV Archive’s Scientist Expert Panel consisting of seven 7 experts in HIV/AIDS research and program development: Carlos Cáceres, Ph.D., Don Des Jarlais, Ph.D., Quarraisha Abdool Karim,Ph.D., Jesse Mbwambo, Ph.D., Donald Morisky, Ph.D., Susan Pick, Ph.D., and Suniti Solomon, M.D. In selecting EBIs for the Global Archive, this Panel used criteria analogous but not identical to HAPPA’s including: (1) quality of program implementation (content quality and faithfulness of implementation); (2) scientific rigor of evaluation (appropriate design and methods, with comparison group); (3) follow-up assessment occurred a minimum of three months after the end of the intervention period, preferably longer (HAPPA required a six-month follow-up period); and (4) demonstrated positive impact on one or more of these HIV-related behaviors and/or HIV infection rates for one or more subgroups of persons -- sexual risk behaviors, drug injection risk behaviors; pre-and perinatal transmission risk behaviors; antiretroviral therapy adherence; STI/HIV infection rates; viral load. Program packages for 10 global EBIs are available in both their original language and in English, in printed as well as digitized pdf formats. For a complete list of Global HIV Archive programs, please refer to the table titled HIV/AIDS Prevention EBIs in the Global HIV Archive provided on the Sociometrics website (http://www.socio.com/pdf/happasupplementary.pdf).
A compendium of effective structural interventions, Best Evidence Structural Interventions for HIV Prevention 19, was published in 2013 by Springer Science + Business Media. Functioning as a resource to HIV prevention service providers, planners, policymakers, funders, and others, this collection of effective structural interventions includes information and materials to learn about, replicate, or adapt structural interventions (interventions that seek to change the physical, social, cultural, political, economic, legal, and/or policy aspects of the risk environment; 20). This resource includes 18 case studies of evidence-based structural interventions (ESIs) selected as efficacious by the same Scientist Expert Panel that chose the HAPPA EBIs. Five structural interventions have been shown efficacious in reducing injecting drug abuse (IDU) risk, five in reducing non-commercial sex risk, five in reducing commercial sex risk, and three in increasing HIV screening and access to HIV antiretrovirals. For a complete list of the structural intervention programs included in the compendium, please refer to the table titled Best Evidence Effective Structural Interventions for HIV Prevention provided on the Sociometrics website (http://www.socio.com/pdf/happasupplementary.pdf).
Summary and Conclusion
Over the last 15 years HAPPA has selected, under the guidance of its Scientist Expert Panel and based on objective criteria of efficaciousness, 55 HIV prevention programs as EBIs, or evidence-based behavioral interventions. In collaboration with developers of these selected EBI programs, HAPPA has then gone on to create publicly available program packages (also known as replication kits) for 24 of the 55 EBIs. The HAPPA replication kits are the largest private sector collection of HIV-related EBIs. HAPPA’s pioneering work has foreshadowed several current scientific zeitgeists such as “evidence-based medicine”, “research to practice”, and “translation science”.
Over the last 12 years, over 4,000 of HAPPA’s replication kits and associated materials (evaluation instruments, facilitator’s manuals), in printed and digitized/downloaded formats, have been acquired by schools, clinics, local and State governments, and community-based organizations. The materials have been obtained for grant-writing, research, implementation, adaptation, and/or re-evaluation purposes 7, a testament to HAPPA’s success in promoting the broader uptake and implementation of evidence-based HIV prevention programs.
In addition to presenting an up-to-date list of current U.S. HIV-related EBIs, this paper has also presented up-to-date lists of effective global HIV-related interventions, whether behavioral or structural in their approach. HAPPA will shortly be facilitating access to these global EBIs and structural interventions, via printed and downloadable materials that will allow schools, clinics, or community-based organizations to replicate or adapt them.
HAPPA has been a major complement to the CDC’s REP and DEBI projects. Although the HAPPA and CDC EBI lists overlap, they are by no means identical, because of slightly different inclusion criteria used. Thus HAPPA offers an alternate, complementary supplementary lens through which HIV-related EBIs can be viewed 21.
In the decade since the original paper describing HAPPA was published 4, the field of evidence-based medicine has advanced. There has been much analogous movement in the development, evaluation, and identification of HIV prevention EBIs. This paper has noted such trends in the field, by studying the content and scope of programs identified as EBIs in HAPPA’s first (1997-2001) versus second (2007-2011) selection rounds.
Acknowledgements
This paper was funded by the National Center for Minority Health and Health Disparities (Grant R44 MD005177-02). The HIV/AIDS Prevention Program Archive (HAPPA) is funded by the National Institute of Allergy and Infectious Diseases and the National Center on Minority Health and Health Disparities. The Program Archive on Sexuality, Health and Adolescence (PASHA) is funded by the National Institute on Child Health and Human Development and the U.S. Office of Adolescent Health.
The authors wish to thank Dr. Carmela Lomonaco and Ms. Tamara Kuhn for their help with the manuscript.
References
- 1.National Institutesof Health (NIH,2008).Dissemination, implementation, and operational research for HIV prevention interventions (RO1). Retrieved from http://grants.nih.gov/grants/guide/pa-files/PA-08-166.html
- 2.W E Norton, K R Amico, D H Cornman, W A Fisher, J D Fisher. (2009) An agenda for advancing the science of implementation of evidence-based HIV prevention interventions. , AIDS and Behavior 13(3), 424-9.
- 3.M J Rotheram-Borus, Swendeman D, Flannery D, Rice E, D M Adamson et al. (2009) Common factors in effective HIV prevention programs. , AIDS and Behavior 13(3), 399-408.
- 4.Card J J, Benner T, Shields J P, Feinstein N. (2001) The HIV/AIDS Prevention Program Archive (HAPPA): A collection of promising prevention programs in a box. , AIDS Education and Prevention 13(1), 1-28.
- 5.M S Neumann, E D Sogolow. (2000) Replicating effective programs: HIV/AIDS prevention technology transfer. AIDS Education and Prevention, 12(Suppl A) 35-48.
- 6.Collins C, Harshbarger C, Sawyer S, Hamdallah M. (2006) . The Diffusion of Effective Behavioral Interventions project: Development, implementation, and lessons learned. AIDS Education and Prevention, 18(Suppl A) 5-20.
- 7.Cunningham S D, Card J J.(in preparation). Realities of replication: Implementation of evidence-based interventions for HIV prevention in real-world settings.
- 8.Centers forDisease Control and Prevention (CDC,2003). Advancing HIV prevention: New strategies for a changing epidemic—United States,2003. Morbidity and Mortality Weekly Report. 52(15), 329-332.
- 9.K A, Maiorana A, Vernon K, Myers J, C D Rose et al. (2007) Implementation of HIV prevention interventions with people living with HIV/AIDS in clinical settings: Challenges and lessons learned. , AIDS and Behavior 11, 17-29.
- 10.Centers forDisease Control and Prevention (CDC,2007). HIV prevention strategic plan: Extended through 2010. Atlanta, GA: Centers for Disease Control and Prevention. Retrieved from http://www.cdc.gov/hiv/resources/reports/psp/pdf/psp.pdf.
- 11.Noar S, H G Black, L B Pierce. (2009) Efficacy of computer technology-based HIV prevention interventions: A meta-analysis. , AIDS 23, 107-115.
- 12.D B Portnoy, Scott-Sheldon L A J, B T Johnson, M P Carey. (2008) Computer-delivered interventions for health promotion and behavioral risk reduction: A meta-analysis of 75 randomized controlled trials,1988–2007. , Preventive Medicine 47, 3-16.
- 13.C H Klein, Card J J. (2011) Preliminary efficacy of a computer-delivered HIV prevention intervention for African American teenage females. , AIDS Education and Prevention 23(6), 564-576.
- 14.G M Wingood, J, Er D, Solomon J, Braxton N et al. (2011) Preliminary efficacy of a computer-based HIV intervention for African-American women. , Psychology & Health 26(2), 223-234.
- 15.Card J J, Kuhn T, Solomon J, Benner T A, Wingood G et al. (2011) Translating an effective group-based HIV prevention program to a program delivered primarily by a computer: Methods and outcomes. , AIDS Education and Prevention 23(2), 159-174.
- 16.Card J J. (2001) The Sociometrics program archives: Promoting the dissemination of evidence-based practices through replication kits. , Research on Social Work Practice 11(4), 521-526.
- 17.Card J J, Lessard L, Benner T. (2007) PASHA: Facilitating the replication and use of effective adolescent pregnancy and STI/HIV prevention programs.40,275.e1-275.e14.Journal of Adolescent Health.
- 18.C H Klein, Lomonaco C, Pavelsak R, Card J J. (2013) WILLOW: Reaching HIV-positive African-American women through a computer-delivered intervention. doi: 10.1007/s10461-013-0479-z AIDS and Behavior.
- 19.Golden R E, Collins C B, Cunningham S D, Newman E N.. Card JJ.(forthcoming)Best Evidence Structural Interventions for HIV Prevention,Springer Science +Business , Media, NY .