Research and Evaluation Efforts

PSA’s research and evaluation efforts are in the following areas:

• Organizational Management
• Risk Assessment
• Supervision
• Treatment
• Forensic Analysis

Organizational Management

• Performance Improvement Center – PSA’s Performance Improvement Center is operated by the Office of Strategic Development and Office of Information Technology to provide the Agency with aggregate views and detailed reports of PRISM data. Staff are able to access the information through dashboards, pivot tables linked to the data warehouse and reports.
• PSA’s Evidence to Practice Series – In 2013, PSA’s Office of Strategic Development (OSD) began an internal seminar series designed to enhance management’s ability to use measurement and other data to improve performance and get results. Four nationally recognized experts are scheduled to speak on outcome measurement and organizational management and offer follow-up sessions to PSA management, specific offices and program areas. All PSA staff are invited to participate and external justice partners are welcome to attend.
• Annual Judicial Survey - PSA considers judicial officers in both the local and federal courts to be its primary “customers.” To help gauge the opinions of judicial officers about the Agency’s performance, PSA conducts biennial surveys of these stakeholders. The surveys address satisfaction with PSA’s responsiveness, staff professionalism, the quality and benefit of PSA reports, PSA’s supervision of higher risk defendants (including those with mental health and substance dependence issues), and the provision of treatment services. Survey results allow the Agency to assess its role, staff and quality of services.

Risk Assessment

• Validation of Risk Assessment Instrument – In FY 2009, PSA contracted with the Urban Institute and Maxarth to conduct an independent validation of its risk instrument. Study objectives included:​
  •  
  • Identification of statistically significant and relevant predictors of pretrial risk by defendants considered for pretrial release by judicial officers in D.C. Superior Court and the U.S. District Court for the District of Columbia.
  • Revision of the risk assessment to include identified risk factors appropriately weighted by their correlation to pretrial failure and proper designation of low, medium and high risk categories under appearance in court and public safety matrices.
  • A process review of current PSA risk assessment procedures to determine causes for staff overrides of the assessment tool and other potential issues in implementing the revised instrument.
  • Determination if specific risk factors are more common to higher-risk defendant populations or the commission of violent offenses while on supervision and if these factors should receive more weight in a single risk assessment or be used to create a separate risk assessment tool.

The Urban Institute and Maxarth completed their study in FY 2012. PSA Management is considering the contractors’ recommendations and expects risk assessment implementation to begin in May 2012.

Supervision

• Predictors of Pretrial Outcomes – In FY 2010, PSA contracted with Abt Associates throughout the strategic planning period to evaluate the relationship between defendant characteristics and supervision and treatment interventions to pretrial outcomes. Abt’s preliminary research found differences in failure to appear and rearrest rates by supervision type and time under supervision, but these were not statistically significant. The research also confirmed findings from local and national risk assessment studies that lengthy prior criminal histories, persistent drug use, time at residence, and lower employment levels were contributors to misconduct. Since FY 2010, PSA and Abt have refined the evaluation design and supporting research to allow Abt more flexibility in considering appropriate quasi-experimental research designs that better identify the effect of supervision suppression on pretrial misconduct.
• Random Versus Fixed Drug Testing – In FY 2009, PSA launched a pilot project on the use of random drug testing for defendants in New Directions, one of the Agency’s programs for defendants with substance disorders. The purpose of the pilot was to evaluate whether randomly drug testing defendants—rather than using a fixed testing schedule—would improve defendant compliance with drug testing conditions, increase detection of illicit drug use, increase abstinence from substance use or create efficiencies in PSA operations. Based on the encouraging findings from this pilot, as of FY 2011, random testing is required for all New Directions defendants and will be required for Drug Court defendants in FY 2012.
• Evaluation of the D.C. Superior Court Drug Intervention Program – An evaluation of the impact of two D.C. Superior Court experimental interventions on drug-involved defendants in Washington, DC, published by the National Institute of Justice in 2000.

Treatment

• Evaluation of Drug Court – In FY 2010, PSA commissioned an independent evaluation of the D.C. Superior Court’s Drug Court This study gauged the Drug Court against national guidelines established by the National Association of Drug Court Professionals.

In Phase One of the Drug Court study, the contractor:

• Conducted various interviews with Drug Court staff, program and Agency leadership, defendants and other members of the Drug Court steering committee, including judicial officers, public defenders and prosecutors;
• Administered a structured survey to the Drug Court team to better understand the design and elements of the program and protocols;
• Completed a literature review on best practices for drug courts and examined the Drug Court program in relation to the standards and research; and
• Provided final recommendations based on the assessment to PSA senior leadership and a final report.

During Phase Two, the contractor’s most critical task has been conducting a week-long training for PSA staff on the Drug Court model and providing support to PSA on the multiple recommendations to be implemented. These include providing information on how PSA can implement a wider variety of sanctions and rewards, and adding recognized best practices to Drug Court, such as regular Drug Court staffings, increasing the tenure of judges assigned to Drug Court, and assigning a small number of dedicated defense attorneys to Drug Court. As a follow-up to the week-long training, PSA is leading a sub-committee of Drug Court stakeholders to develop detailed proposals regarding implementing changes recommended by the assessment team and approved by the Drug Court Steering Committee.

• Evaluation of PSA STARS – In FY 2010, PSA commissioned an independent evaluation of PSA STARS (Support, Treatment and Addiction Recovery Services). This study consisted of a qualitative and quantitative review of PSA STARS’s effectiveness in better matching treatment to assessed need and the quality of new intensive group treatment regimens and interventions.

In Phase One of the study, the contractors have completed:

• A literature review of recognized evidence-based practices for intervening with substance abusing adult defendants;
• Individual interviews with most PSA Treatment Program staff, leadership, and external stakeholders, including D.C. Superior Court judicial officers;
• Focus groups with defendants under PSA treatment/supervision to elicit their feedback on programming and services; and
• Data analysis comparing current PSA STARS participants to defendants who participated under PSA’s former treatment protocol.
• The assessment team presented its preliminary findings to PSA leadership in September and is now completing the final Phase One report. They will begin Phase Two following discussions with PSA leadership regarding which of the team’s recommendations the Agency wants to implement.

• Focus Groups with Defendants in Sanction-Based Treatment – In August 2008, Operations and the Office of Strategic Development (OSD) began the first of on-going focus groups of defendants under sanction-based treatment in New Directions and Drug Court. These focus groups were intended to assess defendants’ impressions of PSA’s in-house treatment programs, identify treatment strategies defendants believed worked best in helping them remain drug-free, and determine the services PSA should provide to help defendants remain drug-free following treatment participation. Since the inception of focus group interviews, participating defendants consistently have rated their treatment experience with PSA as either “good” or “excellent” and have given similarly high ratings to PSA’s treatment staff. Among the elements of treatment mentioned as the most effective at encouraging future sobriety were “talk therapy” groups and interaction with PSA staff, other treatment clients and the Court.

Forensic Analysis

PSA’S Office of Forensic Toxicology Services includes the Office of Forensic Research (OFR) which conducts basic research that leads directly to practical scientific enhancements for the Agency. OFR collects and analyzes statistical data on subjects related to drug trends and drug use patterns; and distributes findings to the Agency and to the community. This often includes examining emerging drug problems, particularly those that require unique analytical approaches not routinely done by OFTS’ laboratory.