Preventive vs. Remedial Screening Tools
The Economic Roundtable has developed five predictive screening tools to identify and prioritize high-need homeless individuals who will have high future public costs because of ongoing crises in their lives that are resolved in expensive institutional settings, including jails and hospitals. Two of the tools are preventive – they identify people on the path to persistent homelessness. Three of the tools are remedial – they identify people who are already harmed by homelessness and will be frequent users of public services in the future. These tools are in the public domain and can be downloaded here.
Individuals experiencing homelessness bear an immense cost, as do public systems that have limited resources to meet growing demands. However, the need for services is not evenly spread among individuals. For example, in Santa Clara County, California, we found that five percent of homeless people account for 47 percent of all public spending on the homeless. These high-need individuals face multiple obstacles to being housed, including unreliable low-wage jobs, physical and mental health problems, criminal justice system involvement, and social isolation.
Many separate pieces of information provide evidence about someone’s likely future homeless trajectory, but no single piece of evidence by itself provides enough information to make this prediction with certainty. The five predictive screening tools that can be downloaded here address this problem by combining the predictive power of many different pieces of information to produce an estimate of the likely course of homelessness for an individual.
Each of the five tools identifies high-need individuals for whom the escape route from homelessness costs less than the problem of remaining homeless. All of the tools predict what will happen in the future rather than simply describing what has happened in the past.
A model with high predictive accuracy yields high numbers of true positives (high-need individuals in the target population who are correctly identified) and true negatives (individuals who do not have high-needs and are not in the target population) and low numbers of false negatives and false positives.
The biggest job the triage tools do is to correctly exclude most people with low costs. After that, they distinguish subtle differences between people with significant problems and a record of using public services to predict who will have high risks in the future.
One of the challenges the screening tools must contend with is abrupt changes in homeless status, health conditions and service utilization from the years preceding the screening to the years following the screening. Some health, mental health and justice system occurrences are one-time events, resulting in risks that spike and then decline. On the other hand, if individuals remain homeless, problems tend to worsen over time.
The high predictive accuracy of the five screening tools is demonstrated by the fact that they make correct predictions an average of 92 percent of the time. That is, true positives and true negatives make up an 92 percent of the screening predictions. Most false positive predictions are for individuals whose needs and risks are close to the threshold for inclusion in the high-need population targeted by each tool. These are close misses and flag individuals who also have urgent needs for help.
Two Preventive Tools for Identifying Individuals on the Path to Persistent Homelessness
The first type of tool is for early identification and quick, effective preventive intervention for young adults and workers who lose their jobs who are likely to become persistently homeless. These tools are for preventing the social disconnection and legal, medical and behavioral health problems that proliferate after people are persistently homelessness. Comprehensive support leading to employment in living-wage jobs is a key preventive intervention for these individuals after they are identified.
Both tools were developed using a database linked agency records of more than one million Los Angeles County public assistance recipients who experienced homelessness. The unemployment tool is for workers who have recently lost their job. The young adult model is for youth 18 to 24 years of age. Both models can be used for unemployed young adults.
The traumas experienced by high-need individuals in both populations have strong predictive power. Out of the 32 variables used in the employment model, factors that have the greatest predictive power include histories of homelessness, incarceration, unemployment, or substance abuse, ethnicity, and household structure.
Out of the 22 variables used in the young adult model, factors that have the greatest predictive power include histories of foster care, homelessness, incarceration, mental illness, or substance abuse, and ethnicity.
The two models have similar predictive accuracy. Outcomes from these two preventive tools are shown in the accompanying chart.
Three Remedial Triage Tools for Identifying Homeless Individuals who will be in the 10th Cost Decile in the Coming Year
The second type are triage tools for identifying homeless individuals who will be in the 10th cost decile in the coming year. Because of the level of distress in their lives as well as their high public costs, they should have first priority for permanent supportive housing. These are remedial tools – they identify individuals who are already suffering great harm from homelessness.
Outcomes from the three remedial triage tools for identifying homeless individuals whose future costs will be in the 10th decile are shown in the accompanying chart. These outcomes are the average of results from separate tests with homeless records from both Santa Clara and Los Angeles Counties.
The first tool was developed using data about encounters with multiple public agencies in Los Angeles County, the second tool was developed using similar data from Santa Clara County (Silicon Valley), and the third tool was developed using only hospital data from Los Angeles County.
All three tools were tested twice, first using homeless records from Santa Clara County, and second using homeless records from Los Angeles County. The tool developed using multi-agency data from Los Angeles County is the most accurate for identifying individuals who will have 10th decile costs in the future, followed by the Silicon Valley tool, and then by the Los Angeles County hospital tool.
All three of the remedial tools performed well using homeless records from both counties. This indicates that the triage tools will have strong predictive performance in other metropolitan regions.
Trade-Offs in Using Predictive Screening Tools
The first objective in using the tools is to identify as many homeless persons as possible who will be persistently homeless in the future, and in the case of the triage tools, also have high public costs. These are true positive screening outcomes. The second objective is to avoid decreasing potential program effectiveness by including homeless persons who will not be persistently homeless or have high costs in the future. These are true negative screening outcomes.
The five screening tools offer different trade-offs between shortfall and burden, with the LA Hospital tool, which uses only medical data, offering the least advantages trade-offs. Three-quarters of the target 10th decile population is screened out by this tool (2.6% true positives vs. 7.4% false negatives). The advantage of this tool is that it enables screening in hospitals where only medical data is available.
The trade-offs can also be managed by changing the probability cut-off score for inclusion in the target population. For each individual who is screened, the screening tools produce a predicted probability of being in the target population that can range from 0% to 100%.
Cut-off scores are typically set at the level where targeted individuals in the population being screened have an equal likelihood of being above the cut-off score as below the cut-off. The false positives that accompany this cut-off point are an acceptable burden. However, if the program carrying out the screening has only a small number of slots for participants, a higher probability score will reduce the burden of false positive outcomes, and the increased shortfall from more false negative outcomes will not result in leaving program slots unfilled. The reports that explain each screening tool provide detailed explanations of these trade-offs.
Information about each screening tool and the tools themselves can be downloaded below.
Early Intervention Preventitive Screening Tools
- Webpage for Preventive Screening Tools
- Report Explaining Tools: Early Intervention to Prevent Persistent Homelessness
- Preventive Screening Tool: High-Risk Unemployed Workers
- Preventive Screening Tool: High-Risk Young Adults
10th Cost Decile Triage Tools
- Webpage for Silicon Valley Triage Tool
- Report Explaining Tool: Silicon Valley Triage Tool: Identifying and Housing High-Cost Homeless Residents
- Triage Screening Tool Built on Multi-Agency Data from Santa Clara County: Silicon Valley Triage Screening Tool
- Webpage for Los Angeles Triage Tool
- Report Explaining Tool: Crisis Indicator: Triage Tool for Identifying Homeless Adults in Crisis
- Report on Outcomes from 15 Hospitals Using the Screening Tool: Getting Home: Outcomes from Housing High Cost Homeless Hospital Patients
- Triage Screening Tool Built on Los Angeles County Multi-Agency Data: 10th Decile Triage Screening Tool
- Webpage for Hospital-Based Triage Tool
- Report Explaining Tool: Hospital to Home Triage Tool for Identifying 10th Decile Homeless Hospital Patients
- Triage Screening Tool Built on Los Angeles County Health System Data: 10th Decile Screening Tool Based on Medical Records
