No Risk in Trying New Approaches to Finding Children Most in Danger

In my last column, I discussed the new approaches to identify and target high-risk families for special attention in child welfare. Los Angeles and Allegheny County, Penn., as well as New Zealand are working on risk assessment algorithms. Rapid Safety Feedback (RSF), which has been implemented in Florida and is being adapted to other states, targets for special attention families with characteristics associated with high risk to children.

It is my opinion that these new approaches to protecting children have demonstrated their potential and deserve to be put into practice. In particular, the results of Project AURA in Los Angeles demonstrate that risk assessment algorithms can be a powerful tool.

As I described in my last column, AURA did an impressive job of predicting future harm to children based on child and family characteristics. Among families that had at least one previous referral, the 1 percent of referrals with the highest risk scores had 57 percent of the deaths and serious injuries.

Targeting the top 10 percent of referrals for special treatment would have identified 76 percent of the cases where a serious event would occur within the next six months. Targeting the top 20 percent would have identified 83 percent of the serious events. That is a lot of children saved from death or serious injury.

The power of AURA stems from its ability to access data from other systems and translate many factors into a single score. Among the factors that contributed most to the likelihood of a future AURA event were a previous referral within six months, parental history of narcotics treatment, age (younger) of the child, and previous substantiated allegations of abuse and neglect.

Some have expressed concerns that like medical tests, AURA produced a significant number of “false positives.” For example, among 1 percent of referrals with the highest risk scores, about 30 percent had AURA events within 6 months and the other 70 percent did not. However, some of these families had a critical event outside the 6-month period measured by AURA.

Moreover, comparing a high AURA risk score to a finding that cancer is present or absent is misleading. A high AURA score is more like a finding that one has a genetic marker for cancer and needs to be monitored. There is no harm and probably a lot of good in targeting high-risk families for extra services. Of course agencies should not remove children from their families based on a high risk score alone because that removal could be very harmful to the child and family.

The RSF tool being used for in-home cases in Florida and now being replicated in several states for CPS is a different type of instrument. Rather than calculating a risk score for each child, it targets cases with proven risk factors for special attention. The lack of child deaths in Hillsborough County in-home cases in the three years since implementation (compared to nine in the three years before) is certainly suggestive, though there is no proof that RSF is behind the drop in deaths.

But RSF has not yet been used with investigations and it has not been assessed, as was AURA, to show whether it would target the cases in which deaths or injuries would occur. This needs to be done before conclusions can be made about its effectiveness compared to AURA and similar tools.

Opponents of the new risk assessment tools argue that they might lead to more children of color being removed from their homes. But as Emily Putnam-Hornstein, one of the leading experts on child maltreatment data, noted in a forum covered by the The Chronicle, social workers making these decisions have their own biases. An objective tool should decrease bias, not increase it.

Moreover, there is a more fundamental problem with the racial profiling argument. There is strong evidence that tools like AURA can identify children who are in danger of death or injury. Opponents of the approach need to ask themselves whether they would prefer to see children of color die or face severe abuse rather than be protected at home or removed from a dangerous situation.

Surprisingly, the Commission to Eliminate Child Abuse and Neglect Fatalities (CECANF) does not appear to have heard about the encouraging results of the AURA study or another set of positive results in New Zealand. CECANF is wrapping up its deliberations and completing work on its final report, and based on remarks at the last meeting, appears to have concluded that predictive analytics is not yet ready for the big time. It would be a shame if the commission misses an opportunity to educate the public and policymakers about this promising tool.

This column was published in the Chronicle of Social Change on February 9, 2016.