Evaluation of Foster Care Agency Should Look Beyond ‘Performance Scorecard’

Accountability systems can be harmful when service providers are evaluated based on outcomes they cannot control. It is common knowledge that schools in low-income neighborhoods are often blamed for the poor performance of their students. This problem occurs as well in child welfare, where a public or private agency can be rated as poorly performing because it works with a harder-to-serve population.

On January 6, 2016, The Chronicle published my column about a critical foster home shortage in the District of Columbia. The court monitor for the child welfare system blamed the crisis in large part on the decision by D.C.’s Child and Family Services Agency (CFSA) to end a contract with a private agency (Foundations for Home and Community) that had cared for many of the oldest and most troubled youth in the system.

In my last column, I discussed the impact of Foundations’ closure on the overall number of foster homes in the District. But the closure had another serious impact—on the children who lost their homes when their foster parents quit the system. I did not know this number until CFSA sent me a long-delayed response to a Freedom of Information Act (FOIA) request I made a year ago and repeated last December.

The numbers I finally obtained were startling. The entire transition process took seven months, from November through May 2015. During that period, some young people aged out of care, attained permanency, or were able to stay in the same home because their foster parents transferred to another private agency. But many foster parents chose not to transfer, often because Foundations used private funds to pay them more than other private agencies in the District would have.

When the smoke cleared, CFSA reported that 40 youths who were previously in Foundations foster homes had moved to new placements during the process of closing the agency. That is a total of 40 young people who had to leave their homes because CFSA decided to shut down their foster care provider. While we don’t know if any of those young people might have moved anyway during the seven-month transition process, it seems fair to assume that most or all of them had to move because of the closure of Foundations.

The new document also sheds light on the reasons behind the decision to close Foundations. It explains that the decrease in CFSA’s caseload due to its emphasis on keeping families together resulted in a decreased utilization of available foster care “beds.” In order to “right-size” the system, the agency decided to close the two “lowest-performing providers.”

I consulted the “Performance Scorecards” that CFSA uses to rate its private providers. Indeed, Foundations did have the worst average score for the first three quarters of FY 2014. (Ratings for the last quarter were not available at the time the decision was made to close the agency.)

But as a veteran of two private agencies, I know that these agencies are rated on things over which they have no control. Most importantly, two indicators of permanency (percentage of youth achieving permanency within the past 12 months and those achieving ‘timely’ permanence’) account for 40 percent of an agency’s total score. Foundations did particularly badly on these ratings.

Yet, agencies that work with older kids who have been in foster care for a long time have little control over permanency outcomes for their clients. Foundations took in the hardest-to-serve youth. Many of these youths had been in the system for years, and all of their family connections had been exhausted. Their age and behavioral problems made them difficult to place. Expecting the agency to miraculously find permanent homes for them was not realistic.

When they made the decision to close Foundations, CFSA leaders may not have realized that it would result in the loss of homes for 40 foster youth and a citywide shortage of foster homes. But as I described in my last column, CFSA knew there was a problem as early as January 2015, two months into the seven-month transition.

Once agency leaders knew that many foster parents from Foundations were leaving the system, they had plenty of time to reverse their decision. Other organizations, like the agency charged with monitoring CFSA for the court, the City Council, and advocacy groups also had advance warnings but did not speak out.

This tale has national implications. First, child welfare professionals should beware of benchmarks that penalize agencies for factors over which they have no control. Second, legislators, advocates, and reporters need to be much more aware of what child welfare executives are doing. They should not rely on rosy reports from leaders who fear the career implications of admitting mistakes.

This column was published by the Chronicle of Social Change on January 20, 2016.