Skip to content

Improving Human-AI Partnerships in Child Welfare: Understanding Worker Practices, Disagreement, and Desires for Algorithmic Decision Support (A Conversation with Haiyi Zhu & Toby Li)

    Join us for a CHIWORK conversation on February 16 at 11am EST. Click here: Download Calendar Invite

    AI-based decision support tools are increasingly used to augment human decision-making in high-stakes, social contexts. It is critical that we understand the frontline workers’ experiences with these AI-based tools in practice and the impacts of adopting these tools. We studied AFST (Allegheny Family Screening Tool), the pioneering AI-based decision support tool designed to assess a family’s risk level when they are reported for child welfare concerns in Allegheny County. I worked with my collaborators to conduct a series of interviews and contextual inquiries at a child welfare agency, as well as data analysis of child welfare call screen workers’ decision-making over four years. Our studies showed patterns of when, whether, and how much the frontline workers decide to rely upon algorithmic recommendations. Also, we found that from 2016 to 2018, the algorithm (AFST) recommendations had 20% racial disparity because the algorithm on its own would’ve investigated 71% of Black children and 51% of white children. Over that same time period, the workers reduced the disparity in screen-in rate between Black and white children from 20% to 9%, by disagreeing and overring the algorithmic recommendations. Our qualitative data show that workers achieved this by making holistic risk assessments and adjusting for the algorithm’s limitations. Our analyses also show more nuanced results about how human-algorithm collaboration affects prediction accuracy, and how to measure these effects. These results shed light on potential mechanisms for improving human-algorithm collaboration in human service decision-making contexts.

    Speaker: Haiyi Zhu

    Associate Professor at Carnegie Mellon University

    Haiyi Zhu is the Daniel P. Siewiorek Associate Professor of Human-Computer Interaction and the Director of HCI Undergraduate Programs at Carnegie Mellon University. Haiyi received a B.S in Computer Science from Tsinghua University and an M.S. and a Ph.D. in Human-Computer Interaction from Carnegie Mellon University. Haiyi has received multiple NSF awards (CRII, Cyber Human System, EAGER on AI and Society, Fairness in AI, and Smart and Connected Communities), several paper awards in venues such as CHI, CSCW, and Human Factors, and an Allen Newell Award for Research Excellence.

    Conversation Details: