Current:Home > InvestOregon is dropping an artificial intelligence tool used in child welfare system-LoTradeCoin
Oregon is dropping an artificial intelligence tool used in child welfare system
View Date:2024-12-23 20:58:39
Child welfare officials in Oregon will stop using an algorithm to help decide which families are investigated by social workers, opting instead for a new process that officials say will make better, more racially equitable decisions.
The move comes weeks after an Associated Press review of a separate algorithmic tool in Pennsylvania that had originally inspired Oregon officials to develop their model, and was found to have flagged a disproportionate number of Black children for "mandatory" neglect investigations when it first was in place.
Oregon's Department of Human Services announced to staff via email last month that after "extensive analysis" the agency's hotline workers would stop using the algorithm at the end of June to reduce disparities concerning which families are investigated for child abuse and neglect by child protective services.
"We are committed to continuous quality improvement and equity," Lacey Andresen, the agency's deputy director, said in the May 19 email.
Jake Sunderland, a department spokesman, said the existing algorithm would "no longer be necessary," since it can't be used with the state's new screening process. He declined to provide further details about why Oregon decided to replace the algorithm and would not elaborate on any related disparities that influenced the policy change.
Hotline workers' decisions about reports of child abuse and neglect mark a critical moment in the investigations process, when social workers first decide if families should face state intervention. The stakes are high – not attending to an allegation could end with a child's death, but scrutinizing a family's life could set them up for separation.
Algorithms raise concerns about racial disparities
From California to Colorado and Pennsylvania, as child welfare agencies use or consider implementing algorithms, an AP review identified concerns about transparency, reliability and racial disparities in the use of the technology, including their potential to harden bias in the child welfare system.
U.S. Sen. Ron Wyden, an Oregon Democrat, said he had long been concerned about the algorithms used by his state's child welfare system and reached out to the department again following the AP story to ask questions about racial bias – a prevailing concern with the growing use of artificial intelligence tools in child protective services.
"Making decisions about what should happen to children and families is far too important a task to give untested algorithms," Wyden said in a statement. "I'm glad the Oregon Department of Human Services is taking the concerns I raised about racial bias seriously and is pausing the use of its screening tool."
Sunderland said Oregon child welfare officials had long been considering changing their investigations process before making the announcement last month.
He added that the state decided recently that the algorithm would be completely replaced by its new program, called the Structured Decision Making model, which aligns with many other child welfare jurisdictions across the country.
Oregon's Safety at Screening Tool was inspired by the influential Allegheny Family Screening Tool, which is named for the county surrounding Pittsburgh, and is aimed at predicting the risk that children face of winding up in foster care or being investigated in the future. It was first implemented in 2018. Social workers view the numerical risk scores the algorithm generates – the higher the number, the greater the risk – as they decide if a different social worker should go out to investigate the family.
But Oregon officials tweaked their original algorithm to only draw from internal child welfare data in calculating a family's risk, and tried to deliberately address racial bias in its design with a "fairness correction."
In response to Carnegie Mellon University researchers' findings that Allegheny County's algorithm initially flagged a disproportionate number of Black families for "mandatory" child neglect investigations, county officials called the research "hypothetical," and noted that social workers can always override the tool, which was never intended to be used on its own.
Sen. Wyden backs national oversight of technology used in the child welfare system
Wyden is a chief sponsor of a bill that seeks to establish transparency and national oversight of software, algorithms and other automated systems.
"With the livelihoods and safety of children and families at stake, technology used by the state must be equitable — and I will continue to watchdog," Wyden said.
The second tool that Oregon developed – an algorithm to help decide when foster care children can be reunified with their families – remains on hiatus as researchers rework the model. Sunderland said the pilot was paused months ago due to inadequate data but that there is "no expectation that it will be unpaused soon."
In recent years while under scrutiny by a crisis oversight board ordered by the governor, the state agency – currently preparing to hire its eighth new child welfare director in six years – considered three additional algorithms, including predictive models that sought to assess a child's risk for death and severe injury, whether children should be placed in foster care, and if so, where. Sunderland said the child welfare department never built those tools, however.
veryGood! (51)
Related
- Sister Wives’ Kody Brown Explains His Stance on His Daughter Gwendlyn Brown’s Sexuality
- John Krasinski Revealed as People's Sexiest Man Alive 2024
- Social media star squirrel euthanized after being taken from home tests negative for rabies
- Wendi McLendon-Covey talks NBC sitcom 'St. Denis Medical' and hospital humor
- Wreck of Navy destroyer USS Edsall known as 'the dancing mouse' found 80 years after sinking
- Garth Brooks wants to move his sexual assault case to federal court. How that could help the singer.
- Spirit Airlines cancels release of Q3 financial results as debt restructuring talks heat up
- Charles Hanover: A Summary of the UK Stock Market in 2023
- FSU football fires offensive, defensive coordinators, wide receivers coach
- Moana 2 Star Dwayne Johnson Shares the Empowering Message Film Sends to Young Girls
Ranking
- What is best start in NBA history? Five teams ahead of Cavaliers' 13-0 record
- Rachael Ray Details Getting Bashed Over Decision to Not Have Kids
- Duke basketball vs Kentucky live updates: Highlights, scores, updates from Champions Classic
- Pistons' Tim Hardaway Jr. leaves in wheelchair after banging head on court
- Eminem, Alanis Morissette, Sheryl Crow, N.W.A. and Janet Jackson get Songwriters Hall of Fame nods
- Travis Kelce's and Patrick Mahomes' Kansas City Houses Burglarized
- Bev Priestman fired as Canada women’s soccer coach after review of Olympic drone scandal
- Armie Hammer Says His Mom Gifted Him a Vasectomy for His 38th Birthday
Recommendation
-
Sister Wives’ Madison Brush Details Why She Went “No Contact” With Dad Kody Brown
-
Gossip Girl Actress Chanel Banks Reported Missing After Vanishing in California
-
FC Cincinnati player Marco Angulo dies at 22 after injuries from October crash
-
NFL MVP rankings: Does Steelers QB Russell Wilson deserve any consideration?
-
'I was in total shock': Woman wins $1 million after forgetting lotto ticket in her purse
-
Controversial comedian Shane Gillis announces his 'biggest tour yet'
-
Why Officials Believe a Missing Kayaker Faked His Own Death and Ran Off to Europe
-
RHOBH's Kyle Richards Addresses PK Kemsley Cheating Rumors in the Best Way Possible