Human review of AI decisions needs legal protection, says IT industry body
The right to human review of decisions made fully by computers should not be removed ‘while AI is still in its infancy’, the professional body for IT has warned.
A government consultation on personal data suggests human appeal against some automated decisions by AI (which might include recruitment or loan eligibility) could be unnecessary.
But because AI doesn’t always involve personal data to make decisions about us, true protection of our right to revisit must consider wider regulation of AI, according to BCS, The Chartered Institute for IT.
The consultation Data: A New Direction, launched by the Department for Digital, Culture, Media and Sport (DCMS), is looking to update the UK’s version of the GDPR.
DCMS is seeking further evidence before forming firm proposals on reform of the UK’s existing data legislation, including considering the removal of Article 22 of the GDPR. Article 22 focuses specifically on the right to review fully automated decisions.
Dr Sam De Silva, Chair of BCS’ Law Specialist Group and a partner at law firm CMS, explained: “Article 22 is not an easy provision to interpret and there is danger in interpreting it in isolation like many have done. “We still do need clarity on the rights someone has in the scenario where there is fully automated decision making which could have significant impact on that individual.
“We would also welcome clarity on whether Article 22(1) should be interpreted as a blanket prohibition of all automated data processing that fits the criteria or a more limited right to challenge a decision resulting from such processing.
“As the professional body for IT, BCS is not convinced that either retaining Article 22 in its current form or removing it achieves such clarity.
“We also need to consider that protection of human review of fully automated decisions is currently in a piece of legislation dealing with personal data. If no personal data is involved the protection does not apply, but the decision could still have a life-changing impact on us.
“For example, say an algorithm is created deciding whether you should get a vaccine. The data you need to enter into the system is likely to be DOB, ethnicity, and other things, but not name or anything which could identify you as the person.
“Based on the input, the decision could be that you’re not eligible for a vaccine. But any protections in the GDPR would not apply as there is no personal data.
“So, if we think the protection is important enough it should not go into the GDPR. It begs the question - do we need to regulate AI generally - and not through the “back door” via GDPR?
“It‘s welcome that government is consulting carefully before making any changes to people’s right to appeal decisions about them by algorithms and automated systems - but the technology is still in its infancy.”
BCS supports consultation
BCS, the Chartered Institute for IT supports the consultation and will be gathering views from across its membership. The consultation says that “automated decision making is likely to increase greatly in many industries in the coming years. The need to maintain a capability to provide human review may, in future, not be practicable or proportionate, and it is important to assess when this safeguard is needed and how it works in practice.”
It acknowledges that there may be “legitimate need for certain ‘high risk’ AI-derived decisions to require a human review, even if this restricts the scope of use of such systems or makes them slower.”
DCMS said it is seeking further evidence as part of the consultation before forming firm proposals on Article 22 and the right to request a review of fully automated decisions.
Latest News from
Seven steps to reset IT management17/09/2021 14:20:00
IT management must go beyond technical expertise and focus on broader organisational leadership - according to industry experts who have developed seven digital principles for those working in business technology.
Copying China's gaming ban would harm UK children, warns BCS expert09/09/2021 14:10:00
China’s limit on computer game time risks harming children’s education and should not be introduced in the UK, according to Professor Andy Phippen, a BCS Fellow and online harms expert.
AI Breast Screening needs stronger evidence before it is safe for clinical use – says professional body for IT03/09/2021 14:10:00
Better quality research is needed before artificial intelligence can be trusted to diagnose breast cancer in the full range of UK patients, according to the professional body for the IT industry.
Government’s post-Brexit global data plans must promote ethics as well as trade31/08/2021 14:10:00
The Government’s post-Brexit global data plans to boost trade and healthcare are welcome but must also incentivise ethics and a focus on climate change, according to the professional body for IT.
Computing is the fastest growing STEM A level, says professional body for IT12/08/2021 11:20:00
Record numbers of young people have been awarded Computing A-level and are choosing to study computer science at degree level, analysis by BCS, The Chartered Institute for IT reveals.
Computer Science fastest growing STEM A level, says professional body for IT11/08/2021 09:10:00
Record numbers of young people have been awarded Computer Science A-level and are choosing to study computer science at degree level, analysis by BCS, The Chartered Institute for IT reveals.
Call for governments to back IT's 'powerful role' in tackling climate change10/08/2021 14:10:00
Tech can play a leading part in achieving net-zero – if there is the global political will and funding to do so, according to the professional body for IT.
Apple's CSAM plan could pave the way for tracking 'political memes and text messages'09/08/2021 14:10:00
Apple’s plans to find child sexual abuse material (CSAM) on US customers' devices ‘fundamentally break the promise of end-to-end encryption’, according to an expert at the professional body for IT.
Britain can set 'gold standard' in ethical artificial intelligence - industry report05/08/2021 14:10:00
The UK can lead the world in creating artificial intelligence that cares about humanity - provided more people from non-tech and diverse backgrounds choose to study and work in the field, according to a new BCS report.