I would like to help today and donate

Next
Thank you! Your submission has been received!
Oops! Something went wrong while submitting the form.
Thank you! Your submission has been received!
Oops! Something went wrong while submitting the form.
Next
Thank you! Your submission has been received!
Oops! Something went wrong while submitting the form.
Thank you! Your submission has been received!
Oops! Something went wrong while submitting the form.
CLOSE
NEWS

EU AI legislation threatens fight against discrimination and injustice

Fair Trials is calling on the European Union to radically improve its new legislation on artificial intelligence to include meaningful safeguards to protect against discrimination, prevent uses of AI which undermine the presumption of innocence, and address significant loopholes which render any safeguards meaningless. 

Griff Ferris, Legal and Policy Officer said:  

“The EU’s new AI legislation needs radical changes to prevent the hard-wiring of discrimination in criminal justice outcomes, protect the presumption of innocence and ensure meaningful accountability for AI in criminal justice. 

“The new legislation lacks any safeguards against discrimination, while the wide-ranging exemption for ‘safeguarding public security’ completely undercuts what little safeguards there are in relation to criminal justice. 

“The framework must include rigorous safeguards and restrictions to prevent discrimination and protect the right to a fair trial, which includes restricting the use of systems that attempt to profile people and predict the risk of criminality." 

Fair Trials has serious concerns about the use of AI and automated decision-making systems in law enforcement and criminal justice. AI systems are used by law enforcement and criminal justice authorities to predict and profile people’s actions in order to justify surveillance, stop and search, questioning, arrest and detention, as well as other non-criminal justice punishments such as the denial of welfare, housing, education or other essential services. In doing so, these systems engage and infringe fundamental rights, including the right to a fair trial, privacy and data protection rights, as well as discrimination based on race, socio-economic status or class and nationality. 

We welcome the fact that the EU is taking a much-needed legislative approach to regulate and limit the use of AI, and even more so that it recognises that the use of AI in criminal justice is “high-risk”. However, this legislation does not go nearly far enough to prevent certain fundamentally harmful uses and provide strict, mandatory safeguards, which will have damaging consequences across Europe for a generation.  

There are no protections in the proposed legislation against the significant threat of discrimination in AI and automated decision-making systems. 

Fair Trials is calling for significant improvements to the legislation to: 

  • Restrict the use of AI and automated decision-making systems to predict, profile and assess people’s ‘risk’ of criminality, generate reasonable suspicion and justify law enforcement action, arrest and pre-trial detention. The use of these systems undermines the presumption of innocence and must not be allowed. 

  • Make rigorous bias testing mandatory for all AI and automated decision-making systems used in criminal justice, and no system should be allowed to operate without it. The criminal justice data used to create, train and operate these systems is reflective of systemic, institutional and societal biases which result in Black people, Roma and other ethnic minorities being aggressively overpoliced, detained and imprisoned across Europe. As a result, these systems have been shown to directly generate and reinforce biased and discriminatory criminal outcomes. These issues are so fundamental that it is questionable whether any such system would not produce such outcomes, but a rigorous testing regime is the bare minimum required. 

  • Ensure that individuals can challenge criminal justice decisions that are assisted by AI. AI and automated decision-making systems should be open to public scrutiny in the same way that all decision-making processes by public entities should be. Commercial interests and technological design should never be a barrier to transparency and any criminal justice decisions assisted or influenced by AI or automated decision-making systems must be fully open to meaningful scrutiny and challenge by any individual subject to it.  

If you are a journalist interested in this story, please call the media team on +44 (0) 7749 785 932 or email [email protected]

Keep up to date

Receive updates on our work and news about Fair Trials globally

Some activities in the following sections on this website are funded by the European Union’s Justice Programme (2014-2020): Legal Experts Advisory Panel, Defence Rights Map, Case Law Database, Advice Guides, Resources, Campaigns, Publications, News and Events. This content represents the views of the authors only and is their sole responsibility. It cannot be considered to reflect the views of the European Commission or any other body of the European Union. The European Commission does not accept any responsibility for use that may be made of the information it contains.