Risk Scoring and Algorithmic Systems in Law Enforcement: Explainability, Human Oversight and the Right to Challenge
Report D-25 examines the use of risk scoring and algorithmic systems in law enforcement and public decision-making, focusing primarily on the European Union and the United Kingdom. It asks what safeguards are required when an algorithm influences the intensity of scrutiny, the direction of an investigation, access to alternatives to prosecution or the substance of a public decision.
Its central argument is that an algorithmic score acquires legal significance not when it is calculated, but when it changes a person’s position. The entire chain must therefore be examined: the origin and quality of the data, processing and scoring, interpretation of the output, the decision of the responsible official and the possibility of effective review. High technical accuracy alone does not establish the legality of this process. ARGA_D-25_Risk_Scoring_Algorith…
The report pays particular attention to the EU AI Act. It explains that the prohibition of certain forms of individual criminal-risk prediction does not amount to a general prohibition of algorithmic tools in law enforcement. Following legislative changes in 2026, some requirements governing high-risk AI systems also have revised application dates. Existing data-protection and procedural safeguards nevertheless continue to apply independently of that timetable.
Cases involving PNR processing, SCHUFA, Dun & Bradstreet, Bridges and SyRI illustrate different legal problems: the verifiability of criteria, the actual influence of a score on a decision, meaningful explanations, discrimination risks and the quality of human oversight. Merely inserting an employee into the decision chain is insufficient if that person cannot understand the system’s reasoning, examine the underlying information or genuinely alter the outcome.
The report ultimately identifies three connected principles: facts, suspicions, predictions and decisions must remain distinct; human oversight must be substantive rather than ceremonial; and a challenge must be capable of reaching the specific element that produced the adverse consequence. A score, an explanatory text and a human signature do not by themselves establish a lawful decision if the connection between the underlying data, algorithmic inference and state action cannot be meaningfully examined.
Citation Rules
- Mandatory source attributionWhen using ARGA Observatory materials, the full name must be cited.
- Date and version indicationFor analytical reports, the year of publication is mandatory.
- Link to the originalElectronic materials must be accompanied by an active link to the official website.
- Context preservationCitations must not be shortened or altered in a way that distorts the original meaning.
- Note on adaptationIf the text is abridged or translated, state: "adapted from ARGA Observatory report".
- No commercial use without written permission from the organization
- Data accuracy preservationCharts and tables must be reproduced without changes.
Ask the ARGA experts
Four short steps. We reply within one business day.