The future of voluntary CSAM detection in the EU – a primer

As we speak, debates are raging in the European Union on a new legal framework for the detection and prevention of child sexual abuse materials (CSAM). These debates are particularly intense with respect to certain online communication services, known as number-independent interpersonal communications services (ICS), such as WhatsApp, Facebook Messenger, and WeChat. This blogpost will focus on the current legal framework for CSAM detection in the EU, and will introduce a White Paper which was drafted by the authors which tackles the question of how voluntary CSAM detection by ICS can be introduced in the CSAM Regulation Proposal.  

CSAM, EU data protection law and voluntary detection 

Interpersonal communications services are subject not only to the EU’s well known General Data Protection Regulation (GDPR), but also to the more specific telecommunications confidentiality provisions of the ePrivacy Directive. As a result, the detection and prevention of CSAM via automated tools by these ICS on their services requires a specific legal mandate that satisfies the requirements of both the GDPR and the ePrivacy Directive. 

Some of the providers of ICS already engage in voluntary detection of CSAM on their services. To do so in compliance with EU data protection law, they currently rely on a provisional regime that was created in 2021, known as the Interim Derogation. This Derogation creates a temporary legal mandate for ICS providers which permits them to implement certain detection technologies to detect, remove and report CSAM on their services, subject to specific safeguards, as an exception to the general confidentiality rules of the ePrivacy Directive. This arrangement has been critical in allowing ICS providers to voluntarily assume responsibility for the fight against CSAM. However, since its inception, the Interim Derogation has faced heavy criticism of privacy organizations in general and both the EDPS and EDPB. These stakeholders strongly voiced concerns with regards to the legitimacy and the proportionality of the Interim Derogation in light of the case law of the Court of Justice regarding an interference with fundamental rights.  

Elimination of a legal mandate for voluntary detection in the CSAM Regulation Proposal  

The Interim Derogation was inherently designed as a temporary solution. It contains a so-called ‘sunset clause’, that would trigger its automatic expiration after 3 August 2024, with the expectation that a new and permanent framework would be established prior to that date.  

It is in this context that the European Commission proposed in May 2022 a Regulation laying down rules to prevent and combat child sexual abuse (the CSAM Regulation Proposal). The proposal however doesn’t focus on creating a new (and permanent) mandate for voluntary action; rather, it focuses mainly on mandatory detection orders for certain service providers issued by competent authorities. In other words, providers of the services covered by the Regulation (which includes the aforementioned ICS, but also hosting services) could be ordered by a competent authority to implement technologies to detect and block or report CSAM.  

The CSAM Regulation Proposal is however remarkably discrete on the possibility of voluntary detection: the Interim Derogation that allows it is still anticipated to expire (although there is an agreement now to extend its lifetime until April 2026, thus buying a bit of extra time to continue the political discussions), and no clear replacement for voluntary detection is included in the current text of the proposed CSAM Regulation.  

A recurring justification for the elimination of a mandate for voluntary detection, is that this constitutes such a powerful intrusion into the fundamental rights to privacy and data protection of the users, that this type of intrusion can only be justified by an order from a competent body, mandated under specific legislation. Critics of the voluntary detection regime would voice that a conditional mandate for ICS providers to act voluntarily under specific safeguards would not be conceptually permissible as a matter of EU law.  

A critical look at the viability of voluntary detection in a Timelex White Paper on the CSAM Regulation 

Legal framework and existing case law 

Against this backdrop, Timelex has drafted a White Paper, funded by Microsoft but with full intellectual and editorial freedom for the authors, to critically assesses the scepticism against the legal viability of voluntary detection. The starting point of the White Paper is an analysis of the current EU legal framework (mainly the Charter of Fundamental Rights, the GDPR and the ePrivacy Directive) and prior case law from the Court of Justice of the European Union (CJEU) on an interference of fundamental rights. The analysis of this case law was of paramount importance in the context of the White Paper as any legislation that allows ICSs to scan their services on the occurrence of CSAM and to report it to law enforcement authorities clearly has a significant impact on the right to private life (including private communications) and the right to protection of personal data.  

The authors find that the legal framework and the existing case law is significantly more nuanced than is often assumed in discussions. The CJEU consistently emphasizes the importance of context when assessing the lawfulness, legitimacy and proportionality of intrusions into fundamental rights. It has, in prior instances, accepted even large scale automated assessments of personal data, provided that these were coupled with clearly defined and effective safeguards that appropriately mitigate potential negative impacts on the persons concerned. This is examined in the paper via prior decisions on Passenger Name Records, the Data Retention Directive, and the Quadrature du Net cases specifically.  

Proposed safeguards to mitigate potential negative impacts on the persons concerned 

In the context of CSAM detection and prevention, the authors are of the opinion that a mandate for voluntary detection could be created that incorporates a broad range of safeguards, including a prior risk assessment of the ICS: 

• to appropriately determine the CSAM risks (taking into account also the user community and the functional and technical characteristics of the service), and  
• to assess the impact, risks and effectiveness of specific detection tools (since e.g. a hash based detection of known CSAM has virtually no false positives, unlike e.g. AI based identification of unknown CSAM, the latter therefore requiring more safeguards).  

Other viable safeguards include: 

• human intervention prior to taking further action (by, for example, obliging ICSs to have in-house trained analysts),  
• better transparency obligations towards users so that they know whether and how their services are monitored,  
• stratified and proportionate response mechanisms that consider the distinctions between e.g. adult offenders mutually exchanging CSAM and teenagers exchanging images of themselves.  

Finally, mature and effective governance measures are needed, such as: 

• prior authorisations by competent authorities in some instances, and  
• post hoc verifications by competent authorities in case of incidents or changes in the ICS, or in the detection technologies used.  

Overall conclusions of the White Paper 

The White Paper concludes that the continued existence of a mandate for voluntary detection and prevention of CSAM by ICS providers is legally feasible in accordance with EU law, by ensuring that these measures are targeted as a result of an ICS-specific risk assessment, and by building in a range of additional safeguards.  

The White Paper does not argue against the introduction of detection orders under the CSAM Regulation, since these orders can indeed play a decisive role in compelling negligent, passive or unaware ICS providers to act appropriately. Nor does the White Paper argue in favour of an unbounded and open mandate for voluntary detection. To the contrary, it notes that the safeguards that currently exist in the Interim Derogation and the ones that are included in the CSAM Regulation Proposal for the detection orders could be strengthened and improved.  

However, the White Paper does present the finding that voluntary CSAM detection and prevention can act as a necessary complement to detection orders, and should be given a clear and unambiguous fiat under the CSAM Regulation, where a prior risk assessment justifies it and certain safeguards are met. This can be done in a co-regulatory model, where the ICSs are allowed to assume responsibility, while building a governance framework that ensures lawfulness, legitimacy and proportionality. In this manner, the Derogation can more effectively contribute to the fight against CSAM, in a manner that aligns with European requirements in relation to the fundamental rights to privacy and data protection.  

For more details, feel free to consult our White Paper, or to contact the authors, Hans Graux and Jolien Clemens.