The European Commission is indeed heading towards new obligations for high-risk artificial intelligence systems. According to a 21 January version of the draft white paper on artificial intelligence (AI), read by EUROPE, the Commission abandons its idea of a temporary moratorium on facial recognition systems.
A framework that is “not overly prescriptive”
The new draft is inspired by the December version, which had already been unveiled by the press (see EUROPE 12406/8). This time, the Commission no longer discusses specific obligations for public administrations or a moratorium on facial recognition systems. It does, however, look further into imposing mandatory criteria for high-risk applications, which it defines as applications that are on a list of key sectors (health, transport, police, and law) and whose use poses a high risk (see EUROPE 12413/6).
“The horizontal framework for AI should not be overly prescriptive so as not to create a disproportionate administrative burden, especially for SMEs”, now adds the text, which focuses on funding needs for research and development as well as safeguards for high-risk applications.
The final text will be published on 19 February along with a data strategy and a report on the broader implications of AI, the Internet of Things, and other digital technologies for the framework as regards security and accountability. It is to be used as a basis for public consultation.
A focus on high-risk applications
In the 20 pages of text, the Commission emphasises that it is not necessary to revolutionise European rules in light of the development of artificial intelligence. However, the Commission thinks that some adaptations could prove to be useful, while there should be specific obligations for high-risk applications.
These criteria—which will have to be respected by professional AI users and developers in every country in the world—could address data, traceability, and human control through a compliance procedure controlled by public authorities in the EU via the following options: data quality and traceability obligations, transparency and human control obligations, and specific criteria for remote biometric identification systems.
These criteria should be used for ex ante and ex post assessments according to the applications’ risk level. (For high-risk applications, the algorithm and data must undergo ex ante compliance assessments through independent audits and assessments by test centres.) In cases where an ex ante conformity assessment is not possible, the Commission envisages an obligation to test algorithms in Europe. For lower-risk applications, the Commission considers that a simple ex post assessment could suffice after an alleged breach or an abuse.
Facial recognition no longer prohibited
In the December draft, the European Commission was considering banning facial recognition systems for 3 to 5 years, while suggesting that this may not necessarily be the most suitable option. The text, dated 21 January, no longer refers to a moratorium at all. It emphasises that the implications for fundamental rights vary considerably depending on the objective, context, and scope and that guidelines should be established in order to define processing. “In the case of mass surveillance, there must be clear criteria by which individuals must be identified”, the text indicates, later adding that—depending on system use and risk level—there must be certain criteria for accuracy.
It should be noted that the Commission is not abandoning the idea of a voluntary labelling scheme for ethical artificial intelligence as an alternative or supplement to the mandatory criteria. The same applies to governance, since the Commission also suggests a European governance structure based on national authorities. Furthermore, it envisages a standing committee of experts that would provide assistance to the Commission. (Original version in French by Sophie Petitjean)