Key Legal Challenges For The Development Of Defense- Oriented AI Systems In Europe Presented by: Dr. Jurić Date: 7 March 2024 How Is Europe Viewed When It Comes To The Al In Defense? - French Al strategy • "An aspiring intermediate power... whose hardline approach to legal and ethical issues may be a strength or a weakness depending on its impact" - Soare, 2023 • "European states exhibit self-imposed ethical and legal restraints, bordering on cultural-technological conservatism, which inhibits an ambitious European agenda on adopting military Al" Direct Regulation Of The Defense Al? - Al Act (in legislative procedure since 2001) - First comprehensive Al regulation in the world - Classification of Al systems into categories, based on risk level: • Unacceptable risk • High-risk • Moderate risk • Low risk Defense Al in the Al Act? - There appears to be understanding about the need to exclude defense Al from the scope of the Act - First version (2021): this Regulation shall not apply to systems developed or used exclusively for military purposes - Second version (2021): this Regulation shall not apply to Al systems developed or used exclusively for military or national security purposes - Current text (Leaked version, January 2024) • 3. This Regulation shall not apply to areas outside the scope of EU law and in any event shall not affect the competences of the Member States concerning national security, regardless of the type of entity entrusted by the Member States to carry out the tasks in relation to those competences. • This Regulation shall not apply to Al systems if and insofar placed on the market, put into service, or used with or without modification of such systems exclusively for military, defense or national security purposes, regardless of the type of entity carrying out those activities. • This Regulation shall not apply to Al systems which are not placed on the market or put into service in the Union, where the output is used in the Union exclusively for military, defense or national security purposes, regardless of the type of entity carrying out those activities. So Are There Realistically Significant Challenges In The New Al Legislation? - No for Al in lethal weapons systems - No for other Al used exclusively for defense and military purposes - Some for dual-use high-risk Al systems (remote biometric identification systems, biometric categorization, emotion recognition, critical infrastructure operation) - Some for general-purpose Al - Very low for generative Al (Al used to produce content) So Where Are The Challenges? - Al = Technology + data Al Becomes What It Feeds On… - French defense Al strategy, p. 4. • "Major digital players, especially American and Chinese,... have access to what really fuels Al: the vast mass of data that their customers provide to them free of charge at each interaction. Having initially sought to know their customers better in order to enhance their products and services, these actors are now using their very deep pockets to pursue greater ambitions, such as driverless cars, smart cities and personalized healthcare. Their products set the standard, and the sheer extent of their use cases makes them attractive to the military, especially in the many dual-use applications. As in the digital sphere as a whole, the defense sector does not necessarily blaze a trail but takes advantage of advances in civilian uses, adapting them to its own particular needs where necessary" How Is Data Regulated In The EU? - Personal Data • Regulation 2016/679 (General Data Protection Regulation) • Directive 2016/680 (Law Enforcement Directive)  Purpose limitation principle  Necessity of having legal basis for every data processing  Limitations regarding data sharing...  Strong human rights protection (constitutional, ECHR, CFEU)  Strong public attitudes about data protection • - Huge potential impact on the availability of datasets for Al systems purposes Non-Personal Data • Regulation (EU) 2018/1807 on a framework for the free flow of non-personal data in the European Union  Prohibition of data localization requirements  Self-regulatory codes for porting of data  Rules for access to data by LEAS • Relatively low / no challenges for using this data for defense Al purposes Do We Really Care About Personal Data Protection When Acting In Military And Defense Domains? - NATO Data Exploitation Framework Policy • Data exploitation should be aligned with core Alliance values, including the protection of personally identifiable information and privacy. - French defense Al strategy • There is a crucial distinction between personal data, from which a person can be identified, and nonpersonal data. The collection and exploitation of data on a massive scale cannot be envisaged without strict compliance with the prevailing personal dana legislation, especially the General Data Protection Regulation (GDPR) What Is Personal Data? - Current position of the CJEU: Personal data is any information related to a natural person whose civil identity can be established, provided that it is capable, at least, of impacting that person's rights or interests - And it is getting broader: - Some national regulators: Personal data is any information related to a natural person who can be singled out in a group of persons, provided that it is capable, at least, of impacting that person's rights or interests - CJEU C-184/20: Ordinary data which indirectly reveals information from the category of sensitive data is to be treated as sensitive. … But The Use Of Personal Data For National Security And Defense Purposes Is Outside The Scope Of EU law, right? - C-511/18 and C-512/18, La Quadrature du Net and Others • Issue (one of): whether activities of national authorities, in so far as they relate to the maintenance of public order and to the safeguarding of internal security and territorial integrity, are part of the essential functions of the Member States and, consequently, are within their exclusive competence? - CJEU: it is necessary to analyses who is carrying data processing operation: - If national authorities themselves > EU data protection rules do not apply - If private parties > EU data protection rules do apply Key Challenges Of Applying EU Legal Framework For Personal Data Protection In The Context Of Defense Al Efforts - Datasets held by privately held companies will be unavailable for national authorities, at least without additional legislation (subject to constitutional, ECHR, CFEU requirements) - It is legally challenging to use existing datasets for new purposes, including to train / operate Al systems - Generally, it is very hard to reconcile operations of Al systems with personal data protection standards - National regulators, courts and the CJEU getting very serious about the enforcement of personal data protection rules - CJEU not particularly sensitive about the needs of national authorities in national security and law enforcement matters (as seen in the data retention judgements).