Debunking the EU AI Act

Debunking the EU AI Act: An Overview Of The New Legal Framework Date: 31 January 2024 Agenda - State of the Al Act - Scope of the Al Act - Classification of Al systems - Key requirements and prohibitions - General Purpose Al Models - What can you start doing today? - Annexes Al Act - Entry Into Application - Co-legislators reached a political agreement on 9 December 2023 - Q1 2024 • Final adoption in the EU Parliament & the Council • Publication in the EU Official Journal + 20 days = entry into force - EU Parliament elections in June 2024 - Q3 2024 • - Titles on Prohibition apply six months after entry into force Q1 2025 • Title on GPAI (and some others) apply from twelve months following into entry into force - Q1 2026 • - General applicability 24 months after entry into force Q1 2027 • Art. 61 (High-Risk Al systems under Annex II) applies 36 months after entry into force When does the Al Act come into application? - 36 months (2027) • High risk Al systems used as a safety component of a product covered by EU harmonization legislation - 24 months (2026) • - - General application 12 months (2025) • Title III, Chapter 4 (Notifying authorities and notified bodies) • Title VI (EU AI Board / National regulators) • Title VIII A (GPAI) • Title X (Penalties) 6 months (2024) • Title I (General provisions) • Title II (Prohibited Al practices Taxonomy of actors under the Al Act - Providers - Deployers (i.e. users) - Importers - Authorized Representatives (for non-FU providers) - Distributors - Affected person - Operators What activities are caught under the Al Act? - Providers • Who place on the EU market or put into service Al systems, or place general purpose Al models in the EU • Applies to providers of Al systems who have their place of establishment or are located outside the EU but where the output produced by the Al system is used in the EU - Deployers • Who have their place of establishment or are located in the EU • Applies to deployers of Al systems who have their place of establishment or are located outside the EU but where the output produced by the Al system is used in the EU - Importers/distributors • - Who make an Al system available on the EU market Product manufacturers • Who place or put into service an Al system in the EU with their product and under their name or trademark Exclusions - Al used for military, defense or national security purposes - Research, testing and development of Al systems or models - Al used by public authorities outside the EU and international organizations (where international agreements are in place) - Individuals using Al as part of a personal, non-professional activity - Al systems and models developed for scientific research & development - Al systems released under free and open source licenses (excluding prohibited Al, highrisk Al, and GPAI models under certain conditions) Definition of an Al System 'Al System' means a machine-based system that is: - Designed to operate with varying levels of autonomy and that may exhibit adaptiveness after deployment, and - That for explicit or implicit objectives, - Infers from the input it receives how to generate outputs such as predictions, content, recommendations, or decisions, - That can influence physical or virtual environments. Classification of Al systems (incl. GPAI models) - Risk-based and cross-sector use case approach - Classification of Al systems • Prohibited Al systems  Subliminal techniques beyond a person's consciousness to distort a person's behavior in a manner that causes harm  Exploit vulnerabilities (age, physical or mental disability).  Social Scoring and biometric categorization  Real time remote biometric identification systems by law enforcement in publicly accessible spaces • High-risk Al systems  Employment, Law Enforcement, Migration, Administration of Justice, etc.  subject to specific requirements  Risk of harm on people's health, safety and fundamental rights • GPAI models & generative Al  e.g. Large Language Models • Limited Al systems  Al systems with specific transparency requirements  e.g. chatbots, emotion recognition systems or deep fakes  allow people to make informed choices or withdraw from a given situation • Unregulated Al systems, e.g. Al-enabled  video games  spam filters - Risk management and compliance obligations along the different classifications (e.g. Al impact assessment) High risk Al Systems classification rules - Category One • Al systems that are a product or safety component of a product that is subject to EU sectoral legislation on product legislation (Annex II); and • - are required to undergo a third party conformity assessment Category Two • Al Systems that pose a significant risk of harm to the health, safety or fundamental rights of natural persons (Annex III) • BUT: Al systems will not be considered high-risk (incl. the ones listed in Annex III) if they do not pose a significant risk of harm to the health, safety or fundamental rights of natural persons, including by not materially influencing the outcome of decision making (Fundamental Rights Impact Assessment) - An Al system shall always be considered high-risk if the system is intended to perform profiling of individuals Requirements for High-Risk Al Systems – Overview - Assessment and Registration - Data Governance and Security - Risk Management and Human Oversight - Technical Documentation, Record Keeping and Transparency General purpose Al systems, models and Generative Al - General purpose Al • Al system that can be used in and adapted to a wide range of applications for which it was not intentionally and specifically designed - General purpose Al model (Foundation model) • Al system model that is trained on broad data at scale, is designed for generality of output, and can be adapted to a wide range of distinctive tasks. - Generative Al • Al systems specifically intended to generate, with varying levels of autonomy, content such as complex text, images, audio or video Compliance Obligations for Generative Al Generative Al: Al systems specifically intended to generate, with varying levels of autonomy, content such as complex text, images, audio or video - Ensure outputs are marked in a machine-readable format and detectable as artificially generated or manipulated - Ensure technical solutions are effective, interoperable, robust and reliable, taking into account specificities and limitations of different types of content, costs of implementation and the generally acknowledged state-of-the-art - Deployers of an Al system that generates or manipulates image, audio or video content constituting a deep fake, shall disclose that the content has been artificially generated or manipulated. - Deployers of an Al system that generates or manipulates text which is published with the purpose of informing the public on matters of public interest shall disclose that the text has been artificially generated or manipulated. General purpose Al models - a two tiered approach - Few obligations if general purpose Al model does not come with a systemic risk - Higher obligations for general purpose Al models with systemic risk - Classification of systemic risk: • High impact capabilities evaluated on the basis of appropriate technical tools and methodologies, including indicators and benchmarks • Based on a decision of the Commission, ex officio or following a qualified alert by the scientific panel that a general purpose Al model has capabilities or impact equivalent to those • When the cumulative amount of compute used for its training measured in floating point operations (FLOPs) is greater than 10^25 • European Commission shall adopt delegated acts to amend these thresholds General purpose Al models - compliance obligations if no systemic risk - Put in place a policy to respect Union copyright law in particular to identify and respect, including through state of the art technologies, the reservations of rights expressed in the exceptions for text and data mining - Technical documentation including training and testing process and the results of its evaluation; make available information and documentation to providers of Al systems - Draw up and make publicly available a sufficiently detailed summary about the content used for training of the general-purpose Al model, according to a template provided by the Al Office - Prior to placing a general purpose Al model on the Union market providers established outside the Union shall appoint an authorized representative which is established in the Union and shall enable it to perform its tasks under the Al Act - Technical documentation and making information available does not apply to providers of Al models that are made accessible to the public under a free and open license that allows for the access, usage, modification, and distribution of the model, and whose parameters, including the weights, the information on the model architecture, and the information on model usage, are made publicly available. - Perform model evaluation in accordance with standardized protocols and tools reflecting the state of the art - Assess and mitigate possible systemic risks at Union level, including their sources, that may stem from the development, placing on the market, or use of general purpose Al models with systemic risk - Keep track of, document and report without undue delay to the Al Office and, as appropriate, to national competent authorities, relevant information about serious incidents and possible corrective measures to address them - Ensure an adequate level of cybersecurity protection for the general purpose Al model with systemic risk and the physical infrastructure of the model Fines - €35 million or 7% of annual worldwide turnover • - Placing a Prohibited Al System on the market. €15 million or 3% of annual worldwide turnover • Most breaches relating to High-Risk Al systems, GPAI and foundation models • - Obligations imposed on providers, importers, distributors and deployers €7,5 million or 1% of annual worldwide turnover • Supply of incorrect, incomplete or misleading information to the regulators in reply to a request Al enforcement - - EU bodies • Al Office (EU Commission) • Al Board • Scientific Panel • Advisory Forum National competent regulators • Left to Member States to decide • Risk of multiple and competing regulators at a national level • Some DPAs have already positioned themselves as the competent regulator (e.g. CNIL) • Spain has already established its Al regulatory body Practical next steps - What are the immediate risks to your organization? - How will you use AI? - Who will lead and contribute to Al governance in your organization? - What internal and customer-facing materials will help you balance innovation, sales and risk? What can you start doing today? - Al mapping and risk-scaling - Al governance process and risk assessment - Increase Al literacy within company (i.e. training, upskilling) - Al ethical guidelines for employees (esp. with respect to genAl) Prohibited Artificial Intelligence Practices - Subliminal Techniques • (distorting people's behaviors by impairing the ability for informed decision, in a manner that causes/ is likely to cause significant harm) - Exploiting Vulnerabilities • (due to age, disability or specific social or economic situations with the objective or effect of distorting behaviors where this causes or is reasonably likely to cause significant harm) - Biometric categorization systems • (categorization of individuals based on their biometric data to deduce or infer their race, political opinions, trade union membership, religious or philosophical beliefs, sex life or sexual orientation) - Social Scoring • (evaluation or classification of individuals based on their social behaviors with a social score leading to detrimental or unfavorable treatment) - Real-time remote biometric identification systems in public spaces • - Crime risk prediction • - (except in certain circumstances by law enforcement authorities) (where based on profiling or assessing personal traits and characteristics) Facial recognition databases • (based on the untargeted scraping of facial images from the internet or CCTV footage) - Emotion detection tools • (in the areas of workplace and education institutions except where used for medical or safety reasons) Annex II - Harmonisation Legislation Covers a mix of EU product and safety legislation applying to: - Machinery - Radio equipment - Pressure Equipment - Toys - Cableway installations - Recreational craft and personal watercraft - Personal protective equipment - Agricultural and forestry vehicles* - Lifts - Appliances burning gaseous fuels - Marine equipment* - Equipment and protective systems for use in potentially explosive atmospheres - Medical devices - In vitro diagnostic medical devices - Motor vehicles and trailers* - Rail system interoperability* - Civil aviation* - Two or three wheel vehicles and quadricycles* * Only limited requirements relating to Commission assessment and Al sandboxes apply. Annex III - High Risk Al Systems - Biometrics • Remote biometric identification systems • Biometric categorization according to sensitive or protected attributes or characteristics • - Al systems used for emotion recognition Critical infrastructure • Al systems intended to be used as safety components in the management and operation of critical digital infrastructure, road traffic and the supply of water, gas, heating and electricity - Education and vocational training • Al used to determine access or admission or to assign natural persons to educational and vocational training institutions at all levels. • Al used to evaluate learning outcomes • Al used to assess the appropriate level of education that individuals will receive or will be able to access • Al used for monitoring and detecting prohibited behaviors of students during tests - Employment, workers management, and access to self-employment • Al used for recruitment or selection of natural persons, to place targeted job advertisements, to analyze and filter job applications, and to evaluate candidates • Al used to make decisions affecting terms of the work related relationships, promotion and termination of work-related contractual relationships, to allocate tasks based on individual behavior or personal traits or characteristics and to monitor and evaluate performance and behavior of persons in such relationships - Access to and enjoyment of essential private services and essential public services and benefits • to evaluate the eligibility of individuals for essential public assistance benefits and services (including healthcare services) to evaluate the creditworthiness of natural persons or establish their credit score (except to detect financial fraud) • to evaluate and classify emergency calls or to dispatch emergency first response service (including by police, firefighters and medical aid) • for risk assessment and pricing in relation to natural persons in the case of life and health insurance - Law enforcement • to assess the risk of an individual becoming a victim of a criminal offence • used as polygraphs and similar tools • to evaluate the reliability of evidence in the course of investigation or prosecution of criminal offences • to assess the risk of an individual offending or re-offending, or to assess personality traits and characteristics or past criminal behaviors • for profiling of individuals in the course of detection, investigation or prosecution of criminal offences - Migration, asylum and border control management • used as polygraphs and similar tools • to assess a risk of irregular migration, or a health risk due to people wanting to enter the territory of the EU • for the examination of applications for asylum, visa and residence permits • for the purpose of detecting, recognizing or identifying natural persons (except to verify travel documents) - Administration of justice and democratic processes • for researching and interpreting facts and the law and in applying the law to a concrete set of facts • for influencing the outcome of an election or referendum, or the voting behaviors of individuals during a vote or referendum Fundamental Rights Impact Assessment - Describe the process in which high risk Al will be used - Period of time and frequency in which a high-risk Al system is intended to be used - Categories of individuals who may be affected by high-risk Al - Specific risks of harm likely to impact those individuals - Human oversight measures that are implemented - Other measures taken such as internal governance and complaint mechanisms