How To Open An Algorithmic Black Box? The Multiple Ways Of Making Transparency: From Source Code To Counterfactual Examples Presented by: Katja de Vries, Assistant Professor in Public Law, Uppsala University Date: 24 February 2021, 18-19.30 CET Introduction - China committed itself to being part of the international human rights system - Transparency - why? - Automated Decision Making (and discriminatory ML) - Multiple ways of opening the Blackbox - Generative ML - Counterfactuals - the good - Counterfactuals - the bad - Concluding thoughts Transparency - Why? - UK GDPR – unlikely that core GDPR ideas are affect • 1 January 2021, the DPPEC (Data Protection, Privacy and Electronic Communications (Amendments etc) (EU Exit)) Regulations 2019 - GDPR, Recital 39 • Any processing of personal data (...) should be transparent to nature persons that personal data concerning them are collected, used, consulted or otherwise processed and to what extent the personal data are or will be processed. The principle of transparency requires that any information and communication relating to the processing of those personal data be easily accessible and easy to understand, and that clear and plain language be used. That principal concerns, in particular, information to the data subjects on the identity of the controller and the purposes of the processing and further information to ensure fair and transparent processing in respect of the natural persons concerned and their right to obtain confirmation and communication of personal data concerning them which are being processed. Natural persons should be made aware of risks, rules, safeguards and rights in relation to the processing of personal data and how to exercise their rights in relation to such processing. - Profile transparency • (Arts. 12-15 GDPR 2016/679) Providing "meaningful information about the logic involved" and about the "significance and the envisaged consequences" of the profiling. - Limitations • profile transparency is subject to limitations (see recital 42 of the DPD 95/46 and recital 63 of GDPR 2016/679) due to trade secrets and Intellectual Property Rights (IPRs) What Is The Algorithm Register? - The Algorithm Register is an overview of the artificial intelligence systems and algorithms used by the City of Amsterdam. Through the register, you can get acquainted with the quick overviews of the city's algorithmic systems or examine their more detailed information based on your own interests. You can also give feedback and thus participate in building human-centered algorithms in Amsterdam. The register is still under development. Sneak-Preview: Counterfactual Examples - Sandra Wachter, et al., Counterfactual explanations without opening the black box: automated decisions and the GDPR, 31 Harvard Journal of Law & Technology (2018). - Holiday rental housing fraud risk • From 1 July 2020, a pilot will be carried out for six months with an algorithm that supports the employees of the department of Surveillance & Enforcement in their investigation of the reports made concerning possible illegal holiday rentals. The algorithm helps prioritize the reports so that the limited enforcement capacity can be used efficiently and effectively. By analyzing the data of related housing fraud cases of the past 5 years, it calculates the probability of an illegal holiday rental situation on the reported address. If Transparency Is The Answer, What Is The Problem? Based on Margot E. Kaminski (2020), Understanding Transparency in Algorithmic Accountability, in The Cambridge Handbook of the Law of Algorithms 121 (Woodrow Barfield ed.): 1. Systemic problems of discriminatory bias 2. Decisions with significant consequences lacking an appropriate justification 3. Objectification of the subject (no dignity)