European flag
>
The next phase of artificial intelligence – Governing for safety, security and human agency Joint EGE GCSA Statement
Screenshot
Statement
16 November 2026
doi:10.2777/5617141

1. Abstract

In this Statement, the Group of Chief Scientific Advisors and the European Group on Ethics in Science and New Technologies argue that AI presents distinctive challenges for public policy, and that traditional guard rails no longer suffice. They recommend avoiding any compromising of existing legislation, and ensuring that regulation evolves in step with these advancing technologies, to continually improve society’s capacity to manage new risks.

The level of ambition in current EU legislation is high, and it will be essential to maintain that level of vigilance in the face of external pressures, and individual and systemic risks. This can be done through the effective implementation and enforcement of the EU AI Act and all other relevant EU-wide frameworks, with safety and security remaining absolute prerequisites for AI development and deployment.

The Statement recommends a nuanced approach to the pace of development and deployment: lower-risk applications may progress more rapidly, but high-risk, highly autonomous or difficult-to-reverse deployments need stronger testing, human oversight and effective safeguards. Europe must expand its capacity to evaluate and govern AI systems independently of the companies and non-European actors that develop them, strengthening its scientific and technical expertise, evaluation infrastructure and technological sovereignty, while reducing strategic dependencies.

A robust AI safety and security ecosystem is of key importance and must continue to evolve, with continuous monitoring, independent evaluations, incident reporting, learning from failures and near misses, and international cooperation. A proportionate, risk-based approach will enable responsible AI innovation, while always safeguarding security, fundamental rights, democratic accountability and human agency.

2. Statement

Artificial Intelligence (AI) systems and their underlying models are becoming increasingly capable, operationally autonomous, and pervasive, while individuals, organisations and societies are becoming increasingly dependent on them. As AI becomes ever more deeply embedded in science, government, businesses, essential public services, critical infrastructure, crisis management, and everyday life, new forms of societal and strategic dependency come into being. In this joint statement, we want to make clear that AI is an exceptional technology in terms of the speed of its development, and in how it affects all aspects of life. Traditional guard rails will not suffice.

This uniqueness of AI changes both the scale and nature of the risks that need to be considered by public policy. Governance must address not only individual applications, but also systemic risks. In our view, and in line with the headline objectives of EU legislation, technological progress must remain in the service of human dignity, human agency, equality and non-discrimination, privacy and data protection, freedom of expression and information, freedom of thought and conscience. Meaningful human oversight must be strengthened and protected in the longer term, as systems become more ‘autonomous’ and decision-making processes grow increasingly complex and opaque.

While this statement focuses on safety and security, AI raises many other concerns that deserve urgent attention. They include discrimination and other violations of fundamental rights, privacy, copyright, working conditions and employment, the integrity of information and democratic processes, the wellbeing of children and young people, and environmental sustainability. These concerns are tangible and immediate and must not be neglected in debates that address more speculative risks for which responsibility is often obscured. The GCSA and the EGE have a long-standing commitment to advising the Commission on how such concerns should be addressed.

While the current security and safety risks posed by AI systems are significant, a wealth of principles, regulation and good practices are already in place. Europe already has a substantial legal and institutional framework addressing many of these concerns, notably the EU Artificial Intelligence Act (AI Act)1, which includes specific provisions on general-purpose AI models and models with systemic risk, and the General Purpose AI Code of Practice2, alongside Union legislation on cybersecurity3, resilience, data protection4, product safety5 and sector-specific regulations that also apply to AI in certain instances. It is essential to ensure that this framework is implemented, applied, monitored and enforced in an effective manner; that the technical and institutional capacity needed to evaluate increasingly capable AI is strengthened; and that emerging risks that may justify additional measures are clearly identified.

Sometimes the obstacle is not absence of legislation, but absence of the political will or institutional capacity to enforce existing obligations, compounded by external efforts to delay or weaken them. When it comes to risk management and accountability, AI should be subject to clear product safety and liability requirements, with responsibility allocated to the actors best placed to prevent and address the risks arising from its development, placing on the market, and use in accordance with the applicable Union frameworks for safety, cybersecurity, liability and fundamental rights. In this context, responsibility should remain with the relevant actors involved in the development, provision and deployment of these systems, in accordance with their respective roles and obligations under applicable law. Existing technical and regulatory safeguards should apply, complemented by new guard rails in the form of public policy.

Against this complex background, the overall principle should be that safety and security remain prerequisites for the responsible development and deployment of AI tools and systems, in line with existing Union law and without creating an exceptional regulatory regime for AI. This is the approach of the EU Artificial Intelligence Act, which requires continuous risk management for high-risk AI systems throughout their lifecycle, with appropriately high levels of accuracy, robustness and cybersecurity. More capable AI models should be developed and deployed only where there is sufficient evidence that the associated risks can be identified, assessed and appropriately managed. In all cases, safety and security assessments should not be responses to harms after they occur, but conditions considered before deployment and monitored throughout the lifecycle of AI, in line with the requirements of applicable Union law.

Frontier AI should be no exception to these principles, and compliance with the AI Act should always remain a core concern, including the specific obligations applicable to providers of general-purpose AI models with systemic risk and the General Purpose AI Code of Practice. We also emphasise the importance of protecting fundamental rights and ensuring that existing rules are effectively applied, implemented and enforced.

As Commission President Ursula von der Leyen emphasised in her State of the European Union (SOTEU) address6, Europe seeks to develop human-centred AI that creates value responsibly, reflecting what she described as ‘the essence of the European way.’ This provides an important political direction for European AI governance: Europe should seek not merely to adapt to technological developments, but to shape the conditions under which they serve European societies and values as well as our global community.

Four principles follow from this approach.

2.1. No compromising of current legislation: safety and security must remain prerequisites for development and deployment, with evidence requirements proportionate to capability and potential consequences

The greater the potential consequences of an AI system, its degree of autonomy, and the difficulty of reversing its deployment, the more robust the evidence and safeguards required before deployment should be, subject to the applicable legal framework. For General Purpose AI models with systemic risk, providers should in particular assess whether the systemic risk is acceptable before proceeding with development, making the model available on the market and/or its use, as applicable under the AI Act and the GPAI Code of Practice.

  • Evidence should address not only performance but also safety and robustness, cybersecurity, controllability, resilience, resistance to failure or manipulation, data and model integrity, and appropriate fallback arrangements. These elements should be assessed using methods appropriate to the model or system and the risks concerned, including the legal classification of the system, the role of the relevant actor in the AI value chain, and the intended purpose and context of use, while always taking account of applicable legal requirements.
  • Higher standards should apply where incidents could have major societal consequences, where systems operate with substantial ‘autonomy’, or where deployment creates dependencies that would be difficult to reverse. This should include consideration of both safety risks arising from failure or unintended behaviour, and security risks arising from deliberate misuse, manipulation or attack. For AI systems, these standards should reflect the risk classification established under the AI Act. For GPAI models with systemic risk, the assessment of systemic risk and the corresponding mitigation measures should reflect the requirements of both the AI Act and the General Purpose AI Code of Practice.
  • Wherever possible, common principles, harmonised testing approaches and evidence requirements should be developed internationally. AI systems and their risks cross national borders, and incompatible national approaches could create both regulatory gaps and incentives to deploy systems in jurisdictions with weaker safeguards. However, the pursuit of global agreement should not prevent the European Union from establishing and enforcing appropriate protections where international consensus cannot be reached, particularly for systems affecting EU citizens, institutions and critical infrastructure.
  • In general, international cooperation should seek convergence around common minimum standards as well as their further development, consistent with the applicable Union legal framework and further developing it where needed. In the EU, particular care should be taken to ensure that additional approaches remain compatible with the harmonised framework established by the AI Act. International cooperation should build on existing instruments and fora, including the Council of Europe Framework Convention on Artificial Intelligence, while supporting convergence around effective application, implementation and evaluation.

2.2. Not every AI system should be developed and deployed at the same pace

Deployment and development conditions should differ according to capability, context and consequence rather than applying the same approach to all AI systems and models.

  • A blanket approach to slowing or accelerating AI development cannot reflect the very different benefits and risks associated with different applications. For GPAI models with systemic risk, for example, the relevant question under the AI Act and GPAI Code of Practice is whether the systemic risk is acceptable, taking into account the model's capabilities, propensities and affordances and the context in which the model is developed, made available or used.
  • Rapid deployment may be appropriate where potential adverse consequences are limited, systems are readily reversible, benefits are potentially large and risks are well understood and manageable, subject to the applicable legal requirements.
  • Stronger evidence and safeguards should be required where AI is deployed in high-risk domains, such as critical infrastructure, crisis management, essential public services including health care or other settings in which failures could have substantial and/or irreversible societal consequences. The applicable requirements should reflect the risk categories and use-case areas established under the AI Act.
  • Reversibility considerations should become a sine-qua-non requirement of responsible deployment. Where AI creates technological or institutional dependencies, or “lock-in” situations that would be difficult to reverse in the event of serious subsequent problems, deployment should require greater caution, including rigorous ex ante testing, safeguards, contingency planning, and effective mechanisms for suspension or withdrawal.
  • Heightened scrutiny may also be warranted for highly ‘autonomous’ or agentic systems where meaningful human oversight becomes more difficult or where systems can take consequential actions without continuous human oversight and intervention. This scrutiny could include more rigorous pre-deployment testing, clearly defined limits on autonomous actions, effective mechanisms for human-intervention, and the ability to suspend or terminate system operation when risks become unacceptable. It is important that these assessments are guided and controlled by public policy and not left to self-regulating technology companies.
  • Where human oversight is required by applicable law, it should be supported by sufficient authority, competence, information and operational capacity to override or discontinue the relevant system, with EU institutional and financial support where needed, in accordance with the human oversight requirements already established under the AI Act.

Differentiated deployment offers an alternative to a binary choice between “pausing” and “accelerating” AI: progress can continue, but the conditions for deployment should become more demanding as capability, autonomy and potential consequence increase within the risk-based framework established by applicable Union law.

2.3. Europe must strengthen its capacity to independently evaluate and govern the AI systems on which it depends

Europe should pursue technological sovereignty without seeking technological isolation. Sovereignty does not require complete self-sufficiency and should not be understood as isolationism or as “going it alone”. It should mean having sufficient capacity to make meaningful choices and to participate in international cooperation as a genuine and capable partner. This understanding of sovereignty is consistent with the direction set out by the Commission President in the 2026 State of the Union and in the Communication on European Tech Sovereignty7, as well as with the approach of the proposed Cloud and AI Development Act8.

This approach requires a strengthening of the EU’s capacity to develop and deploy critical AI technologies in line with European values, while reducing strategic dependencies and the risks they entail, diversifying supply chains, and retaining the ability to evaluate, secure and govern the systems on which Europe increasingly relies. The goal is technological sovereignty, which requires genuine, democratically accountable control over how data and models are used, and who benefits from them.

  • Rather than relying primarily on assessments made by their developers, the EU needs to develop the independent capacity to evaluate increasingly powerful AI systems. This scientific and technical expertise should complement rather than duplicate the governance, monitoring and enforcement structures established under the AI Act. It should be supported by appropriate access to relevant AI models and the evidence needed to evaluate them, as well as by the capacity to design and conduct frontier-model evaluations, including adversarial testing and assessments of cybersecurity dual-use risks and increasingly agentic capabilities. It also includes sustaining human capacity and expertise and safeguarding technological sovereignty over the physical infrastructure and data necessary to conduct these evaluations.
  • The European Union should develop shared evaluation infrastructure and expertise that can be used by public authorities and independent researchers, building, where appropriate, on the expertise and advisory structures supporting the implementation of the AI Act, including the Scientific Panel and Advisory Forum.
  • This evaluation capacity is particularly important for evaluating systems and frontier AI models developed outside Europe on which European science, government, critical infrastructure or essential services may nevertheless depend.
  • Strategic dependencies across models, compute, cloud infrastructure, data and other elements of the AI technology stack should be understood and monitored. The risk-based approach of the cloud and AI sovereignty framework under the proposed Cloud and AI Development Act should be fully exploited for this purpose.
  • Public oversight should be democratically legitimate, exercised in a manner consistent with the rights, freedoms, and principles enshrined in the Charter of Fundamental Rights of the European Union, and supported by sufficient independent technical expertise within public institutions and oversight bodies, complemented by meaningful participation from civil society and affected communities. This should build on the governance, monitoring and enforcement mechanisms already established under the AI Act. Companies should never be the sole arbiters of whether their systems are safe, secure or compatible with fundamental rights.

2.4. Build an ecosystem for AI safety, security and continuous learning

Governance should create the capacity to learn systematically from testing, deployment, failures, near misses and emerging evidence.

  • Independent pre-deployment testing should be complemented by continuous monitoring after deployment, as some risks may only become apparent through real-world use. For models with systemic risk, this should build on the obligations already established under the AI Act and operationalised through the GPAI Code of Practice, including pre-deployment evaluation, post-market monitoring and independent external evaluation.
  • Significant incidents and near misses must be recorded, assessed and, where required by applicable law, reported and investigated using existing legal mechanisms. Further reporting mechanisms should be developed where necessary and proportionate, so that lessons can be learned and safeguards strengthened. The AI Act already establishes serious incident reporting obligations for providers of general-purpose AI models with systemic risk. These obligations should form part of a wider learning infrastructure. In particular, providers of GPAI models with systemic risk are subject to obligations to keep track of, document and report relevant serious incidents in accordance with the AI Act9. Where appropriate and legally permissible, mechanisms for sharing lessons and evidence between relevant actors should be strengthened.
  • Evaluation should include adversarial testing and assessment of vulnerabilities to manipulation, malicious use and cyberattack, alongside conventional safety and performance testing, while also taking into consideration sovereignty risks related to unlawful access to data and service continuity. For GPAI models with systemic risk, evaluations should specifically address systemic risks and should be designed and conducted using methods appropriate to the model and the systemic risks concerned.
  • Evidence and evaluation methods should evolve in step with AI capabilities; safety assessment cannot rely indefinitely on benchmarks designed for earlier generations of systems. Evaluations should use standardised protocols and tools, while evolving as capabilities, risks and evaluation methodologies progress.
  • International cooperation and evidence-sharing should be strengthened so that critical incidents or vulnerabilities identified in one jurisdiction can inform risk management elsewhere. Such sharing should be proportionate and subject to appropriate security assessment, recognising that even individually non-sensitive information may create risks when combined. Mechanisms should therefore allow useful lessons to be shared while protecting security-sensitive, commercially sensitive and other legitimately restricted information.

The aim should be a system that evolves in step with advancing technologies, rather than a system focused on ensuring compliance. Regulation should enable the continual improvement of society's understanding and governance of emerging capabilities, risks, and trade-offs. Experience from safety-critical sectors, including the nuclear sector, shows the value of systematic reporting and learning from both failures and near misses. Similar mechanisms for AI should support proportionate sharing of relevant evidence between developers, researchers and public authorities, while protecting legitimately commercially sensitive or security-sensitive information.

More generally, the benefits and costs of AI systems, including their environmental footprint, energy consumption, and impacts on populations, should be rigorously assessed and taken into account in decision-making, in accordance with Article 191 TFEU10, the AI Act1 and the Energy Efficiency Directive11. Implications for the EU’s greenhouse gas emissions commitments should also be evaluated. Such evaluation should assess not only their direct effects, but also those arising through the behavioral responses and adaptation of citizens and organizations and through broader systemic adjustment to the use of AI.

Together, these measures should complement rather than replace existing Union and national law, in particular the AI Act and the obligations applicable to providers of General Purpose AI models, including models with systemic risk as well as the proposed Cloud and AI Development Act. They support an approach in which Europe continues to benefit from advances in AI while ensuring that this increasing capability is accompanied by increasing evidence, oversight and capacity to manage risk. The objective is to ensure that technological progress can continue, and that our capacity to evaluate, govern and learn from AI develops in step with the technology. Where additional measures are considered necessary, they should be assessed for their added value, consistency with the existing Union framework, and proportionality to the risks concerned. AI poses many additional challenges beyond safety and security, but our concern here is with ensuring that as AI systems become more capable, autonomous and deeply embedded in society, the capacity of developers, deployers, regulators and democratic institutions to evaluate, govern and learn from them develops alongside the technology.

Annex I

References

1. EU Artificial Intelligence Act (AI Act), Regulation (EU) 2024/1689

2. General-Purpose AI Code of Practice, final version, 10 July 2025, prepared by independent experts under the facilitation of the European AI Office pursuant to Article 56 of Regulation (EU) 2024/1689 (Artificial Intelligence Act), available at the European Commission’s AI Office website. The Code provides voluntary guidance on compliance with the obligations applicable to providers of general-purpose AI models under Articles 53 and 55 of the AI Act

3. See, in particular, Regulation (EU) 2019/881 of the European Parliament and of the Council of 17 April 2019 on ENISA (the European Union Agency for Cybersecurity) and on information and communications technology cybersecurity certification (Cybersecurity Act), OJ L 151, 7.6.2019, p. 15; Directive (EU) 2022/2555 of the European Parliament and of the Council of 14 December 2022 on measures for a high common level of cybersecurity across the Union (NIS 2 Directive), OJ L 333, 27.12.2022, p. 80

4. Regulation (EU) 2016/679 (General Data Protection Regulation - GDPR), OJ L 119, 4.5.2016, pp. 1–88, establishes the principal EU framework for the protection of natural persons with regard to the processing of personal data and the free movement of such data. See, in particular, Arts. 5–6 GDPR concerning the principles and lawfulness of processing

5. Regulation (EU) 2023/988 on general product safety (General Product Safety Regulation), OJ L 135, 23.5.2023, pp. 1–51, establishes the EU framework for ensuring that consumer products placed on the Union market are safe, including obligations concerning risk assessment, traceability, recalls and cooperation between economic operators and market-surveillance authorities

6. Ursula von der Leyen ‘State of the Union Address 2026’ (European Commission, 16 September 2026)

7. Communication from the Commission to the European Parliament, the Council, The European Economic and Social Committee and the Committee of the Regions on European Tech Sovereignty, accompanied by an EU Open Source Strategy COM(2026)503 final, 3 June 2026, available at eur-lex.europa.eu/legal-content/EN/TXT/HTML/?uri=COM:2026:503:FIN

8. European Commission, Proposal for a Regulation of the European Parliament and of the Council establishing a framework of measures for strengthening Europe’s cloud and AI ecosystem (Cloud and AI Development Act), COM(2026) 502 final, 3 June 2026, available at: https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=COM%3A2026%3A0502%3AFIN

9. Regulation (EU) 2024/1689 (Artificial Intelligence Act), Article 55, concerning additional obligations for providers of high-risk AI systems

10. Consolidated version of the Treaty on the Functioning of the European Union, Art. 191, OJ C 202, 7 June 2016, p. 47

11. Energy Efficiency Directive (EU) 2023/1791, Article 12 – reporting requirements relating to data centre energy performance, supplemented by the Commission’s guidance on data centre energy performance

Annex II

Author names

Nikola Biller-Adorno, Chair of Biomedical Ethics and Director, Institute of Biomedical Ethics and History of Medicine, University of Zurich (UZH), Vice Dean Innovation and Digitalization, Medical Faculty, UZH

Maria do Céu Patrão Neves, Professor of Ethics at the University of the Azores, President of the Portuguese National Council of Ethics for the Life Sciences

Naomi Ellemers, Distinguished University Professor, Utrecht University, the Netherlands

Adam Izdebski, Professor of Human Ecology, Institute of Advanced Studies, Nicolaus Copernicus University in Toruń, Poland

Martin Kahanec, Professor, Central European University, Vienna, Austria; Central European Labor Studies Institute, Bratislava, Slovakia; European University Institute, Florence, Italy

Migle Laukyte, Associate Professor in Artificial Intelligence and Law at the Law Faculty of Pompeu Fabra University in Barcelona, Spain, Member of the European Group of Blockchain Ethics (EGBE)

Rafał Łukasik, Professor, Director of Łukasiewicz Research Network – Industrial Chemistry Institute, Warsaw, Poland

Paweł Łuków, Dean of the Faculty of Philosophy, University of Warsaw, Poland, Head of the Committee for the Ethics in Science of the Polish Academy of Science

Fruzsina Molnar-Gabor, Professor of International Medical and Health Law as well as Data Protection Law at the Faculty of Law at the University of Heidelberg

Therese Murphy, Professor of Law and Director of the Health & Human Rights Unit, Queen's University Belfast

Barbara Prainsack, Professor at the Department of Political Science at the University of Vienna, Director of the Centre for the Study of Contemporary Solidarity (CeSCoS) and of the interdisciplinary Research Platform “Governance of Digital Practices”

Nils-Eric Sahlin, Professor of Medical Ethics, Lund University, Member of The Royal Swedish Academy of Letters, History and Antiquities, Chairman of The Swedish Research Council's Expert Group on Ethics

Tamar Sharon, Professor of Philosophy, Digitalization and Society at Radboud University Nijmegen, Chair of the Department of Ethics and Political Philosophy, Co-Director of iHub, the Interdisciplinary Hub for Digitalization and Society

Dimitra Simeonidou, Professor of High-Performance Networks and Director Smart Internet Lab, University of Bristol, UK

Rémy Slama, Senior Investigator, Inserm (National Institute of Health and Medical Research), Professor in Environmental Health at ENS-PSL (Ecole Normale Supérieure), Paris, France

Mangala Srinivas, Professor of Cell Biology and Immunology, Wageningen University and Research, Wageningen, The Netherlands

Marcel Jeroen van den Hoven, University Professor and professor of Ethics and Technology at Delft University of Technology, Scientific Director of the Delft Design for Values Institute

Takis Vidalis, Adjunct Professor at the International Hellenic University of Thessaloniki and the University of Athens Senior researcher and legal advisor of the Hellenic National Commission for Bioethics and Technoethics Research Ethics Committee president of the National Centre for Scientific Research "Democritos"

Table of contents