Artificial Intelligence Archives - Blakistons

How Europe’s new AI rulebook would (and wouldn’t) touch autonomous combat aircraft—and what the defence carve?outs really mean

admin.richard — Thu, 06 Nov 2025 18:28:20 +0000

By Richard Ryan, barrister and drone lawyer

How Europe’s new AI rulebook would (and wouldn’t) touch autonomous combat aircraft — and what the defence carve-outs really mean.

In Brief…

Purely military AI systems are out of scope of the EU AI Act. If an AI system is developed or used exclusively for military/defence or national-security purposes, the Act does not apply. (EUR-Lex)
Dual-use is different. If the same autonomy stack, sensors or models are marketed or used for civilian purposes in the EU (for example, civil UAS, border or law-enforcement tasks), the Act can apply — with stringent duties for “high-risk” systems. (EUR-Lex)
Real-world testing is regulated. Pre-market R&D is generally excluded, but real-world testing isn’t — it requires specific safeguards and registration. (EUR-Lex)
Foundation models (GPAI) have their own rules from 2 Aug 2025; the defence carve-out in the Act is written for AI systems, not explicitly for models. If a model is placed on the EU market generally, the provider’s GPAI obligations can still bite. (EUR-Lex)

Context: sUAS News reports that GA-ASI is showcasing its autonomous fighter portfolio (for example, YFQ-42A CCA, MQ-20 Avenger) at the International Fighter Conference in Rome, 4–6 Nov 2025. This post overlays that scenario with the EU AI Act’s rules.

1) First principles: When does the EU AI Act apply?

The Act has extraterritorial reach. It covers (i) providers and deployers in the EU, (ii) providers placing on the EU market or putting systems into service in the EU — even if they are not established here — and (iii) providers/deployers in third countries where the AI system’s output is used in the EU. (EUR-Lex)

However, Article 2(3) draws a bright line: the Act does not apply to AI systems used exclusively for military, defence or national security. It also does not apply where a system is not placed on the EU market but its output is used in the EU exclusively for those purposes. Recital 24 reiterates this and clarifies that non-defence use falls back under the Act. (EUR-Lex)

What this means in Rome:

A closed, defence-only showcase for European militaries: out of scope.
A civil-use pitch, civil flight trials, or plans to sell autonomy modules to EU civilian buyers: in scope (see the high-risk section below). (EUR-Lex)

2) The key defence carve-outs (and their limits)

Carve-out #1 — Defence/military:

“This Regulation shall not apply to AI systems … used exclusively for military, defence or national security purposes.” (Article 2(3))

Two important nuances:

Exclusivity matters. The moment an autonomy stack or sensor suite is also marketed or used for civilian or law-enforcement tasks, the defence exclusion no longer shields those non-defence uses. (EUR-Lex)
Models vs systems. The text explicitly excludes AI systems for defence; it does not create an explicit defence exclusion for general-purpose AI models. If a GPAI model is placed on the EU market, Chapter V obligations for model providers can still apply — even if one downstream customer is a defence user. (More on GPAI below.) (EUR-Lex)

Carve-out #2 — Pre-market R&D:
R&D before placing on the market is generally outside scope, but real-world testing is not. Testing in real-world conditions triggers a dedicated regime (for example, registration, time limits, informed consent or special conditions for law enforcement, incident reporting). (EUR-Lex)

Carve-out #3 — Emergency derogations (non-defence):
For exceptional public-security reasons (or imminent threats to life/health), market surveillance authorities can authorise temporary use of a high-risk AI system before full conformity assessment — subject to strict conditions. Law-enforcement or civil-protection bodies can also use in urgent cases, then seek authorisation without undue delay. This is not a defence-specific carve-out, but it explains emergency deployments outside the military context. (EUR-Lex)

3) If the defence exclusion doesn’t apply, would autonomous fighters tech be “high-risk”?

Very likely yes — for civil variants or dual-use spin-outs:

Annex I (product-safety route). AI that is a safety component of products covered by sectoral EU safety laws is high-risk where those products need third-party conformity assessment. That list explicitly includes EU civil aviation law (Reg. 2018/1139) — covering unmanned aircraft and their remotely controllable equipment. In a civil-UAS configuration, an autonomy stack acting as a safety component would be regulated as high-risk. (EUR-Lex)
Annex III (stand-alone uses). Separate “high-risk” buckets also capture, for example, remote biometric identification and other sensitive functions (if and where permitted by Union/national law), critical infrastructure safety components, and more. If a fighter-born sensing suite were repurposed for civil border surveillance or public-space identification, you quickly hit these Annex III categories. (EUR-Lex)

What “high-risk” demands in practice
Providers must implement a risk-management system, data governance, technical documentation, logging, transparency/instructions, human oversight, and accuracy/robustness/cybersecurity — then pass conformity assessment, issue an EU Declaration of Conformity, and affix CE marking. Deployers also carry duties (for example, monitoring, data relevance, user notification in some cases). (EUR-Lex)

4) Sensors on show: what about face recognition and other “red lines”?

The EU bans several AI practices outright (from 2 Feb 2025), including:

Untargeted scraping of facial images to build recognition databases.
Biometric categorisation inferring sensitive traits (for example, race, political opinions, religion).
Emotion recognition in workplaces or schools (with narrow safety/medical exceptions).
Predictive “risk assessments” of criminality based solely on personality traits/profiling.
Real-time remote biometric identification (RBI) in public spaces for law enforcement — unless strictly authorised and necessary for narrowly defined objectives (for example, locating a specific suspect in serious crimes, preventing a specific imminent threat, finding missing persons), with prior judicial/independent approval and registration. (EUR-Lex)

Implication for a trade-show demo: training a camera on attendees to test real-time RBI in a public venue would likely be unlawful unless those strict law-enforcement exceptions and procedural safeguards apply — which they typically will not at a commercial defence conference. (EUR-Lex)

5) Real-world testing in the EU (civil or dual-use variants)

If a provider runs real-world flight tests in the EU (outside the defence exclusion), the Act requires — among other things — registration, an EU-established entity or EU legal representative, limits on duration (normally up to six months, extendable once), rules on informed consent (with special handling for law-enforcement tests), qualified oversight, and the ability to reverse/ignore the system’s outputs. Serious incidents must be reported promptly. (EUR-Lex)

6) Foundation models (GPAI): obligations can still attach

From 2 Aug 2025, Chapter V sets baseline transparency and copyright-policy duties for providers of general-purpose AI models (with extra obligations if the model presents systemic risks). The defence exclusion in Article 2(3) is framed for AI systems, not models. So, if a foundation model is placed on the EU market, the model provider can have obligations even if a downstream customer is a defence prime. (Open-source specifics and systemic-risk thresholds also apply.) (EUR-Lex)

7) Timelines you need in Rome (as of 6 Nov 2025)

Entry into force: 1 Aug 2024 (20 days after OJ publication).
Prohibited practices + core chapters (I–II): apply from 2 Feb 2025.
GPAI rules (Chapter V), plus other chapters (III §4, VII, XII, and Article 78): apply from 2 Aug 2025.
General application: 2 Aug 2026 (high-risk regime starts to bite broadly).
Article 6(1) Annex III classification trigger & related obligations: 2 Aug 2027. (EUR-Lex)

8) Enforcement and penalties

Violating prohibited practices (Article 5) can draw fines up to €35m or 7% of worldwide annual turnover, whichever is higher.
Other operator obligations can reach €15m or 3%; supplying misleading information can reach €7.5m or 1% (SMEs benefit from caps). Separate fine scales apply to EU institutions. (EUR-Lex)

9) Practical playbook for IFC attendees

If you are a defence OEM showing autonomy stacks:

Map uses: Defence-only (excluded) vs any civil or law-enforcement pathways (potentially in scope). Document the exclusivity of defence deployments if you rely on the carve-out. (EUR-Lex)
GPAI suppliers: If you place a foundation model on the EU market, expect Chapter V duties regardless of defence customers. (EUR-Lex)
No RBI demos on the show floor. Those prohibitions already apply in 2025. (EUR-Lex)
Planning EU flight tests for civil variants? Prepare for real-world testing conditions (registration, oversight, incident reporting). (EUR-Lex)
For civil UAS commercialisation, treat your autonomy as high-risk (EASA product-safety route), budget time for conformity assessment and CE marking. (EUR-Lex)

If you are a European ministry or agency:

Distinguish military operations (out of scope) from law-enforcement or border uses (in scope; watch RBI limits and high-risk duties). Consider Article 46 emergency derogations only in exceptional and documented cases. (EUR-Lex)

If you are a civil UAS integrator:

Expect the full high-risk package (risk management, data governance, human oversight, cybersecurity, logs, conformity assessment, CE). Build compliance into your system architecture, ML pipelines, safety cases, and ops manuals from day one. (EUR-Lex)

10) Quick decision pathway

Is the use exclusively defence or national security?
Yes: AI system is out of scope.
No: continue. (EUR-Lex)
Is it a civil product or law-enforcement/border use?
Civil product with safety function (for example, civil UAS): High-risk via Annex I ? conformity assessment + CE. (EUR-Lex)
Stand-alone sensitive use (for example, RBI, critical infrastructure): Annex III high-risk or Article 5 prohibition applies. (EUR-Lex)
Is there a GPAI model being placed on the EU market?
Yes: Chapter V duties for model providers from 2 Aug 2025, separate from the defence carve-out for systems. (EUR-Lex)
Is this pre-market testing?
Real-world testing rules apply (registration, oversight, incident reporting). (EUR-Lex)

Bottom line for “Autonomous Fighters in Rome”

A military-only display of GA-ASI’s autonomous fighters is outside the AI Act.
Any civil spin-off (cargo drones, civil surveillance, airport ops) or law-enforcement application in the EU will trigger the Act — often at the high-risk level — together with tight prohibitions around biometric uses in public spaces. Plan your compliance architecture accordingly. (EUR-Lex)

This article is informational and not legal advice. Citations are to the Official Journal text of the Artificial Intelligence Act (Regulation (EU) 2024/1689) for scope (Art. 2), prohibitions (Art. 5), high-risk regime (Ch. III), real-world testing (Arts. 57–61), GPAI (Ch. V incl. Art. 53), timelines (Art. 113), and penalties (Art. 99–101).

About the author — Richard Ryan

Richard Ryan is a UK barrister (Direct Access), mediator and Chartered Arbitrator (FCIArb), and a Bencher of Gray’s Inn. He practises across defence, aerospace, construction, engineering and commodities, with a leading specialism in drone and counter-drone law, unmanned aviation regulation, and AI-enabled safety and compliance. Richard advises government, primes and operators on EU/UK UAS frameworks, BVLOS, U-space/UTM and the EU AI Act. He leads Blakiston’s Chambers and contributes regularly to industry guidance and policy consultations.

The post How Europe’s new AI rulebook would (and wouldn’t) touch autonomous combat aircraft—and what the defence carve?outs really mean appeared first on Blakistons.

Military Drones and AI Regulation: A UK Drone Lawyer’s Perspective

admin.richard — Tue, 07 Jan 2025 13:29:44 +0000

Military Drones and AI Regulation: A UK Drone Lawyer’s Perspective

By Richard Ryan, UK Drone Lawyer

On 7 January 2025, The Guardian published an article highlighting the British AI consultancy Faculty AI’s involvement in the development of drone technology for defence clients, prompting renewed questions about where legal, ethical, and regulatory boundaries should lie for AI-driven military applications.
Faculty AI, already prominent for its work with various UK government departments (including the NHS and the Department for Education) and advisory services for the AI Safety Institute (AISI), has reportedly developed and deployed AI models on unmanned aerial vehicles (UAVs) for military purposes. Although it remains unclear whether these drones are intended for lethal operations, the revelations have amplified concerns about how best to regulate or restrict the use of AI in weapon systems.
Below, I explore the key legal issues and examine how the recently adopted EU AI Act—as well as the evolving UK regulatory framework—may shape the future of this sector.
________________________________________
1. Faculty AI’s Defence Work: A Brief Overview
1.1 Government and Public Sector Ties
Faculty AI, known for its work with the Vote Leave campaign in 2016, was later engaged by Dominic Cummings to provide data analytics during the pandemic. Since then, it has won multiple government contracts worth at least £26.6m, extending its work into healthcare (via the NHS), education, and policy consulting with the AISI on frontier AI safety.
1.2 UAV Development
The Guardian reports that Faculty AI has experience in deploying AI models on UAVs. Its partner firm, Hadean, indicated that the two companies collaborated on subject identification, tracking objects in movement, and exploring swarm deployment. While Faculty states that it aims to create “safer, more robust solutions”, details on whether these drones might be capable of lethal autonomous targeting remain undisclosed.
________________________________________
2. The EU AI Act: A New Regulatory Milestone
2.1 Status of the EU AI Act
Introduced by the European Commission in 2021 as a proposed regulation, the EU AI Act has since been adopted via the EU’s legislative process. As of early 2025, it is recognised as a binding regulation designed to harmonise AI rules across all EU Member States. Although the UK is no longer part of the EU, any UK-based company offering AI products or services within the EU must ensure compliance with the regulation’s requirements.
2.2 Risk-Tiered Framework
The EU AI Act operates on a tiered risk basis:
• Unacceptable risk: Certain AI applications (e.g., social scoring) are outright banned.
• High risk: This category includes critical infrastructure, healthcare, and—potentially—defence-related AI systems that could significantly affect people’s safety or fundamental rights. Such systems must meet strict transparency, oversight, and data governance requirements.
• Limited or minimal risk: These uses are subject to fewer obligations, generally focused on transparency (e.g., disclosing AI usage to end users).
For high-risk AI in military contexts, the EU AI Act demands robust human oversight, thorough documentation, and strict compliance obligations, particularly around accountability and the prevention of harm.
2.3 Potential Impact on Military Drones
While national security and defence largely remain the prerogative of individual EU Member States, the EU AI Act’s principles can still influence how companies and governments view the development of autonomous or semi-autonomous drones. Key considerations include:
• Transparent Data and Design: Documenting data sets, development processes, and operational parameters.
• Human in the Loop: Ensuring a human operator is always able to override or intervene in the AI’s decision-making. Other terms such as Human on the Loop and Human outside the Loop are also referred to.
• Liability and Penalties: Breaches can incur hefty fines—up to 6% of global turnover—thus acting as a significant deterrent against unethical or unlawful AI deployment.
________________________________________
3. The UK’s Approach to AI Regulation and Military Drones
3.1 Divergence from the EU?
Post-Brexit, the UK has chosen a “pro-innovation” approach to AI regulation. Rather than adopting a single, all-encompassing statute akin to the EU AI Act, the UK is implementing a sector-by-sector and risk-based strategy, guided by existing regulators such as the Information Commissioner’s Office and the Competition and Markets Authority.
3.2 AI Safety Institute (AISI)
Established under former Prime Minister Rishi Sunak in 2023, the AISI focuses on frontier AI safety research. Faculty AI’s role in testing large language models and advising the AISI on threats like disinformation and system security places the company in a key position to influence UK policy. Critics argue that this may create potential conflicts of interest if the same organisation is also developing AI for military use.
3.3 House of Lords Recommendations
In 2023, a House of Lords committee urged the UK Government to clarify the application of International Humanitarian Law (IHL) to lethal drone strikes and to work towards an international agreement limiting or banning fully autonomous weapons systems. The Government response acknowledged the importance of maintaining “human control” in critical decisions but did not enact binding legislation banning lethal autonomous drones outright.
________________________________________
4. Legal and Ethical Concerns for AI-Enabled Drones
4.1 International Humanitarian Law (IHL)
IHL principles—distinction (separating combatants from civilians) and proportionality (limiting harm relative to military objectives)—are central to discussions on AI-driven drones. Fully autonomous UAVs, capable of selecting and engaging targets without human intervention, raise profound legal questions on accountability, particularly if biases or system errors result in wrongful casualties.
4.2 Allocation of Liability
Traditionally, accountability in military operations lies with commanders and operators. With increasingly autonomous systems, however, liability could extend to technology developers, programmers, or even the purchaser of the system. Clarifying how legal responsibilities are distributed may become a focal point for future litigation and regulatory reform.
4.3 Export Controls
Companies like Faculty AI must also comply with arms-export rules when providing AI-targeting systems or related software to foreign entities. In the UK, export licences for military-grade technology are subject to domestic legislation and international protocols, such as the Wassenaar Arrangement on dual-use goods.
________________________________________
5. Looking Ahead: Balancing Innovation, Safety, and Accountability
5.1 Stronger National Frameworks
Although the UK favours a pro-innovation stance, there is growing pressure from Parliament and civil society for more rigorous, enforceable rules on potentially lethal AI applications. The EU AI Act may serve as a reference point for the UK to consider stricter domestic regulations.
5.2 International Collaboration
Calls for global agreements—treaties or non-binding accords—to prohibit fully autonomous weapons continue to gain momentum. The House of Lords committee specifically recommended international engagement to ensure that lethal force remains under human control.
5.3 Corporate Accountability
Organisations operating at the intersection of commercial defence contracts and government policy—such as Faculty AI—need transparent internal processes and robust ethics boards to mitigate conflicts of interest. Demonstrating genuine corporate responsibility will be vital for maintaining public trust.
5.4 Ethical and Safety Audits
As AI becomes more embedded in defence, mandatory ethical and safety audits may become standard practice. These would scrutinise algorithmic fairness, training data, and how effectively systems can identify and mitigate unintended harms.
________________________________________
6. Conclusion
Faculty AI’s role in developing AI for military drones underscores how high the stakes are when cutting-edge technology meets defence applications. With the EU AI Act now in force as a binding regulation, Europe has provided a blueprint for tighter control over “high-risk” AI systems. In contrast, the UK’s approach still offers substantial flexibility for companies, potentially raising both legal and ethical concerns around autonomy, accountability, and conflicts of interest.
From an IHL standpoint, keeping a human responsible for any life-and-death decision is imperative. As a UK drone lawyer, I urge policymakers, regulators, and industry stakeholders to keep asking: Where do we draw the line between legitimate defensive innovation and an unacceptable risk to civilians? Only by establishing clear, enforceable legal standards—anchored in international law and ethical scrutiny—can we ensure AI-powered drones serve to protect rather than endanger fundamental human values.

Bio – Richard Ryan, UK Drone Lawyer

Richard Ryan is a UK-based drone lawyer specialising in the regulatory, ethical, and commercial aspects of unmanned aerial vehicles (UAVs) and artificial intelligence (AI). Through a series of blogs, Richard Ryan has explored critical issues such as the EU AI Act, the UK’s evolving “pro-innovation” regulatory landscape, and the legal considerations surrounding military drones and lethal autonomous weapons systems.

Drawing on extensive experience in advising government bodies, technology companies, and public institutions, Richard Ryan brings a deep understanding of how international humanitarian law (IHL), export controls, and data protection obligations intersect in modern drone operations. Their writing emphasises the importance of maintaining human oversight in AI-driven systems, championing ethical development and transparent accountability mechanisms.

A trusted voice in the field, Richard Ryan regularly comments on emerging case law, parliamentary recommendations, and global discussions around frontier AI safety. The mission is to help stakeholders—from hobbyist drone operators to established aerospace firms—navigate the complexities of regulation, risk management, and innovation.

The post Military Drones and AI Regulation: A UK Drone Lawyer’s Perspective appeared first on Blakistons.

AI Drone Swarms and the EU AI Act: A Game-Changer in Modern Warfare?

admin.richard — Sat, 16 Nov 2024 17:44:52 +0000

AI Drone Swarms and the EU AI Act: A Game-Changer in Modern Warfare?

By Richard Ryan, Drone Lawyer

The recent trials conducted by the AUKUS nations—Australia, the United Kingdom, and the United States—mark a significant milestone in the integration of artificial intelligence (AI) and autonomy within military operations. The deployment of AI-enabled uncrewed aerial vehicles (UAVs) capable of locating, disabling, and destroying ground targets presents both remarkable advancements and complex legal challenges, particularly in the context of the European Union’s AI Act.

As a drone lawyer with over 20 years of experience in the UK, I find it imperative to dissect the interaction between these groundbreaking trials and the regulatory landscape shaped by the EU AI Act. This discussion aims to highlight the risks, oversight issues, and intellectual property considerations that arise when integrating AI algorithms into military UAV swarms.

Understanding the EU AI Act’s Impact

The EU AI Act seeks to establish a comprehensive regulatory framework for AI technologies, focusing on transparency, accountability, and human oversight. High-risk AI systems, which include those used in critical infrastructure and law enforcement, are subject to stringent requirements. Military applications, while often exempt from certain civilian regulations, still operate under international humanitarian laws and ethical guidelines that resonate with the Act’s principles.

The AUKUS trials demonstrate the use of AI in autonomous systems for military purposes. The AI-enabled UAVs operated collaboratively, sharing data seamlessly across nations. While the Act primarily governs civilian AI use within the EU, the ethical considerations it embodies cannot be ignored in military contexts, especially when such technologies might eventually influence civilian sectors.

Risks and Oversight Challenges

One of the foremost risks is the potential for AI algorithms to make autonomous decisions without adequate human oversight. The EU AI Act emphasizes the necessity of meaningful human control over AI systems, particularly those capable of impacting human lives. In the AUKUS trials, although a human operator was involved, the level of autonomy granted to the UAVs raises questions about compliance with the Act’s standards if similar technologies were deployed within the EU.

Data exchange and interoperability between the three nations introduce another layer of complexity. The seamless sharing of information enhances operational efficiency but also raises concerns about data protection and cybersecurity. Ensuring that sensitive data transmitted between UAVs and control systems is secure aligns with the Act’s requirements for robust data governance.

The Case for a Simulation Sandbox

To address compliance with the EU AI Act, conducting such trials within a simulation sandbox could be a prudent approach. A sandbox environment allows for the testing and validation of AI algorithms in a controlled setting, mitigating risks associated with real-world deployment. It enables developers to assess the AI’s decision-making processes, identify potential flaws, and ensure adherence to ethical and legal standards before actual implementation.

Moreover, a sandbox can facilitate transparency and accountability, key tenets of the EU AI Act. By documenting the AI’s performance and decision rationale within simulations, stakeholders can provide evidence of compliance and readiness for safe deployment.

Intellectual Property Considerations

Introducing AI algorithms into a regulatory sandbox presents intellectual property (IP) risks that must be carefully managed. Proprietary algorithms and technologies shared within the sandbox could be exposed to unauthorized access or misuse. Protecting IP rights is crucial to encourage innovation and maintain competitive advantages.

To mitigate these risks, clear agreements outlining the ownership, usage rights, and confidentiality obligations related to the AI algorithms are essential. Collaborative efforts, such as those seen in the AUKUS trials, require robust legal frameworks to safeguard each party’s IP while promoting shared development goals.

Conclusion

The integration of AI and autonomous systems in military applications is an evolving frontier that necessitates careful navigation of legal and ethical landscapes. The EU AI Act, while primarily focused on civilian applications, provides valuable guidance on managing high-risk AI systems.

By recognising the risks and oversight challenges presented by the AUKUS AI-enabled UAV trials, stakeholders can proactively address compliance issues. Utilising simulation sandboxes offers a viable pathway to refine these technologies within the bounds of regulatory requirements.

Intellectual property considerations remain a critical aspect of this process. Ensuring that AI algorithms are protected within collaborative environments will foster innovation while maintaining legal integrity.

As we advance into this new era of AI-driven military capabilities, a balanced approach that harmonises technological potential with regulatory compliance will be essential. The lessons learned from these trials will undoubtedly shape the future of AI in both military and civilian spheres.

—

About Richard Ryan

Richard Ryan is a leading drone lawyer based in the United Kingdom, with over 20 years of legal experience as a direct access barrister. Specializing in the legal aspects of unmanned aerial systems and AI technologies, Richard has advised government agencies, defense contractors, and private enterprises on compliance, intellectual property, and regulatory matters. His extensive expertise bridges the gap between cutting-edge technological advancements and the complex legal frameworks that govern them.

The post AI Drone Swarms and the EU AI Act: A Game-Changer in Modern Warfare? appeared first on Blakistons.