What approaches are there to regulating the use of lethal autonomous weapons systems (or ‘killer robots’)?
Bernard Merck[1]
To what extent can we delegate decisions that affect human life to machines?
V5 – BM November 2026
Preamble
My modest essay on ‘Killer Robots: Our Humanity in Peril’ dates back almost two years[2] . At the time, I naively believed it was possible, albeit difficult, to establish international regulations governing the use of lethal drones. Whilst I had a clear sense of the emerging trends, I did not foresee the speed of their development and deployment on the ground, making it difficult to turn back the clock. What options remain today?
The growing use of drones raises the question of the conditions under which they are used – a question made all the more pressing by the trend towards the use of Artificial Intelligence throughout the entire operational process. This is why we speak of Lethal Autonomous Weapon Systems (LAWS).
Lessons from the Russia-Ukraine conflict attest to the widespread use of drones by both sides[3] and their lethal impact on the ground. According to several articles, drones are believed to be responsible for three-quarters of the human casualties on both sides.
All the world’s armed forces – and their military-industrial complexes – are closely monitoring events in Ukraine, including countries already advanced in this field (China, Iran, Israel, Russia, Turkey and the USA); some are focusing on the conditions for their use in conflict situations, whilst others seek to master the technology to modernise their military capabilities.
What approaches are there to regulating the use of lethal autonomous weapons systems (or ‘killer robots’)?
What is meant by autonomous weapons systems?
It is a military automated system capable of carrying out lethal actions automatically without human intervention. It may be a mobile device (such as a drone) or a fixed system (such as South Korea’s Samsung SGR-A1[4] ).
Etymologically, ‘autonomous’ means ‘governed by its own laws’. This adjective raises the question of robot obedience, particularly with regard to the continuity of the chain of command.
To be truly ‘autonomous’, a system must be capable, in order to achieve its objectives, of making its own choices – based on its detailed programming for the mission assigned to it and the situation in which it finds itself – from among various predefined options, or options generated by AI where applicable. SALAs can be defined as systems which, once activated, are capable of locking onto a target and using lethal force without supervision by a human operator.
Very early on, a debate arose regarding the difference between autonomous systems and automated systems, depending on the degree of latitude afforded to the machine. As long as conventional programming was involved, the situation was clear: the systems were automated. It was possible to examine their programming code, assess their safety and evaluate the ethics of their deployment. The same causes produced the same effects, although sensors could generate false information. Everything depended on the programming (code) and the reliability of the sensors, but it was possible to audit the system.
With the advent of AI, the decision-making process can no longer be audited; the same situations can produce different outcomes. Worse still, the sensors themselves rely on AI. The decision-making process can only be assessed retrospectively, based on a series of duly documented experiments. Hence a wide-ranging global debate on whether SALAs should be banned or regulated. This debate forms part of another, broader and earlier one concerning the role of machines in our world and the risks they pose. The difficulty with SALAs is that we have no knowledge of either their actual capabilities or the current state of research. Their use in ongoing conflicts gives rise to press releases, the content of which should be treated with caution.
A SALA is the integration, within a single system, of various modules (including an existing weapon system), which communicate continuously with one another to ensure optimum efficiency. Most aerial drones are now capable of taking off, landing and returning to their point of departure entirely on their own, as the procedure is automated. Navigation can also be automated where the mission (such as an attack on a known target) has been sufficiently defined. Take-off and navigation may rely on a form of autonomy – let’s call it programmed autonomy – but this will not account for any navigation or communication difficulties that might be encountered (particularly those devised by the enemy). This is where a higher level of autonomy comes into play, either through a conventional algorithm or via the intervention of artificial intelligence.
Normally, for the attack and opening-fire phase, a human operator intervenes and says: ‘OK, attack!’ because they have validated the target. When the drone loses its connection (due to jamming, for example), it assesses the situation and switches to autonomous mode. It has attack instructions and a list of potential targets stored in its memory. Using its sensors, it will proceed to the designated area and scan its surroundings. If it identifies a vessel or an armoured vehicle whose silhouette resembles one of the targets assigned to it, it will attack it on its own initiative.
Robots, like humans, operate within a cyclical decision-making framework—[5] —which improves reaction times and adaptability in changing situations. It is therefore key to survival during armed conflicts. It involves going through four stages of a loop: Observe, Orient, Decide and Act, in order to make the quickest and most effective decisions. The model is represented by the initials of each stage: OODA.
1. O: observe and gather information about the current situation and environment, and assess the situation.
2. O: Orientation involves combining the information gathered with the mission to be carried out. Orientation enables one to understand the broader implications of the situation and to shape the decision.
3. D: Select a course of action based on observation and orientation. In this stage, one must evaluate the various options and prioritise them in order to take action. Speed is crucial due to the dynamic nature of military situations.
4. A: Immediately implement the chosen decision whilst adapting to changing circumstances. The key idea is to recognise that the action taken alters the situation, but that other factors may also cause it to change.
The reason this model is called the ‘OODA loop’ is that when the situation changes, one must once again observe, orient and decide in order to rapidly adapt one’s actions to reality. The OODA loop embraces the chaos of action.
Thanks to its sensors, the drone constantly cycles through the OO stages and will suggest DA actions to its operator, or carry them out directly. During the decision (D) and action (A) phases – at least the important ones – the human is expected to intervene. But if the link is cut or jammed, the drone will follow its own rules, and, deprived of contact with its operator, it will do everything in its power to complete its mission. Even when the link is working, the mechanisms – which rely on AI – can be so powerful that the drone may give them greater weight than the instructions received from its operator. In fact, it is still difficult to predict what the AI will do.
Views on autonomous weapons
Opponents of the automation of weapons systems rightly fear:
– 1 that this will lower the threshold for entering into conflict by reducing the economic and political cost of using force, thereby increasing the likelihood of conflict. States would be all the more willing to engage in such conflicts if they were assured they could present ‘zero-casualty’ operations to their public.
– 2 That the interdependence and interaction of autonomous systems with one another might lead to uncontrollable strategic escalations.
– 3 That if warfare is delegated to autonomous systems, it will be impossible for leaders to bring it to an end (if there is insufficient control over the weapons systems, or if communication links are cut off).
NGOs argue:
– 1 That the use of autonomous weapons would render international humanitarian law (IHL)[6] meaningless.
2 – That ‘entrusting machines with the power of life and death’ amounts to ‘crossing a fundamental moral line’. For, they explain, ‘only human judgement and an understanding of context enable complex decisions to be made on the battlefield, such as distinguishing between soldiers and civilians, or assessing the proportionality of an attack: all of this would disappear with the use of autonomous robots, which would consequently contravene the laws of war’.
-3 That the use of autonomous weapons would reinforce a sense of impunity and encourage a loss of individual accountability. This would ultimately result in ‘greater suffering for civilian populations’.
The ICRC[7] suggests addressing the issue from an ex post perspective. A treaty requiring significant human oversight over the use of force, for the ‘critical functions’ of target selection and ‘target engagement’, would thus oblige States Parties to maintain reliable records of all data exchanged between their autonomous systems and human operators. In the event of suspected unlawful use of autonomous systems, States could therefore be required to produce these records to facilitate an investigation.
Attempts at regulation
The lack of control over SALAs is a cause for concern amongst military and political leaders, as any mishaps – which would be immediately publicised by the adversary – would tarnish the reputation of the country using them. Conversely, the risk of not possessing such systems, whilst facing an adversary (not necessarily a state) that does possess them, is equally worrying.
When it comes to the use of SALAs, policymakers are torn between the need to equip their armed forces with them to multiply their combat power, whilst keeping military costs in check, and the need not to alienate public opinion, both nationally and internationally[8] . Politically, we are thus witnessing hypocritical behaviour: anti-SALA statements and public calls for international regulation, whilst at the same time there is national encouragement for the development and integration of autonomous systems into the armed forces and a stalling of negotiations.
The autonomy of weapon systems challenges the principle of human control over the use of force. Autonomy and control are, in principle, mutually exclusive. The principle of control over the use of force stems from a compromise between military effectiveness and humanity from a moral standpoint. Thus, when an actor resorts to the use of force, they are expected to control it – that is, to set limits on it and not allow it to ‘run rampant’. This principle underpins all the norms of warfare and, as such, constitutes a ‘meta-norm’. These considerations point to the need to maintain meaningful human control over the way in which force is employed.
The idea of a moratorium on autonomous weapons was first raised in May 2013 at the Human Rights Council, supported by some twenty states[9] , following the report published in November 2012 by Human Rights Watch entitled ‘Losing Humanity: the Case Against Killer Robots’. Following various debates, it became clear that international humanitarian law (IHL) applied in full to SALAs and that the United Nations Convention on Certain Conventional Weapons (CCW) in Geneva was the appropriate framework for establishing regulations governing them.
Over the past thirteen years, the CCAC’s discussions have, in reality, made very little progress, consistently running into issues concerning the precise definition of autonomous weapons, their accountability, their oversight and the role of humans in the decision-making loop. Faced with the questions raised by autonomous weapons – which are ethical (can the decision to take a life be delegated to a machine?), legal (are such weapons compatible with the laws of war?) and strategic (what would be the impact of their development on the balance of power?) – the national delegations and NGO representatives were unable to reach agreement.
In 2016, on the initiative of France and Germany, a Group of Governmental Experts (GGE) on SALAs was established within the framework of the CCAC. It held its first meeting in November 2017. In 2023, a UN vote on regulating SALAs from the perspective of the ‘definition of autonomy’ was supported by 164 out of 177 countries (5 against and 8 abstentions). A report was due in 2024, but has still not been approved. However, a new text was approved by 128 countries on 5 September 2026, which simply emphasises the fundamental rules to be observed. It stipulates:
1. The fundamental principles of international humanitarian law (IHL), in particular the principles of distinction, proportionality and humanity, which remain fully applicable to SALAs.
2 – The principle of human control over decisions to use force is affirmed, without specifying the level of such control[10] .
This text represents a first step towards possible negotiations for an international treaty; however, the future course of the negotiations remains uncertain due to the deep differences that persist amongst the States Parties, both regarding the definition of SALAs and the content of a future legal instrument. This agreement provides a conceptual framework for identifying prohibited acts (without, however, defining them). Compliance with it appears difficult to reconcile with systems capable of operating without human intervention, particularly where the decision-making autonomy of such systems challenges the attribution of responsibility in the event of a breach of international humanitarian law, unless ‘ex post’ oversight is established, as suggested by the ICRC.
The debate on the regulation of SALAs and that on the safety of artificial intelligence address different issues at different levels. They must be kept separate.
To regulate autonomous weapons effectively, and to finally make progress after a decade of discussions, legislators and the public must understand not whether a weapon is autonomous[11] , but to what extent it is. Semantic drift has created a persistent and pernicious problem: terms are used interchangeably to refer to different capabilities[12] . Autonomy is not a binary, cut-and-dried concept; it is a continuum defined by the technology employed and by the human choices involved in the deployment of a weapon system. Policy-makers need a framework to understand this issue; perhaps the GGE will provide the right answers?
What do we mean by autonomy?
Autonomy reflects the degree of discretion a weapon system has in selecting, identifying and engaging targets. However, autonomy is not measured solely at the level of the drone; it must also be analysed when deciding whether to deploy it.
The diagram below illustrates these links
In 1980, the Pentagon deployed the Army’s Patriot missile system and the Navy’s Close-In Weapon System (CIWS), both of which were capable of detecting and engaging threats more quickly than any human, and responding to them autonomously. These two systems are fully autonomous, justified by the need to counter an imminent threat. Human intervention is limited to deciding whether or not to activate them.
It seems relevant to examine the process of deploying a weapons system. The first phase involves deciding to launch a military operation at the strategic level[13] . The second concerns the operational targeting process, which may involve artificial intelligence-based decision support systems[14] (AI DSS) recommending potential targets[15] . Thirdly, the decision to carry out the strikes is then taken by decision-makers, who will select the appropriate weapon system. It is the responsibility of human commanders and operators, in consultation with the relevant authorities, to take the appropriate decisions regarding the use of force before any weapon systems are deployed. In the fourth phase, the targets are transmitted to the weapon systems, which may be more or less autonomous in carrying out their mission. The final phase involves the opening of fire. Throughout these five stages, autonomy must be regulated – by a human engagement decision – and by monitoring compliance with IHL.
Military use of Palantir [16]
Specialised AI systems can be deployed in stages 1 (centred on the concept of ‘digitalisation of the battlefield’) and 2 (target proposals). Their use as a ‘[17] target generator’ makes it possible to broaden the range of information taken into account and to significantly reduce the analysis time (black-box effect) prior to the strike decision. Implementation is also greatly streamlined, as the system automatically transmits mission instructions to the weapons system. The use of such systems exposes users to several risks: decision-making time constraints (less than 20 seconds), standardisation of practices, and the reduction of uncertainty to statistical probabilities. As AI systems become capable of producing integrated recommendations, there is a danger that – in the name of maximum efficiency or under the pretext of available resources – the diversity of perspectives, timeframes and interpretations that traditionally characterise deliberation within the General Staff will be diminished.
At these two levels, an AI does not act in the same way as a subject capable of intentionality, responsibility or moral accountability. On the other hand, it can exert a significant influence on individual and tactical human decisions by directing decision-makers’ attention, prioritising certain options or making certain actions more likely. Its power lies less in its own capacity to act (although …) than in its ability to restructure the decision-making environment within which human actors deliberate under time pressure. By its very nature, AI tends to eliminate uncertainty, yet uncertainty is a prerequisite for vigilance, for the reversibility of reasoning, and for adapting to the uniqueness of a situation and its evolution. Even in situations of ambiguity, insufficient information or unclear instructions, AI tends to generate a proposal or an appealing hypothesis.
In stage 2, the advent of agent-based AI[18] carries the risk that, in the name of shortening decision-making cycles, the human role will be reduced to that of a regulator, or even an observer. Yet we must preserve the human capacity to understand the situation, make an informed decision, halt an action in progress if necessary, and accept the consequences of one’s choice.
With regard to drones, we must examine, module by module, their degree of autonomy and reliability in relation to their contribution to the system[19] . At present, the use of AI in drones is limited (due to constraints relating to weight and energy) and is primarily focused on optical recognition (navigation and targeting). It is not yet clear how large language models (LLMs) could provide operational value to an autonomous weapon system. Testing and evaluation would need to be significantly stepped up to ensure the reliability of a weapon system, a prerequisite for its deployment. For the time being, we are in the experimental phase.
Today, armed forces around the world are drawing crucial lessons from the current conflicts between Russia and Ukraine, and between the United States and Iran, which will shape the future of warfare. The development of even more autonomous weapon systems is probably inevitable. We must therefore consider realistic ways of regulating a trend that is already underway.
What avenues for negotiation are there?
The entire framework governing warfare – encompassing both its methods and its means – is based on the principle of control over force, which we have described as a ‘meta-norm’ governed by the political, military and international spheres. None of these three levels has full control over emerging technologies in the field of autonomous weapons, which call into question the very essence of control over force. Autonomous weapons herald a radical upheaval for humanity in the conduct of war. The fact that force is no longer controlled by humans has consequences for humanity. Whether considered in social, political or psychological terms, the nature of war is human. The concept of human dignity, understood in the Kantian sense[20] , may now be called into question when decisions to fire upon people are made by AI. From a legal perspective, the very definition of autonomy and the minimal human control over the use of force, whilst offering undeniable opportunities, does indeed create new vulnerabilities.
In a less tense geopolitical landscape, it would already be difficult to reach a new international agreement. The current turmoil surrounding global arms control makes this an even more distant goal. The rivalry between the major powers, in particular, but also between other actors seeking recognition who are vying for dominance in the field of military AI, is clearly not conducive to the regulation of autonomous weapons. States’ approaches to the issue differ widely.
For the time being, only the human dimension is beginning to enjoy a degree of consensus in the debate on the regulation of autonomous weapons. Two further dimensions appear to need to be addressed in order to fully grasp the challenges posed by these weapons and translate them into regulatory frameworks: political control of military force and international control of military force.
– Political control refers, on the one hand, to the pressure exerted by non-state actors on the government and, on the other hand, to the limits that the government imposes on the armed forces. States are, in fact, constrained in how they employ force by what they perceive to be the views and expectations of their populations. In this respect, autonomous weapons risk undermining democratic control by making the use of force less transparent and easier to carry out, until such time as a blunder triggers a reaction.
– International control over the use of force – which reflects both the balance of power between competing states and efforts at multilateral cooperation between them.
To these two dimensions, one must undoubtedly add the influence of the ‘military-industrial complex’ and the private sector on these issues. These two sectors differ from one another and shape the former: whilst defence industry firms are engaged in fierce competition for autonomy – which tends to undermine control over the critical functions of weapons systems – the AI research community is proving extremely reluctant to accept the military uses of this technology, preferring its civilian and peaceful applications. The latter speaks out publicly, exerting an influence on public opinion that reinforces that of NGOs.
France and Germany favour the approach of ‘controlled containment’ through the drafting of a non-legally binding code of conduct. Countries such as the United States and South Korea are banking on active participation in the negotiations to bring about a modest agreement, whilst Russia is adopting a strategy of obstruction to block any attempt at a multilateral agreement. China, a major exporter of drones, supports a ban on fully autonomous weapons systems, but stipulates that this should apply solely to their use on the battlefield – not to their development, production or marketing. The Chinese are “working simultaneously on technology and regulation in order to use international law as a means of constraining their competitors[21] ”.
In this context, the only option appears to be the conclusion of a non-binding agreement on best practice – a sort of ‘code of conduct’ (less binding than the TIAN[22] or the NPT[23], but which could lead to prohibitions on use) – emphasising legal obligations under IHL and rigorous unilateral commitments. This approach might address only a few of the on-board modules, such as targeting and the initiation of fire, and the conditions under which a SALA may be deployed by a military authority. However, whilst it attempts to address legal concerns, it does not make it possible to curb all the harmful effects of autonomy, nor to anticipate its effects on international stability. It would require unilateral measurement and control of the degree of autonomy, which could be audited ‘ex post’. At present, only a few States carry out legality reviews of weapons under Article 36[24] , and are under no obligation to share the results. This approach could be a suitable follow-up to the September 2026 agreement.
A very preliminary conclusion
After more than ten years of discussions, the codification of a positive obligation for human oversight of the use of force is at the heart of the debate. This general concept, which has gained momentum in the wake of the call for ‘meaningful human control’ launched by the NGO ‘Article 36’, appears to have been widely adopted by civil society representatives as well as by a growing number of States Parties to the CCW. Consequently, this principle is now widely accepted, both in academic literature and in diplomatic debate, not least because the United States and the ICRC have adopted it.
This framework for human control should translate into requirements for controllability right from the design stage of weapon systems, with regard to human–machine interfaces. To achieve this, it would be necessary to determine the ‘levels of human control’ employed – that is, the techniques and procedures required to maintain human control and accountability intact throughout the system’s operation – which amounts to limiting, or even eliminating, autonomy! It is likely that, if this concept were accepted, it would prove difficult to implement in practice, given that adversaries have deliberately disrupted the human–machine link, forcing the autonomous system to take over and making it impossible to provide evidence that the strike was authorised by a human. However, logs and recordings can be retained to prove that this was the user’s intention.
Researcher Elvira Rosert[25] proposed amending Additional Protocol I to the Geneva Conventions and the preamble to the CCAC to include the principle of meaningful human control. This approach would help to revitalise the GGE’s discussions.
It would be desirable to go further and enshrine an ethical imperative throughout the entire process of ‘the design, manufacture and use of SALAs’, upholding the Levinasian concept of human moral responsibility[26] in the face of systems capable of acting without operator intervention. This would enable us to move beyond purely legal, military, political and national perspectives.
[1] Bernard Merck, director adjunct al departamentului de resurse umane al grupului France Télécom, lucrează în cadrul grupului de aproape 20 de ani și a ocupat, pe rând, funcțiile de director general al Global One France, director de resurse umane al Oda (Pages Jaunes) și, ulterior, al EGT. În calitate de președinte al comisiei de informatică a ANDCP, Bernard Merck este unul dintre specialiștii europeni în domeniul aplicării tehnologiilor informației și comunicațiilor (TIC) în domeniul resurselor umane. Este autorul a câteva sute de articole.
[2] January 2025
[3] Although the figures should be treated with caution, Ukraine’s production capacity is estimated at over 8 million FPV (piloted) drones per year, whilst Russia’s is estimated at 1.5 million, including 36,000 Shahed/Geran long-range suicide drones.
[4] The Samsung SGR-A1 is a South Korean military sentry robot designed to replace human soldiers in the South Korean army within the Korean Demilitarised Zone on the border with North Korea. It is a fixed, autonomous lethal weapons system, manufactured by Samsung Techwin (which became Hanwha Techwin in 2016, then Hanwha Aerospace in 2018), a subsidiary of the Samsung Group.
[5] OODA is a cyclical decision-making framework developed by a military strategist, Air Force Colonel John Boyd.
[6] IHL: a set of rules established to limit the impact of conflicts on civilian populations and to minimise unnecessary violence.
[7] The ICRC recommends: / banning autonomous weapons whose effects are sufficiently unpredictable to prevent adequate compliance with IHL; / banning autonomous weapons designed and used to apply force against people; / strictly regulating other systems through limitations concerning targets, geographical area, duration, circumstances of use and means of neutralisation.
[8] For example, US drone strikes have been poorly received, both by American citizens, who saw them as an affront to their values, and abroad, where enormous anti-American resentment has developed.
[9] which have limited military ambitions.
[10] This level would concern targeting, the choice of weapon system, implementation and control.
[11] Almost every week, as interest in AI grows, new articles appear on ‘killer robots’, ‘fully autonomous weapons’, ‘autonomous weapon systems (AWS)’, ‘autonomous lethal weapon systems (ALWS)’ or, more simply, ‘autonomous weapons’. However, these terms are used haphazardly, as if there were a consensus on their definition. This is not the case.
[12] The rift that came to light this year between the Pentagon and Anthropic, Dario Amodei’s AI company, centres in part on the difference between Anthropic’s definition of ‘fully autonomous weapons’ and the term favoured by the Pentagon, ‘autonomous weapons systems’.
[13] In this regard, military headquarters can rely on tools for digitising the battlefield, which aggregate data collected by all available means (satellites, drones, sensors, human sources, field reports and intelligence).
[14] For example, US Central Command used the Maven intelligent system[14] during Operation Epic Fury to centralise disparate information streams on a single dashboard.
[15] Where targets are suggested automatically, there is a real risk that human decision-makers will approve them without providing any real added value, prioritising quantity over quality. (See Israeli criticism of Habsora)
[16] Palantir is an American company specialising in intelligence and data analysis. It develops software that enables intelligence agencies, the armed forces, police forces and businesses to gather information from multiple sources, cross-reference it and identify links that would otherwise be difficult to spot. $4.48 billion (54 per cent of revenue comes from governments). The co-founder is Peter Thiel. Main products: AIP (applied artificial intelligence), Gotham (intelligence and defence), Foundry (businesses and organisations). Maven is used by the US military to analyse large volumes of military data and help identify potential threats or targets
[17] We’re talking about Fire Factory
[18] The term ‘autonomy’ as applied to agent-based AI systems refers exclusively to functional autonomy, that is, the ability to execute, sequence or adapt certain operations without immediate human intervention. It does not refer to moral autonomy, the capacity for ethical deliberation, or independent responsibility.
[19] The navigation module and the target-locking module will not be treated in the same way, even though both rely on optical recognition.
[20] Kantian ethics is a set of universal moral principles that apply to all human beings, regardless of context or situation. Immanuel Kant refers to these principles as categorical imperatives, which are defined by their morality and their degree of freedom.
[21] As the American researcher Peter Singer explains,
[22] The Tian Treaty establishes a legal framework that renders nuclear weapons illegal for its States Parties, reinforcing the stigma attached to such weapons and supporting the humanitarian perspective, but it is criticised by nuclear-weapon states.
[23] The NPT creates a framework to reduce nuclear proliferation and promote progressive disarmament, but it allows the five nuclear-weapon states to retain their arsenals whilst engaging in discussions on disarmament.
[24] The Security Council may, at any stage in the development of a dispute of the nature referred to in Article 33 or a similar situation, recommend appropriate procedures or methods of settlement.
[25] Elvira Rosert, ‘How to Regulate Autonomous Weapons’, Prif Spotlight, 2017.
[26]: Emmanuel Levinas’s philosophy centres on the ethical question of the Other, with the face (the notion of face-to-face encounter) as a fundamental requirement.