Autonomous Weapons Systems: UN Expert Talks Facing Failure

Fault lines within the GGE

Even before the Russian invasion it was clear that differences of substance within the Group of Governmental Experts pre­cluded rapid agreement.

First of all, the GGE has failed to agree on a common definition of AWS. Most states support the proposal from the International Committee of the Red Cross (ICRC), under which an AWS is a “weapon system with autonomy in its critical functions” that is capable of selecting and attacking targets without human intervention. China, how­ever, would only include systems possessing the capacity to autonomously modify and/ or expand their strategic mission. Another group of states sees no need to define AWS at all, preferring instead to concentrate on the appropriate level, type, degree and form of man/machine interaction.

France took the initiative in 2021 and presented its own definition, which is also sup­ported by Germany. It distinguishes between fully and partially autonomous systems: Fully autonomous lethal weapon systems are capable of selecting their tar­get and initiating an attack without human inter­vention, as well as modifying their stra­tegic mission. Germany and France believe that this category of weapons sys­tems should be entirely prohibited. Partially autonomous systems, on the other hand, select and attack targets within a framework defined by human operators, but can­not make more far-reaching decisions on their own. This latter category, France and Germany argue, needs to be regulated in order to ensure that they are used only in accordance with legal and ethical princi­ples. Other countries including Japan have indicated their interest in deepening the discussion on this proposed definition.

There is also disagreement over the termi­nology used to define the required level of human control. While many participating states prefer the term “human control”, the United States prefers the concept of “human judgment”. The Americans under­stand human control as meaning direct manu­al intervention in the weapons system itself, whereas they prefer to control the effects of the weapons. The advocates of human control want to see control over the weapons systems and not merely their effects.

Another fault line is the arms race be­tween the United States, Russia and China, which is especially pronounced in the sphere of new technologies. China in par­ticu­lar has massively increased its military spending to modernise its armed forces, and intends to become “the world AI leader” by 2030. While China does support regula­tion of AWS in the GGE, its narrow defini­tion raises doubts as to whether it really wants to submit to regulation (presumably in light of its geopolitical ambitions).

The greatest tensions at the moment are obviously those between the United States (or NATO) and Russia. Even if Russia regards “meaningful human control” over AWS as indispensable, it opposes expanding the exist­ing international legal framework. As long as Russia insists on that point it is hard to see a path to agreement in the GGE. This raises the question whether other forums might offer better prospects of progress. Ger­many should prepare for such scenarios. For example the German Federal Ministry of Defence could prepare a national posi­tion on AWS in conjunction with the Ger­man Foreign Office.

Defining a national position on AWS

Unlike France, Germany does not yet have a national position on AWS. The German government’s AI Strategy of 2018 touches only superficially on military uses of arti­ficial intelligence. The only real marker is a commitment in the current coalition agree­ment to actively promote the out­lawing of “lethal autonomous weapons sys­tems that are completely beyond the con­trol of humans”. One reason for the lack of a national position might be found in the differences on AWS between the Foreign Office and Defence Ministry. Also Germany prefers reach agreement with important part­ners, such as the United States, France and the Netherlands, before taking action. In fact both the United States and France have already adopted national positions on AWS; it appears that Germany wishes to wait and see what the coming months bring, and espe­cially how the war in Ukraine progresses.

A national position could include the dis­tinction – made at the 2021 Geneva talks by France and Germany – between fully and partially autonomous systems. It must be noted here, however, that both France and Germany are still only talking about “lethal” weapons systems. This restriction is problematic for two reasons. Firstly, non-lethal weapons can also cause great suf­fer­ing to combatants and civilian populations. Secondly it is difficult to draw a clear line between lethal and non-lethal weapons.

A national position should also lay out the parameters within which the use of par­tially autonomous weapons systems is to be permitted. It must therefore necessarily also address the degree of human/machine inter­action and the concept of human control. Preparatory studies by the International Panel on the Regulation of Autonomous Weap­ons (iPRAW), in which the present authors play a leading role, could supply valuable pointers points in this respect. iPRAW understands human control to require “situational understanding and op­tions of intervention during attack”. Both elements must be ensured by the design of the weapons system (control by design) and implementable during its deployment (con­trol in use). The concrete extent of human control will depend on the specific context of utilisation.

In other words, a national position will need to address the various scenarios and identify the associated red lines. Deployment of AWS in urban environments could be subject to tighter restrictions than in purely military settings, for example by making it obligatory for a human to moni­tor their use and if necessary intervene. Human intervention would be less relevant with autonomous submarines for example.

In light of current geopolitical developments, however, Germany should remain open towards technological innovation and take account of the military benefits of autonomy in its national position. Speed, targeting precision and force protection are some of the examples that mitigate for the use of AWS.

The new geopolitical situation and its influence on Germany’s position on AWS

Calls for Germany to adopt a national posi­tion must also be seen in the context of the transformed security environment: In his landmark speech on 27 February 2022, Chancellor Olaf Scholz declared: “We will now – year after year – invest more than two percent of our gross domestic product in our defence.” He also announced that the government would put €100 billion into a special fund for the Bundeswehr, which has now received the Bundestag’s approval. These developments have two principal implications for a national posi­tion on AWS:

Firstly, they will boost existing defence projects, such as the Future Combat Air Sys­tem (FCAS). FCAS is a system of systems bring­ing together existing and new ele­ments; parts of the FCAS are also to be equipped with autonomous functions. An independent panel of experts on the responsible use of new technologies in the FCAS will ensure that there is no contradiction with the gov­ernment’s commitment to outlaw weapons systems operating outside human control. One of the central points of discussion within the panel is the concept of human control and its concrete operationalisation.

Secondly, in light of the large increase in defence spending announced by Olaf Scholz, there will need to for carefully scrutiny of whether procurements are actually neces­sary and sensible. New technologies might offer worthwhile savings on personnel, for example in the form of AI-based data analysis. Although that particular case does not actually involve AWS, the questions thrown up by increasing use of AI are simi­lar, however it is employed.

So the government’s commitment to out­lawing AWS does not imply a general rejec­tion of technological innovation in the military sphere. On the contrary, a national position on AWS should include a clear com­mitment to the necessity for research and development on military technologies, and name the benefits. Only once such a dif­ferentiated national position has been defined will Germany be able to participate actively in future negotiations in coordination with its partners.

Alternative forums for regulating AWS

NATO, the EU and the UN are the principal alternative forums for regulating AWS (see figure). It is also conceivable that other states might initiate a process outside the institutions altogether.

Recommendations for Germany

It is very likely that the GGE will be declared a failure after this year’s second meeting in July. If Germany is to live up to its com­mit­ment to work to outlaw AWS it needs to develop a strategy for a time after the GGE.

That means preparing a national position on AWS, which should distinguish be­tween fully and partially autonomous weap­ons systems and provide specific guid­ance on the requirements for human/machine in­teraction. It would certainly make sense to include non-lethal systems too. To ad­dress the refusal of the United States and others to accept the term “human control”, Ger­many could sign up to the ICRC ap­proach of speaking of “human control and judge­ment”. The planned German National Secu­rity Strategy could also contain a sec­tion on AWS and new weapons technologies, per­haps modelled on the Swiss Arms Control and Disarmament Strategy 2022–25.

Russia’s invasion of Ukraine appears to mark a paradigm shift in the realm of arms control. Universal forums have been suc­ces­sively sidelined, while regional institutions such as NATO and the EU gain in impor­tance. But AWS issues cannot be settled in regional forums, certainly not inclusively. Yet discussing them within NATO and the EU could at least coordinate partners’ stances more closely and in the medium (and longer) term enable them to enter broader future talks with a single voice.

Germany should also prepare for a dis­cussion on AWS in the First Committee, which the G13 states in particular might initiate. A coordinated approach by the NATO partners and the EU member states could contribute to finding broad majorities in the individual UN institutions.

Under current circumstances none of the potential forums guarantees success. But it would be pointless to allow that to paralyse the AWS discussion. The deployment of new technologies in the war in Ukraine un­der­lines the growing importance of auton­o­my in weapons systems. The announced de­fence spending plans should give occa­sion to reflect on the complexity of auton­o­my in weapons systems, and to develop a national position on AWS. Germany could show the world that it can be counted on to keep its political promises.