AI and algorithms increasingly drive modern warfare, especially when it comes to surveillance and targeting, but they are also hiding significant civilian harm.
In this podcast, Richard Kemp speaks with Luke Moffett, author of Algorithms of War, about the human impact of war and the dangers behind these automated systems.
They discuss why precision is not enough to limit civilian casualties, the issue of accountability when decisions aren’t in the hands of people, and why there’s hope even as we tread further down this dangerous path.
Available to listen here, or on your favourite podcast platform:
Luke Moffett is Professor of Human Rights and International Humanitarian Law at the School of Law, Queen’s University, Belfast.
Scroll down for shownotes and transcript.
Algorithms of War by Luke Moffett is available for £19.99 on the Bristol University Press website.
Bristol University Press/Policy Press newsletter subscribers receive a 25% discount – sign up here.
Follow Transforming Society so we can let you know when new articles publish.
The views and opinions expressed on this blog site are solely those of the original blog post authors and other contributors. These views and opinions do not necessarily represent those of the Policy Press and/or any/all contributors to this site.
Image credit: Mohammed Ibrahim on Unsplash
SHOWNOTES
Timestamps:
01:24 – Why is Zamarai Ahmadi’s story important?
07:11 – What is the difference between weapons being precise and being accurate?
11:51 – Why didn’t the Geneva and Hague law prevent World War 2, and what did the 1949 conventions change?
17:31 – What is the danger of believing in magic when it comes to winning wars?
21:26 – How does accountability work when it comes to LLMs?
25:23 – How did AI affect the scale and speed of bombing in Gaza?
30:41 – What’s the significance of the UN meeting in November 2026?
36:09 – How did Battlestar Galactica inspire you to write this book?
40:51 – How bad could things get, and what gives you hope?
Transcript:
(Please note this transcript is autogenerated and may have minor inaccuracies.)
Richard Kemp: You’re listening to the Transforming Society podcast. I’m Richard Kemp, and on this episode I’m joined by Luke Moffett, professor of human rights and international humanitarian law at the School of Law, Queen’s University Belfast. In his new book, ‘Algorithms of War: The Human Cost of AI in Conflict’, published by Bristol University Press, Luke delves into the promises of artificial intelligence or AI, its use in modern warfare, and the fatal consequences that come with this.
Popular media franchises like the Terminator films and Battlestar Galactica show future visions of a robot class that will one day rise up and destroy humanity. This future is presented as inevitable, that AI will continue to learn and learn until finally it turns on us all. In ‘Algorithms of War’, Luke shows that while killer robots such as drones may already be a reality, the devastating effects they have on military and civilian lives come down much more to humanity than a lot of sci fi, or even the news, would have you believe.
For this book, Luke draws on testimony from conflicts around the world and argues that humanity and a people cantered approach are essential to ensure accountability and strengthen civilian protection and the legitimacy of military operations as AI continues to reshape the battlefield. Luke Moffett, welcome to the Transforming Society podcast.
Luke Moffett: Thanks. Thanks so much for having me.
RK: Thanks so much for coming on. Thanks so much for making the time. Your book begins in August 2021, in Kabul, Afghanistan, with the harrowing death of 43 year old Zamarai Ahmadi, a civilian who was killed by US drone strike along with his children. Could you describe what happened to Ahmadi and his family, and also explain why you chose to open the book with this story?
LM: Yeah, so this strike happens in the very chaotic pull out of Western forces from Kabul in August 19- in August 2021, where the Taliban had effectively taken over the whole country after, you know, end of hostilities peace agreement with the U.S. and Western forces. And so the National Army of Afghanistan collapsed. And you had a situation where they’re trying to evacuate people who are working in the embassies and others, Afghans who worked with Western forces, because there was concern that they would be killed or tortured if captured by the Taliban.
So it’s a really chaotic, really stressful and dangerous time because not only are the Western forces trying to evacuate, you know, over 100,000 people in the space of a few days, but they’re also concerned that ISIS is very active and is looking to carry out attacks against U.S. forces. And this does happen. There’s a, you know, massive suicide bomber attack, and it kills over 160 people at the airport as this evacuation is happening.
And so there’s a concern that it’s going to happen again. And so, U.S. forces are scrambling to try to identify where this next attack is coming from, and they get this intel that it’s going to be a sort of a car bomb. So it’s potentially going to be even bigger. And so they identify a car that’s acting suspiciously.
It’s driving past ISIS headquarters. It’s going on this route, which is unexplainable. It turns out that this is actually the car that’s owned by Zamarai Ahmadi, who is an aid worker working for a U.S. NGO. And the U.S. forces are watching him for eight hours with, you know, drones, and they finally see him going into, you know, a street and a compound.
So it’s a walled house, really. And so they think this is the best point to hit the car with a missile, because it will limit any sort of harm to civilians, than if it’s driven into, like, a marketplace or driven towards the airport. So they carry out this strike believing that this is an ISIS bomb car, because they saw the occupant loading it up earlier in the day.
And, as we know now and after, you know, weeks of an investigative journalists sort of poking the U.S. military to reveal what happens. It was actually a civilian and seven children were killed, some of whom were actually in the car. And what looked suspicious from thousands of feet above was actually just an aid worker going about his work.
He was driving past, he was going to his, you know, different colleagues houses, you know, helping them with different tasks. He was filling up his car with water because there was no water in his neighbourhood. So it’s very much a family man and also like a really good colleague. And he had, him and his family just got visas to go to the U.S..
So it’s sort of, it’s this contrast between trying to see and know everything and being the most powerful and biggest military in the world. That you make fundamental errors and, you know, looking back, you know, the analysis that’s been done about how the U.S. has, you know, carried out drone and airstrikes in Afghanistan over 20 years.
Half the people, half the civilians that were killed were killed because they weren’t identified. They’re misidentified as like Taliban fighters or ISIS fighters. And this is a problem that we’ll talk about this, but like artificial intelligence is just this long evolution, it’s not really a revolution, in how we see and try to understand war because, you know, it’s often talked about the fog of war.
It’s chaotic. You cannot see what’s actually going on the ground. The enemy will also try to, disrupt or distract or camouflage what they’re trying to do. And so it’s very difficult, especially working in urban areas and fighting against like, you know, rebel, guerrilla terrorist groups as they won’t carry weapons openly, they won’t be driving around using tanks.
So it’s very difficult to then to target an enemy fighter. That doesn’t mean that you can just target whoever they want. There are legal rules, and we’ll talk about this. But this story was a just a real contrast because in the aftermath of carrying out the strike, you know, the senior members of the U.S. military and also the president talk about this being a righteous strike and about, you know, we will take out ISIS, whatever threatens our forces.
So it’s just it’s how things, how you can get things so badly wrong. And for me, I believe this is one of the first uses of AI by the US military in a killing strike. And it’s on the basis that the U.S. Air Force Secretary says three weeks after we used AI to carry out a signature strike on a target to stop ISIS carrying out an attack on us.
And this is like the only strike in that time period where they did this in Afghanistan. So the two things match. And there’s been a lot of denial. And, you know, sort of hiding what’s going on. And so really, the book is about trying to improve transparency in how these weapons are used because they’re not clean and they’re not always accurate.
And we may not see the effects, but they’re definitely felt in people on the ground who are subjected to these weapons. And so really what I wanted to do in the book is this is a really complex, difficult, you know, obscured area like there’s very little oversight of how these weapons are used. And so I wanted people to be able to understand. It’s not an academic text.
It’s trying to really help people improve their AI literacy and how we’re using this new technology and the human cost that results.
RK: You talk in the book about the precision of these, of these weapons, like how they are precise. They are, they can precisely hit what their users want to hit. But they’re but there’s a difference, isn’t there, between like, precision and actually getting things right. Can you talk a bit about that please.
LM: Yeah, definitely. And this is not a new issue. In the first Gulf War in the 1990s, the U.S. was targeting what they thought were the leaderships of Saddam Hussein’s government and military by those who were had a satellite phone signal. And so they were bombing these, they were bombing satellite phones really. The same way that, you know, the bleepers were blown up in Lebanon a couple of years ago, that they believed that Hezbollah were using those bleepers.
And in actual fact, you know, you had a lot of people who were handling those like doctors, civilians, and they were, you know, injured as a result. So the difference with precision is that you can precisely hit a target within, you know, five, ten meters, you know, during the Second World War it was like trying to hit a target within five miles and like that’s a big difference.
The difficulty is what are you hitting? And so it’s the difference between, you know, are you accurately hitting the military target or are you precisely hitting what you think is a target. And so we haven’t sort of narrowed the gap in understanding who is the enemy. And it’s also like what has been going on when, you know, countries bombing other countries and like, you know, air strikes or drones is they’re not dropping a bullet on somebody’s head.
They’re dropping like a 2,000 pound bomb often. And these cause massive effects, like, and I grew up in Northern Ireland in the end of the troubles. And most of the villages and towns where I grew up had 1 or 2 bombs that went off, and these were 500 to 2000 pound bombs and just, just single bombs. And they completely wiped out town centres.
So if you’re dropping hundreds of these, thousands of these every day, this isn’t just simply, you know, hitting individual targets or hitting people in very densely urban environments. You’re blowing up whole streets or whole neighbourhoods with 1 or 2 of these bombs. It has a massive effect. So even if you are accurately hitting an enemy soldier, you know, you’re also going to pull in and injure and kill dozens of civilians, hundreds of civilians in one strike.
And so precision isn’t accuracy. And it doesn’t mean that you’re not going to kill a lot of civilians. And these are human beings. These are people like ordinary people who have children or jobs, that have dreams and aspirations, that they’re living their lives, and that’s being snatched away from them by instantaneous decisions that are being fed by maybe mislabelled or, you know, incorrect data about who this target is.
And sometimes it isn’t even, you know, oh we’ve identified this exact person who’s a leader of this terrorist group or leader of this, you know, in military, it’s often based on how they’re moving, where they’re associating and what phones they’re using. And these are sort of inferences. They’re not facts. They’re suppositions. They’re a way of sort of indicating to some extent that this person possibly is a target, but it’s not definitive.
That’s what’s really dangerous, because then anybody could fall into, you know, walking in a certain street at a certain time without realizing and making themselves a target.
RK: That was coming up a lot during reading your book. Just the terrifying nature of just like, especially when the government can, in this case a lot of the time it was the US government being quoted when I was reading this particular bit, but just like we did it legally, therefore, end of story. Full stop.
Basically that was, that’s a terrifying kind of just like a blocker that you can’t get past.
LM: One thing I would say to that is that it may be legal and you may be able to justify it under like the laws of war, but strategically you will lose the war. The U.S. learned that in Afghanistan and Vietnam, where you can’t just bomb everything you think is a target because you will lose the hearts and minds, the support, both of civilians, but also the local government and support.
And so that’s why the U.S. government from the early 2020s was trying to stop the use of, like certain close air support, even from 2009 onwards, in Afghanistan of using like large scale bombs in supporting local troops as they move forward because it was causing massive civilian harm. And that plays badly in the media for, you know, affects the reputation.
And also like for professional soldiers, if you’re calling in an airstrike and you kill those civilians like it weighs heavily on you, you know, it’s like it’s traumatizing the fact that, you know, you got things wrong and you’ve killed somebody else’s children. So like, it’s not just the law, the law isn’t everything.
RK: You mentioned a little bit earlier about the Second World War. And I wanted to ask you about that as well, because you talk about this when talking about the laws of war that the Second World War, which killed between 21 and 25 million soldiers and then also over 50 million civilians that I think, I think you said that’s 3% of the world’s population at the time, which just like, I know we’ve all heard, we’ve all, you know, gone through the school system and heard about the Second World War over and over again.
But just that, that was I don’t know, it was it was interesting for me to be shocked by that again after all this time. But while all that killing was happening, also the Geneva and Hague law already existed. So why did the law fail to prevent all this killing? And what did the later in 1949 conventions change?
LM: Yeah. So in international law, we have the laws of war, which actually the first Geneva Convention goes back to the 1860s. And if you know any history, you’ll know about the past 160 years has been filled with hundreds of wars and, you know, probably hundreds of millions of people dying. So before the Second World War, states tried to learn from the First World War, you know, they banned chemical weapons, they put better protections for soldiers who were captured and made prisoners of war.
And they also tried to limit the way in which aerial bombardments, you know, air strikes could be carried out. That didn’t really work. But when it came to the Second World War, it’s in a situation of countries moving to total war. They’re fighting for their survival. A bit like Ukraine is where their whole industry and society is put on a war footing.
That’s, you know, everybody has to contribute to fight, or support the war effort. And so sometimes, you know, people were thinking about how do we defeat the enemy. We don’t want to be our hands tied by the laws of war. And so, like the US and the UK carried out massive carpet bombing of German and Japanese cities, like hundreds of thousands of people died even before nuclear weapons were used.
And it was a way to sort of force those governments the German, the Nazis and the Japanese imperial forces to stop fighting was trying to ensure peace. But it had no effect, like it was a strategic failure it was that this sort of large scale bombing would change opinions. And like we see it today, where Russia’s constantly bombarding Ukraine, and it’s a weapon of terror to sort of bombard people until they change their minds.
And the, the law, the law has its place. But, you know, law is almost like, like I talk about a lot in the book is like it’s almost like a mindset is that you have to follow the law and see the value in the law. And so after the Second World War, you have the four Geneva Conventions, and they add this fourth convention around protecting civilians, particularly civilians that are living under occupation.
And so a lot of that is talked about today in Palestine in the West Bank and also in, you know, Russian controlled territories in Ukraine, because these are certain rules and certain basic minimums, you have to respect civilians. So you can’t just take their property, you can’t just force educate or, you know, conscript them. There are certain rules.
The difficulty with is then enforcing those rules and so often those rules don’t get enforced. We see sometimes war crimes, war crimes trials, such as like what happened at the international criminal court. And but those are often far and few between. It’s up to states to sort of buy in and follow those laws. And it’s not something just because the law is there.
Like all states came together and negotiated these treaties in 1949. They came back in 1977 and added more rules because they wanted better protection, not only just for those they were fighting, but for their own soldiers, in particular and their own civilians, because they didn’t want them to be subjected to this, because, you know, you had like French, Vietnamese, American forces in 1977 and also after the Second World War who’d lived through the experience of war, had been prisoners of war, who had been civilians who had lost family members.
They wanted to sort of ensure there was some sort of restraint in war because, you know, human beings, we can use so much violence against each other. And at the end of the day, you know, you can’t win everything at the end of a gun, you have to have some sort of room for peace.
And if you fought a war where nobody is willing to forgive you, then you know, like that’s what law is supposed to do. It’s supposed to have a restraint that there is a possibility for peace at the end of hostilities. And so we didn’t see large scale use of nuclear weapons in Korea or Vietnam. We haven’t seen nuclear weapons used since.
We also haven’t seen much use of chemical weapons because we have seen that strategically and, you know, practically they aren’t that effective. But we have seen large scale bombardment using artillery and air strikes and more recently, drones, which are increasingly causing, you know, you know, thousands of deaths every year. And so law has its place but at the end of the day, like technology, it’s just a tool.
It’s how humans use it. And, you know, try to limit and restrain its use because we are flawed creatures. And really we have to think beyond just the next war and winning it, to think about, you know, what is the legacy we’re leaving for humanity and also for, like our children and grandchildren, like, are we making the world more unsafe for them?
And recent wars have shown that we are increasingly globally connected. And so, like starting bombing petrol infrastructure in the Middle East has a worldwide effect. And we can’t, you know, we can’t just ignore rules that are there that should have prevented those sort of wars. They have consequences. And, you know, we have to live with those consequences when people break them.
RK: You quote Craig Martell, the head of the Pentagon’s chief digital and artificial intelligence office, who says “Everybody wants to be data driven. Everybody wants it so badly that they’re willing to believe in magic.” You warn about this belief in magic, or hype, when it comes to the complexity of fighting and winning wars. What do you mean by this?
LM: Yeah. So the author of ‘2001: A Space Odyssey’, you know, Arthur C. Clarke, he says that, you know, when it comes to technology, when we don’t understand it we’re willing to believe that it’s magic. And so, like, we have these systems like even like how like ChatGPT or Claude works like most of us have probably used it.
We could not explain how it is able to, you know, write an email for us based on like a few prompts. And so it’s, it’s the hidden nature of how those systems work that we’ve put, we’re putting a lot of our own material into these systems. And we don’t know then where is that going, how that’s been stored.
And this has been a problem for decades where we’ve had social media. We’ve put up our own photos, and those photos have been tagged, and the data behind those locations have been used to then train algorithms, which you’re now seeing being used in like Gemini and all this other range of. So this is the problem is that the more this technology advances, it’s getting beyond our grasp of how it works.
And so we’re thinking, well, this is this technology is great because it can magically, you know, write, you know, a 2000 word essay that I need for, you know, school tomorrow without me having to actually think of how it works. And that’s really dangerous because again, it’s about us losing control and power of all that, but also it’s about how we hold account, how these systems work.
So, some of them aren’t all that great and, you know, aren’t all that accurate in terms of like so I, I’ve, I found some old family photos and I’ve been using like the very grainy using AI to sort of to colourise them and to make them like, you know, look like somebody took them today. But it changes the face, the person’s face.
And so it doesn’t really look like them. And so these subtle changes which are, you know, are based off, you know, millions of other photos that’s been used before in this sort of work, it’s the bias that these systems have. And so all this here together causes these sort of cascading problems. And so like, there’s been reports that, you know, one of the generative LLMs, you know, Claude by Anthropic, was being used by the US military in how it was carrying out strikes in Venezuela.
And so some of those strikes, you know, earlier in 2026 killed civilians. That’s been very difficult then to understand what was the process being outside the military. But also what role did AI have? We see this in an even bigger scale in the war in Iran, you know, a few weeks later where you had schools being hit and, you know, hospitals, like, you know, why were those made targets?
There’s been very little transparency of then what role AI played in speeding up the way in which these targets were hit, but also then holding to account and making sure it doesn’t happen again, where these systems are used. So it’s a problem of, well, if you are using, you know, systems which you don’t understand, then how are you going to comply with the law.
And like if you’re just believing that this is all just magic and just wonderful, you know, what are the dangers with that? Like there’s a real risk to both, you know, not just the people on the ground, but for militaries. Like there’s risk that, you know, we’ve seen it in the past where, you know, you have the Patriots Missile Defence System, which the U.S. has and been using in the Middle East and also Ukraine.
It shot down British and US planes at the start of the war in Iraq in 2003. And so these automated systems don’t always recognise, you know, what is the target all the time. And so just hoping that it works and believing it’s magic is really it’s really dangerous for everybody, not just enemy and, you know, civilians but also for your own soldiers.
RK: The accountability issue then with this presumably the LLM, Claude, ChatGPT, they’re not accountable when we when we have these accountability courts, how does that work?
LM: Yeah, we have to see. At the end of the day, you know, we don’t put on trial objects. You know, we didn’t put on trial the pens that signed the death warrants in Auschwitz. We put on trial people and not like abstract legal things like companies. So individuals could likely be implicated in criminal trials.
So for the commanders, the generals that are using these systems are choosing and buying and helping funding these systems. They have, you know, legal responsibilities under domestic and international law. In terms of how they, you know, procure them and use them, etc.. So there’s a risk for them. For companies, you know, they could face being sued.
And so we’ve seen already, you know, there’s been a whole range of cases brought against, you know, social media companies, about how they mis-held data and different things, but also in terms of like, with Anthropic, they’re providing a service. And so there’s, you know, we’ve sort of gunmakers in the U.S being sued by victims of, you know, gun violence that they aren’t completely immune, they’ll not likely face criminal trial, but they could likely face, you know, a mass civil action.
We have thousands of claims against them. And so, you know, this will probably take decades, it will, for that to sort of manifest the difficulty is we don’t understand, you know, from like a civil society and external perspective, how militaries are using these systems yet. They’re being very secretive. And obviously they don’t want to give an advantage to the enemy, but there’s a lot of denial and for civilians on the ground, you know, they’re burying their children, you know, or they’re left with, you know, the breadwinner of their family being killed, and they’re, you know, stuck in poverty because they’ve got nobody who’s earning enough money.
And so, or been they’ve lost all their homes and possessions and they’re being displaced. So, like, this is having a real world effect on, and ruining many people’s lives. And, you know, that exposes people to be more vulnerable and creates, you know, refugee populations. It also increases like support for, like rebel groups. And so that has a real destabilizing effect.
Like not just in the Middle East or wherever’s affected. Like, you know, these chickens always come home to roost, you know, and so, like, it’s easy enough to spread this, these things around the world and to test them out. But, you know, the U.S. has been dominant in using drones for the past 25 years. But we’re seeing that, you know, about over 60 rebel groups at the moment are using drones.
We see the Houthi’s are, you know, destroying parts of the capital of Saudi Arabia. You know, it’s airports and damaging key, you know, gas and oil pipelines that are, you know, the international global fuel market depends on. So you know, this the problem is with all these systems is the danger of proliferation.
And this isn’t new, you know, in 1899, when the discussions around like the Hague laws were being negotiated, you have like the Tzar Nicholas the second calling on like states to come together and stop spending so much money on weapons and start like trying to limit and prevent future wars because otherwise, you know, this is in 1899 said that the next century is just going to be a devastating and a wasteless wars, that we’re all going to suffer.
And it was true, that’s what happened. Russia itself, you know, in the First World War and Second World War, you know, committed atrocities and, you know, broke a lot of rules that it you know, helped establish. As did all the other states that signed up to it. So this is the problem that there is this, it’s almost like a seductive pull to invest in technology, because we’re worried about the enemy using that technology against us.
And it’s a real slippery slope that ended up in like World War Three in how this technology is used.
RK: You discuss the speed and scale of airstrikes in Gaza, an area half the size of New York City. I never heard it described in that way before, that really puts things into perspective. 6000 bombs being dropped in just the first week following Hamas’s attack on Israel on the 7th of October, 2023. In one year over 25,000 tons of explosives were dropped, which is more than the equivalent TNT explosion of the nuclear bombing of Nagasaki.
How far did AI make this scale and speed of bombing possible?
LM: It’s easy to understand the speed and scale of how AI is like supercharging these strikes because you can look back at past, you know, conflicts in Gaza where Israel was dropping maybe 50 to 60 bombs a week and in 2021 and in 2014. So, and it’s because they have to use like dozens of, you know, intelligence officers and, you know, military officers to pick out the targets, you know, so that you’ve got legal obligations.
They have lawyers that they go through what their intelligence is, and you know, where likely, you know, Hamas bunkers are or, you know, tunnel networks, you know, so where is the military objective they need to bomb. With AI and systems that the Israelis were using, like Lavender and Where’s Daddy?. They were using like inferences. So the person’s identity and relationship, like the AI system was able to sort of from what we’ve, you know, like investigative journalists like Yuval Abraham and others have sort of pieced together from interviewing IDF officers is how these systems like Lavender have put like different data points together to make a target profile.
So say you’re using a certain phone or walking a certain way, like the way in which it’s explained is like the IDF knew everybody who was in the building based on their phone usage. So they knew that, you know, in one apartment there was two children, and so they could bomb the other side of the apartment. The difficulty, and this is the problem of using high explosives, like 2000 pound bombs in urban areas and in high rise buildings when you drop like, unsurprisingly, a large bomb on a building, it causes its collapse.
And it wasn’t just in Gaza we’re seeing that, we’re also seeing it in Lebanon, which is being heavily bombed by the Israelis as well. And so the AI system was speeding up the scale on which those inferences and those targets could be generated that you weren’t just generating, you know, 50, 60 targets a week. You were generating, you know, thousands, if not tens of thousands every day.
And so, like after the genocide that Hamas carried out, you know, on the 7th of October because that’s, that’s what that was. The Israelis turned to using violence in Palestine and I would say increasing the parameters of what is a target through using AI and using this scale got them also accused of using genocide, because the scale of which the weapons that they were using, you know, that’s currently an investigation at the international criminal court and also an ongoing cases at the International Court of Justice around genocide.
So the scale on which they used was only possible because the AI systems could do it at speed. And for like the operators who were like, you know, flying the drones, the weaponeers who were dropping the weapons, dropping the bombs, you know, usually in that sort of process of, it’s sometimes called a kill chain of where you’re verifying, identifying the target you’re carrying out, what’s the best way to attack the target, you know, using intelligence, etc. often somebody who’s got the intelligence there would be, you know, 20 or 30 of them working a target, and they would have weeks to analyse that target.
What Lavender was doing, it was compressing it down to 20 seconds, where an operator had 20 seconds to decide whether or not this person was an Hamas fighter or not. And also, you know, some of them were saying, well, it was on the basis well, are they male? That’s more than between the ages of 16 and 60. Yeah, likely Hamas. And that’s not how the laws of war work.
A person is civilian, like everybody is presumed to be a civilian up until such time you take a direct part in fighting. And what that means often is that you have to be like firing a weapon or directing attacks or involved in some way. Being a male of military fighting age is not a target. And so this is the whole problem that we saw, you know, 75,000 people being directly killed by these bombs.
Now, like I said at the start, is when you bomb an urban environment, you know, it causes massive damage to buildings but also infrastructure. So like there’s no clean water, there’s no supplies getting in. So I’d imagine that the indirect people being killed are like, I’ve, I’ve seen videos of, you know, of children in hospitals and, and you know, newly born, you know, premature babies in incubators dying because there wasn’t electricity or oxygen.
And that’s, that’s the, the consequences of these bombings of hospitals and the effect it’s like the, the most, the, literally the most vulnerable people in the world are dying not directly from being hit by a bomb, but the effects of the bomb has days and weeks and months after. And so these AI systems like it’s dystopian.
It’s not like the Terminator. It’s how human beings are using these systems to scale up and do things like, you know, it’s carpet bombing for the 21st century. Like, you can’t really use nuclear weapons now, but you’re using weapons which are effectively having the same effect.
RK: Wow, thanks, Luke. So there is a UN meeting coming up in November 2026. Can you, can you tell us about the details of this meeting and kind of what it what implications it has for the future.
LM: For nearly the past 15 years, NGOs like Human Rights Watch, and that evolved into, you know, the Stop Killer Robots campaign has been advocating that we need new international law around killer robots effectively. So AI systems like drones, which can go and hunt the target. Now when the negotiations started, you know the pre negotiations started on this protocol, this treaty which will actually really start next year.
It started in 2014. And at that stage you know we saw you know drones were being used in the Middle East. But we didn’t see the artificial intelligence explosion that we have now. And so there has been no real killer robots until very recently. So and there’s been a couple of strikes where you have armed drones going in and attacking the targets based on does it look like a tank?
Does it look like a military convoy moving ahead? And so basic, you know, the drones camera has like object recognition tools, algorithms based on board. So it’s attacking things, which it looks like that shape. The vast majority of drones that are used in Ukraine are human operated. But a lot of those drones also faced electronic interference in other words, somebody is sending the same signal that blocks the radio connection to the operator.
So the person flying the drone loses control and the drone often falls from the sky. And so what they’ve been building in to these drones is sort of autonomous guidance systems. So in the last mile or the last few hundred meters, it identifies the target, so a tank’s driving along a road and even if it loses signal with its, the guy who’s controlling it, it’ll still hit that target.
So we’re increasingly seeing that autonomy. Now Russia in July launched similar drones, but the object it was going to attack was anything with fuel tanks. And so it was trying to attack at the petrol station. Civilian petrol station. And it flew round the corner of a wall too fast. And as civilians were running away from this drone coming in, three of them were killed in Zaporizhzhia.
So this is probably the first documented incident of like an autonomous drone, a sort of a killer robot killing civilians. And so the efforts in November are to see if there’s going to be consensus. So if all 194 countries in the world can come together and negotiate a treaty. So we’ll see in 2027 if the negotiation could go ahead.
In the past, in 1996, they tried to do something similar in banning landmines. So like mines which somebody, anybody stood on would blow up. And the reason why they wanted to ban those is tens of thousands of people were dying every year from mines that were, you know, buried in the ground decades before, like in a place like Vietnam, Laos, Cambodia.
And so the campaign that started around that finally got them anti-personnel mines, so mines that effect people banned in 1997. Most states signed up to it. Then in a few years later, countries came together and banned cluster munitions, which are a bomb that’s made up like 600, 700 tiny little bombs. About 20% don’t explode. So if you’re dropping millions of these, you have hundreds of thousands of effectively land mines.
And they’re very brightly coloured and so what was happening is children were going along and picking up these yellow objects and then blown up and killing and maiming them. So the efforts now is even if we can’t get an international treaty within the UN, what may happen is that other countries, there’s about 120 countries that want to sort of ban or regulate these systems.
And most countries do so like the UK, France, you know, even China wants to regulate these weapons. The U.S. and Russia are a bit more, let’s say, not on the same page as everybody else. The U.S. has been involved for a long time in negotiation. But, you know, under the current government, there isn’t too much appetite for regulating artificial intelligence.
So there might not, so for the UN process, you need everybody to agree effectively. Or at least 66%, you know, two thirds to agree. If that doesn’t happen, they’ll move to this separate process for a separate treaty. Like there was with landmines like there was with cluster munitions and those like the cluster munitions and the mines treaties have been incredibly successful.
You know, instead of having tens of thousands of people dying from each of those weapons, it’s down to about a thousand a year now. So like, that’s a massive, you know, like, it’s saved probably over a million lives in the past few decades. So, you know, it’s, it can change things. Law can change things, but it’s up to states to be willing to do that and it’s up to all states.
And so, you know, these things can seem so far distance. But like the reason why we banned landmines and cluster munitions is because people around the world came together. It wasn’t just states. It was, you know, people writing to their, you know, their local elected representative, putting pressure on their governments, civil society coming together and organizing and saying that, you know, in war there has to be limits.
And so these weapons, they don’t win wars successfully. It means civilians just bear the costs, you know, more than actually is necessary. So I think there is, you know, a process to sort of limit the worst things that could happen with these weapons as they’re emerging now rather than dealing with it in 20, 30 years where we’re seeing hundreds and thousands of people being killed by killer robots.
RK: I wanted to talk about the remake of Battlestar Galactica from the mid 2000’s, because I was, my friends and I, we’re very big BSG nerds. And when I saw that mentioned at the beginning of your book, I was thrilled. And, and you said about how Battlestar Galactica, in part inspired you to want to write this book.
And I was just wondering, like, could you talk about the story of Battlestar Galactica and also and therefore why or how it inspired you to write this book?
LM: Yeah. So for those who haven’t had the pleasure of watching the remake of the Battlestar Galactica, which was first broadcast, you know, nearly 20 years ago. It’s effectively humans are living across these different planets, and they have at some point, they created robots and the robots overthrew them, and they had this massive, brutal war with these robots called the Cylons. Now where the TV series starts, it is effectively this wiping out of humanity and everybody fleeing into space to try to avoid these robots from killing them.
The difficulty is the robots have advanced their technology, so some of them actually look like humans. And so that really, I think Battlestar Galactica is the best for the reality of surviving in war. And also like because they live under occupation, like some of them are captured, and they also like they talk about using chemical and biological weapons against the Cylons and like the, the, the morality and legality of that.
But when they’re under occupation, it was the same time that the U.S. was occupying Iraq. And it puts the good guys, you know, the main characters, they were actually thinking about making themselves suicide bombers and what that meant and what that meant for, you know, other humans who were collaborating with occupiers. So there’s a lot of like, really a lot of really very human, and difficult issues with war, but also, like with technology, the difficulties of technology and how we use it.
And so like the Battlestar Galactica, the ship itself, you know, the reason why that ship survived the war was because they weren’t connected, you know? And so I think talking like some civil society people today in order to protect themselves from cyber attacks, they have completely closed systems that aren’t connected to the internet. They do not do online video calls around sensitive stuff, they do things in person.
So they’ll fly to meet each other and it’ll be handwritten documents. So like technology gets us so far and you know it’s seen massive advances. But it’s also like the risks it exposes us to now. And like I even think like you know space seems so like, you know, futuristic and stuff. But like my internet is satellite based because I live in a rural area where there is no broadband connection.
And there’s increasing amount of, you know, satellites in space that we rely on for GPS and banking every day and even for like for planes to fly. That stuff is incredibly vulnerable to cyber attack or to one of those satellites crashing into each other and causing massive amount of debris. So it’s, you know, Battlestar Galactica was trying to make that connection between the future and technology, that, you know, it doesn’t matter how things change or where it is, as human beings, what does it mean to be human and live through such, you know, such upheavals that we’re currently seeing at the moment?
And how do we remain human in that? So like in the book, I you know, I there’s some things I quote from Battlestar Galactica, you know, like one of the things they say, and this is like a common, philosophical thing is everything that’s happened before will happen again. And I think writing this book and looking back in history is that, you know, these lessons about how we try and restrain war.
Ultimately, we have to learn them the hard way that we don’t restrain ourselves and regulate these weapons early enough and we’re left to live with the consequences. So, like, you know, one of the things like this is a very late 90s, early 2000, quote, but like the Manic Street Preachers would say, you know, if you tolerate this, then your children will be next.
And I think when it comes to how we regulate AI now, like we’ve seen it with big data, the failure to, you know, properly regulate and to tolerate how these systems kind of would be used, you know, will have long term repercussions. And so like we need to wise up and be, you know, really aware. What I love about sci fi is that allows you to explore these things and to put humans in these difficult situations of how they would react.
And it’s, you know, it’s not trying to be factual, but it’s getting us to think about, you know, how do we limit our worst excesses and be, you know, remain true as humans. So I think, you know, I’m happy to talk about Battlestar Galactica at length.
RK: I wish we could, wish we could have a whole episode on just that! The, you’ve touched on it quite a bit there by talking about Battlestar Galactica, but I would just like to kind of. Yeah, ask a bit more about it. Just like with everything you’ve said today and also in your book, how bad could things get if we continue down this road and what gives you hope?
LM: Yeah, like I think things have got pretty bad already. You know, we didn’t think that… So the problem that a lot of people saw with technology and with the rise of drones, was lethal autonomous weapons. So like these killer robots, you know, Terminator rising up and wiping out humanity. Like AI is, it’s not really that intelligence.
You know, there’s no risk of artificial intelligence wiping us out in that way it’s probably going to be like sort of some guy, using these systems, developing a biological weapon or something. But I think it’s already been pretty bad in terms of how it’s used by militaries in targeting. So we didn’t really expect the scale that we saw violence in Gaza and even in, you know, the Middle East and the broader conflict in Lebanon and Iran, like even the Iranians have been using AI to targets places in Israel, so like they use an AI generated map, and they ended up hitting the hospital, and killing like dozens of people.
So like this is the problem is AI, it can seem so intelligent, so smart. But if you’ve used it, you know, like if any of us have used it, it makes stuff up. It hallucinates. And so we’re using that, like it’ll always have that problem. And those problems can cascade and, you know, become more the more different AI systems you connect together.
So I think things could get worse. And so like I try to end the book on like the good, the bad and the ugly of like AI. Like, stuff that I’m working on at the moment is, you know, how do we better protect civilians. How can civilians be better seen by algorithms? So they’re not just assumed to be targets, you know, could we have a situation like even Star Wars where we have, like, a C-3PO going round and sort of like protecting civilians, but, you know, is that also a risk of, you know, being hacked and, you know, used against them?
And it’s like, this is the danger. The problem is it’s always the asymmetry that the most powerful militaries in the worlds have access to this technology. And those who live with the consequences are completely disconnected, often like they have no mobile connection or no internet connection. Don’t understand, you know, how AI’s being used and that doesn’t really matter because at the end of the day, they have been injured or had a family member killed and it’s like, well, you know, why was that so arbitrary?
Why was my family member killed? You know, that’s just like if it happened, like, if, you know, my family, my children like how would it not drive you into joining the fight against this, this evil empire that was using all this technology against you? And that doesn’t matter where that happens. That’s always been like, that’s an age old human story.
So, like, like, it’s a bit depressing. All of this. I do have hope because at the end of the day, you know, humanity isn’t just about the most powerful people who can, you know, wield the biggest swords. It’s about how we come together and how we work and mobilize to sort of call out bad behaviour, hold those people to account and build a better world.
And I think we’re at that crux with AI where like, we’re not believing the magic anymore. You know, we’re pulling back the curtain and we realize that there’s no Wizard of Oz that’s, you know, it’s magically making all these things happen, it’s just some old dude who’s, you know, pushing all this data and all these products, and I think there is always a possibility of humans to, to be smarter than the technology and evolve things.
So I think we’re always on a continuum. My concern is, you know, how do we do this before it’s too late. Both environmentally, you know, with data centres, you know, beginning to cropping up everywhere, those are becoming targets themselves. But I think the more we rely on technology to fight wars or even to think for ourselves, that is a vulnerability in itself.
And we’ve seen data centres being attacked. We’ve seen like a lot of how the technology and data is collected being poisoned by having false data. And so there’s weaknesses with AI that can’t be overcome. And so I think we’re in at this period of time, we need to recognize that. Because new things and things will evolve in warfare, like nanotechnology and quantum computing, like things which have their own horrors, but I think we need to remain human in all of that.
We need to recognize, like in war, like, how do you explain to the person that you’ve killed their family member or their child, you know, you’ve taken that. One of the people that won the Nobel Peace Prize in Northern Ireland, you know, John Hume, you know, talked about you know, it’s probably better that people are arrested and put into prison because like, you know, people can come out of prisons, but they can’t come out of graves.
And so like it’s easy enough to, you know, authorize and fund all these weapons because some people make a lot of money they will in going down this road. But at the end of the day that we’re going to be, you know, worse off as a people, as a humanity by those people who were killed and lost in these needless and pointless wars.
RK: Luke, thank you so much for coming on the podcast today. It’s been such a such a pleasure to talk to you. I mean, yeah, super heavy, such a heavy subject, but you deal with it so well. Thank you so much for your time today. I’m going to let everybody know where they can find your book in a moment.
But first I wanted to ask, is there anywhere we can find you online?
LM: Yeah, you can find me on my institutions website. Or LinkedIn a little bit and BlueSky, I think, it’s a shrinking space for social media, and speak freely. So if you want to reach out, drop me an email. That’s probably the easiest way to get me. But yeah, thanks so much for having me today. It’s been it’s been a pleasure.
RK: Thanks, Luke. ‘Algorithms of War: The Human Cost of AI in Conflict’ by Luke Moffett is published by Bristol University Press. You can find out more about the book by visiting bristoluniversitypress.co.uk and also transformingsociety.co.uk.
Thanks so much for listening to this episode of the Transforming Society podcast. If you’d like to buy a copy of this book or any other book published by Bristol University Press or Policy Press, we have a 50% discount code valid until the 31st of December.
Just go to bristoluniversitypress.co.uk, select your book and then enter POD04 at checkout. That’s POD04 at checkout.