Artificial Intelligence And Autonomous Military Systems Presented by: Dr. Trzun Date: 31 January 2024 Introduction – New Technologies - This presentation briefly shows what I wrote about in the article. - The topic is very similar to the chapter in the book describing what new technologies can do (very popular).... - But also what risks are present due to the imperfections of these technologies. - Unlike the chapter, in the article, I focused more on the capabilities/imperfections of artificial intelligence... • And autonomous systems that use AI as a replacement for live pilots, co-pilots, analysts, etc. - Modern weapons of today: unprecedented possibilities - For example, small, commercially available drones can attack armored vehicles and soldiers - They serve as a replacement for the highly precise ammunition - Today, they enable Ukraine to maintain some resemblance of balance with Russia (grim situation critical shortage of ammunition for howitzers and missile launchers) - The Russians developing their own tactical drones ZALA Lancet FPV kamikaze drone - How to fight against swarms of small and deadly drones? - Turtle tanks, nets above trenches, etc. indications that armies are actually lost - They're finding "solutions" that are more applicable to the beginning of the 20th century. - They just don't know what to do! Artificial Intelligence - Some new weapons really impressive example large MALE and HALE UAVs (drones)? - Even more impressive: operated by pilots sitting thousands of kilometers away - UAVs fly day and night (24 hrs endurance) above the designated zone, gathering data - This data is analyzed by analysts that use artificial intelligence >> big stream of data - AI sends warning to the commander about potential enemy behavior - Commander (sitting on another continent) issues the command • ....and the drone attacks • ....when it attacks, it will surely destroy: the Hellfire missile designed to destroy tanks and bunkers! ΑΙ - August 29th, 2011 (2 days prior to American troops have withdrawn from Afghanistan): - 10 civilians killed because AI gave a false positive identification - It turned out that the person killed was an aid worker (working for an American humanitarian organization) - I have no ambitions to enter political discussions, purely a technical discussion - Several "red flags": • Often traveled in and out of Kabul • Drove a Toyota Corolla • Seen unloading strange objects from the car containers turned out to be empty water - However: was the threshold for the final decision set too low? Inevitable Arrival of AI - Robots/drones of today: inherent problem due to dependence on the link to the - pilot/base station - Electronic warfare has become increasingly important - Ukraine an excellent example - Ukrainian side: at first, very successful at using UAVs against Russian forces - Many "saviors of Ukraine": UAVs, loitering ammunition, unmanned ground and surface vehicles... - After the Russian side gained momentum and improved its EW and C-UAV capabilities (signal jammers, spoofers, interceptors), the Ukrainian side faced serious problems Inevitable Arrival of AI - Next logical step - equip UAVs with AI +signal-independent guidance systems - AI provides the opportunity to guide a drone without any link to the base station and without a GPS signal - Guidance by onboard sensors or visual orientation - Especially useful in urban operations, a detailed map of the city already exists (e.g., Google Street View) Loyal Wingman Program - UAVs of tomorrow: • AI driven General Dynamics F-16 and Kratos XQ-58A Valkyrie • “Loyal Wingman” project (CCA Collaborative Combat Aircraft, NGAD Next Generation Air Dominance) • UAVs capable of autonomous flight, engaging the enemy, protecting the leading (manned) aircraft; manned-unmanned teaming - At the paper excellent idea (probably the imminent future) - In the reality and in this moment not so perfect as described Problems, Threats, Risk... - The problem with AI paired with sensors: AI can be easily deceived (sometimes too easily!) - For example: by adding some noise in picture • “Explaining and Haunaccine Aduersarial Evomalos!! Coodfollow Chlong Szegedy (2015)” • - AI recognizes panda as gibbon with 99.3% confidence Deceiving Tesla's autopilot with a projector (image of a pedestrian projected onto the road, a tree, or some other background) - Deceiving self-driving vehicles (AI) No. 2: slight changes to a traffic sign result in the AI no longer recognizing it - Adversarial attacks: making slight alterations to input data, introducing changes so subtle that they are practically undetectable to the human eye - AI makes dangerous errors example AI misidentify 3-D turtle toy as a RIFLE - Real danger of AI-driven weapon to wrongly recognize civilians as threats! - Therefore: Human-out-of-the-Loop military systems probably not the best solution (for decades at least) - Human operator - should be kept in the loop (or ON the loop) - Especially for critical operations (activating weapon, selecting targets, definitive decision whether to engage or not...) - Long list of threats - adversarial attacks: • Introducing changes so subtle that they are practically invisible to the human eye • (panda>>gibbon, turtle toy>>rifle) • Deceive image recognition algorithms by distorting the appearance of one's equipment adding false details, altering colors or textures, or even completely changing the visual shape • Masking: camouflage techniques to hide equipment • Distorting sensors and cameras: using flashes, laser devices, or other interfering devices • - etc. List of threats errors in AI training (errors on our side): • Data Bias: if training dataset not diverse or representative enough, the algorithm may learn biased patterns and become unbalanced in its predictions • Overfitting - when the algorithm becomes too tailored to the training dataset, loses the ability to generalize to new data • Scarcity of Data acquiring a sufficient volume of data for training the algorithm challenging particularly in cases involving constrained datasets (military industry!) • Low Interpretability public requires the ability to understand and explain AI's predictions to support it; what parameters taken into account, which features or patterns considered, what is the threshold for decision-making... • etc. Instead of Final Verdict... - Autonomous systems - definitely promising, still many issues to be resolved - The next decades: rigorous testing required before use in real conditions - Always (?): keeping some sort of a safety mechanism (keep Human-IN- or ON-theLoop) - Errors: too expensive (human lives!) Killer Robots - Pros • Fewer casualties  Deploying killer robots can reduce casualties in military operations. • Penetrate highly secured combat zones  With proper planning, killer robots can attack combat zones effectively. • Access to hazardous sites  Military organizations can deploy killer robots for risky missions that may be fatal for military personnel. • Cost-savings  Governments can reduce their military spending with the help of killer robots as their expenses are minimal. - Cons • Flawed distinction process  Killer robots may fail to distinguish between citizens and combatants during an armed conflict. • Lack of accountability  Killer robots cannot be held accountable in case of accidents, unlike military personnel. • Disputes regarding regulations  Developing regulations for killer robots is increasingly complicated. • Poor judgment in complex situations  Killer robots may be unable to make correct decisions in unfamiliar situations.