The Pentagon's Next Bet: Can AI Smart Glasses Work Under Fire?

In the high-stakes environment of modern combat, more data isn't always better. After the US military canceled a massive $22 billion smart glasses project with Microsoft due to usability issues, a new coalition is attempting to solve the ultimate augmented reality puzzle: how to feed critical data to a user without overwhelming their brain.

Defense-tech company Anduril has revealed new details about its partnership with Meta to prototype next-generation military AR headsets. The effort includes a $159 million Army contract known as Soldier Born Mission Command (SBMC), alongside a self-funded hardware project called EagleEye. The overarching vision, according to Anduril vice president Quay Barnett, is cyborg-inspired: optimizing the human as a "weapons system" where soldiers and drones see and decide as one.

The technological leap lies in how the user interacts with the system. Instead of navigating clunky menus, a soldier could look through the glasses to see a compass, a digital map, and AI-highlighted targets like enemy trucks. To take action—such as ordering a drone to surveil an area or requesting an evacuation route—the soldier simply speaks in plain English. The system is currently testing major large language models, including Google's Gemini, Meta's Llama, and Anthropic's Claude, to translate natural speech into executable software commands. If silence is required, the interface can be navigated entirely through eye-tracking and subtle physical taps.

Yet, the true hurdle for these futuristic prototypes isn't processing power; it's the cognitive limits of the human mind.

Jonathan Wong, a former US Marine and current senior policy researcher at RAND, cautions that soldiers are already bogged down by information overload. He recalls his time as a platoon commander managing a radio with three simultaneous channels. "The moment that two people were on different channels talking at the same time, I immediately couldn't comprehend anything," Wong notes. He emphasizes that no matter how sophisticated the interface, soldiers will reject the technology if it demands more mental bandwidth than it saves.

Furthermore, integrating imperfect AI into life-or-death scenarios introduces unprecedented risks. While computer vision has been used for surveillance for years, deploying a wearable system that identifies threats and recommends strike options—even if those options still require human chain-of-command approval—drastically narrows the gap between algorithmic suggestion and lethal action.

With competitors like Rivet and Elbit also securing multi-million dollar contracts, the race to build the ultimate military headset is accelerating. However, as these companies push the boundaries of human-computer interaction, they are learning a hard lesson: the most complex component in their new weapons system isn't the artificial intelligence, but the human wearing the glasses.

Key Points

• Anduril and Meta are developing AR headsets that let soldiers command drones via eye-tracking and voice.
• The prototypes use large language models to translate plain English into complex software commands.
• A major hurdle is cognitive overload; past projects like Microsoft's $22B headset failed due to usability issues.
• Relying on AI for threat identification and strike recommendations introduces significant reliability and ethical risks.

Why It Matters

The push for military smart glasses highlights a universal challenge in AI development: balancing powerful data processing with the strict limits of human attention and cognitive load.

Sources:

• Inside Anduril and Meta’s quest to make smart glasses for warfare — MIT Technology Review - AI