NVIDIA's Jensen Huang warns of risks in autonomous AI agents at Astana event

NVIDIA CEO Jensen Huang spoke about OpenShell, a technology designed to keep autonomous AI agents working safely. Speaking at a Fireside Chat in Astana, he said AI developers must build security into their products from the start rather than fixing vulnerabilities after they appear.

Kazakhstan's Minister of Artificial Intelligence and Digital Development, Zhaslan Madiyev, raised the issue of cybersecurity and asked whether NVIDIA plans to introduce additional hardware-level protection mechanisms to prevent hacks and AI malfunctions. Huang said that for humanity to benefit from AI, it must be developed and deployed responsibly. He compared this to the automotive and mining industries, saying safety must be built in from the very first day.

Huang noted that technology companies themselves bear responsibility for security. OpenShell is designed to restrict autonomous AI agents' access to files, tools and other computer system resources. He compared such agents to self-driving cars or robots operating inside an operating system. Unlike ordinary chatbots, which mainly respond to user requests, autonomous agents can carry out sequences of actions using the tools they are given. Source reporting notes that agents with overly broad access rights could take unwanted actions or touch information unrelated to their task, and that their errors could lead to data deletion, disclosure of confidential information or disruption of corporate services.