The goals are sound, but ultimately they depend upon users reading the dialog windows that warn of the risks and require careful approval before proceeding. That, in turn, diminishes the worth of the protection for a lot of users.
“The standard caveat applies to such mechanisms that depend on users clicking through a permission prompt,” Earlence Fernandes, a University of California, San Diego professor specializing in AI security, told Ars. “Sometimes those users don’t fully understand what is occurring, or they could just get habituated and click on ‘yes’ on a regular basis. At which point, the safety boundary shouldn’t be really a boundary.”
As demonstrated by the rash of “ClickFix” attacks, many users will be tricked into following extremely dangerous instructions. While more experienced users (including a good variety of Ars commenters) blame the victims falling for such scams, these incidents are inevitable for a bunch of reasons. In some cases, even careful users are fatigued or under emotional distress and slip up consequently. Other users simply lack the knowledge to make informed decisions.
Microsoft’s warning, one critic said, amounts to little greater than a CYA (short for canopy your ass), a legal maneuver that attempts to shield a celebration from liability.
“Microsoft (like the remaining of the industry) has no idea how one can stop prompt injection or hallucinations, which makes it fundamentally unfit for nearly anything serious,” critic Reed Mideke said. “The answer? Shift liability to the user. Similar to every LLM chatbot has a ‘oh by the best way, in case you use this for anything essential remember to confirm the answers” disclaimer, never mind that you just wouldn’t need the chatbot in the primary place in case you knew the reply.”
As Mideke indicated, many of the criticisms extend to AI offerings other corporations—including Apple, Google, and Meta—are integrating into their products. Steadily, these integrations begin as optional features and eventually turn out to be default capabilities whether users want them or not.
