A 3rd AI-related proof-of-concept attack that garnered attention used a prompt injection to cause GitLab’s Duo chatbot so as to add malicious lines to an otherwise legitimate code package. A variation of the attack successfully exfiltrated sensitive user data.
One more notable attack targeted the Gemini CLI coding tool. It allowed attackers to execute malicious commands—similar to wiping a hard disk—on the computers of developers using the AI tool.
Using AI as bait and hacking assistants
Other LLM-involved hacks used chatbots to make attacks simpler or stealthier. Earlier this month, two men were indicted for allegedly stealing and wiping sensitive government data. One in all the lads, prosecutors said, tried to cover his tracks by asking an AI tool “how do i clear system logs from SQL servers after deleting databases.” Shortly afterward, he allegedly asked the tool, “how do you clear all event and application logs from Microsoft windows server 2012.” Investigators were in a position to track the defendants’ actions anyway.
In May, a person pleaded guilty to hacking an worker of The Walt Disney Company by tricking the person into running a malicious version of a widely used open source AI image-generation tool.
And in August, Google researchers warned users of the Salesloft Drift AI chat agent to contemplate all security tokens connected to the platform compromised following the invention that unknown attackers used a few of the credentials to access email from Google Workspace accounts. The attackers used the tokens to achieve access to individual Salesforce accounts and, from there, to steal data, including credentials that could possibly be utilized in other breaches.
There have been also multiple instances of LLM vulnerabilities that got here back to bite the people using them. In one case, CoPilot was caught exposing the contents of greater than 20,000 private GitHub repositories from corporations including Google, Intel, Huawei, PayPal, IBM, Tencent, and, satirically, Microsoft. The repositories had originally been available through Bing as well. Microsoft eventually removed the repositories from searches, but CoPilot continued to reveal them anyway.
