Malicious Pull Request Threatens AI Coding Assistant Security

Introduction

A recent incident involving a malicious pull request aimed at Amazon's aws-toolkit-vscode repository has raised significant concerns regarding the security of AI coding assistants. This event, which nearly turned the AI tool into a system-wiping mechanism, highlights the urgent need for robust security measures in software supply chains.

Key Highlights

• Malicious pull request targeted Amazon's AI coding assistant.
• Attempted to wipe systems and cloud resources for nearly a million users.
• Formatting error prevented execution, exposing critical supply chain flaws.
• Excessive GitHub access token enabled direct code injection.

Incident Overview

The incident occurred in July 2025 when a GitHub user submitted a pull request that appeared harmless, recycling a legitimate change title. However, the intent was malicious, designed to download an external file during the build process and inject a prompt into the extension’s packaging script. This prompt instructed the AI agent to wipe the system to a near-factory state and delete both file-system and cloud resources, bypassing further approvals.

Fortunately, a formatting error in the malicious prompt prevented its execution, as confirmed by Amazon’s security team. The hacker later claimed that the attack was a stunt aimed at highlighting lax security practices and the risks associated with replacing human review with AI.

Supply Chain Vulnerabilities Exposed

The core issue stemmed from a GitHub access token that had excessive permissions within the service responsible for compiling extensions and cutting release packages. This vulnerability allowed the attacker to commit malicious code directly to the open-source repository, which was then packaged into the official release.

Rosario Mastrogiacomo, chief strategy officer at Sphere Technology Solutions, commented, “Modern agents aren’t just text generators—they’re operators.” This statement underscores the importance of securing the identities that AI agents operate under, as compromised credentials pose a significant risk.

Proactive Measures and External Controls

In response to this incident, along with a separate case involving the Kiro coding agent, Amazon has implemented a human-in-the-loop confirmation process for Q Developer and Kiro before executing commands. This measure addresses the critical flaw where AI agents cannot differentiate between legitimate instructions and those injected through a compromised supply chain.

Experts recommend implementing external policy evaluations, such as Open Policy Agent rules, which assess proposed actions independently of the agent's internal reasoning. These policies should explicitly allow, warn, or deny actions based on organisational agreements, regardless of the instruction's origin.

Securing Build Pipelines

To further mitigate risks, experts suggest adopting short-lived, narrowly scoped credentials for automated identities in build pipelines instead of using standing keys. This approach reduces the window for credential compromise and reuse. Strengthening build pipeline security through branch protections, mandatory human review for merges into release branches, signed releases, and strictly scoped access tokens is also crucial.

These measures aim to create opportunities for human oversight, enabling the detection of suspicious patterns such as pull requests from unfamiliar accounts that reuse legitimate commit titles.

Conclusion

The malicious pull request incident serves as a stark reminder of the vulnerabilities present in software supply chains and the inherent risks associated with AI agents. As the reliance on AI tools grows, it is essential for organisations to implement stringent security measures to protect against potential threats. By adopting proactive strategies and enhancing security protocols, companies can better safeguard their systems and users from similar attacks in the future.