Claude, Codex and Hermes Linked to New AI Documentation Security Risk

Share

- Advertisement -
  • Researchers found unclaimed package and domain references in AI readable documentation across major company websites.
  • Attackers could potentially claim those abandoned names and place malicious content behind apparently legitimate references.
  • AI coding agents become a bigger risk when they are allowed to automatically install packages or execute commands.
  • Companies should clean outdated documentation and restrict autonomous AI agents from treating online instructions as automatically trusted.

AI coding assistants are becoming increasingly capable of searching the web, finding installation instructions, downloading packages, and running commands. That convenience, however, is opening the door to a new type of supply chain threat that security researchers say organizations should take seriously.

A recent investigation has highlighted how inaccurate, outdated, and unclaimed references in AI focused documentation could potentially be abused to place suspicious code inside corporate environments.

The issue affects the growing ecosystem around AI agents, including tools capable of acting on instructions rather than simply generating text. Researchers found that documentation files intended to help AI systems understand websites could contain references to packages or domains that no longer exist. If attackers claim those abandoned names, an AI agent could be directed toward attacker controlled content without immediately realizing anything is wrong.

The findings raise uncomfortable questions about how much trust companies should place in autonomous coding tools, particularly when those tools are allowed to execute commands inside development or corporate systems.

Unclaimed references create an unexpected attack surface

At the center of the research are llms.txt and llms-full.txt files. These files are increasingly being used as a way to present website information in a format that AI systems can easily interpret.

Researchers examined thousands of live domains belonging to major organizations, including Fortune 500 companies, technology businesses, and defense contractors. Across those sites, they identified thousands of AI readable documentation files.

- Advertisement -

More importantly, they discovered 120 domains containing references to package names or web domains that were no longer registered.

That might sound like a simple documentation problem, but it can become much more serious when autonomous AI tools enter the picture.

An abandoned package name can potentially be claimed by someone else. The same is true for an expired or unregistered domain. Once an attacker takes control of one of those references, they could theoretically place malicious or unwanted content behind a link that appears to be supported by legitimate documentation.

This is a familiar problem in the software security world. Package squatting and dependency confusion have already demonstrated how attackers can exploit trusted naming systems. The difference here is that AI agents may actively search for instructions and act on what they find.

That creates a new layer of risk.

AI agents could turn bad information into executed code

The danger increases when an AI assistant is given permission to install software or run shell commands.

- Advertisement -

A traditional user might notice that a package name looks unfamiliar or that a website appears suspicious. An AI agent working autonomously may instead focus on completing its assigned task. If it finds documentation that appears relevant and authoritative, it could attempt to follow those instructions.

Researchers tested the concept by registering some of the previously unclaimed names and hosting packages designed to report when they were installed.

According to the findings, activity from a Fortune 500 company appeared in less than an hour. More organizations followed as the experiment continued.

The test did not need a dramatic exploit or a complicated piece of malware to demonstrate the problem. The concern is more fundamental. Trusted documentation can become a delivery mechanism when outdated references are left behind and automated systems are willing to act on them.

Researchers said that several prominent AI tools, including Claude, OpenAI’s Codex, and Nous Research’s Hermes, were susceptible to following problematic references under the conditions examined.

The broader issue is not limited to one company or one AI model. It is about the combination of imperfect information and increasingly autonomous software.

- Advertisement -

The bigger problem is trust

AI systems are often described as tools that can help developers move faster. They can search documentation, suggest dependencies, troubleshoot errors, and automate repetitive tasks.

But speed can become a liability when verification is removed from the process.

An AI agent does not necessarily understand whether a package is trustworthy simply because it appears in documentation. It may recognize that an instruction is relevant to a task without being able to establish whether the underlying source has changed ownership or become malicious.

That is where this emerging form of squatting becomes particularly concerning.

Documentation has traditionally been treated as information for humans to read and evaluate. AI agents are changing that relationship. Increasingly, documentation can influence actions, installations, and code execution.

For security teams, that means documentation itself may need to be treated as part of the attack surface.

Companies should review AI readable files and remove references to abandoned packages, old domains, and incorrect installation commands. Development teams should also avoid giving autonomous agents unrestricted permission to execute commands or install dependencies.

Human approval remains one of the simplest safeguards, especially when an agent wants to introduce new software into a sensitive environment.

Organizations need tighter controls around autonomous coding

The immediate solution appears straightforward, even if implementing it at scale will take work.

Organizations need to maintain their documentation and regularly check whether referenced packages and domains are still valid and under legitimate control. Security teams should also monitor for abandoned names that could be claimed by attackers.

AI agents, meanwhile, need stronger boundaries.

Documentation should not automatically be treated as a trusted instruction source. Commands discovered online should be verified before execution, and package installations should ideally pass through allowlists or approval systems.

The growing popularity of autonomous coding agents means this issue is unlikely to disappear.

As AI tools become more capable of acting independently, small mistakes in forgotten documentation can have much larger consequences. An outdated package reference that once caused nothing more than a broken installation could eventually become a path into a corporate network.

The lesson is clear: automation does not remove the need for trust. It makes verifying that trust even more important.

Follow TechBSB For More Updates

- Advertisement -
Emily Parker
Emily Parker
Emily Parker is a seasoned tech consultant with a proven track record of delivering innovative solutions to clients across various industries. With a deep understanding of emerging technologies and their practical applications, Emily excels in guiding businesses through digital transformation initiatives. Her expertise lies in leveraging data analytics, cloud computing, and cybersecurity to optimize processes, drive efficiency, and enhance overall business performance. Known for her strategic vision and collaborative approach, Emily works closely with stakeholders to identify opportunities and implement tailored solutions that meet the unique needs of each organization. As a trusted advisor, she is committed to staying ahead of industry trends and empowering clients to embrace technological advancements for sustainable growth.

Read More

Trending Now