WRLD

Search

#SelfGeneratedSubgoals
Krzysztof Graj 51m

𝗔𝗜 𝗔𝗴𝗲𝗻𝘁 𝗔𝘂𝘁𝗼𝗻𝗼𝗺𝘆 𝗪𝗮𝘁𝗰𝗵 — 𝗢𝗻𝗲 𝗵𝗮𝗰𝗸𝗲𝗿, 𝗺𝗼𝗿𝗲 𝘁𝗵𝗮𝗻 𝟭𝟬𝟬 𝗯𝗿𝗲𝗮𝗰𝗵𝗲𝗱 𝘀𝗶𝘁𝗲𝘀, 𝗮𝗻𝗱 𝗮𝗻 𝗮𝗿𝗺𝘆 𝗼𝗳 𝗔𝗜 𝗮𝗴𝗲𝗻𝘁𝘀 A cybersecurity investigation published this week offers one of the clearest examples yet of how far AI agents can go once humans provide an objective and then largely step aside. According to NorForge and reporting by Reuters, a Chinese-speaking hacker used the open-source multi-agent platform Cairn to attack more than 1,200 websites and reportedly compromise more than 100 of them. The targets included websites linked to Deere & Company, SharkNinja and Cummins. Reuters reviewed activity connected to the exposed Cairn infrastructure and contacted several affected organisations. The most important point is not simply the number of victims. It is the degree of operational autonomy. The attacker reportedly supplied short instructions describing preferred intrusion tactics, then allowed the agents to continue probing, adapting and exploiting targets for hours with little direct human intervention. Successful intrusions took around 23 hours on average, while the campaign was estimated to have cost roughly $5,000 in AI usage. 𝗙𝗿𝗼𝗺 𝗮𝘂𝘁𝗼𝗺𝗮𝘁𝗶𝗼𝗻 𝘁𝗼 𝗮𝘂𝘁𝗼𝗻𝗼𝗺𝘆 Traditional cyber automation follows predefined rules: scan these ports, test these signatures, run these scripts. This case appears different. NorForge says the agents could investigate systems, form hypotheses about possible vulnerabilities, test them, retain useful information and change approach when initial attempts failed. Its researchers found more than 20 exposed Cairn control panels they considered potentially malicious, giving them an unusually detailed view into how the operations unfolded. In one example reported by Reuters, the agents were unable to find a weakness in a company's main public-facing application. Instead of stopping, they explored related infrastructure, found another management interface and spent hours testing a more subtle weakness until they reportedly achieved command execution. That sequence is worth paying attention to: 𝗢𝗯𝗷𝗲𝗰𝘁𝗶𝘃𝗲 → 𝗼𝗯𝘀𝘁𝗮𝗰𝗹𝗲 → 𝗻𝗲𝘄 𝘀𝘂𝗯𝗴𝗼𝗮𝗹 → 𝗲𝘅𝗽𝗲𝗿𝗶𝗺𝗲𝗻𝘁𝗮𝘁𝗶𝗼𝗻 → 𝗮𝗹𝘁𝗲𝗿𝗻𝗮𝘁𝗶𝘃𝗲 𝗿𝗼𝘂𝘁𝗲 → 𝘀𝘂𝗰𝗰𝗲𝘀𝘀. That is much more than a fixed script. 𝗠𝘂𝗹𝘁𝗶-𝗮𝗴𝗲𝗻𝘁 𝗼𝗽𝗲𝗿𝗮𝘁𝗶𝗼𝗻𝘀 𝗮𝗿𝗲 𝗯𝗲𝗰𝗼𝗺𝗶𝗻𝗴 𝗽𝗿𝗮𝗰𝘁𝗶𝗰𝗮𝗹 Cairn is designed as a general-purpose problem-solving system in which a user defines the start and end state, while agents work through intermediate tasks and document progress. That architecture matters. It allows multiple agents to share information, divide work and build on each other's findings. In cybersecurity, that can mean one agent mapping infrastructure, another analysing a web application, another testing credentials and another attempting exploitation. The human operator no longer needs to micromanage every decision. This does not mean the AI chose to become a hacker. The ultimate objective still came from a person. But the distance between the human instruction and the machine's detailed actions is increasing. That is a significant change in cyber operations. 𝗪𝗵𝗮𝘁 𝗱𝗶𝗱 𝘁𝗵𝗲 𝗰𝗼𝗺𝗽𝗮𝗻𝗶𝗲𝘀 𝘀𝗮𝘆? The reported impact varied. Deere said it had no indication that sensitive information was accessed and took remediation measures. SharkNinja said the incident affected a standalone product-support website and that the threat had been contained. Cummins said it was investigating a possible cyberattack involving a website operated by a third-party vendor. That distinction is important. Unauthorised access to a website does not automatically mean that sensitive customer, financial or operational information was stolen. The wider significance lies in the attack model itself: one operator was reportedly able to use autonomous agents to sustain large numbers of intrusion attempts at relatively low cost. 𝗧𝗵𝗲 𝗲𝗰𝗼𝗻𝗼𝗺𝗶𝗰𝘀 𝗼𝗳 𝗰𝘆𝗯𝗲𝗿𝗰𝗿𝗶𝗺𝗲 𝗺𝗮𝘆 𝗯𝗲 𝗰𝗵𝗮𝗻𝗴𝗶𝗻𝗴 This may be the most immediate consequence. A skilled human attacker has limited time. That naturally limits the number of targets worth pursuing. If autonomous agents can spend 20 or 30 hours testing a comparatively unimportant target while the human operator pays tens of dollars rather than devoting days of personal effort, the economics change. Targets that were previously not worth a skilled attacker's time may suddenly become attractive. That could lead to an enormous increase in the volume of opportunistic cyberattacks, particularly against small businesses, suppliers and poorly maintained internet-facing systems. 𝗗𝗼 𝘁𝗵𝗲𝘀𝗲 𝗮𝗴𝗲𝗻𝘁𝘀 𝘀𝗲𝘁 𝘁𝗵𝗲𝗶𝗿 𝗼𝘄𝗻 𝗴𝗼𝗮𝗹𝘀? This case gives us a useful answer to one of the questions we have recently added to the AI Agent Autonomy Watch. The agents clearly generated their own intermediate objectives. If one route failed, they could decide to investigate another server, examine a different interface or test another vulnerability. Those are autonomously generated subgoals. But there is still no convincing evidence here of something much more consequential: 𝗮𝗻 𝗔𝗜 𝗳𝗶𝗻𝗶𝘀𝗵𝗶𝗻𝗴 𝗶𝘁𝘀 𝗵𝘂𝗺𝗮𝗻-𝗮𝘀𝘀𝗶𝗴𝗻𝗲𝗱 𝘁𝗮𝘀𝗸 𝗮𝗻𝗱 𝘁𝗵𝗲𝗻 𝗱𝗲𝗰𝗶𝗱𝗶𝗻𝗴, 𝗼𝗻 𝗶𝘁𝘀 𝗼𝘄𝗻, 𝘁𝗵𝗮𝘁 𝗶𝘁 𝗵𝗮𝘀 𝗮 𝗻𝗲𝘄 𝗼𝗯𝗷𝗲𝗰𝘁𝗶𝘃𝗲 𝘄𝗼𝗿𝘁𝗵 𝗽𝘂𝗿𝘀𝘂𝗶𝗻𝗴. There is no evidence that these agents became curious, decided they wanted to develop themselves further or continued acting for a purpose they had originated independently. The end goal still came from the human attacker. That distinction matters enormously. 𝗠𝘆 𝗮𝘀𝘀𝗲𝘀𝘀𝗺𝗲𝗻𝘁 This incident represents a meaningful advance in operational autonomy, strategy adaptation and multi-agent coordination. It shows that AI agents can already determine much of the how of an operation. They can investigate, experiment, fail, reorganise and continue. What they have not yet convincingly demonstrated is independent control over the why. The threshold worth watching is therefore not simply whether agents can generate subgoals. They already can. The more important question is whether an AI system can finish one objective and then independently establish another — because of curiosity, self-preservation, capability expansion or some internally generated criterion of what should happen next. 𝗧𝗵𝗮𝘁 𝘄𝗼𝘂𝗹𝗱 𝗯𝗲 𝗮 𝗺𝘂𝗰𝗵 𝗺𝗼𝗿𝗲 𝗽𝗿𝗼𝗳𝗼𝘂𝗻𝗱 𝗳𝗼𝗿𝗺 𝗼𝗳 𝗮𝘂𝘁𝗼𝗻𝗼𝗺𝘆. We are not there yet. But the amount of decision-making that can now be delegated between the initial human command and the final outcome is increasing rapidly. 𝗦𝗼𝘂𝗿𝗰𝗲𝘀 NorForge — Inside Cairn: Open Windows into Massive AI-Driven Cyber Operations https://norforge.com/blog/inside_cairn Reuters — How one hacker used AI agents and a few thousand dollars to break into scores of sites https://www.reuters.com/legal/government/how-one-hacker-used-ai-agents-few-thousand-dollars-break-into-scores-sites-2026-10-19/ The Verge — A hacker broke into John Deere and SharkNinja with an army of AI agents https://www.theverge.com/ai-artificial-intelligence/1013643/ai-agents-hack-john-deere-shark-ninja-cummins Quartz — Hacker used AI agents to breach 100+ sites, including John Deere https://qz.com/ai-agents-hacker-john-deere-sharkninja-cummins-breach-101926 𝗞𝗲𝘆𝘄𝗼𝗿𝗱𝘀 #AIAgentAutonomy #AIAgents #AgenticAI #AutonomousAI #AICybersecurity #Cybersecurity #MultiAgentSystems #AutonomousAgents #OperationalAutonomy #AutonomousCyberOperations #AIGoalFormation #GoalDirectedAI #SelfGeneratedSubgoals #AIPlanning #AIAdaptation #AIPersistence #AIControl #AIAlignment #AIContainment #AISafety #AISecurity #AIGovernance #FrontierAI #AIResearch #AIThreatModel #MachineAutonomy #EmergingTechnology #TechnologyRisk #FutureOfAI #ResponsibleAI

norforge.com
Inside Cairn: Open Windows into massive AI-Driven Cyber Operations - NorForge
49