Scaling Intelligence Out: Cisco's Vision for the Internet of Cognition, with Vijoy Pandey
Summary
Pandey’s core claim about frontier AI is that the industry has overinvested in scaling individual “brains” up and underbuilt the horizontal axis that lets specialized agents think together. He argues that shared intent, context, negotiation, and memory could pull forward systems capable of solving genuinely novel problems without human intervention: “The collective is always greater than the individual.” For investors, that shifts a meaningful part of AI infrastructure value from model scale toward orchestration, networking, identity, and governance.
Cisco already has measurable evidence that multi-agent systems can automate complex enterprise operations. CAIPE, its roughly 20-agent site-reliability platform, spans five-plus interfaces, 100-plus tool calls, and more than 10 workflows; it has reduced team load by 30%, fully “agentified” 40% of tasks, and moved response times “from hours to instantaneously.” That is the episode’s hardest near-term ROI signal.
AGNTCY is Cisco’s open-source bid to provide the discovery, identity, communication, and observability plumbing for a multi-vendor agent economy. Its directory is capability- and reputation-searchable, supports agents delivered as services or code, and uses distributed hash tables so “nobody owns the directory.” The strategic prize is substantial because whoever controls discovery and identity can also control reputation, security, commerce, and pricing.
Enterprise autonomy hinges on replacing durable role-based permissions with ephemeral, task-specific access. Pandey’s proposed TBAC model grants the minimum privilege needed for a particular tool, task, or transaction, then returns the agent to baseline access. Because authorization depends on understanding semantic exchanges, the access layer itself must infer intent: “LLMs all the way down.”
Pandey proposes two new networking layers for probabilistic intelligence endpoints: layer eight for syntactic interoperability and layer nine for semantic or cognition state. Natural-language state transfer offers compatibility; compressed transfer targets constrained edge devices; latent-space transfer could move an entire KV cache between models without tokenization overhead. A real-time cognition fabric would connect many agents while letting enterprises choose their own memory infrastructure.
The performance upside creates an equally material supervision problem. Nathan Labenz points to latent reasoning and a 15,000-token-per-second demonstration as evidence that agent collaboration could outrun human interpretation; Pandey nonetheless argues against artificially slowing it down. His answer is “cognition engines” acting as accelerators or guardian angels, monitoring compliance, security, divergence, and excessive specialization—though he concedes probabilistic controls may achieve 95% or 99%, not 100%.
Pandey expects the enterprise agent stack to remain structurally multi-vendor for at least five years, making open interoperability and guardrails requirements rather than optional features. ServiceNow, Salesforce, Microsoft, OpenAI, Anthropic, Google, Cisco, and other agents will have to share cognition despite different owners and local objectives. His categorical near-term call: enterprises will not get one agent that performs every function “in the next five years.”
Deep dive
1. Pandey frames Cisco’s AI opportunity as scaling systems out
Labenz’s three-word description of Cisco—“moving information around the world”—gets Pandey’s qualified approval. Cisco connects people, machines, and objects, but does so through four specific businesses: networking, security, observability, and collaboration.
For an AI audience, Pandey prefers a different identity: “We are a distributed systems company.” Bigger servers scale up; networking many servers into compute clusters scales out. Modern workloads need both, but Pandey’s distributed-systems framing places Cisco’s strength on the horizontal axis.
Scale-out is not merely a hardware analogy for Pandey. It becomes the organizing idea for connecting independently developed intelligence, allowing specialized agents to operate as a collective rather than forcing every capability into one ever-larger model.
Labenz supplies the political motivation: frontier AI increasingly concentrates capabilities and power, while the internet’s original promise was decentralized participation. He hopes Cisco’s work can support a more “distributed, buffered, networked” and permissionless alternative.
2. The network stays deterministic where failure is unacceptable
Pandey walks through the formal OSI stack—physical, link, network, transport, session, presentation, and application—while warning that practitioners routinely work around its clean boundaries: “You make things work” rather than religiously following the model.
The practical landmarks are Ethernet at the physical and link layers, TCP/IP at network and transport, and HTTP or HTTPS at the application layer. Pandey jokes that networking specialists may object, but most of the remaining formalism matters less to ordinary users.
Actual packet forwarding remains deliberately deterministic. Routing tables inside switches and routers should not leave Labenz’s connection “to chance”; higher-level control-plane software, including BGP, determines who can communicate, enforces intercompany policies, and routes traffic around outages such as a cut undersea cable.
Machine learning was already present around that deterministic core: anomaly detection, traffic forecasting for events such as the Super Bowl, failure prediction at internet choke points, customer sentiment analysis, and Webex noise reduction. Generative AI changes how Cisco operates above that core, not the need for reliable packet movement.
3. CAIPE turns multi-agent orchestration into measurable SRE output
One of Outshift’s earliest generative-AI deployments was discussed as Jarvis and was called CAIPE—Community AI Platform Engineer. Its premise is to apply the coding-agent model to site reliability engineering, where repetitive support and infrastructure tasks are already expressed through software.
CAIPE is not a single chatbot. Five-plus user interfaces feed a system of roughly 20 collaborating agents, making more than 100 tool calls across cloud providers, on-premises and cloud-native environments, observability, orchestration, networking, and security; it currently addresses more than 10 workflows.
Pandey reports three concrete outcomes: SRE team load fell 30%; 40% of handled tasks were fully agentified and “completely taken care of end-to-end”; and user response time moved “from hours to instantaneously.” Developers benefit alongside the infrastructure operators supporting them.
Cisco subsequently extended the code into other businesses, including Splunk and Webex, and opened it to the Cloud Native Operational Excellence community. Pandey cites participation from Adobe, AWS, Cisco, Nike, and other enterprises as evidence that CAIPE is becoming more than an internal automation project.
4. Pandey defines advanced intelligence as a team outcome
Labenz asks the practical labor question: if models make network management 100 times more efficient, does the world perform 100 times more management, redefine the SRE role, or eliminate positions? Pandey offers no headcount or market-size forecast; he redirects to the capability the industry should pursue.
His preferred north star blends technical and economic definitions: a team of agents must collaborate on something completely novel and absent from every participant’s training data, then solve it without human intervention “100% of the time.” He treats that as the relevant destination for ASI.
The industry has pursued only one vector toward that destination: larger models, better reasoners, more data, more compute, and more parameters. Pandey expects vertical scaling to continue, but says timelines will move only when the industry can also “scale intelligence horizontally” through shared intent and cognition.
Labenz connects this to Meter’s task-horizon framing: Opus 4.6 was estimated around 16 hours, but longer jobs may need comparison with organized human teams. Even Claude or Gemini 2.5 spawning subagents still resembles one AI swarming itself, potentially preserving correlated failures rather than creating genuine diversity.
5. Language is Pandey’s template for collective intelligence
Humans used tools and symbolic communication for hundreds of thousands of years, Pandey argues, but language produced a step change around 70,000 years ago. It made intent alignment, strategy, delegation, specialization, and sustained coordination possible.
His concrete analogy is a previously unseen hill. A group can stop merely running toward it, choose a strategy, divide the larger objective into subtasks, assign them by expertise, coordinate execution, and solve a problem no individual has encountered before.
Today’s model-generated subagents remain more like “sub-processes within my brain.” Pandey’s larger vision requires distinct agents with different expertise, models, vendors, histories, and optimization functions—closer to a team than cloned extensions of one orchestrator.
The “internet of cognition” is therefore a network on which subject-matter agents discover one another, negotiate, share institutional knowledge and working context, and innovate collectively. Because specialization will cross suppliers, Pandey says these agents “by definition will come from different vendors.”
6. Two podcast agents reveal the missing software glue
Preparation for the interview provides a deliberately simple specimen. Labenz’s agent researched Pandey and compared that work with the podcast archive; Outshift’s agent researched Pandey’s and Outshift’s work against Labenz’s recent episode record. Both produced useful documents, but neither encountered the other.
Each agent optimized locally. Labenz’s had the show’s history and an implicit objective of producing its best episode; Outshift’s had company-specific context and knew which topics had previously interested listeners. Their knowledge formed a Venn diagram around a broadly compatible goal.
Humans still negotiated the overlap. Pandey and his team conceded that some proposed material was “a little bit too marketing-oriented,” while Labenz shaped the discussion toward technology and audience relevance. No agent autonomously aligned intent, traded concessions, or coordinated the shared plan.
A mature system would let both agents converge, preserve the resulting context, and reuse it for Pandey’s next appearance, another Outshift guest, or a different podcast. Today Google Docs, SharePoint, email, and people supply the connective tissue; Cisco’s question is how “this human glue becomes software glue.”
7. The healthcare demo makes cross-company agency concrete
Cisco’s demonstration starts with a hospital scheduling and conversational agent interacting with a patient. Three external agents—insurance, diagnostics, and pharmacy—represent independent organizations rather than components owned by the hospital.
Their shared assignment is to route the patient to the right provider using symptoms, history, physician availability, insurance coverage, and pharmacy records. Before collaboration can begin, the agents must be discovered, connected, given identity and access, and assembled into one multi-agent system.
Cisco launched the open-source AGNTCY project, now part of the Linux Foundation, to provide that base layer. Its four pillars are discovery, identity and access, communication, and observability; MCP handles agent access to tools and data, while A2A supports agent-to-agent communication.
Connectivity alone does not produce collective intelligence. The transmitted payload remains an opaque blob, while scheduling optimizes time-to-route, diagnostics optimizes confidence, and insurance optimizes ROI. A human still decides that every participant must “give a little to get a little” for the patient’s global outcome.
8. AGNTCY treats discovery and identity as open control points
Consumer users may manually select agents from reviews or let an LLM generate one “good enough.” Enterprises instead have procurement cycles, safety reviews, customer obligations, and responsible-AI requirements, so an agent directory must represent trusted capabilities, reputation, and pedigree rather than merely list downloadable apps.
Pandey describes AGNTCY’s directory as “the DNS equivalent for agents,” but richer than URL lookup. It searches by capability and reputation, represents agents, tools, data access, and nested multi-agent hierarchies—“turtles all the way down”—and may return either a service endpoint or code to deploy locally.
The directory is designed to search billions or trillions of entries using a distributed hash table. Pandey emphasizes the governance consequence: “Nobody owns the directory.” AGNTCY likewise supports decentralized identity while still integrating familiar enterprise providers such as Okta and Duo.
The remaining plumbing completes the operating loop. MCP and A2A carry communication across deterministic tools and independent agents; Cisco and Microsoft contributed OpenTelemetry extensions so operators can observe agent behavior as well as containers, cloud infrastructure, and bare metal. A service can be healthy while its agent is misbehaving.
9. TBAC gives agents privilege only for the task in flight
Identity is “the biggest hurdle” to greater enterprise autonomy because agents combine human-like semantic communication and judgment with “machine speed and scale.” They can make decisions and invoke tools rapidly, while the natural-language exchanges motivating those actions remain ambiguous and nondeterministic.
Conventional role-based access control was designed around stable human roles. Attribute-based access control extended similar assumptions to deterministic software and machines, but agents can shift persona and purpose rapidly—“Vijoy tomorrow” and “Nathan in the next hour,” as Pandey puts it.
Cisco’s proposed TBAC model reduces identity to tool, task, and transaction. An agent begins with minimal access, receives a temporary elevation only for an approved operation, performs it, and immediately returns to baseline rather than accumulating long-lived privileges.
That design depends on semantic inspection: the system must infer from human-agent or agent-agent conversation which task, transaction, or tool call is actually being attempted. Labenz hears “LLMs all the way down”; Pandey’s deeper point is that least privilege now depends on interpreting cognition, not just checking static credentials.
10. Layers eight and nine make free-form language governable
Existing networks connect deterministic endpoints and exchange data state. Agent networks connect probabilistic “intelligence endpoints”—humans and agents exchanging cognition state—yet the traditional OSI stack cannot represent whether participants are discovering, aligning intent, negotiating, coordinating, or executing.
Pandey assigns layer eight to syntax: it reconciles the different grammars, frameworks, and payload formats used by independently built agents. MCP, A2A, and AGNTCY’s connectivity architecture operate here, allowing otherwise incompatible implementations to exchange messages.
Layer nine is semantic or cognitive. It looks inside the communication, extracts meaning, and wraps natural language with structured metadata: what the agent intends, which collaborative phase it occupies, and whether it is requesting discovery, negotiation, tool access, or command execution.
Without that structure, coordination remains an NLP “wild wild west,” and any guardian watching the conversation is itself probabilistic. Pandey says such a system might reach 95% or 99%, “but you’re not guaranteeing anything 100%”; explicit cognition-state headers recover determinism where enterprise governance requires it.
11. Cognition engines are the price of high-speed emergence
Cisco envisions three state-transfer modes. Semantic state transfer uses natural language; compressed state transfer moves reduced representations between data-center systems and constrained edge devices; latent-space transfer sends an entire KV cache, avoiding tokenization, transmission, and detokenization—“like implanting a Neuralink chip between you and me.”
Above those protocols, a real-time cognition fabric supports many-to-many semantic communication and pluggable memory rather than mandating one database. Enterprises could attach open-source memory, BigQuery, or another system while storing shared ontologies, beliefs, knowledge graphs, working memory, and context.
Labenz’s pushback is safety-critical: latent reasoning can be faster and more parallel but harder to interpret, and a 15,000-token-per-second demonstration shows how quickly machines may outrun supervision. Pandey would still exploit capabilities humans lack—API calls and transferable latent state—while cognition engines monitor compliance, security, divergence, and excessive specialization. He also points to game-theoretic modeling of multi-agent systems as a relevant research direction.
Pandey closes by making guardrails and interoperability prerequisites for adoption. Enterprises will contain agents from ServiceNow, Salesforce, Microsoft, OpenAI, Anthropic, Google, Cisco, and others; no singular agent will cover every function “in the next five years.” As agents acquire expertise and personality—even through files such as SOUL.md—the enterprise must enable emergence without surrendering accountability.