## Overview This case study presents LinkedIn's journey in deploying production multi-agent systems, specifically the Hiring Assistant product launched in October 2024, and the technical evolution toward autonomous agents. Adam Kaplan, a senior staff engineer at LinkedIn working on the agents platform within Core AI, shares insights from months of production operation serving recruiter customers. The presentation is particularly valuable because it moves beyond theoretical multi-agent design to address the practical operational challenges encountered when scaling agent systems in production environments. LinkedIn's vision centers on creating economic opportunity for the global workforce, with agents positioned as essential tools for helping workers accomplish tasks more efficiently. The Hiring Assistant serves as the concrete example grounding the discussion, though the talk quickly pivots to the broader architectural and operational challenges the team has identified while working toward more autonomous agent capabilities. ## The Hiring Assistant Product The Hiring Assistant represents a multi-agent system built using a supervisor pattern with four specialized agents, each handling distinct phases of the recruiting workflow. The architecture begins with a preference model constructed by analyzing recruiters' historical decisions and actions across past and current projects, enabling personalization to individual recruiter styles and preferences. The four agents operate in a coordinated workflow. The intake agent guides recruiters through the process of creating job roles and descriptions. The sourcing agent searches for and generates comprehensive candidate lists. The evaluation agent helps recruiters narrow down to the most promising candidates. Finally, the outreach agent initiates personalized communications with candidates, tailoring messages to both the candidate's profile and the recruiter's communication style. The system has been in production since October 2024, delivering what LinkedIn describes as "significant positive impact" to recruiter customers. The technical infrastructure supporting this system comprises both open source components and proprietary technologies developed to fill gaps in the available ecosystem. While the speaker notes this isn't intended as a product pitch, the multi-month production deployment provides important context for understanding the operational challenges discussed. ## From Multi-Agent Systems to Autonomous Agents Kaplan positions multi-agent systems not as an emerging frontier but as a "crashing wave" already delivering value in production. The forward-looking challenge involves evolving toward autonomous agents—systems that can take in multiple environmental signals simultaneously, maintain memory across contexts, and operate in more open-ended react loops rather than following predefined workflows. The definition of an autonomous agent provided distinguishes it from specialized agents through its ability to process multiple signals from its environment, use those signals along with memory to orient itself, and then enter a react loop where it reasons about the next action, takes that action using tools, incorporates feedback, and iterates. This subtle but impactful difference creates several significant operational challenges. ## Memory Architecture and Isolation The discussion of memory architecture reveals one of the most complex operational challenges in production agent systems. Traditional agent memory follows a three-tier structure. The first tier consists of in-session memory—the active conversation including user messages, agent responses, tool calls, and their results. This represents high-resolution, immediately accessible context. The second tier involves summarization of conversation history along some axis of classification when the active conversation becomes too large for context windows. These summaries preserve the broad strokes, learnings, and key takeaways from completed interactions, allowing the agent to reference past work without maintaining every detail in active context. The third tier emerges when even summarized histories become too numerous. At this point, information is decomposed into primitives—facts, phrases, and structured data—and stored in retrieval systems for RAG (Retrieval Augmented Generation). The speaker mentions both traditional embedding-based retrieval and more sophisticated agent-powered retrieval systems as options at this tier. However, autonomous agents that process multiple signals from different users introduce a critical complexity: memory isolation. The presentation illustrates this with a scenario involving an event coordinator agent named Evie working with two users, Jane and John. Jane is the primary contact planning a conference, while John is aware of the event and contacts Evie to request 10 additional rooms. In a naive implementation with isolated memory stacks per user, Evie might independently contact the hotel to book the additional rooms, creating confusion by having multiple points of contact making decisions. The solution requires introducing what Kaplan terms "common knowledge"—a shared memory space where the agent is aware of its activities across different contexts without necessarily exposing all details. The common knowledge layer allows Evie to recognize that John's request relates to Jane's event and route the information appropriately, perhaps informing Jane of the change rather than taking independent action. However, implementing common knowledge presents substantial challenges spanning trust, regulatory compliance (including DMCA considerations), enterprise policy requirements, and basic human expectations about information sharing. Kaplan expresses optimism about solving this challenge by drawing an analogy to human memory. Humans naturally navigate memory boundaries—knowing what information is appropriate to share in which contexts, how memories decay over time, and what constitutes normal versus pathological forgetting. Because these boundaries and expectations can be expressed through language, they potentially become tractable as reasoning problems that models can learn to handle. ## Tool Discovery and Management Autonomous agents require access to significantly larger tool sets than specialized agents operating in predefined workflows. The historical progression illustrates this scaling challenge: approximately 10 tools was considered reasonable two years ago, 30 tools last year, and current models like GPT-4.0 can handle around 100 tools. While this represents impressive 10x growth in 18 months, the total number of available tools has grown 1000x in the same period. The emergence of the Model Context Protocol (MCP) and similar frameworks is accelerating this explosion by making it easier for existing APIs to expose their capabilities as LLM-callable tools. This creates a persistent need for tool discovery mechanisms that help autonomous agents explore their environment and identify relevant tools for their current tasks. ## Tool Calling and Safety Validation LinkedIn's production experience has revealed that tools fall into two fundamental categories with very different operational requirements. Idempotent tools like knowledge retrieval operations are relatively safe—they don't modify state and can be retried without consequence. These represent only about 10-20% of available tools. The remaining 80% are potentially destructive tools that perform actions with consequences: sending emails, committing code, commenting on documents, planning events, or any form of communication. Because these tools essentially represent communication actions, LinkedIn treats them with the same safety considerations applied to direct user-facing messages. This manifests in two critical validation points. First, tool parameters undergo safety validation before execution to ensure they're not hallucinated and are appropriate for the intended action. The speaker emphasizes that once data enters the context window, it becomes extremely difficult to remove, making proactive validation essential. Second, feedback returned from tool execution undergoes anti-jailbreaking and other safety checks before being added to context. If content filters (such as Azure's content filtering) catch problematic content after it's in context, recovering gracefully becomes challenging. An agent working on a long-running task may have executed multiple subsequent actions that can't simply be forgotten, making it difficult to backtrack to a safe state. LinkedIn's approach communicates validation feedback as tool feedback itself, allowing the autonomous agent to reason about what happened and make decisions. The agent might search for alternative tools, adjust parameters, devise novel approaches to reach its goal, or gracefully give up if the task proves infeasible with available safe actions. ## Computational Efficiency Through Complexity Classification A critical operational challenge in production deployment is the scarcity of GPU resources, particularly for expensive reasoning models. While reasoning models enable impressive capabilities, they don't scale economically to handle all agent interactions. LinkedIn addresses this through complexity classification that routes requests to appropriate model types. Many user interactions can be satisfied using only existing context without requiring tool calls—simple questions about the agent's capabilities, references to previous conversations, or straightforward informational requests. These interactions are routed to faster, lower-latency completion models rather than expensive reasoning models. By implementing this complexity-based routing, LinkedIn moves workload off slow, expensive reasoning models onto more efficient completion models, allowing reasoning models to focus computational resources on genuinely complex problems requiring multi-step planning and tool orchestration. ## Modular Agentic Architecture The presentation describes a modular architecture that has emerged from production experience. The core react loop incorporates several specialized components: - Tool Discovery: Helps autonomous agents explore available tools and capabilities - Tool Calling: Executes selected tools with validated parameters - Safety Validation: Proactively validates tool parameters before execution - Anti-Jailbreaking: Checks tool feedback before incorporating into context - Complexity Classification: Routes requests to appropriate model types This modularity reflects the reality that production agent systems require substantial infrastructure beyond the core LLM reasoning loop. Each component addresses specific operational requirements identified through real-world deployment. ## Design Philosophy: Adapting Agents to Human Shape The presentation concludes with a philosophical stance informed by production experience: attempting to bend human expectations and the human world to accommodate agentic systems leads to failure. Instead, successful deployment requires adapting agents and their supporting infrastructure to conform to human-shaped expectations and workflows. Kaplan argues the world is fundamentally designed for humans—its communication patterns, expectations, work processes, and trust models all reflect human psychology and social structures. Rather than trying to change these deeply embedded patterns, effective agent systems must conform to them. An illustrative example involves trust and information security. People expect models to be perfect and never leak information, yet when organizations hire humans, they onboard them and provide access to sensitive systems based on behavioral expectations and trust models, not guarantees of perfection. Understanding the behavioral psychology underlying human trust and designing agent infrastructure to conform to those expectations represents a promising path forward. ## Critical Assessment This case study offers valuable insights from genuine production deployment, distinguishing it from purely theoretical or prototype-stage discussions of multi-agent systems. The focus on operational challenges—memory isolation, tool safety, computational efficiency—reflects real engineering constraints rather than idealized architectures. However, several caveats warrant consideration. The presentation claims "significant positive impact" for Hiring Assistant without providing quantitative metrics or detailed evidence. While understandable in a conference talk context, this limits our ability to assess actual production effectiveness versus engineering sophistication. The memory isolation and common knowledge challenge is well-articulated, but the proposed solution remains largely aspirational ("we're working on it"). The analogy to human memory is compelling but doesn't constitute a technical approach. Production deployment of shared memory across privacy boundaries involves substantial unsolved technical, regulatory, and social challenges. The tool safety approach is pragmatic and reflects real production concerns, though the presentation doesn't address potential latency implications of adding validation steps to tool calling paths, or how the system handles edge cases where validation itself might be uncertain. The complexity classification routing represents a sensible engineering optimization, but the presentation doesn't discuss how classification accuracy affects user experience, what happens when simple requests are misrouted to completion models and fail, or how the system learns to improve routing over time. Overall, this case study provides authentic insights into the operational challenges of production agent systems at scale, grounded in multi-month deployment experience serving real users. The challenges identified—particularly around memory architecture, tool safety, and computational efficiency—represent genuinely hard problems that any organization deploying autonomous agents will encounter. The solutions discussed are honest about remaining gaps while offering pragmatic approaches that have enabled production deployment.