Varick Agents addresses the challenge of scaling deep customer engagement in enterprise AI transformation without exponentially increasing headcount. The company embeds forward-deployed engineers (FDEs) within client organizations to map existing workflows, re-engineer processes around AI, and deploy agents on top of legacy systems like NetSuite, SAP, and Salesforce. To overcome the bottleneck of finding enough high-quality FDEs who combine technical expertise with strong communication skills, Varick developed an internal FDE Agent system that augments their engineers through three stages: an engagement agent that synthesizes documentation and answers queries, a workflow agent that assists in building processes within their platform, and an autonomous assistant that handles routine client requests. The solution uses dependency graphs to represent company operations, custom post-trained models based on open-source foundations for clearer analysis, and reinforcement learning to improve knowledge graph traversal. This approach enables department-wide transformations delivering 25-75% ROI while maintaining the human-centered consulting experience clients require.
Varick Agents is a consulting firm focused on enterprise AI transformation that has developed sophisticated internal LLMOps tooling to scale their forward-deployed engineering model. The company works with large enterprises to transform entire departments through deeply embedded engineers who understand business processes and re-engineer them around AI capabilities. This case study is particularly interesting from an LLMOps perspective because it represents a meta-application: using LLMs in production to help deploy other LLMs in production for clients.
The central thesis presented is that execution of work is no longer the primary bottleneck for AI adoption in 2026, as model intelligence and integration capabilities have matured significantly. Instead, the critical constraint has shifted to understanding business context deeply enough to design workflows that AI can execute reliably. Every enterprise operates differently, with unique edge cases, exception handling, and informal processes that aren’t captured in documentation. Varick’s approach requires forward-deployed engineers to spend extensive time embedded with clients, mapping current workflows by interviewing process leads and uncovering the reality of how work actually happens versus how it’s documented.
The company identifies three core responsibilities for forward-deployed engineers. First, they must map how humans currently perform work, going beyond documented processes to understand edge cases and failure modes. For example, in a finance department, an FDE would interview leads across accounts payable, accounts receivable, card reconciliation, banking, billing, and financial planning to understand not just the golden path but what happens when things go wrong. This often reveals undocumented dependencies and informal processes that are critical to actual operations.
Second, FDEs must re-engineer processes around AI rather than simply applying AI to existing broken workflows. This is positioned as a key reason why many AI pilots fail to reach production or deliver ROI. The approach requires finding a balance: changing workflows enough to capture substantial efficiency gains while keeping them familiar enough that adoption doesn’t suffer. A typical redesign might automate four of eight steps completely, add human-in-the-loop intervention for three steps, and leave one step entirely manual due to risk or lack of clear value from automation.
Third, FDEs deploy agents on top of existing systems of record. Varick’s positioning here is that enterprises are married to platforms like NetSuite, SAP, Dynamics, and Salesforce after investing millions of dollars and years in migrations. Solutions that require migration away from these platforms are non-starters. Their Varick OS platform is designed to layer agents on top of existing systems with integrated governance and evaluation suites.
The challenge from an LLMOps perspective is that effective FDEs need to be in the top percentile technically while also possessing exceptional communication and interpersonal skills. This combination is rare and difficult to scale through hiring alone. Clients generate constant communication demands, sending hundreds of pages of documentation and emails around the clock as different process owners pull FDEs in different directions.
To address this scaling challenge, Varick developed an internal system called the FDE Agent, described as “Codex for FDEs.” The system has three stages of increasing autonomy.
The engagement agent serves as an intelligent assistant purpose-built for FDEs, replacing their previous workflow of uploading 150 pages of documentation to Claude and waiting for verbose, often incorrect analysis. This agent integrates multiple data sources including meeting notes from tools like Granola, synthesized documentation, and PowerPoint slides. FDEs can query it to answer questions like identifying whether Sarah mentioned in an email is the same person as Sarah in a Slack message, or determining who owns a particular process. This seemingly simple capability addresses a major time sink: frontier models struggle with entity resolution across documents, yet this is critical for understanding organizational workflows.
From an LLMOps perspective, this stage represents a retrieval-augmented generation system tailored to the specific information needs of forward-deployed work. The challenge is maintaining context across diverse document types and communication channels while providing responses that are both detailed enough to be useful and concise enough to be actionable.
The workflow agent embeds the engagement agent’s capabilities directly within Varick’s platform interface. As FDEs construct workflows in the platform, the agent provides real-time assistance by identifying missing edge cases, verifying process ownership, and ensuring that the workflow being built accurately reflects the process that needs to be engineered. This represents a co-pilot pattern where the AI works alongside the human engineer during the construction phase rather than just providing pre-work assistance.
The technical implementation involves having the agent work in conjunction with whatever model the FDE is using, whether Claude, Codex, or others. This suggests an architecture where the workflow agent maintains context about the specific client’s operations and can interject relevant information or warnings as the FDE builds out automation logic.
The third stage, still in development, aims to handle routine maintenance requests autonomously. When clients send emails requesting changes like redirecting a QC report to a different email address, the autonomous assistant should be able to process the request, query the company’s knowledge representation, make the appropriate workflow modification in the platform, and execute the change without FDE involvement. This frees FDEs to focus on high-value activities like client interviews and process discovery rather than minor configuration changes.
This stage represents a significant LLMOps challenge: building sufficient trust and reliability that an AI system can make production changes to business-critical workflows without human review. The system must understand natural language requests, map them to specific workflow components, verify that changes won’t break dependencies, and execute modifications correctly.
The foundation of the FDE Agent system is a dependency graph serving as a single source of truth for company operations. While acknowledging that various graph database solutions exist, Varick uses a dependency graph structure because enterprise workflows are typically linear with cycles. Process owners naturally think in terms of dependencies: person C shouldn’t handle something until persons A and B have approved it.
This choice of representation has important implications for LLMOps. Dependency graphs provide structure that can be validated and reasoned about programmatically. They make it easier to identify circular dependencies, missing approvals, and other structural issues in workflows. The graph structure also provides a more tractable search space for retrieval compared to unstructured documents.
A particularly interesting aspect of this case study is Varick’s decision to move away from frontier models for certain tasks through post-training of open-source models. The company identifies two distinct problems in the FDE workflow: generating high-quality output from extracted context, and extracting the right context in the first place.
For the first problem, frontier models like Claude proved surprisingly inadequate. The specific issue is verbosity: when performing long analysis tasks, frontier models fail to distinguish between details that matter to the client and details that can be glossed over. This is characterized as a skill that human consultants excel at, understanding what level of detail is appropriate for different audiences and contexts. Varick addresses this through post-training on top of open-source models, with specific mention of Qwen 2.5 as a preferred base model.
This represents a pragmatic LLMOps decision: recognizing that frontier models optimized for general capability may not perform optimally for specific enterprise use cases, especially when the desired output characteristics involve subjective qualities like appropriate level of detail. Post-training allows them to encode domain-specific knowledge about what makes a good process analysis from a consulting perspective.
The training data for this post-training presumably comes from examples of high-quality analyses produced by experienced FDEs, though the presentation doesn’t detail the data collection and curation process. This is a common challenge in LLMOps: building training datasets that capture expert knowledge in a specific domain.
For the second problem of extracting the right context from the knowledge graph, Varick employs reinforcement learning to train agents on custom tools designed for graph traversal. These tools address specific challenges that emerge in enterprise workflow mapping.
One tool focuses on entity resolution, ensuring that person A and person B are actually the same person. This is described as a frequent problem because companies often have multiple people with common names, and frontier models struggle with disambiguation across documents. Another category of tools identifies structural issues in the graph like redundancy cycles or violations of directed acyclic graph properties.
The RL environment exposes these custom tools and trains agents to use them effectively for context extraction. This approach is interesting from an LLMOps perspective because it represents a middle ground between pure prompting and full end-to-end training. Rather than trying to teach a model everything about graph traversal through prompts or fine-tuning, they build specialized tools that encapsulate domain logic and train the agent to orchestrate those tools appropriately.
The RL approach also allows for iterative improvement as new edge cases and challenges emerge in production. When the system makes mistakes in context extraction, those examples can feed back into the training process to improve tool selection and usage.
While the presentation focuses primarily on the internal FDE Agent tooling, it also touches on how Varick deploys AI for clients in production. The Varick OS platform is described as including governance and evaluation suites, suggesting mature LLMOps practices around monitoring, testing, and compliance.
The emphasis on building agents on top of existing systems of record rather than replacing them represents an important deployment philosophy. From an LLMOps perspective, this means the agents must interact with complex enterprise systems through their existing APIs and interfaces. This is likely more technically challenging than building greenfield systems, but it’s positioned as essential for enterprise adoption.
The company’s approach to process re-engineering also reflects LLMOps maturity. Rather than automating everything possible, they make explicit decisions about which steps should be fully automated, which need human-in-the-loop intervention, and which should remain fully manual. This risk-based approach to automation is more sophisticated than all-or-nothing strategies and likely contributes to higher production success rates.
Varick claims to deliver department-wide transformations that achieve 25-75% ROI through revenue uplift, cost savings, and risk mitigation. This is contrasted with point solutions that might deliver only 5-10% ROI by automating a single subprocess. The broader transformation approach is enabled by the deep engagement model that FDEs provide.
The company positions 2026 as the year of the forward-deployed engineer, with unprecedented demand for deep customer engagement to understand business use cases and drive AI adoption. However, they also acknowledge that this model is difficult to scale through hiring alone given the rare combination of skills required.
The FDE Agent system is presented as the solution to this scaling challenge, allowing individual FDEs to manage multiple client communications and processes that would otherwise require much larger teams. This represents a meta-application of AI: using LLMs in production to increase the productivity of the people who deploy LLMs in production for others.
While the presentation makes compelling points about the importance of deep business understanding for successful AI deployment, several claims should be examined critically. The assertion that knowledge work is “almost entirely solved” from an execution perspective is bold and not universally accepted in the industry. Many organizations still struggle with basic AI reliability and accuracy even when context is well-understood.
The statistics cited about AI pilot failure rates are described as “semi-outdated” by the presenter himself, though still relevant. It would be valuable to understand whether these failure rates are improving as tooling matures and whether Varick’s approach genuinely achieves higher success rates than alternatives.
The technical details about the FDE Agent system are relatively high-level. Key questions remain about training data quality and quantity for the post-trained models, the specific RL algorithms and reward functions used for knowledge graph traversal, and how the system handles errors and edge cases in production. The autonomous assistant stage is acknowledged as still in development, indicating that full automation of FDE tasks remains aspirational.
The dependency on human FDEs for initial process mapping and re-engineering also represents a potential bottleneck even with AI augmentation. While the FDE Agent may help individual engineers scale, the fundamental model still requires significant expert human time embedded with each client.
That said, the case study represents sophisticated thinking about LLMOps challenges in enterprise contexts. The recognition that frontier models may not be optimal for all tasks, the investment in custom tooling and training for specific use cases, and the careful attention to deployment on top of existing systems all reflect mature practices. The meta-application of AI to scale AI deployment is also an interesting pattern that may become more common as the industry matures.
This case study offers several lessons for LLMOps practitioners. First, context is king for enterprise AI applications, but extracting and maintaining that context at scale requires sophisticated tooling. Simple RAG approaches may not suffice when dealing with hundreds of pages of documentation, informal knowledge in emails and meetings, and complex dependency relationships.
Second, frontier models are not always the answer. Post-training open-source models for specific domains can deliver better results for particular use cases, especially when desired output characteristics involve subjective qualities like appropriate level of detail or tone.
Third, combining multiple AI techniques in a system can be more effective than relying on a single approach. The FDE Agent combines retrieval augmentation, custom model training, reinforcement learning, and structured knowledge representation to address different aspects of the problem.
Fourth, production deployment in enterprises requires meeting organizations where they are, working on top of existing systems rather than requiring wholesale replacement. This is more technically challenging but dramatically improves adoption prospects.
Finally, the human-in-the-loop remains critical for high-stakes business processes. The most successful deployments may be those that thoughtfully allocate work between AI and humans based on risk, value, and capability rather than automating everything possible.
Snorkel developed a specialized benchmark dataset for evaluating AI agents in insurance underwriting, leveraging their expert network of Chartered Property and Casualty Underwriters (CPCUs). The benchmark simulates an AI copilot that assists junior underwriters by reasoning over proprietary knowledge, using multiple tools including databases and underwriting guidelines, and engaging in multi-turn conversations. The evaluation revealed significant performance variations across frontier models (single digits to ~80% accuracy), with notable error modes including tool use failures (36% of conversations) and hallucinations from pretrained domain knowledge, particularly from OpenAI models which hallucinated non-existent insurance products 15-45% of the time.
Digits, a company providing automated accounting services for startups and small businesses, implemented production-scale LLM agents to handle complex workflows including vendor hydration, client onboarding, and natural language queries about financial books. The company evolved from a simple 200-line agent implementation to a sophisticated production system incorporating LLM proxies, memory services, guardrails, observability tooling (Phoenix from Arize), and API-based tool integration using Kotlin and Golang backends. Their agents achieve a 96% acceptance rate on classification tasks with only 3% requiring human review, handling approximately 90% of requests asynchronously and 10% synchronously through a chat interface.
OpenAI's Forward Deployed Engineering (FDE) team, led by Colin Jarvis, embeds with enterprise customers to solve high-value problems using LLMs and deliver production-grade AI applications. The team focuses on problems worth tens of millions to billions in value, working with companies across industries including finance (Morgan Stanley), manufacturing (semiconductors, automotive), telecommunications (T-Mobile, Klarna), and others. By deeply understanding customer domains, building evaluation frameworks, implementing guardrails, and iterating with users over months, the FDE team achieves 20-50% efficiency improvements and high adoption rates (98% at Morgan Stanley). The approach emphasizes solving hard, novel problems from zero-to-one, extracting learnings into reusable products and frameworks (like Swarm and Agent Kit), then scaling solutions across the market while maintaining strategic focus on product development over services revenue.
