## Overview HubSpot achieved a significant milestone by becoming the first CRM provider to launch a Deep Research Connector with ChatGPT, representing a pioneering implementation of the Model Context Protocol (MCP) for production AI agent integration. This case study demonstrates how HubSpot engineered a sophisticated LLMOps solution that democratizes AI access for over 250,000 businesses while maintaining enterprise-grade security and user-level permissions. The project showcases the evolution from local MCP implementations to remote, production-ready systems that can scale across large enterprise environments. The initiative emerged from HubSpot's recognition that their initial local MCP server, while functional for early testing, created significant barriers to adoption. Users had to manually create Private App Access Tokens and configure MCP clients locally, which limited usage to technically sophisticated developers. The new remote MCP server eliminates these friction points by providing browser-based access through trusted OAuth flows, representing a fundamental shift in how AI tools can be integrated into enterprise workflows. ## Technical Architecture and Implementation HubSpot's engineering team made a strategic decision to build their MCP server infrastructure in-house rather than utilizing third-party solutions like Cloudflare's turnkey offerings. This "build vs. buy" decision was driven by their need to leverage existing enterprise infrastructure including CI/CD pipelines, auto-scaling capabilities, observability tools, and region-aware API routing built on Cloudflare Workers. The homegrown approach also provided direct control over OAuth integration and user-level permissions, which proved crucial for enterprise security requirements. The core architecture centers around a Java-based implementation using the MCP Java SDK within a Dropwizard microservice framework. This choice aligned with HubSpot's existing technical stack, where backend libraries and infrastructure tooling are standardized around Java microservices. The team extended the MCP Java SDK to support their specific enterprise requirements, demonstrating the flexibility of the SDK even in complex production environments. A critical technical challenge involved implementing Streamable HTTP support for the MCP protocol. At the time of development, the Java SDK lacked official support for the newly standardized Streamable HTTP transport protocol, which provides formalized structure for both streaming and non-streaming use cases. HubSpot's team implemented custom transport protocols to bridge this gap, creating a solution that avoids the complexity of Server-Sent Events (SSE) while maintaining full MCP protocol compliance within a stateless HTTP environment. The team deliberately chose to avoid SSE transport protocol for several practical reasons. First, their use case focused primarily on tool calls rather than server-to-client notifications. Second, SSE requires long-lived connections that can be challenging for load balancers, especially in environments with frequent auto-scaling. Third, managing stateless HTTP requests proved significantly simpler than session management. Their custom transport implementation creates ephemeral sessions for each HTTP call, maintaining protocol compliance while optimizing for their infrastructure constraints. ## Service Discovery and Tool Registration One of the most innovative aspects of HubSpot's MCP implementation is their distributed service discovery system built on top of CHIRP (Cool HubSpot Inter-process Remote Procedures), their internal RPC framework. This system automatically discovers and registers MCP tools across their entire infrastructure through a simple annotation-based approach. Services can mark RPC methods with `@ChirpTool` annotations to indicate they should be exposed as MCP tools, democratizing AI integration across hundreds of development teams. The MCP Gateway service periodically polls for services deployed with these annotations, fetches their specifications, and automatically converts each RPC to MCP spec-compliant tools. This approach eliminates manual configuration, gateway redeployments, and coordination overhead that typically plague enterprise AI integrations. Teams can deploy updates to their services and have their MCP tools immediately available to AI agents, creating a self-service model for AI tool development. This distributed approach enables rapid innovation at the edges while maintaining strategic coordination at the center. However, the team acknowledges the need for governance and guardrails to ensure customers experience a cohesive, unified interface rather than a fragmented reflection of their organizational structure. The balance between enabling team autonomy and maintaining user experience consistency represents a key LLMOps challenge for large organizations. ## Security and Authentication Architecture HubSpot's security implementation for their MCP server addresses complex enterprise requirements around user-level permissions and OAuth integration. Their historical approach to app integrations involved account-wide installations with admin-level access, which worked well for background data synchronization but proved inadequate for user-facing AI interactions. The MCP specification's emphasis on user consent and data access control required significant adaptations to their existing OAuth infrastructure. The team implemented a phased approach to user-level permissions rather than overhauling their entire public API infrastructure simultaneously. They created a dedicated gateway API at mcp.hubspot.com specifically for CRM search, associations, and property fetching, fully instrumented to respect user permissions in the MCP context. This pragmatic approach avoided breaking existing integrations while enabling proper permission enforcement for AI agent interactions. Their OAuth implementation captures four critical pieces of information: client_id (app/client info), hub_id (account ID), user_id (installer identity), and scopes (granted permissions). The system intersects app permissions with user-specific permissions, ensuring that AI agents cannot access data beyond what the user would normally see in HubSpot's native UI. This granular approach contrasts with their historical model where API tokens were intentionally decoupled from specific user identities to support background operations. The permission enforcement system operates at multiple levels. App-level scopes provide coarse-grained access control (e.g., `crm.objects.contacts.read`), while user-specific permissions handle granular restrictions like team-based access, unassigned records, and field-level permissions. The MCP server computes the intersection of these permissions at runtime, accepting some performance overhead to ensure precise access control for the nondeterministic queries generated by AI agents. ## Query DSL Design for AI Integration A particularly sophisticated aspect of HubSpot's implementation is their design of a query Domain Specific Language (DSL) that bridges the gap between natural language user inputs and complex CRM search requirements. The challenge involved compressing HubSpot's sophisticated Public Search API, which supports rich data structures for complex queries, into a single query string that AI models can reliably generate. The token-based DSL follows a `key[:operator]:value` pattern with space-separated tokens, supporting everything from basic object searches (`object_type:contacts`) to complex filters with operators like `contains_token`, `has_property`, and `in/not_in` for multi-value comparisons. The system handles associated object searches (`associated_companies:12345`), pagination (`limit:100 offset:20`), and sorting (`sort:lastmodifieddate:desc`). The DSL design philosophy prioritizes predictable structure and clear examples over maximum flexibility. The team deliberately excluded features like boolean OR logic, nested expressions, and relative date syntax because these constraints actually improve AI model reliability. Their key insight is that AI models excel at pattern matching, so a consistent, example-rich DSL outperforms more flexible but ambiguous query languages. Comprehensive examples in tool descriptions dramatically improve model performance. For instance, showing that "Find contacts in companies acme & hubspot" translates to `object_type:contacts associated_companies:in:acme,hubspot` teaches the model to understand the pattern. The system also implements helpful error messages to allow models to self-correct when they generate invalid queries. ## Production Deployment and Scalability HubSpot's MCP server implementation demonstrates several key LLMOps principles for production deployment. The system leverages their existing enterprise infrastructure including auto-scaling, observability, and region-aware API routing, ensuring that the AI integration benefits from the same reliability and performance characteristics as their core CRM platform. The stateless HTTP design enables horizontal scaling without session affinity requirements, crucial for handling variable AI agent workloads. The team accepts some performance overhead from runtime permission computation in exchange for precise access control, representing a classic LLMOps tradeoff between security and performance. The distributed service discovery system scales naturally with their microservices architecture, allowing hundreds of teams to contribute AI tools without centralized bottlenecks. This approach enables rapid iteration and innovation while maintaining system stability through established CI/CD processes. ## Future Evolution and Lessons Learned HubSpot's implementation provides valuable insights for organizations building production AI integrations. Their decision to build in-house rather than using third-party solutions enabled deeper integration with existing infrastructure and security models, though it required significant engineering investment. The phased approach to user-level permissions demonstrates how organizations can incrementally evolve their systems to support AI use cases without breaking existing functionality. The team identifies several areas for future enhancement, including expanding their service account model to all APIs, implementing OAuth 2.1 standards more broadly, and contributing their Streamable HTTP implementation back to the open-source MCP Java SDK. Their experience suggests that successful LLMOps implementations require balancing rapid AI development cycles with enterprise security and reliability requirements. The case study illustrates how sophisticated enterprise software companies can successfully integrate AI capabilities while maintaining their existing security posture and user experience standards. HubSpot's approach demonstrates that LLMOps success depends not just on AI model capabilities, but on thoughtful engineering of the surrounding infrastructure, security models, and integration patterns that enable AI to work seamlessly within existing enterprise workflows.