## Company Background and Evolution Patho AI, founded by CEO Ching Kyong Lamb, began as an NSF SBIR grant-funded company two years ago, initially focused on LLM-driven drug discovery applications. The company has since pivoted to building expert AI systems for large corporate clients, moving beyond traditional RAG (Retrieval Augmented Generation) systems to develop what they term "Knowledge Augmented Generation" (KAG) systems that perform research and advisory roles based on clients' specific domains of expertise. The fundamental challenge that Patho AI identified was that many enterprise clients were requesting AI systems capable of sophisticated reasoning and advisory functions that traditional RAG systems couldn't adequately address. While RAG systems excel at retrieving relevant documents based on semantic similarity, they struggle with complex numerical calculations, multi-hop reasoning, and the type of strategic thinking that human experts employ in their decision-making processes. ## Core Innovation: From RAG to KAG The company's primary innovation centers on transitioning from traditional vector-based RAG to Knowledge Augmented Generation (KAG), which integrates structured knowledge graphs to provide more accurate and insightful responses. The key philosophical difference is that while RAG systems primarily retrieve information, KAG systems are designed to understand and reason over interconnected knowledge structures. The foundation of their approach rests on a conceptual framework they call the "wisdom graph" - a systematic method of preserving and connecting domain expertise through interconnected relationships. This graph represents not just static information but the dynamic thought processes and comprehensive taxonomy of specific domains of expertise. The wisdom graph architecture consists of several interconnected components: wisdom (the core reasoning engine), knowledge (facts and data), experience (historical patterns and outcomes), insights (derived patterns and strategic recommendations), and situational awareness (current context and conditions). ## Technical Architecture and Implementation ### Multi-Agent Orchestration with Node-RED One of the more interesting technical decisions made by Patho AI was their choice to use Node-RED, typically known as a workflow automation tool for IoT applications, as their orchestration platform for multi-agent LLM systems. While this might seem unconventional, the CEO draws parallels to IBM's approach in IoT projects where Node-RED served as a no-code interface overlaying more complex underlying systems. The Node-RED implementation allows for flexible prototyping and rapid development of complex state machine workflows. The system includes specialized AI agent nodes that can interface with different LLM providers including OpenAI, Anthropic, and on-premises models. This multi-model approach provides both flexibility and redundancy, allowing clients to choose models based on their specific requirements for accuracy, cost, or data privacy. ### Knowledge Graph Implementation The system utilizes Neo4j as the graph database backend, leveraging Cypher queries for complex graph traversals. The knowledge graph structure goes beyond simple entity-relationship models to capture the nuanced decision-making patterns that human experts use. The graph is continuously updated by different specialized agents, each responsible for maintaining specific aspects of the domain knowledge. The extraction process for populating the knowledge graph uses a hybrid approach that combines automated LLM-based extraction with human expert validation. This hybrid model addresses one of the key challenges in knowledge graph construction - ensuring both coverage and accuracy. The automated LLM graph transformer handles the initial extraction, while domain experts prune the graph by removing irrelevant relationships and validating the taxonomy structure. ## Production Use Case: Competitive Intelligence System ### Problem Statement A specific client implementation involved building a competitive intelligence system for a company that previously relied on their marketing department for competitive analysis. The client wanted to automate this process using AI while maintaining the strategic depth and insight that human analysts provided. Traditional RAG systems proved inadequate for answering sophisticated strategic questions like "How can I win against my competitor in this market space?" because such questions require multi-faceted analysis across various data sources and the ability to perform complex reasoning over numerical data. ### System Architecture The competitive intelligence system maps the abstract wisdom graph concept to concrete business functions: - **Wisdom Engine**: Acts as the supervisory orchestration agent that makes high-level decisions about information gathering and analysis strategies - **Knowledge Component**: Stores and manages market data, competitor information, and industry intelligence - **Experience Component**: Maintains historical campaign data and past strategic initiatives with their outcomes - **Insight Component**: Houses industry analysis and emerging trend identification - **Situational Awareness**: Provides real-time performance metrics and current market positioning - **Strategy Generator**: The decision-making output component that synthesizes all inputs into actionable recommendations ### Multi-Agent Workflow The system employs multiple specialized agents working in coordination: - **Insight Agent**: Monitors social media and other public data sources to track product sentiment and competitor activities - **Market Analysis Agent**: Processes quantitative market data and performs numerical calculations - **Strategy Agent**: Synthesizes inputs from other agents to generate strategic recommendations - **Wisdom Agent**: Oversees the entire process and ensures coherent decision-making across all components All agents contribute to and draw from a centralized knowledge graph that serves as the system's memory and reasoning foundation. This centralized approach ensures consistency while allowing for specialized processing by domain-specific agents. ## Technical Challenges and Solutions ### Limitations of Vector RAG The presentation highlighted several key limitations of traditional vector-based RAG systems that their KAG approach addresses: **Numerical Reasoning Deficiency**: Vector stores excel at semantic similarity matching but struggle with complex numerical calculations. For marketing analysis requiring precise quantitative insights, traditional RAG might return relevant passages containing numbers but cannot perform the calculations needed to derive actionable metrics. The KAG system addresses this by storing numerical data in structured graph format and enabling function calls for mathematical operations. **Multi-hop Query Complexity**: Traditional RAG systems often fail at questions requiring multiple logical steps or relationships. The graph-based approach naturally supports traversing multiple relationships to answer complex, multi-faceted queries that are common in strategic business analysis. **Context Preservation**: Vector similarity can lose important contextual relationships between entities. Knowledge graphs maintain these relationships explicitly, enabling more nuanced understanding of how different business factors interact. ### Hybrid Architecture Benefits The system is designed to work alongside traditional RAG when appropriate, creating a hybrid architecture that can adapt to different types of queries: - Simple product information queries can be handled by traditional vector RAG with ChromaDB - Complex strategic questions requiring multi-hop reasoning are routed to the knowledge graph system with Neo4j and Cypher queries - The system can dynamically choose the appropriate approach based on query complexity and data requirements ## Performance and Evaluation ### Benchmarking Results Patho AI conducted internal benchmarking that showed significant improvements over traditional RAG approaches across several metrics: - **Accuracy**: 91% - attributed to the system's ability to work with structured data and maintain explicit relationships - **Flexibility**: 85% - reflecting the system's adaptability to different query types and domains - **Reproducibility**: High deterministic performance due to structured reasoning processes - **Traceability**: Clear audit trails showing how conclusions were reached - **Scalability**: Effective handling of growing data volumes and complexity While these metrics appear impressive, it's important to note that the specific benchmarking methodology and comparison baselines were not detailed in the presentation, which is common in vendor presentations but limits the ability to independently verify these claims. ### Real-world Impact The practical impact for the competitive intelligence client included automation of previously manual processes, more consistent analysis across time periods, and the ability to handle more complex strategic questions than traditional business intelligence tools. However, the presentation did not provide specific quantitative measures of business impact such as time savings, cost reduction, or improved decision outcomes. ## Deployment and Operational Considerations ### Technology Stack Flexibility The system architecture demonstrates thoughtful consideration of production deployment needs. The use of Node-RED provides a visual development environment that can potentially be maintained by non-technical stakeholders, while the underlying complexity is handled by more sophisticated components. The multi-model support (OpenAI, Anthropic, on-premises) allows organizations to balance performance, cost, and data privacy requirements. ### Development and Maintenance The hybrid knowledge extraction approach acknowledges the ongoing maintenance requirements of knowledge graph systems. By combining automated extraction with human expert validation, the system aims to balance automation benefits with accuracy requirements. This approach recognizes that domain expertise remains crucial for ensuring the quality and relevance of the knowledge representation. ### Open Source Components Patho AI has made some components available as open source, including what they describe as an "LLM graph RAG stack" that can be deployed via Docker. This approach to sharing core infrastructure while maintaining proprietary expertise in application and customization reflects a common pattern in the enterprise AI space. ## Critical Assessment and Limitations ### Presentation Context and Validation It's important to note that this case study is based on a conference presentation by the company's founder, which naturally focuses on successes and achievements rather than challenges or limitations. The benchmarking results, while impressive, lack detailed methodology descriptions that would allow for independent verification. The specific business impact metrics (ROI, time savings, error reduction) that would typically validate enterprise AI deployments are not provided. ### Technical Complexity Trade-offs While the KAG approach addresses genuine limitations of traditional RAG systems, it introduces significant complexity in terms of knowledge graph design, maintenance, and updating. The success of such systems heavily depends on the quality of the initial knowledge modeling and ongoing curation, which can be resource-intensive. The hybrid approach requiring domain expert involvement for graph pruning may limit the scalability benefits that organizations expect from AI automation. ### Generalizability Questions The competitive intelligence use case represents a specific type of analytical problem well-suited to knowledge graph approaches. The generalizability of this architecture to other enterprise use cases, particularly those with less structured decision-making processes or more diverse data types, remains an open question that would need to be validated through broader deployment. ## Innovation and Industry Implications Despite these limitations, Patho AI's approach represents meaningful innovation in addressing real limitations of current RAG systems. The integration of structured reasoning with LLM capabilities, the thoughtful use of multi-agent orchestration, and the hybrid approach to knowledge extraction all represent practical advances in enterprise AI system design. The work also demonstrates the evolution of LLMOps beyond simple question-answering systems toward more sophisticated expert system capabilities. This trend toward AI systems that can perform advisory and strategic functions represents a significant step toward more valuable enterprise AI applications. The choice to build on established tools like Node-RED and Neo4j, while potentially limiting in some ways, also demonstrates a pragmatic approach to enterprise deployment that prioritizes reliability and maintainability over cutting-edge technical innovation. This balance between innovation and practicality is often crucial for successful enterprise AI deployments. In the broader context of LLMOps evolution, this case study illustrates the growing sophistication of production LLM systems and the need for more complex orchestration, knowledge management, and reasoning capabilities as organizations move beyond basic chatbot applications toward systems that can truly augment human expertise and decision-making capabilities.