## Overview John Snow Labs has developed a medical chatbot product that serves as a comprehensive medical research assistant, with a particular focus on automating the traditionally labor-intensive process of conducting academic literature reviews. The presentation, delivered by Thea Bach (Head of Product at John Snow Labs), showcases how the company has productionized domain-specific LLMs to address real-world challenges in medical research workflows. The core problem being addressed is the significant time and resource burden of conducting literature reviews in academic and professional medical settings. Traditional literature reviews can take weeks to months, require highly experienced research teams, and involve managing information overload while ensuring balanced, unbiased synthesis of available research. The solution leverages LLM technology combined with retrieval-augmented generation (RAG) to automate the first four to five steps of the literature review process, dramatically reducing the time required from weeks or months to minutes. ## Technical Architecture The medical chatbot is built using a RAG (Retrieval-Augmented Generation) architecture that combines information retrieval with text generation. This is a critical LLMOps design decision that allows the system to ground its responses in verified, up-to-date medical literature rather than relying solely on the LLM's parametric knowledge. The system uses proprietary LLMs that have been specifically built and fine-tuned by John Snow Labs for the medical domain. According to the presentation, these models have demonstrated state-of-the-art accuracy on benchmarks used by the Open Medical LLM Leaderboard, reportedly surpassing other high-performance models including PaLM 2, Med-PaLM 2, GPT-4, and Llama 3. However, it's worth noting that these are claims made by the vendor themselves, and independent verification of these benchmarks would be advisable for those considering adoption. ## Knowledge Base Infrastructure A significant component of the LLMOps infrastructure is the comprehensive medical research knowledge base that indexes major medical publication databases. The sources mentioned include: - PubMed - arXiv - bioRxiv - MDPI - Other open access sources (with the list continuously growing) The knowledge base is updated on a daily basis, which is an important operational consideration for maintaining current research coverage. This represents a significant data pipeline operation that must run reliably in production to ensure users have access to the latest publications. The system also supports custom knowledge bases compiled from users' proprietary documents. These custom knowledge bases support automatic ingestion of PDF and text documents and can detect and incorporate changes observed in provided document repositories. This suggests an event-driven or polling-based document ingestion pipeline that monitors for updates. The system is described as being "adapted to handle large sets of documents," indicating scalability considerations have been addressed in the architecture. ## Production Features and Capabilities The chatbot includes several production-grade features that demonstrate mature LLMOps practices: **Explainability and Evidence Citation**: Every response includes references to the sources used to compile answers. This is critical for medical applications where evidence-based responses are essential. The literature review feature specifically provides reasoning for why each filter was validated or invalidated, and evidence from original papers supporting data point extractions. **Hallucination Prevention Safeguards**: The system includes explicit safeguards to prevent hallucination, which is particularly important in the medical domain where incorrect information could have serious consequences. The presentation mentions these safeguards as a key feature, though specific implementation details are not provided. **Adaptive Communication Styles**: The system can adapt to different tones, styles, and formats according to user preferences, and enterprise customers can configure custom brand voice and communication styles. **Smart Reference Ranking**: References are intelligently ranked to prioritize the most relevant information when responding to user queries, suggesting some form of relevance scoring or reranking mechanism in the retrieval pipeline. ## Literature Review Workflow The automated literature review feature represents a sophisticated application of LLM technology to a structured research workflow. The process involves several steps: **Step 1 - Source Selection and Keyword Search**: Users select target knowledge bases and enter search keywords. The system provides immediate feedback showing the list of relevant studies matching those keywords. In the demonstrated example, searching for "tissue regeneration" and related terms across selected databases returned approximately 1,600 relevant articles initially. **Step 2 - Data Extraction Definition**: Users define in plain English what data points they want to extract from each paper. Examples given included: material used for scaffolds, proposed improvements, implementation period, in vivo experimentation duration, type of bone defect considered, and similar structured data points. **Step 3 - Inclusion/Exclusion Criteria**: Users specify filtering criteria in natural language. Examples included "studies which are validated in vivo" for inclusion and excluding "technical reports on materials without in vivo testing" or review papers. This natural language interface for defining complex research criteria is a notable UX design choice that lowers the barrier to use. **Step 4 - Additional Filters**: Users can apply filters based on publication date, impact factor, or article type. These filters are applied on top of the LLM-based inclusion/exclusion criteria. **Processing and Results**: The system processes all matching documents, with results color-coded: white indicates insufficient information to determine inclusion status (requiring manual review), red indicates exclusion by defined criteria, and green indicates inclusion with all necessary data extracted. Each extraction includes mouse-over access to the reasoning and evidence supporting the LLM's decision. ## Performance and Scalability The demonstrated example processed 271 documents (after filtering from an initial 1,600) in approximately 7 minutes and 10 seconds. This represents a dramatic improvement over traditional literature review timelines of weeks to months. However, users should note that post-processing work is still required, including data normalization (as measurement units and reporting formats vary across papers) and the actual writing of the literature review paper itself. The presentation mentions that support for these additional steps is "in progress" for future releases. The enterprise version is described as "built to scale and accommodate a growing number of documents and interactions very smoothly," supporting unlimited knowledge bases, users, and groups. This suggests the infrastructure has been designed with horizontal scalability in mind. ## Deployment Options The product offers two deployment models, reflecting common LLMOps patterns for different customer needs: **Professional (SaaS)**: A subscription-based offering accessible via browser at chat.johnsnowlabs.com. This includes all core features with a 7-day free trial. **Enterprise (On-Premise)**: Allows the chatbot to be installed and run on the organization's own servers for enhanced security, privacy, and control. This includes single sign-on (SSO) integration and API access for developers to integrate chatbot features into broader processing workflows. The API offering is particularly significant from an LLMOps perspective, as it enables programmatic access to the literature review and other features, allowing integration into automated research pipelines or custom applications. ## Limitations and Considerations While the presentation is promotional in nature, several honest limitations were acknowledged: - The response context in standard chat mode is limited to maximum 20 references - Data normalization is still a manual effort required post-extraction (units, time periods, etc. vary across papers) - The "white" color-coded results indicate cases where the LLM couldn't make a determination, requiring human review - Users may need to iterate on their prompts if inclusion/exclusion criteria are too restrictive or data point definitions don't yield useful extractions - The actual writing of the literature review paper is not yet automated (though mentioned as a future feature) ## Additional Features Beyond literature review, the chatbot includes several other production features relevant to medical LLMOps: - **Document Q&A**: Upload up to 10 documents per session for direct querying, summarization, or translation - **Medical Agents**: Approximately 9 specialized agents for tasks including de-identification, entity extraction, relation extraction, and assertion status extraction from unstructured documents - **Conversation History and Bookmarking**: Users can track previous runs, bookmark important conversations, and clone literature reviews to use as templates for follow-up analyses The ability to clone literature reviews for incremental updates is a thoughtful feature for ongoing research, allowing users to pick up new publications without reconfiguring the entire analysis from scratch. ## Summary Assessment John Snow Labs has built a production-ready LLM system that addresses a genuine pain point in medical research. The RAG architecture, domain-specific model tuning, daily knowledge base updates, and explainability features represent solid LLMOps practices. The system appears to be well-designed for the target use case, though prospective users should validate the benchmark claims independently and understand that human oversight and post-processing remain necessary components of the workflow. The offering of both SaaS and on-premise deployment options demonstrates flexibility in meeting different organizational security and compliance requirements common in healthcare settings.