## Overview This panel discussion features experts from multiple companies discussing generative AI use cases, return on investment, and the challenges of deploying LLMs in production across various industry verticals. The panelists include Camelia Aryafar (Senior Engineering Director at Google Cloud AI), Shibhi Roror (CEO and Co-founder of Symbol AI), Shingai Manga (Head of AI Education at Chain ML), and Manas Bhuyan (AI and Data Leader at Deloitte). The discussion provides a cross-industry perspective on where generative AI is delivering value and what obstacles organizations face when moving from experimentation to production. ## Panelist Backgrounds and Company Focus Areas The panel represents diverse perspectives on LLMOps. Symbol AI is described as a purpose-built AI platform for understanding and transforming unstructured conversation data, working with organizations that build products on top of communication channels like Zoom, call recordings, and email. Chain ML has developed an open-source platform called Council AI that uses agents and chains for various tasks to advance prompting capabilities, with a focus on self-service analytics where users can ask questions in natural language and receive responses after the LLM handles code generation. Google Cloud AI's teams focus on building search, recommendations, and personalization solutions, many augmented with generative AI. Deloitte's practice provides advisory services on data platform modernization and business value generation from cloud, data, and AI/ML technologies. ## Primary Use Cases for Generative AI in Production The panelists identified several core business functions where generative AI is being deployed: **Customer Service and Operations Improvement**: This emerged as a primary area where organizations are seeing real use cases. The ability to automate repetitive tasks and provide personal assistance creates productivity gains across customer-facing workflows. **Communication Data Processing**: Symbol AI's work focuses specifically on unlocking value from audio recordings, video meetings, emails, and chat data. Before LLMs, AI was already being used for intent identification, sentiment analysis, and transcription, but LLMs enable deeper use cases. Call summarization was highlighted as a particularly strong use case—while summarization was possible before, LLMs enable "20,000 different ways" to personalize summaries for each user receiving them. **Search and Recommendations**: Google Cloud's perspective emphasizes that generative AI is best viewed as an enhancement to existing applications rather than entirely new capabilities. Adding gen AI to search takes existing capabilities "to the next level" by enabling better summarization, personalization, and conversational search interfaces. **Self-Service Analytics**: Chain ML's focus on enabling users to ask questions in natural language, with LLMs handling code generation and returning responses in natural language, represents a significant productivity enhancement for business users who previously needed technical skills to query data. **Developer Productivity**: Multiple panelists mentioned coding assistance and copilot-style tools as unexpected but delightful applications that emerged from generative AI capabilities. **Healthcare Applications**: The discussion touched on how LLMs unlock previously difficult-to-use data sets. Clinician notes in hospital contexts were cited as an example—a startup called Hero AI, led by Dr. Devin Singh, is using LLMs to analyze clinician notes as an additional data point for reducing wait times in hospital emergency rooms. ## Four Pillars of Communication AI Applications Symbol AI's CEO outlined a framework for categorizing generative AI applications in communication-centric workflows: **Augmentation**: Using AI or language models to assist humans in their existing workflows while keeping humans in the loop. The goal is enhancement rather than replacement. **Automation**: Fully automating workflows that previously required human effort. The example given was CRM data entry after sales calls—salespeople should spend time selling rather than entering notes, and this workflow can now be fully automated. **Search**: Reducing the time required to find the right answer and information by searching through historic and current data, fusing multiple data forms together, and creating consensus. **Analytics**: This includes predictive analytics, passively recommended analytics, deep-diving into single information sources, or analyzing multiple sources together. ## Barriers to Production Deployment The panelists identified several significant challenges preventing organizations from moving generative AI from experimentation to production: **Regulatory Constraints**: Highly regulated industries like healthcare and finance face unique challenges. As one panelist noted, "failing fast is not an option in a healthcare setting," which limits the experimental approach that works in other contexts. **Organizational AI Literacy**: Companies that lack the foundational literacy for AI adoption struggle to leverage the technology effectively. This includes both the general workforce's ability to use AI tools and senior leadership's capability to deploy AI responsibly. **Evaluation Framework Development**: Product teams are currently doing a "two-way process"—experimenting with different use cases while simultaneously trying to figure out evaluation paradigms based on their data and customers. Questions around human-in-the-loop approaches, guardrails implementation, and infrastructure for model redundancy all require significant work outside of the actual language model usage. **Data and Platform Maturity**: The panelists emphasized that organizations exist at different maturity levels. Some industries, particularly TMT (Technology, Media, and Telecommunications), are ahead, while others are still using paper-based systems for production operations. As one panelist put it, trying to adopt generative AI without sorting out data and platform foundations is "jumping onto the 3.0 version without doing 1.0." **Foundational Model Selection**: Most companies won't build foundational models in-house and need to choose from hyperscalers or open-source options. This choice matters significantly for production outcomes. **Experimentation Culture**: AI development differs from classic software development, requiring a mindset of iteration, evaluation frameworks, and patience before getting things into production once they show value. **Scaling Challenges**: Much of current generative AI usage is "scratching the surface." Scaling to billions of users or large document volumes creates challenges for system reliability, responsiveness, and meeting SLOs. ## Recommendations for AI Adoption The panel offered practical guidance for organizations looking to adopt generative AI: **Value-Driven Approach**: Technology should not be the driver—start with business problems and find solutions using multiple technologies, of which generative AI can be one component. Identify what generative AI excels at: repetitive tasks where it can do 30-40% of work in seconds, and creative work like brainstorming where it provides instant ideation. **Start with Existing Applications**: Rather than building entirely new applications, look at where AI is currently being used and see how generative AI can augment and extend those capabilities. This reduces risk compared to greenfield development. **Leverage Service Providers**: For new applications, working with solution providers is often preferable to building from scratch, allowing organizations to validate use cases and business cases before significant investment. **Align with Objective Functions**: Start with the problems the organization was trying to solve before AI—those that drive the bottom line. Business leaders must articulate specific metrics (not vague goals like "maximize customer happiness") so AI tools can be applied to advance measurable objectives. ## Emerging Trends and Future Directions The panelists identified several trends expected to shape generative AI adoption: **Agents and Chains**: The ability to do prompt engineering in ways that allow additional control flow at runtime—adding agents for fact-checking, redaction, or other specific functions—provides options for more sophisticated applications. **Self-Service Analytics Growth**: The ability to literally "speak to your data" is expected to grow as a major application area. **Domain-Specific Models**: Rather than one general model solving all problems, organizations will likely use suites of specialized models based on their specific data and workflows. Real-time model selection based on performance evaluation for particular use cases is anticipated. **Productivity Emphasis**: Applications focused on improving productivity, whether through agents, internal developer tools, or enhanced search, will be major areas of emphasis. **Industry Consolidation**: Following patterns seen in cloud and big data, the expectation is that within 24-36 months, a few major players will build end-to-end technology stacks for generative AI capabilities, and industry will converge around those platforms. **Autonomous Agents (Cautionary View)**: While interest exists in autonomous agents, some panelists expressed caution, noting that even expert understanding of LLM behavior remains incomplete. The observation that current best practices for prompting include phrases like "take a deep breath and help me step by step" suggests the technology has significant maturation ahead before autonomous agents become reliable. ## Team Composition for GenAI Initiatives When asked about building generative AI teams, the panelists recommended a balanced approach. General software engineers can form the majority of the team since much of generative AI has been democratized. Infrastructure and ops backgrounds are valuable for compliance and scaling concerns. A few AI specialists are still needed, but the landscape has shifted from requiring mostly AI specialists with a few software developers to a more balanced composition. Solution engineering roles that bridge AI experts and industry needs have become increasingly important because AI tools are now more sophisticated than traditional analytics tools. For regulated industries like healthcare, having someone with relevant compliance experience (e.g., HIPAA) is also recommended.