Remitly presents an interesting case study in applying LLMs to solve a critical business challenge in the highly regulated financial services sector. The company, which facilitates international money transfers across 170 countries and serves approximately 8 million active customers, faced significant challenges in ensuring marketing compliance across multiple jurisdictions and languages. The Problem Space: The company operates in a complex regulatory environment where marketing materials must comply with different regulations across 170 countries. Their traditional process involved multiple rounds of review between the creative team and legal team, with feedback scattered across various communication channels (Google Docs comments, Slack messages, meetings). This led to significant inefficiencies, slower time-to-market, and constant context switching for marketers. Technical Implementation: The development team approached the solution with a carefully planned strategy: Initial Scope and Architecture: * Started with a focused scope: English language and text-only content * Built a system that analyzes content against internal guidelines derived from various regulatory requirements * Implemented automated checks for multiple compliance aspects: * Claim verification * Required disclaimers * IP infringement detection * Spelling and grammar * Other regulatory compliance checks Data Preparation and Training: One of the most significant challenges was creating structured training data. The team had to: * Collect and organize historical feedback from various sources (Google Docs, Slack, meeting notes) * Create a structured corpus of content-feedback pairs * Develop evaluation datasets for model testing * Iterate through multiple versions of the model before beta release The development process took 15 days of actual work spread across 3 months, with much of the time invested in data preparation and model refinement. Model Evaluation and Quality Assurance: The team implemented a robust evaluation framework: * Achieved 95% accuracy and 97% recall in issue identification * Balanced precision and recall through multiple iterations * Incorporated both objective metrics and user feedback * Refined the presentation of issues and guidance to improve usability * Conducted beta testing with select marketers before wider deployment Production Implementation: The production system follows a straightforward but effective workflow: * Marketers upload content to the system * The AI analyzes the content against guidelines * System identifies potential issues and provides specific guidance * Marketers can iteratively improve content based on AI feedback * Final legal review serves as a quality check rather than a detailed review process The system leverages: * Internal guidelines stored in Confluence pages and documents * Custom-engineered prompts * An analysis engine that combines the guidelines and prompts Future Roadmap and Scaling: The team has outlined several important expansion areas: * Extending capabilities to handle audio, image, and video content * Adding support for additional languages beyond English * Integration with native creative tools (Adobe, Figma) to provide real-time compliance feedback * Enhanced automation of the legal review process Lessons and Insights: Several key learnings emerged from this implementation: * The importance of high-quality training data and the significant effort required to create it * The value of iterative development and testing before wider deployment * The need to balance technical metrics with real user feedback * The importance of making the tool fit into existing workflows Technical Challenges and Solutions: * Data Structure: Converting unstructured feedback from various sources into structured training data * Model Evaluation: Developing appropriate metrics and evaluation processes * Integration: Planning for future integration with various content creation tools * Scalability: Designing the system to handle multiple content types and languages Impact and Results: While the system is still in the process of full deployment, early results show: * Significant reduction in review cycles * Improved efficiency in the marketing content creation process * Better adherence to compliance requirements * Reduced workload for the legal team * Faster time-to-market for marketing materials This case study demonstrates a practical application of LLMs in solving real-world business problems, particularly in regulated industries. The methodical approach to implementation, focus on data quality, and attention to user needs provides valuable insights for similar projects in other domains.