Databricks developed an AI-generated documentation feature for automatically documenting tables and columns in Unity Catalog. After initially using SaaS LLMs that faced challenges with quality, performance, and cost, they built a custom fine-tuned 7B parameter model in just one month with two engineers and less than $1,000 in compute costs. The bespoke model achieved better quality than cheaper SaaS alternatives, 10x cost reduction, and higher throughput, now powering 80% of table metadata updates on their platform.
# Building a Custom LLM for Documentation Generation at Databricks
## Background and Initial Implementation
Databricks implemented an AI-generated documentation feature to automatically generate documentation for tables and columns in their Unity Catalog system. The initial implementation used off-the-shelf SaaS-based LLMs and was prototyped during a quarterly hackathon. The feature quickly gained traction, with over 80% of table metadata updates becoming AI-assisted.
## Production Challenges
The team encountered several significant challenges when moving to production:
- **Quality Control**
- **Performance Issues**
- **Cost Constraints**
## Custom Model Development
The team opted to build a bespoke model with these key characteristics:
- **Development Metrics**
- **Training Data Sources**
## Model Selection and Evaluation
- **Model Selection Criteria**
- **Selected Architecture**
- **Evaluation Framework**
## Production Architecture Components
- **Core Infrastructure**
- **Key Features**
## Performance Improvements
- **Quality**
- **Cost Efficiency**
- **Throughput**
## Production Optimization Techniques
- **Prompt Engineering**
- **Infrastructure Optimization**
## Monitoring and Maintenance
- **Quality Assurance**
- **Deployment Strategy**
## Key Learnings and Best Practices
- **Model Development**
- **Infrastructure**
- **Cost Management**
## Results and Impact
- **Business Impact**
- **Technical Achievements**
The case study demonstrates that building custom, fine-tuned models for specific use cases can be both practical and advantageous, offering better control, lower costs, and improved performance compared to general-purpose SaaS LLMs. The success of this implementation provides a blueprint for other organizations looking to deploy LLMs in production for specific use cases.
Start deploying reproducible AI workflows today
Enterprise-grade MLOps platform trusted by thousands of companies in production.
