AskNews developed a news analysis platform that processes 500,000 articles daily across multiple languages, using LLMs to extract facts, analyze bias, and identify contradictions between sources. The system employs edge computing with open-source models like Llama for cost-effective processing, builds knowledge graphs for complex querying, and provides programmatic APIs for automated news analysis. The platform helps users understand global perspectives on news topics while maintaining journalistic standards and transparency.
AskNews is a news intelligence platform developed by Emergent Methods, founded by Robert Caulk, who has an academic background in computational mechanics and machine learning. The platform represents a compelling case study in applying LLMs to the challenge of news analysis at scale, with a particular focus on reducing bias, enhancing diversity of perspectives, and making global news accessible across language barriers.
The company was founded in the wake of ChatGPT’s emergence, with the explicit goal of leveraging new AI capabilities to solve long-standing problems in news consumption and analysis. Caulk brings both technical expertise and a personal connection to journalism—his father was an editor at the Rocky Mountain News for 20 years—which informs the platform’s emphasis on journalistic standards and ethics.
One of the most distinctive aspects of AskNews’s architecture is their use of edge computing with open-source LLMs. Rather than routing all processing through cloud-based APIs like OpenAI, they deploy models like Llama 2 and Llama 3.1 at the “edge”—meaning the point where articles are ingested. This architectural decision has several important implications for their LLMOps strategy:
At the edge, their LLMs perform structured extraction on each article to capture:
This metadata extraction transforms unstructured news articles into structured data that can be queried, filtered, and compared. The ability to distinguish between facts, analysis, and opinion within articles is a key differentiator from traditional news aggregation.
After enrichment, articles are converted to embeddings—vector representations of their semantic content. With 500,000 articles processed daily, this creates a massive vector space that enables:
The clustering approach groups semantically similar articles together (e.g., all coverage of “Russia-Ukraine” or “Israel-Palestine” into topic clusters), which then enables cross-source analysis within each cluster.
Beyond simple clustering, AskNews builds a knowledge graph by extracting relationships between entities. This graph captures connections like “person X has relationship Y with organization Z” across all ingested articles. The graph serves as the foundation for their “graph chat” capability in the News Plunker analyst tool, where users can query not just individual articles but the interconnected web of entities and relationships across global news coverage.
The architecture employs a tiered approach to LLM usage that balances cost, capability, and transparency:
This hybrid approach acknowledges that while transparency is paramount for the data processing layer, users may want to use different models for their analytical queries.
A core philosophical principle of AskNews is what Caulk calls “algorithmic enforcement of diversity.” This is not about demographic diversity but rather ensuring that news synthesis includes perspectives from multiple countries, languages, and political orientations. Within each topic cluster, they actively sample from different source countries—France, Russia, Ukraine, Mexico, etc.—to ensure the meta-narrative isn’t dominated by any single perspective.
Importantly, the platform deliberately avoids pre-labeling sources with political leanings (unlike competitors such as Ground News). Caulk argues that labeling Fox News as “right-leaning” a priori prevents users from recognizing when that outlet might produce a balanced report. Instead, bias is detected at the article level based on the content itself—its reporting voice, use of sensational language, and subjective framing.
The platform even includes sources like RT (Russian state media), not because it’s reliable, but because transparency about what different sources are claiming is itself valuable. Often, these sources surface in the “contradictions” analysis, showing where they diverge from other coverage.
The consumer-facing interface allows users to query global news and receive synthesized summaries with citations back to original sources. Users can explore topics, identify contradictions between sources, and “talk to the news” through natural language queries.
An analyst-focused tool that allows users to:
Partners like the University of Texas Autonomy Lab use the API to align and detect misinformation. Their system takes Reddit posts and queries AskNews to find diversified sources related to the claim, enabling assessment of whether the post aligns with or contradicts established reporting.
Key customer segments include:
Approximately 60% of AskNews’s processing is devoted to non-English languages. The platform maintains a transparency dashboard showing real-time statistics on language distribution and source coverage. This multilingual capability is presented as a major differentiator, as it allows analysts to access distilled information from sources they couldn’t otherwise read.
The translation process is viewed as a form of transformation that adds value while also potentially addressing some copyright concerns—translated and distilled content, often reduced to single sentences per article, is distinct from simply republishing original content.
AskNews navigates the contentious copyright landscape through several mechanisms:
The platform embeds transparency at multiple levels:
Upcoming features include natural language alerts that can be customized and shared to WhatsApp or Telegram groups—particularly valuable for NGOs monitoring fast-moving crises who need immediate, translated, and distilled updates pushed to their coordination channels.
An interesting validation moment came during the 2024 US presidential election, when AskNews’s bot predicted a Trump victory based on signals about underestimated Republican turnout, while Caulk personally expected a Democratic win based on mainstream media consumption. The bot’s correct prediction demonstrated the value of algorithmically-enforced diversity in cutting through potentially biased reporting.
The company operates with a team of approximately 5 people but claims to function at the level of a 20-person company due to AI augmentation of their own workflows—a practical demonstration of the productivity gains possible when AI is deeply integrated into operations.
Snorkel developed a specialized benchmark dataset for evaluating AI agents in insurance underwriting, leveraging their expert network of Chartered Property and Casualty Underwriters (CPCUs). The benchmark simulates an AI copilot that assists junior underwriters by reasoning over proprietary knowledge, using multiple tools including databases and underwriting guidelines, and engaging in multi-turn conversations. The evaluation revealed significant performance variations across frontier models (single digits to ~80% accuracy), with notable error modes including tool use failures (36% of conversations) and hallucinations from pretrained domain knowledge, particularly from OpenAI models which hallucinated non-existent insurance products 15-45% of the time.
Stripe, processing approximately 1.3% of global GDP, has evolved from traditional ML-based fraud detection to deploying transformer-based foundation models for payments that process every transaction in under 100ms. The company built a domain-specific foundation model treating charges as tokens and behavior sequences as context windows, ingesting tens of billions of transactions to power fraud detection, improving card-testing detection from 59% to 97% accuracy for large merchants. Stripe also launched the Agentic Commerce Protocol (ACP) jointly with OpenAI to standardize how agents discover and purchase from merchant catalogs, complemented by internal AI adoption reaching 8,500 employees daily using LLM tools, with 65-70% of engineers using AI coding assistants and achieving significant productivity gains like reducing payment method integrations from 2 months to 2 weeks.
SuperLinked, a company focused on vector search infrastructure, shares production insights from deploying information retrieval systems for e-commerce and enterprise knowledge management with indexes up to 2 terabytes. The presentation addresses challenges in relevance, latency, and cost optimization when deploying vector search systems at scale. Key solutions include avoiding vector pooling/averaging, implementing late interaction models, fine-tuning embeddings for domain-specific needs, combining sparse and dense representations, leveraging graph embeddings, and using template-based query generation instead of unconstrained text-to-SQL. Results demonstrate 5%+ precision improvements through targeted fine-tuning, significant latency reductions through proper database selection and query optimization, and improved relevance through multi-encoder architectures that combine text, graph, and metadata signals.
