|
Welcome back! Anthropic and OpenAI had better hope more companies don’t follow the example of AT&T. The telecommunications firm plans to keep its employees’ spending on Anthropic and OpenAI models flat in the coming years by using more open-source models such as Nvidia’s Nemotron, according to Mark Austin, an AT&T vice president. Austin oversees AI used by the company’s 100,000 employees for everything from coding and financial analysis to tools for HR and customer support staff. While AT&T also offers some AI-powered features to customers, like a chatbot in its mobile app, the majority of its AI use is internal. Over the past year, he said, AT&T boosted its use of open-source models to the point where such models now power 40% of employees’ AI queries. AT&T plans to raise that figure to between 60% and 70% in the coming years, he said. Austin said he’s found that open source models are “just as good or better” than older models sold by the likes of Anthropic and OpenAI. For instance, AT&T’s software developers still rely on cutting-edge models for complex tasks like generating code, but can use cheaper open source models for less intense tasks like generating summaries of previously submitted code, he said. “We expect that to just keep getting better going forward.” AT&T also has gotten savings from using so-called model routers that automatically route employees’ queries to cheaper models based on the complexity of the task. After the company began using router provider LiteLLM, its AI coding costs fell 56% while the quality of the AI’s performance fell just 2%, Austin said. AT&T started developing its primary internal AI system, Ask AT&T, in 2023 which initially relied entirely on OpenAI models the company purchased through Microsoft’s cloud, Austin said. The company has since expanded the number of models that power the service, which employees use for a range of tasks: software developers generate and edit code, employees ask about company’s HR policies, salespeople take contemporaneous notes or summarize client calls, and customer support staff look up information while speaking to customers. Software developers using Ask AT&T can choose a specific model or AI agent such as GitHub Copilot, Devin, Claude Code or Codex, but their spending is capped, Austin said. The Ask AT&T service now processes 45 billion tokens—or small pieces of text—per day, Austin said. While Austin didn’t specify how much the company spends to run it, using closed-source models from OpenAI, Anthropic and others to process that many tokens would cost at least hundreds of millions annually, based on those companies’ publicly listed prices. Besides Nemotron, AT&T uses other open-source or open-weight models such as Meta’s Llama and Google’s Gemma, Austin said. The company runs some of these workloads using Nvidia and AMD servers chips in its own data centers, which is often cheaper than renting servers to run such models from cloud providers, he said. AT&T also is analyzing Chinese open source models from DeepSeek and Moonshot but isn’t currently using them for work. The company is still evaluating potential risks from using them, he said. Austin said that in AT&T’s experience, open source models have typically been around six to 10 months behind frontier models in terms of capabilities, but the gap seems to be narrowing.
| 
Gift Subscription
