Welcome to Runtime! Today on Product Saturday: ST Microelectronics and AWS collaborated on a new data center chip, VAST Data's core product now supports block storage and Kafka streaming, and the quote of the week.

(Was this email forwarded to you? Sign up here to get Runtime each week.)

Ship it

Photo finish: AWS designed a lot of the hardware it uses across its cloud infrastructure, as well as several of its own chips. Now it is collaborating with STMicroelectronics on the development of a new networking chip that contains ST's "next generation of proprietary technologies for higher-performing optical interconnect in datacenters and AI clusters," it said in a press release.

The computing requirements of the AI frenzy are creating a bottleneck inside data centers, as GPUs crunch data faster than networking chips can route that data to its destination. ST's new chip will enter production later this year and it is the company's first chip that uses silicon photonics, in which light is used to transfer data rather than electrons.

On guard: The rush to build generative AI applications will inevitably create software bugs, which can be particularly scary when apps that connect to a ton of sensitive data are involved. Pangea has raised $51 million to tackle AI security, and this week its AI Guard and Prompt Guard services became generally available.

AI Guard "prevents sensitive data leakage and blocks malicious and unwanted content," and Prompt Guard "analyzes user and system prompts to block jailbreak attempts and organizational limit violations," the company said in a blog post. "You cannot really deliver enterprise quality AI capabilities without having these security guardrails," Pangea co-founder and CEO Oliver Friedrichs told Security Week.

Block and tackle: Companies that have found ways to deploy generative AI apps in production tend to have already done the hard work of modernizing their approach to data, as Monica Caldas of Liberty Mutual pointed out in the last episode of How We Built It. VAST Data added two important services to its data storage platform this week that could convince companies looking for an upgrade to pull the trigger.

The company's flagship product now supports block storage, which makes it far more useful for companies using VMware, Kubernetes, or container-based applications, and it also introduced Event Broker, a data-streaming service based on the popular open-source Kafka project. Event Broker "will prove valuable to enterprises with performance data needs, such as advanced analytics and front-end to AI pipelines," NAND Research's Steve McDowell told SiliconAngle.

A problem indeed: Enterprise software (and really the web in general) would not exist without APIs, which connect applications across the internet. But modern infrastructure design choices — as well as, of course, the generative AI boom — have led to a huge increase in the number of APIs that companies need to maintain in order to keep everything up and running.

Boomi introduced an API management service this week that "ensures that APIs — whether shadow, legacy, or managed — are cataloged, monitored, and controlled, reducing risks and improving efficiency," it said in a blog post. "Every major trend that has shaped digital transformation has triggered a scramble to keep up as technology stacks seem to take on a life of their own," Boomi's Ann Maya told Forbes.

Whoops: On Thursday Sakana AI, a startup that has raised $244 million from investors like Nvidia, announced that it had created "The AI CUDA Engineer, the first comprehensive agentic framework for fully automatic CUDA kernel discovery and optimization." The company claimed that its agentic engineer could convert PyTorch code into kernels designed to run on Nvidia's GPU software "with speedups of 10—100x over common PyTorch operations" according to AI benchmarks, which sounds great!

However, layperson AI engineers quickly realized something in Sakana's code was totally wrong, and using the agentic engineer actually "resulted in a 3x slowdown — not a speedup" when converting PyTorch code, according to TechCrunch. Sakana owned up to the mistake and said it would fix the code, but it's another reminder that AI benchmarks are just as unreliable as, well, every single benchmark that has ever existed.

Stat of the week

Cloud "optimizations" were the talk of the industry during 2023 and 2024, as companies faced up to the reality that IT budget growth was going to be flat — at best — during an era of rising interest rates and inflation. Cutting back on SaaS sprawl has been one of the main priorities, and new research from The FinOps Foundation found that 65% of "large cloud spenders" are planning to better manage their SaaS spending over the next year.

Quote of the week

"Having competed against Oracle and IBM in client-server, I knew that the buyers will not tolerate winner-take-all. Structurally, hyperscale will never be a winner-take-all because buyers are smart." — Microsoft CEO Satya Nadella, talking to Dwarkesh Patel about the evolution of the enterprise AI market but also making an interesting argument about B2B antitrust.

The Runtime roundup

Cisco confirmed that the hackers behind the Salt Typhoon attack, which compromised telecommunications companies around the world, exploited a flaw for which a patch was available to gain access to devices running its networking software.

LaunchDarkly acquired Houseware, a product analytics startup that had raised $2.1 million in funding, for an undisclosed amount.

Thanks for reading — see you Tuesday!