Can Muse make us forget the metaverse?

This is a column about AI. My fiancé works at Anthropic. See my full ethics disclosure here.

The first time I attended a Meta developer conference devoted primarily to bots was a decade ago. In 2016, the company then known as Facebook released a bot platform for Facebook Messenger. Rudimentary machine-learning models let developers from 1-800-FLOWERS, CNN, and other companies build automated systems to interact with customers, helping them to make purchases or get headlines delivered to them directly.

At the time, Silicon Valley had been consumed by an interest in bots, which its boosters hoped would shift more and more internet activity to messaging apps. "Messaging is going to be the interface — or the anti-interface — of the next phase of the internet," the entrepreneur Robin Chan, who then ran a bot-based e-commerce app named Operator, told me at the time. Meta, which owned most of the biggest messaging platforms of the moment, believed it might be in the best position to capitalize.

Within a couple years, though, the opportunity had proven to be a mirage. The bot-building apps I profiled in The Verge all shut down, and by 2018 Meta was winding down the bot platform. David Marcus, who then ran Facebook Messenger, left to pursue the company's next big idea: creating its own cryptocurrency. (That one never came to fruition, either.)

I thought about all this on Wednesday during the long ride back from Menlo Park, where I had just watched Mark Zuckerberg and his top lieutenants lay out their developer pitch for the year. The future of the company, Zuckerberg said, lay in Muse, the personal agent that is not even a month old.

"The centerpiece of our vision for what we’re building is Muse,” Zuckerberg said, promoting the app to a central place in the company's family of apps only a few days after most people first heard of it. “In the coming years, I expect that Muse is going to grow into the personal superintelligence that billions of people around the world are going to use to accomplish their goals and improve their lives.”

Muse represents a more consumer-friendly version of OpenClaw, the open-source AI agent I used for a few confusing days earlier this year. If you're willing to connect it to your Google Workspace, bank accounts, and other online services, you can ask it to perform a variety of tasks for you for free: filling out forms, grabbing reservations, paying bills, and so on.

Soon, Zuckerberg said on stage, you will be able to chat with Muse using your voice. It will be integrated into the operating system for the company's virtual reality products, and the company is also racing to put out a novel piece of hardware: the Muse Charm, a sort of strapless smartwatch with a screen that lets you see and interact with your animated agent. The Charm is scheduled to ship later this year; pricing information has yet to be released, though Bloomberg reported, citing people familiar with the plans, that it would be priced like a smartwatch.

All of this strikes me as a rather extraordinary level of investment in a brand-new product that has only barely begun to show signs of product-market fit. Zuckerberg said on stage that "millions" had tried Muse, only a day after The Information reported week-one usage at more than 500,000 people. By contrast, the company's Threads app hit 100 million users in five days, and Meta has yet to come up with a hardware plan for that one.

So while Meta's sudden focus on Muse no doubt reflects real enthusiasm about early feedback from testers — some of which has bordered on ecstatic — I suspect it's just as much about conveying a sense of momentum during a year when Meta plans to spend up to $145 billion on infrastructure and other capital expenditures.

Investors have typically been patient with Meta; its core advertising business is so strong that shareholders hardly blinked as the company has lost more than $85 billion on its Reality Labs division since 2021. But this year's capital spending alone is set to exceed those cumulative losses, and the company finds itself in need of a story to sell.

On Tuesday, I questioned some of the logic behind that story. Muse can no doubt do some impressive work, just as OpenClaw could. Personally, I find it tiresome to manage agents over a long period. They constantly need new tasks to do, need you to approve various actions on their behalf, and need you to review their work. They feel much more like interns than they do coworkers, and this gives Muse the feel of enterprise software: something I can easily imagine the managerial class trying out, but have more trouble seeing as a fast-growing consumer product.

During his keynote, Zuckerberg acknowledged that he originally thought virtual reality and the metaverse would become mass-market phenomena before AI did. But once AI did, the company changed course. AI, he said, "will be much bigger and more important. So we have shifted our focus to building the best devices for personal superintelligence."

This is almost certainly the right thing to do, and yet I noted how gingerly Zuckerberg sidestepped what had been an expensive miscalculation. It wasn't long ago that Zuckerberg had deployed the metaverse in developer-conference keynotes the way he did with Muse this week: as a bright spot to rally the company around amid never-ending litigation and social backlash related to the company's core products, which went all but unmentioned on Wednesday.

It wasn't long ago that Zuckerberg told us that the metaverse would be where billions of people were spending their time. And so, when he says that — never mind — it's actually Muse where everyone will be spending their time, there's more than a little reason to doubt him.

Intellectual honesty department: Meta VR Glasses

Whatever skepticism I feel about Muse, I must acknowledge being genuinely impressed with Meta's other big announcement on Wednesday. The Meta VR Glasses, which effectively shrink the Apple Vision Pro into a device that is one-sixth the weight and roughly one-third the price, appear to be a landmark development in the history of virtual reality.

Every year since I became a tech reporter, I have trudged to a handful of demos from Apple, Meta, Snap, and other companies working to make VR a mass-market phenomenon. And every year, no matter how much the device I tried improved on its predecessors, it always came with some tradeoff that destined it for quick storage in the family garage. It was too big and bulky; it overheated; it ran out of battery; the display was terrible; it cost too much money; and what could you do on it, anyway?

With the VR Glasses, the engineers at Meta have managed those tradeoffs better than any other company has to date. In a roughly hour-long demo, I found the glasses — which weigh just 100 grams — perfectly comfortable on my face. Thanks to the attached puck, where most of the computing takes place, the glasses never overheated. (The puck can also be plugged in, extending the promised three-hour battery life indefinitely.)

I was shocked to learn afterward that what the company is calling a 5K display has a little under half the pixels of the Vision Pro, since during my demo I would have called the two displays roughly on par. The images were crisp and immersive as I watched segments of a Billie Eilish concert and highlights of an NBA Finals game, and I played quick rounds of Beat Saber and Forza Horizon 6.

One of my favorite touches is what Meta is calling the "surface keyboard," which paints a full keyboard on the surface in front of you when you place your hands on it with your palms down. I found it reasonably responsive as I typed out a few sample Instagram comments, and while I probably would not want to write a whole column with it, I could certainly imagine banging out a few emails on the plane. (Though at that point I'd probably just toss a Bluetooth keyboard into my bag and use that.)

Of course, all the old questions about VR still apply. Is this a gaming device, a productivity device, something to watch movies on, or what? Meta's answer is that it's all three, but that's been Meta's answer since the earliest days of the Oculus Quest, and that particular combination was not enough to bring the metaverse to life. Whether you can make the use cases more compelling simply by improving the hardware remains an open question.

Still, as feats of engineering go, it's really something. Particularly after the disastrous showcase last week of Snap's $2,200 Specs, which aim for similar functionality but arrived with a design that has all the charm of a scuba mask.

It still seems entirely possible that, as with Muse, VR Glasses will ultimately be remembered as a novelty that didn't have much staying power. And we should reserve final judgment until we can test them outside demo conditions.

Still, I think anyone with a remote interest in VR will want to at the very least try them on. It turns out you can't lose $85 billion on something without learning a thing or two.

On the Instagram Live this week: If you've been following the handsome countdown clock at machinegods.fm, you know there are still 24 days until launch week. But to ensure we don't get rusty during our dark period, Kevin and I will be going live on Instagram on Friday at noon PT to discuss the following topics:

• The news story we would be talking about if we still had a podcast
• What really happened on the shoot for our trailer
• Listener questions

If any of that sounds fun, follow me and Kevin on Instagram and the Meta Corporation will helpfully notify when we are live. Or just have your Muse watch and fill you in later!

Also it's a great time to add the show in your podcast player of choice:

YouTube | Apple | Spotify | RSS

Following

An OpenAI agent hacked an Australian Medicare website

What happened:  Australia’s prime minister shared that an OpenAI agent gained unauthorized access to a public-facing Medicare portal, where it accessed public and private files and wrote files to an internal server. 

“This situation is obviously unacceptable,” Prime Minister Anthony Albanese told reporters. “And today, I spoke with the CEO of OpenAI, Sam Altman, to express Australia’s extreme concern about this incident.”

The Medicare website breach happened on June 18, but the Australian government was only informed three months later, in a September 10 email. During his call with Altman, Albanese said he “expressed my disappointment that it took the company way too long to inform the government what had occurred.”

Australia is currently conducting a forensic investigation into the incident.

AI research firm Transluce published an investigation suggesting that the agent executed the hack during a mundane Internet research evaluation. The agent seemingly performed the hack in order to find average costs for skin treatments in the Australian healthcare system.

“This data reveals that malicious cyber activity is not limited to agents tasked with cybersecurity-related tasks” — like those that prompted the Hugging Face incident — “and can arise instrumentally to solve mundane tasks like information retrieval,” Transluce observed.

The investigation found that agents also targeted the University of New Mexico’s digital library and Data USA, an API that provides visualizations of public US government data. Transluce observed agent activity as early as March 6.

Australia’s Deputy Prime Minister Richard Marles said the Australia incident was “relatively minor” because no private information was accessed.

Still, Marles said, although “the impact of this event is minor,” it is “a very serious incident because, in an unintended way, an AI agent has entered into an Australian government website in a way which is unauthorised.”

Why we’re following: AI agents have been used to hack governments before, as when a hacker used Claude to retrieve sensitive tax information from the Mexican government early this year. But, Transluce wrote, “this appears to be the first reported instance of an agent autonomously choosing to attempt to compromise a government website;” that is, the first time an AI chose to do it on its own.

These incidents don’t give additional evidence of exceptional hacking ability; the agent wasn’t hacking into Australia’s most sensitive records, and the other attacks recorded by Transluce seem to have failed.

But it’s really striking that OpenAI’s agents seemingly folded under zero pressure — resorting to hacking universities and governments in evaluations that didn’t prompt them to do so at all.

And as Albanese pointed out, OpenAI should report cyber incidents to affected parties far, far more promptly.

What people are saying: Former OpenAI board member Helen Toner wrote on X, “On the one hand it's *almost* reassuring that all these new incidents are from the same May-July period when OpenAI's training setup was clearly fucked... meaning maybe they've now fixed it.”

But, she added, “what might be happening now — at OpenAI or elsewhere—that we're not going to learn about until December..??”

—Ella Markianos

Those good posts

For more good posts every day, follow Casey’s Instagram stories.

(Link)

(Link)

(Link)

Talk to us

Send us tips, comments, questions, and posts: casey@platformer.news. Read