Hi there! My name is Janko Roettgers, and this is Lowpass. This week: Why phones may be better for AR than headsets, and how Meta makes video work globally.

When Mark Drummond was working on the Vision Pro at Apple, he had a bit of an epiphany that didn’t really fit Cupertino’s preferred narrative.

Drummond was managing the Character Intelligence Team, which among other things built the Encounter Dinosaurs demo. Preinstalled on the Vision Pro, the demo puts viewers eye to eye with interactive prehistoric creatures. 

“We built that with Jon Favreau,” Drummond says, referring to Apple’s long-standing partnership with the Mandalorian director. “Before the headset was available to us to take down to Burbank, we used iPhones and iPads,” he recalls. Relying on mobile devices for demos made sense. VisionOS is essentially an iPadOS fork. “It did actually work out. We had a really good experience with iPhone and iPad,” he says.

Through that experience, Drummond realized that these mobile devices weren’t just decent stand-ins for the headset. “What we learned looking for sources of surprise and delight with interactive characters in mixed reality is that the headset is actually not the best [device] for this kind of thing,” he says.

“I still think it's a pretty fabulous piece of hardware,” Drummond says. However, headsets can also be alienating, and separate viewers from the world and from the people around them. “It's kind of lonely,” Drummond says. Having an AR app on a phone, on the other hand, makes it much easier to show it to others. “People can lean in over your shoulder,” Drummond says.

That’s why, shortly after leaving Apple in 2023, Drummond embraced mobile devices for augmented reality storytelling. For the past two and a half years, he has been working on a new iPhone AR app called Pixi that’s all about mobile-first interactive storytelling.