Welcome to Lowpass! This week: Amazon is adding LLM-powered search to Fire TV, and Sightful is launching the first AR laptop.

Ten years in, Fire TV is getting an AI upgrade 

Searching for content on Fire TVs is going to get a lot more nuanced: Amazon is launching a new LLM-powered search experience on Fire TV devices today, which allows consumers to ask more complex questions, and hopefully get better content recommendations in return.

Amazon first previewed the new search experience at its fall devices event last year, and is now bringing it to select Fire TV OS devices in the U.S., with plans to be live across the entire domestic Fire TV footprint in the coming weeks.

All of this happens ten years after Amazon released its very first Fire TV streamer. That’s why I caught up with Amazon’s Alexa & Fire TV VP Daniel Rausch this week to talk about the company’s bet on AI search, the push and pull between smart TV platforms and publishers, and the future of external streaming devices in a world where you basically can’t buy TVs without built-in apps anymore.

How Amazon built its LLM-powered TV search. When Amazon launched its very first streaming adapter in 2014, it shipped the device with a remote with integrated microphone, betting that consumers would prefer voice search over on-screen keyboards (or whatever this was).

• Now, Fire TV voice search is getting its biggest update to date, with Rausch telling me that it “has been rebuilt from the bottom up with large language models.”

• The new search experience is powered by an Amazon-built LLM, which has been trained on data from Amazon’s IMDb. “It gives you access to metadata that includes the entire cast, key lines, scenes, plot points, et cetera,” Rausch said.

• This allows consumers to ask for movies in which a human and a dog are best pals, movies with a certain dialog line, or, as Rausch put it, “the movie where Tom Hanks ends up talking to a volleyball for the whole [time].”

• To make AI-powered search work, Amazon purposefully kept the functionality narrow and focused on entertainment use cases, Rausch said. “It's not aiming to help you with your high school term paper.”

• One of the challenges now is for Amazon to actually get people to make use of these new search features. The past decade of voice search and ambient computing has resulted in people training themselves to use very specific, precisely-worded requests, tailored to be understood by assistants that turned out to be not all that smart.

• Now, the company has to come up with ways to educate Fire TV owners about letting go of some of those forced phrases, and, well, just talk. “Large language models handle utterances that weren't imaginable with prior state-of-the-art technology,” Rausch said. “It is important to make sure customers understand the full breadth of what's available.”

Why people still buy streaming dongles. Third-party device makers started shipping TVs powered by Fire TV in 2017, and Amazon began selling its own Fire TV models in 2021. Smart TVs have since become the company’s fastest-growing part of the Fire TV business, but Amazon keeps releasing new streaming sticks every year. So why won’t dongles die already?

• Without naming names, Rausch essentially blamed the company’s competition for making TVs that don’t live up to people’s expectations. “What continues to shock me, frankly, is the amount of times that a Fire TV stick is plugged into a so-called smart TV,” he said.

• Amazon knows this because Fire TV sticks support HDMI CEC, a communications protocol for HDMI that, among other things, allows devices to identify themselves to each other. It’s how your smart TV knows that you have an Xbox on HDMI 1 – and how Amazon knows who made your television. “We know the TV make and model a stick is plugged into,” Rausch confirmed.

• “Well above 80 percent of the time,” a Fire TV stick is plugged into a TV that comes with apps built-in, according to Rausch.

• “We call them so-called smart TVs for a reason,” Rausch said. “They're just not that smart.”

Why apps aren’t the future of TV. Fire TV was the first major smart TV platform to introduce what’s known in the industry as a content-forward UI, betting a decade ago that people would prefer direct links to movies and shows to long lists of app icons. That paradigm has been adopted by almost every other manufacturer since, validating Amazon’s approach.

• “We continue to believe that customers care about TV shows and movies, not apps,” Rausch said. “The best way to find things is a user interface that gets all of the content out of those applications and in front of you, instead of gating things behind experiences you have to go in and out of.”

• Such an app-centric approach makes sense on mobile, where apps have very distinct features and value propositions, Rausch argued, adding:  “It's a terrible living room experience.”

• The problem with content-forward UIs is that some publishers don’t really like the idea of presenting all of their shows next to those distributed by competitors. If it was up to the streaming services of the world, then you’d spend all of your time in their apps, and never leave.

• Rausch downplayed that tension when I asked him about it, telling me that publishers ultimately knew that content was king. 

• However, he also had some advice for publishers: “It's not a stable long-term strategy to assume that customers will come into your app and stay there,” Rausch told me. 

• As an example, he referred to a trend that’s become popular among TV app developers, who increasingly hijack the back button of TV remotes to make their app more sticky. “To say: Okay, I will make it so [that] you have to confirm the back button again if you want to exit the app – that's not a long-term strategy.”

The first AR laptop is here, but it’s not a spatial media machine

After testing the waters with an early access product last year, Israel-based spatial hardware startup Sightful announced the first full-featured version of its AR laptop Thursday. Dubbed Spacetop G1, the laptop ships with a pair of tethered AR glasses instead of a screen, which offer access to a spatial desktop that the company describes as a “100-inch virtual workspace.”

I got a brief demo of the new Spacetop last week, and walked away with the impression that it is an interesting laptop with some obvious flaws, and a fairly pedestrian use case. Basically, Spacetop is all about web-based productivity. Think doing your email, writing documents, browsing the web, and the likes.

That’s very much by design, according to Sightful’s co-founders Tamir Berlner and Tomer Kahan, who previously worked at Magic Leap together. While Magic Leap, at least in its early days, wanted to be the AR solution for everything ranging from gaming to enterprise work, Spacetop is all about productivity. “With Magic Leap, one of the big questions was: What would people do with it,” said Berlner. “Why would anyone choose to put the glasses on every day?”

Sighful’s focus on productivity means that the company is approaching spatial computing fairly conservatively. The laptop integrates Xreal’s new Air 2 Ultra glasses, which support 6DOF tracking, thanks to two integrated cameras. 

However, the Spacetop doesn’t ship with any 3D apps, and Sightful primarily is looking at 3D as a way to spatialize conference calls at some point in the future. Similarly, the laptop’s spatial desktop can be placed further away or closer to the user, but everything is still locked to a single plane, with no ability to arrange individual windows spatially.

Spacetop is based on Android AOSP, but doesn’t offer access to any app store out of the box. “We chose a web-first approach,” Berlner said. He added that the company would welcome XR app developers to the platform, but a spokesperson told me that Sighful won’t support WebXR or 3D video viewing at launch.

All of this means that Spacetop is basically a kind of Chromebook for AR. If that’s all you need, then the device may offer some advantages over Apple’s Vision Pro, including a better travel mode, and a lower price tag (the Spacetop G1 will cost $1900 when it becomes available later this year). But if you’re at all interested in developing or even just consuming spatial entertainment, passthrough headsets like the Vision Pro (or the Meta Quest 3, for that matter), are still a better bet.

What else

YouTube opens up Playables games to everyone. The casual games, which include Cut the Rope and Angry Birds Showdown, can be played on desktop and mobile.

AI can now help write your kids’ bedtime stories. My latest for Fast Company is a story about Naria, an AI children’s book creation service made by former Vevo employees.

Fubo readies free tier. The internet-based pay TV service wants to keep people who cancel hooked.

Sonos CEO Patrick Spence talks headphones, app. Spence explains why the company’s new headphones don’t work on Wifi, and why Sonos launched its controversial new app.

Google-owned Owlchemy Labs brings VR games to Vision Pro. Job Simulator and Vacation Simulator are launching on Apple’s headset with hand-tracking support; Owlchemy’s CEO previosuly told me that hand tracking is the future of VR gaming.

Marvel’s What If…? Vision Pro app is here, and The Verge’s Wes Davis is not impressed.

Apple may be working on an Android app for Apple TV. That’s unless the Android developer the company is looking for in an ad spotted by Bloomberg’s Mark Gurman is simply meant to work on a revamped Android TV app.

Meta may have teased its new VR UI in two ads. Surprise! The company is taking some cues from Apple.

That’s it

If there’s one thing you need to watch this week, it’s the new season 3 trailer of “The Bear.” And if there’s one thing you need to rewatch, it’s season one and two of … okay, I’ll stop. But seriously, such a good show!

Thanks for reading, have a great weekend!