Roku starts using AI, explores crowd-sourcing for better picture quality

Update the iPad. Scan the aging computer for viruses and spyware. Turn off motion smoothing on the TV. Many people double as tech support for their relatives during the winter holidays, and optimizing TV settings has been one of those tasks for years. Now, Roku wants to do away with that.

The streaming device maker is introducing a new feature called Roku Smart Picture for smart TVs made in-house as well as by its partners this spring that automatically adjusts a TV’s picture setting based on the content that is being watched at any given moment.

Roku Smart Picture wants to bring better visual quality to the masses. “Our research has shown that more than 90 percent of the customers do not change picture quality modes at all,” said Roku’s director of picture quality Erwin Bellers during a CES press preview event. “If you're watching a movie, then it ought to be the best picture mode to set the TV (to) movie mode. If you're watching sports, it will be best to set the TV (to) sports mode – but really, no one is doing that.”

• Roku executives said that the company is using AI to automatically detect the type of content being watched, and then adjust the picture mode accordingly.

• They didn’t elaborate on how this is done, but a company spokesperson told me that Roku TVs use a combination of methods, “including artificial intelligence, machine learning, and data from content partners” to make these adjustments, suggesting that there may be both automated content recognition as well as computer vision at work.

Changing the picture mode may just be the first step. Bellers said that his team at Roku had “a lot of more exciting works on the horizon.” He didn’t elaborate further, but the company has filed some interesting patent applications that suggest the company is exploring the use of crowdsourcing to adjust a variety of audio-visual settings.

One of the downsides of using automated content recognition for image optimization on every single TV is that the visual settings are being adjusted in real time. “This limits how invasive the corrections can be without creating a jarring experience for the viewer,” one of Roku’s patent applications explains.

By using crowdsourcing, Roku could effectively create its own database of metadata for streamed content, and then adjust settings accordingly whenever a viewer begins watching a video included in the database.

This could include things as simple as whether a content asset looks best in movie or sports mode. However, the patent also mentions other examples that suggest the company is exploring other kinds of optimizations. “The crowdsource server(s) may determine that turning closed captioning on may enhance users' viewing experience at particular portions of the movie (for example, when the soundtrack of the movie is difficult to hear),” the application mentions as one of those examples.

Software is eating the TV world. As TV makers become more software-centric, we’re likely going to see a lot more of these kinds of smart image optimizations. As this happens, we’ll get the answers to two questions: Will streaming services, which often have their own strong opinions on how to best display their content, play ball? 

And will the results actually look better, or will TV makers be tempted to over-optimize, leading family tech support people to turn off these features, much like they have been doing with motion smoothing?

This article was first published as part of Lowpass, a weekly newsletter about AR, VR, streaming and more. Sign up now for free.