Overview of Stadia’s TV app architecture
Since we launched Stadia for Android TV in June, 2021, we’ve had lots of interest in learning more about how we built this experience. Visually, folks have noticed similarities between the Stadia experience on Android TV and on Chromecast Ultra.
In this post, we will cover the history of Stadia on TVs, detail some of our implementation decisions, and share a summary of our app architecture for Android TV.
Loading animation on Stadia for Android TV
First: a bit of Stadia history
In order to learn about our Stadia for Android TV architecture, you first need a bit of history on our implementation of Stadia for Chromecast Ultra.
A custom Web Receiver usually involves three things:
- Usage of the Web Receiver API inside of that web app
- Configuration with Google to specify how and when that web app is launched
Our TV app on a Chromecast Ultra. Modified image from our developer docs
Chromecast Ultra devices, like many living room devices, are a form of small, resource-constrained hardware. Generally, Web Receivers are fairly minimalist in terms of UI and functionality, often taking the form of players (e.g., for audio/video) or mirrors (e.g., for Chrome tabs).
As you might expect, gameplay is paramount for Stadia. With limited RAM, compute, and thermals at our disposal, we needed a UI solution that was lightweight yet met the needs of a demanding video game streaming service. It could also be more full-featured than other Web Receivers, due to the Stadia controller being able to function as a remote. This UI solution also needed to work well with the video game streaming client on Chromecast Ultra.
This additionally has some interesting properties, given that we also use Dart (via Flutter) for our native Android and iOS applications. For example, we have been able to share a meaningful amount of Dart code between our TV app and mobile apps, such as parts of Stadia controller connectivity, localization support, and gameplay state management.
As you might expect, this experience was a focus of many optimization efforts in order to achieve the user experience required. This involved a large, collaborative effort across Stadia UI and Playability teams, Google hardware teams, and more.
So we had a web app and OS support for streaming. How did that become an Android TV app?
Back to Android TV
To give a meaningful answer, we have to take a quick digression over to a very cool piece of technology: Cobalt.
Cobalt is a “high-performance, small-footprint platform that implements a subset of HTML5/CSS/JS to run applications, including the YouTube TV app”. In other words, it’s a runtime that enables web apps to run on resource-constrained devices. It was originally developed by the YouTube team, with the main goal of successfully addressing the very wide range of devices in the living room ecosystem.
Cobalt also has a unique property in that it does not directly build against the underlying living room platform. Rather, it uses something called Starboard, which is Cobalt’s porting layer and OS abstraction. Android is one of the platforms that Cobalt supports out of the box, in order to enable developers of web apps to quickly get started.
So we have a way to run a web app on Android TV devices via Cobalt. What about video game streaming?
The Stadia Playability team, which focuses on this core streaming technology, worked to take the streaming stack already in use in our Android mobile app and support the Cobalt model. This meant that, similar to our existing platforms, we could blend high performance video game streaming technology with high development velocity client code.
When everything is combined, we have an architecture that looks something like this:
A simplified view of our architecture on Android TV
There was a lot of work along the way! To make this model work well, there were many changes at every layer of this stack.
In the Stadia streaming client, we worked to maximize the performance and test the quality of video game streaming on many Android TV devices, including inbound video decoding and input latency.
In Cobalt, we added features to support Stadia use cases and extended its functionality via plugins. This enables us to tie in the Stadia streaming client, as well as Android TV platform features, such as gamepad support. These changes also benefit other applications using Cobalt.
In the TV app, we optimized the code to support Cobalt’s feature set, connected flows with the plugins mentioned above, and ensured that none of these changes impacted our production Chromecast Ultra experience.
In addition to changes to each layer, we had to overcome many challenges along the way. Some of these included:
- Game-driven events, such as in-game purchases: these events cause something to occur in the user interface, as well as in the Android TV system, and in turn cause a response in Android-level dependencies, such as Google Play Services.
- Authentication: Our TV app now needs to support authentication via mobile-added credential and casted sessions, as well as users who are logged into their Android TV devices.
- Input: correctly directing input to the various parts of the experience (e.g.: is a given input meant for the user interface or the game session?), as well as testing the wide range of input devices available to users, is a significant undertaking.
One of the most interesting aspects of this model is that the changes to the Stadia TV web app were deployed as part of the existing process for Chromecast Ultra, which enabled us to validate our work in a production environment prior to launch on Android TV.
With Stadia for Android TV available on millions of devices, we’ve increased the types of the devices we need to support in the living room by quite a bit! This was a lot of collaborative work across many dozens of folks within Stadia and across Google, and it wouldn’t have been possible without everyone’s contributions.
Scale is a large driving force in many of these architectural decisions. In order to successfully scale to living room screens, we invest engineering effort in ways that maximize reuse. We don’t want to write–or re-write–applications for each new device we want to support. Additionally, the multiple points of shared infrastructure and code supports our ability to extend to the wide array of living room devices. By iterating on our TV app, enhancing Cobalt, and continuing to promote Dart code reuse across our endpoints, we’re able to plan better for the future.
This work is conducted in collaboration with many people across Stadia and Google teams.
-- Matt Joseph, Engineering Manager and Maksym Motornyy, Senior Staff Software Engineer