Taking Chrome Experiments to the TV

March 27, 2014


Link copied to clipboard
By Igor Clark, Google Creative Lab

With the release of the Google Cast SDK, making interactive experiences for the TV is now as easy as making interactive stuff for the web.

Google Creative Lab and Hook Studios took the SDK for a spin to make Photowall for Chromecast: a new Chrome Experiment that lets people collaborate with images on the TV.

Anyone with a Chromecast can set up a Photowall on their TV and have friends start adding photos to it from their phones and tablets in real time.

So how does it work?

The wall-hosting apps communicate with the Chromecast using the Google Cast SDK’s sender and receiver APIs. A simple call to the requestSession method using the Chrome API or launchApplication on the iOS/Android APIs is all it takes to get started. From there, communication with the Chromecast is helped along using the Receiver API’s getCastMessageBus method and a sendMessage call from the Chrome, iOS or Android APIs.

Using the Google Cast SDK makes it easy to launch a session on a Chromecast device. While a host is creating their new Photowall, they simply select which Chromecast they would like to use for displaying the photos. After a few simple steps, a unique five-digit code is generated that allows guests to connect to the wall from their mobile devices.

The Chromecast device then loads the Photowall application and begins waiting for setup to complete. Once ready, the Chromecast displays the newly-generated wall code and waits for photos to start rolling in. If at any point the Chromecast loses power or internet connection, the device can be relaunched with an existing Photowall right from the administration dashboard.

Tying it all together: The mesh

A mesh network connects the Photowall’s host, the photo-sharing guests, and the Chromecast. The devices communicate with each other via websockets managed by a Google Compute Engine-powered node.js server application. A Google App Engine app coordinates wall creation, authentication and photo storage on the server side, using the App Engine Datastore.

After a unique code has been generated during the Photowall creation process, the App Engine app looks for a Compute Engine instance to use for websocket communication. The instance is then told to route websocket traffic flagged with the new wall’s unique code to all devices that are members of the Photowall with that code.

The instance’s address and the wall code are returned to the AppEngine app. When a guest enters the wall code into the photo-sharing app on their browser, the AppEngine app returns the address of the Compute Engine websocket server associated with that code. The app then connects to that server and joins the appropriate websocket/mesh network, allowing for two-way communication between the host and guests.

Why is this necessary? If a guest uploads a photo that the host decides to delete for whatever reason, the guest needs to be notified immediately so that they don’t try to take further action on it themselves.

A workaround for websockets

Using websockets this way proved to be challenging on iOS devices. When a device is locked or goes to sleep the websocket connection should be terminated. However, in iOS it seems that the Javascript execution can be halted before the websocket close event is fired. This means that we are unaware of the disconnection, and when the phone is unlocked again we are left unaware that the connection has been dropped.

To get around this inconsistent websocket disconnection issue, we implemented a check approximately every 5 seconds to examine the ready state of the socket. If it has disconnected we reconnect and continue monitoring. Messages are buffered in the event of a disconnection and sent in order when a connection is reestablished.

Custom photo editing

The heart of the Photowall mobile web application is photo uploading. We created a custom photo editing experience for guests wanting to add their photos to a Photowall. They can upload photos directly from their device’s camera or choose one directly from its gallery. Then comes the fun stuff: cropping, doodling and captioning.

Photowall for Chromecast has been a fun opportunity to throw out everything we know about what a photo slideshow could be. And it’s just one example of what the Chromecast is capable of beyond content streaming. We barely scratched the surface of what the Google Cast SDK can do. We’re excited to see what’s next for Chromecast apps, and to build another.

For more on what’s under the hood of Photowall for Chromecast, you can tune in to our Google Developers Live event for an in-depth discussion on Thursday, April 3rd, 2014 at 2pm PDT.

Igor Clark is Creative Tech Lead at Google Creative Lab. The Creative Lab is a small team of makers and thinkers whose mission is to remind the world what it is they love about Google.

Posted by Louis Gray, Googler