Video Playback on Mobile Devices

I see 2 common approaches to dealing with video on mobile browsers:

1. Hide the video with CSS “display:none”.  (This results in a partial video download that the user can literally NEVER see).
2. Serve the same video to both mobile and desktop. This video is typically 720p or 1080p, and designed to look great on large screens.

Scenario 1 is one that should fade away: forcing a download but hiding it from users is a poor experience all around… and users are coming to expect video on all platforms.

But what about Scenario 2?  Is it really that bad?  A few months back, I reported on video playback metrics on the mobile web. I found that over 70% of videos served on mobile have a width of over 1080 pixels. Even if the network can support the download, 91% of smartphone usage is in portrait mode – forcing phones to resize the video to fit the screen.  Serving the same video to desktop and mobile viewers will impact playback on mobile devices.

At a Meetup in Cologne earlier this month, Andreas asked if there is a device impact to video playback.  I did not have a good answer, and decided to investigate.

Videos on Mobile

If the video bitrate (measured in MBPS) is larger than the speed of the network connection, we will have playback issues: the network is limiting the playback.  To determine if there is also a device bottleneck, we need to control for the network speed before we can determine if there are device specific issues.

Effect of Network Speed on Video Playback

The website of my alma mater, The College of Wooster, uses a background video of campus (and it appears on all screen sizes).  Since their web design is to show the video to all users, we can use it to test how it plays on various devices.

This video is 1280×720, 45s long, 19 MB, and has a bitrate of 3.5 MBPS.  The default WebPageTest 3G connection has a downlink speed of  1.5 MBPS.  Since the video playback requires over 2x the available bandwidth, we can be pretty sure this video will not play back well on 3G.

To test this on different devices, I hosted the video myself, and then used StreamOrNot (I have blogged about SON here, and docs are on GitHub) and WebPageTest to automate my testing.

There are 6 Android devices on WebPageTest that I used for testing video playback.  Testing the Wooster video on a 3G connection, I found that all of the devices had significant stalling issues:

All of the devices had issues playing back this video, but the Alcatel 1x was significantly worse than the others.  Clearly, this website is not optimised for viewing on a 3G connection.

Does Faster Network Connections Improve Video Playback?

We knew that a video that plays back at 3.5 MBPS would not be able to stream well on a 1.5 MBPS connection.  So, let’s speed up the network to Cable (5 MBPS), LTE (14 MBPS) and Native (no throttling):

For 4 of the 6 devices, the number of stalls reduce to small numbers very quickly, indicating that it was only the network speed that was limited the video playback.  from this, we can conclude that for mid-high range phones, the network connection is the limiting factor in video playback.  If you add video to your website, you can choose how many users (on high powered devices) you exclude from interacting with your website by changing the bitrate of the video.

However, the lower powered Alcatel 1X and Moto E still have a significant number of stalls, no matter what network speed is available.

Stall Time by Device & Network Speed

This is interesting. The Alcatel device stalls more often than the Moto E, but the time in stalls is 2-3x less than that of the Moto E.  I don’t have data on which is worse: lots of very short stalls, of a few longer stalls, but I imagine both lead to video abandonment.  Large videos will stall on low powered devices no matter the network speed.

Either way, it is pretty clear that playing a 1280×720 video requires more processing power than is available on the Moto E or the Alcatel 1X.  Shown below is the Main thread chart when run on a native connection (no chance for a network issue):

The blue lines in the Main Thread are Layout calls, and it is apparent that there are more layout calls on the Moto E than the other devices.  In fact, the Moto E spends 2x the time in layout and painting than the Alcatel, and 6x the time of the Moto G.

 

Does Video Compression Help?

To ensure a positive experience on mobile, many sites just remove video from the mobile screens.  But video compression will reduce the size of the video, and perhaps we can still serve the same video to mobile devices.  There are a lot of great tools to compress videos and measure the resulting quality of the compressed image (more details in an upcoming post).

Let’s take the test video and apply the “q_auto” compression at Cloudinary.  The resulting video is 9.6 MB, and streams at 1.7 MBPS (50% smaller), but still has a high VMAF quality score.

This smaller video reduces the number of stalls (faster devices like the Moto G4 nearly stop stalling even on 3G!). However, even with the compressed video, the Alcatel iX and Moto E devices still have playback issues:

This answers that the question from the Meetup: There is a device component to video playback that we must take into account.  Even strong video compression cannot resolve some of these issues.

 

Changing the Video Dimensions

The Moto G4 screen is 1080 pixels wide, Alcatel 1X screen is 720 pixels wide, and the Moto E is 540 pixels wide, and we are sending in a 1280 pixel wide video.  So, assuming portrait mode, the phone has to resize the video to make it fit on the screen.  The Alcatel 1X removes 68% of the pixels during playback and the Moto E removes 82% of the pixels. It is clear that these devices are unable to process the video fast enough for playback, leading to playback stalls.

Reducing Video Dimensions: Alcatel 1X

Let’s reduce the video to 960 and 540 pixels wide.  When we test these videos on the device, we find the number of stalls drops to below 5, and the time in stalls is below 100ms for a 45s long video, a vast improvement over the 1280 wide video.

Reducing Dimensions: Moto E

The Moto E has a much smaller screen, we still see some stalls with the 960 px wide video, but very smooth playback at 540px.

 

As I reduce the file dimensions, the video playback improves with fewer stalls and near zero time spent in stalls.  This makes sense – it is less work for the device to get the movie on the screen. To ensure good video playback on smaller devices, serve smaller videos to smaller screens.

Solutions

As video becomes more and more pervasive, the current trend of “leave out video on mobile” will no longer suffice.  But, to ensure that we are delivering a quality experience to our mobile customers, it is imperative to examine the bitrate of the videos we are serving.  The bitrate of the video becomes the cutoff for the network speeds our site will playback smoothly on.

Small screen devices cannot handle the 720p and 1080p we are serving the desktop.

Solution 1: Create smaller versions of your videos, and use Javascript to detect the source for smaller screens to ensure a fast and smooth playback on these devices.

Solution 2: Use streaming video.  HLS video streams will deliver an appropriately sized video to the device – abstracting the need to create different videos for different screens. It will also negotiate the network speed, and adapt the video bitrate for the speed of the network you are using.

 

Thanks to Andreas Marschke for the question that inspired this post, Colin Bendell for the discussion and Patrick Meenan for building (and helping me with) WebPageTest.