• Understanding how HLS works
• Understanding the problems with HLS
• Solutions for HLS problems

When responding to an HLS request, the streaming server determines which video quality (i.e., ts file) the client will attempt to play before switching to a lower or higher quality video. This switch depends on available bandwidth and device type.

Switching to the wrong quality first degrades the user experience considerably. For instance, sending a high quality file to a low-end device may cause the video/device to freeze, even on a good connection. And sending a low quality file to a high-end device with a good connection may cause the viewer to experience a low quality viewing experience for a prolonged period.

It may seem that sending a medium quality file first is a good solution, but it’s actually quite lazy. Instead, you can solve for the best solution in every case by using server-less scripting.

Server-less scripting, also known as function-as-a-service (FaaS), allows you to optimize responses on a per-device, per-request basis without touching your origin server’s configuration or code. There’s also no loss of cache-ability, and you can decrease latency further by making the decision at the edge instead of in a far-off data center,

Understanding how HLS works

For developers who are not already aware, HLS works by sending the client an index, also known as a manifest. This index contains a list of ts files called chunks that make up the video. The client requests the appropriate chunk based on play time.

In an adaptive bit-rate implementation, the manifest provides links to several alternative playlists called variants instead of listing chunks directly. All variants have an identical number of chunks and video content, but they differ in bit-rate and resolution (i.e., quality).

According to the HLS spec, clients should request the top listed quality in the manifest, play it, and calculate the available bandwidth (i.e., chunk size and download time). Based on the calculation, the client switches to a higher or lower quality until the bitrate of the video is lower than the maximum available bandwidth. This ensures seamless play.

(The image above is a sample HLS index listing three variants, each with a different bandwidth, resolution, and index file.)

To summarize, choosing the video quality based on the available bandwidth is the responsibility of the client, but choosing the default quality to start with is the responsibility of the server. This default quality is usually either the highest one, or simply the first one uploaded to the server. In all cases, it is the same regardless of which device requests the video and where the video is requested from.

Understanding the problems with HLS

We’ve established that the default video quality isn’t often the best to include in the initial response with HLS, but let’s look at some different scenarios to further understand why.

Scenario #1: High-End Device | High Quality Video | Low Bandwidth | Any Screen Size

In this scenario, the device downloads the master playlist (playlist.m3u8) and starts with the best quality as dictated by the server. The device downloads the first chunk, and it takes a whole minute (62,352ms). During this time, the viewer is waiting, not watching anything. This is a terrible video streaming experience.

Scenario #2: High-End Device | Medium Quality Video | Medium Bandwidth | Small Screen

In this scenario, the device spends seven seconds downloading a 10-second second chunk. After this, it starts to play.

So we’ve fixed the problem in Scenario #1, but we still waited seven seconds for the video to start. But we’re playing in a 800x450 pixel player and medium quality has a resolution of 1280x720 with a bit-rate to match. Therefore, low quality is beyond enough here and waiting seven seconds for a higher quality file is unnecessary. If we started with a screen-appropriate quality, we’d risk less delay before optimization catches up.

Scenario #3: High-End Device | Medium Quality Video | High Bandwidth | Big Screen

In this scenario, the device downloads the initial ts file quickly and, upon downloading, plays 10 seconds of medium quality video before switching to high quality.

So we’ve eliminated the issue in Scenario #2, but we still had to tolerate 10 seconds of lower quality video before optimizing up. And the next time we load another video (even on the same site), we’re going to go through all of this again. It would be better if we could remember the current quality and start with it.

Note: There’s an edge case scenario containing a low-end device and high-quality start, but there’s not much to expand on. As you might expect, the video starts to stutter and lag.

Solutions for HLS problems

Client-side solutions

Naturally, all these scenarios are much better solved for on the client side. For example, VideoJS, one of the most popular libraries for HLS playback on the web, will immediately start with the highest quality whose resolution fits the current player size, eliminating one of the issues above.