Why (and how) to lie with progress bars

Published on 16.06.2020 by Peter Kröner

© Unsplash, Jungwoo Hong

The progress indicator that you see when you run tests inside Warhol's browser extension is fake. It is somewhat connected to reality, but only just. Significant sections of the data the progress indicator is based on are guesstimates and outright fabrication, but all this deception is there for a reason: it still provides a better user experience than any of the alternatives that you might think of. This article explains why we lie to your users with fake progress bars and how Warhol's browser extension does it. Maybe you too will want to bend the truth at least a little next time you build a progress indicator!

Motivation and Requirements for progress bars

Computers are crappy and users (especially technical users) are used to giving up and trying again. The main purpose of a progress bar is to indicate to the user that something is still happening and just waiting a little longer is all that they need to do. And even if the user is not about to abort a running task, a progress bar makes things feel faster than they are, so a progress bar is always worth having.

But not all long-running computer things neatly lend themselves to proper, linear progress bars. A perfect progress indicator requires three things:

  1. A definite end state, known in advance. This could be the number of files to process or the estimated total time a process will take. The end state is when the progress bar is at 100%.
  2. Intermediate data from which the current state can be calculated. If there's 10 files to process and we just finished with number 2, we know that we are about 20% done.
  3. Smooth progress where processing file 3 will not take significantly more time than the first two files. Without smooth progress, updates to the progress bar may jump around unpredictably.

Obviously a huge number of use cases for process indicators do not fulfill all or any of these requirements:

  • Web requests can take a long time, but provide no intermediate data that you can use to fill up a progress bar. This usually leads to animations that indicate that something is happening, but with no way of knowing when and if this something is going to deliver a result. A web request has only a start time and an end time, but there is nothing in between and the total time is unpredictable.
  • Brute-forcing anything can take a very long time, but when a process starts, it is unknowable how long this time will be. There may be some intermediate data (maybe there is a search space that gets exhausted over time), but this intermediate data has no bearing on the time remaining and is thus mostly useless for progress bars.
  • Anything that does not happen in near-constant time has a problem as well. If something huge is being calculated in several sub-steps, but each step's running time varies wildly, a progress bar is possible, but does not reflect the remaining time - just the number of steps still to perform (which most users will not care about).
An Ajax Loader
An ajax spinner indicates that something is (probably) still loading, but nothing more than that.

When you need a progress indicator but the data you have to work with is afflicted by one or more of the above problems, it may be time to get creative. You could always fall back to an AJAX-style spinner animation, but this only works for processes that take relatively little time. It is better than nothing, but does not really indicate progress and most users in 2020 know that a process can stall even when there's a spinning animation on the screen. If at all possible, we should leave the AJAX era and it's spinner animations behind and build something better.

Challenges for Warhol's progress bar

The challenges for the progress indicator in Warhol's browser extension are as follows:

  • A simple AJAX spinner is out of the question. Not only are some web pages so mind-bogglingly complex that sometimes even Warhol's lighting fast tests can take half a minute (way to long for just a spinning animation), but we can't do animation in the first place! Analyzing a web page's styles means thrashing the layout at least some of the time, so any animation would be quite jerky. Plus, we do have some intermediate data that a user might want to see, so a proper progress bar is what we want to aim for.
  • Unfortunately, we can not know how in advance much work we have to do before we do it. About 5% of all elements on any web page are invisible and stay invisible all the time. We don't test those and their children, but we have to look at their styles before we know that these elements need no further investigation. In other words: about 5% of the data we work on does not require any work, but we can't identify the affected 5% beforehand.
  • Test run times are all but constant. Once Warhol's caches warm up, ”tests” turn into relatively fast SHA-1 hashes and lookups in tables.

In short, we don't exactly know our workload, but we can assume that we will get faster and faster as time goes on. Based on this, we built a progress bar that uses flawed data for its first 80% and fabricates the last 20% out of thin air. This results in a progress bar that is only somewhat connected to reality, but which looks acceptable and keeps users from giving up too early.

How to lie with progress bars

First, we guesstimate the number of elements we have to process as follows:

export const guesstimateElementsToTest = (root) => {
  return (
    root.querySelectorAll(":not(link):not(script):not(style)").length * 0.9
  );
};

The use of the not very magical magic number leads to a severe underestimation of the actual number of elements Warhol has on its plate, but works with how the progress in the progress bar is computed. For each test result that Warhol generates, it computes a ”progress” value between 0 and 1, based on the number of elements that have been tested so far. The amount of the ”progress” reported varies for each test results, depending on the complexity of the tests and the number of elements involved. This ”progress” is sent to the progress bar, which takes it at face value as long as the reported progress is below 80%.

A graph showing the relationship between the remaining time and indicated progress
The last 20% of the progress bar are fabricated to still feel like a continuation of the progress updates that came before.

For progress updates over 80%, the progress bar disregards the reported progress data but instead adds half of the remaining progress to the displayed progress value. Every update over 80% is half of the remaining few percent and is completely disconnected from the reported progress value. The resulting updates get smaller and smaller of course, but the update frequency increases towards the end of a test run thanks to caching. As a result, the progress bar reaches 100% pretty much when the test is done. The difference between the progress bar's two modes of operations is not noticeable most of the time and the user remains patient.

In summary:

  1. We deliberately underestimate the total work load
  2. For the first 80% of the underestimated work load, we take the progress at face value
  3. We assume that all caches have warmed up by the time we reach 80% and that whatever work is left will be completed much faster
  4. To hit 100% exactly when the test is done, we forge the last 20% on the progress bar by halving the remaining value for each update, which appear much more frequent at this point

This is why (and how) Warhol lies with progress bars!

The huge difference that a little lie can make

The most striking thing about progress bars is the difference they make on perceived performance. For most of its existence, Warhol did not have a progress bar, but was somewhat faster than it is today. In fact, we made Warhol slower specifically to add the progress bar! Instead of just running a huge test function from start to finish (simple and fast), today's Warhol is based on lazily generating the test result element by element and interleaving the actual test run with requestAnimationFrame(). When measured objectively, this is slower than just running a function, but this approach makes our progress bar possible. And I know this progress bar is at least part fake, because I built to be fake. I know Warhol is slower than it used to be, I made it slow myself, on purpose. It still feels faster than before.

Written by

Peter Kröner

Trainer for frontend technologies, podcaster @workingdraft.

Twitter

More from the blog

Stay up to date

Stay tuned and subscribe to our mailing list! We keep you posted on Warhol's progress.