Designer’s Field Guide

Visualizing the Difference between Analytics & Monitoring

Different workflows for learning and for watching

Tim Sheiner
6 min readOct 26, 2023
Photo by Tamas Tuzes-Katai on Unsplash (edited)

Analytics Enables Monitoring

Analytics and monitoring are different workflows for looking at data collected from a system. They are related to each other like this:

We analyze a system so that we can learn how to monitor the parts of it that are important to us.

While data visualization plays a crucial role in both of these activities, the kinds of visualizations and the contextualizing UX around them is quite different. Therefore, as a designer, it is helpful for you to have a strong intuitive feel for the difference between these workflows. The purpose of this essay is to use visual metaphors to help you develop a basis for that intuition.

In order to get a feel for the difference between analyzing or monitoring data let’s begin by not thinking about data at all!

Instead, imagine finding a spring in a forest.

Water is bubbling up,

Bubbling dry spring of the Leinleiter, 2020

creating a flow

Photo by Peter Livesey on Unsplash

that joins other flows,

Photo by Rostyslav Savchyn on Unsplash

until there is a river

Photo by Jason Schuller on Unsplash

that winds, grows, slows

Photo by Ivars Utināns on Unsplash

and eventually disappears into the sea

Photo By Greg Ritz

Let’s pause and make sure we understand the analogy before continuing:

  • the spring => data source
  • the water => data
  • “bubbling up” => change over time

And the images of flows mixing and eventually disappearing into the sea model the reality that data generated by some element in a system becomes harder to distinguish as an individual signal the further away from the source that we are logically, physically or temporally.

OK, let’s finish our story…

…Now, imagine that, before it disappears into the sea, you want to understand the flow.

There are two different things you can do.

1. You can collect a sample and measure things about it:

2). You can record measurements directly from the flow in real time:

69ADCP Traveller — NIWA

The first approach is analysis; the second, monitoring.

Let’s see how these different approaches to measuring affect the kind of data you get and how it is used.

Analytics

Data

When you analyze a sample from a flow you are concerned with describing features of its composition. Some time after the sample was collected you will get a data set describing its properties something like this:

Water quality data set from Marysville CA WELL 08–01
LAST AND NEXT SAMPLE REPORT (ca.gov)

Let’s notice some things about this data set. First, it is complicated, reporting lots of different properties with different units, some directly and some relative to a standard. Unless you are an expert, it is hard to know what is important. Second, it reports intrinsic properties: characteristics of the sample that are (more or less) independent of time. This kind of data set that emphasizes what over when is characteristic of all analytic workflows.

Usage

Analytic data is usually stored in a relational structure like this:

in order to make it easy to create comparisons with related measures or standards that look something like this:

EWG Tap Water Database

that you tend to find in presentations like this:

Cover of 2021 Marysville CA Water Quality Report
mrl-mrl-2021.pdf (calwater.com)

that are discussed at strategy meetings that feel like this:

Business people discussing a report — photo by rawpixel on Unsplash
Photo by rawpixel on Unsplash

Rules of Thumb for Recognizing Analytic Workflows

  1. The data sets being examined tend to be complicated reports of intrinsic properties.
  2. The use cases for the data are things like planning, preparing reports and other activities where urgency of action is not the most important consideration.

Now that you have a gut feel for the analytic experience, let’s compare it to the monitoring experience.

Monitoring

Data

When you monitor a flow, you are concerned with understanding how it is behaving or changing with time. This means as opposed to analytic data, you care as much, actually perhaps more, about the change between measurements than their particular values. So instead of thinking about a monitoring data set as a collection of details, like an analytic set, think about it as a series of status messages arriving on a frequent, regular interval. Each individual message is usually simple, something like this:

msg = {
timestamp: 01/07/17 8AM,
rate: 10,200,
location: MARYSVILLE GAGE
}

A collection of these messages, organized by the time received, is called a time-series, and might look like this:

YUBA RIVER FLOW in CUBIC FEET/SEC01/07/17 8AM 10,200 MARYSVILLE GAGE
01/07/17 7PM 9,960 MARYSVILLE GAGE
01/08/17 2AM 12,500 MARYSVILLE GAGE
01/08/17 8AM 17,500 MARYSVILLE GAGE
01/08/17 2PM 32,100 MARYSVILLE GAGE
01/08/17 8PM 53,900 MARYSVILLE GAGE
01/09/17 2AM 89,100 MARYSVILLE GAGE
01/09/17 8AM 90,800 MARYSVILLE GAGE
01/09/17 2PM 81,900 MARYSVILLE GAGE

Just as we did for the analytic data, let’s take a moment to notice some things about this data set. First, compared to the analytic set, it is much simpler, reporting just a time, location and flow rate. Second, it reports an extrinsic property of the flow, the rate, which could not be measured later in a lab because it only makes sense as a report of what was happening exactly when and where it was recorded. This kind of a data set containing extrinsic properties organized by the time the measurement was taken is characteristic of all monitoring workflows.

Usage

Monitoring data is generally stored in a flat file structure to enable rapid retrieval and is usually visualized as a simple line chart like this:

which, in this particular case, shows that after a massive rainstorm in California in January 2017 the flow in the Yuba River increased dramatically, creating an serious threat to life and property

Flood_under_the_Old_Route_49_bridge_crossing_over_the_South_Yuba_River_in_Nevada_City

that required an immediate response like this:

https://pxhere.com/en/photo/1276898

Rules of Thumb for Recognizing Monitoring Workflows

  1. The data sets tend to be simple time-series of extrinsic properties.
  2. The use cases for the data are to do with taking action where time is of the essence.

Analysis vs Monitoring => Learning vs Managing

The premise of this essay is that the difference between analytics and monitoring — the reason I am want to pursue one workflow or the other — depends upon whether my goal is to understand a system or manage it. The caveat being, I can’t manage a system until I understand it (at least some of it) which means, generally, analysis is always a precursor to monitoring.

From a UX standpoint, the critical difference between these two workflows comes down to the amount of focused attention they each require. Analysis is about understanding unknowns and so requires high focus, lots of attention, and an iterative approach. My essay The Analytic Workflow describes explicitly the pieces of this experience that need to be considered in the design of analytic tooling.

Monitoring, on the other hand, is about watching a system. Particularly if this can be automated, it requires little of my attention, at least until I get an alert that something is happening that I need to respond to. My essay Metric Display Standards explains the most useful visualizations and interaction models for monitoring.

In a nutshell:

Analysis UX is for understanding;

Monitoring UX is for action.

--

--

Tim Sheiner
Tim Sheiner

Written by Tim Sheiner

System thinker, story teller, designer, husband, father of 3, San Franciscan, Bernal Heights neighbor

Responses (1)