Designer’s Field Guide
Visualizing the Difference between Analytics & Monitoring
Different workflows for learning and for watching
Analytics Enables Monitoring
Analytics and monitoring are different workflows for looking at data collected from a system. They are related to each other like this:
We analyze a system so that we can learn how to monitor the parts of it that are important to us.
While data visualization plays a crucial role in both of these activities, the kinds of visualizations and the contextualizing UX around them is quite different. Therefore, as a designer, it is helpful for you to have a strong intuitive feel for the difference between these workflows. The purpose of this essay is to use visual metaphors to help you develop a basis for that intuition.
In order to get a feel for the difference between analyzing or monitoring data let’s begin by not thinking about data at all!
Instead, imagine finding a spring in a forest.
Water is bubbling up,
creating a flow
that joins other flows,
until there is a river
that winds, grows, slows
and eventually disappears into the sea
Let’s pause and make sure we understand the analogy before continuing:
- the spring => data source
- the water => data
- “bubbling up” => change over time
And the images of flows mixing and eventually disappearing into the sea model the reality that data generated by some element in a system becomes harder to distinguish as an individual signal the further away from the source that we are logically, physically or temporally.
OK, let’s finish our story…
…Now, imagine that, before it disappears into the sea, you want to understand the flow.
There are two different things you can do.
1. You can collect a sample and measure things about it:
2). You can record measurements directly from the flow in real time:
The first approach is analysis; the second, monitoring.
Let’s see how these different approaches to measuring affect the kind of data you get and how it is used.
Analytics
Data
When you analyze a sample from a flow you are concerned with describing features of its composition. Some time after the sample was collected you will get a data set describing its properties something like this:
Let’s notice some things about this data set. First, it is complicated, reporting lots of different properties with different units, some directly and some relative to a standard. Unless you are an expert, it is hard to know what is important. Second, it reports intrinsic properties: characteristics of the sample that are (more or less) independent of time. This kind of data set that emphasizes what over when is characteristic of all analytic workflows.
Usage
Analytic data is usually stored in a relational structure like this:
in order to make it easy to create comparisons with related measures or standards that look something like this:
that you tend to find in presentations like this:
that are discussed at strategy meetings that feel like this:
Rules of Thumb for Recognizing Analytic Workflows
- The data sets being examined tend to be complicated reports of intrinsic properties.
- The use cases for the data are things like planning, preparing reports and other activities where urgency of action is not the most important consideration.
Now that you have a gut feel for the analytic experience, let’s compare it to the monitoring experience.
Monitoring
Data
When you monitor a flow, you are concerned with understanding how it is behaving or changing with time. This means as opposed to analytic data, you care as much, actually perhaps more, about the change between measurements than their particular values. So instead of thinking about a monitoring data set as a collection of details, like an analytic set, think about it as a series of status messages arriving on a frequent, regular interval. Each individual message is usually simple, something like this:
msg = {
timestamp: 01/07/17 8AM,
rate: 10,200,
location: MARYSVILLE GAGE
}A collection of these messages, organized by the time received, is called a time-series, and might look like this:
YUBA RIVER FLOW in CUBIC FEET/SEC01/07/17 8AM 10,200 MARYSVILLE GAGE
01/07/17 7PM 9,960 MARYSVILLE GAGE
01/08/17 2AM 12,500 MARYSVILLE GAGE
01/08/17 8AM 17,500 MARYSVILLE GAGE
01/08/17 2PM 32,100 MARYSVILLE GAGE
01/08/17 8PM 53,900 MARYSVILLE GAGE
01/09/17 2AM 89,100 MARYSVILLE GAGE
01/09/17 8AM 90,800 MARYSVILLE GAGE
01/09/17 2PM 81,900 MARYSVILLE GAGE
Just as we did for the analytic data, let’s take a moment to notice some things about this data set. First, compared to the analytic set, it is much simpler, reporting just a time, location and flow rate. Second, it reports an extrinsic property of the flow, the rate, which could not be measured later in a lab because it only makes sense as a report of what was happening exactly when and where it was recorded. This kind of a data set containing extrinsic properties organized by the time the measurement was taken is characteristic of all monitoring workflows.
Usage
Monitoring data is generally stored in a flat file structure to enable rapid retrieval and is usually visualized as a simple line chart like this:
which, in this particular case, shows that after a massive rainstorm in California in January 2017 the flow in the Yuba River increased dramatically, creating an serious threat to life and property
that required an immediate response like this:
Rules of Thumb for Recognizing Monitoring Workflows
- The data sets tend to be simple time-series of extrinsic properties.
- The use cases for the data are to do with taking action where time is of the essence.
Analysis vs Monitoring => Learning vs Managing
The premise of this essay is that the difference between analytics and monitoring — the reason I am want to pursue one workflow or the other — depends upon whether my goal is to understand a system or manage it. The caveat being, I can’t manage a system until I understand it (at least some of it) which means, generally, analysis is always a precursor to monitoring.
From a UX standpoint, the critical difference between these two workflows comes down to the amount of focused attention they each require. Analysis is about understanding unknowns and so requires high focus, lots of attention, and an iterative approach. My essay The Analytic Workflow describes explicitly the pieces of this experience that need to be considered in the design of analytic tooling.
Monitoring, on the other hand, is about watching a system. Particularly if this can be automated, it requires little of my attention, at least until I get an alert that something is happening that I need to respond to. My essay Metric Display Standards explains the most useful visualizations and interaction models for monitoring.
In a nutshell:
Analysis UX is for understanding;
Monitoring UX is for action.
