L37.0 Finding Insights Using Analytics15%

Analytics

Analytic functions in the chart builder either collapse many MTS into fewer series (aggregations) or reshape each MTS on its own over a time window (transformations). Formulas combine plots, timeshift lets you compare against the past, and filters or Exclude narrow which MTS a function sees. Domain 7.0 carries 15% of the exam.

Think of your metrics as a room full of people each holding a number. An aggregation asks the whole room one question (what is the total?). A transformation gives each person their own coaching (smooth your own numbers over the last hour). Formulas let you do math between two charts, and timeshift lets you hold up last week's room next to today's.

INTERACTIVE

Aggregation or transformation?Squash the diaries, or smooth each one

Four hosts, two in env=prod and two in env=staging. An aggregation combines MTS at each moment, so the line count drops. A transformation works on each MTS over a window of time, so the line count stays the same. Watch the counter.

Picture four diaries side by side. Aggregation reads straight down one page across all of them and writes one total: four diaries become one. Transformation takes each diary alone and smooths it over the last few pages: still four diaries. That's the whole difference the exam wants.

Analytic function
01 · BLUEPRINT 7.1

Totals across sources and group by

To get a total across all sources, add an aggregation such as Sum: Aggregation to the plot. It adds values across all MTS at each time period and outputs one MTS. Mean, Count, Min, Max and Percentile have aggregation modes too.

  • Aggregations combine all datapoints at one moment into a single MTS.
  • Group by keeps one output line per value of the chosen property, for example one mean line per datacenter.
  • MTS that lack the grouped property are excluded from the result.
  • Transformations are a different class: they run on each MTS separately over time.

An aggregation is a class vote. Everyone's number goes in, one answer comes out: the total, the average, or the biggest. Group by means you hold one vote per team instead of one for the whole school, so you get one line per team.

If a student has no team on their name tag (the property is missing), they are left out of the team votes.

Class vote. An aggregation collects every student's number into one answer. Group by splits the class into teams first.

FunctionAggregation or transformationWhat it does
SumBoth (Aggregation, Transformation)Aggregation adds values across MTS per period, per group. Transformation sums each MTS over a window (default 1 hour).
MeanBothAggregation averages across MTS per period. Transformation is a moving average of each MTS (default window 1 hour).
Minimum / MaximumBothAggregation gives the min or max across MTS. Transformation gives it per MTS over a window.
PercentileBothAggregation gives the chosen percentile across MTS (default 95). Transformation gives it per MTS over a moving window.
CountAggregationCounts MTS that have values. Often used to spot missing data.
Top / BottomSelectionKeeps the N highest or lowest MTS (default 5), or those above or below a percentile.
ExcludeFilterRemoves values above, below or outside thresholds.
DeltaPer MTSDifference between each value and the previous one, for each MTS.
Rate of ChangePer MTSLike Delta, but divides the change by the elapsed seconds.
TimeshiftOffsetFetches data from the past so you can compare it to now.
ScaleMathMultiplies each datapoint by a number, such as 100 for a percentage.
Remember for the exam
Aggregations combine all datapoints at a single point in time and output one MTS.doc
Group by combines MTS that share values for the named properties, so mean CPU grouped by instance type gives one MTS per instance type.doc
Data missing the grouped property is excluded when you apply an analytics function and then group by that property.doc
To group, choose Add Analytics, pick a function, then pick a Group by value.doc
Mean: Aggregation without a group by produces one line, the mean across all time series at each interval.doc
02 · BLUEPRINT 7.2

Combining plots with formulas

Each row in the chart builder is a plot with a letter (A, B, C). Choose Enter Formula to compute a new plot from other plots, for example A-B or (A/(A+B))*100.

  • A formula accepts only arithmetic symbols (+, -, *, /), parentheses, numbers and plot letters.
  • A formula can depend on one or more plots.
  • Hiding a plot with its eye icon only removes it from the display. The formula still uses it.
  • If you reorder plots, formulas update to the new letters.
  • Several analytics on one plot run in the order shown, and you can drag to reorder them.

Picture three transparent sheets labeled A, B and C. Sheet C is a note that says "take A, subtract B." You can cover up A and B so only C shows, and C still works, because covering a sheet does not delete it.

Only simple math is allowed on sheet C: add, subtract, multiply, divide, and brackets.

Cover the sheets. Hiding A and B is like covering two sheets with a card. Sheet C still reads them underneath.

Remember for the exam
The formula field only accepts arithmetic symbols, parentheses, numbers and letters that represent plot keys.doc
A formula can depend on one or more other plots in the chart.doc
The eye icon on a plot line shows or hides it, and you can hide plots A and B to display only formula plot C.doc
After you resequence plots, formulas in the chart update to reflect the changed plot letters.doc
When you apply several analytics functions to one signal, they are applied in the order they display.doc
A calculator icon marks a plot that is a math expression, and a ruler icon marks a metric.doc
03 · BLUEPRINT 7.3

Weekly, daily and hourly comparisons

To compare now with the past, clone plot A as plot B, add Timeshift to B (examples: 5m, 2d, 1w), then add formula plot C. A-B shows the difference from a week ago. Timeshift applies to the whole plot and the offset must be zero or positive and at least the data's minimum resolution. Units are w, d, h, m, s.

  • The standalone Timeshift function is a fixed-width shift, such as 30 days.
  • With calendar windows, chart timeshift is cycle-aware (a one-month shift on month cycles lands on month ends) and is available only when partial values are hidden.

Timeshift is a time machine for one line. Copy your chart line, send the copy back one week, and draw both. Now you can see today beside the same moment last week. Plot C is the gap between them.

A normal timeshift always moves a fixed amount, like exactly 30 days. The calendar kind knows that months have different lengths and lands on month ends.

Time machine. Timeshift sends a copy of your line back in time so you can lay it over today's line.

Remember for the exam
Timeshift fetches data from a past offset so current values can be compared with past values, and it applies to the whole plot.doc
Timeshift offsets use the units w, d, h, m and s, and must be zero or positive.doc
The documented weekly comparison clones plot A, adds a 1w timeshift to the clone, and uses formula A-B in plot C.doc
In charts that use calendar windows, timeshift is available only when partial values are hidden.doc
The standalone timeshift analytics function performs a fixed-width shift, such as 30 days.doc
Detectors run on a regular interval called the detector resolution and evaluate their condition each time.doc
04 · BLUEPRINT 7.4

Percentages, ratios and rate of change

Ratios use a formula over two or more plots. Docs example: plot A is zipper.missCount, plot B is zipper.hitCount, plot C is A/(A+B) with Scale set to 100, which gives the miss percentage over time. Scale multiplies each datapoint by a number (default 1) and is the way to turn a ratio into a percent or convert units.

  • Percent change against the past: plot B is A with a 5m Timeshift, plot C is (A/B-1) scaled by 100.
  • Delta is the change from the previous value per MTS. Rate of Change is the same change divided by elapsed seconds.

A ratio is a slice of a pie. If you have misses and hits, the miss slice is misses divided by the whole pie (misses plus hits). That gives a number between 0 and 1. Multiply by 100 and it reads as a percent. The Scale function is that multiply-by-100 step.

Delta says "how much did it change since last time." Rate of Change says "how much per second."

Slice of pie. A ratio is one slice divided by the whole pie. Scale by 100 and it reads as a percent.

Remember for the exam
The documented miss percentage uses formula A/(A+B) with scale:100, where A is missCount and B is hitCount.doc
The documented percent change against 5 minutes ago uses formula (A/B-1) with scale:100, where B is A shifted by 5m.doc
Scale multiplies each datapoint by a number, with a default factor of 1, and is used for percentages or unit conversion.doc
Delta is the difference between each value and the previous one, computed independently for each series.doc
Rate of Change is like Delta but divides the change by the seconds elapsed between intervals.doc
Rename a formula plot so hover text and the data table are informative.doc
05 · BLUEPRINT 7.5

Moving, calendar and dashboard windows

Selecting Transformation instead of Aggregation runs the function on each MTS over a time window, so you get one output MTS per input MTS. Transformation is available for Mean, Minimum/Maximum, Percentile, Sum and Variance. Three window types exist.

  • Moving window: a rolling span such as the past 5 minutes. Default for Mean, Sum, Min, Max and Percentile is 1 hour.
  • Calendar window: Sum, Mean, Max and Min only. Cycles: hour, day, week, month, quarter. A calendar time zone is required, set per chart or detector.
  • Dashboard window: follows the dashboard time picker.

Hide partial values shows only the value at each cycle end. Calendar-window charts cannot have a resolution finer than one hour.

A moving window is a spotlight that slides along your line, always looking at, say, the last 30 minutes, and averaging what it sees. This smooths out the jumpy bits.

A calendar window uses the real calendar: "today so far," "this week," "this month." It starts over at midnight or Monday, in whichever time zone you pick. A dashboard window just matches whatever time range the dashboard is showing.

Sliding spotlight. A moving window is a spotlight sliding along the line. A calendar window is a page of a wall calendar that resets each cycle.

Remember for the exam
Transformations run on each MTS separately over a time window and produce one output series per input series.doc
Mean Transformation is a moving average with a default window of one hour.doc
Sum, Mean, Maximum and Minimum can use a calendar window, with cycle lengths of hour, day, week, month and quarter.doc
A calendar time zone is required, set per chart or detector, and shared by all calendar window functions in a chart.doc
A chart with calendar-window transformations cannot have a resolution finer than one hour.doc
Hide partial values shows only the final value at the end of each cycle.doc
Only charts configured with a dashboard window transformation update when the dashboard time picker changes, and these cannot be used in detectors.doc
06 · BLUEPRINT 7.6

Applying analytics to a subset of MTS

You can limit which MTS a function sees in three ways.

  • Filters on the plot use metadata (dimensions and properties) and narrow the signal before any function runs. Each filter shows as a token on the Plot Editor tab. A grey token means a dashboard variable or filter overrides it.
  • Exclude filters by value rather than by source, dropping values above, below or outside thresholds. Combining Exclude with Sum behaves like a spreadsheet sumif().
  • Top / Bottom keeps only the N highest or lowest series (default 5) or those above or below a percentile.

Order matters, because analytics apply in the order shown.

Before the class vote, you can decide who is allowed to vote. A filter says "only the students in the red team." Exclude says "ignore anyone whose number is over 80." Top says "only the five tallest students."

Because the steps run in the order you list them, filter first, then count the vote.

Who may vote. Filters, Exclude and Top decide who gets to vote before the aggregation counts the ballots.

Remember for the exam
Filters narrow a signal using metrics metadata, and the chart updates in real time as you apply them.doc
A grey filter token means a dashboard variable or filter is overriding it.doc
Exclude filters series by value rather than by source.doc
Combining Exclude with Sum gives a result similar to a spreadsheet sumif().doc
Top and Bottom keep the N highest or lowest series per period, with a default count of 5.doc
Count counts MTS that have values, including extrapolated points, and is usually used to spot missing data.doc
HANDS ON

Try it in your Observability Cloud org

Part A uses send.sh mts. Parts B to D use send.sh analytics: four hosts in two environments (plus web-05 with no env), and cache hits and misses for the ratio.

Steps
  1. Open Dashboards, then Create (+), then Chart. In plot A's Signal field enter lab.cpu.utilization.
    You should see: One line per host (2 with the mts mode, 5 with the analytics mode).
  2. On plot A choose Add Analytics, then Sum. Make sure Aggregation is selected and no Group by is set.
    You should see: A single line equal to the total across all hosts at each interval.
  3. Change the function to Mean with Aggregation, then set Group by to env.
    You should see: One line for prod and one for staging. The host-05 series with no env is missing from the grouped lines.
  4. Click the Mean function's mode and switch from Aggregation to Transformation. Choose Moving window and enter 5m. Remove the Group by.
    You should see: Back to one smoothed line per host, not one combined line. The count of lines matches the count of hosts.
  5. Remove the analytics on plot A. Use the Actions menu to Clone plot A as plot B, then add Timeshift 5m to B.
    You should see: Plot B looks like plot A but lagging by five minutes.
  6. Add a third plot. Select Enter Formula and type A-B. Then alt-click or option-click the eye icon on plot C.
    You should see: Only plot C shows. It is positive when CPU rose over five minutes, negative when it fell. A and B are hidden but still feed the formula.
  7. Edit plot C to (A/B-1), add Scale with factor 100, and rename the plot to CPU change %.
    You should see: A percent change line oscillating around 0.
  8. Start a new chart. Plot A: lab.cache.misses with Sum aggregation. Plot B: lab.cache.hits with Sum aggregation. Plot C: formula A/(A+B), with Scale 100. Hide A and B.
    You should see: A miss percentage of roughly 10 to 25 percent (misses 100-300 against hits 700-900).
  9. On lab.cpu.utilization, add a filter env:prod, then Add Analytics, then Count.
    You should see: The count equals the number of prod hosts reporting (2).
  10. Remove the filter. Set chart type to List and apply Top with count 2.
    You should see: The two highest-CPU series, and web-04 should appear in the list most of the time because it runs 60-95.
  11. Add Exclude (above 70), then Sum aggregation on the plot, in that order.
    You should see: A total that only includes datapoints at or below 70. It is lower than the plain Sum from step 2, like a sumif.
Practice · lesson 3

Check yourself

Questions are worded like the real exam. Press 1 to 4 to answer and Enter to continue.

Questions are worded like the real exam. Read the explanation after each one, even when you get it right.