Two custom SQL monitor upgrades

Metaplane’s custom SQL monitors have always been a powerful companion to Metaplane’s out-of-the-box monitors (like row count, freshness, and nullness, among others). Custom SQL monitors let you apply machine learning-based anomaly detection to any query you can express with SQL. Even better, they’re flexible enough to monitor a single value, or to monitor a group of values, with individual models built for each.

Sometimes, custom SQL monitors are the best way to capture complex logic that out-of-the-box monitors cannot. For example, maybe you want to write a row count monitor that groups your product activity by customer—but the logic that defines a customer can’t be captured in a single column, and you have to do it with SQL. Or imagine you want to create a freshness monitor, but there’s one of several columns that could indicate how fresh the data is.

And while our default custom SQL models work for anomaly detection of all shapes and sizes, the  models for our out-of-the-box monitors are custom-built to spot the specific sorts of issues that tend to pop up when monitoring specific types of metadata—like unusual row counts, or weird freshness patterns.

In light of that, we’re thrilled to announce that our custom SQL monitors are now just a little bit more powerful. In addition to the default custom SQL model, you can now apply any of Metaplane’s out-of-the-box models to your custom SQL monitor. So in the example above, where you’re building a row count monitor with advanced grouping logic? Apply the row count model in lieu of the default model for more finely-tuned predictions. And if you’re creating a freshness monitor that’s evaluating several columns at once? Apply the freshness model so that you can take advantage of Metaplane’s proprietary freshness modeling.

If you’re not sure which model to add to your custom SQL monitor, we’d recommend sticking with the default, which works well for a wide variety of monitoring use cases, from business metrics to sanity checks and everything in between.

Alongside this feature comes a small, quality-of-life upgrade for custom SQL monitors. Previously, if you had a monitor that returned a percent, decimal, or number of seconds, those values would always show up as integers on the monitor page in the table and the graph. Now, if your custom SQL monitor returns one of these types of values, you can select which format you want to see that number displayed as. Choosing “Percent” will render those values as a percent, choosing “Time (in seconds)” will render those values in human-readable minute, hour, day, week, and year increments, and so on.

We hope these updates make your custom SQL monitors a bit more flexible and easier to use.