Launching Today: The Next Evolution in Data Observability

Solving the biggest problems with traditional data observability tools

If you work in data, you've likely encountered challenges in monitoring quality, performance, and spend. You find out about issues too late. You spend more time debugging them than working on core projects. And you know that your warehouse bill is likely way higher than it should be.

But worst of all, you feel that trust in your company’s data isn’t guaranteed.

Perhaps you've already heard of data observability as a solution — hundreds of teams already use Metaplane to ensure trust in their data. But the biggest challenge to getting started with any data observability tool is the time and effort to figure out what you need to monitor for anomalies.

Every data observability tool today works in a similar way: determine what tables or columns you want to monitor, spend time adding each monitor manually, wait a few days for training to complete, and eventually catch issues in the places you know about.

That’s why most people think of data observability as a quarterly objective, something that needs to be scoped, with plenty of time set aside for implementation—not something you can have running on your warehouse in 20 minutes.

From the beginning, we’ve designed Metaplane to be different: it takes minutes, not weeks to connect your warehouse and start monitoring your data. Our mission is to help every team trust their data, regardless of size of team or bandwidth available. Today, we’re announcing a major update to our monitoring platform to further this mission: making it more flexible, more powerful, and easier to use than ever.

We’re making it dead simple to monitor everything

With Metaplane, you’re always the first to know about data quality issues. No more analysts grumbling about weird numbers in their reports. No more customers clogging up your support inbox because of lost data. No more CEOs panicking at two in the morning about a broken dashboard.

That’s because Metaplane helps you monitor all the important things about your data, including the bread and butter of data observability: the freshness and volume of tables and views, the distribution of values in your numerical columns, and the nullness or uniqueness of your most critical data.

But Metaplane’s new monitoring platform empowers you to monitor so much more.

Snowflake spend monitoring

If you’re a Snowflake customer, we’re willing to bet that managing the cost of your Snowflake warehouse is top of mind. Perhaps you’ve even had a particularly slow query that racked up several credits in a single go, or users who unexpectedly caused your compute spend to spike. When spikes occur, you need to know about them as they’re happening—not weeks after the fact, when you get your Snowflake bill.

Metaplane can monitor your Snowflake account and ping you with an alert every time we spot an unusual increase in spending. Those timely alerts will help you keep your costs under control and make sure you’re using your resources most efficiently.

Group by monitors

Tables often have logical segments — the channels feeding into you marketing table, or the individual customers within your customers table, for example. Those segments can hide fascinating insights, like when a marketing channel stopped sending data, that might be missed when monitoring the table as a whole.

With Metaplane’s group by monitors, you can break down your tables into individual segments and monitor those segments for row count, freshness, or any of Metaplane’s other monitor types. Configuring is as simple as choosing a column (or multiple) to group by, or as powerful as writing complex SQL to outline your grouping criteria.

dbt job monitoring

dbt’s powerful transformation platform is a favorite among Metaplane customers. But for users who are looking for deep insights into their dbt jobs, Metaplane monitors job durations to help you catch stuck or long-running jobs, preventing issues downstream.

Source-to-target monitoring [coming soon]

In modern data stacks, data often flows from transactional databases to data warehouses. Data engineers want to be the first to know when this process breaks—all the data didn’t arrive, or it looks funky—so that they can triage any effects on downstream systems and BI tools. Metaplane’s upcoming source-to-target monitor will give you peace of mind that the data is flowing as it should

Auto-magical setup and insight into your data

You’ve heard it before: knowledge is power. The same is true for knowledge of your metadata. The more you now about how your data behaves, the better you can manage your data stack. Metaplane has tools to help you make sure bad data never slips through the cracks.

Automatic metadata collection

Most companies choose not to monitor everything in their database—for good reason. It can be noisy, and not every data issue is worth getting an alert for. But that doesn’t mean that the data in those unmonitored tables isn’t important, or that you won’t sometimes want to dig into the metadata.

For Snowflake and BigQuery users, Metaplane captures regular snapshots of the freshness and row count of every table or view in your warehouse. This historical metadata can be a gold mine for troubleshooting pipeline errors when things go wonky in a table you don’t have a row count or freshness monitor on.

Suggested monitors

You probably have a pretty good sense which tables are your most critical, or what columns are in your company’s most vital dashboards. But data environments are complex, with dozens of schemas, hundreds of tables, and thousands of columns. All that complexity makes it difficult to keep track of what's used on your C-suite's favorite dashboard, or what data powers important business logic—especially when your data is constantly growing and changing.

That’s why Metaplane develops monitor suggestions, perfectly tuned for your individual data environment. The suggestions are based on numerous factors: usage, column-level lineage, and insights we’ve gleaned from the hundreds of other data teams who use Metaplane. You’ll never have to worry that you’ve missed adding monitors to something important.

This new platform is an unusual milestone for Metaplane. Startups don’t usually take chances like this. We’re told to move fast, to break things, to do only what it takes to acquire the next user.

But hundreds of companies trust Metaplane to help them spot and fix their data issues, and we’re building our platform for those customers. We’re building it to last.

And we’re building it for companies like yours. If you’re interested in giving Metaplane a try, give it a try (for free) today, or feel free to get in touch with our team of experts to discuss your data observability needs.