Partner Highlight - Orchard Analytics

😱 My dashboard looks broken, and I’m about to join a meeting! Can you fix it right away? 

Roland Cassirer and Ryan Brennan, the Co-Founders of Orchard Analytics, used to encounter these messages from clients all too often. Their team would immediately jump into action, trying to identify the root cause and resolve the issue—all before the client meeting started.

Over time, having to answer questions like “why has this data been broken for three weeks?” left the Orchard team feeling uneasy - while they didn’t cause the underlying issues, their team did own and maintain the data stack. Any time one of these questions was asked, they could feel their clients losing  trust in not only the underlying data, but also their consulting services.

Dealing with dreaded data incidents

One way that Orchard Analytics immediately helps their clients from day one is by educating them on how data engineering is not a one-and-done process. It requires the right infrastructure, but also a discipline in maintaining data systems.

“When you have an average of 30 to 40 data sources, something critical is bound to break at least once per week,” said Roland. “You need data engineering support in place to manage these incidents.” 

Take replication errors as an example. Stitch is often used to extract data from different ad platforms and load this data into a warehouse. When business stakeholders create new ad groups, the extraction process sometimes fails and the data isn’t loaded as expected. As a result, the client’s ad spend appears to skyrocket or plummet. When the business stakeholder views their ad spend or marketing dashboard days or weeks later, they realize the data looks wrong—and the dreaded cycle begins again.

Another common issue they ran into was AWS DMS replication issues. When data wasn’t properly replicated from source systems into Redshift, critical reverse ETL flows stopped working which translated into marketing automation emails not being sent. At any time of the day, they would receive a message from their clients’ CMO asking why revenue dramatically decreased.

In addition to Stitch and DMS source issues, the team ran into other typical data quality issues caused by erroneous manually entered data, discrepancies caused by homegrown custom ETL software, UX bugs in upstream web applications, and reconciling differences between payment providers and billing systems.

As a result, the Orchard Analytics team were spending up to 15% of their time, or 24 hours per week across a 4-person team, dealing with data incidents like these. The issue could sometimes be resolved in minutes; other times, it took hours or even days to fix. 

But their challenge wasn’t solving the problem; it was discovering it existed in the first place, so they could take action before downstream dashboards and reports were used by business stakeholders. Unfortunately it sometimes took business users weeks to discover that they were using incorrect data.

Searching for a data observability solution

In an attempt to address this challenge, the team tried writing dbt tests. While this is a great way to test a small set of critical models, it’s difficult to scale to hundreds of tests without introducing noisy alerts and maintenance costs. Even with dbt tests, they were still unable to catch silent issues that occurred earlier in their ETL pipeline, and test failures didn’t give them a sense of how urgent the issue was. 

They also tried altering their data modeling design patterns in Looker to make things break instead of failing silently, but this didn’t actually solve the problem.

Eventually, the team evaluated a handful of data observability platforms but were unable to get started. “Unfortunately, we never had a tool because it was either too expensive or too difficult to set up. We’d largely rely on business users to flag issues for us.”

Metaplane was different. With a free-forever plan and usage-based pricing, the tool scales with their clients as they grow. It can also be implemented in a matter of minutes, not months. 

“Relative to the value the product delivers, the cost is low,” said Roland. “That made choosing Metaplane a no-brainer for us.”

How Orchard Analytics uses Metaplane

With Metaplane, Orchard Analytics is now the first to know about data quality issues. This not only builds trusted data sets for their clients, but also establishes Orchard Analytics as a trusted partner.

“Metaplane is the only tool we roll out to all of our clients,” said Roland. “It’s genuinely useful no matter the company size or industry.”

With automated, real-time anomaly detection, they are instantly alerted to emerging data incidents that impact business critical metrics. This helps their team proactively fix issues, rather than reactively find out about data issues that happened weeks ago.

Now, when Stitch doesn’t properly replicate data or the data is incorrect, the Orchard team receives a Slack alert about the anomalous volume and freshness of Stitch schemas and tables. When DMS stops replicating data, Metaplane proactively notifies the team and the issue can be fixed immediately, avoiding sudden drops in revenue they used to experience.

“Metaplane helps us be proactive,” said Roland. “It empowers us to quickly detect, investigate, and resolve data quality issues—before they have a chance to cascade and compound.”

In addition to timely data quality alerting, functionality like lineage and usage analytics help them understand the impact of data issues, how their data is being used, and lets them effortlessly prioritize their work now with context such as alert severity.

Metaplane has helped Orchard Analytics build trust in not only the underlying data, but also in them as a consultancy. This has resulted in stronger client relationships, and opened new doors to exciting and valuable work.

“When we’re on top of our client’s data quality issues, they see how valuable we can be,” said Roland. “As a result, they often ask us to work on higher-value projects.”

***

Are you in need of a trusted data partner to help take your data stack to the next level? Get in touch with Orchard Analytics →

If you are a data consultancy looking for a data observability platform to help your clients trust their data and get ahead of data quality issues, get in touch or become a partner by filling out this form.

‍