Event Transcript

Introduction [00:00:00]

Kevin Hu: Hello everyone. Thank you for joining us on a Wednesday for a very special conversation with two friends, Jake and Keith from Sigma and Snowflake. And today we are gonna be talking about three very interesting topics which Keith will introduce in just a second. We feel like these. the topics that are most timely and timeless for data teams today.

And we're gonna speak to how Snowflake, Sigma and Metaplane, not only touch on those three topics independently by themselves, but also how we all use each other to. Help get beyond, you know, tool by tool siloing so that you have full end-to-end coverage using, in this case the Snowflake Metaplane Sigma stack, which we cutely call the SMS stack.

But, you know, definitely put a thumb down in the chat if it doesn't resonate. But just to kick us off, I'm Kevin. I'm the CEO, co-founder and proudly the Chief SQL Wrangler of Metaplane. We are a data observability tool that helps you be the first to know of data issues by helping you prevent them and also helping you detect them downstream and giving you the context you need to triage.

But Jake, who are you?  

Jake Hannan: Yeah, yeah. I realize I didn't put my, my company tag here, but hi everyone. My name is Jake Hannan. I lead the data platform team at Sigma Computing. If you're not familiar with Sigma, we are a spreadsheet-first kind of BI and exploration tool built for the cloud. So friends like Snowflake and some others BigQuery, Redshift.

But we're really all about changing the game, you know, the way that business folks interact with data and how the data team might operate with them as well. So, excited to chat with y'all.  

Keith Smith: Hey everyone. My name is Keith Smith. I'm a Senior Partner Sales Engineer at Snowflake and in that role I work with several of our technology partners, software companies like Metaplane, like Sigma, and several others in a couple of different categories focused on really making the products work better with Snowflake. So that our joint customers using the products together, see success and can, can realize their business goals and see some great value.

I've been in this space for over 20 years just past my second year at Snowflake. And obviously love working here and love working with these partners. Super excited to be here today.

Kevin: Congratulations on the two year mark, and also a super successful quarter that was announced from Snowflake recently.

Keith: You know, we're proud of all of our quarterly results. Wall Street doesn't always get as excited as we are, but look, we still think the world is our oyster and there's so much potential and growth available in this market. So we're still all so excited.

Kevin: It's still day one and I'm long SNOW.

Yeah. What do those banks, yeah. What do they know they're not using? But I saw that snowflake has approaching 8,000 customers.

Keith : We certainly do. And again, that quarterly customer growth, both in the number of customers, the long term commitments, the consumption contracts that they sign to consume the product you know, continue to grow. You know, we think out of this world Net Promoters score, NPS score, customer satisfaction metrics, Forester... for six years now, a hundred percent of our customers would recommend Snowflake according to Forester. So, yeah, every quarter it's exciting to see where we land with, with the financials and new innovations happening in the product.

Kevin: It's amazing to keep up with all the product news as well. And also before I jump into some questions, we're trying a new format with this webinar where we're gonna be asking each other questions, but I think the most important questions come from the audience. So if you have anything, any comment, anything that you're wondering about, feel free to put it in and we should have the time to address those.

Snowflake: Flexibility and Power at Scale[00:04:12]

Summary: Snowflake is a modern data cloud with many tools for scaling companies. Snowflake's flexibility, scalability, and trust are built to help their customers meet their business goals using data. Snowflake's newest tools, like Snowpark and Unistore, help get even closer to this goal.

Kevin: Keith, across all this incredible growth and approaching to 8,000 customers, what are some of the themes that you see are top of mind for data teams today?

Keith: Yeah, thanks Kevin. So you and I, and, and Jake had some prep for this and kind of decided upon what might make sense to talk about.

I've certainly got some ideas there. Just as a quick introduction for the audience, if you don't know much about Snowflake, we, over 10 years ago, were the first data warehouse built for the cloud. Not an on-prem technology that just happens to run in the cloud, but truly built for the cloud, the infinite elasticity and scale of the cloud running on all three public clouds, Amazon, Microsoft, and Google around the world.

We've started to shift the messaging there and we no longer like to box ourselves into just a data warehouse company, but the data cloud is now how we describe ourselves and what is the data cloud. The data cloud for us is our platform. The product, the architecture, the great innovation happening with our storage, compute, and cloud services, intelligent infrastructure running in the cloud.

But also the second part of it, in addition to the product, is the content, the content available within our data marketplace. The content available that happens with data sharing between companies, data sharing that happens within an organization, data sharing that can happen privately between organizations, customers, suppliers, and partners.

And so that's how we are now thinking about ourselves as the data cloud, the platform and the content to help customers meet their business goals and outcomes using data and technology. And so as we think about these partnerships, companies like Metaplane, companies like Sigma, companies like ours, some things came to my mind that are important for users and customers and buyers to be able to see the benefits and to be happy with their experience.

And the three that I landed on are flexibility, scalability, and trust. So just to go into each one of those, you know, flexibility. Well, what does that mean? Well, one thing it means for us is in addition to being a data warehouse company and that being one of our workloads, we now talk about all the different workloads that we support.

Right. In addition to that, it's data lakes. It's our newly announced unistore. Which is supporting both transactional and analytical data in one platform. Data engineering, data sharing and collaboration. Cybersecurity and ML and data science are these different workloads that we support.

So we want customers to have a lot of flexibility in using Snowflake and not just be tied to one type of workload, but open up all these workloads and support all these workloads natively in the product. And we can talk more about that scale. Obviously we launched as a cloud first company wanting to architect the product in a way that could leverage and take advantage of the infinite scale of the cloud, both storage and compute, and be able to just have those available on demand without really giving it much thought. And of course, trust. And that's where companies like Metaplane are really critical for our joint customers to have trust in using these solutions so that the data being analyzed, visualized, ingested, transformed and used for decisions. There's confidence in that. And so those are some themes that I think are important for everyone watching and certainly that we talk about at the companies that we work at.

Kevin: Flexibility, scale and trust. That makes a lot of sense. And I'm almost imagining it like across like three dimensions, right?

Across different use cases, the depth of use in those use cases and the adoption by the teams. Like how much did they trust the data? And you mentioned like Snow Park and Unistore, I mean, incredible technologies. Is there one feature, I mean one workload that data teams you feel like should definitely check out today?

Keith: Yeah. Thank you. I actually forgot to mention Snow Park and that opening statement, but certainly we love to talk about Snow Park as that just adds flexibility to our offering. Snow Park, for those of you that don't know, is a way that Snowflake supports all types of developers to bring the code to the data and develop in their language of choice. So originally we were a SQL company, SQL still doing a ton of that, but now supporting Java, Scala, Python. Went GA last November. So we want the workloads, we want the data, we want the usage.

We want the developers using Snowflake because it's easy to use, it's cost efficient, and so flexible with both the types of workloads that we offer and support and also the languages and tools that people want to use when working in Snowflake.

Kevin: Amazing. Yeah. Plus one, especially, I mean, speaking for someone with, come from the data science background.

Like you had to pull me to learn sql. I'm glad I did, but being able to write Python and bring the data to the code is very powerful. And, I can't help but ask when you mentioned scalability, cost is a question that [00:10:00] is very top of mind for data teams today in this environment.

And how does Snowflake balance the considerations of a consumption-based pricing model while also keeping costs low for customers?

Keith: Yeah. Great. And just a quick correction, Kevin, again, bringing code to the data, not data to the code, but we want to bring the code to the data. Running within the platform.

So just a friendly little reminder to the audience there, what we are we're focused on. Yeah. So look, with scale, we're always innovating, we're always introducing new features. We are obsessively focused on continually improving the performance of the product, especially our core database engine, being central to our philosophy and we're always releasing performance improvements to our customers that require no effort or cost to adopt. When it comes to perception in the market, you know, we, we do hear that you're expensive compared to vendor X, Y, and Z. It, it is a consumption model. You pay for what you use, you pay for the storage that you use in Snowflake running in these cloud provider environments, you pay for the compute that Snowflake uses in these cloud environments per second of compute.

So certainly can be expensive if you just go in with kind of a buffet all you can eat mentality. We've seen that customers for so long have been operating in this world of scarcity. You think of the old on-prem server world, a data center running onsite and scarce hardware resources and having to make some tough decisions on which workloads get priority, which schedules get priority.

And now operating in a world of abundance in the cloud, where scarcity is a thing of the past and there's just so much compute and storage available. So we encourage customers to think about the total cost of ownership, right? Not just the price of the product or the software that you're using, but the human effort to work in these different platforms and keep things running.

We're always doing our own price and performance benchmarks against the competition. We feel we score very well actually beating our competitors, but certainly we're giving customers the tools they need to be successful, like built-in cost controls, our most successful customers often start their Snowflake journey with a one week quick start engagement with our professional services teams to get them started on the right foundation and understand the built-in cost controls that are there.

Things like auto suspend on a compute warehouse. Things like resource monitors to not go above a certain credit threshold and things like that. So again, in this world of abundance, it can get expensive if you're not focused on your business outcomes and leveraging the technology in the right way to meet those.

But we certainly want customers loving their experience. And again, our, our NPS scores and our customer satisfaction surveys support the success and enthusiasm that our customers talk about.  

Jake: And Keith. I love the point you made about total cost of ownership. Often when you're evaluating tools, you might not really think through that lens, but as a data team, you know, regardless of size, when you're asked by the business to deliver on things, but yet you're still spending time maintaining clusters, they're doing things outside of what drives core business value it's hard to articulate the value your team provides as a data team. A tool like Snowflake, a platform like Snowflake gets us away from that and we can focus on core business outcomes that help drive revenue rather than be caught up in, oh, what's, what's, what do we have to tune today?

Or things like that. So it's been really great for us here at Sigma.

You bet.

Kevin: And likewise, both as a customer and as a partner. Those cost controls, they work. Especially to be able to do that programmatically, there's always organizational and process work to be done, but to have those hard technical guardrails, yeah, it's definitely helped us out a lot.

With that, I will pass the mic over to you, Keith.

Keith: Thanks, Kevin. Great, great questions. And I would just, again, emphasize some of the things I talked about, you know, total cost of ownership of a solution. Like Snowflake and others, not only the, the sticker price of using the product, but the human effort involve the complexity and how hard it is or easy it is to achieve your business goals.

We want customers coming back. We want also to expand within an account and go beyond our original use case. So we're super focused on that.

Sigma: Powerful Insights in a Familiar Setting [00:15:00]

Summary: Sigma is unique in that it allows users to explore data in a familiar interface, like a spreadsheet, while utilizing the scalability and security of Snowflake. Jake demonstrates how Sigma can quickly analyze large data sets and create visualizations in real-time.

Keith: So I'm gonna kind of turn it around and talk to Jake a little bit. I work closely with Sigma, one of my technology partners and elite Snowflake BI partner.

We have some of the same investor backing in both companies, so certainly a company we know and love very well. So Jake, let me just start with if you could just talk a little bit more about Sigma as a product, as a company and kind of what makes you guys unique and special.

Jake: Yeah. Thanks Keith.

You mentioned some great points about Snowflake's background over the past 10 years. Built truly for the cloud. And when you look at kind of the competitive landscape that Sigma operates in a lot of these tools came out before the advent of the cloud data warehouse. And I think fundamentally our goal here at Sigma is to make use of the best things that a cloud data warehouse can provide.

So we're landing all of this data. In a platform like Snowflake, we've got all these great compute resources and all the controls that you mentioned, role-based, access control, all these different things. And really what Sigma does is just bring out the best of that in a familiar interface that everyone has used before or, you know, 90% of folks with the spreadsheet.

And so this is fundamentally different, I think, in my opinion versus the other folks in our competitive space. And so what ends up happening is you're dealing with large data sets. You've got data from all these disparate sources. You can bring it into one interface that feels like the spreadsheet that we've used before and not worry about any of those scale concerns or any of that.

So we're kind of flipping the script on what the old operating model was. My 10 years of working in the data space, the workflow had always been, there's this request, okay, let me go add this to this dashboard. I try to answer all these different questions where really with a Sigma workbook, it's sort of just the starting point and so you can go from, here are some useful KPIs, but I want to explore and answer another question. And it has always been, Hey, I've got this other question. Let me export it to Google Sheets, or do something and manipulate the data in a familiar way. But now we're bringing that, all of that to the platform.

And again, the scalability and security that Snowflake provides, we're able to just use all those best bits. So yeah, I think when I look at my past experience and my ways of working today relative to other products, it's fundamentally changed. I'm much more of an enabler rather than just answering tickets and trying to fulfill requests.

So it's been very rewarding for me working at Sigma and I think that what we're trying to do here is very different. And again, building off of what Snowflake has to offer, we just pull through the best bits there, which is great.

Keith: Yeah. Amazing. And myself, having been in this world for so long, I've had a fair amount of exposure to lots of these tools and certainly as I got to know you guys better really recently, actually, since joining Snowflake, a couple of years ago, was and continue to be super impressed by the Sigma product. And you've got your own set of very passionate, enthusiastic customers that we share, which is amazing. So, Jake, as we continue to talk, and I mentioned some of these things that I think are common themes flexibility, scale and trust.

Do you have any comments on those and Sigma's position? Or approach there to help in those areas?

Jake: Yeah, I mean, scale is the biggest thing that we always talk about. I sort of mentioned it before but traditionally, you know, with other tools, I could talk from experience here at Sigma, we have some pretty large source tables.

So on the scale of billions, 45 billion maybe to be exact. But when you're working with that type of data, traditionally you would have to do some kind of flattening or aggregations. And what ends up happening is you get away from that granular data that's most important. And so as you're doing all these transformations upstream, you lose insight into the actual granular detail that you care about.

But again, given that we just use the best bits of Snowflake, if we are maintaining our warehouse in a performant manner, you know, whether it's things like the query acceleration service that Snowflake just recently released and offers whether it's clustering on certain tables we're able to provide results in lightning speed, almost faster than what the Snow site UI might give back in certain instances because we do some interesting things there. You mentioned elasticity being kind of unlimited, right? And so, so long as we've configured our warehouses in the correct way, within Snowflake, we just take those best parts.

All we're doing essentially is compiling sql, running it against the Snowflake engine and serving those results back. So the experience is great. And you mentioned some things about being cost conscious and the things like that. What we also do is make heavy use of Snowflake's caching. So when people are constantly reusing results, we'll be able to serve that back.

We also have a really neat feature called Alpha Query. And so as we have those results in a workbook, if we don't actually have to go back to the warehouse we can actually do the computations directly in the browser. So this again, your notion about being cost conscious, we also play a role there, but ultimately I think what's important to note.

giving this level of freedom and flexibility to the business is something we haven't done before. So if they're able to use data to find an insight that generates millions or hundreds of thousands in revenue, we kind of flipped the script on what really wasn't possible before, in my perspective.

Keith: Amazing. Jake, did you have anything you wanted to show us? Actually, like a demo or a...

Jake: I feel like we should stop talking about it and just get into it. Let me go ahead and share my screen.

Give the people what they want.

Yeah, yeah, absolutely. So this is kind of just a demo workbook. So this is, this probably feels familiar to those folks. You know, what a standard dashboard might look like. So we've got our KPIs we've got some visualizations about sales to target for some background plugs, electronics, we sell electronics all across the country. And so, with traditional platforms, this is probably what you would have, you'd be able to set some filters if you wanted to. Maybe you wanted to just look at the Midwest. But what I want to kind of talk about is what I alluded to earlier about this notion of exploration.

So, in traditional tools, if there was some other question that needed to be answered, I would have to go back on the data team and create some new visualization or make some updates. But what's so great about Sigma is that we are just compiling SQL right on the fly. And so if I'm looking at something like sales in the west, and I want to drill down into this, there's no predefined drill paths that need to be set by the data team.

Historically you might have to create group sets or different drill paths. But if I just wanna look at okay, within the west what are our top selling stores, I can go ahead and do that. I can also expand this element, look at the underlying data and start to understand different things.

If I wanna look at a specific store, I can continue to drill down and maybe, let's see which customers are here. And so, again, I love this notion that we call these sort of jumping off points because it's just a place to get started. And if for whatever reason I wanted to kind of expand upon this and maybe change the visualization for some reason, Maybe go to an area chart, whatever that might look like, you have the full flexibility. Again, everything starts with one question and then it sort of tails off into maybe 15 other questions. And so we're allowing that flexibility in a way that hasn't been done before. And I think this is the format of a fully fledged dashboard.

But what is also great is our way to get started with an exploration. So I'll jump over to that as well. So this is essentially what folks get started with. I've got the, the same data set under the hood from the boiler plate demo. But if I wanted to, again, starting with the spreadsheet interface, just start to do some basic calculations.

So if I add a new column here and we'll call this revenue and maybe I just want to do price minus costs. So just like you would in Excel or in Sheets price minus cost, we'll go ahead and compute that. And what you'll notice is that was returned instantaneously and we can see that this was actually done within the browser.

So we didn't actually have to go back to Snowflake in this case, which is pretty neat. I'll just add some formatting here. Maybe I wanna format these as well. Just hitting a hot key here. And then we could start to do some interesting things. So if I just want to group by product type and I want to know our revenue by product type, I can create this grouped column.

Go ahead and add a calculation. And all we're gonna do is say some revenue and we can see our revenue here. And just like Excel, if I wanted to do some conditional formatting to see what the highest revenue by product is, we can go ahead and do that. And I'll just say product revenue, call it product reps. And so this type of flexibility to be able to drill down and expand again, not really possible in other tools. And so I'm just gonna get rid of this group cuz I want to create some visualizations really quick. Let's jump back and let's delete this column. Getting started with a visualization is just as easy. We can do a viz and maybe we want to do, I don't know, let's say customer name and we'll do revenue and maybe I just wanna sort this to say, give me the top 10 customers. Let's convert this to top 10. I'll set a filter here and we'll just say, I want the top 10. And very quickly we're able to do this.

And of course, as we create filters to be able to control parts of the page, we can very easily set this. So, you know, within two minutes we've gone from some basic spreadsheet to some visualizations. Another great thing that we have is the ability to parse out JSON. And so if folks aren't familiar with working with JSON, we actually make it super straightforward.

So if you have a JSON field, like we have our customer JSON here, we can actually extract these columns. And if I just want to grab what the loyalty program is, how long they've been a customer, I can easily extract those fields and we can use them in the visualizations down below. Basically, anytime that we have something in the parent element here, we can bring it into the child as well.

And so, again, there's 4 million rows here. You've seen this being returned. This isn't pre-recorded or anything. I could say from experience, one of our data models we're working with is 500 million rows. And again, Keith, like I mention, We have clustering set on that. So you can see results returned in lightning speed.

And I think that kind of speaks to what we're all about here. It's like, how do we get data into stakeholder's hands immediately so they can explore and do the things that they want?

Keith: Amazing, Jake, so exciting to again, watch you work with that product on top of Snowflake. Many of us working in this field and that have worked with data for so long, certainly all of us, I think, have used Excel spreadsheets on our machine locally

Jake Hannan: Yeah.

Keith Smith: With extracts and those of us in the field for a while may remember the row limit in Excel, I think it was 60 something thousand rows. Right. And to see you. At that scale on top of millions, tens, hundreds of millions, maybe even billions of rows live against a cloud data warehouse or the Snowflake data cloud.

In that familiar spreadsheet like interface and experience, I didn't see you write any code. You're just dragging and dropping. You're adding columns, you're pivoting all live. Using the scale of the cloud, I mean, it's a beautiful thing.

Metaplane: an investment in data trust [00:26:30]

Summary: Metaplane, a data observability tool that monitors for anomalies and data quality issues. The tool integrates with cloud data platforms like Snowflake and BI tools like Sigma to provide end-to-end lineage and usage tracking. With Metaplane, data engineers are the first to know about data issues.

Jake: All right, so Kevin, you opened and let's put Kevin on the hot seat and, and learn a little bit more about, about Metaplane.

And Jake, I'm gonna flip it to you to kind of keep our format going here round Robin of host and moderator and answerer of questions. Go ahead, Jason. Yeah, absolutely. So a little bit about my experience with Metaplane really quick. Some background about why I think data observability is so important.

It's 3:00 AM on a Sunday and your CEO decides to give you a call or a Slack. I'm not sure why he or she is up at 3:00 AM but regardless, it happens. And the statement is, this looks wrong, right? And so if you're on a data team, you've probably dealt with this before, and that is probably the most soul crushing thing.

One, it takes away from your time. And two, you lose trust in the business. So we use Metaplane internally to essentially monitor for anomalies, whether it's a table dropping in row count or some column value coming out within some bands that we define. And so Kevin, how do you think about Metaplane playing in this role for BI and on cloud data platforms like Snowflake?

Kevin: Great question. I couldn't have said the value prop better. Being the first to know is important. Trust and time are two things that are very easy to lose, and it's hard to get back. And when that CEO or that VP of sales wakes up at 3:00 AM you know what they're thinking. But data's supposed to be correct, right? What the heck, right? Why am I dealing with this issue? But we know that data is meant to be useful, and it takes work to make it perfect. It will never be perfect, but quality is on a spectrum, and it's based on how people use it. Keith, if you mentioned Snowflake, as the first data cloud built for this modern era, and how Sigma is really adding a unique and powerful workflow on top of that.

Well, Metaplane and data observability tools, to be frank, wouldn't exist without Snowflake. And I'll tell you why. It's because data quality issues are as old as data issues, right? Like you best bet that on every IBM Z series out there, there are data quality checks. The problem is that it impinges on the core analytical workloads, right?

And without Snowflake, both allowing users to horizontally scale the compute and providing the sort of metadata you need to track number of rows, the freshness of different tables. Data teams can't be blamed for choosing analytical workloads over data quality and integrity checks. And that's been the trade off until Snowflake, essentially.

And with tools like Sigma, it answers the immediate follow-up question. So Snowflake makes it possible to collect this metadata about your data. Hence data observability. The immediate question is always, does this matter, right? If a table is delayed by 24 hours, but no one is using it, you can punt it a little bit, but if the CEO is looking at it at 3:00 AM, I also don't know why they're doing it at 3:00 AM, but let's say that they're refreshing their Sigma dashboard at 3:00 AM that's a P zero now, and because at the end of the day when we talk about flexibility ,right? The ability for a tool to support different workloads is important for us to recognize on the data observability side is that trust doesn't only lie with the data team. It also lies with the people who consume data and the people who produce data, right? Like they need to trust the data as well.

Thankfully we're working with amazing partners like Sigma to be able to get more and more of this data quality in front of consumers themselves, so that in context as they're consuming the buffet of data, to use Keith's term, right? They get the, they get the nutrition menu right next to it at the same time.

Jake: Yeah, I love it. And, you know you and I talked about this, Kevin, success for me, and I think for most data teams, no one at the company outside of our team knows about Metaplane. And what I mean by that is there's never any issues. Everything's great. It's all hunky dory. Everything's fantastic.

But you know, as your product scales and evolves, how are you thinking about, you mentioned some things around integrations, but how are you thinking about better ways to help articulate the information provided by Metaplane to other stakeholders just outside of the data team.

Kevin: There's definitely a organizational component as well as a tool component where there is a cultural shift unfortunately, that has to happen a lot of the time around how people think about data, like we said before, many times there's a perception that data is meant to be perfect. Data is an oracle in a way, and it is like truth with a capital T, but that that is not necessarily realistic and often sets up expectations that are destined to fail unfortunately.

So there is this aspect of how do we at Metaplane, but also data teams like yours, communicate to users in a way that's contextualized to their use case. Are we gonna say the words, data quality, data observability and data lineage? Maybe. But perhaps there's some other language that we can use, like a dashboard going down. Being able to go into a QBR, if you're a CSM, and know that these numbers are correct, being able to go to a board meeting with full confidence. This is what matters. And the reverse is true when you go upstream. Let's say you're an engineer writing code, right? So there's an organizational component, but also the technical component where getting a lineage and usage from Snowflake, from Sigma is key because you're not going to slap someone with a lineage map.

This is your full end-to-end. Figure out if your data is correct. No, it needs to be tailored to the use case. Like what workbook am I using right now? How fresh are the tables back in that workbook? If there's something going wrong, who should I talk to? What is upstream? So it needs to be highly contextualized if we want a good shot at making it better adopted.

And by it, I mean all the practices around trust and data.

Jake: Yeah, no, totally agree. I mean, with all that said, I think I wanna see a demo if you're ready. I'm kind of excited to.

Kevin: Yeah. Well I'll be honest, is that while you were while you were demoing Sigma, I went into your R Sigma workbooks too.

Jake: Amazing.

Kevin: We're very lucky to work with Jake and the data team at Sigma, and we are lucky to be Sigma customers ourselves at a Metaplane. And for a little bit of context Metaplane is a product-led self-serve tool. So of course you can talk with us, but if you want to get started and see how it looks and feels with your data, you can sign up and there's a 14 day for each trial as well as a free tier.

And usually the onboarding process, we skip the sign in is very simple. Connect your database, Snowflake. Very easy. A transformation tool or an ELT tool, and then a BI tool like Sigma. And once you connect everything, then you get a couple of things for free. One of those things is schema changes, right?

There's tables, columns, and schemers being added, deleted, and renamed. You get that for free. You get lineage for free, which I'll go into in just one second. And you also get a usage where across different tables and schemas, we parse your query history to understand who is using these tables, how often and when.

And we use this combination of the schema that we ingest, the lineage and the usage to lay the foundation for everything else that's in Metaplane. I started with that for one reason, which is because that is our use case for Sigma, where it's as a product-led company, we need to know who is using Metaplane so that the moment they do, the moment they perform some key actions, we can get in touch with them and offer our support.

Perhaps they never wanna talk to us, and that's totally fine, but if they do want to talk with us, we wanna make sure that it's at the most important moments. So for us, every day we go to two key tables that we pull in Sigma. One is recent teams that have connected a warehouse and an alerting destination like Slack, right?

If you have a warehouse and an alerting destination, you're good to go. And that's the first key event where we think, okay, this person and this team has shown interest in Metaplane. And then over time, track for every team. So for example, here's 11,000 teams that have tried out Metaplane, when their trial was created, when their account was created in a number of tests they have and the number of sources that they've connected.

You can see a lot of Snowflake on there and, I was just experimenting with what you're saying, that the drag and drop to see how to create these visualizations by drag and dropping and yes, it's extremely easy. And we're a little bit behind on the Hightouch integration or we should be using Hightouch to put this back into our CRM so that we can automate some of the actions which we're currently performing cuz we look at these dashboards and say, okay, here's the new trial, we should reach out to them. But I bring that up because we use Fivetran, DBT, Sigma, and Metaplane and Snowflake as the foundation upon which all of it is built. And this is where the lineage comes in. We haven't announced it publicly yet.

However, we have column level lineage and parsing and visualization. If you have it with both within the warehouse, upstream in the warehouse to tools like DBT in your transactional databases and also downstream to to BI tools. So for Sigma, we automatically parse out, okay, what queries depend on this table, right?

What workbooks contain those queries? And you can traverse this all the way up and all the way down. And this is one of the key foundations of Metaplane. Where from here, after we've synced your information schema, then we can begin to start adding tests, like suggesting tests based on the usage of different rows based on the downstream lineage.

Once you start adding monitors, then everything becomes right. You can kind of add 'em in bulk, suggest them, tune them. We'll start to send you alerts based on historical patterns, and those alerts will also contain your downstream lineage. So in this case, if there is a table refresh, which is delayed from 30 minutes to 20 hours, we'll say, okay, here are the downstream sigma workbooks that are affected by this.

So that's the furthest downstream and the furthest upstream is with our GitHub integration, where as you make changes to a dbt model for instance, we will tell you which Sigma workbooks and queries are affected by this change in your model, and also simulate the effect of that change on your  data itself so that you can prevent issues from happening in the first place.

Because our philosophy is, okay, issues that can be prevented, should be prevented, but not all of them can be prevented. And those ones you should detect earlier rather than later. And that's the end to end and I really do mean it where data quality has always been a problem, but we haven't always had at tractable attack on that problem until the data cloud and until partners, other tools you use within your data ecosystem, started having robust enough support for metadata tools like Metaplane to integrate and show you end-to-end what's going on.

Jake: Amazing. Yeah. And of course we use Metaplane internally for all the use cases that you outlined. We also actually use it for, you showed the schema changes. We actually use that to articulate to our stakeholders that there's new things available. While it's not Metaplane's forte, data discoverability is something we're always thinking about as well, and so being able to use that automated schema change detection, we kind of just send out an alert. So a lot of really useful tools in, in the toolkit over at Metaplane.

Kevin: And so lucky to work with you all. Really appreciate it.

Keith: I'm wishing now that I had a tool like Metaplane or Metaplane itself in a job I had 15 years ago working at a high tech manufacturer.

I was a new guy doing some analysis with data, helping report on things for my business stakeholders. It was a different team doing the ingestion and transformation of these jobs into our data warehouse. As a downstream consumer and analyst, the assumption was, well, it's all correct.

Let's build some nice dashboard and reports. And one day my stakeholder said, this is all wrong. And you know what? They were right. It was all wrong. And I just didn't have the notification or the awareness about that. Certainly the tools that I was using didn't have that built in. And there's just something meaningful and powerful about the brand of the products that you're creating internally, right? There's a brand associated with a dashboard or a report, and once that brand starts to sour and there's some rot, it can be tough to win people back. And so as the audience thinks about their peers, their stakeholders, their consumers, and keeping the confidence high in the whole stack, you wanna make sure that there's no suspicion of rot under the covers and tools like Metaplane can certainly help with that.

Kevin: I appreciate you saying that, and how could we have known this is the great asymmetry. Data teams, I mean, how many reports and how many tables are you responsible for?

And that's only growing over time. The stakeholder only cares about one, which is the one that they're currently looking at. So yeah, it's not fair. We gotta level the game.

Wrapping up [00:40:58]

Keith: Well, hey, I know we're about ready to close team, certainly love talking to you both and look forward to more success together. Quick commercial for our Snowflake. If you haven't used Snowflake, you can go to signup.snowflake.com. For a 30 day trial, you can load some of your own data. You can load some data from our data marketplace and just get used to this infinite scale of compute and storage. There are data for Breakfast events happening globally right now. You can look those up at a local Snowflake Data for Breakfast to see if there's one near you that you still have time to attend.

And of course, our upcoming June Snowflake Summit event. We're expecting a huge crowd. Both these guys will be there and you can come network with peers, learn more about Snowflake. Whether you are or aren't using it today, it's gonna be a fantastic event. So that's, that's our June Snowflake summit.

I'll let you guys mention what you got going on.

Jake: Yeah, absolutely. Thanks Keith. On the Sigma side, you can get started with a 14 day free trial at Sigma computing.com/free- trial. When you sign up, we also offer a Snowflake sample database. So if you want to understand the power of Snowflake, we offer that connection right then and there.

So within two minutes you can get to building a workbook on Snowflake right then and there. So I would implore you to check it out if you're interested.

Kevin: And while you're on those 14 day trials with Snowflake and Sigma, make sure to check out the 14 day trial with Metaplane. It's very easy. I think one one amazing thing about our space is how easy it has become to adopt these tools.

Sigma and Snowflake, I can vouch for myself as many other people can Metaplane as well. Very easy to get started probably one afternoon and you can have all three up and running with your own data. But yes. See you all at Summit and appreciate everyone for tuning in. If you have any questions or follow up comments or feedback, please DM us and we'll make sure that this recording is available, available for everyone.

‍