dbt Labs deeply cares about its community, and I felt that at this year’s Coalesce Conference. It’s clear in how they recognize community members, continue to invest in dbt core, and host a delightful event. There was a diverse mix of geographies, roles, and backgrounds represented, all with a shared interest in dbt and how it’s shaping data culture.
At the conference, I learned more about how dbt is empowering businesses to create value from their data with the least amount of friction. I also learned about new features and cool custom solutions, discussed data governance and data trust, and partied (as one should) in Las Vegas.
4 dbt features and capabilities I’m most excited about
Here are my top 4 takeaways from the conference and some exciting dbt product announcements:
1. Semantic Layer + Saved Queries + Tableau
Saved Queries in your dbt Semantic Layer provide Tableau with what look like any other data source, but the metrics are related to specific dimensions, so you only see combinations that the data supports. With most of our customers using Tableau and many starting to use dbt, I’m excited to see this powerhouse setup in action.
2. Pipeline health and freshness checks embedded in Tableau
Tired of those “Hey is this table up to date?” messages? Give your analysts in Tableau a pretty embed of dbt pipeline health and freshness — driven by dbt tests and source data freshness checks — so your analysts can answer this question for themselves.
3. Microbatch incremental strategy (core)
For those really heavy data loads that can’t be done in one query, this new incremental model strategy batches data up and executes each load into your table in a separate query. This will be useful for those projects using large scale transactional data. And I love to see dbt Labs continuing to invest in dbt core.
4. Visual editor
Now you can give your analytics team — who are familiar with no-code data pipelines in Salesforce and Tableau — the power of dbt with an interface they already know. Enable them to build models in a domain-specific dbt project where they can quickly iterate and provide value, without “lobbing a request over the wall to the data team.” Instead, they can find and build insights themselves, with their business context and their expertise. Visual Editor also supports reverse engineering models written in SQL to visual nodes, enabling a new experience to find unused CTEs, unnecessary joins, and other issues in dbt model code.
Add “mesh” to anything, and it’s better, right?
No… but dbt is enabling data mesh! The dbt mesh concept enables the separation of responsibilities for separate business domains while giving you shared lineage, shared documentation, and shared context. This experience will feel like one platform even when your data lives on many. Shout-out to cross-platform, cross-project references enabled by the Apache Iceberg open table format! I’m skimming over this huge announcement, but supporting Iceberg indicates to me how dbt wants to empower people where they are, agnostic of what systems and data are in play.
The features dbt provides to enable data mesh are awesome, but the hardest part of taking a data mesh approach is always the governance. What domains own which business metrics? How should domains collaborate? When is it right to move data definitions to a shared integration project? How do we operate during a dreaded “data emergency?”
Some of these questions came up during discussion-based sessions at the conference, which were a welcome change from the sit-and-listen format. They’re hard questions — if you want the answers… I don’t have them for you, but I do know where to look! (Shout-out to the imposter syndrome talk).
Understanding the goals of each domain, their business processes, the information they need, and the information they produce are important facts to get on the table before forming opinions on hard governance questions and establishing processes to support self-serving domains.
dbt enables flexible, collaborative, trustworthy data culture
With dbt, you can start pushing the needle back towards the business domains driving value — enabling them with the flexibility to use data wherever it lives, collaborate across domains with a shared language around data, and build trust across the business that data is accurate, complete, and fresh.
Need help navigating the journey of changing your data culture? Learn more about our consulting services.