"Modern Data Stack™ is dead" begs the question: was it ever alive?
- The hype around MDS obscured the fact that data stacks were simply moved to the cloud. IMO, this happened to every software vertical and it was the writing on the wall — and so it happened to analytics. Beyond this, there was nothing "modern" about the modern data stack.
- The incentives aligned for a fragmented data stack with multiple interoperable tools — until they didn't. As a side-effect of cloud optimization and layoffs, willingness build a data stack with tools from 8-12 (!!!) vendors has reduced a lot. Instead, companies are much more likely today to expect to buy 2-4 products as the core of their analytics infrastructure.
- Analytics products are always purchased from an analytics budget, MDS or not. And AI has (or will very quickly) make it possible to have a no-BS data stack that is a single or few tools that help drive results, not infra-for-infra's sake.
In my opinion, every software purchase decision has to pass the same checklist:
- Is it actually useful and does it justify it's cost?
- What are the hidden costs (people, servers, etc)
- Is it reliable? Is there a cost associated with making it reliable?
Most data stacks fail this basic litmus test on all accounts.
Businesses care about moving the bottom line, not whether your dashboard is powered by 10 cloud-native tools.
More so now than ever — It is clear that data is a growth center, not a cost sink. Analytics is critical, but expending time, energy and focus in maintaining large teams of data engineers and independent services is a massive undertaking with a lot of latent costs.