At my company, a team is setting up a new data vault 2.0 based data system.
When it was introduced, we were presented slides that showed it would retrieve data from different internal and external sources, consolidate that data, with Master Data Management that would re-inject the cleansed data into the source data system to increase data quality. Overall the system would become the basis for all our internal and external reporting needs, even with provisions for data scientists to experiment in. It would replace our Kimball data warehouse, as a Kimball based DWH is hard to maintain and adjust to new requirements. The new data system would be agile and adjust to any change in any data source or business requirement way faster than a Kimball DWH.
That's what the PowerPoint slides claimed.
At this moment the project is a year late. A team of seven senior programmers, all certified data vault 2.0, are working on it full time. And the project has only delivered a few minor things so far. Business is frustrated because all work on the Kimball data warehouses is set on hold. They were used to get the changes they asked for delivered in a few days.
As the person who build the Kimball data warehouses has left the company today, here's my question:
I witnessed one guy who created and maintained a few Kimball data warehouses, while also working on other projects, who was successful, fast and cheap.
On the other hand I see a team that has not been very successful so far, with slow progress and that's expensive. I hear they have recurring performance and missing/duplicate data issues.
What are your thoughts on setting up a general data system that can do the things the slides promised?
All architectures promise about the same things. What works in reality?