Common data in an enterprise

Greetings,

Company I work at folks love them some excel. We do a lot of stuff in excel. We also use a lot of 3rd party tools in house or cloud based.
Between the excel files, the 3rd party tools, each department spins up data that might also be used by other departments. Meaning dept1 has data for Company. dept2 also has data for Company. Mrs Old Timer also has companies she tracks on her excel spreadsheet. dept1 names Amazon as AMZ, dept2 spells it out Amazon Inc, (yes with a comma :disappointed_relieved::rage:) Mrs Old Timer uses Amazon
You see where I am going with this. So I would like to standardize this junk. Because now they want to compare data, do reports (in excel) etc
How would you describe this consolidation and standardization effort? what are the terms I need to use to read up on this data domain / domain data type of stuff?

Data governance?

Thanks much

Yes... it's called "Data Governance". If you search for the term, the wording will change from site to site that you may find but the basic idea is to make ensure that data has high levels of...

Availability
Usability
Integrity
Security

Basically, that's also the definition of what a DBA should do, as well, and I consider them to be the 4 basic laws for data. For DBAs, though, all of the items are (should be) equally important with no exceptions.

I'm not sure if the following is what you're looking for but I've been through such things over and over quite literally since the dawn of personal computers both as a young man and an "old timer" and thought I'd share my basic formula for success on changing people's "junk". :wink:

Heh... and the proper use of commas by "Mrs. Old Timer" should be something that you can easily accommodate so that she can spend more time doing the good job she's been doing all these years and less time on hating you for causing her to make unnecessary changes. :wink:

There are a couple of things that you have to remember if you want to make changes to fix their "junk". First and foremost, they don't think it's "junk" and it's actually not "junk". It's obviously been working for years or the company wouldn't exist. That fact and the fact that people are of the ilk that "We've never done it that way" and "We've always done it this way" are things that you have to be seriously empathetic about, especially with the old pros, like Mrs. Old Timer, who may actually know more about the company than you could imagine and is largely responsible for the success of the company.

You also have to remember that, although change is inevitable, it's frequently not for the better. This happens because someone thinks they know what's better and they jam stuff down people's throat that doesn't actually work better because they think that people like Mrs. Old Timer are outdated and useless even though precisely the opposite is true.

If you want to fix their "junk", and you should, then you need to actually spend a large amount of time learning exactly what and how and WHY people do things. You can make some early and nasty enemies if you go in with even a hint of an arrogant, condescending, or all knowing attitude. Rather, do a deep study on what their PROBLEMS are and find a way to fix those in a easy and helpful manner. You need people to be on your side or it will be a terribly arduous task that has very high potential of failure... especially when you later need THEIR help to do something right (and, trust me... you WILL need their help someday!).

Remember the 4 rules of data I posted above. The order of your customers' (the people doing the "junk") concern is from the top down with Availability and Usability being their top priorities and they take a whole lot of pride in doing their jobs right and so Integrity is a "given", as far as they're concerned. You MUST respect that or fail! Your order of concern is from the bottom of that list up with Security and Integrity being of equal importance. The truth is that if you can't solve the first two (Availability and Usability) in the eyes of the people actually doing the work, then all the rest of it is going to fail especially if you even imply that their methods somehow lack "Integrity" unless THEY'RE the ones that bring that subject up.

Make friends with the people on the floor and learn what they do, how they do it, and what their problems are and do it in such a fashion where they perceive that your efforts will make what they do and THEMSELVES more valuable. The Integrity and Security will come with all of that if YOU do your job right and have made yourself more valuable to the company, as well. :wink:

The bottom line is that Data Governance is incredibly important but is a huge task (especially when first attempts at implementation take place) and there's no way that you or even a whole department can do it correctly unless you've helped people like Mrs. Old Timer feel like she was a part of the change and that the change was actually valuable to her. If "DevOps" (a stupid term for something that should have always existed, IMHO) doesn't exist between IT and the people actually doing the work, attempts at Data Governance is going to fail in one way or another no matter what the plan is (and, yeah, you need to make a plan with multiple incremental milestones).

old timer here too :wink: thanks much for the great points I agree 100% with. hard to balance the people skill when company is hemorraging because of process issues

Heh... understood and appreciated. It all involves things like swamps and alligators. :wink:

The Microsoft-suggested solution is Master Data Services.

It looked pretty good to me (haven't used it, but reviewed an online course on MS VIrtual Academy).

1 Like

@andrewbb ,

I looked for it but didn't find a specific course on it by MS Virtual Academy. Do you have a link?

Heh... no wonder I couldn't find it. Thank you for the link.

1 Like

Actually, I don't know how I watched that either. I think they must have changed the title or something.

Thank you Sir Andrew (British Accent)

1 Like

Take a look at this post

1 Like

thanks Jotorre! So now how does this differ from home grown DW with DimCompanies. digging in further.

MDM and MDQS are really tools for your knowledge workers to engage, coordinate and cooperate with DBAs to establish and standardize across the enterprise under the direction of you management. Ms. Old Timer might be given ownership and control, along with other designated strategic business users, of the quality and consistency of your enterprise's data. The business users, report creators become the drivers rather than the roadblocks to accomplish the corporate goal of reliable data. Your DW ETL can then leverage this store of metadata to map data from various sources to the standards your company adopts.