Over the past three years, I’ve worked with several clients that are reaching a pain threshold dealing with the uncontrollable growth of unstructured data. Why is this a painful topic? Decades of distributed systems have created islands and fiefdoms that hinder an organizations ability to create a master view of data. Why create a master view? Leverage, analytics and reporting, compliance, security, disposition management, storage growth, cost avoidance, on and on…

How did organizations get to this point? Distributed business applications, policies and management processes have created discrete islands of information most of which is unstructured (Documents, records, images etc…). To compound the problem, the business areas have general worked within their own vertical area with minimal collaboration between areas (islands) and the legacy business processes are a direct reflection of this. To realize the business efficiencies and cost reductions, roles, responsibilities and business processes must be resolved and integrated into a master view. Therefore some of the greatest challenges are not technical but instead organizational.

In David Loshins book Master Data Management, he talks about the how the decentralization of IT has created vertical silos and fiefdoms that impede horizontal collaboration. Why is this important? To create an integrated master view of data, the organizations business areas must cooperate. Specifically, the artificial fiefdoms overseen by individuals with political agendas are not incented to create a central master view since this would disrupt their island of power.

Loshin goes on to explain the important areas that must be addressed in order to create and implement a master view of data:

  • Organizational preparedness – Are you prepared for change and do you have a plan to minimize disruption? Does everyone understand the goals? How it will impact them? Will they cooperate? Who will twist arms when the games start?
  • Data Governance – Who is responsible? Who is accountable for data quality? What are the policies and procedures? How will they be enforced and their enforcement measured?
  • Meta data management – How will you establish a controlled vocabulary? How will you tag information? What is required to support search, workflows and technical integration?
  • Technology integration – How many document and records management systems do you need to integrate? Do you want to sunset a few? How will you integrate search?
  • Anticipating change – The journey will require organizational change which effects people, process and policy. How do you anticipate this and plan for it?

A great read, I recommend this book for those working in the field of MDM and its related areas.