Saturday, October 4, 2008

The Debate Continues

In his September 2008 article for B-Eye Network, Bill Inmon commented on a study which found that applications storing BI data in Data Marts typically only have a life span of about 18 months. Bill wondered why this might be true and posited the following answers:

  • The traditional data model used for Data Marts (dimensional star schema) is "not conducive to change." Bill says that "the star schema is good for static requirements, not fluid requirements." So you build it for a specific purpose and then find out a year later that it doesn't meet your needs anymore.
  • The traditional tool used for Data Marts (OLAP) is not conducive to change. For slicing-and-dicing of data, the front-end tool and the back-end data model go hand in hand. If the data model no longer meets your needs, the GUI presentation will not either.
  • Business requirements are changing. If you have a inflexible data model, it is not going to meet your needs in the future.
  • Data Marts tend to be departmental solutions for a small pocket of users. Others in the organization may not know about these applications, which leads to their decline and disuse.

"There are probably are plenty of other reasons why data marts have such short lives. And interestingly, ALL these reasons are at play at the same time. It is not just one factor that causes a data mart to go into disuse. Instead, it is ALL of these factors working at the same time.

The net result of the fast expiration of data marts is that data marts start to accumulate in the corporation in large numbers. First, there are four or five data marts. Then, there are 50 or 60 of them. Then, there are hundreds of them."

Of course, this is just a continuation of Bill's long-running "Data Warehouse versus Data Mart" debate with Ralph Kimball. These guys don't draw quite the attention of other debaters such as Biden and Palin or Obama and McCain, but Inmon and Kimball have been going at it for years.

The back-and-forth is often entertaining. One of my favorite discourses was when Ralph said that a Data Warehouse was just a collection of Data Marts. Bill quipped back that you can collect all the minnows in the sea, put them together, and still not have a whale.

Inmon's concept is to create a very flexible data model of enterprise data available for any business intelligence need. Bill considers the data warehouse to have characteristics such as being integrated, serving all people, having a broad audience, and providing all data details in a relational form for flexible reporting. On the other hand, Kimbill's ideas are more practical, in that you create small collections of data for a specific business purpose and group of individuals. Bill pretty much agrees with the definition, saying that the data mart serves a smaller group of people and provides only summary data in a less flexible structure, such as the star schema. He just doesn't like that approach.

That is why Bill is quick to point out that with all the work you put into a BI application with a Data Mart, it is only going to last a year or so. Now multiple this times the number of different Data Marts you have within your organization and Bill thinks you will slap yourself on the forehead and say, "Wow, I coulda had a DW!"

No comments:

About Me

My photo

I am a project-based consultant, helping data-intensive firms use agile methods and automation tools to replace legacy reporting and bring in modern BI/Analytics to leverage Social, Cloud, Mobile, Big Data, Visualizations, and Predictive Analytics. For several world-class vendors, I led services teams specializing in providing software implementation and custom application development. Based on scores of successful engagements, I have assembled proven methodologies and automated software tools.

During twenty years of technical consulting, I have been blessed to work with smart people from some of the world's most respected organizations, including: FedEx, Procter & Gamble, Nationwide, The Wendy's Company, The Kroger Co., JPMorgan Chase, MasterCard, Bank of America Merrill Lynch, Siemens, American Express, and others.

I was educated at Valparaiso University and the University of Cincinnati, graduating summa cum laude. In 1990, I joined Information Builders, the vendor of WebFOCUS BI and iWay enterprise integration products, and for over a dozen years served in branch leadership roles. For several years, I also led technical teams within Cincom Systems' ERP software product group and the custom software services arm of Xerox.

Since 2007, I have provided enterprise BI services such as: strategic advice; architecture, design, and software application development of intelligence systems (interactive dashboards and mobile); data warehousing; and automated modernization of legacy reporting.