Saturday, October 4, 2008

The Debate Continues

In his September 2008 article for B-Eye Network, Bill Inmon commented on a study which found that applications storing BI data in Data Marts typically only have a life span of about 18 months. Bill wondered why this might be true and posited the following answers:

  • The traditional data model used for Data Marts (dimensional star schema) is "not conducive to change." Bill says that "the star schema is good for static requirements, not fluid requirements." So you build it for a specific purpose and then find out a year later that it doesn't meet your needs anymore.
  • The traditional tool used for Data Marts (OLAP) is not conducive to change. For slicing-and-dicing of data, the front-end tool and the back-end data model go hand in hand. If the data model no longer meets your needs, the GUI presentation will not either.
  • Business requirements are changing. If you have a inflexible data model, it is not going to meet your needs in the future.
  • Data Marts tend to be departmental solutions for a small pocket of users. Others in the organization may not know about these applications, which leads to their decline and disuse.

"There are probably are plenty of other reasons why data marts have such short lives. And interestingly, ALL these reasons are at play at the same time. It is not just one factor that causes a data mart to go into disuse. Instead, it is ALL of these factors working at the same time.

The net result of the fast expiration of data marts is that data marts start to accumulate in the corporation in large numbers. First, there are four or five data marts. Then, there are 50 or 60 of them. Then, there are hundreds of them."

Of course, this is just a continuation of Bill's long-running "Data Warehouse versus Data Mart" debate with Ralph Kimball. These guys don't draw quite the attention of other debaters such as Biden and Palin or Obama and McCain, but Inmon and Kimball have been going at it for years.

The back-and-forth is often entertaining. One of my favorite discourses was when Ralph said that a Data Warehouse was just a collection of Data Marts. Bill quipped back that you can collect all the minnows in the sea, put them together, and still not have a whale.

Inmon's concept is to create a very flexible data model of enterprise data available for any business intelligence need. Bill considers the data warehouse to have characteristics such as being integrated, serving all people, having a broad audience, and providing all data details in a relational form for flexible reporting. On the other hand, Kimbill's ideas are more practical, in that you create small collections of data for a specific business purpose and group of individuals. Bill pretty much agrees with the definition, saying that the data mart serves a smaller group of people and provides only summary data in a less flexible structure, such as the star schema. He just doesn't like that approach.

That is why Bill is quick to point out that with all the work you put into a BI application with a Data Mart, it is only going to last a year or so. Now multiple this times the number of different Data Marts you have within your organization and Bill thinks you will slap yourself on the forehead and say, "Wow, I coulda had a DW!"

No comments:

About Me

My photo

I am a project-based software consultant, specializing in automating transitions from legacy reporting applications into modern BI/Analytics to leverage Social, Cloud, Mobile, Big Data, Visualizations, and Predictive Analytics using Information Builders' WebFOCUS. Based on scores of successful engagements, I have assembled proven Best Practice methodologies, software tools, and templates.

I have been blessed to work with innovators from firms such as: Ford, FedEx, Procter & Gamble, Nationwide, The Wendy's Company, The Kroger Co., JPMorgan Chase, MasterCard, Bank of America Merrill Lynch, Siemens, American Express, and others.

I was educated at Valparaiso University and the University of Cincinnati, where I graduated summa cum laude. In 1990, I joined Information Builders and for over a dozen years served in regional pre- and post-sales technical leadership roles. Also, for several years I led the US technical services teams within Cincom Systems' ERP software product group and the Midwest custom software services arm of Xerox.

Since 2007, I have provided enterprise BI services such as: strategic advice; architecture, design, and software application development of intelligence systems (interactive dashboards and mobile); data warehousing; and automated modernization of legacy reporting. My experience with BI products include WebFOCUS (vendor certified expert), R, SAP Business Objects (WebI, Crystal Reports), Tableau, and others.