Monday, December 22, 2008

Using IBM zLinux for Enterprise BI

Mainframe Executive Magazine recently wrote how the State of Oklahoma Department of Human Services (OKDHS), one of my clients, is moving legacy Business Intelligence applications to web-based versions on the IBM zLinux platform.

OKDHS lowered their Total Cost of Ownership (TCO) while delivering "seamlessly and transparently to users" a modernized system for child welfare. Their "KIDS" system now uses Information Builders' WebFOCUS product for enterprise BI.

Chris Little, the zLinux administrator, talks openly about OKDHS' decisions, challenges, and fears.

A big part of the project was achieving a comfort level. At the time we started our migration, Linux was still a dirty word in much of the IT industry. Linux was considered a ‘hobby’ or a ‘toy’ system then. We knew that the end result was to bring missioncritical applications to a new platform—and along the way, we had to feel comfortable by first performing the database migrations with databases that weren’t considered missioncritical. In this way, if an outage occurred and we had to move back to our original platform, we could do so without impacting a lot of people.

For more information, read the entire article at Mainframe Executive's website.

Thomas Communications started publishing "Mainframe Executive: IT Management in the Mainframe-Centric Enterprise" in March of 2008. For a subscription, click here.

Sunday, December 14, 2008

Reflections on Gate 24

It only took me a few minutes to walk the entire length of the Will Rogers World Airport in Oklahoma City, Oklahoma. The gate numbers went to 24, but I counted only 17 doors out to the planes. Perhaps half a dozen airlines were represented there, with each company allocated 1 to 3 gates. The flight choices were limited -- Atlanta, Cincinnati, Dallas, and so forth.

With little else to do while waiting for my Delta flight, I picked up a copy of the SkyMiles magazine and read an excerpt from the 2008 Inforum debate between Jimmy Wales, creator of Wikipedia, and Andrew Keen, author of "The Cult of the Amateur: how blogs, MySpace, YouTube, and the rest of today's user-generated media are destroying our economy, our culture, and our values."

I found one disagreement particularly interesting. Keen said his "biggest problem" with Wikipedia was that there is no "hierarchy of knowledge" -- no centralized editorial staff judging the quality or amount of content for each entry.

There is no one at the heart of Wikipedia saying that the entry on Pamela Anderson should be shorter or . . . longer than that on Hannah Arendt. There’s no one in Wikipedia who determines that the entry on Stephen Colbert’s concept of “truthiness,” which is really a footnote to an early-21st-century comedian, is longer or shorter than the entry on truth—the core concept in the Western canon, the core concept in the history of Western philosophy. So I may seem old-fashioned, but I believe in knowledge hierarchies. There has to be someone who makes a call on whether Pamela Anderson is more or less important than Marie Curie or Hannah Arendt. . . . Ultimately, for those people who go to Wikipedia who have no media literacy, who lack the skepticism, the education which we all have, I am fearful that in this open-source, free-knowledge universe we are going to be educating people who are not able to evaluate—not so much the accuracy of information, but the importance of information.

Wales thought Keen's statement "strange" and guessed it came from an archaic viewpoint of space limitations associated with physical books. Wales responded:
[Wikipedia is] not like an encyclopedia where we say, look, we’ve got thirty volumes and so we have to cut, we have to limit somewhere."

Is Keen still living in the Gutenberg world while Wales has found his way into the Google world? It seemed they could not communicate because of different worldviews.

With the physical limitations of the Gutenberg world, somebody had to make value decisions -- you just couldn't print everything because you ran out of room. In the new Google world, however, you basically have unlimited space to keep anything and everything, regardless of quality or value.

Will Rogers World Airport obviously has physical limits. There are only 17 gates, so you can't have thousands of airplanes waiting outside. Inside the terminal, you have a few food options; there is just room enough for one fancy bar, Sonic (headquartered in OKC so it gets a special spot), Schlotzsky's, and TJ Cinnamons. At each vendor, you see limits -- Schlotzsky's is only going to make certain types of sandwiches at the airport.

Of course, Keen is a smart guy. He knows that earlier physical limitations on printed material do not apply to data stored out in the virtual "cloud." Keen is probably not bothered by relative lengths per se. Instead, he aspires to a high standard of somehow organizing knowledge according to its value.

But the challenge there is that some human being would make value judgments for the rest of society.

I ate at Schlotzsky's and their original sandwich was fine. Somebody decided that this airport site could not stock the ingredients to make the Angus Pastrami Rueben, so I was out of luck. But no problem, I can pick up a Rueben back in Cincinnati.

But consider the issue of our stored knowledge -- what committee would have omniscient abilities to judge what we need to know and remember? I would be concerned about giving somebody the power to say, "I decree that information about Pamela Sue Anderson will not become part of human knowledge -- strike her from the record! Next on the list... some guy named Hasselhoff."

Archaeologists base some important understandings of past civilization on what they found while digging through the "middens" -- the garbage piles. They learn a great deal about human activities and behavior by analyzing what people discarded -- things which at the time did not seem important enough to keep. Perhaps even if we decide to discard knowledge, we still need to throw it somewhere for safe-keeping.

In business intelligence, we often make value judgments on what to store for later decision making. In this new Google world, physical storage limits are going away. Our ability to handle large amounts of data is improving. We should consider saving everything (after all, one person's trash…).

Sunday, December 7, 2008

Free BI for Higher Ed

For over five years, Microstrategy has been part of an educational service from Teradata allowing universities free access to an online learning portal focused on business intelligence and data warehousing technology. In November of 2008, Microstrategy expanded this offering and announced a new program to give BI software and courseware, free of charge, for use in university curriculum.

MicroStrategy has provided its software for teaching BI courses to numerous universities in the U.S. and Europe, including University of Colorado Denver Business School, RWTH Aachen, University of Bamberg, and HTWK Leipzig.

Universities can build or augment their business intelligence course offerings with MicroStrategy and teach students about BI in a tangible way, providing hands-on experience that students can take directly into the workplace. By incorporating MicroStrategy software into a BI curriculum, students can learn valuable skills, such as designing business intelligence reports, performing data analyses, and monitoring business metrics.

Barbara Wixom, Associate Professor and Director MSMIT at the University of Virginia commented, “Companies want to hire students who can hit the ground running – they want students with an understanding of both foundational skills and contemporary tools that exist in the workplace. Having the ability to teach my students business intelligence using software like MicroStrategy is incredible. It means students experience business intelligence in a relevant, real-world way. And, it means that my students can be productive as soon as their careers begin.”

Microstrategy is making a smart move. Companies that have already implemented a BI product often struggle to find knowledgeable resources; being able to employ college students with formal BI training reduces the hiring challenge. That is good for Microstrategy as well - they are planting seeds in organizations that may not have their products yet. Plus, it is difficult to sell products that nobody knows how to use.

An Indiana company selected WebFOCUS as their BI product. However, before buying and implementing it, they wanted to hire a BI developer. Unfortunately for Information Builders, the software vendor, the would-be customer searched unsuccessfully for months trying to find a local professional with the right skills.

The two parties found themselves in an old-fashioned barter stand-off: "I'd give you that wagon wheel if you had a cow to trade fer it." Except it was: "I'll give you money for your BI software if you can introduce me to an experienced WebFOCUS developer." Yikes -- there were no available BI Hoosiers to be found.

Eventually, the organization settled on a quality person experienced with a different web-based BI tool and invested time and money training him in WebFOCUS. While Information Builders was able to make the sale, they should have also recognized the red flag being waved in their face. It is worth repeating -- it is difficult to sell products that nobody knows how to use.

If you are attending higher-education courses, ask your university if they offer business intelligence topics -- important skills needed by today's information-intensive firms. If you work for a software vendor, make sure that you are developing talent pools for your client base (sorry, offering $3K-per-day vendor consultants does not count). If you are a customer in the market for BI products, make sure one of your selection criteria is being able to easily hire technical resources.

Tuesday, November 25, 2008

Baconators, Batteries, and BI

Many organizations have a "Just the Facts, Ma'am" attitude, which misses the point of BI. Having the world's largest data warehouse on display in the Guinness book of records provides far less value than simply making good use of the collected information.

Evan, my teenage son, recently started driving my old Honda. He kept telling me about maintenance issues: the air in the front tire is low, there is a funny noise when starting the car in the morning, the Valvoline sticker says the car is 1000 miles past an oil change. When he brought me these facts, I would just respond with, "Sure, we'll have to do something about that."

The problem is, my son didn't and I didn't. We had the facts; we just never made thoughtful decisions that might have led to proactive tasks. Instead, we were ultimately forced to respond to crises.

Evan called me one evening from the nearby Wendy's, "Hey, I stopped to eat a Baconator and now the car won't start." The battery was dead. A week later another call came to me, "Hey, I was driving and something happened to the tire." It was flat.

Business Intelligence is not about just gathering facts. Even with all the facts, bad things can still happen to you. If you want real value from BI, you have to think about the facts and take the appropriate action.

At our local DAMA event, Bill Inmon spoke about his clients' gigantic data warehouses. Some amassed hundreds of thousands of terabytes -- call records for a telecommunications giant; cash register receipts for a large retailer. Bill estimated the infrastructure cost of managing each terabyte at somewhere between $750,000 and $1 billion USD.

Bill then said something strange: most companies use only a small fraction of what they put into the data warehouse. At first, it sounded like the creator had indicted his invention. Since Bill is unlikely to do that, he was evidently expressing the sad fact that most companies do not effectively use what they are paying so dearly to capture.

You must have the facts, but if you forgo decisions and never act, you may end up with the corporate equivalent of a dead battery and flat tire.

Friday, November 14, 2008

Power of the Stack

This week, I saw the power of the stack. Talking to local business people, I learned their business intelligence (BI) software decisions were influenced by earlier choices of stacks.

By "stack," I mean that a single mega-vendor provides an integrated product group that meets all or most of a customer's technology needs. If you went with the IBM stack, for example, you would have various hardware and operating system choices, the DB2 database, Cognos BI, WebSphere for the web tier components, and so forth. If you wanted the Microsoft stack, you would get the Windows operating system, SQL Server database, BI in the form of Reporting and Analytic Services, Internet Information Services for web, etc.

If your organization built custom applications, you might choose your stack based on the underlying operating system or databases -- IBM, Microsoft, or Oracle. If you implemented a packaged solution, however, you would probably go with that ERP vendor's stack -- SAP or Oracle (more of a heap of applications than a stack: Oracle, Peoplesoft, JDEdwards, Hyperion, Siebel).

Automobile manufacturers have always offered these "mega-vendor" choices. Buy an Accord and Honda selects the components for you. While you can configure some options, you do not worry about the brand of radio, carburetor, cruise control, or security system. Your real decision is to decide upon a specific vendor -- Honda, Toyota, Ford, Chevy, Porsche, and so forth -- and then pick their specific product that best meets your needs -- Accord, Civic, Element, Pilot, etc.

One of my enlightening conversations this week was with a company using the Oracle ERP and database products. When ready to start a new BI initiative, their strategic decision was easy: use whatever BI tools are provided by Oracle. The next day, I met with an organization ready to swap out their BI product. They decided to go with a purely Microsoft platform, which meant their existing "best-of-breed" reporting tool was no longer welcome.

These examples show an advantage to having a standard, single technology stack. When you want a BI product, you no longer research all of the available products, process RFI/RFP documents, watch vendor presentations and demos, do a competitive bake-off, work with the top vendor on a proof-of-concept, and then put together legal agreements. Instead, your incumbent mega-vendor just tells you which BI product you will use.

It also points out that making your initial stack decision is important. Buying the wrong car is a relatively painless mistake; deal with it for a few years and then replace it. Implement the wrong enterprise technology, however, and you might experience pain. In fact, if you picked the stack, you might be what is traded in for something different.

While some companies may still decide to use "best-of-breed" BI products that fall outside of the stack, most will go with mega-vendor choices -- IBM, Microsoft, Oracle, or SAP.

Monday, November 10, 2008

No 'No Wake Zone' in BI

The mega BI vendors are getting bigger while the little guys are struggling. That's the picture painted by recent 2008 earnings reports. IBM, Microsoft, and Oracle all grew in their third-quarter revenue, while MicroStrategy and Actuate suffered declines.

IBM takes the award for top BI growth. Partially due to its acquisition of Cognos, IBM's total software revenue grew 11.8% in Q3 2008 over the same period the previous year, with their Information Management segment expanding 26% to $2.849 billion. Information Management grew faster than any of its IBM peer software products -- WebSphere, Lotus, Tivoli, and Rational.

Microsoft's total quarterly revenue improved to $15.061 billion; a 9.44% increase from Q3 2007 revenue of $13.762 billion. Isolating Microsoft's revenue to just server and business division products, we see their BI-related software grew over 19% ($8.352 billion, up from $7.016 billion).

Likewise, big vendor Oracle announced quarterly earnings of $5.331 billion, up 17.71% from the previous year. Their new license revenues increased 13.8% to $1.237 billion.

Earlier in the year, the $2 billion privately-held SAS claimed a 16.6% increase in analytics software revenue over the previous year.

The wake of these giant ocean-liners may be causing the small BI dinghies to take on water and could threaten to capsize them.

MicroStrategy was hit with declining total revenues, bringing in quarterly revenues of only $33.7 million. The previous year was higher with $34.7 million. MicroStrategy's sales force struggled to sell products to new customers; new license revenue dropped 18% to $24.8 million from the previous year's $30.2 million.

Harder hit was Actuate, the smallest commercial vendor in the BI space. Actuate's quarterly revenue fell by 2.88% to $33.7 million. New license revenues for the quarter plummeted 25.37% to only $10 million.

For years, privately-held Information Builders has struggled to break past $75 million each quarter, with new-name customers probably providing for about 60% of that figure. The big vendors consolidating the BI market are wrecking havoc on that small BI vendor as well.

Actuate, MicroStrategy, and Information Builders are all facing challenges trying to sell their products to new customers. When buying BI software, customers are choosing those mega vendors most likely to stay afloat for the long term.

(All figures are in U.S. dollars)

Wednesday, November 5, 2008

Software Worldviews

After reading articles and listening to experts speak on spreadsheets, it has slowly occurred to me that these individuals have an underlying worldview of software management that shapes their perspectives and subsequently their opinions, much like how people's mental concepts of a transcendent divine power or a total absence of one determines how they see the world.

Take for example, a November 2008 article written by John Myers, the founder of the Blue Buffalo Group, for the B-Eye Network. In his article "Legalizing the Spread(marts) of Business Intelligence," John compares Microsoft Excel users to marijuana smokers (all meant in good humor, I would assume). Despite those in IT who would like to regulate this "soft drug," users continue to covertly utilize spreadsheets for either recreational or medicinal purposes. John's underlying assumptions about the IT department are fairly obvious:
"For many years, inflexible data governance organizations and IT departments have put a stranglehold on the development, distribution and consumption of analytical business intelligence applications – or at least they thought that they did. These analytical applications leaked out of IT’s 'walled garden' in an almost intoxicating fashion. "

The worldview being expressed here is that of the IT group consisting of elite professionals making regulations and putting restrictions on the common folk who live on the outside of the restricted-access temple. While these smart individuals establish IT laws to prohibit activities dangerous to the organization as a whole, the hoodlums on the street ignore the rules and just duck into a dark alley where they can light up and avoid being caught.

At a recent conference, Howard Dresner recommended to the IT folk that they "stop spreadsheet applications" and limit the tool to only appropriate usage. Howard said that BI needs to be owned by the corporation, run by the CFO, and used to create a "performance-based culture." This would imply that Howard's Priestly worldview of software management is similar to that expressed by John Myers. BI is all about important financial decisions, and those are best made at the top of the organizational pyramid.

On the same day, Boris Evelson of Forrester Research said that spreadsheets play an integral role in business intelligence and that end users need to have a private BI workspace for analysis and what-if simulations. Boris tried to comfort his IT audience with, "you fought a good war and lost," as they have been unable to eliminate user applications of Microsoft Excel and Access databases. In a different session, Boris stated his opinion that "IT can't own BI."

Unlike Howard, Boris seems to espouse the Libertarian worldview of software. The company consists of business users who need to have BI tools in their hands to do their jobs. Some of these individuals are Power Users who can handle BI application development tasks just like the best of the IT professionals.

These two differing perspectives may derive from the idea of the complexity of the BI work and products. One camp has decided that BI is too difficult for the general users and needs to be performed for them by a trained group of professionals. The other camp feels that technology should be accessible to the common person and not controlled and regulated (they admit deficiencies in the products' usability that makes achieving this goal difficult).

If you want to discern where somebody really stands on the issue of software management, just ask them how they feel about spreadsheets.

Wednesday, October 29, 2008

Leave a Legacy (follow-up)

Earlier this month, I did a blog on Leaving a Legacy (BI product, that is). In particular, I talked about how many large companies still have old 4GL application development tools such as FOCUS, NOMAD, and RAMIS. Not only are there hundreds of firms with these 1970s- and 1980s-era 4GLs installed, most are probably still paying annual maintenance fees of 15% to 20% of the purchase price.

Following that post, I received inquiries into how difficult it really is to convert these products. Management within these organizations are demanding, "Just replace it -- it's only a reporting tool!" but their IT groups are struggling to comply with the mandate.

Many make the decision to convert to a data warehousing ad-hoc tool such as Business Objects, moving the data off the mainframe. That creates its own issues, but there are two main reasons why this does not work well. First, the 4GL is a full application development tool while Business Objects is an ad-hoc report writer. These are apples and oranges; the replacement does not have all the features of the original product and it is not the paradigm that the end-user expects.

Another reason is that the 4GL provides real-time access to mainframe data. A data warehouse tool does not. It only allows the user to see copied data; perhaps not up-to-date, maybe not to the level of detail they need, and often without all of the needed columns.

A better solution might be to keep things where they are and convert to a new mainframe BI technology such as WebFOCUS or the zLinux Cognos. At one mainframe client, we automated the 4GL conversion and dramatically reduced the time, cost, risk, and skill-set requirements by baking translation logic into a software utility. We also reduced the amount of effort by keeping all the back-end data the same by staying on the big box. To help with support issues, we switched to a new Unix System Services operating system.

Even a conversion utility is not a magic wand, of course, and your enterprise BI conversion initiative will still be difficult. If somebody is telling you otherwise, you want to be careful moving forward with their advice.

Thursday, October 23, 2008

Be Careful What You Blog

Just last week, I did a seemingly innocuous blog posting on hot SAP skills. SAP has been doing creative things to find more people to fill tens of thousands of SAP openings.

Well, SAP's creativity reached out and bit me.

While I was happily typing away in my blog, my director of marketing and sales was on the phone talking with SAP about a open position leading their North American adult learning programs. Unfortunately for me, this is a match made in heaven for him and SAP. Based on his background, he is a perfect candidate for the job.

I'm happy and wish him the best of luck training individuals to work in these important SAP positions. Perhaps I should do a blog on the hot openings within my own organization?

Wednesday, October 22, 2008

Cincinnati Fast 55!

I am happy to announce that the Business Courier of Cincinnati recently named my organization one of the 55 fastest growing privately-held businesses in the local area.

In just two years, we have grown to $6 million in annual revenue and have assembled an excellent staff of over a dozen individuals working in the corporate office and 40 consultants in the field. We have offices in both Cincinnati and Columbus as well as a consulting presence in Indianapolis. I want to thank everybody who has been involved in this success.

For more information on the honors, click on the picture to read the news article.

Business Courier of Cincinnati names PPS Finalist in their Fast 55!

Software Vendors, Repent!

John Thompson, the CEO of Symantec, is warning other leaders of software companies that they need to rethink their business models. He admits that it is not an immediate problem, but he sees something on the horizon for the legacy software industry.

In an interview with Financial Mail for their October 24th issue, John talked about the emerging trend of "software as a service" (or Saas) as opposed to the traditional way of selling shrink-wrapped products. Here is an excerpt from that article:

"[John] believes software companies don't need to start panicking - yet. In the long term, though, their profit margins will be eroded as the industry moves to a service model, he says. 'Software businesses have gross margins in the range of 80%-90%, while services businesses are in the range of 60%-65%. A well-run software company will have pretax operating margins of 25%-35% and a services business of about 15%. It's a very different model.'

Thompson says software companies have to reinvent themselves as services businesses, which will be difficult for many. But if they don't do it, they could find their future in doubt.

'Some company, without a legacy-installed base or a legacy revenue stream, is able to run up and take your business away from you,' Thompson says.

But the shift will take longer than many in the industry assume, he says. This will give the big software players time to adapt.

'The revenues of Saas companies will not overtake the revenues of [traditional] software [companies] for a long, long time," Thompson says. "It's not a big threat to the profitability of software companies in the near term.'

He says the annual turnover of the world's largest dedicated Saas vendor,, is dwarfed by Microsoft's yearly revenue. 'If [ CEO] Marc Benioff were sitting here, he'd say software's dead. That's just crap. Marc's a good friend of mine, but Marc is just promoting what he's doing.'"

Back in the days before companies could afford their own mainframe computers, people would connect up to somebody else's system and pay for the time they used it. Software vendors would install their applications there and let different customers access it on the shared system.

The 1970s relationship between Tymshare and Information Builders is a good example of how BI software was offered as a service at an early point in computing history. When platforms became affordable, vendors began selling products to each company individually.

John is right that software pricing models will definitely need to change. For example, Information Builders still prices their BI product based on the size of the hosting computer. Never mind that the development costs are the same for Windows, UNIX, and System z and that they only have a single code base now anyway. If you want to run their BI on your Windows box, the price tag is about $40,000, but tell them you want to run it on your mainframe and their number goes up to probably around $1 million.

Their pricing model is based on the assumption that the bigger your computer, the more business value you will get, so therefore you should pay more. They offer you a perpetual license, so you pay them 20% of a purchase price every year for annual maintenance ($8,000 for Windows and $200,000 for the mainframe) which allows you to always have the latest release. If you get a bigger machine after the BI software is installed, you are going to be found out and have to pay the infamous "upgrade fee."

Pure software vendors will pay attention to certain financial ratios, one of which is "revenue per headcount." They want to divide their annual revenue by the number of employees and come up with a ratio of about $250,000, but the higher the better. To improve their score, they only have two choices: either increase revenue or reduce the number of employees. If they have challenges on the revenue side, they will get rid of unnecessary employees (some go so far as to convert employees to partners or subcontractors so these individuals no longer impact the official ratio).

Software consulting firms are going to have much lower revenue per headcount ratios as they are limited by the number of hours in the year that any given employee can bill for services. Unlike the software vendors, the services organizations might be happy with a ratio of $150,000 to $180,000. Because Saas companies are not necessarily pure services businesses, they may be able to achieve a high ratio here, although they may have troubles attaining the high figures of today's pure software businesses.

For the sake of their customers and their own employees, hopefully legacy software vendors will pay attention to John calling out in the desert, "Repent!"

Wednesday, October 15, 2008

Human Decisions

In an earlier blog, I discussed how Howard Dresner says that his term "Business Intelligence" really only applies to those software applications that help humans make decisions.

Timo Elliott (Employee #8 at Business Objects) blogged on a similar topic back in 2007 with his "Will Computers Ever Help With Decisions?" article (I hate to give away the ending, but Timo answered his own question with "No"). It's an interesting blog about why people are still unhappy with their computer appplications, so be sure to read it.

Timo gives us a definition of "decision" itself:

"A decision is a situation where information is lacking by definition. Many executives define their jobs as "making decisions" -- i.e. tasks that can't be automated. As computers take on the lower-level tasks, they're free to move on to more complicated choices."

Like Howard, Timo points out a significant dividing line between applications that automate tasks and those that present information to humans for a final decision. It the computer has all of the information necessary to reach a conclusion, then the result cannot be considered a real "decision." Instead, it is an automated task.

Timo concludes his blog with:

"But to decide (and err) is human. Recognizing this and setting expectations appropriately can help smooth the relationship between the people that consume information and the groups that provide it."

Notice our natural biases both for and against the human race. We think that only humans can make decisions; computers are beneath us in that capacity. However, we're quick to admit that humans make mistakes, unlike computers which are inerrant (except for when software applications make the mistakes we accidentally tell them to make -- too bad computers cannot decide to not follow our instructions!).

Monday, October 13, 2008

Hot Jobs

Back in the late 1990s when Y2K was looming, people with SAP skills were in high demand. A decade later, SAP skills are still hot although with a slight twist. In the September 29th issue of InformationWeek, Marianne Kolbasuk McGee wrote a NewsFilter article on the topic:

"A scarcity of experienced talent combined with the growing popularity of SAP's products has pumped up pay considerably for those with SAP expertise, according to a report by research firm Foote Partners. Despite the weak U.S. economy, premium pay for a dozen SAP-related skills rose by as much as 30% over the past six months, and in some cases up to 57% in the last 12 months. The study evaluated the pay of 22,000 IT pros in the United States and Canada.

SAP is aware of this increasing demand and is moving to get more talent into the pipeline through alliances with universities, lest it lose potential customers to other ERP platforms, like Oracle, amid fears of a labor crunch."

In May of this year, Marianne had written that SAP was working hard to resolve this issue:

"SAP officials estimate there's a current shortage of 30,000 to 40,000 experts needed to meet the global demand for implementations and support of SAP software. And if you talk to employers looking to hire SAP talent, they'll tell you that finding these people is no easy mission."

While that much demand sounds like a great problem for a software vendor to have, Joe Westhuizen, an executive for SAP, says that "very quickly success can also become a noose around one's neck." Joe reports that the situation has actually improved, as just a year ago there were unfilled openings for over 50,000 SAP specialists worldwide. SAP is trying a variety of ways to encourage more people to learn SAP, including quadrupling the number of universities offering courses in the product.

Many of the hot SAP skills in the 1990s were related to technical implementations, while today's in-demand specialists are more on the functional, business side. Companies need people with interpersonal skills who can help work with and explain the SAP functionality. This is a trend not only for SAP professionals, but technologists in general. In an August article, Stacy Collett wrote:

"The most sought-after corporate IT workers in 2010 may be those with no deep-seated technical skills at all. The nuts-and-bolts programming and easy-to-document support jobs will have all gone to third-party providers in the U.S. or abroad. Instead, IT departments will be populated with 'versatilists' -- those with a technology background who also know the business sector inside and out, can architect and carry out IT plans that will add business value, and can cultivate relationships both inside and outside the company."

You can have a hot job working in the software industry or in the kitchen of a McDonald's (I've done both). Your wage for doing either of these jobs will follow the same simple principle: You will be paid based on how difficult it is for the employer to replace you.

If the quick service restaurant can train somebody new in an hour to do your job flipping burgers, then you will always only make minimum wage. However, if McDonald's has to train somebody for months to come up to speed on their SAP environment, then they will pay you a premium for that expertise.

Sunday, October 12, 2008

Leave a Legacy

Despite living in the 21st century, companies today still use a variety of legacy reporting tools created in the 1970s and 1980s. Early software vendors created products such as the 4GLs (FOCUS, NOMAD, and RAMIS), QMF, and DYL280 for use on specific host computers, typically a mainframe platform such as MVS/TSO or VM/CMS. They of course designed these tools according to and limited by the capabilities of that decade's technology.

Companies had to train and support reporting end users to log into the mainframe environment and use the command-line processes and menu screens intended for data processing professionals.

Mainframe tools naturally lacked graphical user interfaces (GUIs) and instead used character-based “green screens”. Because the mainframe environment did not communicate with end-user tools such as Microsoft Office, any integration with spreadsheets, documents, or presentation slides was accomplished manually. Likewise, only that data on the mainframe platform could be accessed (e.g., typical data sources include sequential files, tape cartridges, VSAM, IMS, DB2, and others).

The new web-based BI products, on the other hand, can be architected as multi-tier environments, separating the user experience from the backend data sources. Just because the data and access mechanisms are on a mainframe does not mean that the user must use a dumb, green-screen terminal. Instead, the user can interact with the BI engine via a web tier and a simple GUI web browser. The BI engine can automatically pass the formatted reports back to the user in the desired output formats, such as HTML, Microsoft Word and Excel, PDF, and others.

The complexity of using a mainframe operating system is now hidden from the end user and performed automatically by the BI system. In addition to accessing the mainframe legacy data, the BI engine is now typically able to reach a wide range of enterprise data on multiple platforms. Relational database management systems (e.g., Oracle, UDB, Sybase, Informix, and MySQL), multidimensional cubes (e.g., Essbase), and other non-legacy sources can now be used in BI requests. In addition to data, modern BI engines are also able to communicate with enterprise packages (e.g., PeopleSoft, SAP, and Oracle) to easily obtain application data.

With the proper web security in place, enterprise data can now be shared easily with clients, suppliers, business partners, and others who were outside of the mainframe environment. Reporting tools are now online at a corporate website for the proper individuals, enabling them to perform business intelligence as easy as ordering a book from Amazon.

In 2008, it has become increasingly hard for a firm to support a reporting tool that was installed before 1988. One reason is the dearth of knowledgeable individuals in old BI technologies such as FOCUS, RAMIS, and NOMAD. Take COBOL as an example of a technology that is still in widespread usage in businesses today. Despite the need, few colleges teach COBOL to their students. If universities no longer teach a critical business programming language, how can we expect them to instruct on an obscure 4GL like RAMIS?

If a company still uses NOMAD to run mission-critical human resource and payroll functions on the mainframe, will they really be able to hire somebody to replace Harold, the life-long 4GL programmer, when he retires next year? This is a significant business problem for companies using legacy reporting tools.

Some products have been traded among vendors like bent Topps baseball cards of little-known right fielders. NOMAD and SQR are good examples. RAMIS and others were acquired by Computer Associates. The original creator of the FOCUS 4GL still owns the technology, but they have effectively replaced it with a web version. The legacy products may get a frequent white-washing and some new features painted on occasionally, but they still fail to provide today's organizations with current BI functionality.

You may already know this dirty little secret: the vendors of legacy BI tools do not really sell these products anymore. Nobody today buys a reporting language whose integrated development environment consists solely of a text editor. Instead, these vendors hope their legacy customers will continue to hold onto these cash cows so they can be milked for annual license fees, which is typically 15% to 20% of the purchase price.

I worked with a Fortune 50 financial services firm that had installed NOMAD in the late 1980s. At one point this company had over 500 4GL users, but after 20 years the usage had slowly declined to about 100 people. Few of these individuals were the original developers; they had merely inherited programs when earlier workers had left. Few people knew the actual workings of these important business processes. Few were excited about having to do this support work.

Manually converting these applications posed a major obstacle to this organization. In order to justify moving to newer BI technology, we had to reduce the time, cost, and skill requirements by building an automated translation process.

Another Fortune 50 client has over 800 business users supporting FOCUS applications and they have struggled for a decade trying to manually replace this legacy tool with a Business Objects product. A large Fortune 500 insurance company has a similar situation with over 500 FOCUS users. Decisions come down from on high saying, "It's just a reporting tool. Throw it out and replace it with Business Objects." Sorry, it's not that easy.

I worked with an IT group who had been given the "just replace it" directive. Unbeknowst to them, savvy users in remote manufacturing locations had used the 4GL to build complex, mission-critical processes to schedule their shop floor operations. Oops, who saw that coming? After all, it was just an end-user reporting tool. End users shouldn't do that kind of thing.

Over a 20-year period, legacy end-user reporting tools like FOCUS spread through organizations like a unchecked virus. Early MIS departments nonchalantly turned over reporting responsibilities to business developers. Management did not pay attention to how widespread the usage was. Now, decision makers find themselves ill and seeking powerful legacy antibiotics, as it could cost them millions of dollars to treat their disease and manually replace user-written applications with new products in order to get to a healthy state where they can meet today's challenging business demands.

If your company still depends on a legacy reporting tool, you have valid reasons to be worried. We are trying to cope with global competition, rapidly changing technologies, and shaky financial situations. We are in a consolidating BI market that contains leader products, emerging new technology, and declining older tools. It is definitely time to leave your legacy.

Friday, October 10, 2008

Actuate's Pay-to-Play Deal

Actuate Corporation recently announced a partnership opportunity for other software vendors to embed the Actuate BI product into their applications and resell it as their own.

OEM deals can be great for the original vendor. Just look at Oracle, whose relational database management system was embedded with large applications in the 1980s and 1990s. If vendors could get success like Oracle's through these partnerships, you would think that they would be begging other companies to resell their products.

With that in mind, consider the following details of Actuate's OEM partnership offer:

"'Actuate’s new OEM Quick Start Program provides our ISV partners with the software and services they need to enhance their applications and drive new sources of revenue,' said Nobby Akiha, senior vice president of Marketing at Actuate. 'With our Rapid-Time-to-Market methodology for addressing our partners’
development, marketing and sales challenges, they can reap the full benefits of Actuate-based applications and pursue the rewards of the increasingly heated Rich Internet Applications market.'

Additionally, the OEM Quick Start Program provides superior design environments for creating new products with increased performance, reduced development time, improved end-user experience and increased profit margins. Actuate partners also receive support and development platforms, providing a fast, effective means to deploy Actuate enhanced applications to customer communities of any size and add capacity easily and cost-effectively as their needs grow.

The OEM Quick Start Program provides participating partners with a kit that includes the following:

  • Development licenses for Actuate BIRT-based environments for up to five users on a single server

  • One online training course for one student

  • One day of application installation and setup support

  • Four days of consulting with an Actuate Professional Services expert (does not include travel and associated expenses)

  • High-quality and responsive support via phone and email
    throughout the development and prototyping process

  • Significant discounts on deployment software when the Actuate-based solution is ready to go to market

The Actuate Quick Start OEM Program is immediately available and priced at $35,000."

For 35 grand (plus expenses, of course), you can get copies of open-source BIRT software, a single person trained for one day, one day of installation assistance, and four days with a high-paid advisor flown in from a far-away locale. When you are ready to sell your application to customers, Actuate will give you some discounts, because you evidently have to pay for even more software licenses at that time.

That's a hefty chunk of change to get the rights to resell Actuate. If application vendors have to incur this type of cost during the upfront development effort and then pay for additional licenses each time they sell their product, how can they be profitable? Well, they try to pass along the cost to the customer and inflate their asking price. But I wonder how long customers will continue to be willing to pay high costs for software applications, especially those made with open-source BI tools.

Back at the end of September, Christopher Dawson posted a ZDNet blog entry on how he looked into buying SAS to do statistical analysis for his local school district in Massachusetts. Christopher choked on SAS's quote of $5,000.

Having worked for software vendors most of my career, I can pretty much guess that the SAS sales rep thought he was being extremely nice to Christopher and basically giving away the product for an unrealistically low dollar amount. But no, Christopher thought that was too expensive and decided instead to try to use the open-source R statistics package.

Of course, Christopher is with a small, non-profit educational organization. His story may not represent a real trend for purchasing BI software. Perhaps the mega-companies are still willing to spend big bucks for BI software products.

What's this? Today, GM is selling for under $5 per share? Ford Motor is selling for $2 per share? P&G's value just lost $10 per share? Xerox just lost half its market value? Now, that is the trend making the news. That will change the way BI software is purchased.

Maybe Actuate's offer is more of an "Empty Your Pockets" deal, where they are looking for cash. Hopefully, those vendors who want to make an investment to play with Actuate will still be able to cover the $35,000 expense. Even more, I hope they will still have customers able to buy their finished application.

BI Consultants

Occasionally, I speak with individuals who think that using outside consultants for Business Intelligence projects is a luxury and only feasible for those large organizations with substantial budgets. On the contrary, consultants often provide the most cost-effective way to achieve your BI goals.

One reason is that BI consultants are specialists with solid training and expertise in BI products. They have probably dedicated years of their careers to specific products, such as Cognos, Business Objects, WebFOCUS, SAS, Microstrategy, or Actuate. They hit the ground running and accomplish your goals efficiently.

In addition to experience and skills, an outside consultant has the time to address your BI needs. While your staff must handle day-to-day operations and put off strategic initiatives, an external consultant can focus strictly on the work at hand.

Consultants are a flexible resource for your projects; they are temporary, come in only when you need them, and leave when they are done. They can also provide your organization with a fresh, objective point-of-view. Because the typical consultant has worked with a variety of clients, he or she has seen aspects of BI with which your staff may be unfamiliar. While onsite, the consultant can transfer some of this knowledge to your associates, leaving you with a better team.

Value-added features such as these make BI consultants effective and efficient. Having performed this type of work before and being dedicated to your assignment, consultants can achieve results more quickly than your internal resources.

At my consulting organization, we have developed software utilities to automate much of the manual effort for BI activities (e.g., automatically translating legacy programs and generating the modern BI code). We put the know-how of doing the BI work into an application, which means less time, risk, and cost.

Armed with knowledge, expertise, and tools, outside BI consultants may actually provide a lower-cost option than doing the work yourself.

Thursday, October 9, 2008

Psst: SAP buying Teradata. Pass it on...

The New York Times posted some speculation today that German megavendor SAP might be considering yet another BI company to go along with its recent purchase of Business Objects.

See Ashlee Vance's blog in the October 9th BITS section (Business, Innovation, Technology, and Society). Now somebody is really climbing the laddder of inference here if the only fact upon which this rumor is based is the upcoming retirement of SAP's Klaus Kreplin. More than likely, there is also an inside leak happening here.

Somebody is guessing that SAP wants to buy a data warehouse vendor such as Teradata or Netezza.

Teradata people in Dayton, Ohio, are sure to be nervous about this. Many are just now regaining their balances after the 2007 spin-off of Teradata from NCR. Their big concern then was that Teradata might leave the Midwest and send employees to a development site on the West Coast. But hey, it's time for a new perspective: El Segundo is a lot closer to family than Waldorf.

This is a logical next step for SAP/Business Objects in order to have a full stack of BI offerings. Today, they can provide BI for their SAP operational and Business Warehouse products, but they probably want to expand into more generic BI data warehouses; purchasing a vendor like Teradata or Netezza makes sense.

If SAP does get Teradata, watch for them to try to swap out competitive databases (e.g., those from IBM and Oracle) at their customer sites.

Saturday, October 4, 2008

The Debate Continues

In his September 2008 article for B-Eye Network, Bill Inmon commented on a study which found that applications storing BI data in Data Marts typically only have a life span of about 18 months. Bill wondered why this might be true and posited the following answers:

  • The traditional data model used for Data Marts (dimensional star schema) is "not conducive to change." Bill says that "the star schema is good for static requirements, not fluid requirements." So you build it for a specific purpose and then find out a year later that it doesn't meet your needs anymore.
  • The traditional tool used for Data Marts (OLAP) is not conducive to change. For slicing-and-dicing of data, the front-end tool and the back-end data model go hand in hand. If the data model no longer meets your needs, the GUI presentation will not either.
  • Business requirements are changing. If you have a inflexible data model, it is not going to meet your needs in the future.
  • Data Marts tend to be departmental solutions for a small pocket of users. Others in the organization may not know about these applications, which leads to their decline and disuse.

"There are probably are plenty of other reasons why data marts have such short lives. And interestingly, ALL these reasons are at play at the same time. It is not just one factor that causes a data mart to go into disuse. Instead, it is ALL of these factors working at the same time.

The net result of the fast expiration of data marts is that data marts start to accumulate in the corporation in large numbers. First, there are four or five data marts. Then, there are 50 or 60 of them. Then, there are hundreds of them."

Of course, this is just a continuation of Bill's long-running "Data Warehouse versus Data Mart" debate with Ralph Kimball. These guys don't draw quite the attention of other debaters such as Biden and Palin or Obama and McCain, but Inmon and Kimball have been going at it for years.

The back-and-forth is often entertaining. One of my favorite discourses was when Ralph said that a Data Warehouse was just a collection of Data Marts. Bill quipped back that you can collect all the minnows in the sea, put them together, and still not have a whale.

Inmon's concept is to create a very flexible data model of enterprise data available for any business intelligence need. Bill considers the data warehouse to have characteristics such as being integrated, serving all people, having a broad audience, and providing all data details in a relational form for flexible reporting. On the other hand, Kimbill's ideas are more practical, in that you create small collections of data for a specific business purpose and group of individuals. Bill pretty much agrees with the definition, saying that the data mart serves a smaller group of people and provides only summary data in a less flexible structure, such as the star schema. He just doesn't like that approach.

That is why Bill is quick to point out that with all the work you put into a BI application with a Data Mart, it is only going to last a year or so. Now multiple this times the number of different Data Marts you have within your organization and Bill thinks you will slap yourself on the forehead and say, "Wow, I coulda had a DW!"

Thursday, October 2, 2008

BAM BI 3: The Blog

Seth Grimes expanded his "Is BAM BI?" question from within LinkedIn and wrote about it in his blog, summarizing the different viewpoints. Read more about it here.

Monday, September 29, 2008

Public Service Announcement: Haiti

Over the weekend, caring volunteers in southwestern Ohio did a special "Pack-a-Thon," getting over 20 tons of food on its way to Haiti to help people there deal with basic survival needs after the hurricanes. Working with "Kids Against Hunger," about 1000 compassionate people from around the Greater Cincinnati area spent two hours each sealing pouches with rice, dried vegetables, soy flour, and flavored protein powder. In just two days of work, they were able to prepare about 348,000 meals and get them loaded onto trucks, ready for a flight to Haiti's neediest children and adults.

Donations to help cover the cost of the food on this Haiti trip are still being accepted.

Kids Against Hunger in Cincinnati will continue to package food on most Saturdays. In addition to sending food to other countries, the Cincinnati chapter of KAH works with local food pantries and churches (for example, with Pastor Mike Tabor at the Happy Church in the Appalachian region of southern Kentucky and Pastor Kareem Smith at the River of Life Church in downtown Cincinnati) to help feed those in need who live nearby.

Open Source BI now Mainstream?

In today's press announcement, Actuate released the findings of a survey of 1000 international professionals on their adoption of open source software.

"Exploring in depth organizations’ use of and attitudes towards open source, across four important territories, the findings categorically confirm that open source software is not a 'here today, gone tomorrow' phenomenon, rather it has been broadly recognized and embraced for its ability to offer organizations sustained competitive advantage. These findings support Gartner’s projections that, by 2012, at least 80% of all commercial software solutions will include substantive open source components*."

*See press release titled ‘Gartner Highlights Key Predictions for IT Organisations and Users in 2008 and Beyond’, January 31, 2008.

Actuate's survey, which they have been doing since 2005, found that European organizations had a higher adoption rate of open source than did those in North America. For example, over 60% of the individuals in France and Germany said that open source was the preferred option when purchasing software, compared to only 40% in N.A..

Actuate itself made a strategic decision years ago to hitch its wagon to the open source star:

"The Actuate platform boasts unmatched scalability, high-performance, reliability and security. Its proven RIA capabilities and highly collaborative development architecture are backed by the world's largest open source information application developer community, grounded in BIRT, the Eclipse Foundation's only top level Business Intelligence and reporting project."

The Actuate survey reported that the biggest challenge with using open source software was the lack of skilled individuals. Actuate says this lack of skills is actually a good sign, because it indicates a high adoption rate. The demand is using up the pool of available people.

The implied claim here is that as more companies use open source software, more technical people will see it as a good career course and become skilled in it. That would be nice. However, what could happen instead is that decision makers say, "Gee, I can't find any experienced people in this open source technology, so it is easier and perhaps cheaper to go with a more mainstream solution that has broader adoption and support. Where's that phone number for my IBM rep?"

Lack of human resources hurts not only the open source BI community, but the smaller software products and legacy technologies as well. As BI professionals center their careers around the four pillars (IBM, Microsoft, Oracle, and SAP) and get under that consolidated BI umbrella, people working with other technologies will feel left out in the rain and will probably seek shelter by developing new skills.

Thursday, September 25, 2008

BAM BI 2: Return to the Forest

My last post on BAM BI prompted my ex-IBI colleague and friend, Larry Eiss, to reveal my apostasy and urge me back into orthodox BI. See Larry's comment below.

Larry points out that Mr. Dresner's position is that business intelligence is complex and requires human decisions. Larry ties this back to some of my earlier blogs on the issue of the evolving User Interface in BI software. Thanks, Larry.

Also, if you want to see some great nature photography, check out Larry's website.

BAM BI, or, The Importance of Being Human

In a LinkedIn discussion group, Seth Grimes posed a simple question: "Should Business Activity Monitoring (BAM) be considered a Business Intelligence (BI) application?"

In other words, is BAM BI? This question went out to all individuals in the LinkedIn EPM Business Intelligence group, but who should respond to Seth's question but Howard Dresner himself, the ultimate definer of BI. Now that is a little like kneeling to pray for guidance and immediately having God tap you on the shoulder.

It is not like Seth is some newbie BI person asking silly questions. Far from it. He chairs the annual North American and European Text Analytics Summits and is contributing editor for Intelligent Enterprise magazine (CMP).

Here is Howard's response to Seth's question:

"By definition, BI is about enabling end users with access to and analysis of data. Hence, I believe BAM is BI when the user is involved in the decision-making process. BAM can also be used for decision automation, which would not be BI. "

Howard, of course, means by his definition. Howard created the business intelligence world in 6 sentences and rested on the 7th blank line. And Howard saw that it was good.

Some naïve LinkedIn members disagreed with Howard and they were promptly struck down with fire and brimstone. Anybody who questions Howard on his own definition of BI would merely be Job shouting to the heavens, "Hey, I think you made a mistake down here, God!"

Despite the risk of BI swords, pestilence, and famines, let's look closely at Howard's position.

By His intelligent design, in a BI application, thou shalt have humans. If people are there to use the analytic results, then an application passes the Dresner BI Criterion. On the other hand, if the software provides intelligence to another non-human application, then it is no longer BI. It is merely "decision automation."

So if I examine stock trends online and decide to buy some shares of Starbucks, I am using a BI application. But if I set up an automated agent to watch the stock market and it invests in Starbucks for me, then it is not BI. Interesting.

But wait, I am a human shareholder now despite the manner of having bought the stock, so doesn't that count for me being involved somewhere in the process? After all, I put my decision-making process into the rules of the application, so I am there in proxy.

Granted, I didn't shout, "Eureka!" and move my mouse to click on the buy button, but the application knew that I would because I had already advised it to perform as if it were me. Shouldn't that count as BI? Or do BI qualities somehow decline the farther downstream the user is from the software application itself?

Perhaps this is like not hearing the tree that falls in the forest if you are not there. If an intelligent activity happened and no user was sitting at the screen to see it, then did the BI not really happen?

Continuing our inquiry into this religious doctrine of BI, let's consider an analogy. Delta Flight 1700 flies from Atlanta to Cincinnati with passengers, so it is obviously a passenger plane. While in Cincinnati, the Delta people remove the seats and fly the same plane back to Atlanta with pallets of boxes filled with canned beans. We would agree that this is now a cargo plane. But it is still an airplane, right? It is definitely still part of our transportation system.

Or would not having any human passengers inside immediately turn an airplane into something else? This is deep stuff, Howard.

Perhaps I should post my own question on a LinkedIn forum: "What sort of penance do you think is adequate in Howard's eyes for my BI blasphemy?"

Saturday, September 20, 2008

Writing in Cursive

My youngest son, a senior in an American high school, will be old enough to vote in the upcoming 2008 Presidential election. I helped him fill out his new voter's registration form with some trepidation, as I assume he will always lean in a completely different political direction than his father.

He went through the standard U.S. voter qualification checklist, claiming such obvious things as being of sound mind, being an adult, not being incarcerated somewhere, and so forth. When he was done with that, I said, "OK, when you sign here, be sure to use your full name."

"You mean, sign my whole name in cursive, even my middle name?" he asked in a disbelieving tone.

"Yes, that is your legal signature that you use for formal statements," I answered, wondering at what early age he stopped believing that I might sometimes be right.

"Well…" He paused and then admitted, "I'm not sure that I know how to write my middle name in cursive. I only use cursive for my first and last names."

What? He doesn't know how to write in cursive? I thought that every American third-grader learned to write in cursive!

But my teenage son not being able to write in cursive should not really come as a surprise to me. If you are never forced to use something you learned early in life, will you really retain it in memory or be able to use as an active skill when you are older?

Way back when I was in school, my teachers would explicitly state that some homework had to be written in cursive. Today, educators are probably instead telling children what file format to use.

"Remember children, I will only accept your essay on 'How I Spent My Summer' if it is in a Microsoft Word 97 format. If you have Word 2007, be sure to save in a backward-compatible format! (And no more PowerPoint, Billy!) Oh, and Arial 11, please. If you have any questions, text me. I will follow your Twitter to see how you are doing. OK, line up quietly at the door for dismissal. Don't forget your iPods!"

If you are the parent of a teenager (God bless you, by the way), see if you can get them to stop texting friends for 15 seconds and have a simple conversation with you. No hard topics, just a question or two. When did he or she last write a document in cursive? Is cursive writing important anymore?

If you are a vendor of legacy BI tools, ask your customers if they are still writing new applications in your technology. What? They are no longer writing programs in NOMAD, RAMIS, FOCUS, DYL280, or SQR? They own the product, have trained users, and still pay the annual license fees! When did they stop writing in legacy code?

Sometimes, people stop doing things that they did in the past. Things that we assume will always be done in the future. This declining usage may happen slowly and we do not always notice until long after the fact. Legacy BI tools, for example, may still be out there, installed somewhere on the enterprise, but they may not be actively used.

If your organization owns legacy BI tools, consider consolidating them to newer, more productive, technology. By the way, if you have a teenager, encourage them to vote (even if they are going to make different choices than you would).

Friday, September 19, 2008

Big Blue Cognos

After acquiring Cognos Corporation in early 2008, IBM wasted no time porting the Cognos 8 BI product onto their System z platform. When asked "Why port Cognos to the mainframe?," Mike Biere of IBM answered:

"Cognos has a portfolio of BI and PM solutions based upon a modern, SOA architecture. It’s been one of the leaders in this space for awhile with an enormous install base and many satisfied customers. IBM and Cognos had been working closely within IBM’s software brands to examine ways their mutual technology stacks could benefit.

Given IBM’s leadership position in SOA and portal, Cognos 8 was deemed a perfect fit with little overlap in features and functions. It offers an integrated platform with many BI and PM capabilities. It sits upon a common metadata layer allowing the end user to select from a deep set of functions (including query, reporting and dashboards) to deliver critical business information. This information may be provided to the end user in many formats or user interfaces, such as Web browsers or personal digital assistants."

Mike also says that the mainframe is the perfect server choice for BI applications. Advantages of using the System z include:

  • Server consolidation
  • Mission-critical BI and PM
  • 24-7 operation and availability
  • Security Compliance issues
  • Environmental (green) factors
  • Managing mixed workloads and BI on a common system
  • Near real-time BI operations
  • Closer proximity to data captured at the enterprise level
  • Operational BI
  • Lower total cost of ownership and higher return on investment

Most BI tools did not run on the mainframe, so companies often moved their reporting and analysis off to a distributed environment. But this caused more problems than it solved. Now you had redundant data in the enterprise which needed to be "synced" on a regular basis using data transfer processes that suck up CPU, network bandwidth, and disk space. The copied data idea started as being "yesterday's data" until users demanded more real-time information, which then meant implementing sophisticated data-change-and-replicate processes. IBM Cognos may have the real solution: put BI back on the mainframe with the data.

To read Mike Biere's entire story, see his July/August 2008 article in the IBM Systems Magazine.

Wednesday, September 17, 2008

Why BI Doesn't Work

Doug Henschen, the editor in chief at, recently posted the results of a BI survey asking 385 technology professionals about the barriers to their success with BI software. You can read about this in his September 2008 In Depth / State of BI article titled, "Business Intelligence Gets Smarter."

Here are the rankings of BI barriers, from highest to lowest:
  1. Complexity of BI tools and interfaces
  2. Cost of BI software and per-user licenses
  3. Difficulty accessing relevant, timely, or reliable data
  4. Insufficient IT staffing or excessive software requirements for IT support
  5. Difficulty identifying applications or decisions that can be supported by BI
  6. Lack of appropriate BI technical expertise within IT
  7. Lack of support from executives or business management
  8. Poor planning or management of BI programs
  9. Lack of BI technology standards and best practices
  10. Lack of training for end users

The IT individuals in the study place most of the blame on the BI vendors and then point to their organization's data as the next big reason for problems. Their remaining 7 reasons are internal issues that prevent BI from being used successfully.

Their message is that the BI software itself is too hard to use and too expensive. Within the company, the IT group and the business users do not have the proper training nor are there adequate resources. To make matters worse, upper management either is not taking the BI initiatives seriously or they do not understand the amount of work involved.

Monday, September 15, 2008

Gartner Summarizes BI

In a June 2008 statement, Gartner reported that the BI software market in 2007 exceeded $5.1 billion, which was a 13% increase from the previous year. During that year, the 3 largest publicly-traded BI vendors were acquired (Business Objects, Cognos, and Hyperion Solutions). The big software vendors selling single "stacks" of technologies (Oracle, SAP, IBM, and Microsoft) now own 67% of that multibillion dollar BI market.

Dan Sommer, a senior research analyst at Gartner, says that this "marks a pendulum shift from a market driven by best-of-breeds to one dominated by megavendors." That doesn't necessarily mean that the big guys will take the entire market. Sommer adds, "However, many smaller independent BI vendors grew faster than the market, and we expect continued innovation and new vendors to enter the market.”

Sunday, September 14, 2008

SAS Global

Last week, at a business event in Mumbai, India, SAS announced an expanded partnerships with two of the world's largest outsourcers, Tata Consulting Services (TCS) and Wipro. SAS is the largest of the privately-held BI vendors (see blogs on Information Builders as well) with over $2 billion in annual revenue. Today, that is twice the size of Cognos, which was recently acquired by IBM.

TCS reports that 10% of their almost $6 billion dollar revenue comes from their BI practice. As part of the announcement, TCS says that they will speed up their consultant training on SAS. Likewise, Wipro says that they will expand the Global Center for Excellence on SAS from 300 to 700 trained SAS consultants.

The day before this announcement from Mumbai, SAS had released a statement about their global growth:

SAS is experiencing its fastest rate of revenue growth in the emerging markets of Asia-Pacific, Eastern Europe and Latin America. In 2007, SAS revenue grew 28 percent in Brazil, nearly 50 percent in India, almost 60 percent in Russia and 67 percent in China. It’s no secret that the BI and analytics sectors of the software market are growing faster than the market as a whole. Worldwide BI platform software revenue exceeded $5.1 billion in 2007, a 13 percent increase from 2006, according to a June 2008 report from analyst firm Gartner.

In business since 1976, SAS achieved $2.15 billion in 2007 annual revenues, up 15% from the previous year. SAS says that "intense demand for business analytics -- BI, analytics and data integration -- propelled this growth."

“As companies become more global, they look to the next generation of software solutions to help them optimize revenue, increase customer loyalty and satisfaction, and enhance competitiveness,” said Mikael Hagström, SAS’ Executive Vice President for EMEA and Asia Pacific. “BI and analytics are increasingly the answer, whether in mature markets like Western Europe and North America or rapidly emerging markets like India, China, Russia and Brazil.”

The activity in Mumbai doesn't mean that SAS ignores American workers. Just the opposite; SAS has always been a favorite career spot for IT professionals.

SAS’ recognition of employee value drives the company to offer such amenities as onsite child care, an eldercare information and referral program, an onsite healthcare center, wellness programs, a 58,000-square-foot recreation and fitness center with natatorium, an extensive package of benefits and many other work/life programs. The policies derive from the employee-focused philosophy behind SAS’ famed corporate culture: if you treat employees as if they make a difference to the company, they will make a difference to the company. With one of the lowest turnover rates in the industry, SAS is reaping the rewards of its work/life programs and family-friendly policies.

In addition to experienced individuals already in the workforce, SAS is helping to address our youth's declining interest in STEM areas (science, technology, engineering, and math). SAS pre-empted the Mumbai announcements with one about training American high school students in their products. After a successful pilot at a North Carolina high school, SAS is expanding a training program to 9 other high schools and will make the material available throughout the United States in 2009. SAS will train qualified high school teachers at their headquarters in Cary, NC.

SAS skills are in high demand by companies, governments and organizations worldwide that use SAS to analyze huge amounts of data to make better decisions. SAS careers vary from entry-level programmers to executive positions requiring data warehousing, data mining and analytical expertise. The biggest demand is currently in the pharmaceutical and financial services industries, which use SAS in research and development, marketing, fraud detection and clinical trials. Students who graduate high school with SAS knowledge carry a distinct advantage into post-secondary education, where SAS is used in many college courses requiring quantitative analysis, such as psychology, statistics, mathematics, business, and public health.

The around-the-world activities of SAS represent a microcosm of our global business macrocosm. SAS is global because the rest of the world is global. With companies openly considering North America as a "mature market," attention is turned to "emerging" countries such as India and China. Americans must work to avoid our growth cycle from going to "maturity" to "decline."

Saturday, September 6, 2008

Personal Jet Packs

In his 2006 book, "The Change Function: Why Some Technologies Take Off and Others Crash and Burn," investment writer Pip Coburn made some predictions on future winners and losers in the marketplace. And the winners are...

  1. Flat Panel Display Televisions
  2. Mobile Enterprise E-Mail
  3. Business Intelligence Software
  4. Satellite Radio

OK, these predictions may seem about as wild as me guessing my son's college will be expensive next year, but since Pip mentions BI, let's hear him out.

Pip writes that BI software "is an antidote for complexity - the greatest curse in the enterprise." While the idea of getting to all of your enterprise data sounds great, Pip says that the high "total perceived pain of adoption," or TPPA, prevents companies from really doing BI. Pip continues:

"Very few of the people who want what business intelligence software provides have ever interfaced with a database for more than a short period and, therefore, have little clue as to how powerful these tools can be. But the culture will steadily shift and education about the power of the tool will expand steadily over the next decade at which point many CEOs will carry a pocket device they can use to query databases all over the world in an integrated, near real-time fashion. The migration from here to there will be powerful. Vendors that make business intelligence simple to the end user will dominate those focused on additional features."

That is all fine and good as long as the future CEO does not try to use his iPodWorldQuery device while strapped into a personal jet pack. Always practice safe flying; just mind-link back to the home robot and let Rosie take care of the BI.

But seriously, Pip does have a good point. Ms. CEO wants to be able to get accurate information right now, presented in a way that is useful for her to make important decisions. She doesn't care where that data resides or in what arcane format it is. She is not going to write some SQL query in her spreadsheet macro to get that information. Instead, she needs to tell the user interface what she wants and let it figure out the howz-its and whatz-its.

We are still waiting for this super user interface that can handle BI requests.

Thursday, September 4, 2008

Hitting Stumps

I grew up in a small farming community just seven miles south of Nowhere. Not much happened in my hometown, so we learned to really appreciate even the small events. In particular, the town had one major claim to fame: it was the site of the very first automobile accident in the entire world. The premier, never-before, first-of-its-kind fender-bender.

Yes, all this is absolutely true. Looking out my bedroom window, I could see this significant spot at the intersection of Liberty and Carmean Streets.

Each summer we celebrated this major historical event with a festival, holding a parade, turning the entire downtown block of Main Street into a carnival, and choosing a king and queen in honor of the wreck. But you probably want to know more about the accident.

Back around 1890, my hometown had a smart guy named John who tried to make an improvement on the ubiquitous horse-drawn buggy. He put a one-cylinder gasoline engine on a buggy chassis that had two back tires and a single tire in the front. For steering, John attached something like a hockey stick to the front tire, which he had borrowed from his wheelbarrow.

A picture of John's automobile (before the accident, of course)Eye witnesses said that John could get that souped-up runabout moving to about 15 miles per hour. Of course, this was before anybody considered enforcing speed limits on the city streets.

In fact, nobody had even thought of paving the roads yet, much less having seat belts, air bags, car insurance, or 24-hour tow trucks.

On one joyride around town with his buddy Jim, John must not have been paying attention to the road. When he got to my street, John either didn't see that tree stump or he was unable to maneuver around it, what with going at such a high rate of speed and steering a front wheelbarrow tire with a stick. He crashed his one-of-a-kind automobile and made history. Less than a hundred years later, we put up a nice sign commemorating John's poor driving ability.

Jim helped John push the damaged car back to his house a few blocks away, planning to make the world's first accident repairs.

John's luck was about to get worse. In some strange twist of fate, his garage burned down, destroying his unique wrecked car.

At the time, his neighbors were unaware of the historic significance of John's undertaking. Instead, they probably gave him a hard time about trying to be smarter than everybody else. Why replace oat-fed horses with noisy gasoline-powered engines? Especially since gasoline cost two cents per gallon!

John responded accordingly by packing his bags, saying goodbye to his soon-to-be famous friend (after all, Jim was the world's first automobile passenger accident victim), and leaving. Not only did John leave my small town behind, he left the state and moved to a emerging western territory called Indiana.

In a booming state at the beginning of the 20th Century, John quickly went to work on Automobile 2.0 (probably implementing new features such as four tires and an improved steering device) and made a name for himself in this hot industry. However, when you have an innovation with the potential to make money, lots of people want in on the action. Before long, John was competing against similar gasoline-powered vehicles with names like Auburn, Cord, Duesenberg, Dodge, Chevrolet, Cadillac, LaSalle, Pontiac, Plymouth, Ford, Oldsmobile, DeSoto, Buick, Mercury, Packard, and Kissell, just to name a few.

Over time, John and many of the original automobile innovators either dropped out, moved in a different direction, or were consolidated into bigger companies. Being a smart guy, John focused his energies on creating 600 patents and let the other firms pay him for his ideas.

You probably do not recognize some of these original automobile brand names as most are long gone. The automotive industry consolidated into a handful of manufacturers, some of which struggle today to survive.

Things change. You start with innovators like John who recognize problems and come up with cool new solutions. Of course, they may barrel down unpaved streets not yet ready for them and hit a stump. While that accident may scare off some early adopters, a few are still intrigued by these new contraptions. They muse, "There may just be something to the idea" and decide to get involved. Diffusion starts and, with at least some innovations, things really do get exciting. When the possibility of lots of money comes to this new innovation, some powerful people often decide to take control of it.

We have seen the same thing happen in the BI software industry. IBM now owns Cognos, an early innovator from the 1970s which had already merged other BI players into its brand. SAP has Business Objects, which had earlier bought Crystal Reports. Microsoft grabbed ProClarity and others. Oracle has Hyperion Solutions, which had already acquired Brio Software.

I read where Larry Ellison said that he sees no more innovation happening in the software industry.

That may be the real trend here. The guy at the early stage of the new idea takes chances and drives off down the bumpy road that may contain an unseen tree stump. If he manages to successfully navigate his way, some big guy may step in and say, "Thanks, John. I'll take it from here." The time for true innovation is then over and now it is business. The focus is on increased market share and improved profits.

No, I am not saying that big companies cannot innovate. Windshields, cup holders, heated seats, anti-lock brakes, navigation systems, and satellite radios are all nice new features and deserve polite compliments. But it's the truly big change like swapping your horse reins for a steering stick that is the stuff of legends for which you throw parades and hold street carnivals.

Did I mention that my close friend's Grandpa Jim was the world's first automobile passenger accident victim? Yes, absolutely true.

1891 Picture (Source: Smithsonian Institute, Washington, D.C.) as shown in the book, "Carriages Without Horses" by Richard P. Scharchburg.

Wednesday, September 3, 2008

Claudia on Operational BI

One of my recent posts reported that Michael Corcoran of Information Builders argued that Business Intelligence should not "be wed to data warehouses." Claudia Imhoff and Colin White seem to back him up with a similar point in their interesting article, "Full Circle: Decisions Intelligence (DSS 2.0)."

Writing for B-Eye Network, they say,
"The tight connection between business intelligence and the data warehouse has led to the assumption that data must be maintained in a data warehouse before it can be used for business intelligence. This assumption is wrong. There are an increasing number of BI applications that do not employ a data warehouse, either because there is no need to store the data in a data warehouse, or because it is not practical or cost effective to do so."

They are quick to point out, however, that "data warehouses are not going to go away."

Imhoff and White feel that the 'business intelligence' term has become confused and linked with just a subset of application usage. Instead, they propose that we bring back the terminology of 'decision support systems' and brand a new 'Decision Intelligence' for the present and future.

They suggest a Decision Intelligence architecture consisting of three conceptual BI subsystems: Data, Process, and Content. What they call the Business Data Intelligence component is basically our traditional BI features for strategic and tactical analytics using structured data, which is where data warehousing plays a role. The Business Process Intelligence component provides for real-time event analytics (BAM), while Business Content Intelligence analyzes unstructured data within the enterprise (e.g., social networks).

As we try to build a broader BI umbrella with which to cover business processes, human interactions, and unstructured content, in addition to our traditional BI structured data, we are perhaps moving not backwards into Decision Support but forward into the Knowledge Management space.

Tuesday, September 2, 2008

Chrome Comics Hot off the Press

Google just released a beta Windows version of their open-source web browser called Chrome. While that is news in itself, I think the way they are explaining the features using a comic book format is fantastic! The graphic presentation of why Chrome exists and how it works is not only quick and easy to read, but intellectually convincing as well.

Monday, September 1, 2008

Sherman Sees BI (Episode 1)

"Gee, Mr. Peabody, why did we use the WABAC machine to take us into a dark building with strange people at desks?"

"Good question, Sherman. We are in 1980 America and these individuals are preoccupied with what was called 'data processing' or just DP. They are managing computer systems which processed business transactions for functions such as sales, manufacturing, payroll, and inventory tracking. I wanted to tell you about Business Intelligence software, but to do that, I thought I should first show you some of the general history of computer systems."

"Wow, Mr. Peabody, that lady is working at a giant keyboard!"

"Not quite, my boy. That is a card punch device, an early way to interface with a computer. She will punch each line of instruction onto a small, stiff piece of paper until she has whole stack of cards that constitutes a program for the company computer to execute. Typically, the instructions were written in special code such as Assembler, COBOL, or PL/1. Alas, that method of inputting commands is almost obsolete even at this this point in time."

"But look over here, Sherman! See that young man in front of the monitor giving off the green glow? That is Elmer Podsnap and he is typing his commands directly into a 'terminal' access point for the company's computer. He is what would be called a 'computer programmer' or a 'coder.' We'll want to keep an eye on him."

"Okay, Mr. Peabody, but that sounds pretty boring."

"You see, Sherman, at our current space-time point, the programmers have simple textual interfaces where they type in commands to interact with a centralized computer. This is definitely an advancement over writing code down on paper, punching a copy onto a stack of cards, and having the computer read them. Plus, the rubber bands from the cards pulled the hairs on Elmer's wrist."

"Now, Sherman, be a good boy and turn the WABAC control knob forward one decade. Let's move slightly ahead and take another look at Elmer's career in DP."

"Gee, Mr. Peabody, it's now 1990 and the computer programmer is still sitting in front of his monitor. Is he wearing the same clothes?"

"Ah, you are missing an important difference here, my boy. Elmer's monitor is no longer just a dumb terminal connected to the computer. It now has its own intelligence built into it (they call it a 'personal computer') and it can now provide Elmer with a richer user interface - WIMP!"

"Gosh, Mr. Peabody, why did you call Mr. Podsnap a name?"

"No, no, Sherman. Elmer isn't a wimp, his new computer interface is! It has Windows, Icons, Menus, and a Pointer - WIMP. Instead of always typing in his commands, Elmer now has a graphical way to communicate with the computer. He can point at instructions and click on a pointing device to tell the computer to execute the commands."

"And here is another improvement for 1990, my boy; not only is Elmer connected to the central server, but he can also reach out to a network of other personal computers and some smaller servers within his company. His managers even stopped calling his organization 'DP' and switched to 'Management Information Systems' or MIS. Now things are getting exciting for Elmer! Quick, Sherman, take us forward to a new century, the year 2000!"

"Here we are, Mr. Peabody, but it just looks like Elmer has a bigger monitor. And he is definitely wearing the same shoes! Oh, and the walls of his office are gone and he is crammed into a weird little cube-shaped space. Did Elmer do something wrong, Mr. Peabody?"

"We can talk about that later, Sherman. For now, notice that Elmer's computer interface has changed. He now communicates with the machine using a simple graphical interface called a 'browser.' His personal computer is smarter than ever, but in some ways the interface method has reverted to a more simple mechanism where, like in 1980, the work is done on a server. But Elmer is not just limited to his own company's servers anymore. He is now connected to a global network of computers!"

"Gee, Mr. Peabody, his computer screen sure has gotten prettier!"

"Yes, now set the WABAC controls to the present, my boy."

"Hey, here in 2008, Mr. Podsnap has a new pair of shoes!"

"Yes, Sherman, but Elmer also has a new, smart interface for communicating with computers all around the world. It is still early in the development, but his browser is quickly becoming a rich user interface."

"Mr. Peabody, are you going to explain what Mr. Podsnap has to do with Business Intelligence software or are you just going to make another bad pun?"

"Sherman, my dear boy, we will understand it better BI and BI."

About Me

My photo

I am a project-based software consultant, specializing in automating transitions from legacy reporting applications into modern BI/Analytics to leverage Social, Cloud, Mobile, Big Data, Visualizations, and Predictive Analytics using Information Builders' WebFOCUS. Based on scores of successful engagements, I have assembled proven Best Practice methodologies, software tools, and templates.

I have been blessed to work with innovators from firms such as: Ford, FedEx, Procter & Gamble, Nationwide, The Wendy's Company, The Kroger Co., JPMorgan Chase, MasterCard, Bank of America Merrill Lynch, Siemens, American Express, and others.

I was educated at Valparaiso University and the University of Cincinnati, where I graduated summa cum laude. In 1990, I joined Information Builders and for over a dozen years served in regional pre- and post-sales technical leadership roles. Also, for several years I led the US technical services teams within Cincom Systems' ERP software product group and the Midwest custom software services arm of Xerox.

Since 2007, I have provided enterprise BI services such as: strategic advice; architecture, design, and software application development of intelligence systems (interactive dashboards and mobile); data warehousing; and automated modernization of legacy reporting. My experience with BI products include WebFOCUS (vendor certified expert), R, SAP Business Objects (WebI, Crystal Reports), Tableau, and others.