Wednesday, December 21, 2011

Being a Trusted BI Advisor

In his book called "How to Win at the Sport of Business," Mark Cuban makes an important point that your customers do not know what they don't know. 

If you are a Business Intelligence consultant, this means that your clients may give you the worst advice on how to build a BI application. They may understand their business problems, but not how to solve them with BI. Instead, you as an expert need to tell them how things should be done. 

Of course, if they are to listen to your advice, they must first trust you. Building trust on your BI engagements is one of the most important things you can do. David Maister once wrote,
"The key point is that trust must be earned and deserved. You must do something to give the other people the evidence on which they can base their decision on whether to trust you. You must be willing to give in order to get." 

You cannot demand trust; instead, you must work hard at developing it. Until you have the client's trust, they will more than likely force you to follow their own mistaken approaches instead of listening to your good advice. 

See this Entrepreneur magazine article for Mark's comments.   

Monday, December 19, 2011

BI Clarity

On a Business Intelligence initiative, much of your success will depend upon a very special word--"clarity." Here is how one online dictionary defines it:
noun: clearness or lucidity as to perception or understanding; freedom from indistinctness or ambiguity.

When performing a BI project, everybody must be clear in their understanding of what is going to be delivered, why, when, how, and at what cost. Ambiguities in the project's mission, tactical approach, or deliverables will lead to failure.

I have recently used a low-fidelity wireframe tool from a software vendor named Balsamiq to reduce ambiguity. This mockup tool helps to quickly define the BI initiative in a collaborative approach.

Not only can business users see an example of what is going to be built, they can interact with it. The Balsamiq Mockup application has simulated links for demonstrating button clicks, listbox pull-downs, drill-downs, and other common user interface controls.

Before any real application development starts, the users can test out your plans and bless the user interface design. Issues can be pointed out immediately, not after weeks or months of costly effort.

Mockups force everybody to put ideas into a quasi-working format and validate feasibility.

I am a fan of Balsamiq Mockups. At $79 USD, it is a cheap way to bring clarity to your BI project.

Friday, November 18, 2011

The Biggest BI Blunder

Many Business Intelligence projects struggle and fail. But there may be just one general reason why they go south: the individuals in charge did not treat the initiatives as software application development projects.

Instead, many people with failed projects saw BI as a business initiative or as a financial exercise and ignored decades of best practices of software development lifecycle.


If you try to run a BI project as something other than a software application development effort, then you are immediately starting off on a bad trip with the wrong people on the wrong vehicle.

Wrong Approach
By not understanding that your BI initiative is an SDLC project, you will not follow well-established practices that would improve your chance of success.

You might not even follow basic project management concepts such as the "Magic Triangle" to define the "scope" of what you promise to deliver by what time and at what cost. Experienced SDLC developers understand that in a project you are stuck with three interrelated variables.

Changing one of the following impacts the others. 
  • Deadline (the length of time to complete the project)
  • Functionality (the scope of the project, the features your provide)
  • Cost (the resources on the project)

If you need your BI project to be done quickly, then you must add resources (increase cost) and/or reduce what will be delivered (reduce functionality).

If you want lots of functionality, then you must push back the deadline and/or add resources (increase your costs).

If you need to lower costs, then you have to be willing to eliminate functionality and/or push back the deadline and work with fewer resources. 

Ka-Boom!
Experienced SDLC developers also know the difference between the "Big Bang" approach and a phased, iterative roll-out.

BI initiatives are sure to fail when an influential person gives vague marching orders such as, "I want an enterprise dashboard where everybody in the company can access all data in any way they want. And I need to demo it at the annual conference in three months. Oh, and make sure it works on my iPad, Bob's Android phone, and Sally's iPhone, and Greg's BlackBerry."

Trying to create an entire complex application in one fell swoop is a sure recipe for failure. 

Burning Your Dollars
Successful SDLC projects follow a phased staffing approach. Not all individuals needed on the project should start on Day One. Yet that is a common mistake on failed BI projects.

Imagine building a house. On Day One, you only have a plan and an empty lot. The first step is to dig a hole and pour the foundation. If your custom home builder told the electricians, plumbers, dry-wallers, and painters to come on Day One and just have a seat in lawn chairs to watch and wait for their services to be needed, you would immediately recognize this as a waste of time and money.

Yet bringing in GUI dashboard developers to sit and wait for the database team to finish a repository is common on failed BI projects. It just is not as obviously a waste since IT people look more productive sitting in a Herman Miller chair in a cubicle rather than out front on a lawn chair. 

Wrong Expectations
Experienced software developers know they will not be successful unless they first have a clear definition of the application they are to build. Successful individuals are able to take a fuzzy idea from the business and work jointly with others to properly define and formally document the specifics of what will be done.

The worst BI projects are those where the ideas for the dashboard are in one person's head. This person is always frustrated that nobody else on the project is smart enough to understand what she is thinking. Without formally dumping ideas from that person's head into a well-articulated requirements document, the BI project is doomed. Yet it often happens.

Note: read the "Tappers and Listeners" section in the Heath Brothers' book "Made to Stick" for more on this topic.

The first problem of not knowing what you are building is that you will create the wrong thing; your project has to fail since it can never meet expectations.

There is another problem of not clearly defining what you are building: you will never be done. At some point, the executive paying for the BI project will start to scream about the time and cost. "Why is this BI project taking so long?"

Well, it was never designed with an end goal in mind and as a result the BI developers are continuously running a marathon on a circular track.

Wrong People
If you do not understand that you are running a SDLC project, then you will not employ the right people.

If you view this as a business initiative, you will naturally pick people who understand your business. It just seems to make sense to have the guy in Finance who calculates your business metrics to run the BI project. But without a formal SDLC understanding, Bob in Finance is not going to be able to run a successful software project.  

Repeatedly, I see a certain type of organization fail in BI initiatives. These have been those where their IT groups have traditionally done operational support; they do not perform application development projects. While they are great at running the networks and keeping e-mail and packaged applications running, they are not software developers. 

When these types of IT groups get complex BI projects thrust upon them, they typically fail. This is not what they know how to do. 

They should not be blamed; their organization is intentionally designed to provide operational support.

They are inexperienced in software development. Yet it does not occur to the executive that he or she is asking for the impossible from these employees. Instead, the project fails and the executive asks, "Why can't my IT development implement a dashboard?"

Plan for BI Success
Business Intelligence is a software application development endeavor; without that basic worldview in place, your BI project will fail. Instead, you will jump into a BI project with the wrong approach, the wrong expectations, and the wrong people. 

A "Just Do It" command from an executive will not make an enterprise dashboard magically appear. To be successful, you will need experienced people following the right methodologies and best-practices to create your clearly-defined BI application. 

Thursday, November 17, 2011

Boris Evelson Predicts BI Future

Well-known Business Intelligence industry analyst Boris Evelson documented his Top 10 predictions for the 2012 BI market. You can read them on his blog.

Boris foresees a continuing trend of end users being able to serve themselves with better BI applications. To meet these needs, software vendors and IT groups must continue to innovate and provide what is needed for self-service BI.

Boris mentions innovations that you would expect, such as mobile BI, big data, BI-specific databases, and the cloud.

I found "Exploration" on his list to be interesting. That must mean "whatever you want to do" type of interaction with the data, going well beyond slicing and dicing a predefined cube structure or painting an ad-hoc report from a simplified data view. Boris seems to be pointing to "limitless BI" data discovery.

I also like Boris' comment about how enterprise standardization on a particular BI product or technology stack is not taking place. Instead, Boris says:
"Enterprises will learn to live with multiple BI tools. Forrester client inquiries about how to live with multiple BI tools far exceed inquiries about platform consolidations."  

As always, Boris makes good points.

Thursday, September 22, 2011

BI Vendors use Communities to Serve Customers

BI vendors tend to primarily follow a software production business model and shy away from too much services. One reason is financial as they are often evaluated by a metric calculated using dollars divided by the number of employees. Because of high margins for software, this "revenue per employee" or "profit per employee" metric is much higher when a vendor focuses on selling software and not services.

For a nice visualization of this, see Pingdom blog from earlier in 2011 showing Apple with profits of about $420K per employee (almost tripled since 2008 when it was just $151K). Google ranked next with $336K in profits for each headcount; Microsoft followed with a measly $245K.

With that said, software customers still need help. At a minimum, software vendors must provide documentation and training services. They also need to offer a customer service desk to call when clients are in a pinch. Sometimes, customers do need onsite consulting services.

Some BI vendors have found a way to provide services without having lots of people on staff: they enable their customers to help themselves.

Qlik Technologies has a "QlikCommunity" of more than 62,000 QlikView BI users from around the globe. They report having one hundred new users sign up each business day.

Doug Laird, the VP of Global Marketing at Qlik Technologies, says:
“Today’s business users need to make quicker decisions backed by clean, relevant information. This means that queuing up and waiting for IT to generate a report just doesn’t cut it anymore. This is why QlikView’s self-service Business Discovery solution – backed by QlikCommunity – is providing a boost to these users by enabling them to quickly react to changing market conditions. This combination has taken our support to the next level, and is a true testament to the compelling content and engaging experiences enabled through QlikCommunity today.”

Information Builders, the vendor of WebFOCUS, has a similar service called "Focal Point." Their community web page states there are 5000 developers posting 88,000 messages on 13,000 different topics. Information Builders employs some individuals who watch over the postings, engage when necessary, but ensure that solutions are clearly documented for future searchers.

With this type of self-service question and answer forum, BI software customers can search for existing answers to their questions and interact with peers instead of opening a ticket with a support representative. This frees the software vendor to focus on what it does best--building software--and to be more profitable in the process. 

Wednesday, September 21, 2011

Text Analytics for Legacy BI Analysis

A stumbling block for businesses trying to modernize legacy computer applications is the sheer volume of program files. An IT organization may own thousands of code libraries, each with thousands of programs. Often, the whereabouts of the original developers of old applications are unknown. 

I have found this to be especially true for legacy end-user 4GL reporting tools such as FOCUS. 

A computer language developed by Information Builders in the mid-1970s, FOCUS became the industry standard as a multi-platform report writer for business end-user communities. With FOCUS, rather than ask the busy IT organization to develop reports, users could build their own.

But instead of being just a report writer, FOCUS was in reality a full application development environment originally designed to replace COBOL. Many enterprising users took advantage of robust features such as online screens, database maintenance, and batch processing  to build very sophisticated systems.

Two or three decades later, some IT shops now struggle to grasp what their FOCUS users developed. Trying to assess the purpose, functionality, usage, and complexity of these legacy applications by manually looking at each program is nearly impossible.

To assist with this type of time-consuming detective work, I developed text analytics software called the "BI Modernization Workbench." Written in C/C++ with a web browser graphical user interface as well as a command-line batch processor, the application has two main features: 1) automated textual discovery; and 2) automated translation into a new BI product.

For now let us consider only the automated textual discovery feature called the "BI Analyzer."  

Text Scanning
Computer programs are not completely unstructured like an e-mail message or the prose found inside a Word document. Instead, almost all computer programs follow a particular formal syntax which forces them to be at least semi-structured text. This simplifies textual analytics since we know what to expect (for the most part, anyway, since there can still be user syntax errors and a fair amount of junk). 

Our textual analytic scanner is smart enough to figure out the code dialect, but we provide it with some starting instructions. For example, we can tell the application to perform a very specific scan such as looking for FOCUS-to-WebFOCUS conversion issues, FOCUS metadata to find data formats, SAS statistical features, JCL batch job features, HTML legacy CGI calls, Crystal Reports features, or to parse SQL commands. 

When in a curious mood, we can perform custom ad-hoc textual searches.

While the results pulled from the text can be just displayed on a screen, it is more useful to save these to a database and later analyze the answer set.

Online GUI and Batch Text Scanning
We started with a GUI front-end, but when working with a large number of libraries it quickly becomes tedious to repeatedly point, click, and run. As a result, we modified the Scan program to be alternatively run using a batch script from the command line.

Not only is it easier to use, the scanner runs much faster since we eliminate generating HTML for displaying results within the browser. On our current engagement, we scan close over 200 mainframe libraries containing over 80,000 programs within 15 minutes.

Keyword Frequencies
For many of the scans, the software performs keyword frequency counts. For example, to evaluate conversion issues related to green-screen application development, the scanner searches the text for a variety of FOCUS keywords whose either presence or absence would be significant: 
  • MAINTAIN
  • -WINDOW, -CRTFORM, -PROMPT, -FULLSCR
  • CRTFORM, FIDEL, FI3270 (used within MODIFY)
  • PFKEY, SET PF

To help with the accuracy of the scanning, we can apply a variety of criteria on searches such as: 
  • Perform case-sensitive search (or uppercase all text first)
  • Perform stand-alone search (or allow the token to be embedded within a string)
  • Ignore blanks between search tokens (since developers often format code using spaces between words)

Pattern Recognition
Using the results of the keyword searches, we can group specific ones together help identify a pattern of usage within the application. For example, if we group keywords found during a legacy FOCUS 4GL scan, we should recognize one or more of the following archetypes: 
  • Reporting App = high number of TABLE (report) requests but few MODIFYs (database updates)
  • Online Reporting App = Reporting App with high number of -CRTFORMs (menu screens) or -PROMPTs
  • Online Maintenance App = MODIFYs, CRTFORMs (transactional screens), and PFKEY usage
  • Batch Maintenance App = MODIFYs with FIXFORM/FREEFORM (transactions) instead of screens
  • Multi-Step Batch Job = JCL with various FOCUS and non-FOCUS steps (which implies this application may be difficult to port to a new platform)

Textual Parsing
For some textual analytics, we actually need to parse the semi-structured code and pull out more than just keywords. For example, we often find SQL (structured query language) embedded within reporting applications. Being structured, SQL follows a strict syntax of blocks of code in a specific order of: SELECT; FROM; WHERE; GROUP BY; HAVING; ORDER BY.

This makes it possible to parse the syntax and extract the names of databases, tables, and columns being used in the application. We can also distinguish between the columns showed on the report versus those being used in the selection criteria or for sorting and aggregation.

Standard Content Analysis
With these textual contents extracted and stored inside a database, we can then perform standard reporting as well as custom queries. For example, one well-known client used the scan results to perform a redundancy of their Business Objects environment to evaluate it being replaced with a new web-based solution.

The business sponsor was completely against a one-to-one conversion of these legacy reports. Instead, from the scanned contents of thousands of reports and SQL files, the client was able to identify commonalities and reporting redundancies which enabled them to categorize their BI needs into a dozen buckets. From there, they built a roadmap for replacing their legacy reporting environment with a collection of highly dynamic reporting solutions.

In addition to analytics, we have standard reports that help with the operational aspect of a modernization initiative such as parallel test plans. 

Building a Textual Analytics Engine
When companies need to modernize an application, they often view it as a one-time activity. With this mindset, they might not invest the time and money to build this type of textual analytics scanner and translator. Because we work with a variety of clients with this common need, it made sense for to create a reusable tool such as the BI Modernization Workbench.

This application has evolved over time. When we first developed it, it handled SQL-based legacy tools. After that, we enhanced it for the NOMAD and FOCUS 4GL. Since then, we have added features for a variety of products such as SAS, QMF/SQL, Oracle Portal, and SAP Business Objects (Crystal Reports, Deski, and Webi).

In addition to the reporting tools, we have added features for handling complementary technologies such as metadata schemas, HTML web pages, and mainframe job control language (JCL). 

Tuesday, September 20, 2011

Survey shows European Decision Makers in Need of Better BI

Information Builders, the software vendor of the enterprise Business Intelligence product WebFOCUS, partnered with market research company Vanson Bourne for insight into how European companies make decisions and to evaluate the impact of those decisions upon business.

In this "No Barrier to Good Decisions" study, Vanson Bourne surveyed over 600 professionals in eight different countries across Europe. Almost all of the respondents (86%) felt that their companies could be making better decisions; 72% felt that a key to achieving faster decisions would be to have better access to information.

However, over half said it was not about more data, but rather easier access to the information already in the enterprise. Forty percent of the respondents wanted simpler BI tools and applications.

The study found that European managers spend over one hour each day looking for information. When viewed financially, this unproductive time costs European organizations with over 1000 employees about 8 million euros annually. The study found that marketing professionals spend even more time than managers looking for data--83 minutes per day (versus 67 for the managers).

Almost all of the people surveyed (88%) feel their organization does not give them the proper tools to do their jobs. 68% believe their company does not provide them with properly formatted information suitable for easy use.

The bottom line? Almost nobody in the survey felt they were capable of making good decisions based on the tools and data provided by their organizations.

Information Builders' Chief Marketing Office Michael Corcoran commented:

"If companies want to increase their competitiveness, they need to improve access to data across all levels of the company. They risk hampering any competitive advantage if they limit the availability of data for business decisions to desktop or laptop users and overlook other mobile devices. In addition, they have to be realistic about the level of control they will have over the devices the data is being accessed on.
At the heart of supporting this goal is the need to create the right infrastructure and support to enable broader access to data over an ever-increasing number of devices, either in real time or using a local device cache." 

Good Business Intelligence goes back to the simple formula of providing the right information at the right time to the right person so he or she can make the right decision and take the right action. Of course, this survey shows that executing a BI initiative effectively is the difficult part. 


For more information, see the press release

Friday, September 2, 2011

Starbucks Urges Fragile America to Drop Partisan Government Ways

Howard Schultz, the CEO of Starbucks, urges Americans to speak out against the partisanship in our government.

See his letter.

Tuesday, August 30, 2011

BI Professionals using Social Networking such as LinkedIn

In a recent study, Jobvite found that 80% of the people they surveyed said their companies use social media sites for recruiting, with the main information sources being LinkedIn, Facebook, and Twitter.

When Jobvite first did this survey four years ago, recruiting through social media was a new idea. Today, it appears to be the predominant method. In fact, almost all (95%) of the 800 U.S. respondents said that they had successfully hired an individual through LinkedIn.

My LinkedIn network has about 2000 direct contacts, many of whom are somehow related to the software industry. Unlike with my Facebook network, I am pretty open to LinkedIn invitations to connect. That brings my "friends of friends" LinkedIn network to over one million people. Of the 120 million people using LinkedIn, I can use my network to search about 16 million of them.

So I was curious to see the statistics for networked individuals displaying business intelligence keywords in their LinkedIn profiles. To keep it manageable, I decided to only look at those individuals living in the United States. I also removed vendor employees from the figures (e.g., for the keyword "Cognos," I ignored individuals who worked for either IBM or IBM Cognos).

Here are counts from my quick research of BI skills claimed by LinkedIn members in my network:
  • SAS: 59,300 people living in the USA
  • Crystal Reports: 43,600 people
  • IBM SPSS: 37,000
  • Business Objects: 34,500
  • IBM Cognos: 28,700
  • Microsoft SSRS: 17,000
  • MicroStrategy: 8,300
  • Oracle OBIEE: 4,200
  • Actuate: 3,900
  • QlikTech Qlikview: 1,400
  • Information Builders WebFOCUS: 1,300
  • BIRT: 1,200
  • Pentaho: 900
  • JasperSoft Jasper Reports: 800


Of these 14 BI products, 80% of the LinkedIn keywords clustered around two statistical analysis products (SAS and IBM SPSS) and the software products from two mega-vendors (SAP Business Objects and IBM Cognos).

The other mega-vendors Microsoft and Oracle were not far down the list. In fact, I might have slighted them by not being able to better search for the product keywords. 

Surprisingly, I saw a lack of R Stat people in LinkedIn despite the open-source statistical package being in the news and high demand of commercial products--only 126 people in the United States mentioned that skill in their LinkedIn profiles.

With so many people with SAS and SPSS skills, why would there be so few with an open source statistical package? 

Supply and demand must be at play here. If the market has limited demand for open-source products, fewer people will bother learning them. The other open-source BI products were also very far down on the list.

Of course, this is just a snapshot in time of the BI software skills in the United States. Some of these skills will decline in number and some will increase. For example, I believe that legacy tools such as Crystal Reports skills will move down the list and emerging products such as Qlikview will move up.

The bottom line for BI professionals is to maximize your opportunities by participating in social media--especially LinkedIn.




To get your own free copy of this Jobvite survey, click here. For a great infographic, see this.

To see my comments about Information Builders's BI product, go to my WebFOCUS blog

Saturday, August 27, 2011

BARC Summarizes BI M&A

In a free analysis, BARC's BI Verdict site offers a nice summary of the business intelligence industry's merger and acquisition activities going all the way back to the 1994 purchase of Pilot Lightship by Dun & Bradstreet.

You can see it here.

Tuesday, August 16, 2011

BI and Car Wrecks

This is the 120th anniversary of an amazing event: the very first automobile accident.

If you do not know about this, be sure to read my earlier post titled "Hitting Stumps." Of course, wrecking cars is just a lead-in to argue that large companies often wrestle away good ideas from innovators and stomp the life out of them.

As an illustrative hands-on example, type in the old URL for Cognos (http://www.cognos.com) or Business Objects (http://www.businessobjects.com).

That's okay, I'll wait for you.

Welcome back. When you tried to go to those product websites, you were instead redirected deep into the bowels of the mega-vendors which years ago acquired those two leading BI firms. Since then, neither brand has displayed much innovation. Both have been too busy being assimilated into their respective 800-pound gorilla corporate cultures.

Few pure-play BI software vendors remain in the industry: Information Builders and MicroStrategy are two that come to mind. It is probably not surprising that these independent vendors have been remarkably innovative lately, jumping into emerging markets such as mobile BI and social media analytics.

But enough about today's BI software, read about the first car wreck that happened 120 years ago. 

Monday, August 15, 2011

R Stats Users Group 2012

This week starts the 2011 useR! conference at the University of Warwick in Coventry in the United Kingdom. Next year, this annual R Stats users group will be held in Nashville, Tennessee, during June of 2012.

And the host already wants an RSVP.

Vanderbilt's Biostatistics group in the School of Medicine must be predicting a large crowd. They have posted a survey focused on where all of the attendees might spend the nights.

If you intend to go, visit their website.


WebFOCUS Closed-Loop Mobile BI

Information Builders, the vendor of enterprise BI product WebFOCUS, announced today "closed-loop BI" support for mobile devices.

This is not a new skill for Information Builders. From the start, their products have had features for full application development tasks. Not only can their BI product display information content to the users, but it can interact and gather raw data. Closed-loop means that the user can view information as well as change it.

While some organizations are concerned about BI applications that update data, others embrace it. For one Partner Intelligence client, we implemented a dashboard solution that provided their purchasing group with suggestions for saving money. However, if a purchasing decision maker disagreed with the automated recommendation, he or she could click on that line and cause it to disappear from that and future reports.

That "closed-loop" reporting feature updated a flag in the database identifying that piece of information to be excluded from the report. It was an easy task with WebFOCUS, but may have been near impossible with a typical report writing software product.

WebFOCUS also provided a way for that client to enter currently-not-available information. The purchasing person had some inside information about a future agreement that was not yet in the ERP system. Using a BI data entry screen, this person could add this piece of breaking news into the application that would influence the savings advice for everybody the next day.

During nightly processing, WebFOCUS picks up the new information, builds a formal transaction, and passes it along to the ERP system so that this new information can become active. Had the client needed more real-time synchronization, we could have made the ERP communication happen on a data-change basis.

Many people still dismiss handheld devices such as phones and tablets as being devices only good for "consumption." They want to pigeon-hole mobile devices to reading information, unable to process transactions and update data. Information Builders may help change this attitude.

For more information on Information Builder's closed-loop mobile BI offerings, see their press release

Tuesday, April 26, 2011

Free Forrester Research on Future of BI

Endeca is offering a free Forrester Research on the future of Business Intelligence.

In it, Boris Evelson talks about the BI trends he sees for this year and beyond. In fact, Boris predicts that BI in the year 2020 will "be so different as to be unrecognizable."

Here is the executive summary:
Forrester continues to see ever-increasing levels of interest in and adoption of business intelligence (BI) platforms, applications, and processes. But while BI maturity in enterprises continues to grow, and BI tools have become more function-rich and robust, the promise of efficient and effective BI solutions remains challenging at best and elusive at worst. Why? Two main reasons: First, BI is all about best practices and lessons learned, which only come with years of experience; second, earlier-generation BI approaches cannot easily keep up with ever-changing business and regulatory requirements. In this research document, Forrester reviews the top best practices for BI and predicts what the next-generation BI technologies will be. We summarize all of this in a single ├╝ber-trend and best practice: agility. BP pros should adopt Agile BI processes, technologies, and architectures to improve their chances of delivering successful BI initiatives.

As always, Boris effectively communicates his insight into the BI software market. Be sure to download this free research paper today.

Friday, April 15, 2011

BI Dashboards with QMF Enterprise Edition

During business introductions, I used to provide people with the exact year that I started working in the software industry. But then I noticed the youngsters in the room would get wide eyes as they did mental calculations.

But just to give you a milestone, I started "coding" after Ted Codd had formulated the System Relational which would be the foundation of an entire relational database industry based on SQL, the structured query language. I also started after IBM released an implementation of the relational database on their mainframe VM/CMS platform, called SQL/DS. My first corporate job was about the time IBM released another version of the relational database for their MVS/TSO platform, called DB2.

Way back then, IBM also developed a SQL database reporting tool called QMF, which stood for Query Management Facility.

QMF has stood the test of time. It is still in use out there on mainframes and has not been forgotten, either by technical users or the software vendor. In October of 2010, IBM released QMF Version 10 with significant new features.

Like me, Mike Biere is a fellow Cincinnati resident who has been in the industry for a while. A senior manager with IBM, Mike recently wrote an article in the April/May 2011 issue of the z/Journal magazine providing readers with details that they might not know about QMF.


For example, did you know that you can build graphical web dashboards with QMF?


This is not your Father's QMF. The green-screen dumb terminals are a thing of the past.

Like the 4GLs that emerged about the same time in mainframe history, Mike reminds us that "QMF was initially designed to be a straightforward query and reporting tool. The assumption was that the information provided to the users from source tables was suitable for reporting purposes. It was intended to provide relief from the constant pressures users placed on IT to produce reports."

Today, IT has new pressures. For example, their business users expect to be able to interact with data visually and through a web interface. IBM has added those features to a new QMF Enterprise Edition for either a rich workstation client (Windows, Linux, and Solaris) or a thin client (Windows, Linux, Solaris, AIX, HP-UX, iSeries, Linux on System z, and z/OS).

Mike wants you to know about the latest changes to QMF, such as:
  • New Metadata Layer (definitions/virtual data sources)
  • Visual, Interactive Dashboards 
  • Enhancements for Business Analytics (150 new functions and OLAP support)
  • New Data Types and Sources (JDBC support)
  • New Data Environments Feature
  • Enhanced Charting Creation and Deployment (QMF for Workstation)
  • Technical Core Improvements


For more information about IBM's latest enhancements to QMF, visit their website.

Also, be sure to check out Mike's recent book, "The New Era of Enterprise Business Intelligence," which was released in August 2010 and is available in both physical and mobile e-book reader formats.

Thursday, April 7, 2011

Living in the Cloud

I had not really planned to move to the cloud, but I find myself there.

While I am still tethered to Microsoft technology for some important Windows application development activities, mobile devices and universal user interfaces are pulling me away.

One major step in my new life in the cloud was "exchanging" Microsoft Exchange/Outlook. Instead of having a personal notebook computer bogged down with gigs of Windows software and years of e-mail attachments, everything is now available from the cloud using Google Mail.

My Google Mail, Calendar, and Documents are available from any of my mobile devices or from any computer with a web browser.

In addition to Google Docs, I have started using a free Dropbox account, which allows me to store any type of document, access them from any desktop browser or mobile device, and share them with associates by disclosing a simple URL. While I like Google Docs, Dropbox's easy integration with the Windows desktop gives it an advantage.

Many of my favorite software applications have moved to universal user interfaces.

Amazon was one of the first company I noticed advertising the value of this. Their commercials show people reading electronic books on any number of mobile devices: the Amazon Kindle, the Apple iPad, iPhone, Android phone, etc. The announcer says that it does not matter if you change phone providers; you can still get to the books you bought on Amazon.

Another great universal application is TripIt. I can easily create travel plans just by forwarding my e-mail reservation confirmations from hotels, car rentals, and airlines to TripIt. I can then view and manage my itineraries from my mobile phone, iPad, or desktop.

Likewise, software vendor Appigo has created a great ToDo application (based on David Allen's GTD-Get Things Done time management methodology) with a universal front-end. While I used to manually sync up my Microsoft Exchange with the iPhone and iPad ToDo packages, I now pay just $20 a year to keep my to-do lists in the cloud. From this central location, all of my access devices are automatically synced. Like TripIt and the Amazon Kindle, I can e-mail things to a personal ToDo account.

I spend quite a bit of time with LinkedIn, which has a very good web browser user interface. On the mobile devices, LinkedIn does an okay job with their iPhone version but they have not yet invested in an iPad application.

Even my music listening is moving to the cloud. Thanks to my oldest son who gave me a web-enabled Blu-ray device,  I can listen to Pandora music streamed through my television. If I were in my den or in my car, I would use Sirius/XMRadio, but more and more I rely on web providers such as AccuRadio or Pandora. Even when mowing the yard, I stream music through my iPhone (sorry, AT&T--but thanks for giving us early adopters those unlimited data plans).

(Note: I will put in a special word of thanks to Sirius/XMRadio for recently rewriting their mobile music application.)

Within the immediate family, we have Windows desktops, notebooks, and netbooks; Apple iPads, iPhones, and MacBooks; Amazon Kindle e-book readers; Nintendo DSi handhelds; Android phones; and web-enabled televisions.

Now that's universal access to the cloud!

Tuesday, April 5, 2011

Data Security Breach Impacts Major Companies (and Millions of Consumers)

Dallas-based Epsilon, the marketing services firm that had a security breach involving consumer e-mails, is not disclosing which companies were involved in the fiasco. Reports indicate that up to 50 firms were impacted, including the financial giants JPMorgan Chase and Capital One.

Yesterday, my inbox was full of apologies from major companies (all pointing a finger at Epsilon) such as Hilton, Walgreens, Best Buy, Disney, Eddie Bauer, and The Kroger Company.




According to Rik Ferguson, director at security software vendor Trend Micro, I should be worried.  Here is what Rik wrote in his blog:
"Not only do the criminals know your name and email address, they know where you go shopping, where you bank, which hotels you stay at and much more. If you are unfortunate enough to have received multiple notifications, just imagine what kind of profile is now in criminal hands."

News reports indicate how common this type of marketing outsourcing is in the industry. Many companies hand over their customer data to a third-party specialist and say, "here, you take care of this for us." Knowing that, hackers have an easier target; they can go after the little services providers instead of the big guys. Instead of just getting one firm's data, they can snag fifty at a time.

Some experts think that this Epsilon event may change the way companies outsource their marketing mass mailings. See ComputerWeekly for an article.

Just imagine the potential value the hacker sees in obtaining the e-mail addresses of every consumer of dozens of the U.S.'s major firms.

Data is valuable and thieves steal valuable things. They then sell it to other crooks who intend to make money off of the stolen merchandise. At some point, their money-making scheme might involve contacting the stolen e-mail recipients and trying to get access to their money.

Hopefully, you use an e-mail provider that is very adept at identifying scams such as phishing. However, with a complete profile of your buying habits, the crooks can now "spear-phish," a term used for a type of phishing scheme that goes after a targeted victim. If you were an Epsilon multiple victim like me, you will need to be personally vigilant against a targeted scam.

Reuters provides some things to do to prevent being a victim.

This should also be a warning to you about the data you store for Business Intelligence. It is valuable and you need to implement the proper safeguards to protect it from unauthorized access. 

Monday, April 4, 2011

Software Advice Analyzes Vendor-Speak

Hunter Richards, analyst at Software Advice, wants to help you understand your software vendor's marketing language. Like the Gartner Magic Quadrant, Richards positions their marketing terms according to an intersection point on two axes.

Richards evaluates a variety of market terms according to 1) the ability to repress meaning, and 2) the completeness of overuse.

For example, software vendors who completely overuse marketing jargon that is totally devoid of meaning earn a position in the "Gibberish" quadrant (as opposed to "Creative Garbage").

To read the full article and to see where your favorite software vendor falls, see Richard's full article

PNC Bank Talks about BI

In a recent February 2011 presentation and round-table in Pittsburgh, PNC Bank discussed their BI experiences. The BI Xchange website provided a summary of that, with comments such as:
  • Executive support for BI is strong 
  • Purchased a data warehouse appliance to speed BI queries 
  • Standard reporting that came with pre-built applications did not meet their needs 
  • Mobile BI is an issue to banks due to security concerns
  • Looking into "Certified Reporting" to confirm figures in various reports are correct 
  • Using an OnShore/OffShore BI application development model 
  • For BI resources, start with good business people and teach them the BI tools (and warnings about that statement since there may be a reason these individuals have not chosen to be involved with the technology) 


The BI Xchange provides this summary of their organization:

The BI Exchange is a platform for professionals interested in exchanging knowledge, ideas and experiences about the field of Business Intelligence (BI). It is a community of both management and technology professionals with a focus on increasing awareness of how BI can transform the business processes. 
We hope that this community will foster knowledge sharing and in the long run we will accumulate a repository of best practices freely available to the entire BI community.


I will comment on just one of their topics: "Certified Reporting." Many organizations employ multiple reporting specialists creating documents with figures that do not match those of other documents. This is especially true in firms where the IT group has not taken an active BI role, leaving the business units to fend for themselves. During one BI assessment, a business reporting specialist confessed to me, "Nobody can prove me wrong."

In other words, there was no easy way to validate or deny the figures he put onto a report. Without a formal standard for business terminology or calculations, this individual could almost make up the results. In fact, when his sponsors did not like the answers on the reports, they gave him different business rules to follow. This is a serious problem.


Companies must formally define how their BI reports are created and have a single version of the truth.

Thursday, March 31, 2011

Press Release: New Strategic Relationship with WebFOCUS Vendor


NEW YORK, NY – March 30, 2011 – Information Builders, an independent leader in operational business intelligence (BI) solutions today announced that Partner Intelligence, a full service BI and data warehouse solutions company, is expanding its role as an Information Builders' Solution partner. Partner Intelligence will continue recommending and implementing WebFOCUS, Information Builders' comprehensive business analytics platform, in customer situations while also tapping the solution to help organizations modernize their existing BI applications.
Information Builders and Partner Intelligence began working together four years ago. Partner Intelligence was already successful, implementing WebFOCUS to Information Builders' new customers while expanding the platform's use with existing customers. Under the terms of this new agreement, Partner Intelligence will now market WebFOCUS to its customer base to dramatically reduce the time, cost, and risk of implementing web-based business intelligence. Partner Intelligence selected WebFOCUS as its BI tool of choice because of its vast experience with the platform and successful joint implementations at organizations like Wendy's/Arby's Group Inc., where it used WebFOCUS to implement an enterprise dashboard, and the State of Oklahoma Department of Human Services (OKDHS), where the company had automated a legacy reporting system conversion into WebFOCUS.
"When I engaged Partner Intelligence to do a major project to convert hundreds of production jobs into WebFOCUS, it seemed like an unattainable undertaking," explained Richard Smith, IT manager at OKDHS. "Not only was I wrong, but it was one of the smoothest projects on which I have had the pleasure of working in years. If you need to convert code, then Partner Intelligence is the company you need to work with!"
"Partner Intelligence provided expertise and skill to get our enterprise dashboard up and running quickly, enhancing functionality and giving our management a clear line of sight into the company's metrics," said Dave Ascah, vice president, IT Enterprise Solutions at Wendy's/Arby's Group.
Doug Lautzenheiser, general manager of Partner Intelligence, said: "We are excited to expand our relationship with Information Builders and the range of capabilities we can offer around WebFOCUS. Organizations looking to replace legacy reporting systems or implement enterprise BI dashboards will benefit from this new partnership."
"Partner Intelligence has had success automating complex conversions of legacy technologies into WebFOCUS," said Gerald Cohen, president and CEO of Information Builders. "Our customers are quite happy with Partner Intelligence's capabilities and we are pleased with the partnership. With our new agreement, we're making it even easier for Partner Intelligence's clients to implement web-based, enterprise business intelligence."
About Partner Intelligence
Partner Intelligence is the BI practice of Partner Professional Staffing. Based in Cincinnati, Ohio, Partner Professional Staffing is a full-service professional services firm, offering a variety of recruiting, staff augmentation, and project-based consulting services. Partner Intelligence provides clients with consultative services, specializing in strategic BI advice, BI application development, and automated BI system conversions.
About Information Builders
Information Builders provides software and services that bring smarter decision-making and streamlined processes to leading organizations in business, government, and education worldwide. The company's software is installed in tens of thousands of locations, servicing millions of users. Its WebFOCUS business intelligence (BI) platform delivers massively scalable information applications, analytics, and customer-facing portals to the new generation of information users to deliver game-changing business results. Its iWay Software integration platform allows enterprises to leverage all of their information resources to streamline internal and business-to-business processes while escalating the overall accessibility and integrity of information regardless of environmental complexity. Headquartered in New York City with 60 offices worldwide, the company employs 1,350 people and has established significant industry partnerships. Visit Information Builders at informationbuilders.com. For ongoing news, go to informationbuilders.com/new or follow Information Builders at @infobldrs.

Friday, March 25, 2011

BI: That was Then, This is Now

In his blog titled "BI Market Evolution," Wayne Eckerson says:

With an endless wave of new companies pushing innovative new technologies, the BI market has been one of the most dynamic in the software industry during the past 20 years.
...[T]he plethora of advertisements about BI capabilities that appear on television (e.g., IBM's Smarter Planet campaign) and major consumer magazines (e.g. SAP and SAS Institute ads) reinforce the maturity of BI as a mainstream market. BI is now front and center on the radar screen of most CIOs, if not CEOs, who want to better leverage information to make smarter decisions and gain a lasting competitive advantage. 

To prove this point, Wayne created a great timeline of the Business Intelligence industry mapping the changes in Users against those of the Vendors.

Prior to 1990, Wayne tells of how companies used 3GL (COBOL, PL/1, etc.) and 4GL (NOMAD, RAMIS, and FOCUS) computer programming languages to handle reporting needs.



He sees 1990 as the point where software vendors began offering tools built specifically for Business Intelligence. Around 2000, the vendors started offering BI applications for managing corporate performance.

Wayne maps out the following key time periods in the BI market's evolution:

  • 1990: Get the Data (data warehousing and BI "suites")
  • 1995: Use the Data (business intelligence platforms)
  • 2000: Optimize Performance (performance management)
  • 2007: BI Goes Mainstream (consolidation; Oracle, SAP, and IBM get in) 
  • 2010: Drive Processes (analytics) 

What's left in the future? Wayne talks about continued changes:
At this point, some might wonder if there is much headroom left in the BI market. The last 20 years have witnessed a dizzying array of technology innovations, products, and methodologies. It can't continue at this pace, right? Yes and no. The BI market has surprised us in the past. Even in recent years as the BI market consolidated--with big software vendors acquiring nimble innovators--we've seen a tremendous explosion of innovation. BI entrepreneurs see a host of opportunities, from better self-service BI tools that are more visual and intuitive to use to mobile and cloud-based BI offerings that are faster, better, and cheaper than current offerings. Search vendors are making a play for BI as well as platform vendors that promise data center scalability and availability for increasingly mission-critical BI loads. And we still need better tools and approaches for querying and analyzing unstructured content (e.g., documents, email, clickstream data, Web pages) and deliver data faster as our businesses increasingly compete on velocity and as our data volumes become too large to fit inside shrinking batch windows.

Sounds like lots of fun is still to come. 

Thursday, March 24, 2011

HP in the BI Space

Might HP become one of the BI mega-vendors? Christopher Baum at Software Advice suggests that it might happen.  He posted a great analysis of merger and acquisition choices for HP. 


Baum plotted a timeline from 2000 to the present showing HP's acquisitions in five major categories:
  • Enterprise Infrastructure
  • Services
  • Personal Systems
  • Software
  • Imaging and Printing 


Two of their really big buys were EDS and Compaq, providing HP with more corporate services capabilities and hardware platforms. Many of its recent software acquisitions deal with corporate IT application management. Let's also not forget that last year HP acquired the struggling handheld maker Palm.

Could HP get into more corporate computer applications? Baum suggests that HP might acquire the ERP giant SAP. Now that would be a really big purchase.

Like EDS, SAP is a large services organization (accounting for around 21% of its total $12.5 billion euro revenues in 2010). Of course, buying SAP would also propel HP into the hot Business Intelligence software space since SAP had earlier acquired Business Objects (which had acquired Crystal Reports and others).

Baum also recommends some other choices from the BI vendor menu.

Take SAS for example. Owner Jim Goodnight has been going strong since starting this software company based on a college project. Now that he is about 70 years old, he might want to celebrate his amazing success and retire. HP could offer Jim a few billion dollars as a retirement gift and SAS would be a great analytical tool for HP.

A similar situation exists at Information Builders. In business since the mid-1970s, Gerry Cohen might also be interested in winding down his time in the software industry. Gerry's WebFOCUS is a good fit for HP's platforms all the way down to the Palm handheld. With WebFOCUS, HP might even have a play for mobile BI.

Baum also suggests that HP might acquire MicroStrategy, one of the few remaining pure-play BI software vendors. Because MicroStrategy does a great job with very large database systems such as Teradata, HP might want to consider acquiring both as complementary BI offerings. MicroStrategy is also one of the emerging leaders in the mobile BI space, which could be of value combined with Palm handhelds.

Baum goes on to list all sorts of different purchases for HP. He says:
With all of these components, HP becomes the only firm that can supply end to end enterprise IT, including hardware and software, either on premises or under the Infrastructure as a Service model. HP could supply all of the IT hardware, including smart phones, handhelds, desktops, workstations, and servers. The expanded BTO offerings allow complete control and situational awareness of the enterprise’s computing and communications systems. The enterprise business software would track, well, all of the business stuff. HP can leverage its existing business continuity sites to jump start an Infrastructure as a Service offering. Adobe technology would allow enterprises to encapsulate business processes within documents and 10gen would store it all. Autodesk, Hermes, and Corel would drop directly into their respective business units.


Software Advice asks for your opinion. Go to their website to read the entire article and to answer a survey on which companies you think HP will acquire.

Wednesday, March 23, 2011

The Three Threes of BI Dashboards

Wayne Eckerson likes the number three. At least, he uses it alot in his book, "Performance Dashboards."

Wayne, research director at TechTarget in Boston, writes:
One of the most salient features of performance dashboards are the "three threes": three applications, three layers, and three types. The "three threes" provide a convenient way to describe the major characteristics of performance dashboards and a litmus test to differentiate imposters from bona fide performance dashboards.

Wayne breaks a performance dashboard into three integrated application components:

  • Monitoring application for conveying information at a glance (dashboards for operational processes and scorecards for strategic goals) 
  • Analysis application for exposing exception conditions and allowing the user to drill into details
  • Management application for improving alignment, coordination, and collaboration 


A dashboard can also be segregated into three different informational layers:

  • Monitoring information utilizing graphical, metrics data for executives 
  • Analysis information utilizing summarized, dimensional data for analysts 
  • Detail information utilizing transactional data for workers 


Lastly, Wayne outlines three types of dashboards:

  • Strategic dashboards for executives (often using the Balanced Scorecard methodology)
  • Tactical dashboards for departmental processes and projects 
  • Operational dashboards for front-line workers utilizing detail transactional 


There is plenty of overlap between Wayne's three informational layers and three types of dashboards. We mess up his "three" paradigm, but we might combine things into a summary such as:

  • Strategic Monitoring Dashboard for executives (highly graphical representative of metrics)
  • Tactical Analysis Dashboard for departments and projects (summary with drill-to details)
  • Operational Detail Dashboards for front-line workers (very current transactional data) 


For more information, see Wayne's book on Amazon.

By the way, if you are considering using WebFOCUS to build BI dashboards, be sure to see my other blog posting.

About Me

My photo

I am a project-based software consultant, specializing in automating transitions from legacy reporting applications into modern BI/Analytics to leverage Social, Cloud, Mobile, Big Data, Visualizations, and Predictive Analytics using Information Builders' WebFOCUS. Based on scores of successful engagements, I have assembled proven Best Practice methodologies, software tools, and templates.

I have been blessed to work with innovators from firms such as: Ford, FedEx, Procter & Gamble, Nationwide, The Wendy's Company, The Kroger Co., JPMorgan Chase, MasterCard, Bank of America Merrill Lynch, Siemens, American Express, and others.

I was educated at Valparaiso University and the University of Cincinnati, where I graduated summa cum laude. In 1990, I joined Information Builders and for over a dozen years served in regional pre- and post-sales technical leadership roles. Also, for several years I led the US technical services teams within Cincom Systems' ERP software product group and the Midwest custom software services arm of Xerox.

Since 2007, I have provided enterprise BI services such as: strategic advice; architecture, design, and software application development of intelligence systems (interactive dashboards and mobile); data warehousing; and automated modernization of legacy reporting. My experience with BI products include WebFOCUS (vendor certified expert), R, SAP Business Objects (WebI, Crystal Reports), Tableau, and others.