Friday, January 29, 2010

Guy Kawasaki's Art of Innovation

In Guy Kawasaki's latest book, "Reality Check," he lists 11 recommendations for companies trying to innovate.  Buy this excellent book for the first ten, but I will give you the last one:

Don't let the bozos grind you down.  The bozos will tell a company that what it's doing can't be done, shouldn't be done, and isn't necessary.  Some bozos are clearly losers—they're the ones who are easy to ignore. The dangerous ones are rich, famous, and powerful.  Because they are so successful, innovators may think they are right.  They're not right, they're just successful on the previous curve, so they cannot comprehend, much less embrace, the next curve.

Guy's comment really hits home for Partner Intelligence's automated BI conversion software.  Seven years ago, I was told it couldn't be done.  Once I did it, I was told that I shouldn't have.  On one hand, they said I was crazy for eliminating so much money that could be made with the status quo of rewriting applications manually.  On the other hand, they denied there was really any money to be made with automated conversions.

Yesterday, one of my clients disclosed that a BI software vendor told them (be sure to hear this sentence in your mind with a very sophisticated accent), "we once considered the idea of automating legacy report conversions but decided there was no market for it."

I was pleased that Guy called them bozos and I didn't have to do that myself.

Modernize QMF/SQL to Web-Based BI - Automatically Convert!

With the 2010 January 28th announcement by IBM and Information Builders on a partnership to provide WebFOCUS as the web-based BI product for mainframe DB2 data warehousing, now is the time to retire your legacy QMF/SQL applications.

Your legacy QMF/SQL applications can now be integrated into your internal and client-facing web portals and accessed on-demand via a web browser.  QMF/SQL application output can be HTML, PDF, Excel, and other standard web-supported formats.  Your legacy reporting applications can be dramatically enhanced with web-based BI features.

Partner Intelligence's BI Consolidator for QMF makes the transition quick and easy. If you are like many mainframe shops, you have decades of legacy applications; converting those into modern web-based technologies seems like an impossible dream. No worries.  Somebody else has done the hard work for you already by creating an automated translation application.

Today, you can automatically inventory, scan, and analyze your QMF procedures, forms, and queries, along with mainframe JCL that runs them. That provides you quick information on the scope of a modernization initiative along with a generated roadmap to follow.
After that, you can automatically convert your QMF applications. Currently, we have the following features in place to translate QMF procedures into comparable WebFOCUS syntax:
  • Comments (--) are preserved and saved as WebFOCUS comments
  • RUN commands are translated into WebFOCUS EXEC commands (processing FORM and symbolic variables)
  • ERASE commands are translated into SQL DROP TABLE commands
  • SET GLOBAL commands are translated into WebFOCUS –SET commands
  • Performing SAVE DATA AS, PRINT, and DISPLAY commands with WebFOCUS features (PRINT is a work-in-progress)
  • BOTTOM commands are ignored

Thanks to WebFOCUS's SQL Pass-Through feature, the BI Consolidator for QMF can leverage your existing SQL routines. If your QMF SQL routines work effectively and efficiently today, they will continue to do so running within WebFOCUS. This dramatically reduces the risk of modernizing your QMF applications -- we keep the core processes exactly the same.
In addition, some additional tasks are being performed for the SQL queries:
  • Comments (--) are preserved and saved as WebFOCUS comments outside of the SQL query
  • SET CURRENT SQLID is being issued to establish user defaults
  • If needed, SET DBSPACE is being issued to establish default database and tablespace
  • If needed, PREPARE SQLOUT is being issued so that WebFOCUS Developer Studio can paint the answer set
  • SQL columns are parsed so that proper names can be referenced when producing output files and reports

The BI Consolidator for QMF can also translate your QMF forms into valid WebFOCUS syntax.
  • Basic form information is being saved and presented as comments within the translated program
  • If there is no QMF form, creating default print output using the SQL columns
  • Respecting the sort sequence of the 1) SQL ORDER BY and then 2) the Form BREAKs and GROUPs
  • Respecting the formatting rules in the Form
  • Converting CALC columns into WebFOCUS syntax
  • Converting most QMF aggregate functions into WebFOCUS syntax (AVERAGE, COUNT, FIRST, LAST, MINIMUM, MAXIMUM, PCT, STDEV, and SUM)
  • Handling BREAK, GROUP, and OMIT instructions
  • Handling matrix reports (BY and ACROSS)
  • Converting T 1110 column instructions
  • Converting T 1210 page heading instructions
  • Converting T 1310 page footing instructions
  • Converting T 1410 report footing instructions
  • Converting T 1402 grand total instructions
  • Converting Break instructions (QMF syntax prior to V3.1)
  • Applying stylesheet instructions to replicate LEFT, CENTER, and RIGHT instructions (issue with using multiple alignments on same line)
  • Converting QMF symbolics such as: &PAGE, &TIME, and &ID

The BI Consolidator offers some additional general features such as:
  • Interactive, one-at-a-time translation and mass, hundreds-at-a-time translation
  • Options to turn on testing features such as code display (ECHO) and turn off code execution (XRETRIEVAL)
  • Option to apply automatic page numbers
  • Applying stylesheets to the report output format
  • Adding WebFOCUS output features such as HTML, PDF, Excel, Active Reports, and runtime user selections
  • Adding pre- and/or post-process steps to the QMF job
  • When needed, adds APP commands (HOLD, PATH, and APPENDPATH)
  • Option to identify specific DB2 subsystems
  • Performs mass string changes during translation process (as an example, change DB2 output tables starting with PROD_ to TEST_ to facilitate testing)
  • Provides an application browser to view programs, stylesheets, metadata, etc.
  • Provides a facility to easily generate stylesheets

I would be remiss to not include some type of disclaimer. Currently, some QMF features might not be automatically translated into WebFOCUS. That doesn't mean that it can't be done technically -- we just have not automated the process yet.  Here are some current items that are being excluded from the translation:
  • Some aggregate functions in the QMF form, such as: CPCT, CSUM, STDEV, TCPCT, and TPCT (flagging as needing manual intervention)
  • Some symbolic variables such as: &ROW and &COUNT (waiting on actual client usage for examples)
  • Some detail formatting rules, such as on which line to place grand total
  • Embedded TSO commands in the QMF procs are currently flagged for manual intervention (no usage at clients yet; parsing/generation TBD)

Tuesday, January 26, 2010

IBI and IBM Partner for System z Data Warehousing

Today, Information Builders and IBM announced a new Business Intelligence partnership for data warehousing on the System z platform.  See the press release.

The new "Data Reporter for DB2" is a special WebFOCUS version specifically for System z DB2 data warehouses. 

Just a few years ago, IBI and IBM announced a similar partnership with the "DB2 Web Query" product -- a special WebFOCUS version made for System i data and intended to be a next-generation replacement for Query 400. 

This announcement raises some questions.  For example, if WebFOCUS supercedes Query 400 on the System i, is this same modernization message true for its mainframe peer Query Management Facility (QMF)?  A bigger question is, why didn't IBM just use its own BI product, Cognos, which also runs on the System z?

You can also read about this on Information Builders' website.

Monday, January 25, 2010

InformationWeek's 2010 Top 10 for CIOs (with a BI twist)

In the 2010 January 8th edition of InformationWeek, Bob Evans listed the top items of concern on CIO's minds in this new year. They include:
  1. The cloud imperative
  2. The 80/20 spending trap
  3. CIO-led revenue growth and customer engagement
  4. Mastering end-to-end business processes
  5. Business Intelligence and Predictive Analytics
  6. External information versus internal information
  7. CIO priorities, CIO compensation, CIO evaluation
  8. Vendor consolidation, with radical exceptions
  9. The mobile enterprise
  10. The transformation quotient

While Bob explicitly lists Business Intelligence as his number 5 item, I can see that BI can play a role in each of his topics.

If CIOs were to draw up New Year's resolutions specifically for their Business Intelligence initiatives, I would suggest:
  • Reduce software license and support costs by retiring legacy reporting tools (e.g., FOCUS, RAMIS, NOMAD, QMF/SQL, DYL280, and Crystal Reports)  
  • Implement and socialize a formal BI strategy that identifies your official BI Team, BI Policies and Procedures, BI Infrastructure, and BI Project Roadmap
  • Choose and enforce a standard BI technology (e.g., IBM Cognos, SAP Business Objects, Oracle, Microsoft, Microstrategy, Actuate, Information Builders' WebFOCUS, or open source)
  • Implement web-based BI that provides users with on-demand, self-service access to information (replace IT report development with easy-to-use browser parameterized user applications)
  • Ensure that web-based BI applications are developed to work seamlessly on mobile devices such as the iPhone
  • Implement an automated information delivery mechanism ("push" BI to internal and external users through e-mail, printers, intranet sites, report libraries, etc.)

For more information, contact me directly.

Sunday, January 10, 2010

Hot Business Intelligence Software for 2010

If your New Year resolution is to learn a hot Business Intelligence software product, which product should you chose? If I base my answer on the growth of job openings in Monster, you would be smart to consider SAS, MicroStrategy, and Oracle's OBIEE.

These three BI products are not necessarily direct competitors; instead, they fit into different BI areas: SAS for statistical analysis, MicroStrategy for ad-hoc reporting against large data warehouses, and Oracle for a variety of enterprise database reporting.

The number of jobs posted in Monster for Oracle's OBIEE doubled from January 2009 to 2010, going from 74 to 158. Not a huge number of jobs, but a positive trend for Oracle.

Job postings for MicroStrategy went up dramatically, from 136 at the beginning of 2009 to 227 a year later. That is a good 40% increase.

But the hottest BI technology continues to be SAS, with 1368 Monster job postings in January 2010 compared to 1195 twelve months earlier. No other BI software product had this type of job demand throughout the year. You would only see this volume of job postings for general application development tools such as Java, C#, C++, and .NET technologies.

A year ago, the closest runner-up to SAS for BI products would have been Crystal Reports with 985 job postings. But client/server desktop reporting technologies are a thing of the past, so demand for Crystal Reports plummeted throughout the year until in January 2010 we only see 608 job postings, a 62% decline.

SAP's Business Objects held steady with 729 jobs in 2009 and 738 in 2010, while demand for IBM's Cognos product dropped 23% from 823 to 667 job postings. This means that Business Object overtook Cognos in the number of Monster job postings and becomes the 2010 #2 demand under SAS.

Demand for Microsoft's Reporting Services had a slight decline, going from 518 job postings in January 2009 to 483 twelve months later. Actuate's low demand continued to drop, going from 62 job postings in January 2009 to only 48 a year later (a 29% decrease).

Information Builders' WebFOCUS product had a positive trend, but the total demand for professionals across the United States was tiny -- just 15 job postings in January 2009 while 18 a year later.

Demand for smaller products such as BIRT, Pentaho, JasperSoft, and QlikTech just never took off throughout the year.

So if you are going to add hot BI skills to your resume in 2010, consider statistical analysis with SAS, ad-hoc reporting with MicroStrategy, and enterprise reporting with Oracle's OBIEE.

About Me

My photo

I am a project-based software consultant, specializing in automating transitions from legacy reporting applications into modern BI/Analytics to leverage Social, Cloud, Mobile, Big Data, Visualizations, and Predictive Analytics using Information Builders' WebFOCUS. Based on scores of successful engagements, I have assembled proven Best Practice methodologies, software tools, and templates.

I have been blessed to work with innovators from firms such as: Ford, FedEx, Procter & Gamble, Nationwide, The Wendy's Company, The Kroger Co., JPMorgan Chase, MasterCard, Bank of America Merrill Lynch, Siemens, American Express, and others.

I was educated at Valparaiso University and the University of Cincinnati, where I graduated summa cum laude. In 1990, I joined Information Builders and for over a dozen years served in regional pre- and post-sales technical leadership roles. Also, for several years I led the US technical services teams within Cincom Systems' ERP software product group and the Midwest custom software services arm of Xerox.

Since 2007, I have provided enterprise BI services such as: strategic advice; architecture, design, and software application development of intelligence systems (interactive dashboards and mobile); data warehousing; and automated modernization of legacy reporting. My experience with BI products include WebFOCUS (vendor certified expert), R, SAP Business Objects (WebI, Crystal Reports), Tableau, and others.