Sunday, October 12, 2008

Leave a Legacy

Despite living in the 21st century, companies today still use a variety of legacy reporting tools created in the 1970s and 1980s. Early software vendors created products such as the 4GLs (FOCUS, NOMAD, and RAMIS), QMF, and DYL280 for use on specific host computers, typically a mainframe platform such as MVS/TSO or VM/CMS. They of course designed these tools according to and limited by the capabilities of that decade's technology.

Companies had to train and support reporting end users to log into the mainframe environment and use the command-line processes and menu screens intended for data processing professionals.

Mainframe tools naturally lacked graphical user interfaces (GUIs) and instead used character-based “green screens”. Because the mainframe environment did not communicate with end-user tools such as Microsoft Office, any integration with spreadsheets, documents, or presentation slides was accomplished manually. Likewise, only that data on the mainframe platform could be accessed (e.g., typical data sources include sequential files, tape cartridges, VSAM, IMS, DB2, and others).

The new web-based BI products, on the other hand, can be architected as multi-tier environments, separating the user experience from the backend data sources. Just because the data and access mechanisms are on a mainframe does not mean that the user must use a dumb, green-screen terminal. Instead, the user can interact with the BI engine via a web tier and a simple GUI web browser. The BI engine can automatically pass the formatted reports back to the user in the desired output formats, such as HTML, Microsoft Word and Excel, PDF, and others.

The complexity of using a mainframe operating system is now hidden from the end user and performed automatically by the BI system. In addition to accessing the mainframe legacy data, the BI engine is now typically able to reach a wide range of enterprise data on multiple platforms. Relational database management systems (e.g., Oracle, UDB, Sybase, Informix, and MySQL), multidimensional cubes (e.g., Essbase), and other non-legacy sources can now be used in BI requests. In addition to data, modern BI engines are also able to communicate with enterprise packages (e.g., PeopleSoft, SAP, and Oracle) to easily obtain application data.

With the proper web security in place, enterprise data can now be shared easily with clients, suppliers, business partners, and others who were outside of the mainframe environment. Reporting tools are now online at a corporate website for the proper individuals, enabling them to perform business intelligence as easy as ordering a book from Amazon.

In 2008, it has become increasingly hard for a firm to support a reporting tool that was installed before 1988. One reason is the dearth of knowledgeable individuals in old BI technologies such as FOCUS, RAMIS, and NOMAD. Take COBOL as an example of a technology that is still in widespread usage in businesses today. Despite the need, few colleges teach COBOL to their students. If universities no longer teach a critical business programming language, how can we expect them to instruct on an obscure 4GL like RAMIS?

If a company still uses NOMAD to run mission-critical human resource and payroll functions on the mainframe, will they really be able to hire somebody to replace Harold, the life-long 4GL programmer, when he retires next year? This is a significant business problem for companies using legacy reporting tools.

Some products have been traded among vendors like bent Topps baseball cards of little-known right fielders. NOMAD and SQR are good examples. RAMIS and others were acquired by Computer Associates. The original creator of the FOCUS 4GL still owns the technology, but they have effectively replaced it with a web version. The legacy products may get a frequent white-washing and some new features painted on occasionally, but they still fail to provide today's organizations with current BI functionality.

You may already know this dirty little secret: the vendors of legacy BI tools do not really sell these products anymore. Nobody today buys a reporting language whose integrated development environment consists solely of a text editor. Instead, these vendors hope their legacy customers will continue to hold onto these cash cows so they can be milked for annual license fees, which is typically 15% to 20% of the purchase price.

I worked with a Fortune 50 financial services firm that had installed NOMAD in the late 1980s. At one point this company had over 500 4GL users, but after 20 years the usage had slowly declined to about 100 people. Few of these individuals were the original developers; they had merely inherited programs when earlier workers had left. Few people knew the actual workings of these important business processes. Few were excited about having to do this support work.

Manually converting these applications posed a major obstacle to this organization. In order to justify moving to newer BI technology, we had to reduce the time, cost, and skill requirements by building an automated translation process.

Another Fortune 50 client has over 800 business users supporting FOCUS applications and they have struggled for a decade trying to manually replace this legacy tool with a Business Objects product. A large Fortune 500 insurance company has a similar situation with over 500 FOCUS users. Decisions come down from on high saying, "It's just a reporting tool. Throw it out and replace it with Business Objects." Sorry, it's not that easy.

I worked with an IT group who had been given the "just replace it" directive. Unbeknowst to them, savvy users in remote manufacturing locations had used the 4GL to build complex, mission-critical processes to schedule their shop floor operations. Oops, who saw that coming? After all, it was just an end-user reporting tool. End users shouldn't do that kind of thing.

Over a 20-year period, legacy end-user reporting tools like FOCUS spread through organizations like a unchecked virus. Early MIS departments nonchalantly turned over reporting responsibilities to business developers. Management did not pay attention to how widespread the usage was. Now, decision makers find themselves ill and seeking powerful legacy antibiotics, as it could cost them millions of dollars to treat their disease and manually replace user-written applications with new products in order to get to a healthy state where they can meet today's challenging business demands.

If your company still depends on a legacy reporting tool, you have valid reasons to be worried. We are trying to cope with global competition, rapidly changing technologies, and shaky financial situations. We are in a consolidating BI market that contains leader products, emerging new technology, and declining older tools. It is definitely time to leave your legacy.

1 comment:

Anonymous said...

COBOL? Yes it in wide spread use. But the reason why they don't teach it and shouldn't is quite simple.

Not in demand in the job sector:

http://www.reed.co.uk/job/searchresults.aspx?k=cobol&jto=false&s=&l=london&lp=&ms=From&mxs=To&st=5&ns=true&da=168

No jobs in London on reed.co.uk

http://www.cwjobs.co.uk/JobSearch/Results.aspx?Keywords=cobol&LTxt=London%2c+South+East&Radius=0&LIds2=ZV

One job in London on CWJobs.co.uk

Now try the same for other technologies like Java.

If you are a recent graduate be careful doing a course in COBOL. People have been saying for years that COBOL will be in demand. Well its still not here in the UK.

Also compare the opportunities in other skills as a comparison. "COBOL is alive" and running but "COBOL is dead" as a career in software development.

About Me

My Photo
Helping companies make better decisions via Business Intelligence. INTP working on the E&J. Traveler, reader, family guy, coffee drinker.