Tuesday, December 31, 2013

WebFOCUS and R for Dynamic Statistical Analysis

Part I
You can leverage the best of two powerful software products by combining WebFOCUS Business Intelligence language with the R statistical programming language.

WebFOCUS from Information Builders provides you with robust BI capabilities such as web access, highly-dynamic scripting, and far-reaching enterprise data access. With the R open-source offering, you have sophisticated statistical analysis, data visualization, and access to web data content.

In this multi-part article, I will show you how easy it is to integrate the two products. 

To help explain this, let's look at a simple application where the two products work together.

I have a text analytics software product called the BI Analyzer, which we use in large legacy modernization initiatives as a preliminary assessment tool. The software mines the applications' text for important keywords and loads the scan results into an inventory database--imagine a small Google engine specializing in indexing your custom computer applications--from which WebFOCUS and R can perform analytics.

Using statistical analysis, I want to determine the complexity of each scanned procedure. I use WebFOCUS' auto-prompting features to generate a simple user interface of options (if I wanted, I could make this a nicer looking web page, but it meets my needs).

To determine a procedure's complexity, I look at various keywords found inside all of the procedures and, based on their "hit" frequencies, categorize each procedure into one of four buckets: Low, Somewhat Low, Medium, and High. 

In addition to showing this breakout in tabular form, I want to display a BoxPlot graph which provides great quartile visualization. 

Using the options I choose from the web user interface, WebFOCUS dynamically generates and executes both WebFOCUS and R scripts. It then displays the consolidated results on the web page (again, this is a very simple application to which I could add more presentation sizzle if needed).  

Notice in the screenshot that I am searching for any Crystal Reports procedures which use JavaScript Pages (identified by a scanned keyword that contains ".jsp"). I only want to consider those reports which are active, ignoring any in the scanned inventory database which were flagged as obsolete. I'm picking the BoxPlot option but have others such as Histogram and Plot.

WebFOCUS produces a tabular report and then combines it with the R graph. 

Picture of WebFOCUS/R User Interface


Had I selected "Update" instead of "View," the procedure would have used the results of its quartile analysis to update the inventory database, flagging each of the scanned procedures referencing JavaScript Pages with its calculated complexity ranking. Later, other reporting features would use this information to calculate time and cost estimates for the legacy BI conversion project.

Here is a summary of what the WebFOCUS procedure does for me: 
  • Via a web browser, interacts with the user
  • Based on the user's selected options, extracts data
  • Dynamically builds a WebFOCUS report script based on user's selections
  • Runs the script and creates a report
  • Dynamically builds an R graph script based on user's selections 
  • Calls R to run the script and create a graph (using the extracted WebFOCUS data)
  • Displays the WebFOCUS and R results on the screen


Visually, the interaction between WebFOCUS and R looks something like this: 

Picture of WebFOCUS and R Interaction



In Part II, I share the technical details of how WebFOCUS and R work together.

Before you read that next section, I'm curious as to what you think. Do you see value in using WebFOCUS and R together?

Please leave your comments here. 

No comments:

About Me

My photo

I am a project-based software consultant, specializing in automating transitions from legacy reporting applications into modern BI/Analytics to leverage Social, Cloud, Mobile, Big Data, Visualizations, and Predictive Analytics using Information Builders' WebFOCUS. Based on scores of successful engagements, I have assembled proven Best Practice methodologies, software tools, and templates.

I have been blessed to work with innovators from firms such as: Ford, FedEx, Procter & Gamble, Nationwide, The Wendy's Company, The Kroger Co., JPMorgan Chase, MasterCard, Bank of America Merrill Lynch, Siemens, American Express, and others.

I was educated at Valparaiso University and the University of Cincinnati, where I graduated summa cum laude. In 1990, I joined Information Builders and for over a dozen years served in regional pre- and post-sales technical leadership roles. Also, for several years I led the US technical services teams within Cincom Systems' ERP software product group and the Midwest custom software services arm of Xerox.

Since 2007, I have provided enterprise BI services such as: strategic advice; architecture, design, and software application development of intelligence systems (interactive dashboards and mobile); data warehousing; and automated modernization of legacy reporting. My experience with BI products include WebFOCUS (vendor certified expert), R, SAP Business Objects (WebI, Crystal Reports), Tableau, and others.