Thursday, September 25, 2014

Lessons from Doorbell Replacement

How hard can it be to replace the button for a doorbell?

That was the task my wife gave me and it appeared within my abilities. Surely, I could be done within an hour.

My wife had already spent thirty dollars on a brushed metal doorbell that looked nice during the day with a back-lit button you could see at night.

On Saturday, I jumped into action and removed the old buzzer button, a simple twenty-year old plastic box. I pulled the doorbell away from the door frame and undid the twisted wires. Quite brittle, they broke easily. To get more length, I tried to pull the wires out farther, but they would not budge. If I was not careful, I would have to instead buy a wireless doorbell.

I examined the new doorbell as I took it out of the package. While the old one had just laid on top of the wood, this one had an inch-long metal piece that was supposed to fit inside a 5/8-inch wide hole. A search of my tool box came up empty for that particular drill bit. Plus, I didn't really believe the doorframe was deep enough.

"Look," I explained to my wife, "this isn't going to work" and presented various reasons to discard her plan.

With open disappointment, she agreed; I put the old button back in place. Ding dong, it still worked.

At the home improvement store, we bought a different thirty-dollar brushed-metal doorbell button; this one could lay flat against the door frame but did not have a light, a compromise.

On the next Saturday, I went back to work. How hard could this be?

I once again removed the old doorbell button. This new one had two pieces: a back that attached to the door frame with screws, and a front that snapped onto the back. The wires gave me grief, but I was finally able to attach the new doorbell button.

This one would not lay flat; some type of plastic protrusion on the back always got in the way. The last doorbell ringer needed a hole, so I considered that as a potential solution for this situation. I got out a power drill and started poking little holes in the door frame.

Ultimately, I was able to get the button to lay flat. When I tried to snap on the front, however, it would not close; something was preventing the snap from catching. I completely removed the doorbell, busted some more holes behind it, hooked it up again, took it off, and repeated several times.

By now I was frustrated. It wouldn't snap close, so I decided to try to keep it shut with some Gorilla Glue.

No luck. With brown glue spots all over the door frame and a ruined doorbell, I had failed. Trying to remove the Gorilla Glue mess, I scrubbed off patches of door frame paint. I tossed the thirty-dollar buzzer in the trash and once again returned the old plastic one to its proper place. Ding dong, it still worked.

Okay, I needed to stop and think about this. What approach was best? I still had the original back-lit doorbell buzzer that my wife wanted. I needed the right tools to do the job.

I made another trip to the home improvement store and bought a 5/8-inch hole drill bit with diamond grit (coincidentally, another thirty dollars). While there, I picked up white paint to cover the Gorilla Glue fiasco.

On the third Saturday, I removed the old doorbell buzzer, drilled the hole, and put in the new buzzer; it just barely fit. With some silicon chalking around the button and some white paint to cover mistakes, all was good. Ding dong!

After replacing this legacy piece of hardware, here are some of my personal insights:

  • I started without assessing the situation 
  • I never had a proper plan  
  • Having never done this before, I did not have the proper know-how, expertise, or skills
  • I did not have the proper tools to do the job 
  • It took longer than expected (especially without plan, skills, or tools)
  • It cost more than expected 
  • I could have saved by hiring a professional

My personal experience with a doorbell buzzer is similar to companies replacing their legacy business systems. How hard could it be, for example, to get rid of old reporting applications and convert all of the existing procedures to newer technology?

Upper management already bought the new BI product, so you just assign the conversion effort to the college intern. How hard could it be? Surely, she can knock it out quickly.

Ding dong: no up-front assessment, no planning, no accurate expectations as to time and cost, no specialized skills or tools, minimal progress every Saturday.

You may consider a legacy system modernization initiative as a one-off project your team can just fumble through and then forget about. That can be the painful approach and you may have to cover up mistakes afterwards. Before you do that, consider there are professionals who have done modernizations before and who have developed methodologies and automated software to reduce the time, cost, and risk.

Don't be a ding dong. 

Wednesday, June 11, 2014

Wendy's Wins Big with BI/Analytics

Congratulations to my friends at The Wendy's Company for being honored yesterday with Information Builders' 2014 Award of Distinction.

IB wrote this about Wendy's enterprise dashboard:

"The Wendy’s Company, the world’s third largest quick-service hamburger company, created a BI portal and dashboard environment that integrates an enterprise point-of-sale system to deliver targeted reports with drill-down capabilities for decision-makers at every level of the company. WebFOCUS helps managers control costs and make informed decisions that improve the bottom line. Thousands of international and domestic franchises currently use WebFOCUS dashboards, helping Wendy’s to improve profit margins at hundreds of restaurants." 

At the beginning, the idea was for an "above-the-store" executive portal where a few individuals could see all of the company's KPIs related to revenue, speed of service, costs, and customer satisfaction. However, it did not take long before thousands of decision makers at different levels of the QSR organization asked for access to that valuable information.

For more information, see IB's press release

Sunday, March 23, 2014

Developing BI/BA Web and Mobile Applications with WebFOCUS

WebFOCUS for BI/BA Application Development

WebFOCUS is a powerful enterprise Business Intelligence and Analytics platform well-suited for today's complex and rapidly-changing environment. The software is produced by privately-held American software vendor, Information Builders.

As part of WebFOCUS, Information Builders provides two BI/BA application development tools, one for business developers and another for IT technical developers:

  • BI Portal and InfoAssist for business developers (web-based and mobile)
  • Windows-based Developer Studio for IT technical developers 

With WebFOCUS, companies can provide business users with self-service intelligence and analytics:

  • Dashboards and scorecards 
  • Self-service guided ad-hoc data exploration
  • Mobile BI with right-time data on any device 
  • InfoApps to easily analyze and manipulate information
  • Deep integration with desktop products such as Microsoft Excel and Adobe PDF 
  • Integration with open-source software such as R statistical programming language and Python 
  • Dynamic report scheduling and distribution, with real-time alerts
  • Integration with enterprise data and information management  

Note: Within the next month or two, Information Builders will release the next generation of Developer Studio which will be called the Application Studio. The new product’s “look and feel” will be consistent with that of InfoAssist. For example, Application Studio will also have a ribbon-based user interface.

BI Portal and InfoAssist

The WebFOCUS BI Portal enables business users to easily create and share sophisticated portals, launch pages, reports, and graphs hosted within the corporate WebFOCUS environment or in the cloud. The users have no software to install.

InfoAssist, the report and graph layout tool, is appropriate for business users as well as for light-weight IT development. Information Builders will license BI Portal/InfoAssist based on a certain number of users (e.g., one hundred) or on an unlimited basis.

WebFOCUS 8 comes with sophisticated multi-tenant security enabling companies to open up web and mobile BI to not only their internal employees, but also to external partners and customers.

Picture: BI Portal with InfoAssist

Windows-based Developer Studio (IDE)

The Developer Studio is a Windows-based IDE enabling IT technical developers to build complex, interactive BI applications hosted within the corporate WebFOCUS environments or in the cloud. 

However, the Developer Studio product is not an appropriate tool for business users. Instead, Information Builders provides for business users the web and mobile InfoAssist product.

Picture: Developer Studio IDE 

Developer Studio Features

The Developer Studio provides an IT BI/BA developer with a variety of software development features, including: 
  • Metadata Management: tool for generating and maintaining the metadata layer 
  • HTML Composer: layout tool for designing and creating web launch pages 
  • Procedure Viewer: layout tool to visually display multiple-step procedural logic 
  • Report Painter: layout tool for designing and creating interactive and dynamic reports 
  • SQL Report Wizard: tool for building reports using SQL requests 
  • Define Dialog: tool for creating virtual columns using business rules 
  • Join Painter: tool for logically connecting tables across the enterprise 
  • HTMLForm: tool for embedding HTML commands inside WebFOCUS procedures 
  • Match Wizard: tool for logically matching tables across the enterprise for either join-like functionality or exceptions
  • For creating graphs, Developer Studio utilizes the InfoAssist product  

Developer Studio Editions

IT technical developers can purchase and install one of two different “editions” of the Developer Studio product:
  • Report Developer Edition
  • Full Edition: includes MAINTAIN development and full personal WebFOCUS image 

Information Builders offers a lower-priced Edition for IT technical developers who do not need to develop online database maintenance applications or have their own personal WebFOCUS environment. Typically, few IT technical developers need all of the features within the Full Edition.

Report Developer Edition

The Report Developer Edition of Developer Studio has all of the features needed to build web and mobile BI applications, including the Procedure Viewer, HTML Composer, Report Painter, Join Painter, Financial Modeling Language Painter, Reporting Server Management, and Change Management features. 

An IT technical developer can download and install this edition of the Developer Studio on his or her workstation for a reasonable one-time license fee plus on-going annual maintenance fee.

Full Developer Studio Edition

The Developer Studio IDE can also be purchased in the “Full Edition” version with its own personal, stand-alone WebFOCUS environment—a WebFOCUS Client Tier (web server, Java app server, client components, security repository, etc.) and a WebFOCUS Reporting Server. This provides a personal “sandbox” for testing WebFOCUS functionality. 

In addition, the full edition of Developer Studio comes with the MAINTAIN development product for creating web and mobile database maintenance applications. This full edition is limited to those IT technical developers who perform MAINTAIN development or to those few individuals with a valid business case for a personal BI environment.

Note: The IT technical developer’s version of Developer Studio must be kept in sync with the other components of WebFOCUS environments. In other words, the developer should not install a client release higher than the software release of the WebFOCUS web tier and BI server.

GUI Code Generators

These WebFOCUS tools enable a visual application development process by providing layout tools that can translate the design and generate the necessary computer programming instructions.

When developing BI applications with WebFOCUS, several languages are involved:
  • InfoAssist: generates the FOCUS 4GL 
  • Developer Studio’s Report Painter: generates the FOCUS 4GL 
  • Developer Studio’s HTML Composer: generates HTML, JavaScript, and XML 

Being a business user tool, InfoAssist does not provide access to the generated instructions. Developer Studio, on the other hand, lets the IT technical developer get to a text editor from where he or she can modify the generated instructions.

There are really only two valid situations where the IT technical developer needs to manually type instructions: the FOCUS 4GL’s Dialogue Manager and the HTML’s JavaScript. Both are procedural scripting languages that accompany the main logic.

There is no reason for an IT technical developer to manually change any of the actual HTML/XML logic generated by the Developer Studio’s HTML Composer other than to add JavaScript. 

Likewise, the IT technical developer must be careful when modifying the 4GL non-procedural code used for creating reports, graphs, and extract files. It is possible to make manual changes not recognized by the Developer Studio’s parser which will cause the layout tools to fail. Where this might happen is when the developer creates a highly-dynamic layout that cannot be painted.

WebFOCUS technical developers should consciously avoid developing BI/BA applications that cannot be opened in the graphical layout tools. While necessary in some complex situations, having “non-paintable” code dramatically increases the skill-set requirements, shifting from individuals who can use visual GUI development to those who must know how to hand-code the 4GL instructions.

Should You Use WebFOCUS?

A list of Information Builders' customers looks like the "Who's Who" of global businesses and government agencies. For over a decade, Gartner has included Information Builders in the Leader's quadrant for enterprise BI products. 

WebFOCUS runs on a variety of platforms (e.g., Windows, UNIX, Linux, and mainframes) and can scale to handle millions of users. It provides super-secure web access to sensitive data, which can be stored in many diverse formats (Information Builders has always been a well-known provider of enterprise data adapters through their iWay Software brand, partnering with many of the other leading software vendors). 

WebFOCUS provides business users with easy-to-use yet robust BI/BA tools. It provides IT developers with a sophisticated IDE to produce robust self-service web and mobile applications. 

WebFOCUS is highly dynamic. Unlike old-fashioned report writers like Crystal Reports which were limited to a single static layout, a WebFOCUS procedure can dynamically restructure itself to produce thousands of different report versions.  

Legacy tools can be modernized into WebFOCUS. For example, SQL routines can be directly embedded inside the WebFOCUS language to leverage legacy assets. For many other reporting tools, there are utilities to automatically assess and transform into WebFOCUS in order to reduce the time and risk of the BI modernization initiative. 

For more information, visit the WebFOCUS website

Saturday, March 1, 2014

How to Ask Smart BI/Analytics Questions

Many people rush when posting a "how-to" question on a software vendor case, a LinkedIn group, or some other web forum, as well as when sending an e-mail question to associates. Getting a good answer to the question can be dramatically improved by first properly structuring the request.

For a great article on how to ask smart questions, see author Eric Raymond’s article on that topic. Eric has written several well-known books on open-source software and the Linux operating system, one of his most famous being “The Cathedral and the Bazaar.”

Eric advocates that “Google is your friend” and you must always search before asking for help. 

He recommends that, before posting a question in a web forum, you try to find the answer yourself by going through these steps:

  1. Search the archives of the forum to which you plan to post
  2. Search the web
  3. Read the manual
  4. Read a FAQ
  5. Inspect or experiment
  6. Ask a skilled friend
  7. If you're a programmer, read the source code (e.g., the WebFOCUS JavaScript engine)

Eric also makes the point that “All diagnosticians are from Missouri,” meaning that the people reading your answer will ask you to “Show me your problem’s symptoms in chronological sequence.” 

Following Eric's advice should help you get answers to your BI/Analytics questions.

Have you been frustrated by people asking for help (or perhaps, have you found yourself guilty of asking vague questions)? 

Sunday, February 16, 2014

Tableau Software Continues to Win Data Visualization Fight

The three big contenders for BI/Analytics data visualization are Tableau Software, QlikTech's QlikView, and TIBCO's Spotfire.

Using the job market as an indicator shows that demand for Tableau is growing much faster than that for the other two.

Picture (2014 February 16)

Do you agree with this simple assessment? Post your comments here if you have insight into the Data Visualization software market.

For more information on these three software products and some additional details on Tableau, see my earlier blog article.

Tuesday, February 4, 2014

Closer Look at R Statistics Scripts Called by WebFOCUS

Part IV
Let's take a closer look at the R scripts WebFOCUS generating within my sample text-mining application. 

In this simple application, WebFOCUS generates one of three graph options: box plot, histogram, or a plot, depending on the user selection.  

If you do not have R already installed, you can get a copy and full implementation instructions from the Comprehensive R Archive Network--a kind of App Store for all things R. The hyperlink for the CRAN is:

For WebFOCUS to be able to communicate easily with R, we want to install R on either the same computer as the BI product or within its network of accessible drives. 

When you code in the R programming language, you are commonly using assignment statements and functions. 

For example, I want to assign the values in my WebFOCUS-generated text file into a named R data structure. Assignments are done with a symbol that puts together a left-caret with a dash. 

To pull out those values, I must first read the text file using a read function. Here is the basic command:

I want a data structure called "keywords" to hold my WebFOCUS file. The read command has a variety of options; I am using the "table" option to read a simple tabular structure. My table has a row identifying the column headers, so I inform read of that with a parameter called header. My table values are separated by commas, so I tell read about that using the sep parameter.

After that command, R has loaded my WebFOCUS data table into its memory. 

Before I actually create an output graph, I need to tell R where to store it. Had I been working with R interactively, the graph would just show on the screen. Here, I need to save the graph into a bitmap file and later in the process tell WebFOCUS to display it on the screen. 

I do that with the R png command (PNG stands for Portable Network Graphics). Here is a sample:

Along with the output file's full path, I also include some optional parameters to tell R how tall and wide to make the graph. 

Two R commands down and just one more to make the graph! 

If the user requested a box plot, I would use the R function called, naturally enough, boxplot. My sample statement looks like this: 

The first parameter for the boxplot function tells it to use a column called "Count" in my keywords data structure. Notice that R uses the US dollar sign to separate the structure and column name (in this example, keywords$Count). I am also passing in some optional variables to make the box plot lay down horizontally, give R to option to vary its width, and set the X axis label to say "Search Counts." 

The boxplot function gives me a bitmap graph such as this:

The other R graph functions look very similar. For example, my histogram command is simply: 

The difference here is a parameter called main where I am blanking out the main title. My output histogram might look like this:

Next is the simple graph of points plotted, whose R function looks like this: 

This gives me a graph such as the following:

Quick Clean-Up!
Before the R script ends and returns control to WebFOCUS, I am going to tell R to close the output graph file (otherwise, R controls the picture and I cannot open it). To do that, I call a function called "dev" (short for device) and turn off the graphics device connected to the output. It is simply this:

One Last Thing
The three types of graphs used above are all standard R features. Some, however, require that an add-on "package" to be installed and referenced. You can read about this on the CRAN website.   

For example, I might want WebFOCUS to create a file of keyword counts and then produce a word cloud. Within R, I would need to install the package containing that feature. Once installed, WebFOCUS needs to reference that package with the "library" function. 

Here is an example of WebFOCUS generating a word cloud R script: 

If you do have R scripts that use a package, you must have WebFOCUS start R using the R_LIBS parameter to identify where you installed the package. See an earlier article on how that works

Do You Have WebFOCUS and R? 
I hope you enjoyed this multi-part article on integrating WebFOCUS and the R statistical programming language. Do you have both products and, if so, do you use them together? Please leave a comment and let us know your experience. 

As always, if I can ever be of service, just let me know.

Sunday, January 5, 2014

Generating Dynamic R Scripts with WebFOCUS

WebFOCUS would only have a fraction of its power without the procedural scripting language called Dialogue Manager. Based on user input, a WebFOCUS procedure can leverage Dialogue Manager to basically rewrite its own code and become something completely different.

This robust flexibility makes WebFOCUS a great complement to the R statistical programming language.

In Part I of this multi-part blog posting, I showed you a simple user interface for graphing data obtained from text mining of legacy reporting applications.

When automatically determining the complexity of any particular reporting procedure, I give the user an option of seeing the results--in addition to the tabular data--graphically. The user can select BoxPlot, Histogram, Plot, or none of the above.

Based on the user's choice, the WebFOCUS procedure needs to generate the appropriate R code.

Here is the line of WebFOCUS code that created the Graph pull-down list:

At run-time, WebFOCUS will have a symbolic variable named &GRAPH containing one of four possible values: BOXPLOT, HISTOGRAM, PLOT, or NONE. If the user was not interested in visualizing the data and selected "NONE," we can skip all of the R logic.

For the three R graph options, however, we need to provide R with several key pieces of information: 
  • Where to find the input file (stored in a WebFOCUS Reporting Server folder called "r_data")
  • Which R script to run (stored in a WebFOCUS Reporting Server folder called "r_scripts")
  • Where to store the output graph (we'll use the same "r_data" folder for input and output)

Visually, here is the interaction between WebFOCUS and R:

Below is the WebFOCUS logic for doing that. First, we set a symbolic variable named &COUNTPATH with the full path of the file containing named pairs of text scan values and their counts.

Next, we need to deal with the R script. We create a symbolic for the name of the R script (&SCRIPT) as well as provide some file definition details so that WebFOCUS knows where to write the script (a file allocation called "RPROGRAM").

In the third step, we set &GRAPHPATH with the full path name of where R should write the output graph. So far, so good.

Our next step is to create an R script that will use the mined counts from the text scanning to produce the user-selected graph.

Based on the user's graph selection, the WebFOCUS procedure will write one of three different R scripts. Note: I could have made this logic much more succinct and dynamic but I decided to make it more verbose and simpler to read for the purposes of this blog article.

In this step, we write statements to the file allocation "RPROGRAM" which points us back to the particular R script defined earlier as "r_dynamic_script.r" stored in the "r_scripts" folder.

The great thing about WebFOCUS is that, at run-time, it can dynamically change any aspect of the R scripts. My example here is fairly simple, with the only dynamic code being the names of the input and output files (&COUNTPATH and &GRAPHPATH, respectively).

After writing the proper R script, WebFOCUS will then go a block I named "RUN_RSCRIPT."

Here is where WebFOCUS uses the "call_r_dynamic" FOCEXEC discussed in Part II. This call passes in the name of the R script (the value of symbolic variable &SCRIPT) so that R can run the proper code. Within those instructions, we have told R where to find its input file, how to generate the output graph, and where to store the results.

Click here to go to Part IV, where we can take a closer look at the R scripts. 

Wednesday, January 1, 2014

Using WebFOCUS to Call R

In this second of several articles on integrating WebFOCUS and R, I will show how the enterprise BI product from Information Builders can easily call the open-source R statistical programming language. If you have not already, go back and read Part I for an overview.

One simple way for WebFOCUS to call the R product is to use the Dialogue Manager scripting language and the SYSTEM function.

For this example, I created a WebFOCUS procedure named "call_r_dynamic" intended to be executed from within other WebFOCUS procedures. The calling procedure uses a symbolic variable named &SCRIPT to pass in the name of the R script to be run.

Using the WebFOCUS Dialogue Manager command to create symbolic variables, I set the values of three different folder locations. Now typically, symbolics are intended to provide dynamically-changing values for parameters, but I am really just leveraging them here for readability purposes.

They are:
  • &R_BINARY variable tells WebFOCUS where the R executable is stored
  • &R_LIBS variable tells WebFOCUS where R packages are stored
  • &R_HOME variable tells WebFOCUS where the R scripts are stored

Based on my Windows environment, here is the WebFOCUS code for my three folder variables:

-SET &R_BINARY = '"C:\Program Files\R\R-2.15.1\bin\i386\R"';
-SET &R_LIBS = 'C:\Users\Doug\Documents\R\win-library\2.15';
-SET &R_HOME = 'C:\ibi\apps\r_scripts';

Notice that if any of your folder names have spaces, you need to enclose them within double quotes. Otherwise, the procedure gets confused when it hits the blank and thinks the folder name ends there. Also notice that I store the R scripts in an application folder on the WebFOCUS Reporting Server (I will talk more about this in a later post).  

The name of the R script really is a parameter from the calling program--for example, something like "r_dynamic_graph"--so I dynamically set its full path as follows:

-SET &R_SCRIPT = &R_HOME || '\' || &SCRIPT;

With those four symbolic variables, I will put everything together into a single instruction call to R:


At run-time, WebFOCUS will substitute all of my values into a string such as this:

"C:\Program Files\R\R-2.15.1\bin\i386\R" HOME=C:\ibi\apps\r_scripts R_LIBS=C:\Users\Doug\Documents\R\win-library\2.15 --vanilla  < C:\ibi\apps\r_scripts\r_dynamic_graph.R

This command string will call R, telling it to use my R libraries. Notice that I included a start-up option called "vanilla" which basically instructs R to do minimal work. Also important is the left caret, telling R to run the script whose name I provided.

I still need to execute this command string, which is where the WebFOCUS SYSTEM function comes into play. This will pass the command string to the underlying operating system and send back a return code showing success or failure.

The WebFOCUS SYSTEM function takes three parameters:

SYSTEM(commandstring_length, commandstring, returncode_format);

Here is my actual WebFOCUS Dialogue Manager code using the SYSTEM function to call R: 


I put the SYSTEM's return code into a global symbolic variable called &&RETCODE so that the calling procedure can verify everything worked. 

In Part III of this article, I will show how to use WebFOCUS to generate and run dynamic R scripts. If you have any questions, contact me. 

About Me

My photo

I am a project-based software consultant, specializing in automating transitions from legacy reporting applications into modern BI/Analytics to leverage Social, Cloud, Mobile, Big Data, Visualizations, and Predictive Analytics using Information Builders' WebFOCUS. Based on scores of successful engagements, I have assembled proven Best Practice methodologies, software tools, and templates.

I have been blessed to work with innovators from firms such as: Ford, FedEx, Procter & Gamble, Nationwide, The Wendy's Company, The Kroger Co., JPMorgan Chase, MasterCard, Bank of America Merrill Lynch, Siemens, American Express, and others.

I was educated at Valparaiso University and the University of Cincinnati, where I graduated summa cum laude. In 1990, I joined Information Builders and for over a dozen years served in regional pre- and post-sales technical leadership roles. Also, for several years I led the US technical services teams within Cincom Systems' ERP software product group and the Midwest custom software services arm of Xerox.

Since 2007, I have provided enterprise BI services such as: strategic advice; architecture, design, and software application development of intelligence systems (interactive dashboards and mobile); data warehousing; and automated modernization of legacy reporting. My experience with BI products include WebFOCUS (vendor certified expert), R, SAP Business Objects (WebI, Crystal Reports), Tableau, and others.