Shift Happens

See the latest version of "Shift Happens" for trends in technology and globalization.  It provides amusing (we're not reading newspaper in the toilet anymore), interesting (by 2020, mobile devices will be the dominant mode of accessing the Internet), and scary trends (if you're an American, that is).

SAS iPhone BI

After reading my blog on using WebFOCUS to run iPhone BI applications, Sy Truong of MetaXceed sent me the following information on an iPhone architecture for SAS:

SAS iPhone App Architecture

The delivery of SAS data and reports to an iPhone application is unique and requires a different architecture as compared to traditional client server systems. This is similar to web applications in how it is delivered to a browser but in this case, the application replaces the browser. The diagram below illustrates the components of the iPhone SAS application architecture including: the iPhone application, a web server and the BI Flash server.

In this example, the iPhone application on the iPhone communicates through standard TCP/IP protocol to a web server. The web server then communicates to an application server which is actually a SAS session processing SAS programs and data. The output resulting from the SAS program is then delivered back to the iPhone in a similar way a web browser would access a web page the serer. The distinction however is that the iPhone application is not a web browser and the SAS session running on the server is more dynamic compare to a static page. The SAS data and macro program it executes may be simple and standard but facilitating the communication takes a little more effort. The request from the iPhone application and the delivery of information from the server is handled by BI Flash. This makes the experience more dynamic and delivers the full power of SAS on the server. The following steps are taken in order to facilitate the access of SAS data from an iPhone App.
 
Step 1 - Download iPhone Application

One of the most unique and successful aspects of the iPhone is how users can easily download an iPhone App directly from the iPhone App Store. There is a user friendly way of searching for and downloading the application to your iPhone. In this case, you can download BI Flash App which is the client component which enables the communication between the iPhone and the SAS server.

 Step 2 - BI Flash Application Server

More details on this step will be explained in the "Application Server" section but an administrator would execute the BI Flash server. This server functions as a listener waiting for a request from the iPhone. Upon receipt, it would process the request similar to how you would submit SAS programs from display manager. The server would generate a SAS log and output results in XML which is then sent back to the iPhone to be viewed.

Step 3 - Connect iPhone to Application Server

Most iPhone applications have configuration settings. This allows for users to easily configure settings during setup and then use the application without further changes unless future configuration changes are needed. In order for the BI Flash example iPhone app to access SAS, the user would need to configure the following:

 

 

Host Name - This the name of the server or an IP address of the SAS server.

 
User Name - A valid user name that has been defined on the server needed during authentication

 
Password - A user defined password to secure access

 
There may be other configuration options which will set the default behavior of the application but the parameters above show the minimum requirement in order to connect to a SAS server.

Step 4 - Run Application

The final step taken by the user to access SAS data is to execute the SAS macros from the iPhone. This request is initiated from the iPhone app and sent directly to the server with user selected options. The results are then returned to the iPhone displaying the most updated information on the server.

The system architecture in this example is rather simple compared to other systems that require multiple layers of middleware. This is similar to the SAS/IntrNet where users are on a web browser accessing SAS data and programs on the server through the broker and SAS application server. The difference however is that the client is not a browser, but rather a dynamic iPhone application.

Searching through the iPhone App Store, I could not locate the BI Flash application that Sy references.  Regardless, the architecture that Sy describes is very similar to that used by WebFOCUS which I will document in the near future.

Accidental iPhone BI

I admit it -- I converted to Apple. It started with an accidental iPhone purchase.

I never intended to buy an iPhone. Instead, I planned to replace my Treo with a brand new Palm Pre from the same carrier. Unfortunately for Sprint and Palm, the teenagers working at the strip-mall store lost my order. After weeks of strange phone calls with kids who could never find my order nor know when truck shipments would arrive ("Dude, if I could predict the future, I would be rich and not work for Sprint!"), I simply went to the Apple Store and came out with an amazing iPhone.

While waiting in the red velvet line (this was the weekend of the 3Gs release), I was introduced to the Apple Fan subculture. When not tapping away at the touch screen, iPhone users advocated their favorite downloaded apps with cohorts. Apple employees were professionals, passionate about their work.

Literally within seconds, the iPhone was accessing my personal and corporate e-mail accounts and calendars. I could easily connect to WiFi or the AT&T 3G network to browse websites with the Safari browser. Even secure sites were available through a VPN connection.

I even paid the $2.99 per month fee to listen to Sirius XM satellite music on the iPhone. (I also got a sick feeling about having just paid $150 for an MP3 player and $399 for a Sony Book Reader.) Everything I needed to be productive was now at my fingertips; I stopped waiting for the Windows bootup on my PC except for things that required heavy-duty typing.

So far, two individuals have walked away from my personal iPhone demos to go directly to the Apple Store to buy their own. Yes, the same day. Immediately. Others have changed their Christmas, birthday, and other gift-buying/gift-getting plans. One of my straight-laced employees responded to my iPhone enchantment with, "Sorry, I want my phone to be a phone. That's all it needs to do."

So I added some real sizzle.

On my iPhone, I logged into a secure executive dashboard for a well-known restaurant. The Safari browser visually showed corporate-wide key performance indicators for sales, speed of service, food costs, labor costs, and cash register operations. All right there in my hand, blazing fast. From the total level, I tapped down through the company hierarchy until I reached a specific store. I looked at their raw item inventory to see if they had enough cheese. I pointed out a red stoplight where the manager had failed to staff enough people on cash registers the previous day, causing a negative impact to speed of service and revenues. I tapped to get a trend of the past 20 days to see if that was just a new occurrence.

My friend responded, "Boy, I have to get one of those."


We had created this business intelligence application using the WebFOCUS software product from Information Builders. Initially, the plan was to have on-demand access through a PC web browser and push out static reports to e-mails, printers, and Blackberry devices.

Just like my personal use of the iPhone, the restaurant's corporate use of iPhone was accidental. During the development project, somebody tried to access the dashboard directly with a Blackberry but with no luck. It seems that the WebFOCUS BI Dashboard uses web techniques not supported by the Blackberry. Even if the Blackberry user could get past the initial dashboard screens, the reports did not display well on the handheld.

Not so on the iPhone. The Safari browser is top-notch and allows zoom in-and-out of the screen content. Even client-side scripts, cookie techniques, and secure LDAP integration worked on the iPhone. The iPhone even supported dashboard exports of data to PDF and Excel.

My accidental iPhone purchase has caused all sorts of changes for me and the people around me. I would write more about it, but I have to go sign for a FedEx package -- an Apple MacBook Pro just showed up.

Take the Technology Job Trend Quiz

Here is a quiz you will not see in Cosmopolitan: How much do you know about today's technology job market?

In each of the following pairs of career activities, there is an A and B option. See if you can select the one in which you are most likely to find a job today:

1. Building computer applications using A) RPG or B) the iPhone programming language?

2. Writing A) COBOL programs or B) Business Objects ad-hoc queries?

3. Creating A) SQR reports or B) ESRI online maps?

4. Writing web applications in A) .NET or B) Java?


In 1980, I encountered my first computer programming job when a friend took me to his workplace to see RPG coding done on an IBM mid-range computer. For decades, RPG developers were in demand. However, if you had searched Monster.com for RPG job postings at the beginning of 2009, you would have only found 117 openings across the entire United States. The number of openings has steadily declined and in August of 2009 was down to 88.

The correct answer to Question 1 is B -- in August of 2009, there were over twice as many Monster postings for people who can develop handheld iPhone applications than for RPG programmers. Apple has sold millions of iPhones and the demand for new applications grows daily.


If you went to college for computer programming, you probably took COBOL. In 1997, Gartner Group estimated that 80% of the world's business applications were using COBOL. But if you picked A for Question 2, you would be wrong. Demand for COBOL programmers has been steadily declining. Monster started the 2009 year with 348 COBOL job postings and went down to 286, an 18% decline. The Business Objects business intelligence product, however, has seen solid demand despite being acquired by SAP, going from 729 job postings in January to 771 in August 2009, a 6% increase.


Specialized technologies for report writing have been popular. For example, the SQR language is a mix of SQL (structured query language) and COBOL. Today, however, you would have troubles finding a job writing reports using SQR. Monster postings declined from 86 at the beginning of 2009 to only 45 in August. Jobs to create geographic information systems using the ESRI technology, on the other hand, increased from 150 to 165 in the same timeframe.


Question 4 is tricky. The jury is still out for standards for building web applications. Possible winners are Windows products such as .NET, C++, and C# or non-Microsoft technologies such as Java. But if you base your guess on the number of Monster job postings, the verdict seems to be Java. At the beginning of this year, there were 3885 .NET job postings. Today, that number has dropped 9%. However, Java jobs have consistently stayed above 5000 most of the year (Monster caps the number of hits at 5000).


As you evaluate your skill sets, here are some things to keep in mind about today's technology trends:

Most companies prefer packaged applications over custom applications.
Companies want low headcount, hence they are willing to buy and implement systems developed by third-party vendors. Applications such as human resources, payroll, accounting, accounts payable, accounts receivable, and so forth are standard and easily acquired. Few firms are so unique that they cannot use a packaged ERP from a mega-vendor such as SAP or Oracle. You are better to have skills related to software package implementation, project management, and functional know-how than in custom application development (unless you want to work for the software vendors).

When companies have troubles finding resources for a legacy technology, they move away from it.
Organizations with mainframe computers were unable to find technical people who knew how to run their large platforms. The result? They looked for alternatives such as small, distributed systems. IBM was forced to try to switch from their proprietary operating systems to an industry standard such as Linux which had a broader knowledge base. It may be counter-intuitive, but you are better to have fewer skills in emerging technologies than be an expert in legacy ones -- don't get suckered into being the old-time guru who is honored today and discarded tomorrow.

To reduce costs, companies consolidate multiple technologies.
Over time, companies collect a variety of technologies. There comes a time when they have to clean house. For example, large firms purchase a myriad of report-writing technologies -- for example, COBOL, RPG, EzTrieve, FOCUS, Crystal Reports, SAS, and Oracle products. It becomes too expensive to deal with multiple vendors, pay lots of annual maintenance fees, perform multiple upgrades, and support and train employees on so many different technologies. At some point, companies select a winner and eliminate the other similar products. You are better to build expertise in the clear leader within a technology segment.

Companies are moving away from host-based and client/server technologies.
Today's applications must be integrated with web technologies and ubiquitous desktop products such as Microsoft Office. Stand-alone platforms will continue to decline. Even the popular client/server reporting tool Crystal Reports experienced a 30% drop in Monster job postings this year. If your skills are limited to legacy technologies, it is time to renew yourself.

Companies are moving away from paper-based and manually-intensive systems.
Systems that are heavily paper-based and require people are quickly becoming obsolete. Online systems and forms will replace paper. If a person's manual process can be automated, it will be. If your job is repetitive and time-consuming, find a way to automate your tasks and move yourself into a more essential and protected position.

Companies are offloading work to the customer through web-based and mobile applications.
For example, instead of having employees take telephone orders, pizza shops provide customers with an online ordering website and an iPhone app in order to reduce operating costs. Keep an eye on emerging cost-saving technologies and change with the times.

Companies will still build applications if those will save them money.
Application development is not dead.  Companies will gladly fund development initiatives that promise to save them money.  For example, executive dashboards with key performance indicators that visually point out positive and negative trends are popular projects.  If you love creating computer applications, be sure to have current web application development skills.



If you picked B for all 4 questions, you are aware of technology trends and probably keep your job skills fresh and in demand. If you guessed A for any of the options, you should investigate current technology trends and consider making some changes to protect your job (and stop wasting your time with quizzes in Cosmopolitan).

Another BI Vendor Acquired

The big software vendors continue to consolidate the BI market.

Today, IBM announced plans to acquire statistical analysis software vendor SPSS for $1.2 billion in U.S. cash. IBM and SPSS had already been partners to integrate predictive analytics features into the Cognos BI product.

We know that consolidations will continue, so let's just get them over with. Here is some advice on how to move things forward faster.

Which of the original companies are still standing in the weeds after the big boys pushed them off the field to play the BI game?

Information Builders and Actuate both offer BI products oriented toward developers to build BI applications -- Information Builders with its proprietary 4GL and Actuate using Java. Microstrategy, on the other hand, has an end-user adhoc reporting environment that is popular with companies having large data stores, such as those in retail or insurance.

Because Actuate uses Java and Oracle (after acquiring Sun) is aligned with that technology, let's give Actuate to Oracle. Yes, Oracle already has plenty of BI tools, but they will find a good purpose for Actuate, even if it is only used internally to build BI features into Oracle applications.

IBI and IBM. Over the decades, thousands of receptionists confused IBI reps with IBM when they showed up for sales calls. IBM already resells Information Builders' WebFOCUS product under the name "DB2 Web Query" on the Series i platform. Information Builders has strong mainframe technology and a large client base which IBM can easily leverage. Of course, WebFOCUS competes with Cognos, which IBM acquired and ported to the mainframe. Regardless, each product can be positioned into different niches and made to play nicely together.

For Microstrategy, let's give them to SAP which earlier partnered with the very-large database vendor Teradata. Many of the Microstrategy clients already store their data in the Teradata product, so this makes sense. Even though SAP already acquired competitive product Business Objects, they can put BO into an adhoc user niche while they position Microstrategy as an industry-specific application for organizations with VLDBs.

After these mergers, the only vendors not included will be the open-source BI youngsters jumping up and down shouting, "I wanna play! I wanna play!" The rest of us can take our seats and watch the game.

The Trend for Business Intelligence Software Jobs

In a 2009 June issue, Software Development Times wrote on the scarcity of software developers. Using figures from the U.S. Department of Labor and from Evans Data, SDTimes showed a decline not only in the number of individuals working in application development roles, but in the number of companies producing software products.

Fewer software jobs and fewer people to do work is a worldwide trend. While Evans Data had predicted in 2009 there would be 15.2 million developers globally, they reduced their estimate by 4%. They point a finger at the obvious culprit: the economy.

Is it possible that Business Intelligence software products might go against this bigger trend?

Throughout the 2009 year, I have tracked U.S. jobs for the various Business Intelligence software products using Monster. Today, I see a 10% decline in BI job postings from January through June of this year.

Hardest hit was Crystal Reports, which has 35% fewer job postings than at the beginning of the year, dropping from 985 in January to 639 to now. This decline may reflect a shift from desktop to web tools as well as uncertainty in Crystal's future after being acquired by Business Objects and then by SAP. Changes in licensing costs might also drive customers away.

On the other hand, peer Business Objects jobs saw a 2% growth in spite of a general downward trend. While Crystal Reports is a strong tool for building standard production reports, Business Objects offers a more flexible tool for ad-hoc reporting. Companies may be changing from Crystal Reports to Business Objects as a move toward flexible end-user architectures instead of having developers build user reports (especially if there are fewer software people).

The leader in BI job demand continues to be SAS, although their 1195 postings in January declined 4%. Another top BI product is IBM's Cognos, which shed 14% of its open jobs, going from 823 to 706 postings. Openings for Microsoft Reporting Services jobs went down 3%.

Declining demand for Crystal Reports developers is consistent with that for the legacy COBOL programming language (now 50 years old), which also saw a 35% decline. COBOL jobs went from 348 Monster postings in January to 226 now. C++ jobs also went down -- 3759 to 3049 -- for a 19% drop.

Web-based development positions are not exempt from this downward trend. While Monster did not show the actual number of Java jobs in January -- just that fact there were more than 5000 -- today the postings are at 4948. Openings for Microsoft .NET programmers went from 3885 in January down to 3351 now, a decline of about 14%.

In the Top 5 List of BI openings, the landscape is changing. While SAS remains at the top with the most demand for jobs and Microsoft stays at number 5 with Reporting Services, Crystal Reports dropped from number 2 to number 4 on the list. Business Objects took advantage of its peer's decline by moving up two notches in the list, going around Cognos to take the number 2 spot.

At the middle of 2009, the top 5 BI products in demand are:

1. SAS (increasing as a percentage of overall BI jobs -- now 25% of the BI jobs that I track; was 23% at the beginning of 2009)
2. Business Objects (increasing -- 16%; was 14%)
3. Cognos (steady/declining -- 15%; was 16%)
4. Crystal Reports (declining -- 14%; was 19%)
5. Reporting Services (steady/increasing -- 11%; was 10%)
   

In my work, I have seen demand for BI application development in two main areas. First, companies want to save money by eliminating legacy tools. Because of this, companies are consolidating tools and even changing computer platforms in the process.

Also, companies want to build BI applications such as KPI metric dashboards where cost-savings can be visually identified and acted upon.

Either way, it's the economy driving Business Intelligence software jobs. Companies want to save money by supporting fewer software products and employing fewer people.

Information Builders' Summit 2009

Information Builders' annual users' group for 2009 is over. As a sponsor of the event, I can say that that it was a great week in Nashville with several hundred customers attending. Speakers from a variety of companies did an outstanding job explaining how they are using Business Intelligence software to improve their organizations.

USTRANSCOM told how BI has enabled them to better transport wounded soldiers while reducing the number of beds and other medical needs. Ford Motor Company lowered operating costs by making warranty costs openly available on the web to their automobile dealers. Nationwide Insurance built an urban school portal to address student proficiency -- while Nationwide would not take credit, the city increased graduation rates from less than 50% to 75% and is on its way to a goal of 90% graduation rate.

You can read more about Summit 2009 on my WebFOCUS BI blog.

While there, I presented case studies on automating BI application development and consolidation tasks. For those of you who could not be at my sessions, I made a copy available on the web.