Feel Like Your Pants Are Down – Check Your Data

Few things are as embarassing (cool double entendre) as having your pants pulled down in public.  While it is great fun to see it done figuratively to someone else (real world on the grade school playground), it can really cramp both careers and management practice in the corporate world.

Just when you have the fought for budget money to put into some cool analytics or reporting…the project is up and running, you’re day dreaming about keeping your job (maybe a raise!). Then, the project status rolls across your desk: bad data, instant project over-run.  The problem is worst-case scenario; bad data tends to be discovered just as the project goes live with no quiet opportunity to fix it.  The money is gone and everybody is looking (yep, just like second grade, lunch money is gone and…).

Don’t you just love going back to the C-Suite for more  project budget (feel like you’ve just been pantsed?)  Ouch!  This one needs to be avoided at all costs.  How to do it is the tough part.  Does anybody really want to go into the C-Suite and say, “Our corporate data may not be, uh, kosher?”  There is a real career enhancer for any CIO.  So you need to do what any red-blooded CIO would do, bury the cost in another part of the project.  No one wants to champion a project entitled “Fixing Corporate Data Assets”.  How did they get messed-up in the first place?  Guess who gets blamed?

No one wants to hear about legacy code and budget reductions.  There is no wide-eyed fascination regarding aging databases suffering from too many corporate mergers and grafted on must-have emergency applications.  Well, if they don’t want to hear about that, why go through it.  Don’t tell me problems, get me solutions!  Well, here you go, as requested.

Instead, early in an analytics or reporting initative, put in a sub-task to test existing data for proper compliance to the stated information delivery  goals (see that doesn’t feel scary, just numbing).  Overall match to corporate data governance and meta data management standards can be tested at the same time, who could argue with that?  Just a little bit of testing, call it project (career)  insurance.

The next step should be obvious, even to a CIO.  Slide one of these sub-tasks in each project initative (they go togther like “sliders, beer, and baseball”!).  Before you know it the Corporate Data Assets are in tip top shape, without the nasty embarassment of a major budget line item.

It’s the End of the World As We Know It!

The Holidays are a great for watching “End of the World” shows on the History Channel. They were a great comfort, actually almost encouraging, because all of the prophecies target 2012.  “The Bible Code II”, “The Mayan Prophecies”, and the Big 2012 Special compendium of End of the World scenarios, covering Nostrodamus to obscure German prophets, all agree that 2012 is the big one (Dec 21 to be exact!)  What a relief!, the rest of the news reports are trending to canned goods, shotguns, and gold by the end of the year.  We really have almost a whole 48 months before everything goes bang (I wasn’t ready anyway, procrastination rules!).

Unfortunately, we need to do some IT planning and budgeting for the new year and probably should have some thoughts going out 36 months (after that see the first paragraph).  As I discussed in a prior blog, the reporting, BI/CPM/EPM, and analytics efforts are the strongest priority; followed by rational short cost savings efforts.  All organizations must see where they are heading and keep as much water bailed out of the corporate boat as possible.  Easy call, job done! 

Then again a horrifying thought occurred to me, what if one of these initiatives should fail? (see my nightmares in prior blog posts on Data and Analytics).  I am not saying I’m the Mad Hatter and the CEO is the Red Queen, but my head is feeling a bit loosely attached at the moment.  Management cannot afford a failed project in this environment and neither can the CIO in any company (remember CIO=Career Is Over).

The best way to ensure sucessful project delivery (and guarantee my ringside lawn chair and six-pack at Armageddon in 2012) lies in building on best practice and solid technical architecture.  For example, the most effective architecture is to use a layer of indirection between the CPM application (like Planning & Budgeting) and the source data systems (ERP, Custom transactional).  This layer of indirection would be for data staging, allowing transfer to and from fixed layouts for simplified initial installation and maintenance.  In addition, this staging area would be used for data cleansing and rationalization operations to prevent polluting CPM cubes with uncontrolled errors and changes.  In terms of best practice, libraries and tools should be used in all circumstances to encapsulate knowlege rather than custom procedures or manual operations.  Another best practice is to get procedural control of the Excel and Access jungle of wild and wooley data which stands ready to crash any implementation and cause failure and embarassment to the IT staff (and former CIO).  When systems fail, it is usually a failure of confidence in the validity or timeliness of the information whether presented by dashboard or simple report.

CPM, EPM, and Analytics comprise and convey incredibly refined information and decisions of significant consequence are being made within organizations to restructure and invest based on this information.  The information and decisions are only as good as the underlying data going into them.  So skimping on the proper implementation can put the CIO’s paycheck at serious risk (Ouch!).

What is Manualytics and why should I care?

Manualytics (manual analytics) is the labor intensive, manual process of creating information. It involves finding, loading, correlating and consolidating data into spreadsheets to answer a particular question.

The business leadership asks a question which initiates a number of analysts to start sift through the silos, usually with spreadsheets.

The answers from different departments don’t agree. That spawns a second round of analysts with spreadsheets trying to prove whose numbers are correct.

The business is left with an answer they don’t trust which leads to decision making with inordinate risk.

Does this sound familiar? Can you give examples in your organization that resemble this process? You are not alone. Manual analytics exists in virtually all organizations because the business can create questions faster than IT can provide answers.

The problems with Manualtyics include:

• It is inefficient and labor intensive
• It produces inconsistent results and can compounded errors
• It buries complex business logic in spreadsheets
• It introduces uncertainty and confusion
• It engenders mistrust of the data
• It leads to risky decisions
• Second and third layer of analysis
• Data Untraceable from Target to Source – compliance anyone?

One of the largest problems is that Manualytics is a hidden overhead cost/activity in many organizations. If the manual spreadsheets (or any desktop system) are run every month and are business critical – then they need to be productionalized and automated.

If you don’t have a program to improve your BI capabilities and limit/reduce the amount of manual analytics then you are treading water at best.

This siloed architecture and manualytics activities describes what we call a typical BI current state. We see this situation to varying degrees, in one form or another, in virtually all organizations.

So how do we break out of this cycle?

How do we position our systems and people to obtain actionable information?

How do we overcome our siloed architectures and maximize our long term IT investments? 

Why Analytics Projects Fail

During an informal forum recently, (whose members shall remain nameless to protect my sorry existence a few more years), analytics projects came up as a topic.  The question was a simple one.  All of the industry analysts and surveys said analytic products and projects would be hot and soak up the bulk of the meager discretionary funds availed a CIO by his grateful company.  If true, why were things so quiet?  Why no “thundering” successes?

My answer was to put forward the “typical” project plan of a hypothetical predictive analytics project as a straw man to explore the topic:

• First, spend $50 to $100K on product selection.
• Second, hire a contractor in the product selected and tell him you want a forecasting model for revenue and cost. 
• The contractor says fine, I’ll set up default questions, by the way where is the data?
• The contractor is pointed to the users. He successively moves down the organization until he passes through the hands-on user actually driving the applications and reporting (ultimately fingering IT as the source of all data).  On the way the contractor finds a fair amount of the data he needs in Excel spreadsheets and Access databases on the user’s PCs (at this point a CFO in the group hails me as Nostradamus because that is where his data resides).
• IT gets some extracts together containing the remaining data required that seems to meet the needs the contractor described (as far as they can tell, then IT hits the Staple’s Easy Button —  got to get back to keeping the lights on and the mainline applications running!).
• Contractor puts the extracts in the analytics product, does some back testing with what ever data he has, makes some neat graphics and charts and declares victory.
• Senior management is thrilled, the application is quite cool and predicts last month spot on.  Next month even looks close to the current Excel spreadsheet forecast.
• During the ensuing quarter, the cool charts and graphs look stranger and stranger until the model flames out with bizarre error messages.
• The conclusion is drawn that the technology is obviously not ready for prime time and that lazy CIO should have warned us.  It’s his problem and he should fix it, isn’t that why we keep him around?

At this point there are a number of shaking heads and muffled chuckles; we have seen this passion play before.  The problem is not any product’s fault or really any individual’s fault (it is that evil nobody again, the bane of my life).  The problem lies in the project approach.

So what would a better approach be?  The following straw man ensued from the discussion:

• First, in this case, skip the product selection.  There are only two leading commercial products for predictive analytic modeling (SAS, SPSS).  Flip a coin (if you have a three-headed coin look at an open source solution, R or ESS), maybe it’s already on your shelf, blow the dust off.  Better yet, would a standard planning and budgeting package fit (Oracle/Hyperion)?  The next step should give us that answer anyway, no need to rush to buy, vendors are always ready to sell you something (especially at month/quarter end – my, that big a discount!).
• Use the money saved for a strategic look at the questions that will be asked of the model: What are the key performance indicators for the industry?  Are there any internal benchmarks, industry benchmarks or measures?  Will any external data be needed to ensure optimal (correct?) answers to the projected questions?
• Now take this information and do some data analysis (much like dumpster diving).  The key is to find the correct data in a format that is properly governed and updated (no Excel or Access need apply).  The key is accurate sustainability of all data inputs, remember our friend GIGO (I feel 20 years old all over again!).  This should sound very much like a standard Data Quality and Governance Project (boring, but necessary evil to prevent future embarrassment to the guilty).  
• Now that all of the data is dropped into a cozy data mart and supporting extracts are targeted there, set up all production jobs to keep everything fresh.
• This is also a great time to give that contractor or consultant the questions and analysis done earlier, so it will be at hand with a companion sustainable datamart.  Now iterations begin – computation, aggregation, correlation, derivation, deviation, visualization, (Oh My!). The controlled environment holds everybody’s feet to the fire and provides excellent history to tune the model with.
• A reasonable model should result, enjoy!

No approach is perfect, and all have their risks, but this one has a better probability of success than most.