Does the following situation sound familiar?
Your complex organization has enormous amounts of operational data stored in everything from Excel sheets and Access tables to clustered database servers from multiple RDBMS vendors. Users from across the management spectrum frequently ask for ways to access that data from different perspectives to inform their decision making. As a competent IT professional you and your team have indentified that a Data Warehouse based Business Intelligence solution is the best approach because its flexibility and high performance will provide long term value to the enterprise. Now it’s time to get buy-in and budget for the project from the management hierarchy.
These initiatives often have high level visibility within the organization because the analysis and decision making they facilitate is done at the C-level and Senior VP level. Unfortunately, at some point in their careers these executives have been disappointed by the results of a “Data Warehouse” project. In some organizations “Data Warehouse” is almost a pejorative term. This is not because there is anything inherently wrong with the technology. It is because too often these projects have failed to deliver the expected results, or failed completely and end up unused. Part of the problem is the way the projects have been executed:
So how do you reduce upper management anxiety and get the buy-in and budget you need to build the Data Warehouse solution that you know will work for them? A good strategy is to use a technology stack and a project methodology that shortens the “Time to Value“ for the project, demonstrates value at the earliest stages, and delivers new production ready functionality every three to four months.
At Pragmatic Works Consulting we have had great success with Data Warehouse projects using the MS BI Stack technologies coupled with the project approach describe above. We call this approach our “Agile BI Methodology”.
We have found that the Microsoft BI stack that comes with SQL Server 2005 and 2008 is hands down the best technology available for building Data Warehouse solutions. BIDS provides a single, consistent developer UI for all phases of the project (ETL, OLAP cubes, and reporting). In addition a vibrant ecosystem of BI developer productivity tools and addins has developed around BIDS that have more than doubled the productivity of our project teams.
The “Agile BI Methodology” we use begins with a “Quick Start” week at the very beginning of the project. During this week we identify a specific, high profile, Business Analysis problem and build a Data Warehouse prototype solution for it. The prototype includes ETL to load at least a subset of the needed data, as well as an OLAP cube with a couple of fact tables and dimensions, plus a presentation layer connected to the cube such as a flexible report or an Excel Pivot Table. At the end of that week we bring in the business stake holders at the highest level and show it to them. At that point we ask them for either a red light if we’re on the wrong track or a green light if they like it. So far it has always been a solid green light, and of course we always get suggestions and new ideas. On one occasion a VP of Markeing commented “Wow. This is answering questions I didn’t even know I could ask.” The absolute key goal of this Quick Start week is to show results to the decision makers early in the investment cycle, and get their buy-in. Once we have it, we map out a project plan calling for a series of 3-4 month long “Sprints” that will each deliver an additional production ready BI/Data Warehouse solution or enhancement. Again, this tight schedule and feedback loop reduces the anxiety for the decision makers because they know it cannot turn into the endless Data Warehouse project from hell that they may have experienced in the past. During the project approval process make sure they understand this.
Another advantage of this project methodology is that it can provide an excellent forum for mentoring a project team up the learning curve for the BI stack technologies. Some clients have asked us to provide a team that executes the “Sprint” build outs completely. But at others our role has been to mentor an internal team during the “Quick Start” week so they can execute the Sprints themselves. In these cases, it has proven effective for the mentor to be available remotely during the project sprints, or to return onsite periodically for additional mentoring and to help get the tougher solution components built. However, regardless of who is doing the work the frequent feedback and deliverables keep the project and solutions aligned with the business stakeholders’ needs and reduces their level of anxiety around the initiative.
A successful BI initiative can deliver significant value that has visibility up to the highest levels of the organization. The IT team that delivers the solution can be real heroes. The “Agile” methodology described in this article can help you get your BI project approved and started quickly, as well as dramatically improve its chances for success.