Your Privacy

This site uses cookies to enhance your browsing experience and deliver personalized content. By continuing to use this site, you consent to our use of cookies.
COOKIE POLICY

DW Acceleration | Data Warehouse Implementation Reimagined

DW Acceleration | Data Warehouse Implementation Reimagined
Back to insights

What is a Data Warehouse?

This blog entry will revisit or reimagine the tired old enterprise data warehouse (EDW) to see if there are any fresh ways we can look at it and its traditional implementations to determine if there are any gains to be made from doing so. I’m calling this exercise data warehouse acceleration.

Before we start this discussion, I think that revisiting the original purpose of a data warehouse may be in order. An enterprise data warehouse is defined as an integrated, line of business and or subject-oriented, collection of time-variant and nonvolatile enterprise-wide data. We can further define each EDW attribute as:

Integrated

Data integration implies a well-organized effort to define and standardize all disparate data elements from throughout the enterprise into one central collection. Integration can be a time-consuming process; however, well-integrated data results in a higher functioning data warehouse. Further benefits include increased data compliance and accessibility for users.

Subject-oriented

The integrated data is arranged and optimized to provide answers to questions coming from diverse functional areas within an organization. Therefore, the data warehouse contains data organized and summarized by topic, such as student demographics and human resources.

Time-variant

The integrated data includes present and historical data, allowing for in-depth historical analysis and therefore, better forecasting capabilities sometimes referred to as predictive analytics.

Nonvolatile

The integrated data that enters the data warehouse is never ―or is almost never – removed. Depending upon the nature of the decision making, data moves from operational entry into the data warehouse at a certain frequency or predetermined schedule. The source system(s) for the data warehouse are usually transactional and therefore volatile, but the source system(s) data is extracted into the nonvolatile data warehouse and preserved for future retrieval. This creates “snapshots in time” of the transactional or volatile source system data and make historical data analysis possible.

Data warehouse implementations, done properly, are massive and expensive undertakings that involve not just the information technology (IT) departments, but all areas of the organization. These projects typically involve months or even years long project timelines and teams of highly paid IT professionals and business analysts as well as subject matter experts throughout the enterprise that must be identified and then highly engaged in the effort. There is an enormous level of “buy in” or investment in thought, manpower and money from the organization necessary to make an EDW project truly successful. Along the road to a successful EDW project there are many common challenges that await such as source system data quality issues, data quality assurance, both efficient and flexible DW design and structure, data movement and ad hoc reporting performance concerns, cost overruns due to inadequate requirements gathering and or poor estimates of effort, and finally, the most important challenge is user acceptance. All the time, effort and associated costs are for naught if at the end of the day (or project, in this case), perhaps due to a lack of training or some other contributing factor, there exists a lack of confidence in the EDW by the users that it is intended to benefit. Gartner estimates that as of 2017, more than 70% of enterprise level data warehouse projects were deemed failures due to one or more of these challenges not being adequately overcome.

What is Data Warehouse Acceleration?

I know, get to the point already! OK, now that we’ve refreshed our memories on what an enterprise data warehouse is and what it can do for an organization as well as the potential pitfalls, let’s turn our attention to defining data warehouse acceleration. Basically any lift or advantage that we can give ourselves in the areas of challenge that were just mentioned can not only make the difference between success or failure in an EDW project, it can also greatly reduce the time necessary to see the positive return on investment (ROI) that is so very critical to any organization implementing such a large scale project. Data warehouse acceleration aims to accomplish exactly that. It is a reexamination of the traditional approaches to each aspect of a DW implementation project to see if there is anything to be gained by non-traditional thinking and approaches to the challenges at hand. Not every area of a DW project will readily benefit from these non-traditional approaches. Sometimes the time honored, best practice approach is still the way to go, but as we mentioned even small advantageous changes can lead to big gains in the overall EDW implementation project. That is not to say that we shouldn’t continue the quest for “the better mouse trap” in all the areas of challenge in the EDW implementation. It is almost always healthy to question accepted paradigms and reexamine traditional approaches. Unless we do so from time to time, the “better mouse trap” will never come into existence.

 

Sources:

Digging In

  • Data & Analytics

    Masking Data 101: Safeguarding PII in Your Organization

    In today’s digital age, data security and privacy are paramount. As organizations increasingly collect, store, and process personal data, protecting Personally Identifiable Information (PII) has never been more critical. One essential practice that organizations can implement at the database level to secure this sensitive information is to obfuscate it through the usage of data masking […]

  • Data & Analytics

    Unlocking the Full Potential of a Customer 360: A Comprehensive Guide

    In today’s fast-paced digital economy, understanding your customer has never been more critical. The concept of a customer 360 view has emerged as a revolutionary approach to gaining a comprehensive understanding of consumers by integrating data from different touchpoints to offer a holistic view. A customer 360 view is about taking an overarching approach to […]

  • Data & Analytics

    Microsoft Fabric: A New Unified Data Platform

    MicroPopular data services and tools often specialize in specific aspects of the data analytics pipeline, serving teams in the data lifecycle. For instance, Snowflake addresses large-scale data warehousing challenges, while Databricks focuses on data engineering and science. Power BI and Tableau have become standard tools for business intelligence tasks. So, where does Microsoft Fabric create […]

  • Data & Analytics

    Improve Member Experience: Maximize Engagement & Value for Associations

    As you know, member engagement is key to providing value and retaining members over time. However, you must also recognize that member needs and preferences are evolving rapidly, especially as they desire more seamless digital experiences. Additionally, member expectations for personalized, omnichannel interactions have risen in recent years, and this means that associations must strategically […]

  • Data & Analytics

    A Guide to Data Strategy Success in Your Association

    While countless organizations aim to harness the potential of data, few possess a clear strategy to transform raw information into actionable insights that fuel their operations and marketing efforts. Don’t fall into the trap of investing in limited, tactical solutions.

  • Data & Analytics

    ChatGPT & Your Data Strategy – Revolution or Evolution?

    You would be hard-pressed to find a single person who was not some degree of impressed when they first tried out ChatGPT. After its public release, the conversation in the tech space seemingly changed overnight about how AI would change everything. But much like past hot topics in the tech world – such as the […]