Unlocking Data Insights for a State Education Agency

Challenge: Segmented Data Environment Hindering Legislative Reporting Requirements

Our client had 13 years of data they were managing across 3,000+ data sources. Despite having this robust amount of data, they had difficulty responding to legislative reporting needs, causing delays and inaccuracies. Their legacy data was siloed, creating a cumbersome process for storing and retrieving data. Further, the data sources were all varied in quality — raw, aggregated, clean — making it nearly impossible to know which data version employees were processing.

This existing data environment meant they: 

• Lacked a cohesive repository for storing and managing data 
• Relied on laborious manual reporting processes across various teams and departments 
• Were unable to quickly respond to governmental reporting demands 
• Were unable to conduct effective data profiling  
• Lacked transparency into their data 

Strategy: Modernize and Streamline Data Reporting Capabilities

Our initial priority was helping our client meet state-legislative reporting requirements. The agency needed a data environment that enabled them to seamlessly provide on-demand and predictive analytics. Also, at any point, their reporting requirements could change, meaning their solution had to enable their teams to swiftly respond to any new criteria. To support this dynamic capability, the state education agency needed a data mart that could become the best source for any descriptive reporting on academic success. To do so, we worked in two phases: 1. Discovery and 2. Implementation.

Phase 1: Discovery 
To meet our client’s goals of upgrading and centralizing their data environment, we first had to do a data deep dive. We worked with their team to profile their data and organize it, so we understood what they had, what was valuable, and what they no longer needed. Further, we needed to identify how the data must work for them, so we could define the best build approach to meet their needs. 

Phase 2: Implementation
Once we conducted this discovery phase, we began building their data platform. We had to ensure our solutions met these goals: 

• Create two data structures to consolidate their thousands of legacy data sources. They needed a data lake to organize historical and ongoing data sources, as well as future operational reports. They also required a relational model that contained non Ed-Fi data and Ed-Fi–based data store and allowed reporting across these two systems.
• Design a data-ingestion process to load each of the legacy data sources into the new data structures. This process had to be seamless while enabling them to meet future reporting demands.  

As we worked with our client, we became a steadfast partner to our stakeholders and helped represent the business interests of how and why to best implement each solution.  

Outcome: A Data-Enabled Agency Empowering the Public with Actionable Information

Today, our client is a data-enabled agency that can swiftly and accurately report education-centric data at any moment. Their cloud-based data repository centralizes and stores all data, providing a cost-effective solution for predictive-reporting capabilities around academic success. With more data visibility, they can turn insights into actionable strategies. Further, thorough data profiling capabilities enable their team to better understand their data sources and how to best organize the information. They also have purpose-driven data warehouses focused on specific subjects, which they can further build out, as needed. This environment has optimized their robust analytics and data science capabilities and set them up for future demands. By building this foundation, all stakeholders — from the parents to legislative bodies — have more visibility into academic success, driving more informed decisions and actions.