00:12 Reid Colson: Good afternoon and welcome everyone to our LinkedIn live event. I'm really excited today. We've got a great group joining me on this call. We're going to be talking about AI and data governance AI is everywhere and getting a ton of attention. I think all of us on this call know, you can't really leverage AI without good data and good data requires good data governance. Those of us who have been doing it for a while, I know it's sometimes hard to get that sponsored and we're starting to see some patterns where business executives are getting their first, what I call real hands on experience with AI through LLM and folks are finding ways to harness that experience to advance their data governance agendas. So we're going to explore the intersection of AI and data governance today and we've got a great panel to do that. These are folks that I have looked to for data governance counsel and advice for a long time now, My name's Reid Colson. I'm the consulting practices leader at UDig. We offer custom software development design, data analytics and AI as well as automation. Joining me today are Wayne Eckerson, John Duncan and Josh Bartels. And I'd to give you guys each a minute to introduce yourself and Wayne will kick it off with you. And then go to John and then Josh.
01:29 Wayne Eckerson: Thanks Reid. Thanks for inviting me. It's great to be here. Yeah, I've been in the data analytic space for over 30 years now. It's hard to believe, but I caught the wave right when it was out in the middle of the ocean and been riding it ever since. And now of course, this thing is turned into a tsunami. Thanks to AI. But yeah, I've written a couple books, lots of reports, run a consulting firm myself. And I'm glad to be here talking about AI governance with all of you.
02:02 Reid Colson: OK. Thanks for joining us.
02:03 John Duncan: Thanks Reid and again, similar to Wayne. I really appreciate the invite and the opportunity to share some thoughts today. I work for CarMax. We're based in Richmond, Virginia and I have been in this data and analytic space for 23 years and largely spent a good chunk of my time focused on data governance, data privacy, data products and the business lead for our data products, data governance and data privacy. Working closely with our technology partners and other parts of the company around legal and compliance and such. So, AI is a hot topic for us and really excited to be able to, to start to apply some data governance principles to that space and excited to talk about that today.
03:00 Reid Colson: Great. Thanks for joining us, John, and Josh
03:04 Josh Bartels: And hi, I'm Josh Bartels. I'm the Chief Technology Officer at UDig. My background is 20 years of experience in consulting from both data and a technology perspective. And I do a lot to work with Reid and other folks in our organization to help organizations that we work with blend technology and data together to create solutions that achieve tangible business benefits. And I'm really excited to be here to talk about how data and the AI both LLM and, the other technologies that exist along with data and governance blend together to look at what risks and other things that might be associated with that, but also what benefits could be achieved.
03:58 Reid Colson: Thanks Josh, we appreciate it. So kicking us off, let's start with some high-level observations around how AI is impacting data governance and maybe how data governance is supporting AI. John, I'd love to start and get your opening thoughts.
04:18 John Duncan: Sure, thanks Reid. So CarMax is I think doing a decent job really thinking about within traditional AI and now gen AI and large language models how should data governance be applied, where should it be applied? What's the level of rigor? One of the things that I thought was really important as we started gen AI governance was to help some of the business users, particularly if, if the LLM are focused on summarizing large volumes of documentation or knowledge. That's basically unstructured data. We had to spend a little time to just make sure folks understood and could equate knowledge and unstructured data to traditional data and relational databases and help them understand if we can equate the knowledge management of these documents to also be in scope for as data. And then, traditional data governance principles actually are applying quite nicely. There's certainly uniqueness and things that we have to account for. However, once we got the documents that aligned with the data concept internalized, it was a lot easier for the business and technical owners. But to understand how governance and quality management that they may have thought about or been traditional reports or model outputs. It started to click, and so I think that it was really important for us to establish that information and unstructured data early. And then the last thing I would add is, as you think about maybe knowledge management use cases, we're looking to make sure that the outputs are sometimes your customer facing, maybe even your employee facing, have to make sure that the results and the outcomes are of high quality. Certainly no less than if a human was answering those questions or searching for this information manually. And so that's another place where the input of the data and the quality around that is so is so important and that really resonates with executives. They're , yes, of course, if my customers are going to see the output of this LLM or for having associates guide customers, the quality is, is super important.
06:39 Reid Colson: Yeah, that's great. I love the simplifying framework of documents, equal data, right? Put it so the folks don't have to get lost in the language. Wayne, I love to, to hear your thoughts. What are you seeing? Thinking at a high level on the AI and governance intersection?
06:55 Wayne Eckerson: Yeah, it's really interesting. I said I've been in this space for 30 years. Well, 30 years ago, we didn't even know what the word data governance was. Right. That really didn't come into parlance until the.com bubble burst. And we had Sarbanes Oxley and then all of a sudden, the CEO and CFO who are threatened with jail, all of a sudden started thinking about governing data from a defensive perspective. So now fast forward to AI and more recently to gen AI, it, there's been so much press about not only the power but also the risks, more specifically hallucination and data security. And it's almost governance is leading this discussion, especially in corporations, right? So we're seeing a lot of companies, a lot of clients who are, requesting before they've even done anything with AI about how to govern it and putting policies in place before they even implement anything of substance. So it's an interesting role reversal that governance now is almost leading our efforts, at least, in corporations on this score.
08:08 Reid Colson: Yeah. That is interesting moving into the front end. Josh Thoughts.
08:17 Josh Bartels: Yeah. So because AI and gen AI in particular utilize these large language models, they require so much access to data, right? So where I see governance playing in that and a lot of conversations that we're having is because you have to have access to so much data. What's the ownership and I how do I as an organization maker control who sees what? Right. So a lot of the conversation often becomes, OK, well, if I'm going to give it all of the documents that equal data that John's talking about, right? How do I classify that data so that the right people see the right answers, right? And so I think that's one of the large areas of as we give AI or gen AI more access to more of our organizational data, right? How do we use appropriate governance structures to make sure we control who, what has ownership of it and then who can see the results of it, right? Even if it's summarizing it or, or whatever it may do, right? whatever access I have in SharePoint doesn't mean it's the same access AI has, right? So how do we equate to those, those things where, where it's consuming the documents and then having to return answers,
09:30 Wayne Eckerson: Josh, that was the first question a client asked us, they had prototyped an application which was compiling documents that were official documents that anyone in the organization could see. But the next step was, oh well, we have other documents that not everyone can see. So how do we control access to the gen AI output. And to be honest, I've never gotten a good answer about that. Maybe with a little bit of RAG architecture. But you may know more than I,
10:03 Reid Colson: I, I'm wondering, this is thinking out loud, probably shouldn't do this on a call. But, we talk about access control and roles that we set up for individuals within our organization. Maybe there's different AI roles and there's different AI tools that fit different roles where you're going to segment the different AI capabilities through some role architecture that would then allow certain aspects of it to see certain documents and not others. Then maybe you can line that up to the access roles that individuals have,
10:33 Josh Bartels: Thought, it comes back to classification, taxonomies and anthologies, which are part of governance, right? But now we're putting it against the documents and the data that are generated right from those documents. And then Reid said, often we have to figure out how do we put that to roles, right? And access. So now we have appropriate taxonomies onto that, classify the documents and then appropriate roles that get associated with them,
10:59 Wayne Eckerson: But you can't really control what the LLM is going to produce, right?
11:04 Josh Bartels: That's part of the issue.
11:09 Wayne Eckerson: So with this client, their first thought was, well, maybe we have to have a separate chatbot for each role, right? And only feed the chatbot, they got the documents that that role can see. But the other option I guess is to use a RAG based architecture and use the RAG to query the vector database with access controls that get passed along. So only the data. So the RAG only feeds the LLM the data that the person is allowed to see. I don't know if you've seen that work at all.
11:40 Josh Bartels: Yeah. The only instance that I've seen but I haven't seen it work very well yet is that's what Microsoft's trying to do with Copilot. And the one that they're opening up to to all office users or office 365 users. So they are using some version of that, the vector only takes in what that user has access to and only responds to what that user has access to. So if I ask about something in some document repository and I don't have access to it, it doesn't generate a response.
12:12 John Duncan: I know you probably want to get to the next topic, but I would just add quickly that I think another piece that's really important is the methodology and the SDLC approach that you apply to building and deploying these LLMs. We think often about certifying the data that is both responsibly ethical but of course, meets the minimum thresholds of quality who's the owner of that data. And being really intentional before allowing it to integrate into the production model. I think the other piece that's really important is how effective are you as an organization to review the outputs and, and train the model who's watching, particularly the early days of the outputs and how well and what needs to be true about how well it has been tested and certified before you're then making it more broadly available, either at scale or to your customers or to your employees.
13:14 Reid Colson: Yeah. All good John. I'm loving where the conversation is going. You guys are taking it beyond my question. So thanks. I think you've already started to go down this path. But as data governance professionals, how do you guys see AI presenting risks to an organization that they need to start thinking about within their strategy? You've covered some of this already, maybe when I'll just come back to you and, and, and get your thoughts to get us started there.
13:40 Wayne Eckerson: Yeah. So when we were working with one client,we actually compiled the risks that they would need to consider and we came up with 16, believe it or not, which is quite a lot, everything that we've mentioned access control data security, if people put prompts in and get data back, including your own data from your own documents, does that get shared with the LLM provider? How fresh are the answers, right? Is the model of giving you year old answers to explain ability, transparency, environmental considerations these things chew up a lot of CPU and chew up a lot of energy as a result and it just goes on and on. So there's just a lot of things to consider. once you're implementing gen AI and, and we need to consider all of them and implement appropriate controls to ensure that we don't take unnecessary risks.
14:47 Reid Colson: Yeah, Josh, curious to get your thoughts on how you think about the risks that AI presents to organizations.
14:57 Josh Bartels: Yeah. So, I think Wayne brought up quite a few, I think another one that I think about a lot is bias in fairness. I think Wayne said hallucinations earlier, right? And so how do you, how do you make sure that you're training the model appropriately? Right? So, thinking about it and how governance potentially could play into that risk, right. So, if we've done something that reads, let's say, resumes for hiring, right? As an example right, and it used past historical data, there could be bias in how we hired historically in the past, right. So how do we reflect on that and governance? This is where governance starts to expand, right. So data governance now becomes part of AI governance. And John was talking about this a little bit of making sure that we have a standard and and we approve and vet our model, right? So now there's this governance approach about biased detection and mitigation and making sure that we audit what AI is doing and, and providing to us, right? So that we potentially get equitable results or at least even results that we can trust, right? And so I think that's one of the risks, but I think governance is a great place to help address that risk, and put some of that in the framework that you have for governance because it's really relying on the data that's underlying and underneath.
16:21 Reid Colson: And John, I'll let you close this out here.
16:24 John Duncan: Yeah, I think that Josh and Wayne have covered a lot of the bases. I would just say that I would definitely emphasize Wayne's point around really being clear on the tools that you're using and how they do or don't integrate back to the LLM provider or the interface provider an unintended consequence, for example, could be even a simple use case within that prompt you're putting in as maybe a draft strategy document for your, for your company just so that it can summarize all of the things that make it more consumable for your executives. Well, you got to be really careful where that output goes and does it integrate back with the provider? that would be risking IP for for your company potentially, particularly if it's not public. So, I think that they're just really, really thinking about the security, really, really thinking about unintended consequences of data sharing of those results or of the input prompts as well.
17:24 Reid Colson: Yeah, all great ideas. I think the one thing I would add just from conversations that we've had with clients that from an organization they're starting to think about is a lot of the things that AI is able to do is taking the role of, say, an apprentice in the beginning of a professional career. And so as they start to use AI for things, research briefs and in the legal space or underwriting assistance in insurance, how do they train staff to become the next experts if they're going to be using more AI in the spaces? So there's a whole talent management piece. That's a risk that's presented to organizations to think through as well with, with how they're thinking about deploying and leveraging AI
18:07 Wayne Eckerson: One more thought about risk is that they're going to get compounded as companies become more sophisticated in the use of these things because they're going to start chaining models together to do things, planning and, and task management. And so it's not just one model that you have to manage and monitor and evaluate. It's going to be multiple models, maybe even thousands, right.
18:33 Reid Colson: A lot of thoughtful inspection as you continue to grow with it that you're going to have to do. Yeah, for sure. So, with those risks, what do you guys see as best practices in managing the risk, how should governance teams think about their roles going forward? Josh, I'll let you start as we round robin this and get your thoughts there.
18:55 Josh Bartels: Yeah. So I think we talked a little bit about it when we were bringing up the risks, but if I'm thinking about what's the reflective side of it, I do think data governance expands into this team that has a mix of traditional data governance responsibilities and potentially has to have some innovative practices that go along with it because traditionally, right governance has stewards that own or have ownership related to the data and responsibilities related to it. But as AI expands in the the data space, that group needs to conform and become potentially larger, have more roles and responsibilities, right? We talked a little bit about bias mitigation, but we've talked a little bit about so just to name a few, right? Privacy protection, right. So how does that team start to look at enhancing anonymized data or encryption techniques as they're passing it between whatever tool they're using or whatever data sets they're using for regulatory compliance, right? We are in the early stages of AI, there's not a whole lot of regulation is coming out, right? And how do both federal regulation and state regulations apply to organizations and how do they have to handle that if they're using AI when you brought this up, transparency and explainability? Right. So how do we mitigate risk by trying to figure out how transparent and what is in the model so that we can explain it to our stakeholders, our executives, right? As we try and get more resources for governance that needs to be part of the story that we need to tell and then lastly continuous monitoring, right? How do we continue to check in on what's going on and monitor that practice and become more efficient at making sure that things are going right? And governing AI really.
20:43 Reid Colson: John, I'm really curious to get your thoughts because you're living this in a company right now, right? How are you guys thinking about it? And what's the role that your team is starting to take on?
20:52 John Duncan: Yeah, I think, I think many of the points made are right on. I would say probably a little bit tactically or operationally. I think having a great 360 view of your risks, data governance being one of them is really important. And so if you've got a use case review team, someone that a group of folks, hopefully fairly senior folks that are intentionally thinking through and, and fielding the use cases that folks are asking to be solved with JAI or with an LLM and honestly having a matrix. I think one of the things data governance folks can do well is help people think about risk versus value on a bit of a two by two where you're really saying, hey, while we're still in learning mode and trying to just figure out if there's any value in this particular LLM or the the problem we're trying to solve. Is this the right tool to use? I think that you could be a little bit more relaxed and a little more looking for folks to be innovative. Not necessarily day one being really, really concerned about every risk, but I think that you've got to have a way to think about the progression of that use case. When will it go from just learning to maybe a pilot or MVP? When will it then move from that pilot to scale and whatever scale means for your business or for your agency? And I think that having that use case team team being accountable for not just helping define the risks and the value, but intentionally signing off and helping drive when we move from from being a learning exercise to a pilot to scale. I think that operationally is something that I think data governance professionals can add a lot of insight on how to structure that and set that up even though the risks and the value are more than just the data governance components.
22:46 Reid Colson: Do you see roles evolving to take on that work? I think about when organizations really start to get serious about privacy, there were privacy officers, credit officer to review use cases, do you see things evolving that way?
23:01 John Duncan: I think that AI governance from a model governance gen AI I think you could look at some of your whoever you've got is either a risk officer. Certainly folks that are accountable for certifying the output of it if you're in the credit business, you've got your credit risk officer, other folks that are already in place to really help, think about that. So it could be that it's a new role if AI generally is not something that's in your shop or in your ecosystem of risks. But I do think that similar to the way that we think about LLM and, and input of documents being equated to traditional data, you could probably look to some of the same C level executives or senior folks to help extend their current accountability to include things gen AI and LLM.
24:00 Reid Colson: Cool. And then, Wayne, I'm going to maybe piggyback the question I asked with when we got on the panel where they're asking, do you see AI governance being a separate thing or more incorporated into broader data governance? So you think about the best practices, your thoughts there and then the different org or alignment models that, that you're seeing?
24:20 Wayne Eckerson: Yeah, I saw that question too. I think it's a good one. , if you have a pretty mature data governance program with established policies, process procedures and committees, I would want to put AI governance under that umbrella and leverage that framework and structure it to create an AI task force or subcommittee that looks at that now, might be a pretty big one, right? Because of all the risks that need to be mitigated in the lifecycle management that John just mentioned. However, if you don't have a very sophisticated data governance operation, you really want to get into gen AI in a big way, you definitely would want to establish it as an independent group, I think.
25:11 Reid Colson: Mm Yeah, I think that makes a lot of sense. So I'm going to pivot a little bit and, and I'll tell you where this is coming from. So, my experience over the past, I don't want to even say how many years? 20 plus 25 I've seen a lot of progress on lots of different dimensions of what we do generally in data, right? How we do storage, how we do compute the algorithms we bring all have made just massive and rapid transformation. A lot of the process of doing data governance is still we were doing in the early days. So I'd be curious to see if you see anything emerging on the AI capability front that has the potential to improve the way data governance works or is operated. And I'll just leave that open and whoever , wants to, to jump in can
25:57 Josh Bartels: From the technology nerd side of the house. I think there's a couple things that I think are potentially interesting, right? so in governance, when we go to set stuff up right there, we talk a lot about cataloging, right? We have to understand all of our data, right? I think AI has the potential to help create efficiencies or automate cataloging creation. I think there's always going to be the human being pair eyes that are going to have to go against it, but it's not going, yeah, we have tools today that can spider into structured data. But now with AI, we can spider into unstructured data and our catalog can become more of this beneficial thing, not only across our structured data but our unstructured data.
26:44 Josh Bartels: I think there's also opportunity for improvement in data quality, right? How can we use AI to help generate better data quality rules or monitor data quality purposes? Right. So we feed AI based upon the data that we have. But if the quality of the data that we have that we're feeding AI with, we're going to get bad answers. It's an old adage and data that always is whatever goes in comes out, right? So how do I think there's potential for AI to help with that? And lastly, I think is there going to be a shift now that we can get efficiencies in some of those things that it can help with the decision making side of governance, right? So hey, we have three different business lines, we have three different definitions of customer. Can AI help us generate a common definition for customer and then help us enhance that decision making process where we all have to come together and create that one eternal definition for customer, right? I think there's some opportunities there. So that there's way more than that, but those are the three that I'll leave it with and, and hand it off to John or Wayne there to give their ideas.
27:52 John Duncan: I would just add maybe 30 seconds more to that. I just so happened to earlier this year, be able to attend Gardner's Data and Analytics conference. It was all focused on the emergence of gen AI. One thing that was clear walking through the hall where all of the the vendors and the technologists lived is all the traditional data governance players are working really hard to extend their products to think about how do they take unstructured data and apply many of the same structured data tools and techniques. And so I think that as with every maturation or evolution, as you described Reid, I think you've got to proceed a little bit cautiously, let the industry settle a little before maybe making big investments in these tools. They need to mature a bit it's a very rapidly changing environment from at least the my lens and the seat that I sit in. But I do think that some of the traditional strong players in the space for traditional data quality are rapidly gearing up to help bring LLMs and AI into them.
28:57 Wayne Eckerson: Yeah, I, I think thinking about AI governance, the hallucinations are really scaring people. And so I think there's been some investment in both a technical approach to that to mitigating that and a human approach. So we're starting to see the advent of model evaluation software such as long chain evaluation that can automatically help you score on the output and its credibility. We're also seeing companies put up the documents as a way of attributing the answers that they're generating to give more confidence to it. But then we're hearing also a lot about red teaming, which is getting people together to try and break the LLM and get it to generate bad answers and what does it take to do that? And what are the parameters by which that would have to be done to cause the thing to hallucinate. So I think there'll be a lot of investment in that area because if at least for corporations if it's giving bad data incorrect data biased responses. It's too risky to use. Right.
30:12 Reid Colson: Very cool. Well, I appreciate it, this is fantastic. I think you guys showed why I always come to one of the three of you when I have questions on data governance. Certainly appreciate your time today maybe if you guys have time, I'd love to give you a minute. Anything else that we didn't touch that, that you'd to share?
30:35 John Duncan: I'd only say Reid that one of the things that I think is important here is when we talk about governance, it tends to be, we can quickly get into textbook, we can think about all the things that we should be doing, whether it's traditional data governance and extends to AI governance. I think that it's just important to be really prioritized and where you want to focus, whether it's at the use case level or the data level. And I think progress over perfection is key. I think that companies can't wait for all of this to be flushed out before they start to really think about the value that you get from LLMs. But I do think that in parallel while that innovation is happening some smart short term high value data governance is important. And but I think progress over perfection.
31:25 Wayne Eckerson: Yeah, I think the first step for a lot of companies is to create a personal use policy. So employees know what their responsibilities are when using these tools and whether they can even use them at all on their own or company provided LLM, what responsibilities do they have to protect the company's reputation, its assets, its data, it's PII data and things like that. So there's a lot of education that really has to happen first and foremost, otherwise you'll end up like Samsung, that shuts the whole thing down because there's a huge data breach. Someone puts a source code into the LLM and it filters out and into the hands of other companies.
32:15 Josh Bartels: Yeah, I think the last thought I'll leave with is that data governance often is the last thing that an organization wants to look at or, or do, right? They always want the pretty shiny object. I think AI is bringing governance and good data practices to the forefront. And I think that's exciting for our organizations because they want to use the new shiny object, which is AI. And now this is an opportunity to really go back to think about the governance of your data and the strategy related to your data and really start to think about what are those use cases. What do we need to do to get ready and, and start to apply so that we can actually hit these AI use cases that we want, right? So it's really bringing to light these things that we've been talking about for, for many years, but now have come to the forefront to make sure that we get good quality results.
33:12 Reid Colson: Yeah, thank you guys. And, and for me, I'd say I've always lived by, at least when I think about trying to get projects sponsored internally, never waste a crisis or an opportunity. I think AI for most of your companies represents both. So figure out how to tie the governance agenda to the things your organization is either afraid of or wants to do with AI to be able to get the resources that you need to do a good job supporting your organization and enabling those, those opportunities with AI. So that's, that's really it in terms of what we had prepared. I really appreciate you guys joining today and making time out of your day. Really insightful thoughts that you guys shared with us. And so I appreciate that and thanks to everyone who was able to join us for the call.
