Innovate with Ludia: The Dynamics 365 Physical Operations Podcast
Join us on the 'Innovate with Ludia' podcast for discussions on Physical Operations, including D365 Field Service, IoT, Mixed Reality, D365 Project Operations, AI, and more. We'll cover industry challenges, new features, and insights from Microsoft, MVPs, and ISVs.
Innovate with Ludia: The Dynamics 365 Physical Operations Podcast
Mastering AI: Navigating Challenges and Ensuring Success with Generative AI and Microsoft's Co-Pilot
Ever wondered why AI implementations can be so hit-or-miss? Unlock the mysteries of generative AI and Microsoft's Co-Pilot with insights from my esteemed colleague Murray Fife, as we tackle the excitement and confusion surrounding AI technologies. We break down the differences between tools like Microsoft's Co-Pilot and Azure OpenAI, offering practical advice on setting up guardrails and structured prompts to ensure your AI initiatives are both advanced and reliable. Learn firsthand the strategies for managing AI projects from defining clear inputs and outputs to navigating the iterative nature of AI development, ensuring your projects align with expected results.
From achieving consistency in AI responses to balancing flexibility, we cover it all in this episode. Hear about the importance of early planning, preprocessing data, and structured test cases to achieve cleaner inputs and better outcomes. Our conversation offers valuable lessons on maintaining reliability and contextual appropriateness in AI outputs, making this episode an indispensable resource for anyone involved in AI project management. Whether you're at the start of your AI journey or looking to refine your existing projects, this episode promises actionable insights and real-world experiences to guide you through the complexities of AI implementation.
Don't forget to take a look at the book: Quick Start Guide to Large Language Models by Sinan Ozdemir
Good morning, good afternoon, good evening everyone. This is Scott LaFonte, your host for Innovate with Ludia. I am here again with, as you probably all see, my esteemed colleague and guest Murray Fife. Murray, how are you today?
Speaker 2:I am just great For anyone who's watching. Then I'm the one without the glasses. I know we both look so handsome and very, very similar.
Speaker 1:That's right and you're the better looking one, so I'll be Pat Sajak. You can be Dana White, sure. Well, murray, I know it's been a while since our first discussion a while back and we were talking with Marcio and Michelle and talking about things around field service, and today I really wanted to talk and share our experiences and ideas and leadership around AI, what it is. Everyone talks about AI, generative AI, microsoft Co-Pilot. I think there's a lot of confusion. There's a lot of excitement, but there's also a lot of confusion. There's also a lot of pitfalls or potential pitfalls that partners or implementers and customers can fall into, and so I think we've learned quite a bit here at Ludia with the various projects that we've undertaken, to see the successes, the trials, the tribulations and everything in between.
Speaker 2:Yeah, and we've learned a lot of things and we'll talk about some of these. But one of the big things is AI is a little bit squirrely. It doesn't always do what you want it to do, but that's just the nature of it. It's got its own subjective sort of format. It's got a mind of its own sometimes when it goes out and does things. So the training or the putting guardrails around the AI is something that we found that what is the bulk of the project? Actually? Getting it to spit something out, that's fine, but getting it to give what we want, then that's a little bit harder to do and and getting it what you want on a relatively consistent basis.
Speaker 1:On a relatively consistent basis is another trial and an area where we've seen some challenges, because everything is based on how you ask a question or how you phrase a prompt versus how I'm phrasing it, and so those are some of the things that we see time and time again. But so if we take a step back, I mean, what would you say in terms of everyone says co-pilot. To me, co-pilot is still a form of generative AI, and so there's co-pilot, which has generative AI functions, and then there's Azure Open AI, which has generative AI functions, and then there's Azure OpenAI. In your opinion, talk to me a little bit about what you see as the differences between a Microsoft Copilot AI and an Azure OpenAI in terms of building those out.
Speaker 2:Yeah, and really for Microsoft, the foundation of all of the AI is the Azure AI. So there's all of these tools out there inside of Azure that things are built on. There's the Azure OpenAI, which is really the GPT implementation of the language model. So that's the general knowledge, the smarts that allows it to string sentences together and take something and process it. And then there's the co-pilots, and the co-pilots are given information and then asked to do something with it. So, for example, a co-pilot may be given some sales information on a customer and asked could you create an email from this? And then so that's going down to the language model in the GPT to go out and create that language model in the GPT, to go out and create that.
Speaker 2:Or, on the finance side, it may be saying look at this vendor and here's all the information that I've got about it. Create a summary, tell me about this vendor so that I don't have to go searching through all the pages to find this information. So the co-pilot is just like what it sounds like. It's almost like a person sitting next to you doing all of the legwork and then summarizing and giving it to you, but it's not doing anything. In the ERP system. For example, it's not going out, and example it's not going out and being actionable and performing things in the system. It's just helping you and guiding you.
Speaker 1:Got you. That's a great summary, because I think that's one of the big things that we see with Copilot is that you still have to do things. When you're implementing copilot, right, and you have your sort of your little buddy next to you, right, you have to tell it. Right, what's my sources? Right, what am I looking at? And then, in theory, potentially right, you have different options, right? Do you want to let just gen a I have at it and come up with responses and hopefully it does or do you want to go ahead and build some topics with some actions? And I mean you can really go down some rabbit holes pretty quickly, depending on how simplistic or complex you want to get with your AI implementation.
Speaker 2:Yeah, and also depends on the tools that you want to use.
Speaker 2:With the Power Platform Copilot that allows you to go out and create bots and create conversations and things like that Then again, those are guardrails that allow you to go out and create a flow that says this is how I want to move through these different steps and I may want to go and grab information based off these questions that are being asked. But the copilot platform is still sort of like those guardrails. When you start getting a little bit more freeform, when you want the AI really to do things for you or be a lot more flexible, so not just be able to answer things like how should I address this credit problem for this customer, maybe you want to say I've got this customer, what the heck am I going to do? And then have it go out and give you more information. Then that's where you move from more of the user-managed co-pilots up to something that maybe someone develops and has some code behind there up in Azure Again, different levels of AI that we're going out and managing.
Speaker 1:So, going from that because what I consider co-pilots so a lot of the co-pilots that we're seeing, that we see, for example, in D365, or we build sometimes for our customers like we're building inside Ludia to scrub through our SharePoint and policies and different information for employees I consider that more conversational. It's really just it's ingesting information and it's trying to summarize and spit back useful information based on what I'm asking it. And then there's the actionable. I think what you're saying is like hey, we could take that a step further and say I got this customer with a credit problem, what are my options or what can I do, and that's where you're really using AI to build some sort of hey, here's some potential solutions that you could do based on the situation at hand.
Speaker 2:Yeah, but with the flexibility. So, in other words, you're starting to trust the AI. Rather than saying I want you to answer this question, you're giving the AI a little bit more leeway and saying what want you to answer this question? You're giving the AI a little bit more leeway and saying what would you suggest? And then it's going out and it's using the language models and the general knowledge that it knows to try to imagine possibilities. And I use the word sort of imagine because that's really what it's doing.
Speaker 2:It's going out and sort of working out what, based off the information and its knowledge, what can it provide, what can it do?
Speaker 2:But that's a lot more flexible and also the results that you get may not be exactly what you want because, just I think of it, even though AI isn't a person, it's as if I went out and I'm just asking someone in general to help me.
Speaker 2:Then they're going to use their background knowledge to go out and create these responses and the AI has got this foundational background knowledge and then it will try to pull in this information, then try to give you some results back, and again, it may not be exactly what you want and I think that's the major hurdle that we've come across in our AI projects that people think that the AI can read our mind. One of the things that we found with our projects is that there's a lot of ambiguity in what we're giving the AI to try to work out what it should answer. It's that ambiguity that is causing the problems, because the AI discovers things or suggests things that we aren't necessarily really interested in. In the projects that we've done. That's something that we've learned that we need to watch out for, and we need to be really careful how we phrase things and what we give the AI in order to get the results back that we want, and I think that's a pitfall and a learning that everyone can pick up from as we're working through these AI projects.
Speaker 1:Yeah, no, that's a great recommendation and I have a perfect example, which is testing our internal co-pilot. I said, hey, what are our holidays? Okay, well, what holidays? Okay? Well, what holidays? Us, canada, 2024, 2023? What is co-pilot supposed to be looking at, right, and do we have data from 20, all these different years in different countries that have all the holidays? Because it's going to then just say, oh, I found it, it's right here, 2021 holidays for ludia, or blah, blah, blah, blah, blah, blah, and it may not be the same as 2024. And it may not be, you know. And so, to your point, right, it's.
Speaker 1:One is how do you phrase the question so that, or the phrase to co-pilot to be as specific as possible?
Speaker 1:Right, because the more specific we can be, you know, the chances are, of course, if it provides an answer, the answer is going to be more relevant, and I think that's where the subjectivity, you know, what I find is a lot, is.
Speaker 1:Of course, is that the success of AI for a project is very subjective. Having that understanding upfront, as you're building the statement of work or you're putting together the project success criteria and acceptance criteria, goals and objectives all of that, I think, needs to be very well understood across the board because of that subjectivity, because how you ask a question and I've seen it and says, okay, well, a AI didn't give me an acceptable answer. Well, the reason is, and if I look at what the question is, what the person's looking to pull back, I start looking at okay, well, let me look at how they phrased the question and find a lot of times it's nine times out of 10, it's possible that it needs to be rephrased appropriately so that AI doesn't get confused. And sometimes I'm also seeing where people are asking multiple questions in the same prompt and not even confusing it even more. It's like whoa, hold on, slow down, you're asking me three questions at once.
Speaker 2:You just confused me. That brings up a good point around scoping projects with AI. When you work with a project that's more of a traditional project like going out and creating a quote, going out and creating an order, doing manufacturing, anything like that it's very cut and dry. You put this stuff in and this is what we want. Maybe we want it in this format, maybe we want these comments to show up, and it's very easy to scope that.
Speaker 2:One thing with an AI project is you need to be a lot more prescriptive or a lot more descriptive as well.
Speaker 2:You can't just say I want to ask a question about orders and get a summary of the customer orders or something like that.
Speaker 2:Well, you can, but the problem is that the results that you get out won't match what you have in your mind, is your concept of what you want to answer.
Speaker 2:So when you have an AI project, a better thing to do is to say when I'm given this information, then this is what I expect to come back, and that also gives you test cases as well, or a benchmark that you can work from, because then you can say it's returning back this information. It may not be in the same format that we're receiving back, but it's returning back information and we're reaching these benchmarks. So when we have AI projects, then these test cases become more and more important, and before we even start working on something and pulling information out, then we need to have the project state, these inputs and outputs in the general sense, and not necessarily this is the sentence that it's going to give us, but this is the information that's going to give us and possibly these are the key things that we want to extract out, because then we can work with that. Otherwise, it's a never-ending process of refinement without really having an end point that you're working to.
Speaker 1:That is a great synopsis and one that I've recently had a customer here at Ludia that did a really great job identifying. Hey, here's our sources and here's up front, here's our sources and here's what we're really looking for from each source. Right, and some of it required revisions to their data just because, right, it wasn't in a format that the co-pilot could easily ingest. But you know, that was part of the process that we went through with them as we were building out either the generative AI or the topic-based items and saying, okay, you know what are we finding and really was able to, in theory, limit our number of iterations pretty significantly, our number of iterations pretty significantly, I mean. So we went from, I think you know, let's just say, like a 40% success rate up front and then, of course, we modified the moderation filters and content. You know, high somewhere, high somewhat medium, call it the three little bears.
Speaker 1:But, like I said, tell everyone internally and even customers that it is a little bit trial and error up front as you're building it, because we don't know. We have a good understanding of what we believe it will return, but we're not 100% sure and we're not 100% sure the second time we ask the question. It'll return the same thing again and again. That's part of the AI training piece, as AI learns what people are asking, what content it should be returning. That raises, I think, a good point that I get a lot. I'm not sure about you, murray, but you know how do we train AI to start returning things more consistently based on how we ask it. You know, asking it consistently the same question over and over again. I mean, what's your take on that?
Speaker 2:The consistency in the format and the returning of the information is harder than most people think, although I'll take that back. Sometimes it's repetitive and people don't like that, and sometimes it's creative and it gives the same response in different formats and some people don't like that either. So I'll quickly talk about both examples, because there was one example where we were creating notes and creating a co-pilot for taking some information and then returning back the results and we had a comment that it looks repetitive, it's returning back the same thing over and over again with different results, and it's because we'd put guardrails in the AI to say this is the format that we want to manage this. You can do that through the system prompts. So the guidelines that you give AI on how to respond to this, the repetitiveness in this case, was good, even though when someone ends up looking at like a thousand of these responses, then it looks like it's just pushing back the same information. That's good. It's because not everyone's going to see a thousand responses over and over. It's going to give you that consistency, the returning back of sort of varied responses, so that if someone's working with it over and over again, it doesn't just spit back the same things over and over, then that's another problem where you need to.
Speaker 2:You've got the knobs and dials that you can change on the frequency and the variation that you have in the responses, and that's part of the tuning what is a good set of variation and what is just completely off the wall, going from one extreme to another, where we can go and manage that. And when we get too much variation, then people don't complain. They say it's not consistent enough. So you've got this range of consistency that you may want or may not want within the AI, and the benefit of the AI is that it's flexible, that it's going to give you back different results based on just the seed or the point that it's starting at when it tries to work out. How do I want to rephrase this Now? If you always wanted a consistent response back based off a set of data, then you might as well just program it. Yeah, but that's a lot of work and it's not flexible.
Speaker 1:If you think about it, you could ask me a question. I can have someone down the street ask me the same question. The response that I give them and yourself could be accurate, but the information and how I provide it in the detail will probably vary. It's not going to be 100% the exact words, phrasing, structure, from the information that I provided you to the information that I provided to the other person. It just right. So I kind of give it that, of give it that little bit of a human element or human feel, so to speak, and so that's where I think it becomes.
Speaker 1:It seems like it's a little bit more of a natural conversation, because you do get those variations, and so that's something that I've come to expect and try to instill in my customers. Like, hey, more than likely you're not going to get the same response time and time again. The question becomes is the content that's being returned to you? Is it relevant, Is it accurate? That's what we're striving for, but the level of detail and some of the other components maybe Copilot having a bad day.
Speaker 2:It wants to keep it real short and it's one of the questions that I ask when we're going through and testing the responses is, like you said, is it accurate, is it correct and is it generally the right response? Not, do you like it? It's not. Do you like the response?
Speaker 1:That's the subjectivity piece, absolutely.
Speaker 2:And be careful not to have the subjectivity as part of the criteria for succeeding, because you'll never win, which is why you go back and you say these are the guidelines on how I wanted to respond and then we can work with that.
Speaker 2:We can put the guardrails up to change it. If you wanted to use AI but you wanted to have information returned back in the same format, then you can still use AI to go and go through the machinations of trying to work out what you're trying to get and getting the results back, and then you can get AI to return it back in more of an API format and then you can write code to then go and reformat it. So AI by itself isn't the answer. There's still development work that needs to be put in place to help you. As an example of that, if you're being fed all this information that you want to use and summarization and so forth, you may want to do some pre-processing of that data through code, through like Python or through whatever language you want to use, and then give it to the AI. So the cleansing sort of helps. It's like give the AI clean information and then it's going to give you better results back as opposed to if you just give it. You go blah, tell me something, then it's just going to-.
Speaker 1:It'll tell you something. Yeah, it just may not be what you want to hear. Yeah.
Speaker 2:Or it may get confused or it may get distracted. It's probably a better word that it goes down the wrong alley here and there when it's trying to return back things.
Speaker 1:Yeah, no, and you mentioned something earlier that I wanted to call out. It has to do with that upfront and early planning, right Understanding what, what the use cases are, and those test cases early on, because if they come at the end it's too late in this particular series. You can't treat it as a normal software development lifecycle project where you're in F&O, you're doing field service or sales and you're building out, you got your process flows and then you can build your test scripts based on that. As you're building everything, you almost need those upfront because that's going to actually help you understand and build the AI models so that you say, okay, now I know what Mr customer is looking for and what they're expecting. Let me go ahead and when I'm building out the AI, I understand, okay, this is my success criteria as I'm testing it, even unit testing it. Can I go ahead and hand this off to the customer saying yes, it's ready.
Speaker 2:Yeah, Sometimes you get excited. Can I go ahead and hand this off to the customer saying, yes, it's ready? Yeah, because sometimes you get excited, it's giving you a result and you think, oh, that's pretty darn cool. And then it's not the result that the customer wants when they're going through. So, yeah, these test cases are even more important and they give you the way of judging things.
Speaker 2:Now you may interpret that the results are meeting the test cases, but maybe they're not based off what the customer is thinking, and that's fair, because there's some subjectivity in there and some variation. But at least you've got a general target that you can aim for. Also, you've got something to discuss with the customer as to why it's working, as opposed to them saying it's not working and giving you the results back. And I've found that once we put a structure and once we have these cases that we can work through, then things go a lot smoother as to people, instead of just saying it's not working. Later on, you may find these boundary cases that aren't being taken into account. You can work on those, but the good thing is then you can have a change order. You can say it's doing what you asked for you want something else and we can do that. We can work on it, but it's not tied to the original statement of work that we're working on, so it's a different way of looking at the outputs that we're getting out of the system.
Speaker 1:Agreed.
Speaker 1:I think that's an important part is understanding that planning and having these discussions with customers early and often in terms of expectations, what we're seeing, how many iteration cycles that we need to do or should we do or are in scope.
Speaker 1:All of this upfront and then reiterating it, I think, helps alleviate some of the anxiety when it comes to AI, because at the end of the day and we've seen it it's evolving. It evolved in the middle of a project, it evolved in the middle of one of the projects that I was just doing, and so when that happens, it's like, okay, hold on. What's the impact? And is it a positive impact or a negative impact? Because in some cases, it's not the change you were really hoping for. It has a negative impact, and then you need to pivot. It doesn't always happen that way, but unfortunately it does happen, and so what I try to tell customers as well is that AI is constantly evolving. We're not anywhere near the end of seeing how this product evolves, and so one of the things is to realize that an AI project, even when it's live, isn't the end. It's really the beginning, isn't the end.
Speaker 2:It's really the beginning and the evolution, or the tools that you have, or the changes that happen throughout the project. That's a really important thing because the technology is changing relatively quickly. If we're working in a programming language, maybe we're working in C Sharp. C Sharp doesn't change that much. Maybe there's a new NET version that comes out once every two years or so. But with AI it's different. There's a couple of things, so maybe there's a new, there's a couple of things, so maybe there's a new NET version that comes out, and that comes out maybe once every two years or so, and that's really not a big deal. But with the AI models, then they're changing a lot. So, for example, we started one project and it was on GPT-3.0, and then there was a new version which was 4.0, which is more expensive. Then they came out with oh, there was version GPT-4, then there was 4.0, and soon there'll be 5. And then do we change the models Because the newer versions are a lot better? We actually got better conversational results back with the later models, but they were also more expensive. What do you do with those? Or there are new tools available. In one project we started looking at one set of Azure tools, but then Microsoft released out another set which were better, so we had to change those.
Speaker 2:The projects need to be flexible in the sense of what we use along the way, because the landscape is changing so quickly. Like you said, sometimes this is good, sometimes this is bad, but for the most part it's good. It just gives us better ways to do things and take advantage of it. You've got to be careful not to get stuck in the mindset that when we started the project, this is exactly how we're going to do it. These are the tools that we're going to use, because it's just changing so quickly. But then you've got to make sure that you don't change for the sake of changing and then get on a path that may be deprecated along the way. So there's things around that, but again, the tool sets are evolving so quickly that you need to be very, very flexible and have an open mind as to what you want to do along the way.
Speaker 1:That's a great summary. I definitely appreciate that and hopefully our listeners are appreciating that as well, so I guess we've got to wrap up here. I know we've had some good chat today. I guess, murray, for our listeners, if there's one or two pieces of advice for them around AI and going and exploring an AI project or implementing an AI project, what would you give them or what would you tell them?
Speaker 2:Don't be scared of the AI. Ai isn't right now. Ai isn't going to take your job, it's just getting. It's helping you. The co-pilots are exactly that. They're helpers that consolidate information, that allow you to get better information back to help you along the way.
Speaker 2:When you think about an AI project, then think about what your end goal is. The more aspirational you get, the harder it's going to get. Ai is not easy. It's not a silver bullet it's going to get. Ai is not easy. It's not a silver bullet that's going to solve every problem out of the box. You may have to do some architecting along the way.
Speaker 2:So don't expect AI to do everything. For you to learn to take over the world, to become Skynet it's not going to do that. For you to learn to take over the world, to become Skynet it's not going to do that right now. That's not what we want to implement with Dynamics or the other projects. We just want some help. It's more like training than programming. What you're doing is you're training the AI. You're not coding. You're not programming it to do a certain set of steps necessarily. So it's a different mindset for the project as to how you want to manage this and the expectations that you've got to get out. In a nutshell, it's different, it's good and it can be a lot of help to you, but it's going to be different from the traditional projects that you've embarked on with business systems up until this point.
Speaker 1:Yeah, that's some great advice for our listeners out there. And, mario, I know we have to wrap up, so I really appreciate your time, as always, and your insight. It's always very helpful and I've learned a lot. I have my large language model book and you'd be happy to know I'm about halfway through this book, so I'm going to do a shameless plug for this book. Even for people that are not technical, it's actually a really good book. I've learned quite a bit. Murray, thank you for recommending that book.
Speaker 2:It's a great foundation book and you probably link to it in the comments or something like that, but we're all learning along the way with these new technologies, so you've got to go out and do your homework.
Speaker 1:Yeah, We'll definitely put a link out there. It's a quick start guide to large language models by I'm going to mispronounce the name, I'm assuming Sanan Ozdemir. I'll just put it up there For those that are interested. Definitely a good book, but I'll put it in If. If you're watching this, then you'll see the book and if you're listening to the podcast, I will put it in the podcast notes for anyone that is interested. With that, we will wrap up for today, Murray, as usual. Thank you so much for your time, your insight, your wisdom and to our listeners. Have a great day. Thank you for listening to the Innovate with Ludia podcast. We hope you enjoyed this episode. Be sure to subscribe to our podcast on your favorite podcast app or follow us on LinkedIn. Until next time, I'm your host, Scott LaFontaine. Thank you.
