Aptage Artificial Intelligence in Project Management talk at DIPMF 2018
John Heintz, Aptage CEO, speaking about Artificial Intelligence in Project Management, at the Dubai International Project Management Forum 2018.
Below is a transcript of John’s talk.
…forecast project success. And we can help you manage toward that success. And I want to spend some time to share some of those ideas with you today. And talk about how we do that. And also pull on some other examples from different areas and some history of how we got to be where we are today.
Here’s what we’re talking about uncertainty, innovation. Today, uncertainty is very hard to deal with. And it’s getting more and more challenging. The reason why is because we’re innovating more, and we or our competition are disrupting us more. Last week was the simplest task you will ever have, again, for managing projects next week, and next month, and next quarter, we’re going to be doing bigger things more exciting work. And it’s going to be increasingly difficult. In fact, I believe that we have far exceeded the human capacity to deal with this kind of uncertainty and this kind of complexity. What do I mean by uncertainty? What I mean, the key questions that are always asked in projects, how long will it take? And how much will it cost? Well, we’ve always had answers for those questions. But they’ve always been partially useful. What I mean today, what’s now possible and what we’re going to talk about in this session, is being able to calculate what is the probability? What are the odds, we’re going to finish by the deadline? Is it 57%? Or 77%? What are the odds, we’re going to fall within our budget, or we’re going to exceed the budget? Those are the kinds of questions that we can ask now and dig into. And I wanted to think we’re at for his his, his comments, especially the augmented teams, we’re going to see that that’s going to dovetail very well into what I’m going to say. And I plan on stealing that phrase. Also, when we’re out was showing the slides about how fortune 1000 companies are failing with increasing regularity. Now, it’s because of the level of innovation and the level of disruption. One of the things that led me to co found afterwards with my business partner, Murray Cantor, is a story that he told me that 10 years ago, he was presenting at the PMI. And what he was presenting was that it is not that software projects fail more often than engineering or construction projects. It is that software projects tend to be highly innovative. We haven’t done them before. And therefore they have lots of risks and uncertainties. And what I would suggest to us here is it is the level innovation. And looking around the city, there’s tremendous innovation just here visibly, it’s the level of innovation that causes projects to be hard to manage. And that’s what we’re going to be talking about. But first, let’s talk about some project management failure patterns. How many people have seen these failure patterns? In your own experience? projects are green, green, green, green, and then red at the very end? Has anybody seen that? Yes, that’s the next one. shuffling resources back and forth, being able to move people around to try and fight the latest fire. Anybody? Yep, more hands go up the game of chicken waiting for another subcontractor or another team or department to fail. So that we don’t have to be the ones to declare How many have a friend that’s experienced this. Not Not you, of course. So these are some of the types of failure patterns that plague projects. Here’s the agenda for what we want to talk about today.
First, I want to go way back millennia ago, and talk about how humans have always created tools to compensate for our weaknesses and augment our strengths. We’re going to go through a few examples for that. Then I’m going to talk about AI and how AI fits into this system of what’s going on today. And what’s true. Now, I’m going to go back to the 1960s. Talk about a game show. And when I’m talking about that game show, I going to have to apologize, some of the topics that we’re going to be talking I’ve got a bad news, good news message around them. And part of what I’m going to be doing is trying to educate all of us about some of the things that some of our psychology makes us good or bad at some things. And some of our project management skills are good or bad at some things, and I want to talk about that. But the good news will be that we’ve got ways forward in the future. We’re going to talk about chess, and Garry Kasparov, we’re going to talk about maps and navigation systems. They are a really good metaphor for how to manage projects and steer to success. And we’re going to talk about how things like Google Maps deal with uncertainty and see some examples from that. Finally, we’re going to actually look at the next generation of project management. How do we forecast how do we steer projects, that’s the point of where we’re going to get to. Alright, industrial tools, these are tools that help us move better, lift more, everything from the lever to the wheel to the pulley on forward with these tools. While humans are not the strongest and most powerful animals on the planet, we can do fabulous things with these tools. Let me take a quick example. This is one of the fastest humans. And he can run about up to 45 kilometers per hour. That’s pretty good, I can’t run that fast. The fastest land animal, though, can run 120 kilometers an hour, quite a bit faster. If we take just a few tools, some wheels and pulley some gears put them together, this bicycle can go with a human on it 129 kilometers an hour. Now that there’s no engine involved in that. So this is just a simple example of how we have for millennia created tools that help us do more, go faster, and be stronger. Let’s look at another group of tools, communication tools, the printing press, the Telegraph, the radio, what used to take months or was impossible now takes milliseconds. And sometimes we complain about how slow it is that milliseconds, right, we have been able to compensate for our limitations and communication with these kinds of satellite networks that now make this possible to record and distribution.
Here’s another set of tools, thinking tools,
the abacus to the calculator to programming languages in the spreadsheet. These are all tools that we’re very familiar with. And many several of them have been around for very long time. These are tools that help us as humans think better. We’re not able to do everything a spreadsheet can do just in our head. And we’re not able to execute the logic that could be encoded in a programming language nearly as well as a computer can. So these are tools that help us think better that we’re very familiar with. So the point that I have with these tools that we have always created tools that help us do better compensate for our weaknesses and augment our strengths across these different areas.
This is something that we’ve been doing for very long time.
This is not new, this is part of the message that we’re going to have jobs in the future. They’re just going to end up being different. All right, let’s talk about AI and where AI fits into things. Who remembers in 1997 when this happened? Oh, wait, that didn’t actually happen. This is just a fictional movie, The Terminator in this movie, the plot was that in 1997 Skynet became self aware, took over the world and tried to eliminate humanity. It was the end of the world. This was the plot. What really happened in 1997 is a little bit more interesting. Garry Kasparov lost to deep blue, a chess AI, a chess algorithm. Now Newsweek branded this as the the brains last stand. This was not the end of the world, but it was the end of the brain humanities done for? Well, we’re going to come back to we’re going to come back to chest and talk about this again. But I want to jump forward to something else that happened. 10 years later, Google Maps released, forecasting arrival time, this was a big deal. This is being able to be on a mobile platform, use real time data to understand what traffic is like and when we’re likely to arrive at our destination.
Right. Now, this example we’re going to dig into more significantly as well. But first, I want to take us back to to a game show. And I want us to play this game. Now the reason I want us to play this game is I want to uncover some of the things that our human mind our intuitions are good at or not good at. And so this is one of those bad news, good news scenarios. This is a game called let’s make a deal. This is started in 1960s is a US game show. This is what it looked like. There are three doors. Monty Hall is the host of this game show. Has anybody heard of? Let’s make a deal? Yes, some people in the audience have. So this is actually known as the Monty Hall problem. And I’ll explain why. But let’s play the game first. So there’s three doors, there’s one prize. Usually it was a car or a vacation. And there were two not prizes like a toaster or a goat were frequent items on the game show. And of course, the contestant wants to win the car. Now if there’s three doors, there’s one car and two not prizes, the odds of picking the right door or one in three. Yes, yes. Okay, excellent. You’re correct. Alright, so let’s play this game. We’re the contestants, we pick door number one. Now, this is where it starts to get interesting. This is the game part of this. Monty knows where the car is. And he opens one of the other doors revealing one of the notch prizes. That’s an interesting twist. Alright, so Monte opens door number three. Now what’s going on? With Monte then does is give us a choice. Do you want to stick with door number one, or switch and choose door number two? All right, alright. I can count. There’s two doors. There’s one car. Now the odds are 5050. Is that right? actually know. The odds are not 5050. This is where it got really interesting. Our intuition steered us wrong. These are the actual odds, the first door is still one in three. The second door now gets all of the probability two out of three. This doesn’t make any sense. It really doesn’t. I’ve got a degree in electrical engineering. I’ve spent a couple of decades programming really interesting software systems. This doesn’t make intuitive sense to me. This isn’t just, you know, it’s not that we don’t know this. It’s it’s that our hunch steers us wrong in this case. Now, let’s talk a little bit about the psychology of that. This is a book called Thinking Fast and Slow. It’s a really good book, actually. But it talks about there two different ways that our brain can process things. One is our fast thinking. That’s our gut feel our hunches are fast thinking is the kind of thing that keeps us alive. If we’re walking along outside, and we see maybe that’s a stick, maybe that’s a snake, how do we not get bitten and injured, right? This is the kind of thing that enables us to drive and hit the brakes quickly without thinking about it. Our fast thinking brain is very important for many things. However, our fast thinking brain is not good at complex probabilities and math. It’s just not. That’s what our slow thinking brain is for. Here’s a takeaway quote from this book that even statisticians are not intuitively good statisticians. Until a statistician does the math and calculates the actual odds and probabilities, their gut feel isn’t correct, either. And that’s their job is to know these kinds of math ratios. So here’s where I’m going with this is that if only three doors can mess with our intuition? What do you think your complex project plans are going to do? all the dependencies, all of the disconnected teams and vendors and subcontractors all of the possibilities, they far outstripped our ability to process this in our head, there’s just no way. We need to build tools that can augment ourselves to handle this kind of situation and probability.
All right, let’s come back to chest. And this is where I want to talk about the augmented teams again, since 1997. When a human plays against a computer, a chess computer, it’s predictable. Now the chess computer is going to win. That’s kind of a foregone conclusion, in the last year.
The AlphaGo is another AI engine that beat the world’s greatest the world champion go player, which is a more complex game than chess. So this has been happening for a long time. Now, today, doctors, lawyers, other professionals are sometimes coming into conflict with different AI engines trying to trying to do what they do. So this conflict has been painted in the in the media in many ways. Garry Kasparov, and others after 1997, when he lost began creating things called Open chess tournaments. And an open tournament, let’s any team join. The team could be an AI, the team can be a human, the team could be a few people, humans, or an augmented team. And guess what happens most of the time, when an augmented team of humans plus AI, play chess against just a computer.
Right. That’s what happens, the augmented team wins more often. This is why it is man plus machine. This is why it is the augmented team together that is going to get results in the most successes. So this is what we want to be able to do more often than not where I started was with tools. And I mentioned thinking tools. When we build AI, right? It is a thinking tool that helps us think better. And that’s how that’s what we’re trying to create. And earlier today, I think that PM Auto was trying to create that as well. And so I want to to to repeat that same message that we’ve heard before. Okay, let’s talk about maps and navigation. This is route from Austin, Texas, to Dallas, Texas. I live in Austin, Texas. And the route, I’ve actually driven this quite a few times. So I kind of know this one. It’s a little bit difficult to see on the screen here. But this is forecasting, Google Maps is forecasting that it will take me two hours and 51 minutes to reach to drive from Austin to Dallas at this point in time. Now, how many people see this number on the on the navigation system and think that’s the time to beat? And I’m going to beat that time? I do. All right. Unfortunately, this is another bad news. Good news scenario. Most of the time, most people driving should be that time. We’re not actually as awesome drivers as we think we are. I’m sorry. But the good news is we get to use this as an example of explaining project uncertainty, and what it means and how we can leverage that to better manage our own projects. Ready. Alright, so here’s the route Austin to Dallas. This is on Interstate 35. And there is a forecast of two hours and 51 minutes that we’re going to arrive. Now. This is a forecast, it’s not that Google knows that’s the exact moment we’re going to arrive. There’s a bit of a probability distribution around this, there’s uncertainty. And this might be what that uncertainty looks like the triangle at the very top, that is the most likely point that that Google Maps might forecast that we would arrive. But it’s not this is not a normal distribution is scattered, it’s got a different shape to it. So it’s not quite a 5050 chance that we’re going to arrive before after there. But it’s like that. And Google doesn’t want to give us a lower medium probability of when we would arrive, because people would frequently arrive later than that, and be upset. So what Google does is it picks some level of confidence that is higher than 50, or 60%, let’s say 80%. Confidence. And that’s the number that it’s forecasting. And we can do this with our own projects as well, when we have the right math to pull these things together. And so what Google is doing is displaying two hours and 51 minutes is the arrival is the duration based on some level of confidence, that is light, more likely than the highest probability points that we will arrive. Right? This is about as deep as we want to go into the math for this this talk. But it’s important to understand this. Now, what happens when we start driving? And the number starts to shrink? Are we awesome drivers? Or are we reducing uncertainty, let me show you what’s happening. If everything goes normal, if everything just kind of proceeds as it would we start driving forward, the uncertainty because we’ve made some progress and some of the road is behind us. And we already know that it’s done, the uncertainty narrows a little bit. And the 80% comes in a little bit. But that triangle didn’t change. We can move again, uncertainty narrows again, let me just do these one more time. Here we go. And watch that it shrinks a little bit. And the orange line moves. This is simply making progress and reducing uncertainty, the 80% confidence number comes in. So it’s not necessarily you know, you guys might be awesome drivers. But apparently I’m not this is this is just happening for for normal things. What this does, though, is uncover how we can look at our projects. And if I go forward again a little bit further. And we add one more aspect to this to this drive. In this case, now we’re adding risks. I know, because I’ve lived through it and driven through it a few times that traffic in Austin, traffic in Dallas, and construction
in Waco are the three big problem areas for this this route. And what we can do then is we can use these to get even one more level of depth into what we’re doing. So these are roughly where these would occur along the road. If we move our car forward a little bit, perhaps we learned that traffic in Austin is worse than we had hoped. What that does is two things. And I’m going to replay that that animation again. But one important thing is it pushed out the likely times of arrival, it moved the whole thing, not just the shape the width of the curve. So if I do that, again, watch the watch, the distribution is going to move and expand. Both are true in this case, because this risk, it was realized traffic was worse, it was a little bit misty in Austin, and that caused some more problems. Now, if the traffic turns out to be way less than we thought, then the probability comes back and shrinks. Now we actually are planning, not just 80% confidence, but the likely all of the likely times are going to be shorter in this case. All right. Now one more thing. So project management often involves critical paths, uncertainties, risks, many things like that. Here’s one of the things that on this route, we can’t do. But we could do in our projects. What if we wanted to front load all of our risks? If we could do this, then, at the first day of the project, we still have the same uncertainties and risks, but we’re going to learn and uncover them right away. And if we can measure and shrink the uncertainty and the distribution, then we’ve made better choices to help us learn the hard things. What are the good news right away instead of waiting until the middle or the end of the project to find those things. So it’s not always Can this be done. But if it’s possible to do this, we want to so that we can learn all the hard and good things right away. And this is another aspect of managing and learning and doing things in terms of a project management approach, that driving from Austin to Dallas doesn’t, doesn’t give us that opportunity.
One more thing that I would comment on. And it’s difficult to see in this picture, but I’m sure that you all know this is Google Maps gives you different routes and scenarios, there are different choices to be made. Sometimes when I drive to San Antonio, Texas, I will choose a route that is a slightly longer time because I know there’s less chances of something really bad going going on in that route. Instead of taking the expressway, I take a secondary road, that is a more pretty drive, and avoids the very bad traffic accident risks that could happen on the expressway. So I choose to spend seven more minutes on the road by intention as an offset against that kind of risk. So being able to do scenario based management of scope and scheduling resources are things that we can do in real project management, and a little bit in the routes that we take in in terms of driving and moving on the road. Alright, so the takeaways from this example, are that we’re not only dealing with uncertainties and real time updates to things, but we’re actually predicting based on percentage of confidence. And what percentage of confidence we want in our project plan is a choice that we get to make. Is this a very innovative project that if it goes long, it’s not going to surprise anybody? Or is this a project that has a very fixed deadline, and we need to manage with a very high degree of confidence. Those are, those are choices.
Alright, so let’s now talk about the next generation of project management. How do we measure and steer projects to success? We want to measure uncertainty and risks in projects, just like we can do in a drive where the uncertainty of hitting traffic or not hitting traffic can happen or not. We can do that in our project plans. We want to be able to see problems months ahead of time, not weeks after they’ve already occurred. We don’t want the Green Green Green red problem to affect us. We want to be able to see things earlier. We want to tune plans based on our risk tolerance is 80% confidence good enough, or do we need 90% confidence, being able to calculate and figure in the right levels of slack that are responsible is difficult in projects when we don’t know how to measure uncertainty. And this provides the avenue for doing that. And we want to be able to act with confidence to get projects back on track if they’re if they’re off track. So let me show you what some of that looks like. So we can forecast when a project will finish. Now this will look a lot like the Google Maps distribution. And for the same reasons, this one has two different things. It’s got colors green, and red. The white line is where the project’s deadline is, this project has a seven month deadline. If we missed the deadline, we’re in the red, if we make it honor before the deadline, we’re in the green. And that’s what the coloring means in this case, being able to calculate the odds a 74% chance that we’re going to hit this deadline, given everything that we know today, that gives us an opportunity to be able to make high confidence decisions and act not with stress but with with real progress and, and, and manage towards this successful outcome. Now, that kind of analysis can work for very, very simple projects. And here’s one example actually created a back of the napkin plan this back of the napkin plan, make some assumptions around how long we’re going to be working on it and eight month project, it makes some list of the type of work that we’re working on. I will point out a few of the interesting characteristics for this when we’re working on a project. Sometimes we do know things with higher confidence. Sometimes we have less confidence, you can see that one of these items is flagged with a risky flag. That means it’s got more uncertainty more width in terms of the probability distributions. One item has a question mark, what does that even mean? How do we deal with that? Well, in the case of a default project, if we know nothing else, that question mark is filled in based on what are the other work items look like. And there’s a very broad distribution based on the other ones that we learn from. And, of course, rate based estimates, best case, worst case and expected case can be supported. We’ve got one range on here and 25 to 30. This back of the napkin plan? Well, the odds that we’re going to get it done in eight months, given what we know, are fairly low 16%. This looks like a six to 12 month project. But that’s a pretty wide range, although we’re just working with a napkin. So that’s not unreasonable with just that amount of information. That uncertainty perhaps is very natural in this case.
How about tracking progress.
If we can measure uncertainty and risk in a project, then we can track it over time, month over month, we can measure the plan and determine if we’re on track or not. One of the stories from one of our customers is that this year in 2018, our customer was running a one year project to create a new manufacturing line. And in April, one of their vendors was beginning to show significant signs of risk that they were putting the whole project at risk. Now what was very interesting in this case is that our system was detecting this about three months before that risk was going to occur. Here’s how that worked. And this is if you’re looking at the screen, when this problem spiked up, their risk spiked up to about 90% odds, they were not going to make the final project deadlines because of this vendor. What we were doing was learning from the vendors delivery early in the project when they were not yet on the critical path. But they were not starting or finishing their items on time. And this began to create enough evidence that our systems started acknowledging the risk that when they were on the critical path later, they would continue to not deliver on time. And that was very, that was the crunch time. Now our client was able to talk to the vendor with no results, talk to them again, and then replace them. And the new vendor came in began delivering what they said they would do and project with, with the risks went right back down. And they were on track. And today they’re on track for delivering the ability to see and learn from things and predict the future is really, really exciting and helps us avoid emergencies and late failures. Now this, of course works for complex systems. This isn’t integration with Microsoft Project for a construction system, the ability to measure based on existing project plans and existing progress actually versus baselines. That’s what, that’s what we’re about in terms of being able to augment a project manager to know what’s really going on in ways that are well beyond their own mental capacity to calculate the odds and the dependencies and the all of the all of the very complex uncertainties that exist in every real interesting project. Being able to see all of these things together is what provides that that critical value. Here’s a slightly detailed example, that goes down into the visuals that are exposed when looking at a case where for example, this one vendor was putting the team at risk. Here’s the task in in a Gantt chart, this task has some dependencies that feed into it. And there’s some risk that this task won’t start on time, there’s a little bit of red in that picture, mostly green, some red. In this case, we are able to learn that this team for is actually going to be adding risk to the future plan because they’re probably going to be adding a lot of delay in terms of when they finish. That’s why this finish of the task has so much read in it. Now how could we How can we learn the team for is adding that future real? I mentioned before that when a team is demonstrating that they do what they will say they are going to do and finishing things on time and providing that evidence of actions versus baselines? Well, then we can measure that. That’s some of the Basie and evidence. In this case, though, the team originally thought they were going to be delivering more and faster.
And it turns out in reality they’re not. That’s the gap, the white line is the original and the gap down to the colored high confidence green medium confidence yellow and low confidence red, that there’s simply not delivering as much as they thought they would be. And this is how we can measure and forecast to the future risks that they’re creating. So when a team is going slower, that’s an indication that we can start taking action before it becomes an emergency. We we can also apply these same types techniques to hybrid projects with multiple different work streams and vendors go to market with marketing, development, engineering, work, and so on. All of this can be wrapped up together, providing the risk overviews for different starts and finishes for various various different activities with different work processes. And when we can measure uncertainty of efforts, we can also apply those same techniques to uncertainty of return and apply that to generate probabilistic j and s curves and do portfolio management with uncertainty analysis built into it. We should have innovative projects in our portfolio, we should have routine projects in our portfolio. Being able to measure and provide the right balance of both of those gives us an ability to optimize our systems in ways that help us make better decisions. Here the takeaway points that I hope everybody remembers that our intuition is simply not good at complex uncertainty. We need augmentation systems to help us with us. AI is a thinking tool that can help us think better. It is humans and machines. augmented teams is the right way to think about this.
And today, we now have the ability to create next generation tools that help us manage projects with all the benefits that I can bring. Thank you very much.