Build Your AI Team With These Roles (Guest: Keith McCormick)

Andreas Welsch: 0:21

Today we'll talk about how you can build your AI team. Who better to talk to about it than someone who's been helping teams do just that? Hey Keith, thank you so much for joining.

Keith McCormick: 0:31

Thank you for the invite. I enjoy your show.

Andreas Welsch: 0:35

Thank you so much. Why don't you tell our audience a little bit about yourself, who you are and what you do?

Keith McCormick: 0:40

I've been doing this kind of machine learning thing for a long time since the nineties. I actually started out as a SPSS statistics trainer, believe it or not, many years ago, but then moved more towards machine learning and consulting. And now I do some teaching on LinkedIn Learning. I speak at conferences and I'm also a consultant at a small AI consultancy called Pandata.

Andreas Welsch: 0:59

Awesome. I know we met at the AI Summit in New York almost a year ago and we were part of the LinkedIn Creator Accelerator program. So it's awesome having you on the show. Thank you so much for being on. And for those of you in the audience, if you're just joining the stream, drop a comment in the chat where you're joining us from. I'm always curious to see how global our audience is. I can't believe from which parts of the world I see you join. Keith, what do you say? Should we play a little game to kick things off?

Keith McCormick: 1:26

I've seen a couple of your shows, so I knew something like this was coming, but I also know it will be random. So go for it.

Andreas Welsch: 1:34

Yeah, exactly.

Keith McCormick: 1:34

Let's see where my brain takes us.

Andreas Welsch: 1:36

The game we usually play is called In Your Own Words. And when I hit the buzzer, our wheels will start spinning, and when they stop, you'll see a sentence. I'd like you to answer with the first thing that comes to mind and why? In your own words.

Keith McCormick: 1:49

Sure.

Andreas Welsch: 1:50

And as always, to make it a little more interesting, you have only 60 seconds for your answer. I'll take a look at the clock here. And for those of you watching, if you would like to answer as well and why, please put it in the chat, too. Keith, are you ready for What's the BUZZ?

Keith McCormick: 2:03

Sure.

Andreas Welsch: 2:04

Excellent. If AI were a song, what would it be? 60 seconds on the clock. Go.

Keith McCormick: 2:12

The song that comes to mind is, I don't even know if everybody's heard it, Rush by Troye Sivan. Which is probably the last song anybody would think of, but I think it's because we're all a little over caffeinated on this stuff. It's a very high beats per minute song, if people haven't heard it. I think we have to slow, we have to slow our roll a little bit. That's my random thought.

Andreas Welsch: 2:34

Thank you. And look, there, there is a lot of rush going on and a lot of hype as we're riding the hype cycle like a roller coaster. It's one of my favorite analogies. So awesome. Thank you. And thanks for thinking on the spot. Now, with that rush right there, there's obviously a lot of focus, a lot of rush towards building something, getting something out there, whether it's in the hands of your own teams, or if you're in more of an IT environment, incorporate AI into the applications that you build for your own teams. Or if you're more on a product team where you sell SaaS products and so on, how do we get it there? And over the last couple of episodes, I've talked to guests about methods to do that. And we've covered RAG, Retrieval Augmented Generation, for example. We've talked about open source. We've talked about fine tuning and prompting. Maybe the question to you too is right with everything that's going on, some things that are new for AI teams and for data science teams, how far can teams even get with just prompting or with augmenting prompts from what you see?

Keith McCormick: 3:37

There's so much talk about this. I don't think pure prompt engineering is really for us as AI folks and AI consultants. I think where people are going to be doing that is even things like journalism and so on, and the folks that are doing things like marketing research. I think they're really the ones that are going to be focused on just pure, straight up prompt engineering. I was asking my colleagues about what they thought about similar things, and they recommended a talk quite recently, which I thought was really, great. And what it was talking about was generative AI isn't particularly good at consistency of response sometimes, and you have to control that. My quick answer to the question is I really think, certainly if I was thinking about somebody to assign to a project or to collaborate with, I want somebody that really understands natural language processing more generally not just prompt engineering. So this particular talk that I was referring to was doing a lot of parsing and post processing on the back end. And I see that because I've been talking to some folks. For instance, there's a legal project that we've been working on. You might be pulling things from forms, or you might have to populate forms. And that's not going to be only prompt engineering. There's all kinds of other stuff going on. If I was working with a colleague, and they were tackling it only from the prompt engineering side, I would think that was way too limited. Not only that we want to deploy these solutions. We've got all kinds of pipeline and API issues going on, and that goes beyond prompt engineering too. I know there was excitement about it in the spring, but I don't think we're going to be talking about prompt engineer as the career of the decade much longer. I think that's gonna fizzle out. It's a broader, bigger problem.

Andreas Welsch: 5:26

I mean, very concretely, and the way that I've been using generative AI and probably many of you have been using it, too. That way is even if you say generate a summary of this or that text for or you know with about 500 or 800 words, you can tell by the context window and how much you put in that, sometimes all you get is four or five lines of text as output. I told you I want 500 words. What's so hard about this?

Keith McCormick: 5:55

Imagine taking one step further, right? Let's say you had to populate some SEC filing or something with information. There's just no way that all, in and of itself, is going to be able to pull from... It's a big difference between structured data and unstructured data. The power of it is its ability to do some pretty remarkable things with unstructured data. But there are some use cases that are about structured data. And I don't just mean tabular data. More like this legal form type thing. It's a whole different area. So you need more in your bag of tricks.

Andreas Welsch: 6:30

I'm taking a look at the chat here and I see folks joining from Charlotte, North Carolina, from Germany, from Chicago, from California, from Costa Rica. Super exciting. Thank you for being with us today. Now, Keith we've, just talked about prompt engineering and that it will evolve, it will most likely not be the sexiest job of the next century unlike data scientists, right? What skills do you need on your AI team beyond prompting? I think there's certainly a need to do some basic prompting if you're building applications, but what else do you need on your AI team?

Keith McCormick: 7:07

Folks that know me well... And we've got to know each other pretty well, Andreas. So forgive me if I'm revisiting themes that I've become somewhat famous for, but you have to invest a lot of time and patience with external consultants. So I often think in terms of clients. But you could be in a corporation and it's the internal client, you've really got to sit down and figure out what they're trying to do. Clients usually come to you because they have some kind of a problem, right? They're trying to get better resolution or better recognition, I guess you would say, in their computer vision model or something like that. But when folks approach an expert about getting help on large language models, they usually start with that. They usually don't start with a problem or a question. They say, I want to do something with large language models, right? Which is really a problem. So I think the most important skill to have at the team and a brand new data scientist typically won't have it is this dialogue with a client or internal customer about what exactly they're trying to do. And somehow you've got to marry that with a business problem that has money attached to it. And it's not to be overly obsessed with the money aspect, but that team has to pay for itself. And again, that's if it's internal as well. And there are little things that turn out to be not so little things if you ignore them. Like trying to get projects that are going to start and end in the same fiscal year and not let them just drag on. So that ability to carefully interact with somebody that needs something done. And even learn to push back on them a little bit. Are you sure that's what you want, right? Or how is this going to derive value? You would think that the person who needs something done is always going to have the skills themselves to know what to ask for, but in my experience, they don't. So somebody on the team has to do that. And I find usually, a new data scientist has to shadow somebody in doing that problem definition for two or three projects. And then they really get the hang of it. But you just can't throw a new data scientist into that without them observing it and really interacting and like client interaction and the whole nine yards. I know that's not a technical skill, but it's so critical to doing this. Because if you don't have good problem definition, you end up doing the wrong project basically.

Andreas Welsch: 9:38

I couldn't agree more. I've seen that in previous roles myself, where there's a strong focus on technology, whether that is from your internal or external stakeholders, we need to do something to your point of view. There's a renaissance of that going on right now with large language models. And then coming in with a technical lens of, hey, here's a model or here's an approach that we can take, without having done your due diligence and asking what is it that we're actually looking to optimize for? And what's the business problem? What's the business metric? What's the value that we can realistically create? And to your point, how do we measure it? How do we know that once we've built and delivered this, it does actually solve the problem that our stakeholders were trying to solve to begin with?

Keith McCormick: 10:22

You just mentioned business metric, and of course I agree, but that ties into large language models, too. Just simply prompt and response, whether it's chatbot or document search or whatever, that in and of itself isn't going to have any business metric, tied to it. Actually in the episode with Tobias, I thought he had a great example of that, because there are some folks that might just say, Ooh, let's upgrade our chatbot with generative AI. But he had a great example about how they were spending so much to maintain the back end. It was actually a cost savings to do that. So now you've got your business metric, but you can't assume the business metric is there. You've got to discuss it from the beginning because if you don't, you can't measure success at the end.

Andreas Welsch: 11:05

Exactly right. And then it gets so much harder to build that momentum and to justify why you're making these investments to begin with. I'm taking a look at the chat here and I see a question from Franziska, which I think is a great one that we should touch on. And her question is, can AI with the right prompt engineering replace the need for creative people or creative problem solving to some extent in corporations? So could prompt engineering be the new skill that replaces creativity?

Keith McCormick: 11:32

I had an interesting experience recently. I went to this event in London called Thinkers 50, and it was all business authors, elite business authors. The Harvard Business Review crowd, really interesting. And someone that has become a friend over the last couple of years, Dori Clark, was giving a talk on exactly this. And she was saying, if your writing only goes as far as what ChatGPT is going to spit out, then you're really not establishing much of a personal brand. So she took it partly as a challenge, but she did mention some examples that I just thought were brilliant. So she's approaching this as a business author almost from a journalistic point of view, but she has a number of bestselling books. But she mentioned a prop that she uses that I just think is brilliant. She said she takes her name because she's written hundreds of articles. And then another expert. She's sometimes asked ChatGPT, what topics do we have in common? And think about how helpful that would be if you were going to be on someone's podcast or co-authoring an article with them and so on, finding those points of intersection. So I think it's going to be an amazing aid to researchers, but researchers are going to have to bring their best game. Because if they're just doing mediocre work, then they're going to be in trouble. But if they're truly innovating, I think it's going to be an aid to them. That's my two cents, but I'm not the author that that she is, but I thought that was a compelling talk that I heard last week.

Andreas Welsch: 13:05

Especially now with things like ChatGPT being able to pull in data from the web, you don't necessarily have to worry as much about cutoff dates in knowledge and model training. If this thing can go off and find information by itself, or if you give it a few sources that you wanted to search and summarize and aggregate. So great point there. Yeah. Now obviously Beyond prompting, beyond the business skills and being able to facilitate that dialogue and distilling what is it really that we want to optimize for with AI, be it generative or be it the traditional machine learning type things. There are also other roles on the team, right? You need data engineers, you need to have data scientists if you want to build things yourself. Why do we still need those in your point of view when it's all about large language models, prompting a bit of augmentation, being able to reach out to the web? Why do we still need data engineers and data scientists?

Keith McCormick: 14:04

Before we went live, I was mentioning that the YouTube algorithm sent me down a particular rabbit hole based on the fact that I was revisiting some of your content. And there was someone claiming that you could be a data scientist in 10 minutes, right? Because you could basically query, you could prompt these different questions. The reason that what I call traditional machine learning, and some folks might think I'm hopelessly old school for these kinds of skills, but we're talking, decision tree, ensemble, random forest type stuff. This stuff hasn't suddenly gone away this year. The reason that you still need skills like that, and sure, they're not incredibly hard to learn, but it takes a while sometimes to learn, doing them in a setting and getting value out of them and bringing it all the way to deployment, is we're still going to be doing supervised machine learning for so many things. We still need propensity scores. We still need to detect insurance fraud and potential loan default and predictive maintenance. I've been working on and off on a predictive maintenance project with Internet of Things sensors and all this kind of stuff. Basically seeing if everything is working smoothly on an assembly line. That's clearly not chat engineering. Those things don't go away. All of those established industries that have structured tabular data, that hasn't suddenly stopped being important. I would say on a team, you're absolutely going to need somebody that has some knowledge of large language models. But as I said before, that's going to be natural language processing more broadly. We're starting to see lots of computer vision. It's not surprising. There's been so many advances in that and there's so many moving parts that I think that kind of becomes a specialty. So already you've got potentially two specialties. It doesn't mean that those folks aren't cross trained, but in fact, they almost certainly are cross trained. But those are some specialties. And again, I think you need some folks that can really crank through some very effective, traditional, supervised machine learning. And then, as I've already said, you need somebody that can architect the whole project. Now, hopefully they've got a couple of those four that I just listed, or more. So you need some overlap and you need some cross training, but as a team you need all of that represented. And then of course there's unsupervised and so on as well. In fact, the natural language processing and things like old school unsupervised aren't necessarily separate. Back when I would do bag of words, text analytics, you didn't just simply do the text analytics. You then fed it into predictive models or you did cluster analysis on it. What trends are we seeing in the concept extraction? That skill doesn't go away either. So the natural language processing person has to know how to combine something that's quite new, like word embeddings, with these older techniques like cluster and factor. All the usual suspects.

Andreas Welsch: 17:08

If I take what you said, YouTube sent me down a rabbit hole, you can become a data scientist in 10 minutes. And Svetlana's comment here, problem solving abilities and implementation experience cannot be credited in 10 minutes. I couldn't agree more, right? And I have a feeling 10 minutes is probably what it takes you to update your LinkedIn profile that you're now a data scientist. I think there's a lot of good information that you shared there. I feel because of that hype of large language models we often forget, or the stakeholders we work with, often forget that there are other kinds of. Predictive analytics, machine learning, supervised, unsupervised learning type methods that still have their place. ChatGPT or GPT is not going to do your demand forecast. Please don't use it to do that. But if you're creating a report that should include that information, great, right? It can create all the words and the summary of it, but you still need your traditional forecasting methods to get to a reliable forecast there.

Keith McCormick: 18:05

That always triggers a thought in me when people talk about forecasting. Because, of course, I agree if we spend enough minutes the number of specialties would grow, right? These skill sets haven't gone away. So I agree with you about forecasting. One thing about forecasting though, is I'll sometimes be approached either at a conference or in a client situation and say, oh, can we do something with AI to improve our forecasts? And the first thing I always ask them is what about the existing forecasts and the existing team that's doing forecasting? Because I get really anxious about this idea that the data science team is going to be competing, let's say with the finance team or something like that without them talking about it. That's just going to blow up. And I've always thought that the number one reason that models don't get deployed is organizational resistance. And that is the beginning of organizational resistance right there. Putting two departments against each other without communication.

Andreas Welsch: 19:00

I think you need to work on that relationship and understand how are we teaming up, or how can we help you, if you're in a business function reach your goal? Which again, brings us back to the ability to talk somewhat on eye-level with business teams about what they're trying to solve. There was another question that I thought was really interesting. And it was, when can the discussion move from increasing productivity to increasing creativity? Kind of goes back to the earlier question from Franzisca, but what are you seeing there? A lot of the discussion is really about Hey, we have a co-pilot here that helps you draft an email or create a presentation for you or maybe do some creative work, if you will, whether it's Adobe Firefly or some other image generators. But when do you think that's going to evolve to creativity and maybe how does it relate to AI and data science teams?

Keith McCormick: 19:49

Again, if the goal is a deployed model that you're going to build and then work with an internal or external client to to get value out of this and maintain it, then that kind of process maybe helps you find some research on a particular subject or something like that. But I think where that activity is going to be, and I think it's really important. I was just thinking about this. If you were a huge newspaper, let's say like the New York Times. That's where, not so much on an AI consulting team, but that's where maybe I would want somebody, and very possibly without a computer science background. There's a YouTube video by Bloomberg, and there was somebody that had a history background that was doing a demonstration of prompt engineering. I'd want somebody that was really good at that to be a resource to help the marketing research team, the journalists and so on. So I think there's a place for that, but it's a bit different than the skill that an AI team needs to have to produce deployed models. Harpreet and of course the episode that I'm talking about when he was talking about the technical skills of tying all of this together, when he was talking about RAG, I thought was really quite thoughtful. But it's rather different than what a journalist or a marketing researcher would be doing.

Andreas Welsch: 21:06

So building on that and looking at 2024. Obviously, a lot of teams and leaders have been starting AI projects, generative AI projects, already this year. There's hardly a way around this, and has been hardly a way around this. But looking at next year, what are some of the skills or existing skill sets on the team that are easier to upskill on generative AI, on these new technologies and techniques, I should say, that you're seeing?

Keith McCormick: 21:37

I don't have a detailed answer on this, but I can tell you that I've been intrigued about this since the spring. I think that in a lot of settings, like healthcare for instance, they're gonna have to build and fine-tune their own. But things like Llama 2 and there's going to be many more that are joining folks that are taking smaller, large language models and then building them and deploying them internally. Whether it's all on prem or not, doesn't matter with very sensitive information, I think that's something that I'm still surprised that people don't talk about it more. It's not that there isn't talk about it. It just hasn't dominated the discussion yet. And I really think that's going to be. Not just a skill set, but really like a permanent role in a lot of organizations is maintaining their own internal large language model.

Andreas Welsch: 22:22

Thank you for sharing that. Look, we're coming up to the end of the show, and I was wondering if you can summarize the three key takeaways for our audience today before we wrap up.

Keith McCormick: 22:32

I'm really big on problem definition on the front end. Again, that's a super important topic to me. One of my colleagues at Pandata closes his talks with let's make AI boring together. In other words, make sure that it's practical. Make sure that you're solving a real problem. And then I think another takeaway would be don't think in terms of prompt engineering. Think in terms of natural language processing more broadly. including pipeline type issues and actually deploying a solution that's going to live on just the way that we've done other things. And then finally, I'd say don't, dismiss old school supervised and unsupervised machine learning. It hasn't suddenly gone away.

Andreas Welsch: 23:14

Thank you so much, Keith. Also, thank you so much for joining us and for sharing your expertise with us and for those in the audience for learning with us today.

Keith McCormick: 23:22

I appreciate it. I had a lot of fun.