Andreas Welsch: 0:30

Episode is going to be a little different on short notice. Hey Kiumarse, thank you so much for joining. Maybe if you can talk to our audience a little bit about yourself, who you are and what you do as we now pivot more towards the topic, how can you pick the best AI agent framework for your business? Where are you? What do you do?

Kiumarse Zamanian: 0:50

Hey, thanks very much Andreas. And really I've enjoyed your podcast and great book and also the LinkedIn classes you have. Congratulations on all those. So I'm actually in the San Francisco Bay area. I've been in Silicon Valley for about 30 years. I did my PhD at Carnegie Mellon University, focusing on data management and some AI, and then moved to Silicon Valley and worked in about eight companies so far, focused on analytics and interopability. And for example, I worked for Autodesk, Informatica, Yahoo, Responses, which got acquired by Oracle. And then after Oracle, I joined Walmart Connect. And most recently I've been working in the Gen AI area since kind of early 2023 in the agent areas and exploring ways that we can leverage these agents to automate and make people more productive and also making sure that these things are doing the right thing. Yeah, I'm really excited about talking about a lot of those concepts with you. And I've been evaluating a whole bunch of these AI agent frameworks lately. And I just wrote a little paper, I'll share that after the call.

Andreas Welsch: 2:08

Yeah, absolutely. And I think that gets us right to the meat of the topic, right? There are so many different frameworks already out there, some by large commercial vendors, some by communities, by startups. How do you even know which one you should pick? What have you found and what would you recommend? Where should people start?

Kiumarse Zamanian: 2:30

I think it's really important to understand first of all, what are these agents and what are they supposed to do and what kind of a technology stack is really required to support these AI agents? A little bit of quick background. Agents have been when I was at school, graduate school, we had these agents that were based on rules that were using heuristics and there were some workflows that were manual in terms of how these agents are supposed to communicate with one another. What differentiates this new generation of agents is that they're really leveraging these language models to be more autonomous and to be able to, in fact, plan, learn, correct themselves, and also be able to really orchestrate how to make the decisions. Even though the human is still in the loop, I think what we call human in the loop now, I think the goal is to try to have the human on the loop. So to have these agents actually collaborate with one another relatively seamlessly and independently. So think of it as as you see like a project manager trying to get something, want to set up a meeting for you to brief your clients about the latest features in a particular product. Then you basically task your team and say, This event will happen. This is what we're going to discuss. You may not even tell them what we're going to discuss. You're just going to give them some kind of high level objective. And then this team will basically go figure out among themselves how to organize that meeting, what kind of content they're going to present. That's like putting the slides together. Maybe there's going to be some sort of a venue that they need to make sure they have reservations, to have food coming, all kinds of things that need to happen, right? Individually, we were able to do research using ChatGPT or Gemini to do these tasks individually. But the human was really the main interface with these large language models. You basically, what they call a zero prompt. You basically issue a prompt, you get a response back, and then you issue another prompt. And most of the time these things may contain the context, etc. But now you're actually delegating these tasks to agents. That among themselves need to decide how to break particular objective into multiple tasks and find out if there's an agent that is actually specialized in a particular task. If so, delegated to that particular agent, or maybe there's some fun function that you need to call to basically get that information. A lot of times you may need real time information about flight information or hotel reservations or food, et cetera, that large language model will not be able to help these functions and tools also built in. The way to look at it is the stack that is required. To build these agent orchestration really is composed of really six layers. At the bottom, you have the foundation models that we're all familiar with. The ones from OpenAI and Microsoft, AWS etc, So these are basically a lot of big companies are spending millions of dollars to train these large language models, right? Along with those, you may even have a smaller specialized for a specific area, right? So this is your bottom layer. That's like where you're, a lot of your learning has happened and inference will happen in that, at that level. Then you have got you got to build memory and tools that will support your agents. Just having a chatbot sitting on top of an LLM it might be sufficient for just basically getting some simple answers, but if you want to have context and be able to maintain what type of information has been passed back and forth between these different agents. You need to have memory management. You need to be able to maintain state. So this becomes very similar to how we develop programs. And when you write software, there's all kinds of state management, et cetera, involved. So that layer sits on top of the foundation walls. And then on top of that, you basically have these agent frameworks that are able to utilize. The memory and state and the large language model, et cetera. So these frameworks basically sit on top and are able to help you to create agents and have agents communicate with one. We've seen many examples that half last year or so in this area, particularly from Microsoft AutoGen and LangChain, we're all familiar with. And then after that, LangGraph and LangFlow. So these, are basically, frameworks that help you build agents and have them talk with one another. But then you, once you actually build these agents, you didn't actually host these agents. So you basically expose them to people who are going to use them either through APIs or through UIs or some way that basically they become services that are available, right? So this becomes service for hosting these agents, et cetera. Then once you actually have these hosted, you will have some domain specific agents, right? So you might actually build an agent that is very much specialized in dealing with marketing campaign management. Or you have another one that is very specific to maybe some specific retail or travel or health, or you name it, you will have these very domain specific agents. Now you have this kind of a whole host of agents, some of them might be platform specific, some of them might be working only within a Microsoft AutogGen environment, some of them in Google, etc. Then you need to make sure these agents are collaborating with one another. So again, the analogy with having people. They have special skills and they have this special expertise to do things. These people need to collaborate with one another just as we are collaborating now. These agents need to be able to communicate, pass information back and forth, be able to utilize standards. So that's where you get to these multi agent orchestration frameworks that are just emerging that will really help you grab agents and have them communicate so that they can not only be able to use large language models and small language models to basically find information for you, but also be able to call functions, delegate tasks between each other and make decisions and learn what they have self correct and be able to do planning, et cetera. There are a lot of activity in this area of the multi agent orchestration frameworks that are just happening at a recent. The particularly with big play just announced, AWS just announced theirs. Crew AI has certainly moved into this area. And some of the other players that are core focused on the domain specific like cognizant a neural AI platform, they have very specific areas. So I think it's a long answer to actually evaluate how we're going to use these orchestration frameworks and be able to find out which ones really you have to identify the problem that you want to solve, just like anything else that you want to find the solution. Is this going to be very domain specific? Is this going to be very specific to it? Do you need to build something much more enterprise level that goes across organizations, et cetera? So I think from that perspective certainly having this kind of an openness and the different model support, you want to have something at least at the enterprise level that is can scale. You can actually have openness that you can choose different components that are appropriate for meeting your needs. Second thing you want to make sure that you've got the right mechanisms to have control and governance, right? This is extremely important because you don't want these agents to just be a hundred percent on their own or be reliant on humans you want to be able to delegate the automation to these agents and put the right guardrails and governance rules in place so that you can automatically, or some agents actually are specifically tasked with monitoring what other agents are doing, what information they're passing to one another, etc. Governance is extremely important, particularly in some areas that may deal with health care privacy, as we all know, is extremely important being retrieved. Maybe one group may have access to certain information, etc. So that all needs to be built in. And it's extremely important to have a framework that does give you the flexibility to define these rules and governance rules. And sometimes some of these like the as I mentioned, some of the domain specific frameworks, like the one from Cognizant, actually comes in with a lot of the controls built in and the governance built in, because they're very much focused on very specific domains, but if you're dealing with something like it, LangChain or Crew AI or AutoGen, et cetera they do have some governance capabilities, but that's an area that's extremely important and it's evolving. Then control certainly goes hand in hand with governance rules. Which agent is in control of what, and how do you actually bring a human into the picture? I was reading some news that apparently there were some agents that were developed to go do some, I think, reservation or do some research or something, and they were self checking to see if they actually found the right information or not. And they were continuously failing the check and the they were just being creative. Checking with the large language model to see what the next thing that they should, they were scraping websites, finding email addresses and start sending emails to people, which was really crazy, right? So these agents were just going rogue and doing all kinds of crazy things, right? So you need to be very careful about that in terms of control. Let me just pause there. I've got a few more items to cover.

Andreas Welsch: 11:51

Yeah. Let me jump in real quick because I think there's a you said in, in many ways, these agents we can think of them as human colleagues or as an equivalent in in the sense that helps us think about how we can interact with them. And then my question is we all sign a code of conduct when we join a company or join a large company. There's some kind of an onboarding, there are some ethical standards, some expectations, there's some professional standards if you think of finance and IFRS and others. How do agents first of all acquire that knowledge to know that, hey, these are the parameters of my role, I need to know what IFRS and only act within, and then to also act ethically in alignment with, first of all, but also general principles of how we do business, so they optimize for the right things, right? Additional revenue, and I can increase the margin, but I might do that as an agent at the expense of my customer satisfaction. Exactly. Yes. What have you seen to make sure that these agents actually act ethically?

Kiumarse Zamanian: 13:03

Yeah, that's an excellent point, Andreas. I think ethics is certainly something that could be formalized in certain rules, but I think it's sometimes depending on different, maybe within a company, you have certain very specific rules certain cultures I know at Walmart, there was a tone of voice when the chatbots talk with the customers, it had to actually comply with the tone and like the way that the Walmart All right. Customers need to be treated. I'm sure, in health care, you have similar things. People's ethnicity and demographics and various different things that ethics comes into play. As I said, if these things could actually be put into some rules that could be checked by certain agents as one way sometimes you actually have some of these models if there's a model. That is fine tuned, or you have a small language, you can in fact build in some of those basically the conversations that you've had. For example, let's say there's a law firm that is developing some kind of a chatbot that will maybe help the associates or maybe the clients with certain information. And they want to make sure that their information they're providing is not biased or some HR agency, right? If you actually take this model and then train it on very specific dialogues or type of documents that have been used that are safe. So you basically has been trained to actually know how to interact using the right ethics, et cetera. But you want to be very careful that you want to have some guardrails, very specific guardrails, that when a response actually is generated an agent should check against certain rules, right? Does it meet certain criteria, et cetera? And as long as those rules are codified, then these agents certainly can now do that. So I think that's where you really want to make sure that you've got a framework that allows you to build in those kind of Governance rules and also ability to maybe to call out to a certain APIs and things. Maybe you have services that you have developed already. There are lots of companies that are actually focusing on just governance and ethics and things that there might be services are available that these tools can call to check, right? So there, these are case by case and you got to make sure that your framework supports these kinds of different scenarios and it's flexible enough.

Andreas Welsch: 15:23

That's awesome. And I would love to come back to another point that you made earlier that eventually we will have different teams of these agents and virtual teams and maybe to some extent, even virtual organizations, from starting with individual tasks to departmental or teams, departmental, interdepartmental, marketing talks to finance, what is my budget? Can I get a little more for this campaign? Here's why I think we can achieve better ROI or drive more customer demand to even different companies having a sales agent and procurement agent that figure out, do you have the product? Can you give me a discount? When can I get it? These kinds of things, even if that is still a little further out. Those last two phases of interdepartmental intercompany. What are you seeing in terms of orchestration, in terms of getting these different agents to talk to each other? Because I think also the real the reality in large organizations is that there might be specialized agents by one vendor for more customer experience topics that might be agents specializing on your finance agents on, your HR products, wherever these vendors build in these capabilities, plus, like you mentioned, the Crew AI, AutoGen, other frameworks that are built in on top. Are there any kinds of standards? Is there an ecosystem that you see that either exists or that needs to exist for, that to happen?

Kiumarse Zamanian: 16:48

Yeah, that's an excellent question. And I think that area is just evolving. I come from the enterprise integration. I come from the times that the object model from Microsoft and there was CORBA from object management group. And a lot of people were struggling building object models that were standardized and came and dominated. And with CALM that gives you actually a fairly flexible way of. Defining these objects and being able to integrate. And also at Oracle, it was a big deal to have these, all these different applications to be integrated with. Now they're facing the same challenges there. And certainly with the mobile industry, a lot of apps being developed and how would these apps actually live in an ecosystem that they certainly have standards based on if they're using Android or if they're using iOS, et cetera. So there's some interruption. I think there's a standard from OpenAI is a JSON schema standard in terms of how you're able to pass messages to a large language models, and certainly some of the communications are also getting to some extent standardized, right? Just like anything else, no HTTP protocol all these other protocols that emerge for internet. I think we're going to see some interoperability protocols emerging for agents to communicate with one another. At the end of the day these agents are basically intelligent abstractions of software abstractions that are able to do certain things certainly they're able to communicate with the memory and things that are within the framework itself. Now we're talking about what happens if you actually have an agent community, right? So you publish your agent. Let's say I in fact, there's a I think there's a website that's called age agents.ai or there's actually a place that you can go and develop agents and publish them, right? So it's becoming the ecosystem of agents. You know, you can go and grab as many agents as you want and then you have an orchestration platform that you put these agents together to accomplish something much more extensive. So for these agents to be able to communicate with one another and plug and play to fit them together like a jigsaw puzzle you're going to need to have some interoperability standard. I think those are emerging right now as we speak, but there's going to be, as there are major players in this and they want to have dominance in this area, right? So I think Microsoft is going to probably push for its own agents so is AWS, so is Google. Crew AI is trying to be a little bit more like a Switzerland and making sure that its agents are communicating. People that have the tendency to focus on a particular vendor especially in large corporations because enterprises, they don't want to take any risks dealing with unproven maybe orchestration framework. There's going to be some time for some of these smaller players to gain ground, but I think we definitely need some industry initiatives to be able to define these interoperability standards between agents.

Andreas Welsch: 19:43

Thank you for sharing that. Now, folks, we're coming up close to the end of the show. I was wondering if you can summarize the key takeaway for our audience today. What's the number one thing people should take away when thinking about agents and evaluating what fits best to their business?

Kiumarse Zamanian: 19:59

So I think the best thing is to really understand what these agents are good for, right? I think this, we're basically looking at the kind of viewing AI more from a building systems perspective now. So for people who actually are familiar with building systems and putting things together think of these things as no abstractions that are able to accomplish things. And the main, obviously the main difference here is that you're really relying on these language models that the helping you with the inference or helping you with the planning the advantages that you don't really have to do a lot of times do any programming. There are frameworks that actually there's a lot of them drag and drop low code or no code at all that you can build these things. Really understand what is it that you start experimenting with these there's some tools out there that you don't have any, you don't need any programming experience and they're actually, you can go to some of these companies websites and start experimenting. I think experiment what these agent frameworks are good for, what you want to accomplish, and come to it from a project, maybe development, project management perspective break down your tasks start experimenting defining your agents, giving the persona, give them the the tasks that they need to be specifically good at the tools they need, maybe APIs, et cetera. So you like a, yeah, high level design of a project and then breaking it down into these things experimentation. And then I think more than anything, learning about what's going on. I think I'm reading quite a bit about this. There's some really good people on LinkedIn in this space that is really important to follow. But more importantly, I think it's just get your fingernails dirty a little bit if you want to play around, as I said, sometimes you don't even need to have any programming experience to just basically learn what's going on and try things out. So I'm really going to go big on agents a framework that supports it's open, it's easy to use, particularly if you're not a programmer. It does have the very usable interface. It's user friendly. It's easy to install, easy to maintain. Some of these tools sometimes require a fair bit of technical expertise to even install them. Depending on how much knowledge you have. Pick something that you're comfortable with and then start experimenting. And scalability is very important. I think it's one thing, but you want to put something in production that's going to have thousands of people using it or managing gigabytes of data. Then the scalability comes into play and and ecosystem openness and flexibility. All these are important. As I mentioned, I have a paper that I've written that I'd be happy to share after this call with the participants and others that I've discussed these at length.

Andreas Welsch: 22:39

Alright, Kiumarse, thank you so much for being with us today, for sharing your expertise with us, and for those in the audience, for learning with us.

Kiumarse Zamanian: 22:45

Thank you so much. It was great to be here. Thank you.

Andreas Welsch: 22:47

This is the last episode of the year. I can't believe that the year has gone by so quickly. Next year we'll start season number four of What's the BUZZ? And we already have some excellent guests lined up. I'll share you more about that in a couple of weeks. If you do celebrate holidays in the next couple of weeks, Happy Holidays and a Happy New Year!