Andreas Welsch: 0:30

Today, we'll talk about vibecoding and its impact for developers and investors, and who better to talk about it than someone who's actively working on that, matt van Itallie. Hey, Matt. Thank you so much for joining.

Matt van Itallie: 0:41

Thank you so much for having me. I'm really excited to be here.

Andreas Welsch: 0:45

Wonderful. Hey, why don't you tell our audience a little bit about yourself, who you are and what you do.

Matt van Itallie: 0:50

Sure. So I am the founder and CEO of a software company called Sema. And what we do is bring executive insight into engineering, demystify engineering, software development for non-technical audiences on the C-suite and the board of directors. I'm the son of a math teacher and a computer programmer, so I was born to treat code as data. And have had the great pleasure through sema to help analyze more than a trillion and a half worth of code bases over the last eight years.

Andreas Welsch: 1:25

Wow. That sounds impressive. So I'm super excited to have you on. Should we play a little game to kick things off?

Matt van Itallie: 1:32

Absolutely. Let's

Andreas Welsch: 1:33

do it. Okay. What do you think? If AI were a band, who or what would it be?

Matt van Itallie: 1:41

If AI were a band? Who or what would it be? It's probably a cop out to say electronica electronica who are the folks who do whip it? Is it Devo? A normal but also kooky. That that is how I think about about AI. It is human-like, but also kooky in some and surprising ways.

Andreas Welsch: 2:08

I love that. That's a great answer. I'm always surprised with what my guests come up with when I ask these questions and what the analogies are, but that's definitely a great one. It seems that the, internet lately has gone pretty crazy in the last couple of weeks about a new trend, right? And lots of people are having good time with that too, vibecoding, and with a few prompts and the right tool, anyone can quickly build an app and pretty much any app too, right? All you need is an idea and an AI tool that can generate the quote for you. And now I know you've been in this space for code analysis and, these things for a number of years, like you said. I'm just curious, what are you making of this trend as somebody who professionally looks at these, type of things and, helps companies understand how good is their code and is it usable and is it modern and all these kind of things.

Matt van Itallie: 3:00

Exactly. I always like to start with the, personal take. I, myself have been trying out vibecoding and have used it to build simple prototypes of applications over the last few months. And I absolutely understand what the buzz is about, if I can. I am pretty technical, but definitely not a professional software developer. And I am able using these vibe coding techniques to take an idea heavily refine it first, and then produce a working code that is live. Just through my own efforts. It's personally incredibly exciting to be able to build that way, to have outputs that are accessible accessible to me that weren't accessible before. And it's also really helpful to our business, to be able to really the, next level of prototyping by showing real products in the wild, so to speak. It makes come alive what we're building next, much more so than ideas, words on a page or images even just a clickable prototype, but a real a real live app really goes a long way.

Andreas Welsch: 4:24

I have a couple follow ups for you in a second, specifically around prototyping and, I've been experimenting with white coding as well, and spend a couple long nights, short nights, depending on how you look at it. Going down the rabbit hole of getting excited about something that I built in a couple of minutes and then spending hours trying to get it to where I really want it to be. And I'm, just amazed and excited by the opportunity too. You know me, too. I've learned about software development in, college. I did some, scripting when I was in a, in an IT organization. Also, I've never really been a professional software developer. Like you shared this as well, but I know enough how how you write code what, the different elements and functions and variables and constants and things are and what they do. So being able to, have that booster and, do that much more quickly, I I can only share the excitement. It's, amazing. These days. But I'm curious you, work with a lot of startups and founders probably among many other different kinds of companies, and who, are you seeing who is vibe coding actually for and to, to what degree does it. Democratize that access to software development.

Matt van Itallie: 5:40

Yeah. I use the very narrow term of vibe coding to mean only using human language based techniques in order to produce code and only edit it through the human language. Interface, which is a particular subset, a very important and obviously very sexy subset of the overall approach of GenAI enabled of GenAI enabled software development, which involves not only perhaps starting with human language interface, but then using professional software development techniques to modify. In the strictest definition there are many there's absolutely right a democratization of many folks who are non non coders who are actually trying to build full and complete applications just through vibe coding. I believe the, latest from Y Combinators, it may be a 10% of their products, if I understand it right, of their recent class, are almost 95%. GenAI generated only code and they're able to generate some meaningful revenue from it. We also know that inside product management functions within organizations that vibe coding has has skyrocketed, which makes sense. Traditionally product managers would build very detailed requirements and then turn them over to an engineering team. For prototyping, now for prototypes, and we're gonna talk about the risks and why the limitations of vibecoding, but certainly for a prototype that just needs to get the idea across soft excuse me. Product managers are able to do that themselves and skip a step. And what I would say, thinking about how busy. Our developers are take, remove some of the lower priority work. Basically engineering, software engineering is so, valuable and what they do is so valuable. Anything we can do to take non-essential work away from them and let them focus on the hard stuff, the meaningful stuff. It's good for them, it's good for the organization, it's good for ultimately users. So I know from our experience and more broadly product management is probably the place where vibe coating has taken off the most significantly at, the enterprise level.

Andreas Welsch: 8:08

That's exciting and still also a little bit concerning. I could imagine for software developers who say if anybody can do what I have learned and trained, or maybe even they think they can do it, but they actually can't because I have spent years honing my craft. Where do you see this, going? There's certainly leaders out there, leaders of AI labs who say. By, a certain point in time in the not too distant future, we will only be using AI to code and software development is losing in relevance. What do you make of that?

Matt van Itallie: 9:05

I studied history in undergrad, and I take a longer term historical view here, which is let's say 70 years ago to be a computer programmer meant to manage. Punch cards and keep track of them and make sure they were sorted, and then feed them into the computer. And you were a computer programmer and you literally were working on these cards. And since came in no particular order, version control systems and IDs, like workspaces where folks can get work done and open source code where you're pulling what you want to build, but you get, you don't have to build it yourself. And so in that sense the history of software. Development is one of greater and greater abstraction where you know the idea and then you have more and more adept tools to be able to carry this out even before. Even before the rise of AI for coding if you take the, ratio of software developers working on the Apollo mission, so the space control, NASA space control systems in the seventies, and you compare that to the number of software engineers working at SpaceX, something like a thousand times more productive over the history over that, time period of what individual coders are able to do. Now AI is a continuation of that. It obviously, we all believe that it's a, greater acceleration of what, is possible. But I think the nuance the the understanding of how to manage edge cases, of how to manage complex system, of how to handle. Existing code. It's, different perhaps if you're starting with, new code and how far you can get, but to take existing systems that are powering everything we do since software has been eating the world for some time developers. Developers will remain extremely critical in managing enterprise systems. They'll be empowered by AI, but they'll remain absolutely critical.

Andreas Welsch: 11:11

Something that I've experienced firsthand. I feel it aligns pretty well to some of the research in other areas when it comes to GenAI use, where some of those sentiments where AI helps those that have a certain level of proficiency be even more productive. But it also helps those who are more at an entry level get to a similar level as, those that are, proficient. For me, when I vibe coded my business simulation app, I felt like it was propelling me and, accelerating parts of what I know because I know where some of the gaps are, where some of the issues are, or when I ran into issues that I could troubleshoot it or that I had an idea what to look for and see how I can tweak it. So I think you absolutely have a point there that if you are a professional software developer, this helps you on one hand, accelerate. What do you do? But on the other hand, you need to have that knowledge so you can also either counteract or figure out why it's not working exactly as you want it to. And guided, too. It's almost like pottery. Not sure how many software developers will like that statement, but we'll figure that out in the comments. Now. You mentioned prototyping, right? It helps you if you're in product management or maybe if, you want to create a sales demo or just show the art of the possible to some extent, but I'm, Concerned that people might take this a step further and say it's not just the art of the possible. It is the possible and we deploy it and we maybe monetize it. We put it out in the open. We have a new app but maybe have not applied the, same rigor that professional software developers today would apply. What unexpected challenges do you see that novice coders might run into?

Matt van Itallie: 12:58

Yeah, so there are, four primary risks for using GenAI to code, whether it's. Vibe coding, and it's purely purely using natural language or using GenAI as a start. And then also coding directly the risks are greater with pure vibe coding for sure than GenAI enabled or enhanced coding. The first is security risk GenAI code. Leaves behind or introduces security vulnerabilities that lets bad actors act or information leak out by the way, so does all kinds of code. So does human written code, so does open source code. The point is not that one way necessarily is more riskier than others, but that they all need a human in the loop. They need a review in order to manage that risk. A second is code quality and maintainability and accuracy. Again, a prototype might have a simple number of use cases, but to get the edge cases right, and if you're gonna use it. In an enterprise setting, for sure. It needs to be exactly right. Getting it close really doesn't cut it. It really doesn't cut it. And coming along with that, there's such an increase in volume of code because it's so easy to just have the system spit out spit out so much code that it, it becomes harder, it's harder to understand and therefore harder to maintain. Then there's two other issues that are at least in, in specialized situations. The third is GenAI code will under certain circumstances, likely not get copyright protection a form of intellectual property protection. And so for enterprise organizations who copyright their code, this isn't about copyright infringement, but copyright protection, it seems increasingly clear that they're gonna have to change their approach to ip. Management or and or emphasize the importance of of software developer involvement in the code itself. If you're listening to this and you work at an enterprise that copyrights code, please go talk to your lawyers to make sure that they're helping you on a go forward plan. And the fourth and final reason is something we call exit risk. And what exit risk is the possibility that if you're trying to sell your business that organizations who are conducting technical due diligence, which is say looking under the hood at the code itself might conclude if, your code was entirely vibe coded or GenAI generated, what are we buying? We can, just produce this ourselves. Audience think about, the analogy breaks down a little bit, but think about a a professor assigning a term paper and the student comes back and the professor says, how much did you do? And the student said I just fed it into an LLM and spit out the answer. That's of course, unethical. Although, I guess good, on you for telling the truth. This this student but it certainly does not in any way suggest there is intellectual knowledge in the underlying document. And that's the that's the very real risk. In, diligence situations. And by the way it's, already begun to happen where folks are walking away, organizations are walking away from deals where walking away from deals where the code is almost entirely GenAI generated.

Andreas Welsch: 16:25

And, what are some of the reasons why they're walking away? Is it, because there's no actual IP in the business?

Matt van Itallie: 16:33

Yeah. There's no IP if you can prompt your way to code, nothing's stopping the buyer from building it themselves.

Andreas Welsch: 16:40

Yeah. Now even before vibe coding was a thing. I think investors, PE firms, I know you've worked with some of the largest PE firms on the planet. They started to get interested in this composition. How much is actually human coded? How much is AI coded? What are some of the other reasons you mentioned IP as one part but why should it be careful there on one hand vs. vibe coding, but have the human in the loop.

Matt van Itallie: 17:11

Yeah, exactly. So human, the humans in the loop. Speaking of another reason why developers remain so essential is the best way to address all four of those risks. On the security risk side there is in our view a presumption that if the GenAI code has not been modified well, that means it has not gone through a security review. It's not feasible. Code, it comes straight outta the LLM to be security risk free. So it's a, there's a default presumption that that sellers would have to that would have to justify. And so having humans in loop and obviously involved in modifying or blending that code instead of it pure is what we would say is at least a, is a signal that they they're addressing security warnings quality and maintainability. Same thing if it just came. Out of LLM, there's a presumption that it's definite that it hasn't been deeply understood. And the edge case is added. So the human blending really makes a difference on exit risk. Huge difference between on exit risk. Huge difference between code that started with GenAI and was modified, and which is to say pure GenAI pure GenAI becoming blended GenAI and just GenAI code itself. That stayed pure. From the beginning makes a really big difference in terms of the likelihood of the intellectual property. And then finally, on the on the intellectual property risk side, the copyright side, this is not legal advice. You should most certainly talk to your lawyers if if your organization seeks copyright protection. But the US Copyright Office announced earlier this year that, prompting is not enough to get that protection. You need to have humans actually creating the work. It's a fundamental component. Human involvement is a fundamental component of that IP protection.

Andreas Welsch: 19:07

Thank you for sharing that and I really appreciate how you're speaking with depth and experience on what you're seeing in the market. It sounds so easy. Soon enough we won't be coding. AI will do all, the work in so many different sectors and professions. Software development is as well. But then when you look under the hood and like you just did and you said IP security, exit risk and so on, those are the sort real risks, especially if you want to sell your business at some point or you want to sell your app to, to an investor. We've already covered a lot of ground from by coding as this cool new trend. You don't have to be a programmer. You can just instruct the LLM or the, tool to create an app for you. Using natural language, you can refine it. You either should know what you're doing is what I'm hearing, or you should know somebody who can help you. Figure out what you're doing and, that you're doing it right so that you're protected further down the line. What are some of the other things that you're hearing in, in, in the industry when it comes to coding GenAI coding? Especially as far as in investors, PE firms, private equities are, concerned. What's top of mind for them?

Matt van Itallie: 20:25

Yeah. I'm I'm happy to share with you and your audience the results of of our benchmarking, which we believe is some of the, first benchmarking there is on a Gen Ag code usage, certainly among investor backed companies. And what we found at Enterprise Grade Code by large part investor backed companies, GenAI is approximate for the median organization. I'll just get it right. Organizations are using GenAI to start. About 11% of their code, their production code. That's the 50th percentile. The 25th percentile is 6% and the 75th percentile is 16%. So half the organizations are at the, between starting GenAI between 11, excuse me, between six and 16%. The pure GenAI code, where it started with GenAI but has not been modified is at most. Between three and 9% of the code. That to us, is a really good sign because that means everybody's getting the message that. Starting with GenAI code is a good idea in at least some circumstances which we most certainly agree with. But my goodness, pure GenAI code is, really risky and should be kept kept to the minimum in enterprise settings like this. So developers know it, they know code is a craft. It is not just reducible to mechanical process and they know that they have to be involved to make sure it's correct and building the right thing and building it with care. And also the organizations they are a part of have gotten the message that GenAI is a very, powerful tool and it's only gonna grow in in in importance in, the developer's tool chain. But my goodness, that needs to be done with care care and precision in in the production stack. And, when the code actually gets in the code that actually gets used by enterprise users.

Andreas Welsch: 22:24

So the question that I wanted to ask earlier goes in the same direction is when you hear, again, AI is going to make coding that just easier, but it's going to automate it. When you hear about the, risks on, one hand, right? It's not as simple, it's not as straightforward, especially not if you want to patent it or if you want to copyright the work that you're doing. Take it with a grain of salt. And on the other hand. To your point the, data seems to, show that it's already a great starting point to build upon, to refine it, to extend it. So good, happy medium as in many things in life, right?

Matt van Itallie: 23:04

Exactly we say the goldilock zone. Literally we explain to our clients you too little GenAI code. And you are missing out on a great opportunity for productivity, for job satisfaction, for all the above, but too much and certainly too much pure as opposed to blended code. And you're opening up the organization some real risk.

Andreas Welsch: 23:24

Now, Matt, we're getting close to the end of the show today and I was wondering if you can summarize the key three takeaways for our audience today from our episode.

Matt van Itallie: 23:32

All right. Certainly number one is that if AI was a band, it would be Devo. Wait, that doesn't count. Number one, everyone should be trying everyone should be trying vibe coding, and if that feels like too much, just try vibe writing. Have it produce an article for you in human language. You just, everything we're doing these days is is everything we're doing electronically, at least is going to be AI enabled, if not AI base. And the faster you can wrap your head around it, whether it's picking a movie or or doing something at work, the easier you'll be able to understand its capabilities and its limits. That could be through vibe coding, if you wanted to build something or see what it's like, give it a shot. But if not, pick something and put it to work. Everyone has to be an AI enabled. There's, you'd just be massively missing out. Again, whether it's vibe coding or not. Second vibe coding in particular is a spectacular way for to enable prototyping and especially to help enable product management teams get to testing faster, in particular, by protecting engineers' time. And third, GenAI coding in general and vibe coding in particular is incredibly dangerous in enterprise settings. And it must to mitigate must have a human in the loop, must have developers blending that GenAI code. Pure GenAI code is extremely risky. And our view should not be used in enterprise settings beyond a di minimis amount.

Andreas Welsch: 25:05

Thank you so much, Matt. It's been a pleasure having you on and learning from you and, hearing all these insights from you on how you can use GenAI as a developer and also what it means if you're an investor, if you're trying to sell your app to an investor, things that we should all be aware of in enterprise especially. Matt, thank you so much for joining us.

Matt van Itallie: 25:24

Thanks so much for having me. I really enjoyed it.

Andreas Welsch: 25:27

Wonderful.