.png)
TL;DR
- Treat movable credentials as the root cause behind 80 percent of breaches reported by Mandiant, CrowdStrike, and the Verizon DBIR. Replace shared secrets with asymmetric keys bound to a secure enclave so the signing key never leaves the device.
- Assume any IoT device shipping with shared credentials is one Claude Code session away from compromise. A security researcher seized control of 7,000 DJI vacuums by reverse engineering the credential flow in a single morning.
- Inventory the agents already running in your environment before writing policy. Discover which models, tools, and external endpoints each agent reaches, then restrict the data flow at the boundary rather than trying to constrain agent behavior.
- Decompose AI workflows into probabilistic steps paired with deterministic verifiers. Let the model guess, then loop each guess through a function that can confirm or reject the answer before the workflow advances.
- Hire and promote systems thinkers over punch list executors. The roles that survive agent automation are the ones questioning why a process exists at all, not the ones waiting for the next task.
Transcript
You're tuned in to IoT and AI Leaders, your go to show for insights, predictions, and big ideas on how IoT is reshaping the world of AI.
So I'm Nikhil, and welcome to another edition of IoT and AI Leaders. This week, we're going deep on security, not just device security, which we've talked about a little bit about in the past, but what we are going on is security in a world where IoT and AI come together at very, very rapid speed. And and my guest is a real expert in the field here.
He is Jasson Casey. He's the CEO and cofounder of Beyond Identity. He's got a deep security background, and he talks very nicely building it up as to where we came from, where we are, what's gonna happen in the future, and what we need to look out for as everything gets smarter, everything gets AI enabled. And also, as we're all buying products that are smart enabled and contain agents, What are the pros and cons and what could go wrong?
We've got a few good case studies here, and I give him one as well during this of in the in the payments field of what can go wrong. It's very, very informative podcast. He knows his subject matter inside out, and you'll hear that he is a really good marketeer because he takes a subject like security, which can be full of acronyms. He doesn't use very many of them, but he actually talks about monkeys locked in cages with pens, which really make it easy for everybody to understand what model we need to go forward on.
So with that, I'm gonna hand you over to my episode of IoT and AI leaders with Jasson Casey, the CEO and cofounder of Beyond Identity, and enjoy.
Jasson, welcome to the IoT and AI Leaders podcast.
Thank you for having me.
Well, thank you for being here. And those of you who are watching can tell that Jasson not only is an avid book reader, but has a set of golf clubs in the background, which tells you a little bit about him. And there's a fair chance that a dog may wander around because I think there is a dog in the room at the same time. So what we're gonna be talking about here is obviously, the theme of our podcast for twenty twenty six have been the the merger, IoT, and AI with a particular emphasis on AI. But before we do that, I just wanted to and we spent most of the first part of the podcast on the AI side and then talk about how it's morphing and those issues on security into the IoT world.
So so to start off with, maybe, Jason, for our listeners and viewers indeed, maybe just a quick intro to yourself.
Sure. My name is Jason Casey. I'm the CEO and cofounder here at Beyond Identity.
I have been in the the infrastructure security space since the late nineties. For Beyond Identity, I was the CTO of a company called Security Scorecard, which was a global threat intelligence and risk ratings organization. Before that, I ran engineering for an organization named IronNet, run by the former director of the NSA. And, you know, before there was a before that, and a lot of it had to do with think of it as, like, hyperscale and telco data infrastructure. So I grew up in the world as an engineer, morphed into product management, and then from there, went into the job you see me in now.
Right. And security, as we said at the intro, has always been important. You got a deep background in variety of, by the way, agencies areas. I'm sure there is maybe a few listeners who don't know what NSA is, but maybe you can just explain that acronym for our non American listeners.
Sure. So the NSA is the the National Security Agency. It is one of the the largest government organizations focused on signals intelligence and kind of protecting and defending the US's interest in the cyber domain.
Great. And actually, so let's go deep on that because it's it's been important for a long time, but now it's really, really getting important. I know you have some strong views on this, and indeed your company is kind of built around this, which is how AI is changing the economics of cyber attacks. And one of the things that you've said is the fact that speed now matters more than sophistication.
Maybe you could start off by just explaining that point of view.
Sure. The so there's a there's kind of this famous cartoon that I'm sure a lot of your listeners would have seen before, and it has kind of, like, the the evolution of man. So on the the left side of the cartoon, you kind of see, an early stage, ape of some sort. And as they walk from the left to the right, he becomes more upright and eventually becomes kind of a modern human.
Yeah. The, there's a version of this in the cyber world, which basically is kind of showing the inversion. So it's it starts off as a sophisticated, human on the left and devolves into kind of a less sophisticated animal on the right. And the the story they're trying to say is when this all got started in the seventies and the eighties, the sophistication of the cyber adversary was actually quite high.
By definition, they had to kind of understand and create all of the tools that they used to penetrate your organization. Back then, it was a mainframe over a dial up, but still penetrate your organization and steal the data they were after. As time has gone on, because of things like the proliferation of software, because of things like open source, because of the fact that there's just a lot more people in this space today than before, you don't have to be as sophisticated as you used to be to pull off a sort of cyber attack.
Now enter AI. AI doesn't just take the next step. It takes the next leap. And so whether you're someone who doesn't necessarily know how the tools work or even how to construct the tools, because these AI models have been trained over so much open source software that does these things and already has these patterns, it brings these sophisticated tools essentially to the the the bear of maybe an unsophisticated user.
Or another way of looking at it is for the sophisticated user, it gives them an army. It gives them an army for something that they would have had to have done themselves or farmed out to kind of more low level analysts before. So for instance, if I'm planning an attack on someone, the first thing that I wanna do is I wanna run reconnaissance. I wanna understand what goes on in that organization. Who are the key individuals? What are their key relationships? What does their pattern of life look like?
That would be an operation that would be run by a series of people kind of directing passive and active collection techniques.
In in a world of, in the agentic world or the AI world, I don't really need that team of people in the, across the span of a morning. I can either direct an agent to kinda help me do that initial assessment, or I can construct a custom agent to do that assessment in a constant way so it can kind of compress my attack cycle.
Let's say I'm a reverse engineer and I'm trying to understand the vulnerabilities of software to create an exploit or I'm trying to understand how malware actually works to create a defense.
Historically, I would set up an environment. I would well, even today, I would set up an environment. I would detonate the malware. I would have the software in the environment, and I would do both something called static and dynamic analysis. I would look at what I could tell from it not doing anything, and then I would poke it with a stick. Alright? I would tell it to run, and I would see how it behaves as it runs, and I would build a an understanding.
With the introduction of some of the modern AI tools, I don't have to be present for some of those initial assessments. And, like, the eighty to ninety percent of the drudgery of the work, I can actually kind of automate away. So AI is the natural kind of evolution that we've already seen, which is increasingly sophisticated tools in the hands of more and more actors that aren't necessarily nation states or even organized crime anymore. And AI is even accelerating it more so.
So world of very, very rapid change. And also, we have got now a lot more sophistication at the edge. And I know when we were chatting prior to recording this, we were talking about the the root cause and the and the this issue of really having to understand that a lot of issues also came from the fact that the credentials were movable. And I think you said seventy to eighty percent of your view of issues were actually rooted in the fact that the the credentials themselves were movable. And maybe if you could just talk a little bit about that and what are the consequences of movable credentials?
So when we say movable credentials, first, we probably ought to define it. What we mean is we mean something that's used to either authenticate a human or a device or a service. When we say movable, what we're really saying is it's a piece of data. Right?
So for instance, a password is a piece of data. A biometric, technically, is a piece of data. These things called bearer tokens, which are how you actually, traditionally deposit authentication tech, credentials inside of, like, offline devices, like drones. It's a piece of data.
And so if it's data, it can get copied. And if it can get copied, it can get stolen.
When we say most cyber incidents are the cause of movable credentials, first of all, I'll back it up with a little bit of data. There's three organizations that track the evolution of cyber incidents year over year that we pay attention to and kinda really reference their reporting. So Mandiant, CrowdStrike, and Verizon. And Verizon has a reporting service called DBIR, the database of incident response.
In all three of those port reports consistently, I wanna say for the last six to seven years, and the number dances between, like, eighty and eighty three percent or eighty and eighty five percent depending on the source, The source of all incidents tracked and reported on across those three organizations is an identity system failure and an access system failure. And our argument is that that is describing a symptom That is not actually getting at the root cause. The root cause is what we believe is movable credentials. So when I'm accessing a system, I authenticate myself. The way I authenticate myself is I share this piece of data in some way, shape, or form.
If that data can move, it can be stolen and clearly see that through all of these incidents. So what we argue is it's actually now possible to create a world of immovable credentials for authentication.
And as the world shifts to that, it drastically changes the landscape of cyber attacks, and it actually eliminates the root cause of eighty plus percent of all of the incidents every listener's organization has to deal with on a day in and day out basis.
And if that's the case, does that mean that in your view, when he talks about an immovable credential? I was thinking about, for example, something that that we use every day, you know, chip and PIN. In fact, over here in Europe, we've used it for many, many years. Chip and PIN has been standard for many, many years. Is that what you would call an immovable credential in that it's immovable in the sense that the the physical device is movable. It's the credit card in your pocket, but the credential is bound up in a secure enclave in the chip. Is that sort of one of the key differences is that it's kinda embedded in the electronics?
This is this is a perfect example. So before the modern credit card, we had a magnetic stripe that basically encoded a a number. And this number was the data was a reflection of what was on the physical card.
The number could be stolen, right, from online orders. The number could be stolen from skimmers. And I usually I don't know the origin. I believe it came out of Europe. But chip and pen was this idea of, oh, wait a minute.
We don't need to have, like, the symmetric secret that we shared to prove that we're the right person to pay this bill. We've had modern cryptography for a while. It's possible to create something called an asymmetric key pair and do a cryptographic signing.
And it's furthermore, it's possible to create a little specialized chip. Right? Secure enclave or trusted execution environment. Right?
Whatever whatever term you you favor, where that signing key is locked in that chip, and it's never in a main processor. It's never in memory. So maybe another analogy is imagine a jail, and there's a monkey in the jail, and the jail doesn't have a door, but the monkey has a pen. So in the old world, you would come to the jail and you would ask the the the monkey to essentially give you the pen and then you would go off and sign things.
So that pen was stealable. In the new world, with this new mathematics or the with cryptography, we could basically pass the bill through the bars to the monkey, and we could say sign this. And the monkey would sign it and hand it back to us. And that signature on that bill was our receipt of payment.
In fact, that's very similar to how chip and PIN actually works.
There is a tiny monkey in your credit card, and that monkey is a little bit smarter than the example I gave. When you put your PIN in, that PIN is the monkey's way of knowing that you not only possess the monkey, but you are the same person or at least you have enough of a relationship with the same person who set that monkey up in the car to where that monkey should in fact go ahead and sign that bill for you. Now the US, there there's another interesting part of the story, which is, like, the US and our kind of classic style, we rejected chip and pen forever. Right? Because we we don't wanna change our experience. Blah blah blah blah blah. Yeah.
Yeah.
And and then the funniest thing happened.
Apple came onto the scene with the I iPhone, and Google followed shortly with with Android.
And they quickly realized that, wait a minute, for for the argument of convenience, wouldn't it be nice to pay with your phone? Wouldn't it be nice to just kinda tap your phone instead of instead of something else? And so we ended up getting actually chip and PIN in the US essentially through mobile payments. Yeah. So the hardware that's inside of your phone is actually identical to the hardware that's inside of your modern credit card.
And in fact, it's it's a slightly more advanced version of chip and pen. It's chip and pen or biometric, but the pen or the biometric essentially do the same thing.
So when you pay for coffee or tea at that coffee shop with your phone, you basically have one of two experiences. You tap your phone and your phone says smile or swipe your finger, or you haven't set up biometrics or it doesn't recognize you because you have a mask on or something, and it says, hey. Give me your PIN code. In both of those cases, it's basically they're bringing your biometric or your PIN code to the monkey in the phone.
And when the monkey is satisfied, it you also bring it your bill. It will sign the bill, give it back to you, and you give it to the merchant. That's how that works. That is in fact a perfect example of an immovable credential. The only way of stealing your form of payment is physical theft.
And while I've learned through my trips to London recently that physical theft is certainly very easy, especially for phones Yeah. It is.
It does not scale like online criminal theft does, which is many, many, many, many orders of magnitude more rampant than physical theft.
You know, you introduced yourself as a big engineering background, but I think actually you're a closet marketeer because your analogies are really good and easy for people to understand. And I wanna come back to the fact that the most of the world, you know, we're obviously a IoT company with with six, seven million devices under management for our clients. And the issue of security on the device is huge for us. I'm gonna come back to that.
But but before we do that, the that's great if you control the credit card, if you control the phone. I mean, the monkey is locked inside the the chip on the credit card. The monkey, in terms of, you know, face recognition, I guess, the the the secure side of it is that your your face is locked to your head, which is locked to your body. So but for most devices that we're buying nowadays that are all being marketed as smart devices, that we're buying devices with AI in it.
I mean, just before I came on this podcast, my wife came up to me and and said, what what on earth does this mean? We have a we have an app that and they could be Chinese. I don't know. But we have an app that can turn the lights on and off remotely in our house and set timers and whatever.
And suddenly, they've released an AI version of it. I don't know why I would want an AI version of it.
But but now they're asking for my location, and they're asking for my electricity usage. And I suspect they're trying to help me in some way, but it feels creepy because these people are now asking for access to a lot of information. I just bought a smart plug that could be programmed with on and off times so we can turn the lights on and off while we're outside the house. So Yeah. The that's just an analogy of how you know, it's almost impossible now to buy a device that's not smart, and certainly that's not marketed with the word smart. Do you have any examples or or well known ones or crazy crazy examples of devices? Like, world world failures of maybe consumer devices that that bring this to life?
Yeah. So so there's a very there's a recent one that actually kinda combines a lot of the stories that we're talking about.
There's a Chinese company called DGI. They're very famous for their drones, but they make more than more than that. They make, they make household vacuums, robotic vacuums.
And a researcher recently when I say a researcher, mean, like, a security researcher, was, using one of these, modern AI agents, Claude Code, to reverse engineer, the robotic vacuum. And, actually, actually, I don't know what his actual objective was. I think he was just a natural tinker and just wanted to know, like, how did it work, and could he take control of it, and could he make it, like, vacuum his initials in his house or something like that.
And the first engineers do. Yeah.
The the first step in his process was he had to figure out what kind of credential, where it was located, and how he could steal it or copy it in order to issue control instructions to the vacuum. And so he turned Claude loose, and Claude figured it out for him very quickly. And then he built up his controller, and he's like, alright. Now I wanna actually start controlling my vacuum and make it do things.
And it worked.
It worked so well. And it also kind of took advantage of a a a standard faux pas committed by DGI by DGI. He had taken control of seven thousand or so robotic vacuums across the world, not just his vacuum.
And and so this was in this was all in the news, I wanna say, in mid to early April of this year. So you can if you're interested, you could definitely pull that story and and and get more details.
But it illustrates a couple things. It illustrate like like, clearly, this individual probably could have done it without the help of AI. Right? He's a professional security researcher. However, he was able to do it quickly with AI.
And it also illustrates that this this this root cause of most of our problems is still being violated by large companies that know better.
Right? This shareable credentials seated across all of these IoT devices. So that's kind of a humorous story. There's another one I'll give you is a little bit we'll call it theoretical, but it ties to what you were talking about with your electric switches and light bulbs a minute ago.
And this one is a little bit more, I would say, scary. But, like, somebody might say, hey. What's the what's what's really the harm of someone hacking my light bulb? What's really the harm of someone hacking my my dishwasher?
Like, is it really a problem? Do I really need to worry about it?
Well, there's a simple thing where if I could control enough IoT devices at scale that actually were power devices of some sort, that actually did sync a material amount of current, a a material wattage, then effectively, my adversary now has a big on off switch. Yeah. And the way power distribution works is power distribution is tuned not to the max capacity of everything attached to the network.
It's tuned to what the network's typical usage is. So if I could hack enough IoT devices and control enough IoT devices at scale, and I could flip them on and flip them off and flip them on and flip them off in the right way, I could actually create pathological loads on power distribution, blowing breakers.
And when you start to actually look at the electrical systems across various countries, these breakers are not necessarily commodities. They're not easy to replace. And if you break enough of them Yes. You're creating an outage that lasts a material amount of time. So, like, there are, in fact, very real consequences that I would argue maybe they're knock on or secondary effect, but they're still civilization impacting through being able to hack your light bulb.
And the great thing about well, say the the great slash scary thing about security is that the case studies are really interesting, but they also frighten you at the same time. And, you know, we're moving now into the conversation of how all of this overlaps with with our world in IoT and SI. I'll give you I'll I'll trade you. I'll give you an SI story, and I'll try and make it I'll not be specific as to who it was.
But let's just say we have customers about eight hundred customers, and we're in we have devices in hundred and ninety countries. Customers have devices in hundred and ninety countries. And in one particular country, we power something like thirty five percent of all of the credit card authorization devices that are used in shops and restaurants. We got chip and PIN.
Although in this country, they don't use chip and PIN. They just use a standard SIM card to connect to the telephone network. And because the telephone network is very fragmented, the mobile network is very fragmented. They use our system because we can hop between any any operator, so we're basically selling the promise of a hundred percent connectivity because, you know, you have a chain of thirty restaurants.
It it'd be pretty common that that at least twenty of them couldn't use the same SIM card because the network didn't exist. It's a very big country. Let me put it that way. Well, the purpose of the story was that we were selling them a package of, I don't know, two megabytes usage per month is not much.
Taking payments in a restaurant or a shop. And we got a complaint from a customer saying, are you guys ripping me off? You're you're charging me for four meg a month. No.
Well, you're using four meg. No. I'm not. You know? Here's the engineering proof that I'm only using two meg.
So it turns out that they bought cheap Chinese payment machines.
And the when we did a big trace because we are device people, and we did a big trace, and we traced, you know, the APNs and and the IP addresses. And so there was something in the firmware of the device.
Maybe it was innocent. I don't know. But, basically, it was replicating all the payments for these devices back to China. Shenzhen.
Yep. So every single credit card payment that went through these devices, the data on everything it did to the local bank went to Shenzhen. There was no reason other than the device was manufactured there. And so when we pointed that out to them, I mean, a, they had to pay their bill.
But but, b, it it was kinda we we say, look. We told you. You know, it's all about the firmware. The device and the firmware and and the security codes and a lot of stuff is now gonna be device resident.
And and and that's scary because they sell millions of those devices around the world. So this brings us into sort of the next area, which is what I call the the collision between IoT and AI. Because I said earlier, you know, everything's moving to the edge. Every device you you you now buy is is smart.
And we have a third component that we haven't yet talked about because now devices we have software.
They're not devices, but we have agents. So let's just park devices for the moment, and let's talk about software agents. You you can develop your if you're a big company, you can develop your own agents and all of the big platforms, and you mentioned Claude. But all all of the big platforms will let you develop agents.
But not everybody wants to buy agents. But often, they're buying software, and the software contains agents. Agents, because they're so intelligent and they're exponentially growing capability every year, presumably, are also a a security threat. You don't know really what you're buying and its capabilities.
You know, how can we protect ourselves?
And what are the best practices at least to try and protect yourself from the import of agents, which your employees could be doing just by downloading capabilities, software capabilities, which include agents? I mean, they could be you could be using thousands of agents without knowing it.
Hundred percent. And in fact, you are. So agents are very powerful. Agents are the easy way of thinking about an agent is an agent is really just a simple loop.
And I'm sure most people have used a chatbot agent where they they give the chatbot a prompt or a search to or or it's kinda like a search term, but but not. And the agent takes that and sends it off to an LLM and says, hey. What do you wanna do about this? And the LLM knows how to, based on that sentence, predict what the answer likely ought to be and sends it back down.
Yep. And, you know, slightly more advanced agents will add in what's called tool calls. So when you ask for a thing, the agent actually may have a series of tools. Like, it may have the ability to search the web.
It may have the ability to read Web sites. It may have the ability to write a file. It may have the ability to do x, y, and z. And so when it sends your query up to the LLM, it also says and by the way, here are some tools I have the I have in my facilities if any information from these tools might help you, the LLM, respond stronger or better to the user's prompt.
And so the LLM, than sending just a a final answer back down, it might say, well, you know what? You're right. I want you to go do these three Internet searches and tell me what you find. And so it'll loop. It'll tell it what it finds, and it'll say, alright. Now I want you to go do these additional searches based on what I found.
And it may loop multiple times. And then eventually, when it produces a result that it thinks is final, it then kind of dumps that back out to you. That's the chat agent. People are building versions of that that aren't necessarily driven by users, but driven in a more autonomous way.
Right? Like, maybe you suffer, like I do, from too many emails, from too many people to really be as responsive as you truly need to be. So the agents very, very simply is there to kind of, be an assistant and sort the email and figure out what's junk, what needs a response but not your attention versus what actually needs your attention, and it needs to have access to some certain things. But what's to stop a clever person from emailing me something with a prompt injection that then convinces my agent to use a tool call to divulge my financials back out in the email response to my competitor?
So it is actually important for organizations to know what's going on in their environment right now. The large tentpole is called AI governance. I'm sure people have heard of this before.
The specific area or the specific use case that we're actually now subscribing, I would say, is ShadowAI.
And here's I'll I'll do a little bit of a shameless plug. We actually do have a product in this area. It's called CEROS, c e r o s dot s h. You can take a look, at it online. There's a there's a way for you to try it out.
But it is a tool that helps organizations, essentially discover not just what agents exist in their environment, but what do those agents have access to? What tools do they have access to?
What providers are they do they have access to? Maybe it's okay if they're talking to Amazon Bedrock instance of of Anthropix models. Maybe it's not okay if they're doing your phone home to China for some sort of Chinese hosted model. So number one, understanding what's going on in your environment and then building a plan around what you discover to essentially control the data flow.
It's really, really hard to control an agent. And I don't know if everyone will appreciate the story, so I'll keep it short. But it's not possible to really control what an agent is going to do in itself.
And the reason it's not possible is because while you can trivially say an agent can't make certain tool calls, it's difficult if you let an agent write start writing programs, it's actually difficult to an analyze a program to truly understand what that program is actually going to do without letting that program execute.
And this actually tracks back to a very classic problem that Alan Turing defined back in the nineteen forties. Right? Like, is this program going to halt? It turns out that it's provable that, like, in many cases, you can't answer the question.
So an easier problem to solve isn't what is the agent going to do. An easier problem to solve is how do I track the data that flows in and out of the agent? How do I restrict the mingling of data and the understanding that if I let privileged data into an agent and I let that agent have access to public services, information exfil is possible without actually analyzing that flow. So anyway, we I could I could talk for that for a long, long time.
But at at a high level, the establish an AI governance program, understand what's going on in your environment with some sort of shadow AI use tool, and then build some sort of policy around how you are going to monitor and enforce data access and data controls for agents.
Enjoying this episode IoT and AI leaders? Hit the follow button to get notified when new episodes come out. Now let's get back into the show.
So let's go a bit deeper on that because I think it really is the next train that's coming down the tunnel or whatever analogy we wanna use, and it's a big one. You know, again, I I pick up so many relevant factoids from just listening to the radio as I drive between places. And I think it was yesterday or even on TV this morning. I don't know.
It was certainly recent. Somebody saying, you know, this year maybe a lot of people are saying this. I'm sure they are. This year, twenty twenty six, there will be more agents deployed than there are people on earth.
Now that sounds a crazy statement because there are eight eight billion, and maybe they're referring to people who use computers or whatever. But it it goes back to this issue that, you know, there are companies that are now paying their employees for how many tokens that they are using, which is which is driving you know, use as many agents as possible. But the adoption of agents is proliferating. Agents are not just separate.
They're talking to each other. They're making decisions. They're sending data between each other. So all of this raises the one entity that we haven't talked about yet or the one thing we haven't talked about yet, which is the human being.
So you talked about, you know, you've got a tool for measuring, looking at the inputs and the outputs and, I guess, pattern recognition. And I know that's unusual behavior. Flag it. You may wanna look here a bit more, which is sounds like an extremely valuable tool.
But what about the human in the loop? And the context of this question is one I ask all my guests, which is the the thing I read in the popular press on TV is, you know, AI is gonna take all the jobs away. And I think the and I think you do as well. Think the answer to that is a little bit more nuanced.
One is it's going to be an augmented, a collaborative environment between the human and the technology.
But even then, it's not like a sort of stack model, which is here's a stack of LEGO bricks, and AI can currently go up to brick six, but by next year, it'll go up to brick eight. I think it's you've got a really good view on it's a more nuanced division than simply task based, and it's to do with what the AI is doing that it's good at and what the human being can do that human beings will always be better at. And and I know that you have a view on that. I think that's definitely worth worth worth sharing here and going into as we're getting towards the end of the pod.
So AI is not magic. And get into the gory details of how it works, but we could also safely just treat it like a black box where the black box is a really good guesser. Right? You give it a question, and it's gonna give you an answer.
And because of how that that AI was built, the answer is gonna sound plausible. The answer it it's gonna sound reasonable. Right? It doesn't mean it's gonna be true.
But, again, it's a good guesser. So most of the time, maybe it is true, and it's gonna follow a probabilistic distribution. Right? The answer is gonna be probabilistic.
In domains that matter, people typically pair up these AI guesser with something called a deterministic verifier.
So for instance, there's a lot of problems in the world where we already know whether a solution to that problem is in fact true or not. Right? And so in that case, we would say we have a a deterministic verifier. So for instance, let let's talk about the world of math. If I give you an answer and I give you an equation, it's fairly trivial for you to essentially verify if that answer is in fact an answer of the equation. Right? Now there are other types of math problems where if I ask you slightly differently, is this the most optimal solution for this problem?
All of a sudden, you struggle. And, like, the the problem is not not not that tractable. So in a world of consequence or where things matter, successful folks are figuring out how do they actually partition problems to the probabilistic portion and the deterministic portion. How do they bring deterministic verifiers to bear, and how do they kinda wrap all of this together?
The it's really, really important to just remember that at the end of the day, the these AI tools, they're they're trained after us.
And I wouldn't even say they're trained after the best of us. Right? They're trained on corpus of scientific literature, which which is kinda good. They're trained on the corpus of Reddit, which I think we could probably argue is, you know, it's highly dependent.
Yeah. Exactly.
They're trained on Stack Exchange, which used to be the number one go to for programmers to figure out why their program is not compiling.
And and you can see it when you interact with the AI on some of these problems. You see it basically adopt sometimes the lazy engineer approach where you ask you you give it a a a program, and you ask it a problem about what's going on. And it comes back, and it says, hey. Your your your your question's not even relevant. Like, here's a solution that tweaks this thing and just solves the problem easily.
And that's generally you know, that it's kind of like it's human behavior. Right? It's trained off of human behavior. And if you're trying to understand something principally, if you're trying to actually do a root cause analysis, it's frustrating. But, also, when you take a step back and think about, well, how do you interact with an intern? How do you interact with a fresh graduate?
Like, these individuals are also going to have more distribution style behaviors.
They're going to need oversight. You can't trust everything that comes out of them, but they're generally effective at covering ground and being inure to and getting work done. And so, like, this human agent partnership or or supervised relationship, at least it's where I see us being most effective, certainly within Beyond Identity of how we've kind of done our AI transformation. For harder problems, splitting the problem into so what do I mean when I say split a problem into probabilistic and deterministic?
A lot of you have probably used chatbots. A lot of you may have even used some of these more powerful agents like Claude Cowork or Claude Code.
And, you know, the first instinct you have is you just dump a big problem in the prompt, and then you see what comes back.
Then as you have these sort of problems before, maybe you realize, oh, I can take that prompt, and I can maybe make it a template. So I don't have to type the whole thing anymore. I just say, hey. Use that reasoning style or that skill that I talked about before and solve this problem.
And then your next level of maturity is you realize, well, wait a minute. This probabilistic guesser is trained off the the the typical human interacting on Reddit and and Stack Exchange.
And I've and I can see that. I notice in its results. Like, I'll say, go do this over this list, and it'll, like, give up after eight the eighth item. It won't do nine and ten.
It won't do it won't actually verify the answers for the entire list that I sent it. Right? And you realize, well, wait a minute. Checking like, enumeration is not a probabilistic guess.
Enumeration is a very well known deterministic action. For each one of these items, do x. For each one of these answers, call the verifier.
And so your next journey is you realize, well, wait. Maybe I can decompose this skill to be a series of prompts in a series of verifier calls or a series of function calls, where the function calls are basically doing the probabilistic part, right, for each call the verifier.
And the the prompt parts are doing the probabilistic parts. Like, I I need actual reasoning here. Give me your best guess under these constraints.
And so all of a sudden, you've built something that looks a lot like a classical computer program.
But rather than a collection of functions that are getting called in a very structured way where each function is kind of deterministic, Some of those functions are LLM calls where they're basically probabilistic reasoning. This is kind of more towards the mature or the the slightly more advanced usage and pairing of AI that I see be effective both in the organization.
But I think it also mirrors itself in maybe someone doesn't, like, consciously think about their steps in that way, but they do evolve to that style of behavior over time if they really are kind of working with the AI.
And would your prediction be as we finish here? Well, I think the first question is rhetorical, but I'll ask it anyway. Given the advancement of AI and adoption of AI, which is still in the early stages, as I said, that this issue of probabilistic deterministic interaction and and breaking it into pieces and ask a question, verify ask a question, verify, is going to become more I think the first well, the first question, is it gonna be become more and more important going forward? And secondly, does this in effect, does this define the future skills requirement for success in the workplace? Yes and yes.
So earlier, you said something like, is AI gonna kill all the jobs? And and you think there's a nuanced response there. And I would agree. There are certain jobs and maybe a better way of putting it is there are certain behaviors that I think are not gonna be rewarded with full time jobs now that we have AI.
The person who just wants to be given a punch the the knowledge worker who just wants a punch list to execute and does not wanna consider the business context and does not wanna consider the system context, who just want thinks of themselves as like a a software artisan, if you will, I think their days are numbered. However, person who enjoys systems thinking, the person who enjoys the business context, the person who's questioning the assumptions of, like, wait a minute. As an organization, why do we even behave in this way to begin with? Right?
We're trying to solve this problem for the customer. We did x, y, and z because that was the best technology afforded us at the time. But now with with an agentic approach, I could maybe do things in a slightly different way. That's the person who's gonna get rewarded.
And and finding those people that's another subject for another pod. Finding those people before you hire them and checking they've got those skills is a big issue because that's not what you get taught in the university environments except for early in career, etcetera. And for a lot of people over the age of I'm gonna say forty. I'm I'm I'm a lot older than forty, but I'll I'll just pick forty.
For a lot of people over the age of forty, they they they're not touching AI right now. They're they're not doing that because they think the the younger people the organization do that. The younger people in the organization don't have those skills either because this is coming at them so quick. This has been extremely informative.
I've certainly learned a lot. Again, I compliment you on your analogies. I will remember the monkey analogy. And I wanna give you the opportunity just to mention your ring again.
I think it have I got it right? Is it called Ceros? Did I get that right?
It's called that that is correct.
It's called Ceros, c e r o s dot s h.
Okay. And that can be found on your website?
Yeah. So just go to the website, Ceros, c e r o s, dot s h. It's basically an AI governance solution.
It helps you regardless of where you are in your AI journey. If you're just getting started, it'll help show you what already exists in your environment. You may find it surprising. And if you're little later in your journey, it helps you actually apply some of those controls and governance techniques that I've been talking about.
Before you get too far in. Jasson, this has been great. Thank you very much. Thanks thanks for joining the IoT and AI Leaders podcast.
It's the first time we've gone deep on security. And as those two areas are coming together, not just optimizing the firmware, as we mentioned, but now the AI layer and the agents. So I'm sure a lot of people would have found that very, very interesting. So thanks again for being my guest.
Thanks for having me. And hopefully, will.
Yeah. Great. Thank you.
You've been listening to IoT and AI leaders. We hope today's insights help you drive smarter, faster business innovation with IoT and AI at the center.