From pilots to practice

Episode 10: TechUK Recognition, You Get What You Pay For & The Very First Cyber Attack 🍺

Ten weeks. The longest either of them has been consistent at anything. Neil's briefly out of the doghouse. Kieron's on squash instead of beer. And Leading AI has just had two case studies published in one of the most important AI reports of the year.

Pull up a stool.

TechUK's "From Pilots to Practice" report — and we're in it. Twice. 🏆 TechUK — the UK's leading technology trade association representing over 1,100 member companies — published their landmark report From Pilots to Practice: Using AI in the Public Sector with 19 real-world case studies showing how AI is genuinely transforming public services. It features not one but two KnowledgeFlow implementations: the FE college quality assistant and, in housing, Taff Housing. Being independently selected by TechUK as an example of AI that actually works in practice — not just in pilots — is a significant validation. 

Agentic AI for housing — getting serious with Taff Housing Kieron was with the Taff Housing CEO, director of technology and senior team this week — a monthly meeting they hold to track progress and plan what's next. On the roadmap: a fully agentic tenant inquiry system that triages and instantly answers routine emails, freeing frontline teams for complex cases. And an AI-powered repairs checker that reads job descriptions, checks them against schedule of rates codes, and flags when a contractor's invoice looks a little... creative. Yes, Kieron finally remembered to show the photograph this time.

First draft of a  £500 million bid written in 4.5 hours Neil used KnowledgeFlow's BidWriter to produce the first draft of a 7,500-word tender response for a £500m contract in four and a half hours. What would have taken a week landed at 80% complete before Tuesday lunch. 

McKinsey now tests candidates on AI prompting Not whether they know about AI — whether they can prompt well, challenge outputs, and think critically alongside it. Social workers are already turning down job offers at councils without AI tools. Now the world's top consulting firm is screening people out if they can't work with AI. The direction of travel has never been clearer.

The IQ of your AI depends on what you're paying for Anthropic's Opus 4.7 has an estimated IQ equivalent of around 140 — top percentile of humans. The free tools? Closer to 100. It's like hiring a £20k accountant versus a £100k one. If you tried AI and thought it wasn't impressive, you were probably using the wrong model. You get what you pay for.

The $150,000 overnight token bill A company set an AI agent running overnight. By morning, Google presented them with a $150,000 token bill. KnowledgeFlow is now building automatic kill switches for all client deployments. And a brilliant tip from Nate B. Jones (third plug this series, still not on commission): convert your PDFs to markdown before loading them into AI and you'll save up to 87.5% of your token costs. Most people don't bother. Most people will when it starts hitting them in the wallet.

The first ever cyber attack — 1834 Neil drops a history bomb: the first recorded technology attack happened in France in 1834, when two men hacked telegraph wires to manipulate financial markets. Kieron's response: what's actually changed?

 Ten weeks in. Still going. Still on squash instead of beer. Apparently Kieron has "aura" though, so things are looking up. 

Two mates. A bar. Thirty years of business between them. And all they want to talk about is AI.

Pull up a stool — we'll get the beers in (even if we're not drinking them ourselves). 🍺

Transcript

SPEAKER_01 0:01

All right. Shall we get this pantomime horse of a podcast underway again for week 10? 10 weeks, Kieran, we've been doing this. That's the longest we've done anything like this. As long as we've been consistent.

SPEAKER_00 0:16

That's because of that single piece of feedback we got to encourage us, isn't it? One piece. One piece in ten. Well, here we are. Our tenth anniversary. Can you call it that?

SPEAKER_01 0:29

No, you can't. It's 10 years, you fool. There's no such thing. It's like those people who celebrate it's like our six-month anniversary. It's like, no, no, you've been going out for six months. It's not an anniversary, anniversary of the year, you idiots. So uh anyway, anyway, enough of that old nonsense. How would you know?

SPEAKER_00 0:44

I want to know, did you get yourself out of the doghouse? You last weekend and you were you were firmly in the doghouse. You went away for a weekend with Mrs. Watkins, and um, I'm assuming you're now out of the doghouse and happily living your life.

SPEAKER_01 0:58

Well, what can I tell you? I can tell you that I was out of the doghouse for the weekend, but it turns it turns out it's really easy to get back in the doghouse then. Much easier to get in than out, I find. It turns out once once you leave the uh four-star hotel uh uh after spending a fortune on fine dining and champagne and flowers, uh that actually when you get back to reality, it kind of it all does. I used to joke that my wife miles, not brownie point wife miles have the half-life of Uranium 232, they just disappear like that. So there's no getting them back once they're gone. So never mind. Enough about enough about that old nonsense.

SPEAKER_02 1:40

Yes.

SPEAKER_01 1:42

What was the big highlights from your week?

SPEAKER_00 1:44

My week, well, my highlight. I've had a very uh busy week, which is great. Um, talking to customers, which is even better. Um, I have been talking to multi-academy trusts, I've been talking to housing and university. So um today was University Day, where we were training AI champions um in knowledge flow and giving them the kind of overview of how it works, how it can help them. Um and really good that they've identified the university have identified a bunch of AI champions to help drive what we're doing. So that's really, really cool. Um, and interestingly, on the other side, I I also had a meeting today with the executive leadership team of the university on the driving a bit of accountability into into forcing through some of the things we're doing with them, which is interesting. I I uh one of the things that often turns up, as we as you all would be very painfully aware, is we get someone that buys knowledge flow and then in some cases they're like they're using it and their team are using it, but actually it's not being used more widely. Um so to trying to drive a bit more accountability into some of the projects that we're doing with this university. We've got amazing specialist tools for the uni, I think, as you know, it's um helping them with their compliance annual monitoring reviews, being able to effectively do that with how you throw the raw data at Knowledge Flow, and then Knowledgeflow has been trained to understand how to do an AMR, which is an annual monitoring review and a report that you have to write. Um, and Knowledge Flow does that for you. You give it the data, it tells you here's how it stacks up and here's the things you need to focus on, which is brilliant because and um similar to the offstead quality tools for colleges, what happens when you talk to people who actually do this job and they might be director of quality in a college or they might be CEO. Uh when they sit down to have their review with someone that's produced a report, they tell me often that three-quarters of the time they spend is trying to work out what on earth you actually meant when you wrote down these things. What are you getting at? Because the writing's so bad, and describing data in words is like hard. And knowledge flow does all that for you, so um, they're loving it because you can now actually have the conversation you would hope to have rather than 45 minutes of like, what do you mean now? Really? No, why? Yeah, yeah.

SPEAKER_01 4:10

Did you know you you might not know this, but today I got information from Tech UK, and um, for those of you who don't know uh who Tech UK are, they're a uh uh an organization, um uh I think they're supported by the government to um uh promote uh technology in the UK. Uh and they are a very well-known and respected organization. And they've just published, yes, very respected, and uh they uh have just published a report on uh AI, practical transformation of AI uh in public services, and two, not one, but two of our case studies are in there, and one of them is the quality assistant for FE colleges. So uh a big shout out to our uh one uh audience. Uh if he's still awake, Matt, you did good. Well done, Matt.

SPEAKER_00 5:06

Wake up now. Jolted him awake, you've heard his name.

SPEAKER_01 5:12

What was that? So that was great. Uh so the the the that was one of the custodies. The other one, actually interesting. You mentioned housing associations, the other one was for TAF housing. So um really pleased for uh all of those guys, but also uh the fact that uh knowledge flow has been um uh touted as a um uh uh good solution by one of the most respected tech organizations in the UK. So that's delightful.

SPEAKER_00 5:39

That's amazing. That is cool. And I was with TAF uh on Monday. Are we? Uh with yeah, they're really good in that they've got like proper senior engagement. We get the CEO, uh the their director of technology, and um uh some of their other senior team around we get every month we meet them and talk to them about like what give them the feedback of how things are going and talk about the next bit. And they are really keen to push with us, which is brilliant because we've been trying to get this into housing. Two things a uh a gentic solution for tenant inquiries, so that is being able to immediately just letting knowledge flow answer the emails rather than having um humans do that, which is fine, and particularly when you triage first and you take all the easy stuff. You know, what time do you open? When do I drop the keys off? How do I fix a boiler? Or you know, all the stuff that's just really straightforward to answer, let it answer it, and it gets instant answers. So that's really cool and should be, I think, will give a better tenant experience and relieve a bit of pressure on the team that can then deal with the much more complex, challenging um areas where they have you know tenants with uh additional needs, put it that way, uh, so they can handle that, uh handle those and spend their time on those instead of the sort of more mundane stuff, which is great. And then the other thing is using our API um agentic version of knowledge flow with their repairs, and that being to be able to look at their repairs and and prioritize the triage um and um yeah, just be able to and update, you know, take the data directly from the system and be able to immediately flag up those things that are urgent, those things that haven't been done as they should be, um, and all of that. And then I was talking to another housing association who in a similar space, they spend uh apparently they've just changed their contract for reactive repairs, so that's fixing the stuff that's broken today rather than planned for. Um and they say they they've got a contract to between two and a half and three million uh they hope they expect to spend. They would like it to be two and a half, not three. But but trying to get kind of accuracy from contractors, it's a bit of a challenge because you can imagine some of the some of the reasons why that might be, but um they have um uh schedule of rates codes within housing that helps you set the price that things should be. Uh, those are a very lengthy tome of a document, uh, which we've got TAFAV, a version of that that runs uh automatically so they can use our AI to get answers. But where we're looking to go with this other one though is an experiment to say can we take previous actuals and use that alongside this scheduler rates codes to be able to be really quite precise and to say we think that leaking tap is £28.95 or you know, whatever. And so that'll be really fascinating to do a sort of live check of actual what jobs actually cost, and of course, using the AI's ability to read properly the job, and so it can do a close match of like this is a leaking tap in a third floor tower block, blah blah blah, um, and then uh being able to pick up the the costs from that or from many of those and give a really accurate view. So that'd be really interesting. Really excited.

SPEAKER_01 9:01

Did you um seeing as I've chided you on this podcast before, did you show them the um photograph?

SPEAKER_00 9:08

I did. Uh did you have to get right?

SPEAKER_01 9:10

Did you remember to do it this time?

SPEAKER_00 9:12

I did, and I even told the I told him because he was in the audience, that's how he found us. Oh, was it? He was in the audience for that session, and I said to him that I have been ridiculed by my team for not showing you probably the most impressive thing because exactly that, and as you uh we we were talking about, if the contractor says, Well, that three-meter piece of fence actually wasn't 100 quid, it costs 300 quid because we had to dig a bit of this and uh put a foundation in there. You could ask for a photograph and run that straight into knowledge flow, and probably firstly, in the diagnostics, it could maybe give you a view on that. Secondly, it could probably check and like give you a bit of a it doesn't look like that, son. Looks like you've just nailed that to the tree.

SPEAKER_01 9:55

You should take before and after pictures and store them so exactly.

SPEAKER_00 10:00

Spot the difference. You've just you've just stood that fence up, haven't you? That's not a new fence at all.

SPEAKER_01 10:07

Are we gonna be targeted by um fencing contractors whose uh income are gonna drop significantly due to knowledge flow?

SPEAKER_00 10:14

Possibly, but well that's what the this how this housing association was saying is they don't since COVID really, they've got into they don't check everything as much as they did. I'm sure they didn't check everything, and so the propensity to I'm trying to find the right polite words, but um yeah, get it wrong, uh, is is higher now because no one's usually for accidental errors. There we go, that's that's the right way of putting it. And then the last one there's the multi-academy trust. So our Matt client that I was with configuring their knowledge flow to do lots of amazing stuff. So they've got a pastoral support assistant which is there to help with all of those quite challenging matters. It's got all of their policies, best practices, various things about that, so that you can interact with that ahead of or during indeed a sort of pastoral discussion with a student or staff member. So that's quite neat. Um, and then they've got education data tools which help them they upload their data and it tells them what things they need to go and look at, which is brilliant.

SPEAKER_01 11:13

Yeah, I'm surprised we haven't done more in uh multi-academy trust, is the honest truth. It does uh I think we've talked about before, it's been uh a slow burn. Um uh I actually um uh uh I was a little bit naughty. I'll I'll I'll confess to this. I was a bit naughty and this week, and I have uh joined a competitors webinar and uh listened to them, and uh it was frankly awful. It was just yeah, yeah, well it was it was awful for a couple of reasons, but the main one was it was like really patronizing about it's like here's all the things that you uh should be doing in schools in FE. It's like if I if I was in one of those organizations, I'd have been really bloody cross, is the honest truth. It's like, thanks very much, Mr. Outsider, telling us what we what what our responsibilities are. I think we know better than you. And uh, can you just get on with it and show uh show us what you what you can do? And they and they actually didn't show their tool at all. And and I know for a fact when you get on a webinar, you can't you can't wait to uh open up knowledge flow. You're you're ridiculous for it. You're like, oh right, and here let me just show you this, it's brilliant. And so uh it was quite easy as well.

SPEAKER_00 12:25

I know you've got and I never did video, I don't even have a video backup because I'm that confident in our stuff. Nearly everyone I've ever seen has a video backup, or indeed are playing, don't even have a backup, they are only running the video.

SPEAKER_01 12:37

Well, not being funny, but this particular outfit, uh, they couldn't even get the video working on their uh not nor could they get the slides working for most of it. So it was uh it didn't it didn't inspire confidence by any stretch. So uh yeah, that was that was very humorous. I enjoyed it. Well, I'm glad with an hour I'll never get back.

SPEAKER_00 12:55

Well, there's too many people out there that are not, yeah. I mean I think our public sector ethos is important in everything that we do. It ultimately drives, doesn't it, what we're actually trying to do is make a difference. Uh not and that doesn't mean a difference to the bank account, it means a difference to outcomes for your customers, clients, citizens, whomever they are. So good. I'm glad other people are showing themselves up for the charlatans that they are.

SPEAKER_01 13:24

I was once told by somebody that a guru is just someone who can't spell charlatan. So uh calling gurus.

SPEAKER_00 13:31

Yeah, watch out what you wish for. So, what have you been up to this week, then, apart from joining dodgy webinars and um being bored?

SPEAKER_01 13:39

Dodgy webinars. Well, I have been it's been a busy old week. Um, a couple of things. One is um, I know that you love to do product of the week. Cue Jingle. Yeah. Uh but I want to sh, and then you always love the new shiny stuff, and you you always you're always moving on onto the next, onto the next, on the next. I'd like to have a shout out for one of our very first tools called BidWriter. So um in another part of the forest, I have to um I don't have to, but I do. I help uh uh Eric ICT write bids, and there's a bid out at the moment, and um it just to put it into perspective, it is a 500 million pound contract value. So it's a very big, very important uh piece of work. And uh there stage the stage one documentation came out, and um there are all the usual questions, but there are a series of um text boxes which comes to something like seven and a half thousand words. And I don't know the last time you tried to write seven and a half thousand words, it takes a long time. And if I'd have tried to do it, if uh uh if I'd tried to do the first draft, it probably would have taken me a week previously. And I did it in four and a half hours on Tuesday, and uh the uh it um it's probably only 80% there, but it's 80% there for a first draft that I can give to the rest of the team, and now they can take it and they can sprinkle in all of the relevant examples. Uh, because obviously it's it's pulling from previous case studies and previous bids, and then things have moved on, and we've got new uh and other customers that we could we could reference more more recent um so uh stuff that we could put in there. So I've given it to the team, but I I was given I was given a week to do it, and I was like, I can crack this out in an evening. So I just get a couple of them fake beers in and I'll just uh I'll I'll just knock it out, I'll just knock it out. And um it was it was really good. There were a few um as I say, a few little little foibles, but there always are, and and and AI doesn't replace the thinking, but AI gives you that first uh first set. So yeah, I was really I was really tough with it. Um I think the new uh version running on 5.2 give definitely give better answers than we've we've had previously. So um yeah, a shout out to BidWriter. So there you go.

SPEAKER_00 16:14

And just for our listener who may or may not know about how bid writer works uh in our world in Knowledge Flow and may be thinking, yeah, I use AI for bids as well. Um when you use AI for bids as uh if you use Chat GPT or any of them to answer a bid question, it will give the same answer that everyone else that's using AI is doing. And bid evaluators see it all the time, don't they? And they're saying I'm getting the same thing 15 times, and it's like marked down.

SPEAKER_01 16:43

Yeah, yeah, you just mark it down, don't you? It's kind of this is clearly generic stuff. So being able to um but I was I was doing I mean I've said this to you before, but just being able to kind of load it up, right? Read the read the evaluation, right? How many points would you give me for this answer? And it'll go, I'll give you I'll give you four for this, right? Well, stick in some better examples, stick in an example from this organization or that organization, or or add in some statistics or or whatever else. And you can ratchet it up, but um uh uh ratchet up the answers to to be to get even better because it it works iteratively, it's that kind of it. The first the first pass of it is is quite generic, but as with any any of these tools, the the better you prompt it, the better you question it, the better you challenge it, and say no, that's not good enough. Actually, that's not the right and that's not what I want to say. I want to put it like this or change the answer to this kind of thing. Yeah, but yeah, like I said, four four and a half hours it took me, and um, it would probably would have taken me best part of a week to get to that if I'd if I'd have started from scratch and uh from my from my experience with it, it's also four and a half hours of more enjoyable time.

SPEAKER_00 17:54

I mean, it's hard to say that writing a bit is enjoyable ever, but actually just you know trying to draft copy, unless you absolutely love to write, it is hard to try and write something crisply and you end up deleting the same sentence 13 times and then get rid of that whole paragraph anyway, five minutes later. And it's whereas that part goes, and you can focus on is this answering the question, which is obviously what you're much better to do than you are did I write that in a coherent way. Yeah. I think there's lots to be said for and doing it privately through knowledge flow so you're not sharing your data with the world and giving it to Chat GPT so that everyone else can use it in their next bid is uh pretty important, I would argue.

SPEAKER_01 18:32

So and making sure you don't miss stuff. It's um I remember the evaluators in the DFE used to tell us, you know, we can only we can only evaluate what's on the paper. That's that's our job is to whatever's in that box is all in if you've got 500 words, yeah. Anything over five five hundred and one after that we ignore. So yeah, uh so getting it right um and getting it tight is uh is an iterative process, but to get to the first point in uh you know the first day that we had the uh actual tender documentation just means that uh we can get on with another one. Interestingly enough, I've got another one came through this week for actually for for leading AI, which uh um I I looked at and I thought, oh crikey, look at all those questions. And I was thinking, actually, they're really quite easy. And now the with with the security thing that we set up last week, yeah, I was thinking, no, right, that's gonna be easy. That I'll I'll not I'll knock that out on Monday afternoon, it'll be brilliant. I'll I'll do that. I'll not even gonna worry about it.

SPEAKER_00 19:30

So uh well it's a and it's a tender for for an AI bid writer, isn't it? So you gotta you've got to respond with your AI bid writer.

SPEAKER_01 19:39

There's a question, there's a question there. Did you use AI in the writing of this bid? Of course I did, you idiots.

SPEAKER_00 19:46

Absolutely, thoroughly. In fact, I didn't even have a human look at it. Correct, yeah, yeah, yeah. I will have a human look at it. On your point about how you worked with knowledge flow to be able to get good answers. I read this week that McKinsey are now introducing into their recruitment interviewing process effectively AI tests. Basically, they want to know that you can prompt, they want to know that you can challenge, and they want to know that you can critically think alongside AI, which is brilliant to see. And you know, all these things we talked last time or a few times ago about the councils who have said uh who have reported social workers turning down jobs because they didn't have an AI tool that the social workers needed, and now we're starting to see it at the front end of blocking you from getting in in the first place. Yeah, this is the world. If you're not on the journey, then I don't know what it's good. AI is going to take jobs anyway, but this right now is people with who can understand AI are going to take your job and so get going or progress your learning.

SPEAKER_01 20:47

Can I just say that's how I got into the dog house last time? I'm just checking notice what in the room. Uh yeah, because that that was that was part of the conversation that got me at the dog house last time. Yes. You won't have a job. Oh dear. That's right. I won't go there again.

SPEAKER_00 21:06

No, best smart to it. I also I was doing some uh just asking myself about so um last week uh anthropic released uh Opus 4.7, their latest AI model. Um, and I was asking, what is the IQ of Opus 4.7? Because uh people do a sort of IQ read across for a whole bunch of um AI tools. So first the first answers that you get is well, there isn't one, um, which is uh interesting when I read around it. There's a there's a thing that they there's an AI score that is actually more akin to what AI can do, because AI can smash an IQ test. It's like it's pointless giving it to them, it do the whole thing get 100% instantly in seven seconds. So it's gonna outperform everybody. But it's not a genuine test, so there's other ways, and then you infer the IQ. But here's the thing Opus 4.6 has been has an IQ. Equivalent all over the internet, 133 IQ, and 4.7 is probably 140. An IQ of 140, and that is available to you to do whatever your bidding is. I mean, that is as not many people go beyond that. You're in the very top quarter or in the top percentile, probably, aren't you, of humans? So and interestingly, I heard on a very good podcast that I listened to, AI in Education, they were talking about a similar thing. It was saying um feedback from people, general public about AI, I tried it and it wasn't really that good. Uh was if you the problem is you're trying the free tools. And here's the thing, right? If you hired an accountant for 20 grand, one for 50 grand and one for 100 grand, you would expect very different outputs from them. And if you're paying, you're getting the 100 grand accountant, the Opus 4.7 with an IQ of 140. If you're not, you're on ChatGPT, whatever it might be, and you're getting the 20 grand equivalent and an IQ of about 100, maybe 90. So that is a very big difference. So ultimately, unless you actually have tried the real tools, you know, the kind of thing that's behind knowledge flow, then you're really not really you're not really playing with AI, really. So it's poor capability. You're just kind of seeing some real basic stuff. I thought it was really interesting, that kind of read across into a job role and different salary levels.

SPEAKER_01 23:25

No, I said really interesting um analogy, really, isn't it? That whole you kind of get what you pay for is an old adage, but the um the release of um uh new models, all all the LMMs had new releases last week, and lots of the um gossip on the wires is really about costs increasing and and not just ads starting to appear in order to generate revenue, but actually people starting to charge much bigger numbers for um those later models, and actually creating a two-tier system, actually it would be a three-tier system, wouldn't it? It'd be kind of free models, um uh silver and then gold or whatever. Those who can afford to pay, you know, you 200 you know, three 20 pounds a month, 200 pounds a month is kind of where the sort of standard stuff is right now, but um uh it's expected that those prices will increase. And um uh kind of linked to something we talked about last week, the the rise of methos and and um uh and how that's being very carefully handled, and and only 50 organizations uh in the world allowed to touch it at the moment. And um there was a uh a really interesting podcast on um The Economist about it, and it said that it noted that chat uh chat GPT, I think it was 5.3 cyber, uh was released three days after after Mythos. So it was kind of almost like they just stuck cyber on the end and said, Oh, we got something too, look at us. Which that wasn't the point of the podcast. The point of the podcast was does uh does the rise of Mythos and similar tools um help defenders or attackers when it comes to cybersecurity? And um, it was a it was a there was a long um uh conversation about uh pros and cons. It was talking about um cyber attacks, you know, two or three years ago it was all ransomware, today it's not. So if you look at something like the uh Jaguar Land Rover cyber attack, that was more of a destructive attack. It's not about we're stealing your data, it's we're gonna break things in your infrastructure and we're gonna keep punching that bruise until you pay us. So um that's the kind of um uh sort of destruction attacks that are being uh permeated these days. And obviously, what they're they're looking for are those vulnerabilities so they can get into your systems. Mythos really helps with that. Um, and the conversation was around uh who's got the advantage? Is it attackers or defenders? And and uh broadly speaking, they said, oh, it should help defenders, uh, you know, all things being equal, you know, defenders will be able to fix their problems before the attackers get in. But and it's a really big but it's only the kind of the bigger organizations that are going to be able to do that. Um uh smaller, mid-size organizations are gonna always be playing catch-up. So unless you're really um uh on top of it and understand it, then you're going to be extremely vulnerable and it's not going to be those, as I say, ransomware attacks, it's gonna be destructive attacks. So that's all really quite worrying. But there was um uh I think you've already mentioned interesting facts. So I'm gonna ask you when you thought the very first recorded technology cyber type attack was recorded in history.

SPEAKER_00 26:56

Uh were computers, I said technology. Oh, okay. Go on then. I did I'm not either come on. Was it a plow? We got someone put a screwdriver for a plow, and that was a cyber because of the stuff.

SPEAKER_01 27:11

It was it was a kid and he stuck a he stuck a stick through someone's wheel spokes and he fell off. No, no, no. It was so the first sort of techno the first commonly acknowledged uh technology attack was in 1834, when uh in France two chaps uh hacked into an inverted commas uh the telegraph wires in the in in France to get information on the wars so that they could um uh use market manipulation so they could um buy and sell contracts.

SPEAKER_00 27:45

Yeah, no way for market, not even for intelligence, for military intelligence, just like to make some money.

SPEAKER_01 27:51

Yeah, yeah, yeah. But isn't that true? I mean, isn't that why more cyber attacks happen these days to make money?

SPEAKER_00 27:56

Yeah, yeah.

SPEAKER_01 27:58

What's changed?

SPEAKER_00 28:00

Really funny. You're um yeah, I mean, I think that that whole kind of ransomware, the whole security, what is very evident is the the world is in a high-risk scenario from AI as it goes beyond Mythos's capabilities. Yeah. Who knows what you do to sort it out? I remember when I was, I did um, you might recall, I did some uh CIO round tables for VMware with their kind of best European customers or biggest European customers, and um there was a conversation there where they'd brought in their head of security who was ex-CIA, and he I like the phrase I still remember he said, he said there's two types of CIOs. There's those that know they've had a ransom uh know they've had a cyber attack, and those that and it's basically his point was everybody's got this stuff lying in their systems, yeah, uh ready to be alerted and off it goes and does its nasty things. Um in those days, I mean who knows now, but the the code is is proliferated around your systems and just lies there, you don't know. What they uh he advised about was if you find it, it is a really bad idea to delete one because they are all trained to look for each other, programmed, I should say, to look for each other and note if one's been erased to then unleash hell so that they you know ready to go, they've always been found, and before you can get rid of all of them. So he said actually the first thing you've got to do is get hold of your security professionals, partly trying to drum up VMware business. But um really interesting, I thought.

SPEAKER_01 29:32

Yeah, that whole cyber stuff is is uh is really is really tricky, and um the rise of um agentic and coding. I've it I I'll come back to the the the the the uh agentic in a second. I've done some coding this week for the first time in edges. I did it on clawed it, it's brilliant, really loved it. And I just like I'd stuck in I'd stick in a uh it's like ask it asking me these questions that I don't understand, Matt. I'll just take a screenshot and show all right I can see the problem now. Change the code to this one. It was just really, really clever.

SPEAKER_00 30:03

You've done coding. You actually mean you actually mean cut Claude's done some coding.

SPEAKER_01 30:08

I just I type I did some cut in and pasted, and you always said AI is all about cutting and pasting anyway. There you go, it's colouring in. But coming back to the coding issue, um, lots of uh uh stuff recently on the wires about uh agentic and um getting agents to write code and doing stuff for you, and um uh that's terrifying. And and um there was uh something on the wires this morning that you uh you saw as well, which was the whole um overnight a company had done something where they'd set something running and they got a hundred and fifty thousand dollar token bill from Google the next morning. And Google were like, That's what you burnt. It's like, oh my goodness.

SPEAKER_00 30:57

So we didn't mean to, we didn't mean to.

SPEAKER_01 31:00

That's right. Google, yeah, whatever, where's my money? So yeah, really, really careful about creating and setting things running without understanding. I mean, you yeah, I think you mentioned a couple of uh it wasn't last week, week before, we had a a similar thing with 190,000. For us, it was like $50, it was like it wasn't the end of the world, but it's kind of uh crikey, just terrifying the way it could rack up. And come back to the cost thing, you know, if we if you're using those expensive models, then you know it could be literally hundreds of thousands. So exactly, yeah, you you've got to think about um uh you've got to think about putting limits on on your token burn and uh what that means. But most people don't understand what a token is because it's really difficult to explain to the layers. And it, you know, it could be four characters, it could be one, it could be two, it could be part of a sentence or part of a word. And because it because of the way that works, it's really difficult for most people to understand what the potential costs are. Um so they've got to be really careful, I think.

SPEAKER_00 32:06

Yeah, definitely. And numbers are really so like the a token is sort of broadly the way I kind of estimate if I have to. There are ways you can do it online. You can go grab the tokenizer from OpenAI and or any of the models and see how how much a block of text will be in tokens. Um and it's interesting because it shows you like little coloured splits to show you what a token is in a word, so like strawberries four or whatever it is. And um, but numbers are really numbers, is it it burns truckloads of tokens when you start putting numbers at spreadsheets in because it's it's a very different, uh, very different thing. So, yeah, it's um knowing no one's and you can't keep up with that. That's really, really difficult, and no one's gonna know. But you can have a kill switch, which is what we are now adding to all of our it's a really big one that goes knowledge. And we we have currently um in in all of ours have a a budget trigger, so we'll get a warning if it's um above a certain threshold, so we can at least have a look and see. But ultimately, I mean this kind of thing that Google did in nine hours, it went up 150 grand. So, I mean, you can't that no warning's gonna probably help, you'll probably be 50 grand deep before you could stop it. That's right, yeah. So we're uh we're gonna build in some automatic kill switches uh at whatever threshold, we'll probably make it suitably high, I imagine, so clients don't get stopped unnecessarily. But um yeah, having something that does stop it if it gets fast like five grand or something crazy to switch off immediately. Yeah, are you sure?

SPEAKER_01 33:32

Um I I don't uh here's my uh this week here's this week's um uh plug for Ned B. John's.

SPEAKER_02 33:39

Oh yes, good, good. Hello, Gary. Hello Nate.

SPEAKER_01 33:44

Uh he was talking about these rising costs and token burns and things, and he said the real part of the real problem is that people have just got lazy. So they'll just put in a PDF doc and um it'll it'll obviously burn tokens while it sorts out the header and the footer and the pictures and the logo and everything else. And actually, if you stick in a markdown file, you can save kind of an eighth of the, you know, you can you can charge an eighth, eighth of the cost or whatever. And and people just got lazy by just loading up all of the documents that they think might be useful rather than just giving it a little bit of thought, saying, actually, I don't need that one, or uh I don't need that. Uh um and and how do you train people just to uh or even just do simply if you're gonna do a really big um I mean we've done thousands of documents, but you know, if you just turn those PDFs into uh markdown files or text files, it just serves a massive amount of of token burn. And actually, if it uh people he he described in one of his posts as um we've been drinking at the open bar and the bar's just about to close, but you want to carry on drinking, it's gonna cost you a fortune if you don't change your drinking habit. So uh that amused me given uh uh given my uh uh non-alcoholic beer consumption.

SPEAKER_00 35:03

Yeah, yeah. Well I'm on squash at the moment, so that's the good as well.

SPEAKER_01 35:07

What's your excuse for not having beer?

SPEAKER_00 35:09

I think I don't know, there isn't one. Um there is that interestingly, I was thinking about that same thing from Nate and how it remember software would always be written to be massively uh to uh minimize its storage uh impact. And then now over the last I don't know, probably 10 years, but certainly five years, no one cares anymore because it's yeah, largely it's so cheap. It used to be really expensive, then it went cheaper. It looks like it's getting more expensive again at the moment. But that's really interesting because it's I think we're in that it feels like we might be in that phase of AI of realizing you know we have been as to you to Nate drinking at the open bar and just realizing we there are better ways of doing it and and ways of being more efficient. Yeah, maybe that's the next wave for us to uh Rikey.

SPEAKER_01 35:53

Listen to you talking about Aldwith. Do you remember when we were back at Ford? We're back in the in the last century, Kieran. We were just and there was a a really gnarly old uh guy in the IT department, he was lovely, but he said to me, He said, Oh yeah, I've seen it all. We uh first we we centralized, then we decentralized, then we decentralized now. We're going to centralize again. It just goes around in waves. It takes about 10 years, but it'll be another one. I'll be I'll be well and gone and retired by then. But you'll see, you'll see these come. And he's absolutely he was absolutely right, hilarious.

SPEAKER_00 36:28

Yeah, the old change resistors. I remember that lots of lots of meeting, lots of them. I guess that's one of the things that does get a bit hard work. You you know, as you go on in your career, you're kind of wiser at lots of things, less tolerant of a lot of things, uh, but yeah, I think also a little bit more jaded about some of the stuff that probably can be made to work with the right effort, but ultimately it's easier just to go, oh no, saw that before. I just yeah, interesting, interesting world of whether whether we need more people to just go with it or whether you actually just save yourself the bother and use a bit of wisdom and not bother with that. That's right. I um what was I gonna talk to you about? Oh, yes, AI ambiguity. I often talk, I was talking to the um the uh AI champions of the university that we're working with today, and I was saying how in prompting, one of the things that I have kind of discovered is that we are all as humans really bad at being clear in our writing and not leaving too much ambiguity. Um, and if you leave ambiguity with an AI prompt, it will fill it in. And 90% of the time it'll be bang on correct and wonderful, but 10% of the time it's going to guess incorrectly. And that led me to another thing I saw this week which amused me enormously about words, English words. Um, so uh teaching our six-year-old phonics and how to write how to uh read is quite interesting reflection on how many weird rules we have in the English language and oddities. We seem to have fewer consistencies than than we have all these outliers. But here's an interesting thing which does strike in AI ambiguity is the word sanction has two meanings and they're opposite each other, and that is just bananas for one word. Absolutely crazy. One word that means the opposite of each other if you it's so bananas, and then I was very amused to read that the word cue, you are cueing for the for the shops or whatever, is one letter doing all the heavy lifting and four silent letters all queuing up after it. But it it is ridiculous. The written language of English is so bananas. You try and explain it all anyway. There you go. That was my uh amusing part this week. And I also discovered that uh talking to Louis. So Louis, my 17-year-old son, is doing some work experience for us at the moment, which is great to have him uh experiencing a bit of what it's like in the world of work and uh getting his head further into AI, which is great. But uh he was talking about let's get this having aura, and he to and he he said, uh, my my mates said you've got aura, strong aura. And apparently that's to be cool in the modern nomenclature. So I take that to be good. Uh you know, insomuch as I care what 17-year-old boys think of me, I'll be able to weird. But it was yeah, apparently you need to have aura. So there you go, Neil. You need you need a bit more aura in your life.

SPEAKER_01 39:28

Uh well, clearly, uh I I don't know how to do that, Kieran. I've got me neither. Apparently, you've got it. Well I hear yeah, yeah, yeah. Look at you. All right.

SPEAKER_00 39:39

Um we need to get some.

SPEAKER_01 39:41

Right. Well, on that note, I um I need to go and uh I need to go and pick up my dog. I've left him. Um so it'd been such a busy day. I outsourced dog walking to my octogenarian father-in-law. He's 85 years old. He'll have had the dog up the hill twice and frisbeed him for an hour. So the dog always comes back absolutely shattered, and I'm worried the old boy's gonna break, break my dog. So I didn't need to go and rescue him before he takes him for another bloody walk. So uh there's hope for me. Yeah, I hope I'm as fit as he is when I'm 85 or he's 86. I've got him. Yeah.

SPEAKER_00 40:16

That's very cool. Well, I I wanted to uh share one more amusing anecdote before we before we before we let our listener go to sleep. I was watching, uh, it was on I think it was Sky News, and they had some somebody, some pundit about politics, his like ex-advisor and all that, you know, the usual kind of guy on there, and they were talking about the current conundrum of uh the current government in the UK and all the nonsense that they're uh creating for themselves, it seems to me. Um, but he said, he said, you know what, you've seen that show, The Thick of It. He said, The only the only difference between the thick of it and government is that people on the thick of it don't walk around going, it's like the thick of it, isn't it? I thought that was a fine statement.

SPEAKER_01 41:00

There's been a lot on um my LinkedIn feed of uh little clips from Yes Minster and Yes Prime Minister. And I remember back in the day we used to use those clips in in government to to train people on program and project management, and that one about yeah, how many people how many people are we gonna save? Well, we need another 3,000 minister because uh we can't stop doing anything and uh we need to do extras. So uh yeah, so hilarious, really, really funny. Some some things just never change, Kim, despite all the technology, people will always do dumb stuff. There's there's one thing you can you can rely on in the world today.

SPEAKER_00 41:37

Well, that on that happy, confident thought that can send us all off to off to sleep. So uh put plump up your pillow audience. Yeah, that's right. Night night, Matt. Night night.

SPEAKER_01 41:53

All right, so good evening.

SPEAKER_00 41:54

Catch you there, then you enjoy getting your dog. All right.