AI, trust, misinformation and OSINT with Al Baker

In this episode, Al Baker, co-founder of Prose Intelligence, discusses the latest developments in AI, especially agentic AI and OpenClaw, and their implications for OSINT and security. He explores the limitations of current AI models, the risks of misinformation, and the importance of rigorous research practices in intelligence work.

Transcript

SPEAKER_01 0:01

Hello and welcome to From the Source. This is the podcast from Black Dot Solutions. I'm your host, Matthew Stiver, and today I'm very lucky to have Al Baker, who is co-founder at Pros Intelligence. Great to have you on the show, Al.

SPEAKER_00 0:15

Thank you very much for having me.

SPEAKER_01 0:17

So I want to start with a question I ask all my guests. I'm a fully paid up uh geek. So I'm going to ask you, what are you geeking out about at the moment?

SPEAKER_00 0:28

So I like a huge part of the uh the internet people who are terminally online got very, very excited a few weeks ago when uh what is now called Open Claw got released. So I think probably the thing that I've been thinking that's most relevant to work that I've been most excited about over the last few weeks and doing the most kind of thinking about is what the genuine possibilities are for agentic AI as it currently exists in OSIMP work and in tech in the context of tech startups in general.

SPEAKER_01 1:01

So for um the uninitiated, and I'm very partially semi-hemi-demi-initiated into this, what is OpenClaw?

SPEAKER_00 1:10

So OpenClaw is uh a piece of open source software that appeared uh seemingly from from nowhere a few weeks ago, and it in itself isn't particularly uh I mean it's weird to say it's not innovative because nothing like it has existed before, but all it is really is a way of plugging LLMs into one another and into the internet that gives them capabilities that they haven't had before. So OpenClaw allows an LLM to directly execute commands on your computer, it allows an LLM to directly uh access the internet, anything that a human can do on uh on a computer, open an LLM can do on a computer via OpenClaw. Uh one consequence of this is that there are enormous security vulnerabilities attached to this software, it's a very dangerous piece of software to use recklessly. Uh, but the count the counterpart to that it's also potentially tremendously powerful. Um it means that uh yeah, an LLM can do anything on your computer that you can, potentially speeding up enormous amounts of tasks, doing huge huge quantities of like admin, boring stuff, loads of the stuff that as um you know one of one of a small number of people in uh a tech startup, it would be hugely valuable for me to have uh a robot do for me. The experience has been interesting, fascinating, kind of frustrating in equal measure. Uh the headline is that that there is too much hype, that the the robots are not here to take your boring admin away just yet. But I it genuinely is a matter of uh when and not if. Like the plot the fact that the plumbing is there now and the fact that it works in principle is is genuinely very exciting.

SPEAKER_01 2:55

So if you think about the history of LLMs, and like um chat GPT sort of 1.0 was really idiotic and not very helpful in the way. But now we're on a version that actually seems where where is this agentic AI evolution? Are we on is this version one or are we on version two, three, five point two?

SPEAKER_00 3:16

So I'm actually despite how kind of exciting I find open claw, I'm pretty skeptical of uh the the my prognosis for AI isn't isn't isn't a terrible that of a terribly smooth rise. I think agentic AI is going to be hard limited by the hard limitations of large language models. People, you know, this is it's it's good that people uh are reminded of this every now and again. LLMs are not the only kind of AI, they're one kind of AI, and they're not what you would think would be the most natural kind of AI to deploy in an agentic kind of administrative or ops role. Um, they're a text predictor.

SPEAKER_01 3:59

Um, exactly.

SPEAKER_00 4:03

There's an it's a it's it's a very, very sophisticated autocorrect, and that can do a lot of things, especially in the context of like admin and kind of boring um administrative work where mediocrity is kind of what you're going for. If the if the aim of what you're trying to output is the most normal, mediocre kind of kind of if you don't want to take any risks whatsoever, then an LLM is exactly the way to do it. But as soon as you want to start being remotely creative, like they're not they're not good at it, um, and there's no reason to think they're going to be able to get better at it. Similarly, LLMs aren't agentic. If like LLMs cannot be agentic because they're essentially predictive. Um like if you're if if your raison d'artra is to predict the next token in a sentence, then by definition your your being is not about making decisions or creating anything new. So that puts a hard limit to my mind on what we can expect AI to be able to do, but that still leaves an awful lot of room for it to be useful in the short and medium term.

SPEAKER_01 5:10

I'm I've been dabbling not with OpenClaw but um the agents in Notion, and my business is almost entirely run in Notion, and that has some cleverness. You know, you can ask it to kind of go and get something and then turn it into a table, and it kind of can do some agentic things inside Notion, which are helpful. I think the hard limit on that probably is they want to charge an enormous amount of money for it, and it's on a it's on an unmetered, you know, you pay per thing, so suddenly you get a thousand pound bill. I remember years ago when I was a student going on the pre-deset precursor of the internet where packets switching networks, and I got a bill after a month from BT of like 1,100 pounds, and I'm not doing that anymore, just because if you've got this unlimited billing. So I think billing is going to be an issue for this. Um but you this may you you have a background in philosophy and it may or may not be in your orbit. But back when back when I was a student, I remember reading Marvin Minsky's book, The Society of Mind, and his idea of consciousness was that it was a series of, let's say, by analogy, agents doing different things and sort of having communication, we would say APIs at the moment today. And I wonder whether LLMs might be one element of a society of artificial intelligence mind, but there might be other AI tools that will be talking to them and engaging with the world in different ways. World models is the phrase that's often talked about. Um, and you won't have an AI, you'll have a constellation or a family or a community of them doing your stuff.

SPEAKER_00 6:46

So that's I mean, that's an interesting that that's an interesting kind of set of questions to start to start digging into, and is immediately brought up by the idea of we've all very quickly got very high pollutant here. But when we're talking about agentic AI, it's very it's very important to think about what we mean by by agency in this. And a network of complex non-ogenic objects doesn't necessarily create agency. Agency doesn't necessarily emerge from an interacting network of non-agentic objects. Uh, so like what's a what's a what's a really good example? I mean any kind of computer network, right? If I'm playing um if I'm playing an online game with a bunch of other people, these are non-agentic devices interacting and creating a game. But there's nothing uh agency doesn't uh exist in the game, it exists in the people playing the game. So I think there's a little bit more work to do, unless you want to radically redefine what agency is, there's a little more work to do before we can properly uh say that AI, as we understand them now, are agentic because ultimately they can't um they can't make decisions. One of my favorite, one of my favorite um, what's it say favorite, one of I think the most important questions to ask about AI is whether we can trust it and how we can trust it. Um and this brings us maybe more neatly into uh OSINT-related territory. Because one of the things, one of the the real promises of of AI in uh open source intelligence, threat intelligence generally, is that it can do at least some of the work of um analysts. So we can take a huge amount of data and it can find the insight in the uh in the noise. Now, there is no reason to think that an LLM can't do that. An LLM absolutely can do some of that to some degree. There is even reason to think that future LLMs will be able to do it reliably as well or better than human analysts, human expert analysts. But the missing piece, and what I think is often overlooked, the crucial missing piece is um is is uh trust here. And it's not possible to trust an AI in the way that it's possible to trust. It's certainly not possible to trust an LLM in the way that it's possible to trust uh a human analyst, and that's because an LLM essentially has no skin in the game. So if you you get in a fight with an LLM, right, an LLM dramatically misunderstands you frequently happens to me, it wastes hours of my time because it's gone off on a wrong thing or it's not saved something, or the context is compacted and it's forgotten what it was talking about, and I wasn't paying close enough attention, so I don't have the right notes. So, you know, waste a huge amount of my time. And my reaction then is I get frustrated at this machine. But that is completely pointless. It's about as useful as me getting annoyed at my alarm clock if I forgot to plug it in. Right? The alarm clock doesn't have any agency in in not waking me up. This LLM doesn't have any agency in frustrating my ambitions for the for this project. Um, if I was working with a co-developer, with a human programmer, absolutely if I I can I can be uh I can be disappointed, right? My trust can be frustrated. I can say, I trusted you to do this and you let me down, and I cannot do that with an LLM. And like a tech bros response to this is gonna be let's just make it so we'll just make it better, we'll make it so it doesn't make the mistakes. But I really think that crucially overlooks the fact that we need as much as uh a big part of agency is responsibility for that agency. And if we can't hold an AI meaningfully accountable for its decisions, then there's a real sense in which it hasn't made any decision at all, or like in a meaningful sense, hasn't made the decision at all.

SPEAKER_01 10:45

So that in your area of expertise uh around misinformation, there's a I think uh a risk, isn't there, or a concern that if there's a new technology and you can use it for porn, making money or state action people will do it, right? Um and if if it it i I I I guess what's your feeling about people using AI to generate misinformation?

SPEAKER_00 11:19

It it it is it is So to be to be slightly clear, but I think if there is a technology which can't be used for porn, for making money or for state repression, then that technology will not be successful. Like that is ultimately what all technologies that last are like able to do, even that that's not their own their only or kind of um or main purpose. So the question should one of the huge mistakes I think we made in in responding to social media as as a civil society is that we we managed to to pretend for a long time that it wouldn't be a tool of uh like rampant greed, political interest, um uh you know, pornography, vice, illegality, whatever. Because of course of course it will. This is what um successful societally embedded technologies are are put to. And the fact that we took social media companies at their word when they said, like, no, that's not what these are for, it's not gonna be a problem, it's fine. Like, that's absolute that's absolutely honors. So the the fact that we're making the same mistakes again with AI is is depressing to me in the extreme. So, yes, AI will be put to all these uses. We should be in front of it, we should be uh we should be regulating in advance of it. We're not really, that's our problem.

SPEAKER_01 12:46

No, there's really not any sense of that. And though I I'm old enough to remember the early days of the internet and and the the idealism that existed around it. Um the internet will root around censorship, was a phrase that comes back to me. And there were lots of people who profoundly believed that, and maybe it does. Um and Silicon Valley has a different reputation now, I think. We talk about tech bros and we talk about you know the the big big companies that and I I'm so let's come back to so I'm I'm going off at a slight tangent out of nostalgia mainly. Um tell me about prose intelligence. Let's let's anchor this back in your work at uh in OSINT.

SPEAKER_00 13:30

Uh so prose uh was founded a couple of years ago by me and my friend uh Jordan, based on tech that Jordan had developed during his career as a uh uh a journalist and then later as a disinformation researcher for archiving data off Telegram. Uh the the the tool he developed is still there, it's called uh telepathy, still open source, still free, still reasonably well supported, although it's probably due an update. And it's uh uh basically we got to a couple of years ago and we realized that no one had come up with a better tool so far of um archiving telegram data, specifically for intelligence use cases and for threat intelligence platforms and that kind of thing. So that seemed to us like uh a gap in the market. So we set up pros specific with a couple of different aims in mind. The things that we thought we could do better in terms of providing data for intelligence, for open source intelligence, for um uh you know government research, that kind of thing. The stuff that we thought that we could do better than than the rest of the market, were firstly in taking the needs of OSINT analysts seriously. A lot of data sources and a lot of threat intelligence platforms are marketing platforms that have been repurposed. Um, and we wanted to make sure that the needs of OSINT analysts, which are you know quantitatively different to the needs of marketing analysts or business analysts, which is where you know one of one of the struggles if you're a a data provider is you have to sell the data where where the market is. And you know, we're a tiny company, so for us, the addressable market in the national security space is enormous. But if you're a huge company, then national security is a drop in the ocean for you, and you're much more interested in servicing like these huge enterprise sectors of uh you know risk analysis, business analysis, you know, economics, stock market movements, all of that kind of stuff. And you know, that's good, that's fine, that but that's not what I love, and it's not what I think the important work is. So we wanted to make sure that we were building capacity to support the next generation of um open source intelligence for big data. So we we make sure that we are looking at the right data for open source intelligence use cases, we make sure that the data that we're providing is uh compliant, is rich enough to facilitate all of the fancy analytics tools that these platforms are building. Like there's no point in having a we uh you were kind enough to say we're not obligated to mention black dot, but I will mention Black Dot. They have a special I know they they specialize in network mapping, which is an incredibly powerful um capability for for OSINT teams that I really wish was exploited more. One of the reasons it's not exploited more is that the raw data which is provided into these platforms isn't rich enough to support it. So um really the the the thing that I think is very important, there's all these fantastic analytics capabilities that people are building, but kind of the missing piece is data that is reliable enough, um, rich enough, and like is sourced responsibly enough that the users of these platforms can use it safely and uh and securely.

SPEAKER_01 16:46

And and that's what you're doing with Telegram data?

SPEAKER_00 16:50

Yes, so uh pros, we specialize well, uh Telegram is our flagship data source. If you like, we've now expanded into a couple of other uh fringe social media platforms, um some custom uh scraping uh and so on. Um we try and get to the platform that's going to be kind of essential for intelligence requirements in a couple of years. So right now we're looking at some smaller platforms that have recently uh cropped up um and some not some platforms in non-Western uh context.

SPEAKER_01 17:25

Um and your area of interest, uh uh uh perhaps intellectual and professional interest is misinformation. So help me as a an outsider understand what you mean by misinformation.

SPEAKER_00 17:40

That's an excellent question. Because I don't know exactly what I think misinformation is, uh, but I do know that um which is which is maybe kind of odd for for someone who's who's been working in the field for like five years. But it's certainly not what I know that what I think misinformation is isn't what most people who work in the field think misinformation is.

SPEAKER_01 18:01

Yeah.

SPEAKER_00 18:01

Um and this is probably due to, as you said, my my professional training as an academic philosopher where I I was um before I I came into the open source intelligence space. But when I came into the OSINT space, it was specifically in the misinformation field. I was uh hired um out of my postdoctor logically, where I started as their head of fact-checking. They needed someone to come in and take over and try and uh position uh logically as a third-party fact-checker to Meta and other social media platforms when that was still a thing that social media platforms were uh prepared to pay for. Um so there is an interesting there's a kind of obvious link there between philosophy and fact-checking as such. Well, maybe not obvious, like it's not journalism. I'd done some um you know ed editorial work uh for academic journals, but it's you know, it's not really journalism. But what I did have from my training as a philosopher was a really solid grounding in um argumentation, logic, uh, and epistemology. So I had a better idea than most of what could be reasonably said about things being true, false, misleading, deceptive, what have you, um, and what were good and bad ways of arguing for the truth, falsehood, or otherwise of of a particular claim. Um, and that was extremely helpful um and allowed me to bring I think a degree of clarity to the job which still seems to elude quite a lot of people who uh work working in the space, but a lot of organizations working in the space who are I think still still very um I think a lot of people who are interested in misinformation professionally are kind of deluded about how simple the uh the the issue is, or rather how complex the issue is. And this can be illustrated by the you asked me about definitions earlier. The standard definition of misinformation that you that you hear from the field uh is well, you'll hear it in this way. You'll hear that we're interested in misinformation and disinformation, and that the difference between the two is that misinformation is a falsehood spread recklessly, so spread without necessarily meaning to spread it, and disinformation is falsehood spread intentionally. Of the top now, when you hear that, that seems like quite a reasonable distinction until you start to use it practically as someone working at the front line trying to understand how misinformation operates on the internet. Because what your unit of analysis there is a social media post. So I see a social media post and I say, okay, this is saying that like the moon is made of cheese. I know that that's false, right? So I want to decide our top line framework says like what our most the most important thing we need to decide is is this misinformation or disinformation. And this isn't like the vanity of civil uh civil organization companies either. This goes right to the heart of uh that government uh capabilities in understanding and responding to this issue. Because if something is disinformation, then it's plausibly a matter for national security. If a state-backed actor is deliberately spreading misinf is spreading falsehoods with a view to creating disruption, then that's plausibly a matter for national security. If it's misinformation, then it's not necessarily very arguably not a matter for national security.

SPEAKER_01 21:44

Aren't there where facilitating or supporting or even just sort of bystanding on misinformation can actually be very useful for people. There's still intent behind it. Like if you if you can if you if people are spreading rumours because you've exerted Reflexive control on them to see the world in a certain way, you didn't you might not have put that disinformation into the field, but you've created an environment where misinformation supports your objectives.

SPEAKER_00 22:11

Exactly. And so and so you might want to say, okay, so something that is so if a uh Russian outlet uh posted first the the the claim that the mood is made of cheese, then that counts as disinformation. But then Joe Bloggs from South End retweets it. He doesn't mean to spread something that's knowingly false. So is his retweet misinformation, or is it still disinformation because the ultimate source is the person who spread it deliberately? So this is what I mean. It's not necessarily that that we don't want to care about intent, but that if your pro the primary framework through which you're understanding the issue is this uh uh is this dichotomy that for any given social media post we won't be able to tell which bucket it falls into because we have no idea of intent and we don't know whether the intent transfers between different digital objects. And basically it's clear that we haven't actually thought about how we're gonna deploy this in the in the the real work of researching misinformation. It it uh it is just useless. And no one has done the serious work, don't want to say no one, there is very good work that is being done in academia in responding to precisely these issues. Uh, and almost everyone you talk to in the industry will tell you that like we don't really have a clear idea of the phenomenon yet or what it what it does or how to respond to it. And given that it's been kind of close to the top of the political agenda for exactly 10 years at this point, that's a pretty sorry state of affairs. Um but the uh yeah, so having having worked having worked in the field for 10 years, I I my my big hot take is I still don't think anyone really understands what the phenomenon is that they're trying to address, much less how to address it.

SPEAKER_01 24:00

And the the If you haven't even if we can't, I'm not saying you, but if if as an industry you can't even get to a place where the definitions or the understandings are, it it shows either a lot of flux and change, I suppose.

SPEAKER_00 24:15

Um I I think it shows a lack of uh lack of resolve, a lack of will to take an honest look at where we are and to uh deal with the difficult questions that arise as a result of it. So because the just to take a really obvious example, if we are talking about Russian propaganda, and we're gonna talk about why that's bad, then we also need to talk about the ways in which uh Britain and our allies also make use of propaganda and the way in which that's better. And instead, what we do is we pretend that propaganda is only a tool used by yeah, it's uh and and that's and that that firstly is like takes us all for idiots, and I don't like being taken for an idiot, and secondly, it makes it much easier for um uh for adversaries like uh adversarial actors like Russia to point to us and call us hypocrites because obviously we do make use of propaganda and state messaging, but also obviously the ways in which we make use of propaganda and state messaging are better than the one than the ways that um that than the ways that the Russia do. We we do it like more transparently, with some degree of accountability and with some reference to facts. All of you know, all of this is good and so it's something we should be proud of. Um and similarly, we talk briefly about the I I made a glib comment earlier about the uh in the pre-discussion about the question of where like marketing ends and um and misinformation begins.

SPEAKER_01 25:52

Because of course I'm in marketing and I'm therefore interested in truth, beauty, and justice, right?

SPEAKER_00 25:57

Of course. Um and the qu the question there is a very real one. While people were trying to figure out how to make money out of countermisinformation uh research and technology, one of the obvious places they tried to go to was reputation management for like big corporations, right? Because you can very, very plausibly say uh, you know, if someone's telling lies about Amazon or Coca-Cola or whatever, it's legitimate for Amazon or Coca-Cola or whoever to know about that, to have a you know way of responding to that, and to uh you know, to protect themselves, their shareholders, their business, or all the rest of it. However, as soon as you get the capabilities related to misinformation into the hands of a corporate actor like Coca-Cola, reputation management becomes the use case, not truth management. Right? Coca-Cola, I hope this isn't a libelist thing to say, Coca-Cola does not care about what's true, Coca-Cola cares about what's going to sell Coca-Cola, and more power to them. They are not in the business of uh uh of um improving civic discourse, nor should they be. Uh, but but for that reason it's a mistake to try and think that Coca-Cola or or you know any literally any corporate entity actually has uh uh a business interest in in what's true as such.

SPEAKER_01 27:23

There's a uh I read a book and I'm desperately trying to remember what it was called, but it was um talking about systems thinking and um how systems are perfectly optimized to deliver the results that they get. Um and you know, e you don't necessarily have to impute um wicked intent to corporations for corporations to do wicked things, they might just be getting there on their own, and I don't mean any particular corporation here if anyone's listening, but it it it this brings me back to thinking about we would start off talking about AI and the challenges and opportunities there. Does does the use of technology, the use of AI, particularly empower people who wish to spread disinformation? Does it give them a kind of a quantity or a volume or a quality advantage that that didn't exist before?

SPEAKER_00 28:26

It does, but it also provides the same advantages to uh people who want to improve SOD discourse. I think that the question of empowerment is interesting, but probably isn't central. The the the more interesting question is the systematic effect of these technologies on the if we talk about misinformation on the information environment as such. So the fact that um when when so during the the the rise of social media, the reason that that came along with the rise of misinformation with what what you might call the information crisis is less to do with the fact that people who wanted to tell lies now had an effective tool with which they could tell lies, and more to do with the fact that social media became the way that everybody communiced communicated with everybody else about anything for any reason, and the ways in which social media is designed makes it more likely that using it for those reasons results in poor information quality. So the the issue is not there were all of these people waiting for their opportunity to spread uh misinformative propaganda, and then suddenly you know social media arrives or AI arrives, and now I have this tool that I can do it. The issue is that we had an information environment with one huge set of problems, uh, you know, uh old um uh the old bastions of the media uh uh of the old media establishment, you know, the New York Times, the Sunday Times, the Guardian, or whatever, uh the the Mail, Radio 4, they owned the news agenda, right? They owned uh they got to define what was important, what was worth talking about, what the middle position was, hugely anti-democratic, hugely problematic. And then we replaced that information environment with uh social media, and it solved a lot of those problems, but it also uh creates a whole bunch of new ones because you get rid of all of the institutional safeguards, all of the stuff that made the old media houses gay keepies.

SPEAKER_01 30:36

You you you worked as a journalist and a fact-checker, and I've you worked as a journalist and you know met fact-checker, was fact-checked to the nth degree by Wired editors and people like that.

SPEAKER_00 30:46

Yeah.

SPEAKER_01 30:47

And I I it's an interesting observation you make. You know, they they had it the anti-democratical, the can the can the kind of monopoly control of information in legacy mainstream media, if we can use that phrase. But you understood how it worked, and uh it had some process and some methodology and some accountability, and and I think you're implying that social media as it evolved takes that away.

SPEAKER_00 31:12

It's to some extent, but it also just works differently, right? Right, it's just a day, it's just a different technology to put to print TV or the internet, and so just at the most basic level, you would expect it to work differently and expect it to have different results uh or different impacts, and one of those different impacts is poor information quality. So just to bring it back to your initial question, which was about like the impact of technology, there is a huge impact of technology, but it's less, in my view, less about empowering actors and more about changing the environment that we're operating in.

SPEAKER_01 31:42

So it we're in this epistemological crisis.

SPEAKER_00 31:49

Excellent use, excellent use of the lingo. Almost stumbled over it.

SPEAKER_01 31:56

What does OSINT have to offer to against that? What what's the prognosis? What do we do about it?

SPEAKER_00 32:07

So OSINT is OSINT has a huge amount to offer. It is the way in which the work that is done that will resolve this crisis will be done. Like OSINT describes the series of methodologies that must be employed in order to understand the misinformation problem as it exists and in order to you know analyse relevant information, monitor and respond to it. Um basically looking stuff up on the internet, exploiting information on the internet to discover how information moves around it, what actors are like involved in it, where they are, who they are, all of that kind of stuff. It's vitally important. It's the raw um it's a raw intellectual work of understanding uh of like the the information environment um online. The uh but it's not the only not only is it not is it not the only piece, it's also kind of in a lot of ways has outpaced and outdeveloped OSINT as a practice and as a community of practitioners, has radically outpaced the institutions which it should be serving, which it should be working for, has also radically outpaced the kind of conceptual frameworks that we're trying to use to understand the problems that it's trying to solve. So a roomful of OSINT analysts can do amazing things, even with no technology, literally give a room full of OSINT analysts who've been working for a couple of years a laptop each and a web browser, and they will do incredible, they will solve incredible problems for you. But you've got to define the problems that you want them to solve, right? And you've got to give them the the frameworks to to provide the solution. So if you tell an OSINT analyst, I want misinformation related to the the conflict in Iran, and you don't clearly define what it is that you're looking for, they're not gonna give you useful output, and that's not their fault, and it's also gonna dramatically you limit how useful any intelligence product is. Garbage in, garbage out, you know.

SPEAKER_01 34:14

I I I feel like we could talk all day, and I'm enjoying this so much, but I think we've probably got time. We've run out of time already question. If you are if you are going to give advice to the commissioners, the consumers, the per people who are asking the questions of a room full of OSINT analysts advice about how to use that capability. What what is the most important thing that you need to what would you tell them?

SPEAKER_00 34:53

I give OSINT analysts research questions, not methodological direction. I mean this is just this is just good research management practice, but I have seen I have seen like more of it in in OSINT than than I would like to. Uh it's just bad research, um, it's bad research ethics, bad research practice to tell your people what you expect them to deliver. Tell them the question that you expect them to answer, and then they that it's their job to come up with the the methodology. Also, uh OSINT practitioners come from the good old Wild West of the internet and of open source software and of like piratical, punky investigations. This is wonderful, and it has led to the community being as uh interesting, welcoming, and useful as it is. However, that way of working doesn't lead to a lot of documentation. And OSINT analysts, I think, need to understand, as a community, the far better the importance of um documenting your methodology before you do the research, uh, outlining your research questions before you do the research, being clear about when research questions that you embark on don't turn up any results, be just as noisy about your failures as your successes, you know, all of these kind of good practices from academic research, which just haven't made their way, and other kind of you know, serious um uh EGO research and so on, which just haven't really made their way into uh the OSINT world, but should, and not just because I'm kind of a hoary old academic who thinks that everything should be done in the academic way, but because if you're a government buyer of OSINT services, it is much easier for you to spend money, or if you're a private, even a private sector, but especially if you're a public sector buyer of open service uh of open source intelligence services, much easier for you to spend money if you know that if asked, your supplier will produce a document that says this is the methodology we use, this is who we got to review the methodology for its soundness, this is why it's a sound methodology. You know, basically an audit trail, a paper trail that says the results that you got are um you know predictable, repeatable, uh, and uh you know based on sound methodological practice. We'll never be able to have a peer-reviewed research system in OSINT, probably, but that doesn't mean that we shouldn't try and emulate uh good research practices where they exist.

SPEAKER_01 37:30

That feels like an incredibly powerful insight for us to close on. Um, and uh Al, it's been a delight and a uh um intellectual workout conversation. I'm so grateful. I've enjoyed it very much. Thank you so much for joining me today.

SPEAKER_00 37:48

Thank you very much for letting me hear myself speak and never get bored of it. And thank you so much for talking to Pete.

SPEAKER_01 37:54

And that brings this episode to a close. And if you'd like to learn more about OSINT, Videris or Black Dot Solutions, please visit blackdotsolutions.com. Um, and thank you very much for listening. Uh, goodbye.