What is the state of application security? JB Aviat answered that question, by creating the state of application security report based on data from Datadog customers using the application security and APM products. It provides insights into threat detection, vulnerability detection, prioritization, and general trends on where the most significant risks lie.
We discuss:
FOLLOW OUR SOCIAL MEDIA:
➜Twitter: @AppSecPodcast
➜LinkedIn: The Application Security Podcast
➜YouTube: https://www.youtube.com/@ApplicationSecurityPodcast
Thanks for Listening!
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
What is the state of application security? JB Aviat answered that question, by creating the state of application security report based on data from Datadog customers using the application security and APM products. It provides insights into threat detection, vulnerability detection, prioritization, and general trends on where the most significant risks lie.
We discuss:
FOLLOW OUR SOCIAL MEDIA:
➜Twitter: @AppSecPodcast
➜LinkedIn: The Application Security Podcast
➜YouTube: https://www.youtube.com/@ApplicationSecurityPodcast
Thanks for Listening!
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Jean dedicates to democratizing security by developing the security products of Datadog, leveraging the unique observability capabilities of the platform. Before this, JB was CTO and co-founder of Screen a cybersecurity company providing an automated SaaS solution to protect web applications without traffic redirection or code modification. Screen was acquired by Datadog in April, 2021. JB joins us for his second visit to the podcast. This time he's gonna walk us through the state of application security report that he has created via Datadog, taken data from various customer environments. And anonymized it and given us a view into some very vital statistics for the world of application security. So we hope you enjoy this conversation with JB a. Hey folks. Welcome to another episode of the Application Security podcast. This is Chris Romeo, c e o of Curve Ventures, and also joined by my good friend Robert hba. Hey, Robert.
Robert Hurlbut:Hey, Chris. Yeah, Robert Hurlbut and I'm a principal application security architect at Aquia. Also focused on threat modeling. and, uh, really excited to take a look at this paper, uh, and, and just dig into it. So, uh, very exciting.
Chris Romeo:Yeah, it's super exciting to have JB Avit with us again, uh, for his second visit to the application security podcast. And we're gonna talk about the state of application security, this report that, uh, the Datadog has put out. Um, JB great to have you back and we don't need to hear your origin story because we already heard it once, but great to have you back on the show as well.
JB Aviat:Thanks, Chris. Robert, happy to be here.
Chris Romeo:Yeah, so tell us before we even get into the details of this, uh, state of application security, give us kind of the high level view, like what is this, this report, and even more importantly, why, why create this report?
JB Aviat:Yes. So this report is based on, uh, the Datadog customers that are using the application security products and. APM products. So this, um, application security means a lot of things, right? And you can do, uh, application security in many ways at the code level, uh, maybe as, as, as a pen, um, as a well application firewall. But where we, um, do things, the, the data that way is, uh, from within the application, but also from within. Your, uh, cloud environment. And so the, the real of data today is really the run. So this is where we gather insights and those insights, they may differ dramatically from what you are used to when you work with source code or when you detect the web application firewall, whether it's AWS or CloudFlare, the. Big difference that we have being within the runtime, within the cloud environment of our customers is that we have a very deep context. Uh, I like to say we are full stack because we range from, um, the, the application itself, whether it's Java or Nude or Python. We're a this layer, but we. Also are at the OS layer with the data agent. And we also are at the cloud because, um, use data capabilities to monitor the cloud infrastructure. And when we those three of visibility. We can provide security insights that are much more precise than what you would get with only one of those items. And so this report helps describe what we see as the big, uh, and the big, um, things that you can sense when you leverage those, uh, three items, whether it's in term of threat detection. Vulnerability detection and prioritization and, uh, and more, uh, generally trends on, uh, where we see the more, uh, risks. Uh,
Chris Romeo:Yeah. One of the things that, that I really appreciate about tools such as does Datadog, is being in the runtime environment. You have access to be able to detect did something really happen? Like when you think about SQL injection, you can, you can scan for SQL injection using a number of different tools that are out operating on the source code itself. That may say, Hey, you have a SQL injection here. But when I think about the level of data and information you have inside the runtime, like if you detect a SQL injection, a SQL injection, cuz it just happened, like somebody just requested that. Hit a particular string or particular URL or something. And so from my perspective, that really gives a a much deeper level of insight to what's really happening versus what we think could be happening just by looking at the source code.
JB Aviat:Yes. Uh, you, you, you can think it as, okay. It's, it's good to have the information from your, uh, assess at the level that you have sickle injections. It's good to have the information from your what application firewall that you have someone attacking you, uh, for, for, uh, and trying to discover sickle injections. What's really hard when you have su and is to reconciliate both. And so how do you, uh, prioritize the reaction to a threat if you don't know what's the, uh, targeted application, or if you don't know, what are the underlying vulnerabilities? On the other hand, if your reports dozen, hundreds of injection vulnerabilities, then how do you pick which one you want to, um, go ahead first. And so, um, While we do not directly compare to SAS or, uh, to, um, to, to, what we can do from inside the application is telling you, okay, look here you had an attack that is targeting an endpoint that has a second injection, and that second injection just got exploited. And we know that because we are at, um, application layer. The database driver, so we know the exact query. We know where the like user input goes. And so that's true for injection, right? But that's true for s.
Chris Romeo:Mm-hmm. Cool. So Leia, let's jump in. Let's, let's, uh, I know there's a number of facts that you were able to extrapolate from all the data that you had available, and so let's jump in. Let's start talking about what, uh, what you found in the report.
JB Aviat:Sure. So let's, let's jump to, well, the first fact, right? Um, so this one is about, uh, vulnerability management. Um, so the way, uh, we detect vulnerabilities is at the runtime level. So in the case of, uh, open source software or third party libraries, whatever you call it, Uh, what happens is that when a service instrumented with data starts, it gathers the list of dependencies that are actually required by the service and sends it. To that. So, um, that's widely different. So from what you would get from, I dunno, dependable or something working at the, uh, ci uh, level, at the level what we get is vulnerabilities that are actually required by your, so that's the first prioritization, right? If your test libraries are never used, In your, uh, production application, which you would expect right then, we'll not even see it. And that is good in, in a way because, um, a library that is never used will never cause you a security issue, right? You cannot exploit renewability into something that is never used. So that's like a first layer of prioritization. Then another layer of prioritization is. To find what services matter more, uh, than others. So for instance, a service that has, um, only, uh, a development or staging or pre-production environment is not as important as a production. Well, hopefully you don't have, uh, customer data on it. If the service goes down, um, you not, uh, impact your production metrics, right? You might slow down your developers, et cetera, but. Provided you, you, you have decent, uh, separation of production and environment, but those are reasonable assumptions. So that's one thing. If, if you have a critical vulnerability on a, on a development service, well okay, you might not want to remediate that one first. And then the other, um, layer that we used in order to prioritize the vulnerabilities. Still using the runtime context is whether we detected attacks on that service. And so that's, uh, a metric that I think is very, very interesting because if you get back, for instance, to where everyone was rushing to their applications because you know, log for it was one request, and then you can get remote code execution on the server. if, uh, you have like hundreds of services to remediate, well, you need to prioritize. You cannot do everything. Uh, You cannot do everything the night of the, of the disclosure of the vulnerability. You have to prioritize. And so what would you prioritize? Well, the services that can be attacked, because that's the ones that are of getting scammed by a random, uh, hacker just wrote their script and are trying to, to pop shells everywhere. So if you prioritize the production services and the services that are being attacked, well that's a great way to look at, uh, the things you should do first. So that's very similar way for any and when we. Those two, um, those two dimensions, attacks and, uh, production environments, then we, that critical vulnerabilities remain critical. So the way we see, it's not like, uh, um, uh, rocket science, not machine learning or, or data science. We're just choosing the, the CVSs standard that is like the thing that tells whether vulnerability, critical or low, and cvs. And so the environment of scoring is where we, this is not production reached by cvs.
Chris Romeo:So when we, when I, when I think about this, like, is this, let me, let me kind of bounce an idea off. You tell me if this is what this is saying or if it's not saying this. And so I think one of the big things that people are thinking about a lot in the world of AppSec is exploitability, reachability, and exploitability. Right? And so are you, is this analysis telling me that only 3% of critical vulnerabilities are exploitable slash reachable? Or is it telling me something else?
JB Aviat:So, um, in, in that, uh, specific score, we are not taking into account exploitability as of today. It's something we are working on, and I, I think we'll have something in the next. Uh, so we are working on that independently from.
Chris Romeo:Okay.
JB Aviat:Of the, of the CVSs score. It's, it's part of the CVSs, um, temporal score. Because there is like temporality related to exploitability. You might think that something is not exploitable and two months later, well, someone comes with a, that makes the, the exploitation possible. Um, so that, that's a different way to, to look at it, right? So what we say, Is that Yes, reachability. Because if your service never got an attack, uh, well, given the noise of attacks on the internet, a service that never got any attack means that it's unlikely that it'll get attacked, uh, the, the, the next day. Right? And the other thing is the business sensitivity because we consider that a service that is not production doesn't have a huge, uh, business sensitivity. Does that make sense, Chris? But you are
Chris Romeo:so this is a down, so this is downgrading though, right? So basically you're saying that of all critical vulnerabilities, only 3% of them are actually critical. 32% of those, what people would've said were critical are actually high and 65% of what they, what? What the initial assessment would would've said. They're critical. They're actually lower. They're lower priority than what would've been reported if you just used industry data sources.
JB Aviat:Absolutely. Absolutely. That's correct.
Chris Romeo:getting, I'm, I'm tracking with you now. I'm I'm getting it. Um, so be, before we go any further, what's the, just, people are probably thinking like, what's the size of the data set here? Like, is this across? Yeah. Gimme give us some context about the size of, of the data that you're analyzing to get to these numbers.
JB Aviat:Yes, it's, so, I, I cannot, uh, discuss the, uh, exact number of like, uh, Datadog Qs, but this is the, the full APM Q base that has like, uh, an agent that is prettier today. So we're talking about, uh, thousands, uh, of, uh, organizations. And, and so you can, yes, it's, it's, it's largely, largely that it's this, It's a very significant
Chris Romeo:Okay. All right. I think I, I think I, understand. I think I got number one. I got number one figured out. Now I think we can go to, let's keep going. Let's keep rolling. Um, uh, hopefully I'll get number two faster than I got Number one.
JB Aviat:I think number two is, is pretty simple. So what, what we, what we computed here is, um, risk. So how do you define the service risk? We defined it as the sever. Of the most vulnerability on that service. So if you have a service with one critical vulnerability and 10 medium when we think that the service has a critical risk, alright? If a service has 10 critical vulnerability, then within the service has a critical, um, severity. And so what we, uh, computed is the service risk, depending on the number of, um, dependencies that it. And so what we see, which, uh, is seems very right, but it, it was great to see that, yes, uh, the, the data matches our nutrition. The, obviously the more dependencies you have in one is the higher is the risk, and it's simple to explain, right? If any, uh, dependency has a likelihood of having a vulnerability, well, the. Dependencies, you have the highest likelihood to have, uh, vulnerabilities. And so the highest likelihood to have critical vulnerability. So this is what exactly. The, the, the argument of trying to minimize, uh, the surface of your right, of your services ability on your, on your service.
Chris Romeo:This is, this is good because as you said, it, it backs up. This is data that backs up what I would, I would've argued for this without data, but this gives me data to say, the more dependencies you add, the more, um, chance, the more the service risk grows. You know, based on the number of dependencies. So this just gives me data to be able to say, and I can prove it now. I can't. It's not like my gut says, the more dependencies I add, the more chances I'm gonna have a critical vulnerability and a service. Now I've got data to back it up.
JB Aviat:Absolutely. And so, uh, interestingly enough, what we uh, discovered is that this is not true for net services. And so that, that, that was very, very surprising to us because yes, intuitively you would think it's, and that's not true for, uh, net services. And so our, uh, assumption is that you have much. Less, uh, vulnerabilities that are reported within the ecosystem than in any other ecosystem. And so that's, uh, that's question to us the. I'm not sure, is it that the ecosystem has much less security research than the others? I dunno. The number of dependencies that you had on every services was to what we found with technology. So that was
Chris Romeo:It. Robert's been, Robert's been trying to convince me that dot net's better for years, as long as I've known him and like you're giving him some, uh, some, some, some fuel here to, to, to make his argument even better.
Robert Hurlbut:Although, I will say it's
JB Aviat:interested. Robert, do you have.
Robert Hurlbut:I was gonna say, it's interesting about the net, um, um, Conclusion there that, you know, many.net applications still pull things from other libraries, third party libraries. But so it's, it is just interesting, and I don't know if it's the, the, the amount of data that was collected of, of apps that didn't, but certainly I do know that the, as it said, the.net standard library has certainly, um, continued to add, uh, things that previously were only available from. third party libraries. So I, I see a little bit of both of that in there, but it's an interesting fact,
Chris Romeo:Hmm.
JB Aviat:That, that that's correct. Uh, like the, the, the, the size of the tender library in the is is indeed impressive. So that, that might explain that. But still, since you have a lot of dependencies as well, uh, I dunno.
Chris Romeo:All right, let's, uh, let's see what we got for fact number three. I feel like I understood fact number two, much quicker. I feel like I'm getting better with each one.
JB Aviat:Excellent. Uh, maybe my, my French accent starts to, to be understood,
Chris Romeo:tracking with you. I'm tracking with you.
JB Aviat:Um, so yes. The next fact is that, uh, Java services have the highest risk. So we, we, we computing the, um, do risk per language, right? So Java, Python, and what we, that Javas have the highest. So you can see that we, um, don't track, uh, PHP vulnerabilities, for instance, uh, yet. Um, But that would be an interesting comparison. Now here, one explanation. Um, so to, to I was expecting to as the, as most as technology ecosystem is huge, right? And, and you have like a. Philosophy of including a lot of small libraries in your, in your nodes, um, applications, but still having Java first. And so just like that, we can make hypothesis based on that. Uh, the first hypothesis is that, well, Java is pretty low level language. It's, it's code. So you have a history of, uh, of some, uh, exploitation within the, the, the gvm. And you can think of which is also well, uh, known, um, vector of vulnerabilities, Java. So reason is. Uh, enterprises, uh, and very ecosystem, which may also have a lot of, uh, attention from a, from a security standpoint where the over enterprise, uh, using Java is probably doing much more, uh, and search ecosystem, uh, than languages that are most using in maybe less metro environments. And so that might be a very naive interpretation, but, uh, that, that, that's one that, uh, that we.
Chris Romeo:I mean, I, I think that's, that's consistent when, like, when I look at this chart, Java's been around the longest. Java's got the most legacy applications that are still running in the enterprise. Like this one doesn't surprise me at all because I think about where was the state of AppSec when Node really hit the scene? Where was the state of AppSec when Python hit the scene? It was things were just, We just understood things better. Java's been around as long as, as long as I've been, seems like as long as I've been working in tech, Java's been there with me. Like it's, it's just, it's, there's a lot of older, older stuff in Java that has gotta be skewing those numbers.
JB Aviat:I like, uh, I like this analysis, Chris, that, that's actually a good one. Yes. The, the, and, and I don't think we, um, we wrote that down, but yeah, I, I'm making a mental note of that.
Robert Hurlbut:Yeah. And plus, you know, related to that, uh, and, and some of that.net as well perhaps, but, but certainly with Java, um, as you continue to update the framework, I find sometimes that, um, some organizations I've worked with are a little bit slower on getting, uh, up to date. They might say, well, we're stable. We're gonna stay here for a little bit. We're gonna wait a, a year or two, and then we'll update. And so as a result, you know, updates continue to be applied to the latest versions, uh, typically and, and, uh, not as much backward compatibility. So certainly I can see that might impact them as well. So it's interesting to see, but it confirms what I've, I've seen as well in, in my experience in in the industry.
JB Aviat:good,
Chris Romeo:we're on a roll. Let's keep going.
JB Aviat:Yes. And, um, so the, the, the next one, uh, um, on that case,
Chris Romeo:one hurts. This hurts jb. It hurts a little bit as an apsec professional.
JB Aviat:Yes. I know, I know. And and I agree. Uh, so yes, the fact that a lot our significant, uh, part of organizations still have, uh, vulnerabilities that are pretty whole, uh, yes. And so, uh, injection and SS ssf, uh, from, I think we discovered artists like they were mentioned on. This year. Um, and so yes, we still have five of the, of the organizations that, uh, have injection. Impactful, uh, to, uh, to organizations. But yes, uh, we offered here are, it's, it's, it's, it's a thing of the past or it's, it's, it's not happening again. Well, first injections, we have a number of ways to, to, to protect against it. Still, I think you, you have, we make mistakes, right? Uh, that, that, that still happens. For ssf. On the other hand, while the protections are far from being straightforward, um, obviously you can just make sure that the way you handle URLs. Uh, that depends a lot on your business logic. What are you allow to retrieve? Maybe just as in terms of context, ssf is when you handle user. Doing HTTP request to a server that you are not, uh, expecting to, so
Chris Romeo:Yeah.
JB Aviat:massive where you might be able to credential application.
Chris Romeo:Were you, were you able to tra trace? Like, I don't think of, I thought S S R F was a lot later. Than the nineties. Like, and maybe I just missed something, but what's the, like, do you, how, where, when, when do you, when are you tracing kind of the origins of S S R F
JB Aviat:Yes. So it, it wasn't called necessary at that time, but uh, it was surprisingly enough in the exact same track paper that described chicken injections. Uh, and so it was not called ssf. It was not like, uh, as, uh, generic as what we think, uh, about today, but it was disclosing the exact same papers. That's the same
Chris Romeo:Hmm. Yeah.
JB Aviat:rights grew a lot, uh, with the, uh, specifically provider. Right where, where, where with the single ssr you can capture all of the, um, all of the cloud credentials of the application. And that's when that vulnerability really, again, like it's, it's, uh, um, in French, I would say, but the English website, but became like a very impactful vulnerability.
Robert Hurlbut:This, this seems to match. I think what we've also seen in the Owas top 10. Uh, S S R F, I believe, uh, dropped in 2017. and now in the, the latest, uh, 2021, um, injection is no longer number one, which or has been almost 20 years. So I, I think, you know, it shows a good trend. Um, but it still shows it's there unfortunately. Uh, but, but I, I like that, you know, the numbers certainly, um, are matching across the industry. We're seeing, uh, this a lot less than we did before, but it's still there.
JB Aviat:And, and that's a very good point, Robert, here, we, we don't have a timeline. Uh, cause you, you know, the, the, the, uh, the product application security product data has been around, uh, for about. And half. But, uh, we don't have meaningful historical data because like our number of torical.
Chris Romeo:Very cool. Very cool. All right. What's next?
JB Aviat:All right, so this one. I love and is, uh, to the, um, data context, right? Uh, and so to my knowledge, uh, uh, was, was never, uh, or even computed before. Um, I'm happy to be corrected if you, if you are aware of that, but what we manage, uh, to, um, to compute with our, uh, full stack observability. Is that 74% of the attacks are misted. So what does it mean to have a mist attack? Well, anyone who looked at the logs of a web application firewall knows that yes, you attack all time, right? And so the difficulty with, to get through, people rarely look at the logs of a web application firewall because it's just, just too much. What we discover is that yes, we can tell when we bring in, when we bridge, um, threat information with drone time context, is that we can discover that an attack is unsuccessful. So if you have a, that targets an application that, that do not outside, outside, or that do not choose user input in the, uh, external well, You are not at risk, right? Someone is attacking you. So that's a valuable information you might want block the ip. Uh, you might wanna, uh, uh, understand that yes, uh, this application is part of your surface, but this the fact that attack was targeting that application is not important for you. You can dismiss that. So it's something that, uh, uh, Really cut noise saying attacks.
Chris Romeo:Yeah, this is an interesting one in that the mist targeted language, like I think of times I have looked in those logs to see, and the thing that I always remember seeing is there's so many PHP scanners that are looking, and I think they're, they're hunting for WordPress, vulnerable WordPress installations, but it seems like the bulk of those, I see so many PHP related. Scans that are happening. And so, but it looks like the data's telling you something different in that the mist targeted language is actually a really small part of this and technology, or is that where it would you call P H P A language or a technology in this bottom part of this analysis.
JB Aviat:Uh, no. So that, that would be, uh, that would be a long way. Absolutely. Um,
Chris Romeo:Okay.
JB Aviat:So the, the technology would be the, the database, uh, the ssf, the, maybe the, the framework if I'm for, uh, so WordPress vulnerability on an application that is using a symphony, for instance, where you have technology. Right. But absolutely, language is exactly that. That's, uh, I dunno, like, uh, look for on a, on a, on a node application, well, okay, you don't care about, And so
Chris Romeo:tells us though, but we can conclude. We can conclude from this though, that there's still a lot of just random scanning happening on the internet, like attackers are not. But certain classes of attackers are still just broad scanning, just hitting the whole internet with, uh, with particular scanners, hoping to get a hit outta something. Um, and they're playing the bulk game of, you know, let's, let's buy a million lottery tickets. So the hope that we win the lottery, like that's, that's what they're trying to do, is buy a lot of tickets and get one or two of them to hit.
JB Aviat:I like the analogy of the, of the lottery ticket. Exactly. And, and that's, um, uh, again, why we still see a lot of like log for shells are things that people had the time to, to patch, right. But that are, uh, still, uh, being, uh, by Byers.
Robert Hurlbut:you
JB Aviat:And so
Chris Romeo:Attackers
JB Aviat:what, uh,
Chris Romeo:Like, and I shouldn't, I shouldn't say this. I'm probably, I'm probably gonna get in trouble or something, but like in general. People that are trying to break into applications. In large part, they're trying to do it the easiest way possible. Like it's, it's not like on the matrix. Okay. It's not like on, uh, hackers or sneakers or, you know, any of the, any of the Hollywood portrayals of, uh, attackers getting into systems like, An attacker will take the easiest way in every time. A good attacker will take the easiest way in every time. They don't have to make it dynamic or, or interesting. They just want to get in and if they can, if they can broad scan the internet and use that as the vector to get into a system, they will take it 100% of the time.
Robert Hurlbut:I, I like that you broke down the paragraph above it. I don't know if you can scroll up a little bit, but in the paragraph above it, it mentions the breakdown of the 74%. Uh, it said, uh, 66%. Uh, endpoints not present, 31%, uh, vulnerabilities related to databases not used by those systems. and then 3% of language is not used in the application. So yeah, I can certainly think of if you're trying to attack, uh, like the database there that you mentioned, uh, are, are highlighting the, you know, sql, for example. Sql, A server has a particular IP address that it uses to listen for requests. Oracle does the same and so on. And so if you're trying to attack those particular IP addresses, Or ips rather to, um, or ports rather, sorry, ports to, uh, to, you know, send in. But they don't even use SQL Server. They don't even use Oracle or whatever. I can definitely, uh, understand that. So it makes sense.
JB Aviat:And I, I think the other, um, interesting part is that. Um, well, for 26% of the attacks, we cannot completely Right? Because maybe you are attempting injections on the service that has a database, right? And so not, uh, we, we.
Chris Romeo:Hmm.
JB Aviat:Still being able to do it in, in three quarter of the cases we believe is a, is a great, uh, simplification on, on how you would handle threats the application.
Chris Romeo:Cool. Let's keep going. What's number six?
JB Aviat:Is the top target of language specific attacks. So to your point of random scanners looking for, uh, WordPress and, and, and similar things. Uh, Chris, what we discover is that when we break down all of the attacks that we have, some of them target language specific things. So what I would not call language specific thing is, You can have SF in any language, the secure injection, any language, but if you target like what president PHP or PhD, like ization, what is PHP targeted for? Java. If you target OG l, okay, it's Java. If you target, uh, it's Java. Uh, and, and, and so that's how we built that, uh, breakdown. And so here, interestingly enough, yes, PHP. Uh, is is the vast majority of, of all the attacks that, uh, that we detect attacks that are
Chris Romeo:Yeah. Word WordPress rules the internet. We know it. That's why the internet exists to host WordPress pages. WordPress sites. All right. Let's look at this last one. This, this, this last one is I, I scanned ahead a little bit and it kind of, it seemed, um, it seemed like fact seven is a bit of a hidden. I, I didn't, I, I looked at this and I went, wait, what? So like, I, I wanna learn more about this one.
JB Aviat:Sure. So people that, uh, use data, they usually tag the environments, right? So you have your production environment, that's the one on which you'll have the page duty. If something goes wrong and you have the dev and pre-production environments, here is no page duty. You use that to ensure that what you're about to deploy is performant and safe, et cetera. What we did is break down the attacks that we have, poor environments. And so what we found is that, so first of all, we don't know all of the environments. That's the 17. This one is just people don't tag or use that we are not able to integrate. But for, uh, everything we, we understood that like 11% of the attacks target. Non-production environments. That's your staging, your pre-production, your development environment that is actually being targeted. And so same point where we environments that's. But when such an environment is attacked, well that's still a concern, right? And so often when people map their attack, they're focusing, that's where you data. But if you think that pre-product staging are also attacked, well, there is a huge risk because if you go developer staging environments, Handling the secrets as carefully as your production, for instance, as.
Chris Romeo:Yeah, it's, uh, it's making me think about, or making me ask the question, are attackers practicing? In non-production environments before they launch an attack against a production. Because if I'm going after a system and I don't have any way to test it locally, an attacker, I don't have a way to model it and, and run my own copy of it so that I can test in, in quiet and private where no one else can see what I'm doing. The next best thing is to, is to practice on the non-production environment before and, and create an attack complete sequence, make sure it works, and then hit the production environment one time. Perform the action and get out of, get out of town right before any other problems. So, um, that's, I don't know if the, I don't know if the data's probably not gonna back up that idea as to whether that's true or not, but that's just what I'm thinking about when I, when I look at this
Robert Hurlbut:I, I, I think of a couple of things on this. Uh, one maybe that, I mean, that certainly would help, right? If you found, uh, a staging server that's a, a publicly available and maybe not monitored as closely, uh, it certainly can help you to, uh, practice, uh, the real attack against the production server. I think that this is also potentially a product of unfortunately, um, cloud environments that we have now where we don't have anything on premise, uh, anymore. We don't have, you know, our, our, uh, web service and so forth set up, we may have pushed them all out to cloud and depending on, you know, how we access our staging environments versus production environments. If, you know, we're not careful in terms of managing, uh, the access and so forth, they could be still available still out there. And so both could be available, and that may be part of this. But again, I, without seeing the actual data and seeing why, but that's just a, a hunch is that previously we would, uh, have all that stuff hidden. It would be internal. Nobody ever gets to it except for the internal network. um, and only production would be a available publicly, but I think now potentially all your environments are, uh, available because the only way you're gonna get there is over the internet to get to those environments unless you have something set up, uh, to create VPNs and so forth. But, you know, even that you, you may be potentially opening yourself up. So it would be interesting to sort of follow up on this and, and maybe dig in a little bit more. But it, it's an interesting statistic.
JB Aviat:And so I, I, I guess, uh, this, uh, to your point, I think the trend of development will just make that statistics higher and higher. Again, I would love to see what happens next, uh, year, uh, to, to, to start having a trend. But if you think of all like, um, Low code or no code development tooling. Well, the people using those tools, they, they have no choice. Right. But to use like instance to, so design is accessible. It be, but it'll be accessible. We've, I've been, I'm a big fan of, um, uh, and so you just, it's, it's something to preview, um, like single applications. You just plug, uh, your GI there and it'll automatically build and give, you will publicly expose, uh, the next, uh, version of your, of your single page application. So all of the. Those tools by design, uh, build a non-production, a preview environment that is chooses production, APIs. APIs a different and.
Robert Hurlbut:Yeah. And, and think of accounts too. Um, you know, in your, in your non-production environment, you're not using the real accounts, you're using a bunch of test accounts, and so the security around those may not be, uh, as, as well thought out as you would for production. So it's, again, it's very interesting, uh, statistic to look at.
Chris Romeo:Yep. So my big takeaway have no staging environments and no dev environments. Put everything in
Robert Hurlbut:you go.
Chris Romeo:Test and production. Okay. That's not, that's not what I actually, that that was a joke. That wasn't, that, that's not an actual recommendation. Um, I think staging and dev have
Robert Hurlbut:have their place.
Chris Romeo:their, uh, their parts have their place in the overall application deployment, uh, scenarios. So, uh, JB this is, this is excellent. Thank you for, for walking us through the report. Um, we'll definitely add a, a link to the report in the show notes so people can go study this in more depth. Um, I'm looking forward to the future years. I wanna see how this data updates and I wanna see how, how things change over a period of time. Cuz you know, I feel like I spend a lot of time thinking about AppSec, but there's, there were a few things in here that. Gave me stuff to think about, uh, as to kind of challenge some of my assumptions about where I thought things were in the, in the industry now. So thanks for sharing that with us, and we look forward to, uh, talking to you some more in the future about, uh, a as you continue to watch this data and, uh, and are ready to report on it, uh, in the, in the next year.
JB Aviat:Happy to looking forward to it as well. Thank you so much Chris and Robert.