Go back

EP01 - Rethinking Offensive Security Assurance

44m 2s

EP01 - Rethinking Offensive Security Assurance

The transcript highlights critical gaps in how organizations conduct offensive security activities, particularly external penetration tests, to achieve true security assurance. Key issues include organizations providing static, incomplete scopes (e.g., IP lists that are outdated or represent only a fraction of the real attack surface) and failing to include discovery phases to validate their perimeter. This leads to a false sense of security, as testers are constrained to narrow scopes while adversaries have unlimited time and resources. Additional deficiencies include unrealistic constraints like requiring prior approval for every command, which wastes testing time, and poor preparation of prerequisites (e.g., credentials or allowlisting) that reduces effective testing effort by up to 33%. The hosts emphasize that penetration tests should simulate realistic attack paths—not just vulnerability scans—and that organizations must clearly define their objectives (e.g., breach simulation vs. technical audit). They advocate for collaborative scoping with wildcard domains, subsidiary discovery, and advance planning to ensure tests are thorough and meaningful. Ultimately, without addressing these gaps, organizations risk misallocating resources and failing to reduce their likelihood of compromise.

Transcription

7094 Words, 41532 Characters

English
[Music] Welcome to Cyber Conversations, where experienced industry professionals provide insights into the dynamic world of cybersecurity in Australia. I'm Chris Elliott, a senior Red Seamer with experience performing both penetration testing and adversary simulation engagements. And I'm Jacob Larson, an offensive security lead and threat researcher with a diverse background in strategic cybersecurity advisory. Together we have provided both strategic and technical security advice to hundreds of Australian organisations. In each episode we'll share our insights, discuss emerging trends and delve into the challenges and solutions affecting the Australian security industry. Whether you're a seasoned professional or you're in the field, our goal is to provide you with valuable perspectives to help you navigate the complexities of cybersecurity. So grab your headphones, settle in and join us for Cyber Conversations, where we explore the front lines of Australia's cybersecurity industry. Today we'll explore the critical gaps in how organisations conduct offensive security activities for security assurance and the use of third-party professionals to execute this. In our experience, deficiencies in security assurance results in a misallocation of resources which fails to minimally impact whether an organisation is likely to be compromised. So let's dive deep into this topic now. Chris, can you give us a definition about what security assurance actually means? Absolutely. So the intention of security assurance is to provide the organisation with confidence that its security controls and defences are effective in reducing risk and a capable of preventing or detecting potential threats. So obviously we've both worked as pan-testers and I think particularly from our perspective in the consulting realm, we get engaged by internal security teams that want a pen test performed on a specific scope. Now, two of the most common pen-tests are an external pen test and an internal pen test that both start in different starting points from a position of privilege. And so we'll focus particularly on an external pen test for this first point. And the objective of an external pen test is to evaluate whether an adversary can compromise the organisation's public perimeter and obtain a foothold to any kind of internal network or system that allows them to access information that otherwise be restricted only to employees of the organisation. Now when we work with third parties often organisations will say this is the scope of the pen test. It might be a list of demands and a few IP addresses. And obviously when you're working with a third party because of the legality of ethical hacking, it's really critical to have that scope tightly agreed before the penetration testing starts. But one of the areas that is a deficiency that I offer an encounter is organisations saying, "Yep, this is our public perimeter and they provide the scope." And they're not willing to perform any kind of initial discovery to validate that what they're provided as their scope is their actual public perimeter. So what this results in is effectively the pen tester having their assessment constrained to what has been provided. In some cases I've seen examples where an IP address is provided for the test scope. And there's not a single port open with any service running on the external. So there's nothing actually to assess or remunerate or report any security issues on. Can you give your perspective on this problem as well? Definitely. So yeah, a lot of the time an organisation will just either give us a list of IP addresses or yeah, they're external side arranges. And some key issues with that is we've seen things before where an organisation will just go to Amazon and dump down a list of IP addresses and say, "Yep, here's all our EC2 instances." But they're actually dynamic IPs and what happens is between the time they've actually scope the engagement and we start testing those boxes have been rebooted, those IPs have been released and renewed. And it's just no longer accurate. Essentially it's for, could be for some completely different organisation. The other part of that is what an organisation actually thinks is their security perimeter, isn't necessarily what their actual security perimeter is. So things like where they've got multiple cloud providers, the party providers, different storage mechanisms and that sort of thing, all these wrap into that. What is the organisation's security perimeter and are actually a tax service that adversaries will look at in trying to get a foothold. Without thinking about those sort of information sources as part of the external penetration test, it's just not realistically measuring the sort of likely hood or success rate of an adversary being able to get in. It often leads to organisations getting a false sense of security. Absolutely. And I think another good point to discuss is how a lot of the time this information is captured in the scoping process when engaging a third party. The cost and internal security team might have the intention of evaluating the security perimeter. By the time the penetration test actually gets assigned to do the work, if they do discover something else that's important that should be added in scope, by that point the commercials have already been agreed. And the allocation of effort has already been constrained to what the initial agreed scope was. If in the initial scoping activity, usually the internal security teams shared by 10 IPs and they've been allocated four or five days of testing effort. But once the discovery commences, you know, an additional 30 IPs or 10 demands, etc is discovered, sure with an agreement it could be added into scope. But it's not realistic that a full evaluation of those, that attack service will be performed in that time. No point by the time the commercials have already been won and the engagement started, does any party want to have to go back and renegotiate, particularly not the security team where they've already received the approval for management or their size, or the president of their business case. The last thing they want to do is go back and ask for more funding. It's really important that when your, if your internal security team and you're engaging a third party, that you ask them to either perform that, include an activity for discovery of the external attack service. And that is included as like a key activity before the penetration testing starts. So that if any additional high risk end points or legacy IPs that might still have services running on them can be added into scope. And just in general providing a wildcard scope for the assessment so that the adversary or the penetration tester can discover other assets that might be important to look at. Yeah, that makes sense. One thing that so contractors and people that are actually delivering these penetration tests, they just do their best on the information provided to map what they've been given back to a certain amount of time. And let's say the client's only giving IP addresses across. What if those IP addresses are actually reverse proxies for hundreds of websites, right? And so all of a sudden you've got maybe 10 IP addresses which you've been given, but it's actually for 10,000 websites, right? You've scopeed based on this number of posts, whereas in fact the attack surface is much wider than what you originally anticipated. And as penetration test is we do what we can within the time we're allocated, but an adversary they have unlimited time, unlimited resources. It's just it's really a false sense of security to look at that sort of scope and say, yeah, we were not able to find a way in in the time that we were given. So from a solutions perspective, we need to address from both sides what should organizations be doing to make sure that when they're engaging a third party or conducting their own external pen test that the full scope is considered. And I think the key answer here is making sure that when the penetration test starts, that a discovery activity is performed. You know, making sure that any websites, the full not just the domain name is provided, but a wildcard subdomain is provided which indicates to the tester that they can effectively enumerate and attack any subdomain of that host. Additionally, providing any of the subsidiaries or owned companies entity names so that they can be reversible cups from like an ASM ownership perspective to see if any other IP ranges can be discovered and added into scope. What we're trying to do is basically be in a position where the internal security team provides as much information as possible to enable the penetration tester to fully assess the external primary of the organization. Another point that's probably worth talking about as well is how often an external pen test, the objective is like we mentioned to evaluate whether an adversary can breach the perimeter. But an adversary can sometimes breach the perimeter through things like valid credentials, phishing, credential staffing, which are often considered out of scope. And it's important to be aware of what those out of scope activities are because if you're expecting that all potential attack paths are going to be evaluated in an external pen test and say a key activity like phishing isn't performed. That might also result in a false sense of security assurance. What's your perspective, Chris? Yeah, I think it's interesting. And it goes back to what is the organisation actually looking for as part of a pen testing scope of works. People have different opinions of what a pen test is. And I feel like there's a misunderstanding or a miscommunication between different people in industry what that looks like. My perspective, and I think this goes back to what the roots for penetration test is, is finding realistic attack paths to get from point A to point B. So in the case of an external penetration test, point on the internet, unauthenticated, to earn internal foothold within the organisation. Let's say it's Citrix Box or on a web server in the DMZ or something, just that initial internal access. But some companies are approaching it as almost like a vulnerability assessment, where you've got the external scope. Let's run a bunch of automated scanners across them, get an overall look at what the security perimeter is in terms of things like the patches up to date or the web servers configured with strong ciphers and what does the TLS infrastructure look like or the certificates? Whereas that's not really a penetration test in my eyes. That's more of like a security audit or a technical security audit, let's say. It's not actually showing the organisation what sort of lens would an attacker take to get into our environment and how would they perform it. So I think that should be one of the first conversations that's had when trying to scope these things. Is it actually a pen test you're looking for or do you just want a security, a technical security audit that looks across your entire scope that you provide us and looks for things like vulnerable systems, missing patches, et cetera, et cetera? For sure. And it can be hard because sometimes with vulnerable patch, you know, outdated components and software on the external perimeter, that might be a point of entry into the organisation. You know, often adversaries are exploiting end-a-vonor abilities. You know, obviously zero days in some cases, but primarily end-aise to, you know, get command execution or in some way obtain a foothold. And obviously, in those cases, you definitely want your pen test to be evaluating. Here's your VPN outdated and can it be exported for a foothold, et cetera. But you're not necessarily going to be looking for outdated JavaScript libraries in your main website as a way to obtain an initial foothold. So again, it's about prioritising what is important to you. Do you want a, you know, do you want a mile wide and an inch deep in terms of look at the entire scope with a kind of basic level of misconfigurations, or do you want actually to evaluate can the perimeter be breached? So it's about bringing that context into that conversation and engaging that third party. Absolutely. I think we can probably move now into some of the other constraints and deficiencies in security assurance. And one of the most important things to understand is that when you're engaging in penetration testing company or doing it internally, any sort of penetration test is a time bound exercise. There's a limitation of effort that's available to do the assessment. And it's really important that the no constraints are introduced to limit the quality that can be provided in that limited time frame. So in some cases, I've seen examples where organisations will say, you can't run any command without prior approval. Or how do I perform an immigration to run end map to a numerate? So as a running, you know, if every single command that needs to be run requires prior approval, it can result in a significant amount of lost effort and a severely reduced quality of service. Now, I think it's important that whenever these things come up to actually kind of understand the intent of why that constraint might be in place. Now, in some cases, an organisation that says they want, you know, all commands approved, they might have had a previous example where a penetration test or someone has, you know, just ran it around and exploit and hit something offline and result of the demolished service. But it's also not important to let that kind of event impact all for the testing from there. Can you think of any other examples in relation to constraints for testing, from your perspective, Chris? Yeah, absolutely. Yeah, just sort of my thoughts on having to provide every command that you're potentially going to run. That's just not a realistic way of approaching penetration testing. You don't actually know what commands you're going to run until you have a look at what's in the environment, right? Whether you're attacking web servers or SQL servers or whatever. It's just, you know, you'd be basically printing out a million plus lines of potential. It's just, no, not feasible. You've got to have trust with the people that are delivering your pen tests to do the right thing. And if that trust is not there, then why are you engaging them to sort of do a penetration test in my eyes? In terms of additional constraints, so excluding systems from scope for various reasons, potentially, they're a critical system. And they're used by customers on a daily basis, like social media platforms, internet banking portals, et cetera. Excluding things like that because of that sort of risk, you've also got the risk of what happens if you don't test them, right? And so I'd prefer to see an approach where it's more collaborative and communicative, where you're sort of working with us to test those in a more controlled and safe way, whether that's a staging environment or doing it at a certain time period, where the risk of an outage is reduced. You've got a test to know whether there is demonstradable risk there. Additional things like considering where your users are all good accessing your sites from. So things like geo-pensing and that. If you have these technologies in use, you absolutely need to let your penetration testers know before they start testing. Global organizations where they've got access points from all over the world. If some of those are only accessible from, say, the US and some from Australia, as penetration testers, we absolutely need to know that. So we can make sure we're scanning from the right places, so they're actually seeing a true representation of your external attack surface. 100%. And one of the most common things, particularly when engaging a consulting or contractor third party for penetration testing, is the understanding that it is a time-lapse assessment. You're paying by the day for their services. And when they engage you and let you know the start and finish date, they need any prerequisite, whether that's allow listing for their source IP, credentials being provided, et cetera, well before that start test date. You know, if you're paying thousands of dollars per day for an engagement, you don't want it to be spending the first two days still providing your assigned party with the credentials and the prerequisites they need. In your mind, you might think, oh, well, we'll just start later and finish later, but the reality is it doesn't happen. That effort and that time is burnt. So there's the immediate cost, which is the loss of money on that time where nothing product is being done. But then there's the double loss where you're losing variable time. And if the pentest is, say, allocated six days, and the first two days are spent establishing those prerequisites, you've just reduced the time allocation of the test by 33%. That's a significant impact to the quality of the test. So it's really critical that when you're engaging third parties for testing, that you're establishing well in advance what those prerequisites are. If it's account creation, things like this, which often takes time, you submit a service ticket, needs approval, might take three to four days of paying on your own internal organisation. You want your test to be providing you those prerequisites well in advance, at least two weeks at a minimum. So it's about being aware of that. Yeah, could you add anything else to that, Chris? From your perspective? Yeah, I think it's a two-way streak, right? So we'll do our best to let you know well in advance of what we require. Let's say, you know, a month before even, like, hey, we need these accounts for initial access if it's an internal sort of engagement. And with us, you know, giving you the heads up and that sort of thing, we hope that that means we'll be able to test our access in advance, make sure when the engagement starts, that we're ready to hit the ground running, start trying to identify risk in the environment. Yeah, you make a really good point. It is a two-way streak. And I don't want to come across as, from the third party's perspective, all these deficiencies from the internal security team's perspective, it is really true. I've seen examples where, you know, a third party, a company signs a contract for penetration test and a schedule for the next day. There's absolutely no way that the prerequisites are going to be met in 24 hours. I've only had many one out of 10 cases where they've been met. And it depends on the scope and what the service is. If it's an external network of Pantist, all you need to do is say, yes or no to vote to survive these, that's at least something that you can facilitate quite quickly. But if it's creating any kind of account, that's going to take at least a couple of business days. So it's being aware of, well, if you do have a requirement for a Pantist, engaging that third party, well in advance, you might have a board reporting obligation or a compliance requirement to have the report by a certain day, please just help us and help yourselves any guide just. advance so that we can ensure that by the time the test actually starts that we're getting effective use of that time to enumerate is usually environment. What you don't want to do is schedule a test last minute. There's inadequate time to gather the prerequisites. They've provided midway through the engagement and then in the day you get a report that you share with your board that says you've got a low security risk foster, but the reality is it's a false sense of assurance because the test only had two out of the five days allocated for testing to actually look into the inscribed systems. I think we can probably move on there as well and talk more about red timings specifically. Now Chris, this is one of your specializations. Can you talk more about when do you actually need to conduct a red team? Absolutely. So I'll just give my sort of definition of what a red team excise is before we get stuck into it. And when I'm talking about red teaming here, I'm talking about essentially trying to simulate an adversarial attack on an organisation. So a red team is essentially, it should be a long-term engagement, potentially a month plus, ideally starting at no access, trying to get in via reasonable means the realistic threat actor would take. And then once getting to that point, trying to remain undetected, be all privileged within the environment and start working towards trying to compromise key critical systems to complete the engagement objectives. And when I'm talking about realistic attack vectors here, I'm talking about things like trying to get in organically through external facing portals, external facing applications, etc. Vishing, vishing, smishing, all those sort of social engineering-based attacks, potentially trying to interact with people via teams or using their services that they use to interact with each other internally. I think some red teams will often go onto site or send in physical implants and that sort of thing. That can definitely be included in the scope, but I think it's really important to tie the sort of attacks back to what is a realistic threat actor going to do. So if the threat actor's targeting your vertical are all based overseas in Russia or China, a country like that, and all the evidence for other attacks is showing everyone's getting in via fishing, you're probably not going to want to invest a lot of time in trying to sneak into the corporate office and drop an implant. Not saying that that's not feasible definitely is, and you need to take internal security seriously, physical security. But yeah, it should be trying to simulate a threat actor in a realistic way. So yeah, a question we often get asked is when should you red-team, when should you pen test? Are we ready for a red-team? Are we not ready for a red-team? The reality is you're constantly getting attacked by adversaries over the internet, all from different levels of maturity and sophistication. I think if you're finding that you're doing a one-week external pen test and you're the consultants that you've hired to do the engagement of getting in, and you're not seeing any high fidelity alerts in your scene and things like that, you're probably not spending your money in the most effective way by hiring a red-team to come in a sessure environment because it's more than likely that they're getting in a few days, completely objectives, and yeah, essentially they would have planned for a month and one week later they're basically done. Your time's much better spent working out why are they getting in within such a short period of time. Once you're getting to a point where your external pen tests are only returning low findings, I think that's a good point when you can start trying to take things up a notch and going a little bit harder. One week is not a lot of time to try and get into a company at all, having potentially two weeks a month on a red-team, multiple operators working together. It just gives them so much more time to go so much further in depth into your external security perimeter and try and find that weak point unless you're in. Yeah, really question. So some people might be familiar with Purple Team, which is a more collaborative approach between the red team and the blue team. From your perspective, is there a maturity point where after an organisation has done Excel Network Pantas, internal network Pantas for at least two weeks, elapsed times, and they're not breaking into the perimeter and they're not necessarily elevating to domain admin or completing objectives. Is there any value in conducting a purple team and assessing detective controls or preventive controls first before doing a red team? And do you find that, you know, two-handed question, do you find that when you're engaged for a red team yourself, have organisations already completed purple teams first? No, that's a really good question. I think they're two completely different types of security exercise in terms of outcome. So I don't think there's one that needs to predate another one. If you've got the resources, I'd definitely be doing both. Because what's happening is the red team's essentially trying to make a level of noise in your environment, trying to test common TTPs or Active Directory enumeration methods, et cetera. And yeah, their ability to go in depth in a purple team is limited, but on the other hand, you've got the blue team there. They're working with the customer to try and see whether these attacks are being detected. So there's that time overhead for the red team to talk to the blue team going, we tried to do a cover-oasting attack. Did you see that? Can you work with the customer to look in their scene, et cetera? So yeah, we're supposed to an external network pen test or a red team. It's purely focused on finding that way in. So in an ideal world, I think both would be happening to certain different purposes. So red team focused on finding vulnerabilities, same in pen testing, and purple teaming, trying to assess the detection and prevention maturity of an organisation. Yeah, that's a really good point. I think from a lot of the answers we've provided today, it's been from the perspective of ourselves primarily working in consultancies and being engaged by security teams. From your perspective, looking at internal red teams, is there much difference in terms of how the exercise is scoped and delivered versus engaging a third party? Yeah, and I think this sort of ties back into different people in the industry, how different definitions of what a pen test is, what a red team is. So how I described a red team earlier might be completely different to how you understand a red team. And I think it's at the point where there's no right or wrong answer to be honest. I think it's just important to be crystal clear when we're talking about these things. It's like, we're going to do an external pen test, and by that, it means we're going to look at the scope you provide to us, try and find vulnerabilities in that scope and use those to try and find a exploit or a demonstrable way of using those vulnerabilities to breach this service. I think, yeah, it's not that black and white anymore, unfortunately. So when an organization has an internal red team function, that function might actually be there just to do internal technical risk assessments of applications or internal web application pen testing, I feel like the word red team has a lot of glamour and marketing potential associated with it. So when organizations are hiring a red team, it sounds almost sexy, right? But the reality is that function may actually just be an internal technical security function. Now on the other hand, you've got organizations which actually have internal red teams. And I reckon in Australia, I can count the number of organizations that have this capability on one hand. What do you think? Do you agree with that? Yeah, I think it's a important decision. I think often, like you said, there's a sense of glamour with being in a red team position. And so it's almost attractive where if an organization posts that the job title for their hiring is a red team, but the responsibility of it is actually a pen tester, the pen test is a motor better to take that role. Because a lot of people do want to move into that space. But ultimately, if your responsibility on your day-to-day duties is to not perform evasive actions, assess web applications and review third parties reports of red teams, are you actually a red teamer? Probably not. Another important thing is that a part of being a red teamer requires having a really creative approach to developing a touch-chains to simulate adversaries. And the best way that that creativity is grown and nourished and developed is reviewing many different organizations' environments. And so it kind of comes to the question of should you even have an internal red team, like a real internal red team, because in that internal position, if you're already, red-turning the same company over and over and over again. One, you might struggle to actually emulate an adversary that has no knowledge of the environment because you do have insider knowledge. And two, you're seeing the same issues over and over again, and fixing the same things. Whereas an adversary that has compromised many other organizations or an external red team that's compromised many different organizations, might have a different unique and creative approach to breaching your organization. And so, yeah, what do you think about that? - No, I completely agree. And I think I've got this philosophy that I think clients should always make sure that so you might get a pen test done by one pen tester one round, the next round. It's in your best interests to have a different pen test that actually look at your environment. Because there's so much creativity and how I have so many different attack types and everyone has different specialities in different areas. It's important to get multiple lenses on the one problem. Like this isn't the sort of thing where it's, you go get a blood test and they measure one if you're hormones or a mineral in your blood. And it comes to the same outcome each time with the same input. You could have one pen test to look at your environment. They find absolutely nothing. And you have another one and they find this critical vulnerability which every past pen test for the last 10 years is missed. But it's a gaping hole and it lets someone straight into your organization. Very important point. Great. I think we can probably move forward now into a few other areas that we thought would be important discuss. And I think we'd like to cover another disconnect that we see between internal security teams, like the security function of an organization, and then our source red teams. And a lot of the time security functions when they're developing RFP to engage a third party to deliver a red team. They have a lot of good intentions. Understanding they're seeing an industry, different industry verticals or similar verticals being compromised by certain ransomware groups or certain APTs. May want to understand more, could we for victim with the same attacks? And so they asked to emulate a ransomware group. Can you talk about more about that from your perspective? Yeah, definitely. So I've definitely seen scopes where it's requested for the red team to emulate, say, ContiRants and Wear TTPs. I think this one was quite common probably a year or so ago. And all that does is restrict the red team to essentially emulate or simulate what we know the ContiRants and Wear operators do. And that's not really what happens on a red team. What happens is you get your foothold, and you look at, OK, what do I have available in the environment? And let's say one ContiRants and Wear TTP would be they use any desk or something to come in. But you're coming in via-- you've got a co-bolt strike beacon, or you've come in via team viewer, or you've called someone up, and you've managed to get them to open Microsoft Wikicast or something like that. You've got a foothold. It's a bloody big risk in the organization, but it's not emulating the TTPs. And so once you've got that foothold, you're going to start exploring the environment. All right, how are administrative privileges handled? Do they use things like Active Directory Certificate Services? Do their file shares, are they littered with administrative credentials all over the place? What's their SCCM deployment look like? All those sort of things to try and get from point A to point B undetected. And the TTPs just limit the red team so much in terms of what they can actually do. They can't monitor it to high sophistication, sophistication, broker that's trying to gain privileges in an environment. I don't think they're going to limit themselves to what's on a playbook, you know? For sure. And I think, particularly with ransomware groups, it's important, you know, some security teams might not understand that from the point of having no access to ransomware deployed, there are actually multiple conductors that are working together to achieve that final impact on the objectives. So the way that the initial foothold was obtained could be an initial access broke out of selling credentials to a pen tester, not a genuine pen tester, but a producter. And those credentials could have been attained from previous unrated fair-party data breaches. Could be information civil malware and employees personal device. Or it could be brute forcing ID web on the internet. It really doesn't matter. That product will sell those initial access credentials for $500,000 or whatever it is. To a ransomware affiliate, the hacker, who has the objective of escalating from low-provision man user to a position where they can deploy ransomware in every workstation and engage the ransomware as a service ecosystem, whether that be a content club or any other third-party ransomware provider to deploy the malicious software. So to say that you can simulate X ransomware group, the reality is, is that the affiliates work with all different ransomware groups. And the affiliates basically take the path of least resistance. They're on the network. How can they get to a domain admin? Is there a misconfigured active directory certificate service? Is there credentials in a file share? Ultimately, they don't care how they get there. There's not necessarily a playbook. Their playbook might just be the list of quick wins followed by the harder steps. And so when you want to emulate a ransomware group, the group is actually classified as all the affiliates that have their own methodologies within that. So you better off just probably doing a red team than trying to emulate a very specific ransomware group. But when it comes to emulating advanced persistent threats, would your approach differ? And do you think it's still important to consider the TTPs that an advanced persistent threat would use? It might have a more structured playbook for how they compromise an organization. Yeah, that's a really, really good and really tough question, I think. So when we sort of distinguish between advanced persistent threats and ransomware operators, let's say, I think we need to look at the motivations of each group. So ransomware operators are more financially motivated, more like a sort of smash and grab bank-class sort of thing. So they just want to get in there, build privilege, deploy the ransomware and get their payout. Whereas your advanced persistent threats, they're probably going to take the long road, take a more quiet, careful tactical approach. And because of that, the amount of noise and sort of artifacts that they're dropping in the environment will be a lot less. So I still think that the MITRE attack framework TTPs are pretty important to cover because they are known exploitation parts. But I think in terms of trying to emulate advanced persistent threats, it's very difficult as a red teamer, because we just-- we realistically don't have the same resources, the same time, the same money access to the custom tools that they use. These teams might be using zero data exploits, and that's sort of thing to get from point A to point B. I think where we can add value as red teamers there is focusing on more detection side of things. So more complex attack parts, more solutions, which aren't just drop a tool, run the tool, get the outcome. Trying to go for more bespoke systems and that sort of thing, and seeing how well the security team can ingest the logs from those sort of systems correlate different events together. And I think that would give a better representation of how the business would handle an APT based attack. Yeah, cool. I think what we can do now is we give a summary, we'll form a move on into further discussion points of the issues we've discussed, and really what our recommendations are for the security industry to improve how security insurance activities are conducted. So the first thing we discussed was how-- particularly for external network pentes, how tightly-insciting engagements can impede the quality and the depth of testing to truly evaluate that question of whether an adverse recon breach of perimeter. And the solution, basically, is take a wildcard approach, allow the tester to evaluate any IP and any subdomain of the external perimeter. Obviously, don't let them just go hacking random stuff. And anything that needs to be introduced into scope should be done with consent and approval before subsequent testing. But ultimately, the scope should be wide, and the scope should effectively be the full public perimeter of the organization. Regarding the issue we discussed as well around pentes being time-bound exercises, it's really critical that we don't add in constraints for testing methodology. Obviously, when it comes to performing exploits on systems that might be mission critical, there might be reasonable constraints that can be applied, like testing it in a different environment, like a staging environment, or restricting the testing of that system to a different time zone, where if there is downtime, it doesn't impact the business operations on the day to day. It's like testing in the evening, for example. What are some other things in a summary that people should be doing to ensure that their testing constraints doesn't impact the quality of the service? I think it all comes down to communication and trust. So we hope that you're using your budget high. reputable consultants within the industry, but please trust your consultants. I know that they can see a lot of the time get access to sensitive information and sometimes make IT departments not look in the best light, but we do have the same common objective. We want to try and reduce risk so you can find vulnerabilities, reduce risk in your environment, so you can close those gaps before a threat actor comes across and tries to exploit them. So I think things like having an open communication channel while your consultants are doing a engagement is very, very effective, doing regular check-ins, whether it's a five minute stand-up each day or just message summary. And don't try and a successful delivery manager isn't one who's trying to get the time down to the minimum amount that someone's willing to agree to do the pen test for. If we're saying we think that this perimeter needs this amount of time to assess at a reasonable level, there's a reason for that. Yeah, 100% agreed. And one of the other constraints we mentioned was not satisfied pre-represents on time. So for a solution that is scheduling your test in advance, at least one month notice is best-sales outcome. But even if you're doing the two-weeks notice, showing that you have those prerequisites of what you need and that you're providing them at least three business days prior to test commencing. I guess for, let me talk about the red team component as well. It's really critical that all low-hanging through the environment has been assessed and only moving to conducting a red team when you've already had your perimeter evaluated and the internal network assessed where there's no critical or high responings being discovered in that time-bound exercise. So that when you move into the much more extensive time allocation of one of those assessments, the red team is going to dive deep and see what they can find rather than just exploiting common security issues. I think one thing to add to that is it's more than likely that you will have some findings in your environment which someone has flagged as high or critical. Right? So let's say we have an internal web application and it's got a SQL injection vulnerability on it and it's got a level of information in it and someone's assessed it as part of web application penta and said this finding is critical. Right? That's not, that doesn't mean that you do not red team or if you've got an active directory certificate services template and a specific group can use that to exploit what's known as ESC1. You know if you're aware of it and you have some sort of detection or mitigation, some sort of maturity which demonstrates that you are across these things. I think that's a good point where you should red team. In terms of vulnerabilities which are vulnerability scanners, flagging is critical, where that penta is clamped, flagging is critical, it's always important to consider what the risk is to the wider organisation. Those risks have been applied from a technical risk perspective specific to that system just because it's saying it's critical there. It's necessary saying this window is patched is missing and it's critical. I mean you need to take that into consideration of course but it doesn't mean your businesses are at a critical risk of getting done by around somewhere off-road. 100% about understanding that difference between the technical risk and the organisational risk using organisational risk definitions for things like critical high-medium, low and so forth. But yeah I think we've had a good first episode rounding out this topic of Deficient, Eason, Security, Assurance, Actories. We'll see you next time.

Podcast Summary

Key Points:

  1. External penetration tests often fail to validate the full public perimeter, relying on client-provided scopes that may be incomplete or outdated.
  2. Scoping constraints like static IP lists, lack of wildcard domains, and exclusion of discovery activities lead to a false sense of security.
  3. Time-bound testing is further degraded by delays in prerequisites (e.g., credentials, allowlisting) and unrealistic constraints like requiring command-by-command approval.
  4. Misalignment between what organizations want (e.g., vulnerability scanning vs. realistic attack path simulation) reduces the effectiveness of security assurance.
  5. Collaborative scoping, including discovery phases and clear communication of out-of-scope activities (e.g., phishing), is essential for accurate risk assessment.

Summary:

The transcript highlights critical gaps in how organizations conduct offensive security activities, particularly external penetration tests, to achieve true security assurance. , IP lists that are outdated or represent only a fraction of the real attack surface) and failing to include discovery phases to validate their perimeter. This leads to a false sense of security, as testers are constrained to narrow scopes while adversaries have unlimited time and resources.

, credentials or allowlisting) that reduces effective testing effort by up to 33%. , breach simulation vs. technical audit).

They advocate for collaborative scoping with wildcard domains, subsidiary discovery, and advance planning to ensure tests are thorough and meaningful. Ultimately, without addressing these gaps, organizations risk misallocating resources and failing to reduce their likelihood of compromise.

FAQs

Security assurance aims to provide an organization with confidence that its security controls and defenses are effective in reducing risk and capable of preventing or detecting potential threats.

Without validating the scope, organizations may provide an inaccurate public perimeter, leading to a false sense of security. Penetration testers may assess non-existent or irrelevant assets, missing real attack surfaces like dynamic IPs or unknown cloud resources.

IP addresses may be dynamic and change after scoping, or they could be reverse proxies hiding hundreds of websites. This misrepresents the actual attack surface and limits the test's effectiveness.

Include a discovery activity before testing to identify all assets, provide wildcard scopes for subdomains, and share entity names for subsidiaries. This ensures the full external perimeter is assessed.

A penetration test finds realistic attack paths to gain internal access, while a vulnerability assessment scans for misconfigurations and outdated software. Organizations should clarify which they need to avoid false assurance.

Such constraints waste time and reduce test quality, as testers cannot predict every command needed. Trust in the testing team is essential for effective assessment.

Chat with AI

Loading...

Pro features

Go deeper with this episode

Unlock creator-grade tools that turn any transcript into show notes and subtitle files.