Guest post originally published on Deepfactor’s blog by Seth Knox, Chief Marketing Officer at Deepfactor

How DevSecOps and Developer Security Can Reduce Risk, Accelerate Release Velocity, and Save Developers Time

Over the 20+ years I’ve been working in the cybersecurity industry, I’ve been a SANS student, speaker, and sponsor and found it a great resource across many cybersecurity topics. Given how early in the DevSecOps and Developer Security journey we are as an industry, there are very few sources for best practices data sourced from practitioners. So, when we got the opportunity to participate in this year’s report, SANS 2022 DevSecOps Survey: Creating a Culture to Significantly Improve Your Organization’s Security PostureDeepfactor was all in. The report and webinar dropped last week (9/21/2022) with updated annual data from the last survey.

In this blog, I am going to share 5 key takeaways I had from the report along with some observations of DevSecOps best practices and success criteria from our interactions with Deepfactor customers.

1. Picking the Right DevSecOps Key Performance Indicators (KPIs)

“You can’t improve what you don’t measure.” – Peter Drucker

This classic Peter Drucker quote certainly applies to DevSecOps. When the SANS Survey asked the question, “What are the Major KPIs you use to measure the success of your DevSecOps activities?” there were some great KPI options to choose from and I found the changes from 2021 to 2022 data interesting.

Chart showing major KPIs respondents use to measure the success of their DevSecOps activities in 2021 and 2022

I break these KPIs down into three themes that align with the fundamental value proposition of DevSecOps:

Risk Reduction

The DevSecOps impact on risk has several angles reflected in the KPIs above:

Release Velocity

Line chart showing most of the respondents choose weekly as their frequency of delivery to production within 2017, 2018, 2020, 2021 and 2022

Engineering Time/Cost Savings

2. The ROI and Time Savings of DevSecOps

According to the survey results, “management buy-in” was the number one factor contributing to DevSecOps security programs’ success. Quantifying the value in terms of time saved by catching security vulnerabilities early in Dev and Test is a core principle of DevSecOps that can be used to build an ROI case for management to support your DevSecOps program and help overcome the lack of management buy-in.

What if you could say, “For each vulnerability we find early in dev and test using DevSecOps, we save an average of X days of engineering time. We have Y number of  security vulnerabilities per year. So, investing Z amount would have a strong xxx% return on investment that pays for itself in less than a year.”?

The problem is that there is very little recent data on the amount of time it takes to resolve an average vulnerability after it is released to production compared to if it is identified during coding, component testing, system acceptance testing, or after being released to production. The chart below, originally from NIST and referenced in this deep source blog, shows the exponential cost savings that can be achieved the earlier a vulnerability is fixed.

Bar chart showing relative cost to fix bugs, based on time of detection

One of the fundamental principles of DevSecOps is to find and fix vulnerabilities early in development and testing because it is easier for developers to fix the vulnerability at that point when they are coding that component of the application and while it is fresh in their memory. It is much more difficult to fix a vulnerability for code that was developed months or years earlier, much less if it is a different developer fixing the issue. 

Through the SANS report and webinar, I found data that can help us start to build an example ROI case. Of course, the best ROI business cases will always use your own internal data that is most relevant to your applications and organization.

The first interesting piece of data relating to ROI was in the SANS survey but didn’t actually appear in the report (It would have been a very long report if all 31 charts in the survey were included). The chart below shows the answer to the question, “On Average, how long does it take for your organization to patch/resolve critical security risks/vulnerabilities for systems already in use?

Bar chart showing on average, around 27.3% of organization take 2-7 days to patch/resolve critical security risks/vulnerabilities for systems already in use

The weighted average number of days it takes to fix a vulnerability according to the SANS survey was 21 days, while the most common number of days was 2-7 days. Of course, this will vary by application and organization. 

During the SANS DevSecOps Report webinar, they conducted a poll on the associated slack channel that asked the following question:

How much more time on average does it take engineering and QA to fix and verify a security vulnerability after it has been released to production vs. fixing it in Dev/QA before the release?

Round chart showing around 33% of respondents take 10x for engineering and QA to fix and verify a security vulnerability after it has been released to production vs. fixing it in Dev/QA before the release

This poll, while not scientific, aligns well with the NIST chart above, with the most common response of “10 times more time” to fix a vulnerability after releasing to production. So, now we have an average time to fix a vulnerability after releasing to production (21 days), the increase/decrease in time spent when fixed in Dev/QA (10X). Now all we need is the average cost of a day’s time for a developer. According to U.S. News and world report, the average Software Developer makes $110,140 per year. Generally, benefits and overhead adds an additional 20%. So, let’s call it $132,168 as an annual cost to the company per developer. The average number of working days in a year is 260, which puts the cost per day of a developer at $508.

[Side note: If you know any good developers who make $110,140, Deepfactor would like to hire them.]

So, here is an example of a basic ROI case for DevSecOps:

While your mileage and the variables used above may vary, you can see that there is a compelling business case to be made for DevSecOps. In addition, there are also quantifiable cost savings that can be associated with reducing the risk of a breach, avoiding release delays, and the opportunity cost of developers being able to work on other revenue generating features or applications instead of spending more time fixing security vulnerabilities.

3. DevSecOps Key to Success

According to the Secure DevOps and Misconfigurations 2021 Report by the Cloud Security Alliance, only 30% of security professionals claim to have “fully implemented DevSecOps,” meaning the 70% of security professionals are still working on fully implementing DevSecOps. That’s why I found the following question posed by the SANS 2022 DevSecOps survey interesting:

Bar chart showing the top five factors that have contributed to respondent's security program's success in 2021 and 2022

Again, with so many options, I think it is helpful to break these success factors into three success-factor themes:

Culture and Education

When I talk with Deepfactor customers, shifting the culture of development to view security as part of delivering quality code is one of the most challenging and critical parts of successfully implementing DevSecOps. This requires a combination of both security and engineering management buy-in, individual developer level buy-in, communications between Developers, Security, and Ops teams, and security training that re-enforces secure coding practices.

Key Success Criteria Relating to Culture and Education:

For securing developer/engineering buy-in, many industry analysts and customers I’ve spoken with recommend implementing a “Security Champions Program” or “DevSecOps Center of Excellence” as a strategy. These programs recruit a developer from each team to act as a security champion and receive additional training that they can then impart to their peers and be an advocate for security embedded into each engineering team to speak up about security issues. For more reading on how to implement a security champions program, I recommend the following resource:

Forrester Report: Build a Developer Security Champions Program

When it comes to security training, Deepfactor customers have reported the best results through a combination of continuing education courses (See SANS SEC540 Cloud Security and DevSecOps Automation Course) and automated tooling that provides just-in-time alerts when developers implement applications in a way that introduces risk.


Tooling for DevSecOps is a critical piece of the puzzle to get DevSecOps right. There are several different types of security tooling required to provide complete coverage and reduce risk. One critical thing to remember about DevSecOps tooling is that it should integrate well into the existing engineering tool chains and not require developers to leave their existing tools. If you use Jira to track and prioritize bugs, then the tooling should integrate with Jira as the way for engineers to consume alerts and resolve issues. Don’t make your developer log into multiple other tools or you risk losing engineering/developer buy-in.

Key Success Criteria Relating to Culture and Education:

In the example below, Deepfactor provides a plain english description using an “Unsafe string API,” including detailed resolution information on how to accomplish the same thing in a more secure way, and the stack trace information to pinpoint the code that needs to be changed. This same information can be automatically assigned to an engineer through Jira or other tools using Deepfactor APIs. To learn more about Deepfactor, watch a 20-Minute Speed Demo of Deepfactor.

Screenshot showing Deepfactor page

Automation and Prioritization

One of the biggest challenges in securing developer buy-in is “Alert Fatigue.” With developers struggling to understand and triage “noise” and false positives, they spend more and more time reviewing and investigating alerts that provide a diminishing value to the organization. Overwhelming engineering teams with security alerts often leads developers to ignoring them entirely. In some cases, the flood of information can delay—and even disrupt—remediation efforts as developers struggle with understanding the scope and impact of the issue. We see the problem often in software composition analysis (SCA) tools designed to show a list of all vulnerable dependencies based on scanning a code repository. The solution to “Alert Fatigue” lies in automating the tooling, showing only alerts relevant to individual developers, and prioritizing based on risk and usage.

Key Success Criteria Relating to Culture and Education:

Make sure that your DevSecOps tooling is set up in a way that assigns only the alerts that are relevant to that developer and prioritizes those alerts automatically based on the risk of the vulnerability and usage of that component in the code path. There are often many vulnerabilities that exist in dependencies or components of the application that are never used and therefore should be a lower priority. The Deepfactor approach to prioritizing based on usage is to observe the running application in the QA or test environment and correlate which of the vulnerabilities existing in components that were used when QA is testing the application.

4. Automated Compliance Checking and Enforcement

“So I’ve recently started advising my clients not to write a single security policy without having a means to check it automatically. And this can mean things such as password complexity. Through the whole gamut. So, because we need to be able to inspect what we expect, and the only way that this is going to scale because our companies are not hiring more security engineers at the same pace that they’re hiring developers or employees is we put in a means to be able to do this in an automated manner.”

Kenneth G. Hartman
SANS 2022 DevSecOps Survey: Creating a Culture to Significantly Improve Your Organization’s Security Posture, 9/21/2022

Compliance is sometimes an afterthought when it comes to DevSecOps. Most compliance work is focused on the infrastructure and operational aspect of running applications but there are also compliance implications in the development of an application. Whether it’s SOC 2 Type 2, PCI DSS, ISO 27001, or another compliance standard that is important to your business, put some thought into building compliance checks and enforcement mechanisms into the DevSecOps process.

Bar chart showing compliance policies checked/enforced automatically in 2021 and 2022

In the SANS 2022 Survey data, you can see that there was a big jump from 2021 to 2022 in the number of respondents who checked/enforced 100% of their compliance policies. However, that percentage is still only 18.4% of the total that fall into that desirable category. Finding compliance issues in development or QA through DevSecOps will be much less costly to remediate and will avoid any delays or penalties that could result from a bad audit finding or compliance violation in production.

Deepfactor automatically maps all security vulnerabilities found in application code, dependencies, container images, and web/API interfaces to compliance standards and the section of the standard the vulnerability violates using the secure controls framework. With that information, developers and engineering leaders know immediately when a vulnerability alert has a compliance impact. To read more on this topic, visit What Developers Need to Know About the Impact of Compliance Frameworks on Software Development.

5. Most Useful Testing Practices and Tools

The SANS 2022 DevSecOps survey included asking respondents what security testing practices and tools they found useful on a scale of Not Useful, Useful, and Very Useful. There were 26 different security testing practices and tools ranked in the report and I encourage you to review them all on page 12-14 of the SANS report. Here are the top 5 most useful with the percentage of people who ranked the practice/tool very useful or useful based on the survey:

  1. Web Application Firewalls (WAF) (84%)
  2. Periodic Vulnerability Scanning (83.3%)
  3. Secure Coding Training For Developers and Engineers (82.3%
  4. Automated State Analysis (82%)
  5. Continuous Vulnerability Scanning (79%)

With so many tools and options, I think having this independent view of which tools are most useful is helpful in selecting the tool set and security practices you build your DevSecOps program around.

DevSecOps tooling is one of the areas where Deepfactor can help. Below is a list of the most useful security testing practices and tools from the SANS 2022 Survey that Deepfactor can help address:

Tool or PracticeSANS Survey Responses Useful Percentage
Periodic Vulnerability Scanning83.3%
Continuous Vulnerability Scanning79%
Upfront Risk Assessment Before Development Starts76.2%
Container/Image Security Scanning72.8%
Dynamic Application Security Testing (DAST)70.4%
Compliance Review or Audits by a Third Party70.1%
Interactive Application Security Testing (IAST)62.9%
Cloud Native Application Protection Platforms (CNAPP)59.5%


I’d like to thank SANS and the authors of the SANS 2022 DevSecOps Survey Chris Edmundson and Kenneth G. Hartman for providing such valuable information to the DevSecOps community so that organizations can apply best practices to fully realize the risk reduction, release velocity, and time/cost savings potential of DevSecOps. We are relatively early in the DevSecOps journey and there is a lot of work to do as an industry to complete the shift left represented by DevSecOps and improve our collective security posture. I hope the key takeaways I summarized from the report provided some helpful insights and was a catalyst to continue researching and implementing DevSecOps in your organization.

If you have any questions or I can help you in any way, please feel free to connect with me on LinkedIn.