Introduction 

In recent years, the number of Common Vulnerabilities and Exposures (CVEs) have been on a steady rise, reaching record highs in 2023. In fact, the number of reported CVEs in 2023 alone exceeded 20,000, highlighting the growing complexity and importance of software security in today’s interconnected world. Therefore, scanning source code for vulnerabilities is a critical aspect for organizations to protect their applications from cyber threats.  

Building upon the escalating number of reported CVEs and the critical need for software security, Veracode emerges as a vital solution. Veracode provides organizations with comprehensive tools and capabilities to detect and remediate security vulnerabilities in their applications. Organizations can leverage various scanning techniques such as  

• static analysis (SAST), 
• dynamic analysis (DAST), 
• software composition analysis (SCA) 

 In our project for a major European bank, we used Veracode within our CI/CD pipelines for scanning the services of our microservice architecture before a production roll out. We therefore will explain in this blog article,  

• how we scanned our microservice architecture in the past,  
• the limitations we faced with Veracode with this approach, and  
• how the new Veracode Collections feature, which we were able to utilize as Veracode customers in preview mode, addresses our past challenges by offering a scanning solution customized for microservice architecture. 

We will also take a closer look at the various scanning options, flaws and analyses and examine these in more detail based on our practical experience. 
Before we get to the challenges during our journey, we’ll take a step back and look at how to scan an application with Veracode in general. 

Scanning with Veracode in General 

To successfully perform an application scan with Veracode, three ingredients must be present. First and foremost, the application’s source code or compiled binary, as Veracode conducts scans on either executables or source code. Secondly, either the Veracode UI must be accessible via a web browser or API calls against the Veracode API are possible. Lastly, a clear understanding of the application’s functionality and potential security risk areas is essential to choose the right type of scan and interpret the results accurately. 

Given this information, an application profile must be created which in the context of Veracode is a detailed document or set of information that provides a comprehensive overview of the application being scanned. It typically includes details such as the technology stack used, programming languages employed, third-party libraries and frameworks integrated, specific compliance requirements, and any other relevant information necessary for the accurate scanning and evaluation of the application’s security posture. By creating a thorough application profile, development teams can ensure that the Veracode scanning process is tailored to their application’s unique characteristics and requirements, leading to more precise and meaningful results. 

When using Veracode to scan an application, development teams can leverage a variety of scan types to assess its security vulnerabilities. These include  

• Static Analysis, which examines the application’s source code for potential issues, 
• Dynamic Analysis, which tests the application in a running state to uncover vulnerabilities,  
• Software Composition Analysis, which identifies risks in third-party components, and 
• Manual Penetration Testing, where security experts simulate real-world attacks to detect weaknesses.  

By utilizing these diverse scan types, teams can thoroughly evaluate their application’s security posture and address any vulnerabilities effectively.

Policy Scan vs. Sandbox Scan 

Static scans, like static analysis and SCAs, can either run in a sandbox as a sandbox scan or directly on application level as a policy scan. The main difference between sandbox and policy scans is their focus and purpose. Sandboxing involves the isolation and analysis of potentially risky code or files to detect security threats, while policy scans focus on evaluating the application’s compliance with established security policies and standards. Both approaches play important roles in ensuring the security of software applications, with sandboxes providing a more proactive and exploratory method of identifying vulnerabilities, while policy scans help to ensure ongoing compliance with security standards. Sandbox scans can also be promoted to a policy scan later. 

Image 1: Application profile with three sandbox scans. The first sandbox scan is promoted as policy scan and defines the latest compliance status, which is notified to the central security team

Reports 

After a scan is completed, users receive a comprehensive report detailing the results of the various scan types. This report provides a detailed summary of vulnerabilities identified in scanned applications, including severity levels, recommended fixes, and compliance information. The user-friendly reports can be customized to prioritize remediation efforts and ensure compliance with security standards and regulations. They are essential tools for improving application security.

Scanning multiple applications 

For analyzing multiple applications (e.g. in the context of a microservice architecture) there are no strict requirements or restrictions that the Veracode platform enforces. For example, several applications can be scanned in one application profile, in their own application profiles or something in between. 
Depending on the variant, different challenges need to be addressed. One of those is the pricing model which depends on the number of application profiles. 

Key Considerations and Challenges in Scanning Microservice Architectures  

Before delving into the detailed analysis of how we assessed our architecture, it is essential to provide a comprehensive overview of key considerations for scanning a microservice architecture. We have outlined three distinct categories of challenges that organizations commonly encounter when evaluating how to scan their microservices: 

1. Complexity: One of the key considerations is understanding the complexity of the microservice architecture. Unlike monolithic applications, microservices are decentralized and can be scattered across various systems and platforms. This makes it crucial to have a clear understanding of the entire architecture and how different services interact with each other.
2. Adaption: Organizations must consider the challenge of scanning dynamically changing environments. Microservices are designed to be easily scalable and deployable, which means that new services can be added or removed at any time. This poses a challenge for traditional scanning tools, as they may not be able to keep up with the constantly changing environment.
3. Prioritization: Furthermore, organizations must also address the challenge of managing the extensive data generated from scanning microservice architectures. With many services and endpoints to scan, it can be overwhelming to sift through the results and prioritize remediation efforts. It is important to have a robust system in place to manage and prioritize security findings efficiently. 

Given these challenges, it is important to have a holistic overview of the software security status of all microservices in a microservice architecture for several reasons. Firstly, a holistic security assessment enables the identification of potential security gaps and vulnerabilities that could extend across different services and potentially have a serious impact on the entire application. In addition, such an overview helps organizations to effectively manage risk, as they can better understand risks and take appropriate mitigation measures. This also enables organizations to meet the required security standards, especially in industries with strict compliance requirements. By centrally managing and monitoring all microservices, security processes can be made more efficient, saving time, resources and costs in identifying and resolving security issues. Overall, a holistic view of the security status of all microservices helps to strengthen the security of a microservice architecture and ensure user confidence in the application. 

Considering the result of the assessment of having a holistic overview of the security status, we have two options to implement this using Veracode. The first approach involves utilizing a single Veracode application with multiple sandboxes, an approach we used in the past. The second, more recent option, is to leverage Veracode Collections, a new feature that exactly addresses our challenges with the first approach. We have transitioned from the former to the latter approach and will discuss both options in the following sections. 

Scanning a microservice architecture with Veracode Sandboxes 

Our initial setup off the Veracode platform, consisted of a single Veracode application. Multiple Veracode Sandboxes served as segregated environments for conducting security assessments and evaluations, with each sandbox being exclusively allocated for a specific microservice. 

Scan Types 

To conduct code scanning, we utilize various Veracode API actions. We distinguish between two types of scans that depend on the position in the development lifecycle, each of them performs a sandbox scan: 

1. Pipeline-scan: Referred to as a pipeline scan within our process, this type of scan evaluates the code, generates a Json file of the results, and attaches it to the pipeline for developers’ immediate access. It serves to provide developers with rapid feedback on their code during feature development. 
2. Upload-and-scan: This scan type leverages the Veracode uploadandscan action to upload and scan the code on the Veracode platform, presenting the results on the platform for further remediation actions. This type is best utilized when a feature team releases a new version of their microservice.

Furthermore, each scan type can be paired with a customized scan flavor specific to our repository structure: 

1. Image-scan: The source code is extracted from a docker container, packed into a zip file, and scanned using Veracode. 
2. Repository-Scan: A zip file is created from the development repository and subjected to scanning. 
3. File-scan: This scan method focuses on scanning individual files. 

Challenges arising from Sandbox Scanning 

However, the encountered challenges and technical constraints, mainly with our setup on the Veracode platform, were as follows: 

1. The constrained limit of allowed sandboxes posed a significant obstacle due to the scale of our application development.
2. There was a discrepancy in the use of Sandboxes, as they were intended for isolating environments for individual services but were inadvertently utilized for scanning release candidates, although a policy scan should be used for the central overview. 
3. The promotion of a Sandbox scan to a policy scan had implications for compliance, potentially inaccurately reflecting the overall application’s compliance. 
4. Limitations existed regarding the control of developer access to specific Sandboxes, only permitting access control at the application level, thus granting automatic access to all Sandboxes within that application, including those not developed by the individual. 

Image 2: Promotion of a Sandbox scan (a scan of a single microservice) to a policy scan for the entire Veracode Application

Compliance Workaround for Sandbox Scans 

When assessing this approach in conjunction with the discoveries outlined in the preceding chapter, wherein we underscored the significance of a comprehensive compliance overview of the application, it becomes evident that Veracode harbors considerable untapped potential in this area, a challenge which was subsequently addressed through the implementation of Veracode Collections. Especially that each microservice is scanned in an isolated environment and promoting only one sandbox out of many as a policy scan leads to an uncertain compliance status.  

To obtain exact certainty about the compliance status of our application we implemented a process where, before each Veracode scan, the artifact to be scanned is uploaded to a Google Cloud Bucket. Additionally, we have set up a separate GitHub workflow that runs automatically at defined intervals. This workflow downloads all artifacts stored in the Bucket, consolidates them into a single large zip file, and then conducts a Veracode Policy Scan on the consolidated file. 

Mentioning that this is exactly where Veracode Collections come into play, as they render this additional process unnecessary by consolidating the security status of multiple independent services into a shared compliance status. This is what we will delve into in the following section. 

Enhancing Security and Compliance Management with Veracode Collections 

In response to these limitations, Veracode introduced Veracode Collections. This feature addresses exactly the shortcomings of sandbox approach by allowing us to treat each Microservice as a separate application while still offering a consolidated view of the security status of our entire application.  

Veracode Collections Overview 

A Collection in Veracode consists of assets, where each asset is an individual Veracode application that corresponds to a specific Microservice. This structure allows for each microservice to be scanned individually, while monitoring the security status of the entire system is possible. Various assessment outcomes exist from assessments of assets, indicating varying levels of security vulnerabilities. Collections are classified as “Compliant” or “Not Compliant” based on the aggregated status of the assets within, providing valuable insights for organizations to implement tailored security enhancements in microservice architectures. 

Migration from Sandbox Scanning to Collection 

Since a collection is a dashboard that bundles several applications, the migration of our pipelines to Veracode Collections proved to be a seamless process, as minimal adjustments were required. The Application ID in the API call needed to be changed and assets needed to be created on the Veracode platform for each microservices. Since our scan actions operate within a container, the feature teams only had to increment the image version in their pipelines.  

The migration to Collections also improved the access management and ensured that individuals had the appropriate level of access to the respective applications in the microservice architecture. Teams can be created on the Veracode platform, individuals can be assigned to these teams, and ultimately, one or more services can be assigned to a team. This approach allows for a structured and role-based access control system, where each developer only has access to the applications for which they are responsible.  

Importance of Sandbox Scanning after Migrating to Collections 

While working with Veracode for some time, our focus has been on ensuring that the compliance status of a collection accurately reflects the current state of deployed resources in production. In this regard, utilizing Sandbox scans as part of the process is crucial. It is imperative to perform a Sandbox scan for every new feature or version of a Microservice before deployment in any environment. Subsequently, transitioning the Sandbox scan to a Policy scan upon service update in the production environment is key to establishing the service’s compliance and influencing the overall compliance of the Collection. Directly scanning each service as a Policy scan significantly impacts the Collection’s adherence to compliance standards. This emphasizes the importance of rigorously assessing services, including those in development stages or not fully developed, which could potentially impact a Collection’s compliance status. Additionally, discrepancies may arise, such as the release of a previous version instead of the latest one. If the earlier version has been elevated to a Policy scan, the Collection’s compliance status may not align with the current production environment. 

Conclusion 

The integration of Veracode Collections for scanning Microservice architectures has proven to be a significant advancement in enhancing security and compliance measures. By effectively utilizing Sandbox scans and implementing Identity and Access Management within Veracode Collections, organizations can streamline security assessments and access control processes. The identified challenges and technical constraints before usage of Veracode Collections underscore the necessity for a more structured and tailored approach to security scanning. Veracode Collections offer a solution that allows for individualized monitoring of Microservices while providing a consolidated view of the application’s security posture. The diverse functionalities and benefits of Veracode Collections are instrumental in bolstering security practices, ensuring regulatory compliance, and safeguarding digital assets in today’s dynamic technological landscape. 

Further Reading 

• Using the development sandbox, Veracode Docs: https://docs.veracode.com/r/c_about_sandbox 
• Application security policies, Veracode Docs: https://docs.veracode.com/r/c_appsec_policies 
• uploadandscan composite action, Veracode Docs: https://docs.veracode.com/r/r_uploadandscan 
• What is an Application Profile and what to consider when creating one? , Veracode Community: https://community.veracode.com/s/article/application-profile-and-sandbox-best-practices 
• Open source Software Vulnerability mitigation through automated dependency management, Senacor Blog: https://senacor.blog/open-source-software-vulnerability-mitigation-through-automated-dependency-management/