Use Anchore Engine to improve the DevSecOps tool chain

Category: Tags: ,

I. Introduction
In recent years, containerization technology has developed rapidly, and major Internet vendors have also begun to use containerization technology, and how to ensure container security is one of the purposes of writing this article. One of the functions of Anchore Engine is to scan container images for vulnerabilities based on CVE data to find out whether there are security vulnerabilities and policy issues. This article will explain the use of Anchore Engine from two parts:

Based on the use of Anchore-cli client

Combine with Jenkins to perfect DevSecOps

2. Installation of Anchore Engine
Install using Docker Compose
Anchore Engine supports docker-compose or helm for installation. Here we use the simplest docker-compose for installation and testing.

1). The command is as follows:

# curl https://docs.anchore.com/current/docs/engine/quickstart/
docker-compose.yaml > docker-compose.yaml
# docker-compose up -d

2. Check whether the deployment is successful:

# docker-compose ps
        Name                      Command               State           Ports         
--------------------------------------------------------------------------------------
root_analyzer_1        /docker-entrypoint.sh anch ...   Up      8228/tcp              
root_api_1             /docker-entrypoint.sh anch ...   Up      0.0.0.0:8228->8228/tcp
root_catalog_1         /docker-entrypoint.sh anch ...   Up      8228/tcp              
root_db_1              docker-entrypoint.sh postgres    Up      5432/tcp              
root_policy-engine_1   /docker-entrypoint.sh anch ...   Up      8228/tcp              
root_queue_1           /docker-entrypoint.sh anch ...   Up      8228/tcp     
# docker-compose exec api anchore-cli system status
Service analyzer (anchore-quickstart, http://analyzer:8228): up
Service simplequeue (anchore-quickstart, http://queue:8228): up
Service apiext (anchore-quickstart, http://api:8228): up
Service policy_engine (anchore-quickstart, http://policy-engine:8228): up
Service catalog (anchore-quickstart, http://catalog:8228): up
Engine DB Version: 0.0.13
Engine Code Version: 0.7.1

3. Wait for the update of the vulnerability database to be completed. The update process may take a long time, a few hours or even a day is possible, please be patient. When the value of the RecordCount column is not None, the vulnerability database is even updated.

# docker-compose exec api anchore-cli system wait
Starting checks to wait for anchore-engine to be available timeout=-1.0 interval=5.0
API availability: Checking anchore-engine URL (http://localhost:8228)...
API availability: Success.
Service availability: Checking for service set (catalog,apiext,policy_engine,simplequeue,analyzer)...
Service availability: Success.
Feed sync: Checking sync completion for feed set (vulnerabilities)...
Feed sync: Checking sync completion for feed set (vulnerabilities)...
Feed sync: Checking sync completion for feed set (vulnerabilities)...
Feed sync: Checking sync completion for feed set (vulnerabilities)...
Feed sync: Checking sync completion for feed set (vulnerabilities)...
Feed sync: Checking sync completion for feed set (vulnerabilities)...
Feed sync: Checking sync completion for feed set (vulnerabilities)...
# docker-compose exec api anchore-cli system feeds list //Check the update status, the status is pending represents waiting for update
Feed                   Group                  LastSync                          RecordCount        
github                 github:composer        pending                           None               
github                 github:gem             pending                           None               
github                 github:java            pending                           None               
github                 github:npm             pending                           None               
github                 github:nuget           pending                           None               
github                 github:python          pending                           None               
nvdv2                  nvdv2:cves             pending                           None               
vulnerabilities        alpine:3.10            2020-06-22T03:08:25.647466        1725               
vulnerabilities        alpine:3.11            2020-06-22T03:08:44.098248        1904               
vulnerabilities        alpine:3.3             2020-06-22T03:09:03.977480        457                
vulnerabilities        alpine:3.4             2020-06-22T03:09:09.253906        681                
vulnerabilities        alpine:3.5             2020-06-22T03:09:16.594355        875                
vulnerabilities        alpine:3.6             pending                           1000               
vulnerabilities        alpine:3.7             pending                           None

3. Based on the use of Anchore-cli client
Scan the specified mirror
1). Add mirror to anchore

# docker-compose exec api anchore-cli image add docker.io/library/ubuntu:20.04
Image Digest: sha256:93fd0705706e5bdda6cc450b384d8d5afb18fecc19e054fe3d7a2c8c2aeb2c83
Parent Digest: sha256:52259450119427dab05c0c455121c48d7b04cee2d61b5dbdde1219b2163af572
Analysis Status: not_analyzed
Image Type: docker
Analyzed At: None
Image ID: 74435f89ab7825e19cf8c92c7b5c5ebd73ae2d0a2be16f49b3fb81c9062ab303
Dockerfile Mode: None
Distro: None
Distro Version: None
Size: None
Architecture: None
Layer Count: None
Full Tag: docker.io/library/ubuntu:20.04
Tag Detected At: 2020-06-22T07:19:18Z

2). View the scan status, Analysis Status is analyzed to indicate the end of the scan

# docker-compose exec api anchore-cli image list 
Full Tag                               Image Digest                                                                   Analysis Status                  
docker.io/library/ubuntu:20.04         sha256:93fd0705706e5bdda6cc450b384d8d5afb18fecc19e054fe3d7a2c8c2aeb2c83        analyzed

3). Then check the scan results

# docker-compose exec api anchore-cli image vuln docker.io/library/ubuntu:20.04 all
Vulnerability ID        Package                             Severity          Fix         CVE Refs                Vulnerability URL                                                     Type        Feed Group          Package Path        
CVE-2013-4235           login-1:4.8.1-1ubuntu5              Low               None        CVE-2013-4235           http://people.ubuntu.com/~ubuntu-security/cve/CVE-2013-4235           dpkg        ubuntu:20.04        pkgdb               
CVE-2013-4235           passwd-1:4.8.1-1ubuntu5             Low               None        CVE-2013-4235           http://people.ubuntu.com/~ubuntu-security/cve/CVE-2013-4235           dpkg        ubuntu:20.04        pkgdb               
CVE-2016-2781           coreutils-8.30-3ubuntu2             Low               None        CVE-2016-2781           http://people.ubuntu.com/~ubuntu-security/cve/CVE-2016-2781           dpkg        ubuntu:20.04        pkgdb               
CVE-2018-7169           login-1:4.8.1-1ubuntu5              Low               None        CVE-2018-7169           http://people.ubuntu.com/~ubuntu-security/cve/CVE-2018-7169           dpkg        ubuntu:20.04        pkgdb               
CVE-2018-7169           passwd-1:4.8.1-1ubuntu5             Low               None        CVE-2018-7169           http://people.ubuntu.com/~ubuntu-security/cve/CVE-2018-7169           dpkg        ubuntu:20.04        pkgdb               
CVE-2019-12904          libgcrypt20-1.8.5-5ubuntu1          Low               None        CVE-2019-12904          http://people.ubuntu.com/~ubuntu-security/cve/CVE-2019-12904          dpkg        ubuntu:20.04        pkgdb               
CVE-2019-13050          gpgv-2.2.19-3ubuntu2                Low               None        CVE-2019-13050          http://people.ubuntu.com/~ubuntu-security/cve/CVE-2019-13050          dpkg        ubuntu:20.04        pkgdb               
CVE-2019-18276          bash-5.0-6ubuntu1                   Low               None        CVE-2019-18276          http://people.ubuntu.com/~ubuntu-security/cve/CVE-2019-18276          dpkg        ubuntu:20.04        pkgdb

This is the simplest way to scan the image: add the image to the scan engine, wait for the scan to end, and then view the scan result. But this method is basically not suitable for actual work scenarios, so the next part of the article will explain how to apply Anchore Engine to actual work.

Four, combined with Jenkins and applied to DevSecOps
In the traditional development process, security work is usually carried out as the last step. As a result, once a problem is discovered, the cost of manpower and time for repair will be high, and usually security issues may affect not only development, but even changes in requirements and architecture. The purpose of moving “safety left” is to reduce the cost of repairing the problem.

At present, the rapid development of containerization technology has caused many manufacturers to change the traditional deployment methods and use a combination similar to K8S+jenkins+harbor. First look at the architecture of this classic combination:

1. Developers submit code to code warehouses such as gitlab

2. Download the latest code in the code warehouse by manually executing the jenkins build (or configure webhook to trigger the jenkins execution build)

3. Generate jar package through mvn compilation, and perform a series of static analysis, unit testing and other work

4. After the test is successful, start to build the jar package into a mirror through the docker build command

5. Push the generated image to the harbor mirror warehouse

6. Use k8s to pull the images on the harbor to create containers and services, and the final release is complete

According to the principle of safe shift left in DevSecOps, we choose to perform mirror scanning after the mirror is built in the fifth step. Below I will use an example to show how to use jenkins+anchore to realize automated mirror scanning.

Install plugin
Select Manage Plugins from the Jenkins main menu.

Click [Optional Plug-in], then enter [anchore] in the filter box, select [Anchore Container Image Scanner] and click [Direct Install].

Configure anchore plugin
Click [Manage Jenkins] -> [configure system], find [Anchore Container Image Scanner], and enter the detailed information of anchore engine:

Engine URL: The URL address of the anchore engine (the default is http://your_anchore_engine_IP:8228/v1)

Engine Username: Username (default admin)

Engine Password: Password (default foobar)

Add docker warehouse account
Click Credentials->System->Global Credentials->Add Credentials, add a hub.docker.com account, fill in the id casually, and fill in hub.docker.com here. Click [OK] to finish adding

Add scan mirroring to the pipeline
In this example, we will use the pipeline to build:

Create a new task in jenkins and select pipe line, enter the following script in [pipeline] and click save

pipeline {
    environment {
        registry = "zj1244/demo"  //Warehouse address, used to push the mirror to the mirror warehouse. Modify according to actual situation
        registryCredential = 'hub.docker.com'  //Credentials used to log in to the mirror warehouse, modified according to actual conditions
    }
    agent any
    stages {   
        //jenkins downloads code from the code repository
        stage('Cloning Git') {
            steps {
                git 'https://github.com/zj1244/docker-dvwa.git' 
                }
        }
        //Build image
        stage('Build Image') {
            steps {
                script {
                    app = docker.build(registry+ ":$BUILD_NUMBER")
                }
            }
        }
        //Push the image to the warehouse
        stage('Push Image') {
            steps {
                script {
                    docker.withRegistry('', registryCredential ) {
                        app.push()
                    }
                }
            }
        }
        //Mirror scan
        stage('Container Security Scan') {
            steps {
                sh 'echo "'+registry+':$BUILD_NUMBER `pwd`/Dockerfile" > anchore_images'
                anchore engineRetries: "240", name: 'anchore_images'
            }
        }
        stage('Cleanup') {
             steps {
             sh script: "docker rmi " + registry+ ":$BUILD_NUMBER"
             }
        }
    }
}

Finally, click [Build Now] to start building

View Results
After the build, click [Anchore Report (FAIL)] to view the scan report

The report will show whether the scan result is FAIL or PASS. By default, a vulnerability will cause the build to fail

Integration results
In actual work, there are often dozens of releases a day. It is obviously inconvenient to view the scan results on jenkins at this frequency, so it is necessary to have a graphical interface for statistics. The anchore enterprise version does provide a UI interface, but the enterprise version requires a fee, so I simply made a UI interface to integrate the scan results

 

To sum up
This article introduces some basic usage of Anchore Engine, including how to combine with jenkins, welcome to discuss. . Finally, give the github address of anchor ui.

https://github.com/zj1244/anchore_ui

 

Reviews

There are no reviews yet.

Be the first to review “Use Anchore Engine to improve the DevSecOps tool chain”

Your email address will not be published. Required fields are marked *