Are you familiar with COBIT's DevOps Audit Program, a framework for IT auditors to review an organization's development and service processes in a DevOps environment? It's an extension of the traditional IT audit framework, so the domain areas are the same, but some things are different, and I'll try to combine what I know and interpret what needs to be added for each domain.

First, before we get into that, let's review the DevOps characteristics.

# DevOps features and associated risks

If you haven't read the article below, I think it would be helpful to do so first.

 - http://tokenterrace.com/eng/197?sca=IT+Audit

1. It relies heavily on automation

DevOps is all about streamlining processes and efficiency in order to be agile in the marketplace and provide fast response and good quality to customers. For automation and orchestration, you need expertise in tools like Jenkins, Ansible, Chef, and if your organization uses the cloud, you need expertise in cloud platforms. 

[Example: Chef Automation Tool]

Source: https://intellipaat.com/blog/what-is-chef/ 

A major part of DevOps thinking is continuous integration (CI) and continuous delivery (CD), which is the process of automatically building and testing code as developers make changes, and continuous deployment (CD) is the process of automatically deploying code changes to production. Continuous testing is the process of continuously testing code changes throughout the development process, while constantly monitoring the performance of the application and infrastructure to ensure that any issues or errors are acted upon immediately. 

In general, there's less risk in something that's done automatically, assuming it's set up well, than there is in something that's manually controlled by a human. Computers are often more accurate than human error.

However, even with automated processes, problems can arise if they are operated without proper controls. 

IT auditors should consider evaluating the automation of development and deployment processes, as well as test automation. Automation of code changes and deployments is a key element of this process, but it should also include how previous versions are managed in case a deployment goes wrong, what logic is used to automate deployments, and what elements of the process are subject to human intervention.

Based on the foregoing, the audit methodology should consider whether the IT controls designed as a baseline for a traditional organization are sufficiently mitigated by the controls in the automated processes designed by the company, and the controls should be designed and operationally assessed. If they are insufficient, realistic and appropriate improvements should be proposed considering the company's market environment and business processes.

2. Cultural differences 

DevOps aims for collaboration between development and operations organizations. In this process, the boundaries between development and operations are blurred, which can lead to unclear roles and responsibilities between developers and operators, resulting in blurred roles and responsibilities for development, operations, security, etc. 

As a result, IT auditors should assess whether segregation of duties is clearly established, and evaluate whether the company is performing SoD monitoring on a regular basis. In addition, change management and remediation processes (e.g., monitoring controls) may differ from a typical organization and should be a focus. While a traditional organization would assess most of these areas, it may be more efficient to consider the IT risk as high due to the heavy reliance on automation, and to assess how the design of the fieldwork phase is prioritized over other controls.

Source: https://www.linkedin.com/pulse/safepaas-introduces-sod-scanner-segregation-duties-risk-adil-khan

3. Vulnerabilities

DevOps can increase the risk of vulnerabilities and security breaches due to the high pace of change for the purpose of rapid releases, which can lead to poor documentation. 

There is a risk that change management, vulnerability identification, and various remediation processes may not follow industry standards or comply with the company's internal policies.

Rather than simply identifying non-compliance as a deficiency, IT auditors should consider whether the company's internally designed, hands-on processes are sufficiently mitigating the IT risks addressed by the company's internal policies. Furthermore, it should recommend updating documents such as policies and guidelines to reflect current practices. 

4. Understand the various tools or cloud environments 

As described in the previous article, DevOps tends to utilize various tools or use cloud services to improve scalability, flexibility, and efficiency. 

As such, having an auditor with expertise in this area should be a top priority for audit organizations. The engaged IT auditor should review relevant matters such as data and infrastructure security, contracts with third-party vendors, service level agreements, etc. in a cloud environment.

# Domains in the IT governance framework COBIT 5

An IT audit is also an assessment and review of an organization's IT processes and controls, and COBIT 5 is an IT governance framework that can be effectively performed by understanding and auditing it.

The four domains of COBIT 5 are as follows

1) (APO) Align, Plan and Organize: Consistency and efficiency of development and service processes.

2) (BAI) Build, Acquire and Implement: Proper documentation of processes and compliance with regulations

3) (DSS) Deliver, Service and Support: Proper audit trails and reporting for development and service processes

4) (MEA) Monitor, Evaluate and Assess: Applying appropriate development and service process management and monitoring mechanisms

Source: COBIT 5 Area and Domain 

Because the four domains cover the overall management and operational processes of an IT organization, familiarity with them helps IT auditors quickly identify possible risks in each domain.

For example, if you understand the monitoring and evaluation domain, you can perform a review of IT performance measurement and monitoring and an assessment of your organization's compliance with regulations.

(In fact, these were probably already built into the methodology at each accounting firm, but as an IT auditor or internal control practitioner, you need to understand the context in which these controls were created.)

You may be wondering why I'm suddenly talking about COBIT 5.0, but it's important to understand these areas so that you can tailor your audit methodology to the unique circumstances of the organization you're auditing, rather than mechanically applying a one-size-fits-all approach. Did you know that there's even a framework specific to DevOps organizations? We'll explore that in our next post, "DevOps Audit Program".