Jenkins Security: Keeping Your CI/CD Pipeline Safe and Secure
Jenkins Security: Keeping Your CI/CD Pipeline Safe and Secure. Jenkins is an automation server commonly used for continuous integration and continuous delivery (CI/CD). It runs on multiple platforms such as Linux, macOS, and Windows. Besides, it provides a web interface for managing your builds. Jenkins streamlines the software development process, from building and testing to deployment. It helps DevOps teams to collaborate on projects and create applications faster.
To run secure CI/CD pipelines you have to ensure you have proper security configurations in place. Jenkins holds valuable code scripts, and its security is critical for secure software development. This article discusses Jenkins Security: Keeping Your CI/CD Pipeline Safe and Secure.
Jenkins Security: Keeping Your CI/CD Pipeline Safe and Secure
1. Restrict Access Using Role-Based Access Control
Using Role Based Access Control (RBAC) in Jenkins manages user permissions. RBAC allows admins to assign permissions to roles instead of individual users. Useful in organizations with a large number of users. Since Jenkins handles sensitive information, it is crucial to limit users to access resources they only need to perform their tasks. Configuring access control ensures users access data as per their roles, and not beyond that. This helps protect your Jenkins CI/CD pipeline from unauthorized changes to configurations or even potential breaches.
Restricting access involves viewing configuration details, starting builds, or installing plugins or modifying system settings. Manage it through Jenkins’ built-in security settings or third-party plugins that provide advanced access control features.
Besides, review and update access control regularly.
2. Use Content Security Policy
Content Security Policy (CSP) is a security standard that helps protect Jenkins pipelines from cross site scripting (XSS) attacks. Basically, it is an HTTP response header to static files with restrictive default permissions that protect Jenkins from malicious HTML/JS files. Define a CSP to restrict the specific resources that are loaded by the user’s browser when accessing the Jenkins UI.
Here is the Jenkins default for the Content Security Policy:
sandbox; default-src 'none'; img-src 'self'; style-src 'self';
These rules prevent running Javascript, using inline CSS, or web fonts. By enforcing a strict CSP, you significantly mitigate XSS attacks, which occur when malicious code scripts are injected into web pages. When defining your CSP, be as specific as possible in order to limit potential attack vectors.
3. Use Secrets Management Tools
Secrets are authentication credentials used during software development. These include usernames, passwords, SSH keys, encryption keys, and API tokens. These secrets carry crucial authentication data, and if left exposed, can lead to unauthorized system access. Luckily, there are lots of secret management tools to help you secure your Jenkins pipeline. These tools include HashiCorp Vault, AWS Secrets Manager, Keywhiz, Strongbox, and Azure Key Vault, which securely stores and manages these secrets.
Secret management tools have a variety of features that enforce security. They also address security security standards and help meet regulatory compliance.
4. Regularly Audit and Rotate Secrets
Regularly auditing and rotating of secrets helps tighten the security of your Jenkins CI/CD pipelines. You need to regularly review your secrets to ensure that they maintain security and are only being used where necessary. Through regular auditing, you identify any anomalies or unauthorized use of secrets, enabling you to act quickly.
Also rotate secrets at regular intervals. That reduces the risk of exposure due to prolonged use. Secrets that are always in use for long may end up being discovered or misused.
5. Avoid Building From the Master Node
One of the Jenkins security best practices is to avoid building on the master node (controller). Building on master comes with potential security shortcomings as such builds can read or modify files in $JENKINS_HOME. This includes all sorts of activities, from the Jenkins installation and creating new jobs to reading credentials and installing plugins. It’s best to configure the master node to have no executors.
This configuration forces the master node not to run any builds. Alternatively, you use a plugin such as Job Restrictions Plugin to limit the jobs that run of certain nodes such as the Master node.
6. Implement Infrastructure as Code (IaC)
Another crucial security best practice is to implement infrastructure as code. Basically, provisioning and managing infrastructure using machine readable files. These files allow you to automate the process of setting up, changing, and versioning your CI/CD infrastructure.
The IaC approach ensures consistency across your infrastructure. That reduces the risk of security issues caused by misconfigurations. Moreover, it you easily track and review changes to your infrastructure. In addition, use pre-approved, version-controlled templates for your infrastructure setup. This minimizes the likelihood of unapproved or insecure configurations.
7. Filter Unsafe Environment Variables
Environment variables in Jenkins are used to pass parameters to build scripts. They have multiple security implications, such as affecting the behaviour of builds. For instance, if a pull request is built, environment variables may contain sensitive information like the pull request ID or name, which can be exploited if not handled correctly. Therefore, Jenkins administrators need to pay careful attention to the potential issues surrounding environment variables.
Special environment variables significantly change the behaviour of build scripts. A key example of such variables is PATH (or Path in Windows systems), which is used to locate the programs executed during a build. If overridden, it causes unexpected behaviours by launching different programs than expected. Other less familiar variables, like LD_PRELOAD on Linux or DYLD_LIBRARY_PATH on macOS have similar effects. As a result, scrutinize plugins abilities to add arbitrary environment variables from sources that don’t require permission to configure jobs.
Besides, you need to filter or sanitize environment variable values. For instance, Windows Batch’s cmd.exe evaluates the values of variables as they’re read. In the event that a variable value contains valid Batch commands, Jenkins might execute these commands. To mitigate this implement global filters for environment variables. Also, you use plugins such as Safe Batch Environment Filter and Generic Build Step Environment Filters to filter out unsafe environment variables.
8. Run Jenkins with Least Privileges
Implement the principle of least privilege. Each Jenkins instance has various permissions that are granted to users. These permissions include Overall/Read, Overall/Administer, Overall/SystemRead and Overall/Manage.
The default permissions include Overall/Read, which grants access to some Jenkins features and APIs. The Overall/Administer permission grants unlimited access to Jenkins. The highest level of permission, allowing users to install plugins, run arbitrary code, and even alter the Jenkins configuration. For safety, limit the level of access to only those who absolutely need it.
Optional permissions like Overall/SystemRead permission provides read only access to the global config of Jenkins. Useful in scenarios where the Jenkins configuration is managed externally. Another optional permission, Overall/Manage, allows for access and modifications to a subset of administrative options, providing a middle ground between the Overall/Read and Overall/Administer permissions.
9. Monitor Jenkins Environment and Analyze Logs
Well, Jenkins CI/CD logging and monitoring is crucial for building secure projects. Monitoring Jenkins to detect any anomalies. In addition, logs provide detailed information of all events in your system. The combinations of both monitoring and logging helps detect suspicious activity, troubleshoot issues, and investigate security incidents.
There are various ways to monitor your CI/CD pipelines. First, you use built in plugins such as Versions Node Monitors, Job/Queue/Agents Monitoring, and Slave Monitor. On the other hand, you use server monitoring tools such as Prometheus and Grafana, New Relic, and Datadog, all designed to provide display metrics about your Jenkins environment.
Collecting log data is also as essential as monitoring, as you review logs later to detect the origin of issues in your Jenkins. Jenkins has various plugins for logging, but also use third party log analysis tools. They include information about system events, user activities, build results, and error messages. By analysing this data, you identify patterns and trends that might indicate a security issue.
10. Disable SSHD
SSH is a network protocol for secure data communication, but also leads to unauthorized access to your systems. By disabling SSHD in Jenkins, you turn off the Secure Shell (SSH) service on the Jenkins master node. When you enable this configuration, you block unauthorized access via the SSH protocol.
11. Implement CSRF Protection
Cross-Site Request Forgery (CSRF) where an attacker tricks a victim into performing an action that the victim did not intend to do. For Jenkins, this could mean making changes to jobs, executing builds, or even modifying the Jenkins configuration. A CSRF attacker with Overall/Read permission connects to their specified URLs. Jenkins offers CSRF protection through a mechanism called a crumb, a unique token created for each user session. This token must be provided with any form submissions or actions resulting in changes to the Jenkins instance.
This protection ensures that actions cannot be carried out with another user’s token and provides a safeguard against CSRF attacks.
However, some outdated plugins that send HTTP requests to Jenkins do not function properly with CSRF protection enabled. In this case, it’s necessary to disable CSRF protection temporarily. However, you should avoid this scenario as much as possible, as it renders the specific Jenkins instance unsafe.
12. Enable Markup Formatters
The Markup Formatter is a feature in Jenkins that allows users with the right permissions to input descriptions for various objects such as builds, views, jobs, etc. This is crucial for two reasons; it enhances user experience by allowing users to incorporate rich formatting in their descriptions. Also, it shields users from the threat of Cross Site Scripting (XSS) attacks. Set up the markup formatter by applying filtering to the descriptions, which ensures that harmful content does not pass through.
During Markup Formatter configurations, there are several options available, including the default option of Plain Text. This option filters and renders all descriptions as entered. It escapes unsafe HTML metacharacters such as < and &, and converts line breaks into HTML <br/> tags. Another common selection is the Safe HTML option, which is provided by the OWASP Markup Formatter Plugin. This setting permits the use of a restricted, yet safe, subset of HTML.
However, Markup formatters come with security implications. For instance, if a formatter permits arbitrary HTML, it may be unsafe. While permissions such as Job/Configure and Build/Update are limited to fully trusted users, any user with an account edits their profile description, which is formatted by the chosen markup formatter. Mostly, this risk is higher on public Jenkins instances utilizing tools such as GitHub, GitLab, or Google accounts, where anyone could potentially login and edit their profile.
13. Use SSL/TLS for Encryption
SSL/TLS encryption helps provide data confidentiality and integrity. This SSL encrypts data in transit, meaning unauthorized users cannot decrypt it. Whenever a hacker intercepts traffic between the Jenkins server and the user, they can’t interpret it unless decrypted.
Well, SSL/TLS helps in ensuring that deployment commands reach the correct servers without being manipulated, further safeguarding your infrastructure. Besides using SSL, ensure you have a central authentication system such as Active Directory or LDAP.
14. Keep Jenkins Updated and Use Latest LTS (Long Term Support) Version
Use the latest version of Jenkins. It comes with security patches for any known vulnerabilities alongside performance upgrades. Also, it’s ideal to maintain the long term support (LTS) version. LTS versions of software are stable releases that are supported for a longer period, typically with security patches and bug fixes. By using the latest Jenkins LTS version, you get all the security fixes introduced since the previous versions.
15. Use a Matrix Based Security Approach
Matrix-based security is an approach that provides granular control over the users and groups that perform specific actions within the Jenkins environment. This approach, determines what each user can do on a detailed level. This feature helps you achieve greater control over your Jenkins, especially in large teams or complex projects.
The Project based Matrix Authorization Strategy is an extension of Matrix based security. It allows for additional access control lists (ACLs) to be defined for each project individually through the Project configuration screen. This way, specific users or groups are granted access only to designated projects. The ACLs defined with Project-based Matrix Authorization are additive, meaning that the access grants defined in the Security screen will be combined with the project-specific ACLs. Both Matrix based security and the Project-based Matrix Authorization Strategy are provided by the Matrix Authorization Strategy Plugin.
In essence, matrix Authorization allows for configuration of the lowest level permissions, such as starting new builds, configuring items, or deleting them, individually. Project-based Matrix Authorization also allows permissions for each item or agent to be configured independently.
Jenkins Security: Keeping Your CI/CD Pipeline Safe and Secure Conclusion
In the modern day world when DevOps is crucial to proper software development, Jenkins plays a key role in streamlining the entire build-to-deployment process. Therefore, it’s crucial to implement the various security best practices to secure Jenkins. All the above configurations come in handy and ensure minimize your server attack service as much as possible. By following these practices, you significantly reduce all associated risks and protect builds from data breaches.
