DevOps provides a set of Continuous Integration (CI) and Continuous Delivery (CD) tools that can automate processes between IT and software development teams. In a CI/CD workflow, each integration is verified through automated builds, including coding, release, and testing, helping developers detect integration errors early. Teams can also deliver internal software to production environments quickly, securely, and reliably.

However, the traditional Jenkins Controller-Agent architecture (where multiple Agents work for one Controller) has the following shortcomings:

  • If the Controller goes down, the entire CI/CD pipeline crashes.

  • Uneven resource allocation, where some Agents have pipeline Jobs queued and waiting, while other Agents are idle.

  • Different Agents may have different configuration environments and require different programming languages. This disparity can cause inconvenience in management and maintenance.

The DevOps extension supports source code management tools such as GitHub, GitLab, Git, and Bitbucket. It supports building CI/CD pipelines through a graphical editing panel (Jenkinsfile out of SCM) or creating pipelines based on Jenkinsfile from code repositories (Jenkinsfile in SCM).

Features

The DevOps extension provides the following features:

  • Independent DevOps projects, providing access-controlled CI/CD pipelines.

  • Out-of-the-box DevOps functionality, without complex Jenkins configuration.

  • Jenkinsfile-based pipelines, providing a consistent user experience and supporting multiple code repositories.

  • Graphical editing panel for creating pipelines with a low learning curve.

  • Powerful tool integration mechanisms, such as SonarQube for code quality checks.

  • Continuous delivery capabilities based on ArgoCD, automating deployment to multi-cluster environments.

DevOps Pipeline Workflow

The DevOps CI/CD pipeline runs based on underlying Kubernetes Jenkins Agents. These Jenkins Agents can be dynamically scaled, meaning they are provisioned or released based on job status. The Jenkins Controller and Agents run as pods on KubeSphere nodes. The Controller runs on one of the nodes, and its configuration data is stored in a PersistentVolumeClaim. Agents run on various nodes but may not always be running; they are dynamically created and automatically deleted as needed.

When the Jenkins Controller receives a build request, it dynamically creates a Jenkins Agent running in a pod based on labels and registers it with the Controller. After the Agent completes its task, it is released, and the associated pod is deleted.

Dynamic Provisioning of Jenkins Agents

Dynamic provisioning of Jenkins Agents offers the following advantages:

Reasonable Resource Allocation: Dynamically allocate created Agents to idle nodes, avoiding task queuing due to high resource utilization on a single node.

High Scalability: Supports adding new nodes to the cluster when tasks are queued for a long time due to insufficient cluster resources.

High Availability: When the Jenkins Controller fails, DevOps automatically creates a new Jenkins Controller container and mounts the persistent volume to the newly created container, ensuring data is not lost and achieving high cluster availability.