Master-Agent Architecture 🖧
Jenkins Master (Server)🗄️
The Jenkins master, also called the Jenkins server, is the main part of Jenkins. It handles:
Job Scheduling: The master schedules jobs based on triggers like code changes, time intervals, or manual starts.
Workflow Orchestration: It manages the execution of workflows defined in pipelines.
Monitoring: The master tracks job statuses, and logs, and shows results on the Jenkins UI.
Configuration Management: All job settings, credentials, and global settings are managed from the master.
The master provides the Jenkins user interface (UI) for creating, managing, and monitoring jobs, and it assigns job execution to agents to ensure efficient workload distribution.
Jenkins Agent💼
A Jenkins agent is a separate machine or container that carries out the tasks defined in Jenkins jobs. Unlike the master, which handles orchestration and management, the agent focuses on doing the actual work. Agents are useful for:
Distributed Builds: By running jobs on different agents, Jenkins can handle more jobs concurrently, improving performance.
Environment Isolation: Agents can be configured with specific environments, such as different OS versions and software dependencies, allowing specialized jobs to run in the required settings.
Scalability: As projects and teams grow, additional agents can be added to scale Jenkins’ job execution capacity.
Each agent is identified by a unique label, which can be used to target jobs to specific agents based on the required environment or capacity.
🎯Task -1
Create an Agent:
- Set up a new node in Jenkins by creating an agent.
AWS EC2 Instance Setup:
- Create a new AWS EC2 instance and connect it to the master (where Jenkins is installed).
Master-Agent Connection:
Establish a connection between the master and agent using SSH and a public-private key pair exchange.
Verify the agent's status in the "Nodes" section.
╰┈➤Create two AWS instances: one for the Master and one for the Agent.
╰┈➤In the Master instance, install Java and Jenkins.
##Java
sudo apt update
sudo apt install fontconfig openjdk-17-jre
java -version
##Jenkins
sudo wget -O /usr/share/keyrings/jenkins-keyring.asc \
https://pkg.jenkins.io/debian-stable/jenkins.io-2023.key
echo "deb [signed-by=/usr/share/keyrings/jenkins-keyring.asc]" \
https://pkg.jenkins.io/debian-stable binary/ | sudo tee \
/etc/apt/sources.list.d/jenkins.list > /dev/null
sudo apt-get update
sudo apt-get install jenkins
sudo systemctl status jenkins
╰┈➤In the Agent instance, install Java and Docker.
##Java
sudo apt update
sudo apt install fontconfig openjdk-17-jre
java -version
#Docker
sudo apt-get update
sudo apt-get install docker.io
sudo systemctl status docker
╰┈➤Now, let's connect the master and agent.
To connect the master and agent, you need to use SSH and exchange public-private key pairs.
➤On the master instance,
ssh-keygen
cd .ssh
ls
cat <Public key>
╰┈➤Copy public key
➤Now, on the agent,
cd .ssh
ls
vim authorized_keys
╰┈➤Paste public key
╰┈➤Open Master EC2:
╰┈➤Add Jenkins port:
╰┈➤Copy Jenkins' initial password from the master machine
sudo cat /var/lib/jenkins/secrets/initialAdminPassword
╰┈➤Paste
╰┈➤Create a username and password
╰┈➤Install Suggested plugins
╰┈➤Now, let's set up a new node. Go to Manage Jenkins and click on Set up a node.
╰┈➤ Enter all details. Here, when we use labels for the agent, your master server will trigger the builds for the agent server.
╰┈➤For the launch method, select 'Launch agents via SSH'. Enter the public IP of the Agent instance in the Host field.
╰┈➤For Credentials, add new credentials (if you don't already have them saved in Jenkins). Select 'SSH username with private key'. Fill in all required details, and in the private key area, add the Private key from the master instance (id_rsa).
╰┈➤Save the configuration details. And you can verify that the Agent is up and running if set up properly.
🎯Task-2
Run your previous Jobs on the new agent
Use labels for the agent, your master server should trigger builds for the agent server.
╰┈➤Let's create a freestyle project that we have deployed before and try to trigger the build on the agent node.
╰┈➤Add the necessary details. Here, select "Restrict where this project can be run" and enter the label of the agent node.
docker build . -t node-todo:latest
docker run -d -p 8000:8000 --name todo-app node-todo:latest
🔴Error- (Read error details carefully )
sudo groupadd docker
sudo usermod -aG docker $USER
newgrp docker
docker run hello-world
➡️Now, build the Jenkins job again.
🟢Job ran successfully
🔶Now, since we deployed the trigger build in the agent. Go to the public IP of the agent instance and port :8000 and you can verify that the docker container was deployed in the agent instance.
◉Commands for Stop, Kill Docker Container
docker ps
docker kill <id>
◉Command for Stop Docker and Jenkins
Agent:
sudo systemctl status docker
sudo systemctl stop docker.socket
Master:
sudo systemctl stop jenkins
sudo systemctl status jenkins
Conclusion📌
By using Jenkins' master-agent architecture, teams can efficiently scale their CI/CD pipelines to manage growing workloads. Setting up a Jenkins agent involves creating a node, configuring an EC2 instance, and establishing a secure connection between the master and the agent. Once set up, jobs can be distributed across multiple agents, optimizing build times and resource use.