Introduction to Continuous Integration
– Continuous integration (CI) is the practice of automating the integration of code changes from multiple contributors into a single software project. It’s a primary DevOps best practice, allowing developers to frequently merge code changes into a central repository where builds and tests then run.
Understanding Jenkins and Hudson relation
– Jenkins and Hudson are Continuous Integration tool, commonly known as CI frameworks. They allow Java project to automate build and deployment process. By using CI tools like Hudson or Jenkins you can create hourly or daily builds automatically, you can also automatically run your unit tests and can deploy build to your QA or production environment.
– By the way Jenkins and Hudson are from same source tree. Hudson was original project which was open source and supported by Sun. When Oracle bought Sun, it took control over Hudson name and logistic platform of Hudson. Many open source contributors were not comfortable with that and this resulted in Jenkins.
Implementing Jenkins through Infrastructure as Code
– Setting up Jenkins is a complex process, as both Jenkins and its plugins require some tuning and configuration, with dozens of parameters to set within the web UI manage section.
– Experienced Jenkins users rely on groovy init scripts to customize Jenkins and enforce the desired state. Those scripts directly invoke Jenkins API and, as such, can do everything (at your own risk). But they also require you to know Jenkins internals and are confident in writing groovy scripts on top of Jenkins API.
– The Configuration as Code plugin is an opinionated way to configure Jenkins based on human-readable declarative configuration files. Writing such a file should be feasible without being a Jenkins expert, just translating into code a configuration process one is used to executing in the web UI.
Understanding concept of Job based automation
– Automation is the use of technology to perform tasks with reduced human assistance. Any industry that encounters repetitive tasks can use automation.
– Automation helps businesses on their path to digital transformation. Organizations today are dealing with major disruption—think Airbnb, Amazon, etc. They’re challenged with supporting their employees and partners, reaching new customers, and providing new, innovative products and services faster.
– By simplifying change through automation, you gain the time and energy to focus on innovation. The automated enterprise’s goal is to get work done faster.
Creating first job in Jenkins
– Step 1) Login to Jenkins
– Step 2) Create New Item
– Step 3) Enter Item details
– Step 4) Enter Project details
– Step 5) Enter repository URL from GitHub (or any other SCM)
– Step 6) Tweak the settings
o Under build,
➔ Click on “Add build step”
➔ Click on “Execute Windows batch command” and add the commands you want to execute during the build process. I have added the
following windows commands:
– Step 7) Save the project
– Step 8) Build Source code
– Step 9) Check the status
– Step 10) See the console output
Enabling Git in Jenkins
– Go to your Jenkins instance’s root page.
– If your Jenkins instance has security enabled, login as a user who has the Overall | Administer permission.
– Select the Manage Jenkins link on the left-hand side of the screen.
– Select the Configure Jenkins link, give the path to the Git executable on the system or let Jenkins install it automatically.
– Go to the Manage Plugins link.
– On the Available tab, select the Git Plugin and click the Download and Install button at the bottom of the page.
– Go to your Jenkins instance job.
– Select the Configure link on the left hand side of the screen.
– In the Source Code Management field, enter the URL of the Git project which typically looks like:
This sets up your job to checkout all branches on the Git repository.
Integrating Git Server with Jenkins for periodic polling of source code changes
Follow the below steps:
– Click on the “Configure” of the job created in the Jenkins dashboard.
– Click on build triggers in the configure settings and select the Poll SCM.
– Enter the desired cron to poll the SCM. Here we have given * * * * * which means the Jenkins polls the SCM every minute.
Import sample java j2ee application on git server
To bring a GitHub hosted Maven project into Eclipse, follow these steps:
– Copy the GitHub URL of the repository to the clipboard
– Open Eclipse and choose Import –> Projects from Git (with smart import)
– Choose the Clone URI option in the Git import wizard and click Next
– Confirm the URI, Host and Repository path parameters and click Next
– Choose the Git branches to clone from the remote repository and click Next
– Confirm the Directory into which the repository will be cloned and click Next
– Choose the Maven project to import into Eclipse from GitHub and click Finish
Trigger Jenkins job from Git
– Step 1: Grant your server access to your private GitHub repository.
– Step 2: Install the Git and GitHub plugins.
– Step 3: Configure a Jenkins job to use your repository.
– Step 4: Grant GitHub access to your private Jenkins instance.
– Step 5: Add the hooks to GitHub.
Understanding Jenkins Deployment Architecture for better performance
Let us have a look at the Jenkins Architecture below diagram depicts the same.
– The developers push the code to GitHub repository and the DevOps team would set up Jenkins job to build the pipeline.
– Jenkins will check the GIT repository for any changes in the source code at regular intervals
– Each Jenkins build requires its own testing environment, which can’t be created on a single server. Jenkins accomplishes this by employing various slaves as needed.
– Jenkins Master will communicate the request for testing to these slaves, as well as test reports.
Implementing Jenkins Master-Slave
The single Jenkins server was not enough to meet certain requirements like:
- Sometimes you might need several different environments to test your builds. This cannot be done by a single Jenkins server.
- If larger and heavier projects get built on a regular basis then a single Jenkins server cannot simply handle the entire load. This is where Jenkins follows a Master-Slave architecture. Steps are below.
→Click on Manage Jenkins in the left corner on the Jenkins dashboard.
→Click on Manage Nodes
→Select New Node and enter the name of the node in the Node Name field.
→Select Permanent Agent and click the OK button. Initially, you will get only one option, “Permanent Agent.” Once you have one or more slaves you will get the “Copy Existing Node” option.
→Enter the required information.
→Enter the Hostname in the Host field.
→Select the Add button to add credentials. and click Jenkins.
→Enter Username, Password, ID, and Description.
→Select the dropdown menu to add credentials in the Credentials field.
→Select the next dropdown to add the Host Key Verification Strategy under Non verifying Verification Strategy.
→Select Keep this agent online as much as possible in the Availability field.
→Click the Save button.
Jenkins Build Trigger
– Let’s understand the details of all these options:
Jenkins Build Steps
Jenkins Post Build Steps
– For dependent jobs, we can say like “If one job build is successful then another job should run” and for the independent job, we can check like “If for one job, the build will be successful then only new build will be checked for another job“.
Below are a few post build steps.
– Aggregate downstream test results
– Archive the artifacts
– Build other projects
– Publish JUnit test result report
– Publish NUnit test result report
– Git Publisher
– Publish Cucumber reports generated with handlebars
– Publish JUnit reports generated with handlebars
– Copy files back to the job’s workspace on the master node
– E-mail Notification
– Publish to Subversion repository
– Set GitHub commit status (universal)
– Set build status on GitHub commit [deprecated]
– Delete workspace when build is done
– Plugins are the primary means of enhancing the functionality of a Jenkins environment to suit organization- or user-specific needs. There are over a thousand different plugins which can be installed on a Jenkins controller and to integrate various build tools, cloud providers, analysis tools, and much more.
– Plugins can be automatically downloaded, with their dependencies, from the Update Center. The Update Center is a service operated by the Jenkins project which provides an inventory of open source plugins which have been developed and maintained by various members of the Jenkins community.
Here is a list of the top 10 must-have Jenkins plugins and features:
- Easy Installation Feature
- Docker Plugin for Jenkins
- Jira Plugin
- Slack Notification Plugin
- Maven Plugin
- Amazon EC2 Plugin
- JUnit Plugin
- Pipeline Plugin
- Mailer Plugin
- Git Plugin
Jenkins Global Configurations
– As a Jenkins admin, Configure System(Option to configure Jenkins settings) is the most prominent section of Jenkins. Here, the admin can define global settings that apply to all the projects that are present in Jenkins. Now, the question is how to navigate towards the Configure System option in Jenkins.
Step 1: Firstly, navigate to the Jenkins dashboard and after that, click on the “Manage Jenkins”
Step 2: Secondly, as soon as we will click on the “Manage Jenkins” option, we will be redirected to the “Manage Jenkins” page. Here, we need to click on “Configure System”
Step 3: Thirdly, we will click on the Configure System link. Subsequently, go to the configuration page
Jenkins Pipeline (or simply “Pipeline”) is a suite of plugins which supports implementing and integrating continuous delivery pipelines into Jenkins. The definition of a Jenkins Pipeline is typically written into a text file (called a Jenkinsfile) which in turn is checked into a project’s source control repository.
There are two types of syntax using which we can define a Jenkinsfile:
- Declarative Pipeline syntax
- Scripted Pipeline syntax
Declarative Pipeline Syntax
The declarative syntax is a new feature that used code for the pipeline. It provides a limited pre-defined structure. Thereby, it offers an easy & simple continuous delivery pipeline. Moreover, it uses a pipeline block.
Scripted Pipeline Syntax
Unlike declarative syntax, the scripted pipeline syntax is the old traditional way to write the Jenkinsfile on Jenkins web UI. Moreover, it strictly follows the groovy syntax and helps to develop a complex pipeline as code.
– As the number of jobs grows, maintaining them becomes tedious, and the paradigm of using a UI falls apart. Additionally, the common pattern in this situation is to copy jobs to create new ones, these “children” have a habit of diverging from their original “template” and consequently it becomes difficult to maintain consistency between these jobs.
– The Job DSL plugin attempts to solve this problem by allowing jobs to be defined in a programmatic form in a human readable file. Writing such a file is feasible without being a Jenkins expert as the configuration from the web UI translates intuitively into code.
– Job DSL was one of the first popular plugins for Jenkins which allows managing configuration as code and many other plugins dealing with this aspect have been created since then, most notably the Jenkins Pipeline and Configuration as Code plugins. It is important to understand the differences between these plugins and Job DSL for managing Jenkins configuration efficiently.
– The Configuration as Code plugin can be used to manage the global system configuration of Jenkins. It comes with an integration for Job DSL to create an initial set of jobs.
– A scripted pipeline is a groovy-based DSL. It provides greater flexibility and scalability for Jenkins users than the Declarative pipeline. Groovy scripts are not necessarily suitable for all users, so Jenkins created the Declarative pipeline. The Declarative Pipeline syntax is more stringent.
Stages vs Jobs
– Stage describes a stage of this Pipeline. It is used in the visualization in the Stage View in a job overview page (Jenkins UI). It is basically just a name.
– Jobs in Jenkins is an improvised or unrestricted build job or task with multiple operations. Operations can be a build, pipeline run, or any script run.