.. code-block:: bash
- git clone https://git.opendaylight.org/gerrit/releng/builder
+ git clone --recursive https://git.opendaylight.org/gerrit/releng/builder
cd builder
mkdir jjb/<new-project>
+.. note:
+
+ releng/global-jjb is a submodule of releng/builder repository which
+ requires a git submodule update --init or using --recursive with git clone.
+ `releng-global-jjb`_
+
Where <new-project> should be the same name as your project's git repo in
-Gerrit. So if your project is called "aaa" then create a new jjb/aaa directory.
+Gerrit. If your project is called "aaa" then create a new jjb/aaa directory.
Next we will create <new-project>.yaml as follows:
.. code-block:: yaml
+ ---
- project:
name: <NEW_PROJECT>-carbon
jobs:
- - '{project-name}-clm-{stream}'
- - '{project-name}-integration-{stream}'
- - '{project-name}-merge-{stream}'
- - '{project-name}-verify-{stream}-{maven}-{jdks}'
+ - '{project-name}-clm-{stream}'
+ - '{project-name}-integration-{stream}'
+ - '{project-name}-merge-{stream}'
+ - '{project-name}-verify-{stream}-{maven}-{jdks}'
project: '<NEW_PROJECT>'
project-name: '<NEW_PROJECT>'
branch: 'master'
jdk: openjdk8
jdks:
- - openjdk8
+ - openjdk8
maven:
- - mvn33:
- mvn-version: 'mvn33'
+ - mvn33:
+ mvn-version: 'mvn33'
mvn-settings: '<NEW_PROJECT>-settings'
mvn-goals: 'clean install -Dmaven.repo.local=/tmp/r -Dorg.ops4j.pax.url.mvn.localRepository=/tmp/r'
mvn-opts: '-Xmx1024m -XX:MaxPermSize=256m'
- project:
name: <NEW_PROJECT>-sonar
jobs:
- - '{project-name}-sonar'
+ - '{project-name}-sonar'
project: '<NEW_PROJECT>'
project-name: '<NEW_PROJECT>'
Please note that the combination of a Packer definitions from `vars`, `templates`
and the `provision` scripts is what defines a given minion. For instance, a minion
-may be defined as `centos7-java-builder` which is a combination of Packer OS image
+may be defined as `centos7-builder` which is a combination of Packer OS image
definitions from `vars/centos.json`, Packer template definitions from
-`templates/java-buidler.json` and spinup scripts from `provision/java-builder.sh`.
+`templates/builder.json` and spinup scripts from `provision/builder.sh`.
This combination provides the full definition of the realized minion.
Jenkins starts a minion using the latest image which is built and linked into the
executed Jenkins will finally attach the minion as an actual minion and start
handling jobs on it.
-Pool: ODLRPC
+Flavors
+^^^^^^^
+
+Performance flavors come with dedicated CPUs and are not shared with other
+accounts in the cloud so should ensure consistent performance.
+
+.. list-table:: Flavors
+ :widths: auto
+ :header-rows: 1
+
+ * - Instance Type
+ - CPUs
+ - Memory
+
+ * - v2-standard-1
+ - 1
+ - 4
+
+ * - v2-standard-2
+ - 2
+ - 8
+
+ * - v2-standard-4
+ - 4
+ - 16
+
+ * - v2-standard-8
+ - 8
+ - 32
+
+ * - v2-standard-16
+ - 16
+ - 64
+
+ * - v2-highcpu-1
+ - 1
+ - 1
+
+ * - v2-highcpu-2
+ - 2
+ - 2
+
+ * - v2-highcpu-4
+ - 4
+ - 4
+
+ * - v2-highcpu-8
+ - 8
+ - 8
+
+ * - v2-highcpu-16
+ - 16
+ - 16
+
+ * - v2-highcpu-32
+ - 32
+ - 32
+
+Pool: ODLVEX
^^^^^^^^^^^^
.. raw:: html
<table class="table table-bordered">
<tr class="warning">
- <td><b>Jenkins Labels</b><br/> dynamic_controller, dynamic_verify,
- dynamic_merge, centos7-java-builder-2c-4g, centos7-java-builder-2c-8g,
- centos7-java-builder-4c-8g</td>
- <td><b>Minion Template names</b><br/> centos7-java-builder-2c-4g,
- centos7-java-builder-2c-8g, centos7-java-builder-2c-8g,
- centos7-java-builder-4c-8g, centos7-java-builder-8c-8g</td>
+ <td><b>Jenkins Labels</b><br/> centos7-builder-2c-4g,
+ centos7-builder-2c-8g, centos7-java-builder-4c-8g,
+ centos7-builder-8c-8g, centos7-java-builder-4c-16g</td>
+ <td><b>Minion Template names</b><br/> centos7-builder-2c-4g,
+ centos7-builder-2c-4g, centos7-java-builder-2c-8g,
+ centos7-builder-4c-8g, centos7-java-builder-8c-8g,
+ centos7-builder-4c-16g</td>
<td><b>Packer Template</b><br/>
- releng/builder/packer/templates/java-builder.json</td>
+ releng/builder/packer/templates/builder.json</td>
<td><b>Spinup Script</b><br/>
releng/builder/jenkins-scripts/builder.sh</td>
</tr>
<tr>
<td colspan="4">
- A CentOS 7 huild minion. This system has OpenJDK 1.7 (Java7) and OpenJDK
- 1.8 (Java8) installed on it along with all the other components and
- libraries needed for building any current OpenDaylight project. This is
- the label that is used for all basic verify, merge and daily builds for
+ CentOS 7 build minion configured with OpenJDK 1.7 (Java7) and OpenJDK
+ 1.8 (Java8) along with all the other components and libraries needed
+ for building any current OpenDaylight project. This is the label that
+ is used for all basic verify, merge and daily builds for
projects.
</td>
</tr>
<tr class="warning">
- <td><b>Jenkins Labels</b><br/> dynamic_robot, centos7-robot-2c-2g</td>
+ <td><b>Jenkins Labels</b><br/> centos7-robot-2c-2g</td>
<td><b>Minion Template names</b><br/> centos7-robot-2c-2g</td>
<td><b>Packer Template</b><br/>
releng/builder/packer/templates/robot.json</td>
</tr>
<tr>
<td colspan="4">
- A CentOS 7 minion that is configured with OpenJDK 1.7 (Java7), OpenJDK
+ CentOS 7 minion configured with OpenJDK 1.7 (Java7), OpenJDK
1.8 (Java8) and all the current packages used by the integration
project for doing robot driven jobs. If you are executing robot
framework jobs then your job should be using this as the minion that
</tr>
<tr class="warning">
- <td><b>Jenkins Labels</b><br/> ubuntu_mininet, ubuntu-trusty-mininet-2c-2g</td>
- <td><b>Minion Template names</b><br/> ubuntu-trusty-mininet-2c-2g</td>
+ <td><b>Jenkins Labels</b><br/> ubuntu1404-mininet-2c-2g</td>
+ <td><b>Minion Template names</b><br/> ubuntu1404-mininet-2c-2g</td>
<td><b>Packer Template</b><br/>
releng/builder/packer/teamplates/mininet.json</td>
<td><b>Spinup Script</b><br/> releng/builder/jenkins-scripts/mininet-ubuntu.sh</td>
</tr>
<tr>
<td colspan="4">
- Basic Ubuntu system with ovs 2.0.2 and mininet 2.1.0
+ Basic Ubuntu 14.04 (Trusty) system with ovs 2.0.2 and mininet 2.1.0
</td>
</tr>
<tr class="warning">
- <td><b>Jenkins Labels</b><br/> ubuntu_mininet_ovs_23,
- ubuntu-trusty-mininet-ovs-23-2c-2g</td>
- <td><b>Minion Template names</b><br/> ubuntu-trusty-mininet-ovs-23-2c-2g</td>
+ <td><b>Jenkins Labels</b><br/> ubuntu1404-mininet-ovs-23-2c-2g</td>
+ <td><b>Minion Template names</b><br/> ubuntu1404-mininet-ovs-23-2c-2g</td>
<td><b>Packer Template</b><br/> releng/builder/packer/templates/mininet-ovs-2.3.json</td>
<td><b>Spinup Script</b><br/> releng/builder/jenkins-scripts/mininet-ubuntu.sh</td>
</tr>
<tr>
<td colspan="4">
- Basic Ubuntu system with ovs 2.3 and mininet 2.2.1
- </td>
- </tr>
-
- <tr class="warning">
- <td><b>Jenkins Labels</b><br/> ubuntu_mininet_ovs_25,
- ubuntu-trusty-mininet-ovs-25-2c-2g</td>
- <td><b>Minion Template names</b><br/> ubuntu-trusty-mininet-ovs-25-2c-2g</td>
- <td><b>Packer Template</b><br/> releng/builder/packer/templates/mininet-ovs-2.5.json</td>
- <td><b>Spinup Script</b><br/> releng/builder/jenkins-scripts/mininet-ubuntu.sh</td>
- </tr>
- <tr>
- <td colspan="4">
- Basic Ubuntu system with ovs 2.5 and mininet 2.2.2
+ Ubuntu 16.04 (Xenial) system with ovs 2.5 and mininet 2.2.1
</td>
</tr>
<tr class="warning">
- <td><b>Jenkins Labels</b><br/> dynamic_devstack, centos7-devstack-2c-4g</td>
+ <td><b>Jenkins Labels</b><br/> centos7-devstack-2c-4g</td>
<td><b>Minion Template names</b><br/> centos7-devstack-2c-4g</td>
<td><b>Packer Template</b><br/> releng/builder/packer/templates/devstack.json</td>
<td><b>Spinup Script</b><br/> releng/builder/jenkins-scripts/devstack.sh</td>
</tr>
<tr>
<td colspan="4">
- A CentOS 7 system purpose built for doing OpenStack testing using
+ CentOS 7 system purpose built for doing OpenStack testing using
DevStack. This minion is primarily targeted at the needs of the OVSDB
project. It has OpenJDK 1.7 (aka Java7) and OpenJDK 1.8 (Java8) and
other basic DevStack related bits installed.
</tr>
<tr class="warning">
- <td><b>Jenkins Labels</b><br/> dynamic_docker, centos7-docker-2c-4g</td>
+ <td><b>Jenkins Labels</b><br/> centos7-docker-2c-4g</td>
<td><b>Minion Template names</b><br/> centos7-docker-2c-4g</td>
<td><b>Packer Template</b><br/> releng/builder/packer/templates/docker.json</td>
<td><b>Spinup Script</b><br/> releng/builder/jenkins-scripts/docker.sh</td>
</tr>
<tr>
<td colspan="4">
- A CentOS 7 system that is configured with OpenJDK 1.7 (aka Java7),
+ CentOS 7 system configured with OpenJDK 1.7 (aka Java7),
OpenJDK 1.8 (Java8) and Docker. This system was originally custom
built for the test needs of the OVSDB project but other projects have
expressed interest in using it.
</tr>
<tr class="warning">
- <td><b>Jenkins Labels</b><br/> gbp_trusty, ubuntu-trusty-gbp-2c-2g</td>
- <td><b>Minion Template names</b><br/> ubuntu-trusty-gbp-2c-2g</td>
+ <td><b>Jenkins Labels</b><br/> ubuntu1404-gbp-2c-2g</td>
+ <td><b>Minion Template names</b><br/> ubuntu1404-gbp-2c-2g</td>
<td><b>Packer Template</b><br/> releng/builder/packer/templates/gbp.json</td>
<td><b>Spinup Script</b><br/> releng/builder/jenkins-scripts/ubuntu-docker-ovs.sh</td>
</tr>
<tr>
<td colspan="4">
- A basic Ubuntu node with latest OVS and docker installed. Used by Group Based Policy.
+ Ubuntu 14.04 (Trusty) node with latest OVS and docker installed. Used by Group Based Policy.
</td>
</tr>
+
+ <tr class="warning">
+ <td><b>Jenkins Labels</b><br/> ubuntu1604-gbp-2c-4g</td>
+ <td><b>Minion Template names</b><br/> ubuntu1604-gbp-2c-4g</td>
+ <td><b>Packer Template</b><br/> releng/builder/packer/templates/gbp.json</td>
+ <td><b>Spinup Script</b><br/> releng/builder/jenkins-scripts/ubuntu-docker-ovs.sh</td>
+ </tr>
+ <tr>
+ <td colspan="4">
+ Ubuntu 16.04 (Xenial) node with latest OVS and docker installed. Used by Group Based Policy.
+ </td>
+ </tr>
+
</table>
-Pool: ODLPUB - HOT (Heat Orchestration Templates)
+Pool: ODLVEX - HOT (Heat Orchestration Templates)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
HOT integration enables to spin up integration labs servers for CSIT jobs
`{odl,docker,openstack,tools}_system_image` followed by image name in the
format `<platform> - <template> - <date-stamp>`.
-.. code-block:: yaml
-
- CentOS 7 - docker - 20161031-0802
-
.. include:: cloud-images.rst
Creating Jenkins Jobs
their XML descriptions and upload them to the sandbox Jenkins server.
We document `installing <Installing Jenkins Job Builder_>`_ `jenkins-jobs`
-below. We also provide
-a `pre-built Docker image <jjb-docker_>`_ with `jenkins-jobs` already installed.
+below.
Installing Jenkins Job Builder
------------------------------
-For users who aren't already experienced with Docker or otherwise don't want
-to use our `pre-built JJB Docker image <jjb-docker_>`_, installing JJB into a
-virtual environment is an equally good option.
-
We recommend using `pip <Installing JJB using pip_>`_ to assist with JJB
installs, but we
also document `installing from a git repository manually
to manage JJB and its
Python dependencies. The `python-virtualenvwrapper`_ tool can help you do so.
-There are good docs for installing `python-virtualenvwrapper`_. On Linux systems
-with pip (typical), they amount to:
+Documentation is available for installing `python-virtualenvwrapper`_. On Linux
+systems with pip (typical), they amount to:
.. code-block:: bash
.. code-block:: bash
- $ git clone https://git.opendaylight.org/gerrit/p/releng/builder.git
+ $ git clone --recursive https://git.opendaylight.org/gerrit/p/releng/builder.git
Before actually installing JJB and its dependencies, make sure you've `created
and activated <Virtual Environments_>`_ a virtual environment for JJB.
(jjb)$ jenkins-jobs --version
-JJB Docker Image
-----------------
-
-`Docker <docker-docs_>`_ is an open platform used to create virtualized Linux containers
-for shipping self-contained applications. Docker leverages LinuX Containers
-\(LXC\) running on the same operating system as the host machine, whereas a
-traditional VM runs an operating system over the host.
-
-.. code-block:: bash
-
- docker pull zxiiro/jjb-docker
- docker run --rm -v ${PWD}:/jjb jjb-docker
-
-This `Dockerfile <jjb-dockerfile_>`_ created the
-`zxiiro/jjb-docker image <jjb-docker_>`_.
-By default it will run:
-
-.. code-block:: bash
-
- jenkins-jobs test .
-
-You'll need to use the `-v/--volume=[]` parameter to mount a directory
-containing your YAML files, as well as a configured `jenkins.ini` file if you
-wish to upload your jobs to the `Jenkins Sandbox`_.
Jenkins Job Templates
---------------------
</td>
</tr>
- <tr class="warning">
- <td><b>Job Template</b><br/>integration-patch-test-{stream}</td>
- <td><b>Gerrit Trigger</b><br/>test-integration</td>
- </tr>
- <tr>
- <td colspan="2">
- </td>
- </tr>
-
<tr class="warning">
<td><b>Job Template</b><br/>integration-patch-test-{stream}</td>
<td><b>Gerrit Trigger</b><br/>test-integration</td>
</li>
</td>
</tr>
+
+ <tr class="warning">
+ <td><b>Job Template</b><br/>integration-multipatch-test-{stream}</td>
+ <td><b>Gerrit Trigger</b><br/>multipatch-build</td>
+ </tr>
+ <tr>
+ <td colspan="2">
+ This job builds a list of patches provided in an specific order, and finally builds
+ a distribution from either provided patch or latest code in branch.
+ For example if someone leaves the following comment in a patch:
+ multipatch-build:controller=61/29761/5:45/29645/6,neutron=51/65551/4,netvirt:59/60259/17
+ the job will checkout controller patch 61/29761/5, cherry-pick 45/29645/6 and build controller,
+ checkout neutron patch 51/65551/4 and build neutron, checkout latest netvirt code,
+ cherry-pick 59/60259/17 and build netvirt, finally it will checkout latest distribution
+ code and build a distribution. The resulting distribution is stored in Nexus and the URL
+ is stored in a variable called BUNDLE_URL visible in the job console.
+ This job should not alter Gerrit votes for a given patch, nor will do anything with the
+ given patch unless the patch is added to the build list.
+ </td>
+ </tr>
+
</table>
Maven Properties
.. code-block:: bash
# If you don't have RelEng/Builder's repo, clone it
- $ git clone https://git.opendaylight.org/gerrit/p/releng/builder.git
+ $ git clone --recursive https://git.opendaylight.org/gerrit/p/releng/builder.git
# Make a copy of the example JJB config file (in the builder/ directory)
$ cp jenkins.ini.example jenkins.ini
# Edit jenkins.ini with your username, API token and ODL's sandbox URL
# Don't push all jobs by omitting the final param! (ctrl+c to abort)
jenkins-jobs --conf jenkins.ini update jjb/ <job-name>
+Alternatively, you can push a job to the Jenkins sandbox with a special comment in a
+releng/builder gerrit patch. The job will be based off of the code your patch is
+based upon. Meaning, if your patch is changing something related to the job you are
+pushing, those changes will exist in the sandbox job. The format of the comment is::
+
+ jjb-deploy <job name>
+
Running Jobs
^^^^^^^^^^^^
Make changes to your JJB configuration, re-test, re-push and re-run until
your job is ready.
-Docker Method
-^^^^^^^^^^^^^
-
-If `using Docker <JJB Docker image_>`_:
-
-.. code-block:: bash
-
- # To test
- docker run --rm -v ${PWD}:/jjb zxiiro/jjb-docker
-
-.. important::
-
- When pushing with `jenkins-jobs`, a log message with
- the number of jobs you're pushing will be issued, typically to stdout.
- **If the number is greater than 1** (or the number of jobs you passed to
- the command to push) then you are pushing too many jobs and should **`ctrl+c`
- to cancel the upload**. Else you will flood the system with jobs.
-
- .. code-block:: bash
-
- INFO:jenkins_jobs.builder:Number of jobs generated: 1
-
- **Failing to provide the final `<job-name>` param will push all jobs!**
-
- .. code-block:: bash
-
- # To upload jobs to the sandbox
- # Please ensure that you include a configured jenkins.ini in your volume mount
- # Making sure not to push more jobs than expected, ctrl+c to abort
- docker run --rm -v ${PWD}:/jjb zxiiro/jjb-docker jenkins-jobs --conf jenkins.ini update . openflowplugin-csit-periodic-1node-cds-longevity-only-master
-.. _docker-docs: https://www.docker.com/whatisdocker/
.. _example-jenkins.ini: https://git.opendaylight.org/gerrit/gitweb?p=releng/builder.git;a=blob;f=jenkins.ini.example
.. _integration-test-wiki: https://wiki.opendaylight.org/view/Integration/Test
.. _jenkins-master: https://jenkins.opendaylight.org/releng
.. _jenkins-sandbox-login: https://jenkins.opendaylight.org/sandbox/login
.. _jenkins.ini: http://docs.openstack.org/infra/jenkins-job-builder/execution.html#configuration-file
.. _jjb-autoupdate-project.py: https://git.opendaylight.org/gerrit/gitweb?p=releng/builder.git;a=blob;f=scripts/jjb-autoupdate-project.py
-.. _jjb-docker: https://hub.docker.com/r/zxiiro/jjb-docker/
-.. _jjb-dockerfile: https://github.com/zxiiro/jjb-docker/blob/master/Dockerfile
.. _jjb-docs: http://ci.openstack.org/jenkins-job-builder/
.. _jjb-init-project.py: https://git.opendaylight.org/gerrit/gitweb?p=releng/builder.git;a=blob;f=scripts/jjb-init-project.py
.. _jjb-repo: https://github.com/openstack-infra/jenkins-job-builder
.. _releng-wiki: https://wiki.opendaylight.org/view/RelEng:Main
.. _releng-builder-gerrit: https://git.opendaylight.org/gerrit/#/admin/projects/releng/builder
.. _releng-builder-repo: https://git.opendaylight.org/gerrit/gitweb?p=releng%2Fbuilder.git;a=summary
+.. _releng-global-jjb: https://gerrit.linuxfoundation.org/infra/#/q/project:releng/global-jjb
.. _releng-builder-wiki: https://wiki.opendaylight.org/view/RelEng/Builder
.. _streams-design-background: https://lists.opendaylight.org/pipermail/release/2015-July/003139.html
.. _spinup-scripts: https://git.opendaylight.org/gerrit/gitweb?p=releng/builder.git;a=tree;f=jenkins-scripts