.. code-block:: bash
- git clone https://git.opendaylight.org/gerrit/releng/builder
+ git clone --recursive https://git.opendaylight.org/gerrit/releng/builder
cd builder
mkdir jjb/<new-project>
+.. note:
+
+ releng/global-jjb is a submodule of releng/builder repository which
+ requires a git submodule update --init or using --recursive with git clone.
+ `releng-global-jjb`_
+
Where <new-project> should be the same name as your project's git repo in
-Gerrit. So if your project is called "aaa" then create a new jjb/aaa directory.
+Gerrit. If your project is called "aaa" then create a new jjb/aaa directory.
Next we will create <new-project>.yaml as follows:
Please note that the combination of a Packer definitions from `vars`, `templates`
and the `provision` scripts is what defines a given minion. For instance, a minion
-may be defined as `centos7-java-builder` which is a combination of Packer OS image
+may be defined as `centos7-builder` which is a combination of Packer OS image
definitions from `vars/centos.json`, Packer template definitions from
-`templates/java-buidler.json` and spinup scripts from `provision/java-builder.sh`.
+`templates/builder.json` and spinup scripts from `provision/builder.sh`.
This combination provides the full definition of the realized minion.
Jenkins starts a minion using the latest image which is built and linked into the
executed Jenkins will finally attach the minion as an actual minion and start
handling jobs on it.
-Pool: ODLRPC
+Flavors
+^^^^^^^
+
+Performance flavors come with dedicated CPUs and are not shared with other
+accounts in the cloud so should ensure consistent performance.
+
+.. list-table:: Flavors
+ :widths: auto
+ :header-rows: 1
+
+ * - Instance Type
+ - CPUs
+ - Memory
+
+ * - v2-standard-1
+ - 1
+ - 4
+
+ * - v2-standard-2
+ - 2
+ - 8
+
+ * - v2-standard-4
+ - 4
+ - 16
+
+ * - v2-standard-8
+ - 8
+ - 32
+
+ * - v2-standard-16
+ - 16
+ - 64
+
+ * - v2-highcpu-1
+ - 1
+ - 1
+
+ * - v2-highcpu-2
+ - 2
+ - 2
+
+ * - v2-highcpu-4
+ - 4
+ - 4
+
+ * - v2-highcpu-8
+ - 8
+ - 8
+
+ * - v2-highcpu-16
+ - 16
+ - 16
+
+ * - v2-highcpu-32
+ - 32
+ - 32
+
+Pool: ODLVEX
^^^^^^^^^^^^
.. raw:: html
<table class="table table-bordered">
<tr class="warning">
- <td><b>Jenkins Labels</b><br/> centos7-java-builder-2c-4g,
- centos7-java-builder-2c-8g, centos7-java-builder-4c-8g,
- centos7-java-builder-8c-8g, centos7-java-builder-4c-16g</td>
- <td><b>Minion Template names</b><br/> centos7-java-builder-2c-4g,
- centos7-java-builder-2c-4g, centos7-java-builder-2c-8g,
- centos7-java-builder-4c-8g, centos7-java-builder-8c-8g,
- centos7-java-builder-4c-16g</td>
+ <td><b>Jenkins Labels</b><br/> centos7-builder-2c-4g,
+ centos7-builder-2c-8g, centos7-java-builder-4c-8g,
+ centos7-builder-8c-8g, centos7-java-builder-4c-16g</td>
+ <td><b>Minion Template names</b><br/> centos7-builder-2c-4g,
+ centos7-builder-2c-4g, centos7-java-builder-2c-8g,
+ centos7-builder-4c-8g, centos7-java-builder-8c-8g,
+ centos7-builder-4c-16g</td>
<td><b>Packer Template</b><br/>
- releng/builder/packer/templates/java-builder.json</td>
+ releng/builder/packer/templates/builder.json</td>
<td><b>Spinup Script</b><br/>
releng/builder/jenkins-scripts/builder.sh</td>
</tr>
<td><b>Packer Template</b><br/> releng/builder/packer/templates/mininet-ovs-2.3.json</td>
<td><b>Spinup Script</b><br/> releng/builder/jenkins-scripts/mininet-ubuntu.sh</td>
</tr>
- <tr>
- <td colspan="4">
- Ubuntu 14.04 (Trusty) system with ovs 2.3 and mininet 2.2.1
- </td>
- </tr>
-
- <tr class="warning">
- <td><b>Jenkins Labels</b><br/> ubuntu1404-mininet-ovs-25-2c-2g</td>
- <td><b>Minion Template names</b><br/> ubuntu1404-mininet-ovs-25-2c-2g</td>
- <td><b>Packer Template</b><br/> releng/builder/packer/templates/mininet-ovs-2.5.json</td>
- <td><b>Spinup Script</b><br/> releng/builder/jenkins-scripts/mininet-ubuntu.sh</td>
- </tr>
- <tr>
- <td colspan="4">
- Ubuntu 14.04 (Trusty) system with ovs 2.5 and mininet 2.2.2
- </td>
- </tr>
-
- <tr class="warning">
- <td><b>Jenkins Labels</b><br/> ubuntu1604-mininet-ovs-25-2c-4g</td>
- <td><b>Minion Template names</b><br/> ubuntu1604-mininet-ovs-25-2c-4g</td>
- <td><b>Packer Template</b><br/> releng/builder/packer/templates/mininet-ovs-2.5.json</td>
- <td><b>Spinup Script</b><br/> releng/builder/jenkins-scripts/mininet-ubuntu.sh</td>
- </tr>
<tr>
<td colspan="4">
Ubuntu 16.04 (Xenial) system with ovs 2.5 and mininet 2.2.1
</table>
-Pool: ODLPUB - HOT (Heat Orchestration Templates)
+Pool: ODLVEX - HOT (Heat Orchestration Templates)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
HOT integration enables to spin up integration labs servers for CSIT jobs
`{odl,docker,openstack,tools}_system_image` followed by image name in the
format `<platform> - <template> - <date-stamp>`.
-.. code-block:: yaml
-
- CentOS 7 - docker - 20161031-0802
-
.. include:: cloud-images.rst
Creating Jenkins Jobs
their XML descriptions and upload them to the sandbox Jenkins server.
We document `installing <Installing Jenkins Job Builder_>`_ `jenkins-jobs`
-below. We also provide
-a `pre-built Docker image <jjb-docker_>`_ with `jenkins-jobs` already installed.
+below.
Installing Jenkins Job Builder
------------------------------
-For users who aren't already experienced with Docker or otherwise don't want
-to use our `pre-built JJB Docker image <jjb-docker_>`_, installing JJB into a
-virtual environment is an equally good option.
-
We recommend using `pip <Installing JJB using pip_>`_ to assist with JJB
installs, but we
also document `installing from a git repository manually
to manage JJB and its
Python dependencies. The `python-virtualenvwrapper`_ tool can help you do so.
-There are good docs for installing `python-virtualenvwrapper`_. On Linux systems
-with pip (typical), they amount to:
+Documentation is available for installing `python-virtualenvwrapper`_. On Linux
+systems with pip (typical), they amount to:
.. code-block:: bash
.. code-block:: bash
- $ git clone https://git.opendaylight.org/gerrit/p/releng/builder.git
+ $ git clone --recursive https://git.opendaylight.org/gerrit/p/releng/builder.git
Before actually installing JJB and its dependencies, make sure you've `created
and activated <Virtual Environments_>`_ a virtual environment for JJB.
(jjb)$ jenkins-jobs --version
-JJB Docker Image
-----------------
-
-`Docker <docker-docs_>`_ is an open platform used to create virtualized Linux containers
-for shipping self-contained applications. Docker leverages LinuX Containers
-\(LXC\) running on the same operating system as the host machine, whereas a
-traditional VM runs an operating system over the host.
-
-.. code-block:: bash
-
- docker pull zxiiro/jjb-docker
- docker run --rm -v ${PWD}:/jjb jjb-docker
-
-This `Dockerfile <jjb-dockerfile_>`_ created the
-`zxiiro/jjb-docker image <jjb-docker_>`_.
-By default it will run:
-
-.. code-block:: bash
-
- jenkins-jobs test .
-
-You'll need to use the `-v/--volume=[]` parameter to mount a directory
-containing your YAML files, as well as a configured `jenkins.ini` file if you
-wish to upload your jobs to the `Jenkins Sandbox`_.
Jenkins Job Templates
---------------------
</td>
</tr>
- <tr class="warning">
- <td><b>Job Template</b><br/>integration-patch-test-{stream}</td>
- <td><b>Gerrit Trigger</b><br/>test-integration</td>
- </tr>
- <tr>
- <td colspan="2">
- </td>
- </tr>
-
<tr class="warning">
<td><b>Job Template</b><br/>integration-patch-test-{stream}</td>
<td><b>Gerrit Trigger</b><br/>test-integration</td>
</li>
</td>
</tr>
+
+ <tr class="warning">
+ <td><b>Job Template</b><br/>integration-multipatch-test-{stream}</td>
+ <td><b>Gerrit Trigger</b><br/>multipatch-build</td>
+ </tr>
+ <tr>
+ <td colspan="2">
+ This job builds a list of patches provided in an specific order, and finally builds
+ a distribution from either provided patch or latest code in branch.
+ For example if someone leaves the following comment in a patch:
+ multipatch-build:controller=61/29761/5:45/29645/6,neutron=51/65551/4,netvirt:59/60259/17
+ the job will checkout controller patch 61/29761/5, cherry-pick 45/29645/6 and build controller,
+ checkout neutron patch 51/65551/4 and build neutron, checkout latest netvirt code,
+ cherry-pick 59/60259/17 and build netvirt, finally it will checkout latest distribution
+ code and build a distribution. The resulting distribution is stored in Nexus and the URL
+ is stored in a variable called BUNDLE_URL visible in the job console.
+ This job should not alter Gerrit votes for a given patch, nor will do anything with the
+ given patch unless the patch is added to the build list.
+ </td>
+ </tr>
+
</table>
Maven Properties
.. code-block:: bash
# If you don't have RelEng/Builder's repo, clone it
- $ git clone https://git.opendaylight.org/gerrit/p/releng/builder.git
+ $ git clone --recursive https://git.opendaylight.org/gerrit/p/releng/builder.git
# Make a copy of the example JJB config file (in the builder/ directory)
$ cp jenkins.ini.example jenkins.ini
# Edit jenkins.ini with your username, API token and ODL's sandbox URL
# Don't push all jobs by omitting the final param! (ctrl+c to abort)
jenkins-jobs --conf jenkins.ini update jjb/ <job-name>
+Alternatively, you can push a job to the Jenkins sandbox with a special comment in a
+releng/builder gerrit patch. The job will be based off of the code your patch is
+based upon. Meaning, if your patch is changing something related to the job you are
+pushing, those changes will exist in the sandbox job. The format of the comment is::
+
+ jjb-deploy <job name>
+
Running Jobs
^^^^^^^^^^^^
Make changes to your JJB configuration, re-test, re-push and re-run until
your job is ready.
-Docker Method
-^^^^^^^^^^^^^
-
-If `using Docker <JJB Docker image_>`_:
-
-.. code-block:: bash
-
- # To test
- docker run --rm -v ${PWD}:/jjb zxiiro/jjb-docker
-
-.. important::
-
- When pushing with `jenkins-jobs`, a log message with
- the number of jobs you're pushing will be issued, typically to stdout.
- **If the number is greater than 1** (or the number of jobs you passed to
- the command to push) then you are pushing too many jobs and should **`ctrl+c`
- to cancel the upload**. Else you will flood the system with jobs.
-
- .. code-block:: bash
-
- INFO:jenkins_jobs.builder:Number of jobs generated: 1
-
- **Failing to provide the final `<job-name>` param will push all jobs!**
-
- .. code-block:: bash
-
- # To upload jobs to the sandbox
- # Please ensure that you include a configured jenkins.ini in your volume mount
- # Making sure not to push more jobs than expected, ctrl+c to abort
- docker run --rm -v ${PWD}:/jjb zxiiro/jjb-docker jenkins-jobs --conf jenkins.ini update . openflowplugin-csit-periodic-1node-cds-longevity-only-master
-.. _docker-docs: https://www.docker.com/whatisdocker/
.. _example-jenkins.ini: https://git.opendaylight.org/gerrit/gitweb?p=releng/builder.git;a=blob;f=jenkins.ini.example
.. _integration-test-wiki: https://wiki.opendaylight.org/view/Integration/Test
.. _jenkins-master: https://jenkins.opendaylight.org/releng
.. _jenkins-sandbox-login: https://jenkins.opendaylight.org/sandbox/login
.. _jenkins.ini: http://docs.openstack.org/infra/jenkins-job-builder/execution.html#configuration-file
.. _jjb-autoupdate-project.py: https://git.opendaylight.org/gerrit/gitweb?p=releng/builder.git;a=blob;f=scripts/jjb-autoupdate-project.py
-.. _jjb-docker: https://hub.docker.com/r/zxiiro/jjb-docker/
-.. _jjb-dockerfile: https://github.com/zxiiro/jjb-docker/blob/master/Dockerfile
.. _jjb-docs: http://ci.openstack.org/jenkins-job-builder/
.. _jjb-init-project.py: https://git.opendaylight.org/gerrit/gitweb?p=releng/builder.git;a=blob;f=scripts/jjb-init-project.py
.. _jjb-repo: https://github.com/openstack-infra/jenkins-job-builder
.. _releng-wiki: https://wiki.opendaylight.org/view/RelEng:Main
.. _releng-builder-gerrit: https://git.opendaylight.org/gerrit/#/admin/projects/releng/builder
.. _releng-builder-repo: https://git.opendaylight.org/gerrit/gitweb?p=releng%2Fbuilder.git;a=summary
+.. _releng-global-jjb: https://gerrit.linuxfoundation.org/infra/#/q/project:releng/global-jjb
.. _releng-builder-wiki: https://wiki.opendaylight.org/view/RelEng/Builder
.. _streams-design-background: https://lists.opendaylight.org/pipermail/release/2015-July/003139.html
.. _spinup-scripts: https://git.opendaylight.org/gerrit/gitweb?p=releng/builder.git;a=tree;f=jenkins-scripts
Code Review / releng / builder.git / blobdiff