.. code-block:: bash
- git clone https://git.opendaylight.org/gerrit/releng/builder
+ git clone --recursive https://git.opendaylight.org/gerrit/releng/builder
cd builder
mkdir jjb/<new-project>
+.. note:
+
+ releng/global-jjb is a submodule of releng/builder repository which
+ requires a git submodule update --init or using --recursive with git clone.
+ `releng-global-jjb`_
+
Where <new-project> should be the same name as your project's git repo in
-Gerrit. So if your project is called "aaa" then create a new jjb/aaa directory.
+Gerrit. If your project is called "aaa" then create a new jjb/aaa directory.
Next we will create <new-project>.yaml as follows:
.. code-block:: yaml
+ ---
- project:
name: <NEW_PROJECT>-carbon
jobs:
- - '{project-name}-clm-{stream}'
- - '{project-name}-distribution-{stream}'
- - '{project-name}-integration-{stream}'
- - '{project-name}-merge-{stream}'
- - '{project-name}-periodic-{stream}'
- - '{project-name}-verify-{stream}-{maven}-{jdks}'
+ - '{project-name}-clm-{stream}'
+ - '{project-name}-integration-{stream}'
+ - '{project-name}-merge-{stream}'
+ - '{project-name}-verify-{stream}-{maven}-{jdks}'
project: '<NEW_PROJECT>'
project-name: '<NEW_PROJECT>'
branch: 'master'
jdk: openjdk8
jdks:
- - openjdk8
+ - openjdk8
maven:
- - mvn33:
- mvn-version: '{mvn33}'
+ - mvn33:
+ mvn-version: 'mvn33'
mvn-settings: '<NEW_PROJECT>-settings'
mvn-goals: 'clean install -Dmaven.repo.local=/tmp/r -Dorg.ops4j.pax.url.mvn.localRepository=/tmp/r'
mvn-opts: '-Xmx1024m -XX:MaxPermSize=256m'
- project:
name: <NEW_PROJECT>-sonar
jobs:
- - '{project-name}-sonar'
+ - '{project-name}-sonar'
project: '<NEW_PROJECT>'
project-name: '<NEW_PROJECT>'
Replace all instances of <new-project> with the name of your project. This will
create the jobs with the default job types we recommend for Java projects. If
your project is participating in the simultanious-release and ultimately will
-be included in the final distribution. We recommend adding the following job
+be included in the final distribution, it is required to add the following job
types into the job list for the release you are participating.
handling jobs on it.
Pool: ODLRPC
-^^^^^^^^^^^^^^^^^^^
+^^^^^^^^^^^^
.. raw:: html
<table class="table table-bordered">
<tr class="warning">
- <td><b>Jenkins Labels</b><br/> dynamic_controller, dynamic_verify,
- dynamic_merge, centos7-java-builder-2c-4g, centos7-java-builder-2c-8g,
- centos7-java-builder-4c-8g</td>
+ <td><b>Jenkins Labels</b><br/> centos7-java-builder-2c-4g,
+ centos7-java-builder-2c-8g, centos7-java-builder-4c-8g,
+ centos7-java-builder-8c-8g, centos7-java-builder-4c-16g</td>
<td><b>Minion Template names</b><br/> centos7-java-builder-2c-4g,
- centos7-java-builder-2c-8g, centos7-java-builder-2c-8g,
- centos7-java-builder-4c-8g, centos7-java-builder-8c-8g</td>
+ centos7-java-builder-2c-4g, centos7-java-builder-2c-8g,
+ centos7-java-builder-4c-8g, centos7-java-builder-8c-8g,
+ centos7-java-builder-4c-16g</td>
<td><b>Packer Template</b><br/>
releng/builder/packer/templates/java-builder.json</td>
<td><b>Spinup Script</b><br/>
</tr>
<tr>
<td colspan="4">
- A CentOS 7 huild minion. This system has OpenJDK 1.7 (Java7) and OpenJDK
- 1.8 (Java8) installed on it along with all the other components and
- libraries needed for building any current OpenDaylight project. This is
- the label that is used for all basic verify, merge and daily builds for
+ CentOS 7 build minion configured with OpenJDK 1.7 (Java7) and OpenJDK
+ 1.8 (Java8) along with all the other components and libraries needed
+ for building any current OpenDaylight project. This is the label that
+ is used for all basic verify, merge and daily builds for
projects.
</td>
</tr>
<tr class="warning">
- <td><b>Jenkins Labels</b><br/> dynamic_robot, centos7-robot-2c-2g</td>
+ <td><b>Jenkins Labels</b><br/> centos7-robot-2c-2g</td>
<td><b>Minion Template names</b><br/> centos7-robot-2c-2g</td>
<td><b>Packer Template</b><br/>
releng/builder/packer/templates/robot.json</td>
</tr>
<tr>
<td colspan="4">
- A CentOS 7 minion that is configured with OpenJDK 1.7 (Java7), OpenJDK
+ CentOS 7 minion configured with OpenJDK 1.7 (Java7), OpenJDK
1.8 (Java8) and all the current packages used by the integration
project for doing robot driven jobs. If you are executing robot
framework jobs then your job should be using this as the minion that
</tr>
<tr class="warning">
- <td><b>Jenkins Labels</b><br/> ubuntu_mininet, ubuntu-trusty-mininet-2c-2g</td>
- <td><b>Minion Template names</b><br/> ubuntu-trusty-mininet-2c-2g</td>
+ <td><b>Jenkins Labels</b><br/> ubuntu1404-mininet-2c-2g</td>
+ <td><b>Minion Template names</b><br/> ubuntu1404-mininet-2c-2g</td>
<td><b>Packer Template</b><br/>
releng/builder/packer/teamplates/mininet.json</td>
<td><b>Spinup Script</b><br/> releng/builder/jenkins-scripts/mininet-ubuntu.sh</td>
</tr>
<tr>
<td colspan="4">
- Basic Ubuntu system with ovs 2.0.2 and mininet 2.1.0
+ Basic Ubuntu 14.04 (Trusty) system with ovs 2.0.2 and mininet 2.1.0
</td>
</tr>
<tr class="warning">
- <td><b>Jenkins Labels</b><br/> ubuntu_mininet_ovs_23,
- ubuntu-trusty-mininet-ovs-23-2c-2g</td>
- <td><b>Minion Template names</b><br/> ubuntu-trusty-mininet-ovs-23-2c-2g</td>
+ <td><b>Jenkins Labels</b><br/> ubuntu1404-mininet-ovs-23-2c-2g</td>
+ <td><b>Minion Template names</b><br/> ubuntu1404-mininet-ovs-23-2c-2g</td>
<td><b>Packer Template</b><br/> releng/builder/packer/templates/mininet-ovs-2.3.json</td>
<td><b>Spinup Script</b><br/> releng/builder/jenkins-scripts/mininet-ubuntu.sh</td>
</tr>
<tr>
<td colspan="4">
- Basic Ubuntu system with ovs 2.3 and mininet 2.2.1
- </td>
- </tr>
-
- <tr class="warning">
- <td><b>Jenkins Labels</b><br/> ubuntu_mininet_ovs_25,
- ubuntu-trusty-mininet-ovs-25-2c-2g</td>
- <td><b>Minion Template names</b><br/> ubuntu-trusty-mininet-ovs-25-2c-2g</td>
- <td><b>Packer Template</b><br/> releng/builder/packer/templates/mininet-ovs-2.5.json</td>
- <td><b>Spinup Script</b><br/> releng/builder/jenkins-scripts/mininet-ubuntu.sh</td>
- </tr>
- <tr>
- <td colspan="4">
- Basic Ubuntu system with ovs 2.5 and mininet 2.2.2
+ Ubuntu 16.04 (Xenial) system with ovs 2.5 and mininet 2.2.1
</td>
</tr>
<tr class="warning">
- <td><b>Jenkins Labels</b><br/> dynamic_devstack, centos7-devstack-2c-4g</td>
+ <td><b>Jenkins Labels</b><br/> centos7-devstack-2c-4g</td>
<td><b>Minion Template names</b><br/> centos7-devstack-2c-4g</td>
<td><b>Packer Template</b><br/> releng/builder/packer/templates/devstack.json</td>
<td><b>Spinup Script</b><br/> releng/builder/jenkins-scripts/devstack.sh</td>
</tr>
<tr>
<td colspan="4">
- A CentOS 7 system purpose built for doing OpenStack testing using
+ CentOS 7 system purpose built for doing OpenStack testing using
DevStack. This minion is primarily targeted at the needs of the OVSDB
project. It has OpenJDK 1.7 (aka Java7) and OpenJDK 1.8 (Java8) and
other basic DevStack related bits installed.
</tr>
<tr class="warning">
- <td><b>Jenkins Labels</b><br/> dynamic_docker, centos7-docker-2c-4g</td>
+ <td><b>Jenkins Labels</b><br/> centos7-docker-2c-4g</td>
<td><b>Minion Template names</b><br/> centos7-docker-2c-4g</td>
<td><b>Packer Template</b><br/> releng/builder/packer/templates/docker.json</td>
<td><b>Spinup Script</b><br/> releng/builder/jenkins-scripts/docker.sh</td>
</tr>
<tr>
<td colspan="4">
- A CentOS 7 system that is configured with OpenJDK 1.7 (aka Java7),
+ CentOS 7 system configured with OpenJDK 1.7 (aka Java7),
OpenJDK 1.8 (Java8) and Docker. This system was originally custom
built for the test needs of the OVSDB project but other projects have
expressed interest in using it.
</tr>
<tr class="warning">
- <td><b>Jenkins Labels</b><br/> gbp_trusty, ubuntu-trusty-gbp-2c-2g</td>
- <td><b>Minion Template names</b><br/> ubuntu-trusty-gbp-2c-2g</td>
+ <td><b>Jenkins Labels</b><br/> ubuntu1404-gbp-2c-2g</td>
+ <td><b>Minion Template names</b><br/> ubuntu1404-gbp-2c-2g</td>
<td><b>Packer Template</b><br/> releng/builder/packer/templates/gbp.json</td>
<td><b>Spinup Script</b><br/> releng/builder/jenkins-scripts/ubuntu-docker-ovs.sh</td>
</tr>
<tr>
<td colspan="4">
- A basic Ubuntu node with latest OVS and docker installed. Used by Group Based Policy.
+ Ubuntu 14.04 (Trusty) node with latest OVS and docker installed. Used by Group Based Policy.
</td>
</tr>
+
+ <tr class="warning">
+ <td><b>Jenkins Labels</b><br/> ubuntu1604-gbp-2c-4g</td>
+ <td><b>Minion Template names</b><br/> ubuntu1604-gbp-2c-4g</td>
+ <td><b>Packer Template</b><br/> releng/builder/packer/templates/gbp.json</td>
+ <td><b>Spinup Script</b><br/> releng/builder/jenkins-scripts/ubuntu-docker-ovs.sh</td>
+ </tr>
+ <tr>
+ <td colspan="4">
+ Ubuntu 16.04 (Xenial) node with latest OVS and docker installed. Used by Group Based Policy.
+ </td>
+ </tr>
+
</table>
+Pool: ODLPUB - HOT (Heat Orchestration Templates)
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+HOT integration enables to spin up integration labs servers for CSIT jobs
+using heat, rathar than using jclouds (deprecated). Image names are updated
+on the project specific job templates using the variable
+`{odl,docker,openstack,tools}_system_image` followed by image name in the
+format `<platform> - <template> - <date-stamp>`.
+
+.. code-block:: yaml
+
+ CentOS 7 - docker - 20161031-0802
+
+.. include:: cloud-images.rst
+
Creating Jenkins Jobs
---------------------
to manage JJB and its
Python dependencies. The `python-virtualenvwrapper`_ tool can help you do so.
-There are good docs for installing `python-virtualenvwrapper`_. On Linux systems
-with pip (typical), they amount to:
+Documentation is available for installing `python-virtualenvwrapper`_. On Linux
+systems with pip (typical), they amount to:
.. code-block:: bash
.. code-block:: bash
- $ git clone https://git.opendaylight.org/gerrit/p/releng/builder.git
+ $ git clone --recursive https://git.opendaylight.org/gerrit/p/releng/builder.git
Before actually installing JJB and its dependencies, make sure you've `created
and activated <Virtual Environments_>`_ a virtual environment for JJB.
(jjb)$ jenkins-jobs --version
+TODO: Explain that only the currently merged jjb/requirements.txt is supported,
+other options described below are for troubleshooting only.
+
To change the version of JJB specified by `builder/jjb/requirements.txt
<odl-jjb-requirements.txt_>`_
to install from the latest commit to the master branch of JJB's git repository:
All jobs have a default build-timeout value of 360 minutes (6 hrs) but can be
overrided via the opendaylight-infra-wrappers' build-timeout property.
+TODO: Group jobs into categories: every-patch, after-merge, on-demand, etc.
+TODO: Reiterate that "remerge" triggers all every-patch jobs at once,
+because when only a subset of jobs is triggered, Gerrit forgets valid -1 from jobs outside the subset.
+TODO: Document that only drafts and commit-message-only edits do not trigger every-patch jobs.
+TODO: Document test-{project}-{feature} and test-{project}-all.
+
.. raw:: html
<table class="table table-bordered">
- <tr class="warning">
- <td><b>Job Template</b><br/>{project}-distribution-{stream}</td>
- <td><b>Gerrit Trigger</b><br/>test-distribution</td>
- </tr>
- <tr>
- <td colspan="2">
- This job builds a distrbution against your patch, tiggers distribution sanity CSIT jobs
- and reports back the results to Gerrit. Leave a comment with trigger keyword above
- to activate it for a particular patch.
-
- This job is maintained by the <a href="https://wiki.opendaylight.org/view/Integration/Test">Integration/Test</a>
- project.
-
- <div class="admonition note">
- <p class="first admonition-title">Note</p>
- <p>
- Running the "test-distribution" trigger will cause Jenkins to
- remove it's existing vote if it's already -1 or +1'd a comment.
- You will need to re-run your verify jobs (recheck) after running
- this to get Jenkins to put back the correct vote.
- </p>
- </div>
- </td>
- </tr>
-
<tr class="warning">
<td><b>Job Template</b><br/>{project}-distribution-check-{stream}</td>
- <td><b>Gerrit Trigger</b><br/>recheck | redistcheck</td>
+ <td><b>Gerrit Trigger</b><br/>recheck</td>
</tr>
<tr>
<td colspan="2">
This job runs the PROJECT-distribution-check-BRANCH job which is
building also integration/distribution project in order to run SingleFeatureTest.
-
- The <b>redistcheck</b> trigger is useful in cases where a project's
- other jobs passed, however this job failed due to infra problems or
- intermittent issues. It will retrigger just this job to save time.
-
- BEWARE: If there were other failed jobs, redistcheck could lead
- to false Verified+1 vote, risking a merge which breaks other projetcs.
- Redistcheck is only for committers who are familiar with the risks involved.
- If in doubt, use the safe trigger word: recheck.
- Recheck triggers every job involved in verifying latest patch set in the Change.
+ It also performs various other checks in order to prevent the change to break autorelease.
</td>
</tr>
Artifacts are uploaded to OpenDaylight's
<a href="https://nexus.opendaylight.org">Nexus</a> on completion.
+ A distribution-merge-{stream} job is triggered to add the new artifacts to the
+ integration distribution.
+
Running the "remerge" trigger is possible before a Change is merged,
- in which case it will cause Jenkins to remove it's existing vote
- if it's already -1 or +1'd a comment.
- You will need to re-run your verify jobs (recheck) after running
- this to get Jenkins to put back the correct vote.
+ it would still build the actual HEAD. This job does not alter Gerrit votes.
</td>
</tr>
<tr class="warning">
<td><b>Job Template</b><br/>{project}-validate-autorelease-{stream}</td>
- <td><b>Gerrit Trigger</b><br/>recheck | revalidate</td>
+ <td><b>Gerrit Trigger</b><br/>recheck</td>
</tr>
<tr>
<td colspan="2">
This job runs the PROJECT-validate-autorelease-BRANCH job which is
used as a quick sanity test to ensure that a patch does not depend on
features that do not exist in the current release.
-
- The <b>revalidate</b> trigger is useful in cases where a project's
- other job passed, however this job failed due to infra problems or
- intermittent issues. It will retrigger just this job to save time.
-
- BEWARE: If there were other failed jobs, revalidate could lead
- to false Verified+1 vote, risking a merge which breaks other projetcs.
- Revalidate is only for committers who are familiar with the risks involved.
- If in doubt, use the safe trigger word: recheck.
</td>
</tr>
<tr class="warning">
<td><b>Job Template</b><br/>{project}-verify-{stream}-{maven}-{jdks}</td>
- <td><b>Gerrit Trigger</b><br/>recheck | reverify</td>
+ <td><b>Gerrit Trigger</b><br/>recheck</td>
</tr>
<tr>
<td colspan="2">
The Verify job template creates a Gerrit Trigger job that will
trigger when a new patch is submitted to Gerrit.
The job only builds the project code (including unit and integration tests).
-
- The <b>reverify</b> trigger is useful in cases where a project's
- other jobs passed however this job failed due to infra problems or
- intermittent issues. It will retrigger just this job to save time.
-
- BEWARE: If there were other failed jobs, reverify could lead
- to false Verified+1 vote, risking a merge which breaks other projetcs.
- Reverify is only for committers who are familiar with the risks involved.
- If in doubt, use the safe trigger word: recheck.
- Recheck triggers every job involved in verifying latest patch set in the Change.
</td>
</tr>
<tr class="warning">
<td><b>Job Template</b><br/>{project}-verify-node-{stream}</td>
- <td><b>Gerrit Trigger</b><br/>recheck | renode</td>
+ <td><b>Gerrit Trigger</b><br/>recheck</td>
</tr>
<tr>
<td colspan="2">
{nodever} containing the directory relative to the project root
containing the nodejs package.json and version of node you wish to
run tests with.
-
- The <b>renode</b> trigger is useful in cases where a project's
- other jobs passed, however this job failed due to infra problems or
- intermittent issues. It will retrigger just this job to save time.
-
- BEWARE: If there were other failed jobs, renode could lead
- to false Verified+1 vote, risking a merge which breaks other projetcs.
- Renode is only for committers who are familiar with the risks involved.
- If in doubt, use the safe trigger word: recheck.
- Recheck triggers every job involved in verifying latest patch set in the Change.
</td>
</tr>
<tr class="warning">
<td><b>Job Template</b><br/>{project}-verify-python-{stream} | {project}-verify-tox-{stream}</td>
- <td><b>Gerrit Trigger</b><br/>recheck | retox</td>
+ <td><b>Gerrit Trigger</b><br/>recheck</td>
</tr>
<tr>
<td colspan="2">
The 2 template names verify-python & verify-tox are identical and are
aliases to each other. This allows the project to use the naming that
is most reasonable for them.
+ </td>
+ </tr>
- The <b>retox</b> trigger is useful in cases where a project's
- other verify jobs passed, however this job failed due to infra problems or
- intermittent issues. It will retrigger just this job to save time.
-
- BEWARE: If there were other failed jobs, retox could lead
- to false Verified+1 vote, risking a merge which breaks other projetcs.
- Retox is only for committers who are familiar with the risks involved.
- If in doubt, use the safe trigger word: recheck.
- Recheck triggers every job involved in verifying latest patch set in the Change.
+ <tr class="warning">
+ <td><b>Job Template</b><br/>integration-patch-test-{stream}</td>
+ <td><b>Gerrit Trigger</b><br/>test-integration</td>
+ </tr>
+ <tr>
+ <td colspan="2">
</td>
</tr>
</tr>
<tr>
<td colspan="2">
- This job runs a full integration test suite against your patch and
- reports back the results to Gerrit. Leave a comment with trigger
- keyword above to activate it for a particular patch.
+ This job builds a distribution against your Java patch and triggers distribution sanity CSIT jobs.
+ Leave a comment with trigger keyword above to activate it for a particular patch.
+ This job should not alter Gerrit votes for a given patch.
- It then spawns the list of jobs in csit-list defined
+ The list of CSIT jobs to trigger is defined in csit-list
<a href="https://git.opendaylight.org/gerrit/gitweb?p=releng/builder.git;a=blob;f=jjb/integration/integration-test-jobs.yaml">here</a>.
- This job is maintained by the <a href="https://wiki.opendaylight.org/view/Integration/Test">Integration/Test</a>
- project.
-
- <div class="admonition note">
- <p class="first admonition-title">Note</p>
- <p>
- Running the "test-integration" trigger will cause Jenkins to remove
- it's existing vote if it's already -1 or +1'd a comment. You will need
- to re-run your verify job (recheck) after running this to get Jenkins
- to put back the correct vote.
- </p>
- </div>
-
Some considerations when using this job:
<ul>
<li>
.. code-block:: bash
# If you don't have RelEng/Builder's repo, clone it
- $ git clone https://git.opendaylight.org/gerrit/p/releng/builder.git
+ $ git clone --recursive https://git.opendaylight.org/gerrit/p/releng/builder.git
# Make a copy of the example JJB config file (in the builder/ directory)
$ cp jenkins.ini.example jenkins.ini
# Edit jenkins.ini with your username, API token and ODL's sandbox URL
.. _releng-wiki: https://wiki.opendaylight.org/view/RelEng:Main
.. _releng-builder-gerrit: https://git.opendaylight.org/gerrit/#/admin/projects/releng/builder
.. _releng-builder-repo: https://git.opendaylight.org/gerrit/gitweb?p=releng%2Fbuilder.git;a=summary
+.. _releng-global-jjb: https://gerrit.linuxfoundation.org/infra/#/q/project:releng/global-jjb
.. _releng-builder-wiki: https://wiki.opendaylight.org/view/RelEng/Builder
.. _streams-design-background: https://lists.opendaylight.org/pipermail/release/2015-July/003139.html
.. _spinup-scripts: https://git.opendaylight.org/gerrit/gitweb?p=releng/builder.git;a=tree;f=jenkins-scripts
.. _spinup-scripts-basic_settings.sh: https://git.opendaylight.org/gerrit/gitweb?p=releng/builder.git;a=blob;f=jenkins-scripts/basic_settings.sh
.. _spinup-scripts-controller.sh: https://git.opendaylight.org/gerrit/gitweb?p=releng/builder.git;a=blob;f=jenkins-scripts/controller.sh
-
Code Review / releng / builder.git / blobdiff