Make bgpcep jobs run with its mri distro

Signed-off-by: Tomas Cere <[email protected]>
Change-Id: I5bb099af2b6c58d1296adcb76f67d7353475dcc8

Fix: JCasC settings file for packaging

Signed-off-by: Anil Belur <[email protected]>
Change-Id: I9889e4f65e31845cdbad69b6afe810381cf7ff20

Chore: Remove unneeded VOLUME_SIZE parameters

global-jjb v0.68.1 resolved an issue with certain instance definitions
requiring a VOLUME_SIZE parameter. As it is no longer needed, remove it
from all instance definitions that do not actually need it.

Change-Id: If43662838b5973d5885df2be6a45a74a5819bd7a
Signed-off-by: Andrew Grimberg <[email protected]>

Chore: pre-commit update dependencies

Change-Id: If6795314cce1c6d42657f05940c71bc591b036c9
Signed-off-by: Anil Belur <[email protected]>

Chore: Update global-jjb to the latest v0.68.1

- Pull in Fix: Pin pyparsing<3.0.0 required by httplib2

Issue-ID: LF JIRA RELENG-4022
Change-Id: I971ad47cc35f55a78f932238b0269de7da96ee79
Signed-off-by: Anil Belur <[email protected]>

Merge "Remove odl-elastic-cloud dependency"

Merge "Failed tests need to report as failed"

Failed tests need to report as failed

Currently only critical tests show as failed.

Change-Id: Id72505f7d82f76e81cf05041ef09bad745dba5d5
Signed-off-by: Luis Gomez <[email protected]>

Remove odl-elastic-cloud dependency

Change-Id: I6d3f586bc35760283f0e46980ffca05164eb0f10
Signed-off-by: Luis Gomez <[email protected]>

Merge "Update docker jobs for phosphorus"

Fix: Update lftools.ini managed config

The CR aff2a708c2a introduced a typo in the
config file. Use the corect credentialID

Issue-ID: https://jira.linuxfoundation.org/browse/RELENG-4013
Signed-off-by: Anil Belur <[email protected]>
Change-Id: I4cd3e6300d202b13f4fcc28166164f90e41908c6

Fix: Update init script to clone releng/builder

The CR aff2a708c2a updated the jenkins-init script to be
picked up from global-jjb. However ODL requires openstack-hot
templates for CSIT stack creation and requires releng/builder
repo in /opt/ciman.

This should fix the issue with the CSIT stack creation
failures.

Issue-ID: https://jira.linuxfoundation.org/browse/RELENG-4014
Signed-off-by: Anil Belur <[email protected]>
Change-Id: I3ae6277803e706429c77843131ab29a38bc5813d

Fix: Update lftools.ini managed config

The CR aff2a708c2a introduced a typo in the
config file. The config file has been updated
to point to the ODL Nexus

Issue-ID: https://jira.linuxfoundation.org/browse/RELENG-4013
Signed-off-by: Anil Belur <[email protected]>
Change-Id: I1e80c7114b16e8686fb4c19abbcbf2f98f696f4f

Fix: Add Sigul signing public keys

The CR aff2a708c2a missed updating the signing
public keys required for verifying release artifacts
used by the release merge job. This CR adds the
keys into JCasC config.

Issue-ID: https://jira.linuxfoundation.org/browse/RELENG-4013
Signed-off-by: Anil Belur <[email protected]>
Change-Id: I78240a956ef03201cc9f734077c7d1ed81a1812d

Update docker jobs for phosphorus

Change-Id: Ic520862b26dffd6cc641c27de059d3eb381c75aa
Signed-off-by: Luis Gomez <[email protected]>

Fix: Correct misspelled serverId in maven settings

The snapshot serverId is supposed to be opendaylight-snapshot.
Additionally, the sandbox is supposed to have the same serverIds as
production in the settings files, just with the "dummy" credential.

Change-Id: I8e0da28230aa88bd8e69d43195e8ca70cf3645a7
Signed-off-by: Andrew Grimberg <[email protected]>

Chore: Remove unneeded cfg merge job

With the transition to using JCasC for managing clouds, env vars, and
managed config files; the cfg-merge job is no longer needed and can
cause potential problems.

Change-Id: Ieaa7e3a13ba3dd5b90a722c924e702f6252349f7
Signed-off-by: Andrew Grimberg <[email protected]>

Fix: Symlink releng global vars as production

The JCasC converter currently only supports 'production' and 'sandbox'
configuration files for the global vars files. As the ODL Jenkins system
uses a 'silo' named 'releng' for the proudction equivalent, this causes
issues with the converter.

Change-Id: I028911bae245b91ab6e4423050b0ac759523b49a
Signed-off-by: Andrew Grimberg <[email protected]>

Fix: Cred ID for packer managed config file

Signed-off-by: Anil Belur <[email protected]>
Change-Id: I1fd508732973d07d33d25d66e8137c096cbbe65d

Feat: Add JCASC managed config files

Issue-ID: LF-Jira RELENG-3731
Change-Id: I31ad0fb4af91274a81a6bb7587515e8cc71e6683
Signed-off-by: Anil Belur <[email protected]>

Chore: Update common-packer to the latest v0.9.4

Signed-off-by: Anil Belur <[email protected]>
Change-Id: I8b2d6da20bab8e1051da1699ea61fdb35320f14b

Chore: Update global-jjb to the latest v0.68.0

- Pull in Fix: Correct bad conversion of OpenStack config

Signed-off-by: Anil Belur <[email protected]>
Change-Id: I127a1eaacc8bb3fbebd6366f12a5e019a05803c9

Chore: Set Cloud AZ and default storage size

The JCasC configuration for OpenStack is requiring that we set an AZ as
well as default storage size. Not doing so causes the outputed JCasC to
have bad data and not load.

Change-Id: I01696797bd709e1eac5c3fb6fd6d93a715827593
Signed-off-by: Andrew Grimberg <[email protected]>

Merge "Update cloud image list docs"

Merge "Feat!: Decommission log shipping on Nexus"

Merge "Chore: Update global-jjb to the latest v0.67.1"

Merge "CI: Update CentOS 8 builder image to latest"

Chore: Update global-jjb to the latest v0.67.1

Signed-off-by: Anil Belur <[email protected]>
Change-Id: Ic0ccafe914774909a39ebf32767c356e9861cff0

Update cloud image list docs

Job: builder-update-image-list/1245
Signed-off-by: jenkins-releng <[email protected]>
Change-Id: I08c955c135d4e5ce4496204200015fba7e7bb776

Fix verify-jdkversion

verify-jdkversion should be openjdk11 for silicon.

Signed-off-by: Sangwook Ha <[email protected]>
Change-Id: Iec2ca3a9bde447de90601975133f1cd4675932a1

CI: Update CentOS 8 builder image to latest

A new image is required to fix the expired LE's root
certificate.

Ref: https://lists.opendaylight.org/g/integration-dev/message/14515
Issue-ID: https://jira.linuxfoundation.org/browse/RELENG-3980
Signed-off-by: Anil Belur <[email protected]>
Change-Id: Ib796b7a38cc4d496579ab2a320eb0d76bc46cfc5

Merge "CI: Update CentOS 7 docker image"

CI: Update CentOS 7 docker image

A new image is required to fix the expired LE's root
certificate.

Ref: https://lists.opendaylight.org/g/integration-dev/message/14515
Issue-ID: https://jira.linuxfoundation.org/browse/RELENG-3980
Signed-off-by: Anil Belur <[email protected]>
Change-Id: I1b66c6bae396bbcf2658f2f883d1ed7d2ae00aa4

Merge "Update cloud image list docs"

Merge "Add infrautils-2.0.x jobs"

Add infrautils-2.0.x jobs

We have branched for phosphorus, create the jobs to support that stream.

Change-Id: Ifd7de860119f9576e1180d55290ef2f149d1b14e
Signed-off-by: Robert Varga <[email protected]>

Fix neutron job config

Neutron is self-managed since Phosphorus, hence does not have a
stability branch. Fix that. Also remove triggers from odlparent/aaa, as
those are release-integrated anyway.

Change-Id: Ic3caeb88f104c4192d81480015b42a08293cfa9c
Signed-off-by: Robert Varga <[email protected]>

Update cloud image list docs

Job: builder-update-image-list/1243
Change-Id: I07cc9b5cd78e61064ea173ee517de57a052bc4fb
Signed-off-by: jenkins-releng <[email protected]>

Feat!: Remove all stable/aluminium jobs

Phosphorus has been released which marks EOL
for aluminum branch.

This change will remove all the Aluminium jobs
from Jenkins CI.

Issue-ID: https://jira.linuxfoundation.org/browse/RELENG-3959
Signed-off-by: Anil Belur <[email protected]>
Change-Id: I5c8ca9a5a258caa853537107e996f6eb169d3b4b

CI: Update CentOS 7 devstack stein image

A new image is required to fix the expired LE's root
certificate.

Ref: https://lists.opendaylight.org/g/integration-dev/message/14515
Issue-ID: https://jira.linuxfoundation.org/browse/IT-22996
Signed-off-by: Anil Belur <[email protected]>
Change-Id: I9e2b9486cb5d67db437cb35f6baceef2925ddf59

CI: Update CentOS7 devstack rocky image

A new image is required to fix the expired LE's root
certificate.

Ref: https://lists.opendaylight.org/g/integration-dev/message/14515
Issue-ID: https://jira.linuxfoundation.org/browse/IT-22996
Signed-off-by: Anil Belur <[email protected]>
Change-Id: I0e25f3b78ca1a77a23701021df98a76ab949d92e

CI: Update CentOS 7 robot image

A new image is required to fix the expired LE's root
certificate.

Ref: https://lists.opendaylight.org/g/integration-dev/message/14515
Issue-ID: https://jira.linuxfoundation.org/browse/IT-22996
Signed-off-by: Anil Belur <[email protected]>
Change-Id: I674e6d53a1a9a9518da3c260545af06cd339baf4

CI: Update CentOS 7 builder image

A new image is required to fix the expired LE's root
certificate.

Ref: https://lists.opendaylight.org/g/integration-dev/message/14515
Issue-ID: https://jira.linuxfoundation.org/browse/IT-22996
Signed-off-by: Anil Belur <[email protected]>
Change-Id: I067c753021ab6d1d3034c3ba5a34a3d63d26c329

Revert "Chore: Update global-jjb to the latest v0.67.0"

This reverts commit 74f2a2b6ab41e4b1cd47a0564f9c9b3b42ae1d33.
The maven verify jobs are failing this change needs
to be reverted to global-jjb version v0.65.3.

Issue-ID: https://jira.linuxfoundation.org/browse/IT-22988
Signed-off-by: Anil Belur <[email protected]>
Change-Id: Ie98ab64d313d1e348c681df7d5b9df50f2f09736

Merge "Fix: Correct docker-name for verify/release job"

Fix: Correct docker-name for verify/release job

Change-Id: I1640d54f738d443d85f198532968312e47949b35
Signed-off-by: Anil Belur <[email protected]>

Merge "Chore: Update common-packer to the latest v0.9.3"

Merge "Chore: Update global-jjb to the latest v0.67.0"

Merge "Update validate-ar jobs for sulfur"

Update Robert's details

Fill in time zone and affiliation.

Change-Id: I4bc7b527dd905d9a5848771a416effa383907e0e
Signed-off-by: Robert Varga <[email protected]>

Update validate-ar jobs for sulfur

Job: autorelease-update-validate-jobs-sulfur/1
Signed-off-by: jenkins-releng <[email protected]>
Change-Id: I239e8f79ab0a72456058a664041f9541ee514df6

Merge "Update validate-ar jobs for phosphorus"

Merge "Prune SXP jobs"

Merge "Cleanup netvirt CSIT naming"

Cleanup netvirt CSIT naming

netvirt has two streams: master (self-managed) and aluminium (managed),
with the latter nearing removal. Prune all other branches and rename
silicon to master to stop automatic proliferation from happening.

Change-Id: I7a28bc1ddfa96023498ac2ff18710b33f3990dcf
Signed-off-by: Robert Varga <[email protected]>

Prune SXP jobs

SXP is a moribund project, which does not really build on its master
branch. Use naming for a self-managed project, pruning all streams
except master.

Change-Id: I050ee7eac21262fcc4ad5e88af3585480c5ae43f
Signed-off-by: Robert Varga <[email protected]>

Correct autorelease/test relationship

autorelease/master is sulfur, hence it should reference
integration-distribution-test-sulfur.

Change-Id: Ibb3b9f9051f2e0786eb0747cb194af0f32a7710d
Signed-off-by: Robert Varga <[email protected]>

Update validate-ar jobs for phosphorus

Job: autorelease-update-validate-jobs-phosphorus/34
Signed-off-by: jenkins-releng <[email protected]>
Change-Id: I2fd2739db00f9b8cf2561b5e13d52131a94e0545

Merge "Chore: update pre-commit repos revisions"

Feat: Add branch stable/phosphorus

Bump master branch to Sulfur and add new branch
stable/phosphorus.

Note: master stream renamed as 'Sulfur' instead of
'Sulpher'.

Issue-ID: LF-JIRA RELENG-3814
Change-Id: I7360f54d433b6523f1df67ad938df10d919f76b5
Signed-off-by: Anil Belur <[email protected]>

Neutron is no longer a MSI project

Switch job stream name from 'phosphorus' to 'master' and remove the
distcheck job as it is no longer relevant.

Change-Id: Ic74e48a02bf9f677e26c98fb8ba28d6570aeed5b
Signed-off-by: Robert Varga <[email protected]>

Chore: update pre-commit repos revisions

Signed-off-by: guillaume.lambert <[email protected]>
Change-Id: Ibf7a90d12efc6951b55f67f39211922049b6228f

Feat!: Decommission log shipping on Nexus

Log shipping has been enabled using AWS S3 buckets
for ODL since Nov, 2020. CR (a5ef8f87a9) confirms its
more than 6 months the jobs are pushing logs onto
S3 buckets.

Also this would help decreasing any performance impact
Nexus2, since logs contain a large number of small files
slowing down IO.

Issue-ID: LF-JIRA RELENG-3957
Signed-off-by: Anil Belur <[email protected]>
Change-Id: I428a98866784c2de35adb9beca91f9b159edc5d0

Chore: Update common-packer to the latest v0.9.3

Signed-off-by: Anil Belur <[email protected]>
Change-Id: Ie095be7a9a07c04605514c4d82bdae97efd40f93

Chore: Update global-jjb to the latest v0.67.0

Signed-off-by: Anil Belur <[email protected]>
Change-Id: Ia25478327159aac522f46332a8fef78dc0e84144

Fix: packaging job format

Fix packaging job format to the recommened JJB job
format to ensure that branch cutting scripts dont
break.

Change-Id: Id19eb7036a2109e2f299b691165fed0cb8c8edf7
Signed-off-by: Anil Belur <[email protected]>

Merge changes I3294ccd3,I61bdf8ab,I401510d9,I9bc41fdf,I6cfe024a, ...

* changes:
  CI: split and improve pre-commit tox job
  CI: make docs jobs logs more quiet
  CI: enforce docs-linkcheck
  Fix: docs broken URLs pointed by docs-linkcheck
  CI: add a docs-linkcheck tox profile
  CI: treat warnings as error in docs tox job
  Fix: docs job warning

CI: split and improve pre-commit tox job

- create pre-commit-install and pre-commit-uninstall tox profiles to
  (un)install hooks locally and (un)perform tests at each "git commit"
  call (i.e. without calling manually the pre-commit tox profile)
- precise pre-commit stages/types in the pre-commit configuration file
  so that hooks are installed correctly. This avoids messages about
  skipped tests when they are run at a wrong stage.

Signed-off-by: guillaume.lambert <[email protected]>
Change-Id: I3294ccd3257385ddaf2bfd8910547a0f24fe3748

CI: make docs jobs logs more quiet

Using sphinx option -q allows to produce less docs logs and keep only
warnings and errors.

Signed-off-by: guillaume.lambert <[email protected]>
Change-Id: I61bdf8abc8512f799d445cc70321d929043f4f7d

CI: enforce docs-linkcheck

Signed-off-by: guillaume.lambert <[email protected]>
Change-Id: I401510d9cf8a289f6e782b7911946b4c46301918

Fix: docs broken URLs pointed by docs-linkcheck

Signed-off-by: guillaume.lambert <[email protected]>
Change-Id: I9bc41fdfbffa94edf759bd9516e9adf6103bfaf0

CI: add a docs-linkcheck tox profile

Signed-off-by: guillaume.lambert <[email protected]>
Change-Id: I6cfe024ae04ede5a349dd004a2af0a75f1177e7a

CI: treat warnings as error in docs tox job

The previous commit fixed the last remaining warning in the docs job.
Adding in this job the option -W to sphinx will make it treat warnings
as error and will ensure there won't be anymore docs warnings that
pollutes jenkins logs in the future.

Signed-off-by: guillaume.lambert <[email protected]>
Change-Id: I5101eceb98aa2bb0ef8b922d871b8e68af0893ef

Fix: docs job warning

Signed-off-by: guillaume.lambert <[email protected]>
Change-Id: Iee942283d9239a24a0e83b9aa65980686bb086e8

Chore: Add new committers to releng/builder

TSC Approval:
https://wiki.opendaylight.org/x/ngkEAQ
https://wiki.opendaylight.org/x/nAkEAQ

Change-Id: Ib5aa523052ad5c17faa68d88d7f18931862ed28c
Signed-off-by: Anil Belur <[email protected]>

CI: Add branch cutting awk scripts

The CR 97b4dbd2fcc9e7018 replaced the branch cutting awk
script with python.

However, the awk scripts written are more stable for branch
cutting but were removed without python script being
tested properly.

The python script fails preserve the comment lines
and does not work when anchors are used in the projects.

Signed-off-by: Anil Belur <[email protected]>
Change-Id: I5318a7cc93f75d21552916935632ce4a4ea0a0cf

Merge "Fix: Include missing quotes around $var"

Fix: Include missing quotes around $var

The CR d3079426375d requires quotes for variable
substitution work as expected within the sed regex.

Issue-ID: https://jira.linuxfoundation.org/browse/RELENG-3931
Signed-off-by: Anil Belur <[email protected]>
Change-Id: I4080e18f0e42f525b23cb32146df814abea922fd

Update cloud image list docs

Job: builder-update-image-list/1227
Change-Id: Ideec007bd4cacb1a82b93d288cb37bf33d1d1e9e
Signed-off-by: jenkins-releng <[email protected]>

ODL docker name will be opendaylight

Change-Id: I70358d6900da145397eca7333ef51ace4a22507d
Signed-off-by: Luis Gomez <[email protected]>

Reconfigure docker jobs to look like O-RAN

Change-Id: I0672130742bdd1a459e326e594ceeac0702266f0
Signed-off-by: Luis Gomez <[email protected]>

Merge "Fix: Include file name in index.rst"

Add karaf_version param to mri-stage

Looks like this param is needed since its used
to lookup the actual artifact id, which is used
to build up the correct bundle_url.

Signed-off-by: Tomas Cere <[email protected]>
Change-Id: I342da21a119b40530b43d83421e1e200fe6a58c9

Fix: Include file name in index.rst

The missing file-name in the index.rst fails the verify job.

Issue-ID: RELENG-3931
Signed-off-by: Anil Belur <[email protected]>
Change-Id: Ib64bbd602e3ba7806e13d155ccede46e71d29641

Do not pass KARAF_VERSION in mri-stage job

Leftover from not being able to test stage to nexus
from sandbox. This is causing issues since its not set in
the mri-stage job and its not needed anyway since we
have KARAF_VERSION set in the downstream job.

Signed-off-by: Tomas Cere <[email protected]>
Change-Id: I74d5c65fbff1f4893435893c8c594d7275e4e4fa

Fix up xpath syntax in mri-stage script

We are using a different xpath version in our stack
so fixup the syntax so it actually runs.

Signed-off-by: Tomas Cere <[email protected]>
Change-Id: Iff38ac1c5b79ffe761c0c6c099033b2169ce704e

Merge changes I473ef076,I036098e2

* changes:
  Add mri-stage job
  Allow usage of MRI distribution in csit

Add yangtools-7.0.x jobs

We are just about ready to branch yangtools-7.0.x, add the jobs to
support that stream.

Change-Id: Ie34a4e7719888052f959e4581fe3f79a83bac99d
Signed-off-by: Robert Varga <[email protected]>

Increase timeout for bgpcep-release-merge

bgpcep is another project which takes longer than 15 minutes to finish,
increase its timeout.

Change-Id: Ia7ea9019add6d44c67740d23a45c7fbff089e917
Signed-off-by: Robert Varga <[email protected]>

Chore: Update global-jjb to the latest v0.65.3

Change-Id: I8e655f7033f0ce49ddb3c2d110b9364c7c942aa8
Signed-off-by: Anil Belur <[email protected]>

Increase job timeout for yangtools-release-merge

This job is now waiting for nexus promotion, which does not complete in
15 minutes. Bump the timeout to 90 minutes to be on the safer side.

Change-Id: Id2189d43c3414329cbd65a936fb60e0444ff0d50
Signed-off-by: Robert Varga <[email protected]>

Increase job timeout for infrautils-release-merge

This job is now properly waiting for artifacts to be promoted and with
current nexus performance that does not happen in 15 minutes. Raise the
timeout to an hour.

Change-Id: Idbefcb710be60e3af52a4059d35450c8699b277e
Signed-off-by: Robert Varga <[email protected]>

Add mri-stage job

This job stages a project and then triggers an mri
distribution test job which runs csit on the staged
artifacts from the staging repo.

Add netconf stage/distribution job definitions
Repoint netconf streams to master

Change-Id: I473ef076e22df17bb5e0c0f7a9e7adcac0bd2822
Signed-off-by: Tomas Cere <[email protected]>

Allow usage of MRI distribution in csit

Add new config values for karaf_version parameter
Add necessary handling of the MRI distros in the
integration scripts
Change netconf jobs to use the MRI distro

Change-Id: I036098e24eaffa979477242b9329c97997cfdd71
Signed-off-by: Tomas Cere <[email protected]>

Increase controller-release-merge build timeout

release-merge job now seems to be correctly waiting for repo promotion,
hence the default 15 minutes do not suffice. Bump to 2 hours as a first
cut (which is ~2x the time it takes to create and test a release).

Change-Id: Ibf26e3175ed90841f1ff21249863a94e6dfe00fb
Signed-off-by: Robert Varga <[email protected]>

Lower genius job memory usage

Genius is a rather small project, there is no reason why it should
require 8GiB of memory to build. Use cheaper 4c-4g nodes instead of
2c-8g.

Change-Id: Ib45b12550ac30c9e01f2e8312aab0d3fed91fa3c
Signed-off-by: Robert Varga <[email protected]>

Lower memory usage in phosphorus ovsdb jobs

8GiB nodes are quite wasteful for ovsdb jobs. Opt for cheaper 4c-4g
nodes instead of 2c-8g nodes.

Change-Id: I4370ae4a61d5123465d40da93bf4da9d79466a48
Signed-off-by: Robert Varga <[email protected]>

Lower memory usage in openflowplugin jobs

8GiB nodes are quite wasteful for ovsdb jobs. Opt for cheaper 4c-4g
nodes instead of 2c-8g nodes.

Change-Id: I6947ff20f048f081523c8cc0583261aaaf8f561e
Signed-off-by: Robert Varga <[email protected]>

Increase netconf-release-merge build timeout

release-merge job now seems to be correctly waiting for repo promotion,
hence the default 15 minutes do not suffice. Bump to 2 hours as a first
cut.

Change-Id: Ieeb59b646f9f821093d0d406dcb22bb8924fdb40
Signed-off-by: Robert Varga <[email protected]>

Increase aaa-release-merge build timeout

release-merge job now seems to be correctly waiting for repo promotion,
hence the default 15 minutes do not suffice. Bump to 1 hour as a first
cut (which is 6x the time it takes to create and test a release).

Change-Id: Ie74ddc1f9931d27a17b53424c000539bd482c342
Signed-off-by: Robert Varga <[email protected]>