## Building Debs
-To build .deb in the Vagrant env, use Debian packaging tools
+The `build.py` helper script is used for building OpenDaylight .debs.
- [vagrant@localhost vagrant/opendaylight]$ dpkg-buildpackage -us -uc -b
+The `build.py` helper script can build a set of .debs based on provided
+version arguments.
+
+ [vagrant@localhost ~]$ /vagrant/build.py -h
+ usage: build.py [-h] [-v [major minor patch deb [major minor patch deb ...]]]
+ ....
+
+ optional arguments:
+ -h, --help show this help message and exit
+
+ Existing build:
+ -v [major minor patch deb [major minor patch deb ...]], --version [major minor patch deb [major minor patch deb ...]]
+ Deb version(s) to build
+ ....
+
+The `-v`/`--version` flag accepts a version number. Any build that matches
+the portions provided will be built. If more than one build matches the
+portions provided, all matching builds will be executed.
+
+For example, `build.py -v 3` would execute the builds that match the regex
+`3.*.*-*`. OpenDaylight 3.0.0-1 and 3.1.0-1, for example.
+
+To only build a single debian package, provide enough version info to make
+the match unique. For example, `build.py -v 2 4 0 1` could only match one
+definition (Helium SR4, 2.4.0-1).
+
+The `build.py` script uses `templates/build_debianfiles.py` to generate debian files for
+ODL .deb packages using JinJa2 templates and dynamic build data provided in `build_vars.yaml`.
## Defining New Debs
-NB: Functionality under construction
-TODO: Docs about how to define debs for new ODL versions
+The dynamic aspects of a build, such as ODL and Deb version info, have all
+been extracted to a single YAML configuration file. For most Deb updates,
+only that configuration file should need to be updated by humans.
+
+The variables available for configuration are documented below. A build
+definition should define all supported variables.
### Deb Build Variables
-NB: Functionality under construction
-TODO: More of these var defs as vars added
+#### `version_major`
+
+The OpenDaylight major (element) version number of the release to build.
+
+For example, Hydrogen is 1.x.x, Helium is 2.x.x, Lithium is 3.x.x and
+so on down the periodic table.
+
+#### `version_minor`
+
+The OpenDaylight minor (SR) version number of the release to build.
+
+#### `version_patch`
+
+The OpenDaylight patch version of the release to build.
+
+#### `pkg_version`
+
+Debian revision for the given ODL major.minor.patch version.
+
+In addition to OpenDaylight's version, .debs themselves have versions. These
+are called "debian revisions". For a given OpenDaylight major.minor.patch
+version, there will be one or more major.minor.patch-pkg_version .debs.
+
+#### `sysd_commit`
+
+Version of ODL systemd unitfile to download and package in ODL .deb.
+
+The version of OpenDaylight's systemd unitfile, specified by the
+git commit hash, is downloaded from the [Int/Pack repo][16] and
+consumed by OpenDaylight's Deb builds.
+
+#### `codename`
+
+Elemental codename for the ODL release, including SR if applicable.
+
+Examples include "Helium-SR4", "Lithium" and "Lithium-SR1".
+
+#### `download_url`
+
+The ODL Deb repackages the tarball build artifact produced by ODL's
+autorelease build. This is the URL to the tarball ODL build to repackage
+into a Debian package.
+
+#### `java_version`
+
+Java dependency for specific ODL builds
+
+#### `changelog`
+
+Entry in the debian/changelog file for specific .deb.
-#### `TODO: Var name`
+A debian/changelog file for specialized ODL builds is generated using these entries.
-TODO: Description
+The changelog entry must follow a specific format. It's best to follow the
+examples provided by existing build definitions closely.
## Using Debs
### Installing
-To install a local OpenDaylight .deb package, use sudo dpkg -i <path to ODL .deb>
+To install a local OpenDaylight .deb package and resolve its dependencies, use `sudo gdebi <path to ODL .deb>`
Here's a walk-through of an install and the resulting system changes.
# Note that there are no ODL systemd files before the install
[vagrant@localhost vagrant]$ ls /lib/systemd/system | grep -i opendaylight
# Install an ODL .deb package
- [vagrant@localhost vagrant]$ sudo dpkg -i opendaylight_0.4.2-Beryllium-SR2-0_amd64.deb
+ [vagrant@localhost vagrant]$ sudo gdebi opendaylight/opendaylight_0.4.2-Beryllium-SR2-0_amd64.deb
# Note that ODL is now installed in /opt
[vagrant@localhost vagrant]$ ls /opt/
opendaylight
# vi: set ft=ruby :
Vagrant.configure("2") do |config|
+ # Configure VM RAM and CPU for VirtualBox
+ config.vm.provider :virtualbox do |virtualbox|
+ virtualbox.memory = 1024
+ # Two cores over default one for faster builds
+ virtualbox.cpus = 2
+ end
- # Start from Debian VM so resulting pkgs will build on all Deb derivatives
- config.vm.box = "debian/jessie64"
+ # Configure VM RAM and CPU for LibVirt
+ config.vm.provider :libvirt do |libvirt|
+ libvirt.memory = 1024
+ # Two cores over default one for faster builds
+ libvirt.cpus = 2
+ end
# NFS is fragile, disable it and use rsync
config.nfs.functional = false
# Sync folders /packaging/deb/ and /vagrant
config.vm.synced_folder ".", "/vagrant"
- # Install Debian package dev tools, helper programs for debian/rules,
- # systemd unit files, Java 7 for ODL Beryllium
+ # Start from Debian VM so resulting pkgs will build on all Deb derivatives
+ config.vm.box = "debian/jessie64"
+
+ # Install pkg dev tools, Python libs for build scripts, gdebi to test install
config.vm.provision "shell", inline: "apt-get install -y --force-yes \
- dpkg-dev \
- debhelper \
+ build-essential \
+ devscripts \
+ equivs \
dh-systemd \
- default-jdk \
+ python-yaml \
+ python-jinja2 \
+ gdebi
"
+
+ # Add jessie-backports
+ config.vm.provision "shell", inline: <<-SHELL
+ echo "deb http://httpredir.debian.org/debian jessie-backports main" > /etc/apt/sources.list.d/jessie-backports.list
+ apt-get update
+ SHELL
end
--- /dev/null
+#!/usr/bin/env python
+"""Build OpenDaylight's .debs using YAML build configs and Jinja2 templates."""
+
+import os
+import sys
+import argparse
+import subprocess
+from string import Template
+
+try:
+ import yaml
+except ImportError:
+ sys.stderr.write("We recommend using our included Vagrant env.\n")
+ sys.stderr.write("Else, install the Python libs it installs.\n")
+ raise
+
+import templates.build_debianfiles as build_debfiles
+
+# Common paths used in this script
+# This file is assumed to be in the root of the .deb build logic's dir structure
+project_root = os.path.dirname(os.path.abspath(__file__))
+
+# Debian templates directory
+templates_dir = os.path.join(project_root, "templates")
+
+# Specialized opendaylight directory for each build
+odl_dir_template = Template("opendaylight/opendaylight-$version_major.$version_minor."
+ "$version_patch-$pkg_version/")
+
+
+def build_deb(build):
+ """Build the .deb described by the given build description.
+
+ :param build: Description of a debian build, typically from build_vars.yaml
+ :type build: dict
+
+ """
+ odl_dir_name = odl_dir_template.substitute(build)
+ odl_dir_path = os.path.join(templates_dir, os.pardir, odl_dir_name)
+
+ # Call helper script to build the required debian files
+ build_debfiles.build_debfiles(build)
+
+ # Build debian package
+ os.chdir(odl_dir_path)
+ subprocess.call(["dpkg-buildpackage", "-us -uc -b", odl_dir_path], shell=True)
+
+ # Install opendaylight's dependencies
+ control_file_path = os.path.join(odl_dir_path, "debian/control")
+ subprocess.call(["sudo mk-build-deps -i " + control_file_path], shell=True)
+
+ os.chdir(project_root)
+
+
+# When run as a script, accept a set of builds and execute them
+if __name__ == "__main__":
+ # Load .deb build variables from a YAML config file
+ build_vars_path = os.path.join(project_root, "build_vars.yaml")
+ with open(build_vars_path) as deb_var_file:
+ build_vars = yaml.load(deb_var_file)
+
+ # Accept the version(s) of the build(s) to perform as args
+ parser = argparse.ArgumentParser()
+ existing_build_group = parser.add_argument_group("Existing build")
+ existing_build_group.add_argument(
+ "-v", "--version", action="append", metavar="major minor patch deb",
+ nargs="*", help="Deb version(s) to build"
+ )
+ new_build_group = parser.add_argument_group("New build")
+ new_build_group.add_argument("--major", help="Major (element) version to build")
+ new_build_group.add_argument("--minor", help="Minor (SR) version to build")
+ new_build_group.add_argument("--patch", help="Patch version to build")
+ new_build_group.add_argument("--deb", help="Deb version to build")
+ new_build_group.add_argument("--sysd_commit", help="Version of ODL unitfile to package")
+ new_build_group.add_argument("--codename", help="Codename for ODL version")
+ new_build_group.add_argument("--download_url", help="Tarball to repackage into .deb")
+ new_build_group.add_argument("--java_version", help="Java dependency for the ODL release")
+ new_build_group.add_argument("--changelog_date", help="Date this .deb was defined")
+ new_build_group.add_argument("--changelog_time", help="Time this .deb was defined")
+ new_build_group.add_argument("--changelog_name", help="Name of person who defined .deb")
+ new_build_group.add_argument("--changelog_email", help="Email of person who defined .deb")
+
+ # Print help if no arguments are given
+ if len(sys.argv) == 1:
+ parser.print_help()
+ sys.exit(1)
+
+ # Parse the given args
+ args = parser.parse_args()
+
+ # Build list of .deb builds to perform
+ builds = []
+ if args.version:
+ # Build a list of requested versions as dicts of version components
+ versions = []
+ version_keys = ["version_major", "version_minor", "version_patch",
+ "pkg_version"]
+ # For each version arg, match all version components to build_vars name
+ for version in args.version:
+ versions.append(dict(zip(version_keys, version)))
+
+ # Find every .deb build that matches any version argument
+ # A passed version "matches" a build when the provided version
+ # components are a subset of the version components of a build. Any
+ # version components that aren't passed are simply not checked, so
+ # they can't fail the match, effectively wild-carding them.
+ for build in build_vars["builds"]:
+ for version in versions:
+ # Converts both dicts' key:value pairs to lists of tuples and
+ # checks that each tuple in the version list is present in the
+ # build list.
+ if all(item in build.items() for item in version.items()):
+ builds.append(build)
+ else:
+ builds.append({"version_major": args.major,
+ "version_minor": args.minor,
+ "version_patch": args.patch,
+ "pkg_version": args.deb,
+ "sysd_commit": args.sysd_commit,
+ "codename": args.codename,
+ "download_url": args.download_url,
+ "java_version": args.java_version,
+ "changelog_date": args.changelog_date,
+ "changelog_time": args.changelog_time,
+ "changelog_name": args.changelog_name,
+ "changelog_email": args.changelog_email})
+
+ for build in builds:
+ build_deb(build)
--- /dev/null
+# Variables that define OpenDaylight's .deb builds
+# Consumed by Python logic to render Jinja2 templates into debian files
+# Each build should define:
+# version_major:
+# The OpenDaylight major (element) version number of the release to build
+# version_minor:
+# The OpenDaylight minor (SR) version number of the release to build
+# version_patch:
+# The OpenDaylight patch version of the release to build (unused pending CR)
+# pkg_version:
+# Deb version for the given ODL major.minor.patch
+# sysd_commit:
+# Version of ODL systemd unitfile to download and package in ODL .deb
+# codename:
+# Elemental codename for the ODL release, including SR if applicable
+# download_url:
+# URL to ODL tarball artifact to repackage into .deb
+# java_version:
+# Java dependency for the ODL release
+# changelog_date:
+# Date this .deb was defined
+# changelog_time:
+# Time this .deb was defined
+# changelog_name:
+# Name of person who defined .deb
+# changelog_email:
+# Email of person who defined .deb
+
+---
+builds:
+ - version_major: "4"
+ version_minor: "2"
+ version_patch: "0"
+ pkg_version: "1"
+ sysd_commit: 07f7c83b0ef46ad3809e5be03e09a77fe554eeae
+ codename: Beryllium-SR2
+ download_url: "https://nexus.opendaylight.org/content/repositories/opendaylight.release/org/opendaylight/integration/distribution-karaf/0.4.2-Beryllium-SR2/distribution-karaf-0.4.2-Beryllium-SR2.tar.gz"
+ changelog_date: "Wed, 11 May 2016"
+ java_version: "7"
+ changelog_time: "17:48:42 +0530"
+ changelog_name: "Akshita Jha"
+ changelog_email: "zenith158@gmail.com"
+ - version_major: "4"
+ version_minor: "3"
+ version_patch: "0"
+ pkg_version: "1"
+ sysd_commit: 07f7c83b0ef46ad3809e5be03e09a77fe554eeae
+ codename: Beryllium-SR3
+ download_url: "https://nexus.opendaylight.org/content/repositories/autorelease-1367/org/opendaylight/integration/distribution-karaf/0.4.3-Beryllium-SR3/distribution-karaf-0.4.3-Beryllium-SR3.tar.gz"
+ java_version: "7"
+ changelog_date: "Tue, 26 Jul 2016"
+ changelog_time: "17:48:42 +0530"
+ changelog_name: "Daniel Farrell"
+ changelog_email: "dfarrell@redhat.com"
+ - version_major: "5"
+ version_minor: "0"
+ version_patch: "0"
+ pkg_version: "1"
+ codename: SNAPSHOT
+ download_url: "https://nexus.opendaylight.org/content/repositories/opendaylight.snapshot/org/opendaylight/integration/distribution-karaf/0.5.0-SNAPSHOT/distribution-karaf-0.5.0-20160809.145534-4240.tar.gz"
+ sysd_commit: 07f7c83b0ef46ad3809e5be03e09a77fe554eeae
+ java_version: "8"
+ changelog_date: "Tue, 09 Aug 2016"
+ changelog_time: "17:48:42 +0530"
+ changelog_name: "Daniel Farrell"
+ changelog_email: "dfarrell@redhat.com"
+++ /dev/null
-
-To start OpenDaylight via systemd, use:
- $sudo systemctl start opendaylight
-
-To uninstall OpenDaylight, use:
- $sudo apt-get remove opendaylight
-
-To uninstall OpenDaylight, while deleting user data and configuration, use:
- $sudo apt-get purge opendaylight
+++ /dev/null
-opendaylight (0.4.2-1) UNRELEASED; urgency=medium
-
- * Release: OpenDaylight 0.4.2-Beryllium-SR2
-
- -- Akshita Jha <zenith158@gmail.com> Tue, 31 May 2016 17:48:42 +0530
+++ /dev/null
-Source: opendaylight
-Build-Depends: debhelper (>= 9), wget, ca-certificates, dh-systemd
-Maintainer: Akshita Jha <zenith158@gmail.com>
-
-Package: opendaylight
-Depends: ${misc:Depends}, adduser, default-jre-headless | java7-runtime-headless, java7-runtime-headless
-Architecture: any
-Description: OpenDaylight SDN controller
-
+++ /dev/null
-[Unit]
-Description=OpenDaylight SDN Controller
-Documentation=https://wiki.opendaylight.org/view/Main_Page http://www.opendaylight.org/
-After=network.service
-
-[Service]
-Type=forking
-ExecStart=/opt/opendaylight/bin/start
-User=odl
-Group=odl
-SuccessExitStatus=143
-
-[Install]
-WantedBy=multi-user.target
--- /dev/null
+#!/usr/bin/env python
+"""Build debian files from YAML debian config and Jinja2 debian templates."""
+
+import os
+import sys
+import shutil
+import urllib
+from string import Template
+
+try:
+ import yaml
+ from jinja2 import Environment, FileSystemLoader
+except ImportError:
+ sys.stderr.write("We recommend using our included Vagrant env.\n")
+ sys.stderr.write("Else, install the Python libs it installs.\n")
+ raise
+
+debian_files_dynamic = ["changelog", "rules", "control"]
+debian_files_static = ["compat", "karaf", "opendaylight.install",
+ "opendaylight.postrm", "opendaylight.postinst",
+ "opendaylight.upstart"]
+
+# Path to the directory that contains this file is assumed to be the debian templates dir
+templates_dir = os.path.dirname(os.path.abspath(__file__))
+
+# Create the Jinja2 Environment
+env = Environment(loader=FileSystemLoader(templates_dir))
+
+# Python string Template, specialized into an Opendaylight directory name per-build
+odl_dir_template = Template("opendaylight/opendaylight-$version_major.$version_minor."
+ "$version_patch-$pkg_version/")
+odl_deb_template = Template("opendaylight/opendaylight_$version_major.$version_minor."
+ "$version_patch-${pkg_version}_all.deb")
+odl_changes_template = Template("opendaylight/opendaylight_$version_major.$version_minor."
+ "$version_patch-${pkg_version}_all.changes")
+
+
+def build_debfiles(build):
+ """Builds Debian files from templates for the given build description.
+
+ Creates a debian dir for the build, copies static build files into it,
+ specializes templates with build-spicific vars to create dynamic files,
+ downloads build-specific systemd unitfile based on commit hash.
+
+ :param build: Description of a debian build, typically from build_vars.yaml
+ :type build: dict
+
+ """
+ odl_dir_name = odl_dir_template.substitute(build)
+ odl_dir_path = os.path.join(templates_dir, os.pardir, odl_dir_name)
+
+ # Clean up opendaylight dir structure if it exists
+ if os.path.isdir(odl_dir_path):
+ shutil.rmtree(odl_dir_path)
+
+ # Delete old .deb file if it exists
+ odl_deb_name = odl_deb_template.substitute(build)
+ odl_deb_path = os.path.join(templates_dir, os.pardir, odl_deb_name)
+ if os.path.isfile(odl_deb_path):
+ os.remove(odl_deb_path)
+
+ # Delete old .changes file if it exists
+ odl_changes_name = odl_changes_template.substitute(build)
+ odl_changes_path = os.path.join(templates_dir, os.pardir, odl_changes_name)
+ if os.path.isfile(odl_changes_path):
+ os.remove(odl_changes_path)
+
+ # Create debian directory
+ debian_dir_path = os.path.join(odl_dir_path, "debian")
+ os.makedirs(debian_dir_path)
+
+ # Copy files common to all .debs to the specific debian dir
+ for file_name in debian_files_static:
+ file_path = os.path.join(templates_dir, file_name)
+ shutil.copy(file_path, debian_dir_path)
+
+ # Copy templated files to debian build dir, specialize templates
+ for file_name in debian_files_dynamic:
+ # Load OpenDaylight debian files Jinja2 template
+ template = env.get_template(file_name + "_template")
+ file_path = os.path.join(debian_dir_path, file_name)
+ with open(file_path, "w") as debian_file:
+ debian_file.write(template.render(build))
+
+ # Use specific systemd unitfile for each build
+ unitfile = "opendaylight.service"
+ unitfile_url_template = Template("https://git.opendaylight.org/gerrit/"
+ "gitweb?p=integration/packaging.git;a="
+ "blob_plain;f=rpm/unitfiles/opendaylight."
+ "service;hb=$sysd_commit")
+ unitfile_url = unitfile_url_template.substitute(build)
+
+ # Build the full path for unitfile
+ unitfile_path = os.path.join(debian_dir_path, unitfile)
+
+ # Download ODL's systemd unitfile
+ if not os.path.isfile(unitfile_path):
+ urllib.urlretrieve(unitfile_url, unitfile_path)
+
+
+# If run as a script, build .spec files for all builds
+if __name__ == "__main__":
+ # Load debian build variables from a YAML config file
+ with open(os.path.join(templates_dir, os.pardir, "build_vars.yaml")) as var_fd:
+ build_vars = yaml.load(var_fd)
+
+ for build in build_vars["builds"]:
+ build_debfiles(build)
--- /dev/null
+opendaylight ({{ version_major }}.{{ version_minor }}.{{ version_patch }}-{{ pkg_version }}) stable; urgency=medium
+
+ * Release: OpenDaylight {{ version_major }}.{{ version_minor }}.{{ version_patch }}-{{ codename }}
+
+ -- {{ changelog_name }} <{{ changelog_email }}> {{ changelog_date }} {{ changelog_time }}
+
--- /dev/null
+Source: opendaylight
+Build-Depends: debhelper (>= 9), wget, ca-certificates, dh-systemd
+Maintainer: {{ changelog_name }} <{{ changelog_email }}>
+
+Package: opendaylight
+Depends: ${misc:Depends}, adduser, openjdk-{{ java_version }}-jre-headless
+Architecture: all
+Description: OpenDaylight SDN controller
#!/usr/bin/make -f
-VERSION_MAJOR = 4
-VERSION_MINOR = 2
-CODENAME = "Beryllium-SR2"
+VERSION_MAJOR = {{ version_major }}
+VERSION_MINOR = {{ version_minor }}
+VERSION_PATCH = {{ version_patch }}
+PKG_VERSION = {{ pkg_version }}
+CODENAME = "{{ codename }}"
VERSION = 0.$(VERSION_MAJOR).$(VERSION_MINOR)-$(CODENAME)
-PACKAGEVERSION = $(VERSION)-$(DISTRIBUTION)0
-TARBALL = distribution-karaf-$(VERSION).tar.gz
-URL = "https://nexus.opendaylight.org/content/groups/public/org/opendaylight/integration/distribution-karaf/$(VERSION)/$(TARBALL)"
+PACKAGEVERSION = $(VERSION_MAJOR).$(VERSION_MINOR).$(VERSION_PATCH)-$(PKG_VERSION)
+TARBALL = $(notdir {{ download_url }})
+URL = "{{ download_url }}"
%:
dh $@ --with systemd
Code Review / integration / packaging.git / commitdiff