poc: Introduce Proof of Concepts (Package building)

This patch sets up foundations for Proof of Concepts that
simply materialize documentation into Ansible instructions
executed in virtualized Vagrant environment.

This Proof of Concept allows to easily build:
1. *.deb packages on Ubuntu 16.04; AND
2. *.rpm packages on CentOS 7.4.
It also sets up DEB and RPM repository over HTTP that can
be used to pull these openvswitch packages with apt-get
or yum from another host.

This particular Proof of Concept is intended to address
following use-cases:
1. for new OVS users to see how debian and rpm packages are
   built;
2. for developers to easily check for packaging build
   regressions;
3. for developers to easily share their sandbox builds
   into QE setups (opposed to manually copying binaries);
4. for developers to add other Proof of Concepts
   that possibly may require full end-to-end integration
   with other thirdparty projects (e.g. DPI, libvirt, IPsec)
   and need Open vSwitch packages.

Tested-by: Greg Rose <[email protected]>
Reviewed-by: Greg Rose <[email protected]>
Signed-off-by: Ansis Atteka <[email protected]>

.gitignore

@@ -16,6 +16,7 @@
 *.lib
 *.pdb
 *.pyc
+*.retry
 *.so
 *.suo
 **/*.sym
@@ -29,6 +30,7 @@
 .dirstamp
 .libs
 .tmp_versions
+.vagrant
 .gitattributes
 /Makefile
 /Makefile.in

Documentation/topics/testing.rst

@@ -389,3 +389,59 @@ validate the suitability of different vSwitch implementations in a telco
 deployment environment. More information can be found on the `OPNFV wiki`_.
 
 .. _OPNFV wiki: https://wiki.opnfv.org/display/vsperf/VSperf+Home
+
+Proof of Concepts
+~~~~~~~~~~~~~~~~~
+
+Proof of Concepts are documentation materialized into Ansible recipes
+executed in VirtualBox or Libvirt environments orchastrated by Vagrant.
+Proof of Concepts allow developers to create small virtualized setups that
+demonstrate how certain Open vSwitch features are intended to work avoiding
+user introduced errors by overlooking instructions.  Proof of Concepts
+are also helpful when integrating with thirdparty software, because standard
+unit tests with make check are limited.
+
+Vagrant by default uses VirtualBox provider.  However, if Libvirt is your
+choice of virtualization technology, then you can use it by installing Libvirt
+plugin:
+
+    $ vagrant plugin install vagrant-libvirt
+
+And then appending  --provider=libvirt flag to vagrant commands.
+
+The host where Vagrant runs does not need to have any special software
+installed besides vagrant, virtualbox (or libvirt and libvirt-dev) and
+ansible.
+
+The following Proof of Concepts are supported:
+
+Builders
+++++++++
+
+This particular Proof of Concept demonsrtates integration with Debian and RPM
+packaging tools:
+
+    $ cd ./poc/builders
+
+    $ vagrant up
+
+Once that command finished you can get packages from /var/www/html
+directory.  Since those hosts are also configured as repositories then
+you can add them to /etc/apt/sources.list.d or /etc/yum.repos.d
+configuration files on another host to retrieve packages with yum or
+apt-get.
+
+hen you have made changes to OVS source code and want to rebuild packages run:
+    $ git commit -a
+    $ vagrant rsync && vagrant provision
+
+Whenever packages are rebuilt the Open vSwitch release number increases
+by one and you can simply upgrade Open vSwitch by running yum or apt-get
+update commands.
+
+Once you are done with experimenting you can tear down setup with:
+
+    $ vagrant destroy
+
+Sometimes deployment of Proof of Concept may fail, if, for example, VMs
+don't have network reachability to the Internet.

Makefile.am

@@ -90,6 +90,9 @@ EXTRA_DIST = \
 	build-aux/sodepends.py \
 	build-aux/soexpand.py \
 	build-aux/xml2nroff \
+	poc/builders/Vagrantfile \
+	poc/playbook-centos-builder.yml \
+	poc/playbook-ubuntu-builder.yml \
 	$(MAN_FRAGMENTS) \
 	$(MAN_ROOTS) \
 	Vagrantfile \

poc/builders/Vagrantfile

@@ -0,0 +1,33 @@
+# -*- mode: ruby -*-
+# vi: set ft=ruby :
+
+VAGRANTFILE_API_VERSION = "2"
+
+Vagrant.configure(VAGRANTFILE_API_VERSION) do |config|
+  config.ssh.insert_key = false
+
+  # Centos-7.4 builder host
+  config.vm.define "centosbuilder" do |builder|
+    builder.vm.hostname = "centosbuilder.dev"
+    builder.vm.box = "centos/7"
+    builder.vm.synced_folder "../../", "/git/ovs", type: "rsync",
+                             rsync__args: ["--archive", "--delete", "-z"]
+    builder.vm.provision "builder", type: "ansible" do |ansible|
+      ansible.playbook = "../playbook-centos-builder.yml"
+      ansible.sudo = true
+    end
+  end
+
+  # Ubuntu-16.04 builder host
+  config.vm.define "ubuntubuilder" do |builder|
+    builder.vm.hostname = "ubuntubuilder.dev"
+    builder.vm.box = "generic/ubuntu1604"
+    builder.vm.synced_folder "../../", "/git/ovs", type: "rsync",
+                             rsync__args: ["--archive", "--delete", "-z"]
+    builder.vm.provision "builder", type: "ansible" do |ansible|
+      ansible.playbook = "../playbook-ubuntu-builder.yml"
+      ansible.sudo = true
+    end
+  end
+
+end

poc/playbook-centos-builder.yml

@@ -0,0 +1,108 @@
+---
+- hosts: all
+  become: true
+  name: builder
+  tasks:
+
+  - name: Create Ansible Local Facts Directory
+    file: path=/etc/ansible/facts.d state=directory
+
+  - name: Initiate Build Numbering
+    copy:
+        content: '{ "release":"1" }'
+        dest: "/etc/ansible/facts.d/builder.fact"
+        force: no
+
+  - name: Set source directory for building
+    set_fact:
+        SOURCE: "/root/rpmbuild/SOURCES"
+
+  - name: Reload Ansible Local Facts
+    setup: filter=ansible_local
+
+  - name: Install "yum-utils", "rpmdevtools", "createrepo", "httpd", "git"
+    yum: update_cache=yes name={{item}} state=present
+    with_items:
+      - yum-utils
+      - rpmdevtools
+      - createrepo
+      - httpd
+      - git
+
+  - name: Remove untracked files from Open vSwitch GIT repository
+    command: chdir=/git/ovs/ git clean -xdf
+
+  - name: Reset Open vSwitch GIT repository to last comitted state
+    command: chdir=/git/ovs/ git reset --hard
+
+  - name: Generate spec files for easy build dependency retrieval
+    shell: sed -e 's/@VERSION@/0.0.1/' {{item}}.in > /tmp/{{item}}
+    args:
+        chdir: /git/ovs/rhel
+    with_items:
+      - openvswitch.spec
+      - openvswitch-kmod-rhel6.spec
+
+  - name: Install build dependencies specified from spec files
+    shell: echo "y" | yum-builddep /tmp/{{item}}
+    with_items:
+      - openvswitch.spec
+      - openvswitch-kmod-rhel6.spec
+
+  - name: Create rpm dev tree
+    command: rpmdev-setuptree
+
+  - name: Run "./boot.sh"
+    command: chdir=/git/ovs/ ./boot.sh
+
+  - name: Run "./configure"
+    command: chdir=/git/ovs/ ./configure
+
+  - name: Run "make dist"
+    command: chdir=/git/ovs/ make dist
+
+  - name: Parse out Open vSwitch version from "configure.ac"
+    command: chdir=/git/ovs autoconf -t AC_INIT:'$2'
+    register: version
+
+  - name: Copy source tarball to rpm dev tree
+    command: cp /git/ovs/openvswitch-{{version.stdout}}.tar.gz {{SOURCE}}
+
+  - name: Unarchive openvswitch source tarball
+    unarchive:
+       src: "{{SOURCE}}/openvswitch-{{version.stdout}}.tar.gz"
+       dest: "{{SOURCE}}"
+       remote_src: yes
+
+  - name: Update release number in spec files
+    lineinfile:
+      path: "{{SOURCE}}/openvswitch-{{version.stdout}}/rhel/{{item}}"
+      regexp: '^Release:'
+      line: "Release: {{ ansible_local.builder.release }}"
+    with_items:
+      - openvswitch.spec
+      - openvswitch-kmod-rhel6.spec
+
+  - name: Build Open vSwitch user space rpms
+    command: rpmbuild -bb --without check rhel/openvswitch.spec
+    args:
+        chdir: "{{SOURCE}}/openvswitch-{{version.stdout}}"
+
+  - name: Build Open vSwitch kmod rpms (only for currently loaded kernel)
+    command: rpmbuild -bb  --without check rhel/openvswitch-kmod-rhel6.spec
+    args:
+        chdir: "{{SOURCE}}/openvswitch-{{version.stdout}}"
+
+  - name: Copy RPM packages to /var/www/html
+    command: cp -r /root/rpmbuild/RPMS/ /var/www/html
+
+  - name: Create RPM Package index file for repository
+    command: chdir=/var/www/html createrepo /var/www/html
+
+  - name: Make sure Apache is running
+    systemd: state=started name=httpd
+
+  - name: Bump up Build Number
+    copy:
+        content: '{ "release":"{{ansible_local.builder.release|int+1}}" }'
+        dest: "/etc/ansible/facts.d/builder.fact"

poc/playbook-ubuntu-builder.yml

@@ -0,0 +1,70 @@
+---
+- hosts: all
+  become: true
+  name: builder
+  gather_facts: no
+  pre_tasks:
+    - name: 'install python2'
+      raw: sudo apt-get -y install python-simplejson
+  tasks:
+
+  - name: Create Ansible Local Facts Directory
+    file: path=/etc/ansible/facts.d state=directory
+
+  - name: Initiate Build Numbering
+    copy:
+        content: '{ "release":"1" }'
+        dest: "/etc/ansible/facts.d/builder.fact"
+        force: no
+
+  - name: Reload Ansible Local Facts
+    setup: filter=ansible_local
+
+  - name: Install "devscripts", "equivs", "apache2", "autoconf"
+    apt: update_cache=yes name={{item}} state=present
+    with_items:
+      - devscripts
+      - equivs
+      - apache2
+      - autoconf
+
+  - name: Remove untracked files from Open vSwitch GIT repository
+    command: chdir=/git/ovs/ git clean -xdf
+
+  - name: Reset Open vSwitch GIT repository to last comitted state
+    command: chdir=/git/ovs/ git reset --hard
+
+  - name: Parse out Open vSwitch version from "configure.ac"
+    command: chdir=/git/ovs autoconf -t AC_INIT:'$2'
+    register: version
+
+  - name: Concatenate full version
+    set_fact:
+        full_version: "{{version.stdout}}-{{ansible_local.builder.release}}"
+
+  - name: Update Open vSwitch version to {{full_version}}
+    command: chdir=/git/ovs/ dch -b -v {{full_version}} Vagrant Build
+
+  - name: Build debian package with Open vSwitch build dependencies
+    command: chdir=/git/ovs/ mk-build-deps -B debian/control
+
+  - name: Install Open vSwitch {{full_version}} build dependencies
+    apt: deb=/git/ovs/openvswitch-build-deps-depends_{{full_version}}_all.deb
+
+  - name: Build Open vSwitch {{full_version}} debian packages
+    shell: DEB_BUILD_OPTIONS='nocheck' fakeroot debian/rules binary
+    args:
+       chdir: /git/ovs/
+
+  - name: Move debian packages to /var/www/html
+    shell: mv /git/*.deb /var/www/html/
+
+  - name: Create Debian Package index file for repository
+    shell: dpkg-scanpackages . | gzip -9c > Packages.gz
+    args:
+       chdir: /var/www/html
+
+  - name: Bump up Build Number
+    copy:
+        content: '{ "release":"{{ansible_local.builder.release|int+1}}" }'
+        dest: "/etc/ansible/facts.d/builder.fact"