Published : 2014-02-10

Poudriere

Poudriere is an excellent tool available on both FreeBSD and DragonFlyBSD that lets you forge a FreeBSD/DragonFlyBSD repository to quickly deploy a fleet of FreeBSD servers or clients (combined with Puppet you can roll out very fast).

Prerequisites

Poudriere needs a ZFS pool. While not strictly required, it is strongly recommended.

Installation

Nothing could be simpler:

cd /usr/ports/ports-mgmt/poudriere
make install clean

Configuration

First, create a directory and mount our ZFS pool onto it:

mkdir /poudriere
zpool create -m /poudriere poudriere /dev/ada1

Now let’s configure poudriere. Open the file /usr/local/etc/poudriere.conf:

ZPOOL=zpool
ZROOTFS=/poudriere
FREEBSD_HOST=ftp://ftp.freebsd.org
RESOLV_CONF=/etc/resolv.conf
BASEFS=/usr/local/poudriere
USE_PORTLINT=no
USE_TMPFS=yes
DISTFILES_CACHE=/usr/ports/distfiles
PARALLEL_JOBS=1
export HTTP_PROXY=http://myproxy:3128
export HTTPS_PROXY=http://myproxy:3128
export FTP_PROXY=http://myproxy:3128

Let’s review these options:

  • ZPOOL: the name of your ZFS pool.
  • ZROOTFS: the root of your pool.
  • FREEBSD_HOST: the host from which to fetch the ports and the distribution.
  • RESOLV_CONF: the DNS client configuration file used by poudriere.
  • BASEFS: the directory where poudriere will place its jails and ports.
  • USE_PORTLINT: set to yes if you want to use port linting.
  • USE_TMPFS: ask poudriere to use a TMPFS for compilation.
  • DISTFILES_CACHE: shared directory between ports and jails.
  • PARALLEL_JOBS: number of parallel jobs. If unset, the number of CPU cores is used.

We’ll need a dedicated ports tree. Ask poudriere to fetch it:

poudriere ports -c -p default

Now let’s create the jail matching our needs:

poudriere jail -c -j "11_1" -v "11.1-RELEASE" -a amd64

In the example above this creates the 11_1 jail, corresponding to FreeBSD 11.1 for the amd64 architecture. (I do not recommend putting . in the jail name, it has unwanted side effects.) To use the pkg package format (now the only one), run the following lines (11_1 is the name of your jail):

mkdir /usr/local/etc/poudriere.d
echo "WITH_PKGNG=yes" > /usr/local/etc/poudriere.d/11_1-make.conf

Now define a list of ports to forge. Open a file (for example in /usr/local/etc/poudriere.d/11_1-ports) and put a list of ports inside:

shells/zsh
ports-mgmt/portmaster

Then ask poudriere to show the build options so you can pick them:

poudriere options -f /usr/local/etc/poudriere.d/11_1-ports -j 11_1 -p default

Finally, kick off the build:

poudriere bulk -f /usr/local/etc/poudriere.d/11_1-ports -j 11_1 -p default

You should get something like:

fbsd91-tests# poudriere bulk -f /usr/local/etc/poudriere.d/11_1-ports -j 11_1 -p default   
====>> Creating the reference jail... done
====>> Mounting system devices for 11_1-default
====>> Mounting ports/packages/distfiles
====>> Mounting packages from: /usr/local/poudriere/data/packages/11_1-default
====>> Mounting /var/db/ports from: /usr/local/etc/poudriere.d/11_1-options
====>> Logs: /usr/local/poudriere/data/logs/bulk/11_1-default/2013-08-01_22h06m39s
/etc/resolv.conf -> /usr/local/poudriere/data/build/11_1-default/ref/etc/resolv.conf
====>> Starting jail 11_1-default
====>> Calculating ports order and dependencies
====>> Sanity checking the repository
====>> Deleting stale symlinks
====>> Deleting empty directories
====>> Cleaning the build queue
====>> Building 3 packages using 3 builders
====>> Starting/Cloning builders
====>> Hit CTRL+t at any time to see build progress and stats
====>> [01] Starting build of converters/libiconv
====>> [02] Starting build of ports-mgmt/portmaster
====>> [02] Finished build of ports-mgmt/portmaster: Success
====>> [01] Finished build of converters/libiconv: Success
====>> [01] Starting build of shells/zsh
====>> [01] Finished build of shells/zsh: Success
====>> Stopping 3 builders
====>> Preparing INDEX
====>> Generating INDEX... done
====>> Compressing INDEX-9... done
====>> Cleaning up
====>> Umounting file systems
====>> Built ports: ports-mgmt/portmaster converters/libiconv shells/zsh
====>> [11_1-default] 3 packages built, 0 failures, 0 ignored, 0 skipped
====>> Logs: /usr/local/poudriere/data/logs/bulk/11_1-default/2013-08-01_22h06m39s

You should see the tree of your packages under /usr/local/poudriere/data/packages/11_1-default (jail name and ports tree name):

fbsd91-tests# ls -al /usr/local/poudriere/data/packages/11_1-default
total 13
drwxr-xr-x  8 root  wheel    9 Aug  1 22:09 .
drwxr-xr-x  3 root  wheel    3 Aug  1 22:06 ..
drwxr-xr-x  2 root  wheel    5 Aug  1 22:09 All
-rw-r--r--  1 root  wheel  348 Aug  1 22:09 INDEX-9.bz2
drwxr-xr-x  2 root  wheel    5 Aug  1 22:09 Latest
drwxr-xr-x  2 root  wheel    3 Aug  1 22:07 converters
drwxr-xr-x  2 root  wheel    3 Aug  1 22:07 devel
drwxr-xr-x  2 root  wheel    3 Aug  1 22:06 ports-mgmt
drwxr-xr-x  2 root  wheel    3 Aug  1 22:09 shells

Updating the Tree and the Packages

You should regularly update the ports tree and re-forge the packages that changed. To do this, simply run the following commands:

poudriere ports -u -p default
poudriere bulk -f /usr/local/etc/poudriere.d/11.1-ports -j 11_1 -p default

You now have a package forge and a FreeBSD repository.

Client Configuration

Now let’s configure the client to fetch its packages from the FreeBSD repository (you’ll need an http/ftp server). First create the directory /usr/local/etc/pkg/repos/ if it doesn’t exist. Open /usr/local/etc/pkg/repos/local.conf:

myrepo: {
    url: "pkg+http://freebsdrepo.lan/pub/FreeBSD/11.1",
    mirror_type: "http",
    signature_type: "none",
    fingerprints: "/usr/share/keys/pkg",
    enabled: yes,
}

Then run the following commands to install the packages we forged with poudriere:

pkg update
pkg install zsh portmaster

That’s it! Source 1 Source 2 Source 3