We run our hosting out of conviction on the basis of FreeBSD and benefit not only from the stability and well thought-out design, but also from the tried and tested technologies ZFS and Jails. Both are the basis of our proServer and enable lightweight, flexible virtualization as a hosting solution in our own data center.
As punkt.de also offers the development of complex websites and applications based on Neos and TYPO3 in addition to hosting, developers and operators have inevitably always worked closely together at our company; incidentally, for longer than the term DevOps has existed. One of the major challenges with regard to the underlying operating system is therefore to reconcile the different interests of the respective factions.
On the one hand, there are the mostly progressive developers with regard to the software. In times when a new JavaScript framework is released almost every week, the calls for the new, hot software are naturally loud: "Please use nginx instead of Apache on the servers and in my developer boxes" or "We need Varnish and PHP 7.1 for our customer's test system because we really want to test how NEOS behaves with it. On the production system, however, PHP 7.0 still needs to be installed and updated with minor fixes." or, of course, "On my favorite OS, I can simply install Redis via my package manager. How does that work in my FreeBSD developer box?". These are just a few examples of the developers' requirements, but they are not plucked out of the air.
On the other hand, there are the operators, who are more conservative in the best sense of the word. Their intention is to provide stable, consistent and updatable systems; no more and no less.
Naturally, this leads to conflicts of interest. So, in the best DevOps spirit, we Opse sat down with our developers, clarified requirements and restrictions and finally found a solution. Our aim was to provide a central package repository that always provides up-to-date software packages that are easy to install.
Package management under FreeBSD
FreeBSD provides third-party software such as nginx, PHP, etc. as so-called ports. These are usually up-to-date, but require compilation from the source code for installation. For large packages, however, this requires a lot of computing time and technical know-how, which a developer understandably neither wants to nor should bother with. In order to maintain the convenience of installing packages in the manner of apt-get, yum or brew, the FreeBSD counterpart pkg can be used. Similar to the package managers of other operating systems, pkg install <PAKET> downloads and installs the appropriate package from a central FreeBSD package repository.
Devs and Ops have thus taken their first step towards each other. But what about major and minor updates of the respective software? How can we operators provide the developers with the latest packages? This is where the poudriere tool comes into play, which can be used to automatically and reproducibly build installable packages. Now it should also be clear why we call our package repository Poudriere.
Building packages with poudriere
poudriere is a command line tool that creates selected packages based on a FreeBSD operating system release. For this purpose, the respective package is selected from the FreeBSD port tree and built within a container, a so-called jail. The portstree is a hierarchically categorized directory structure of package build information that is usually stored under /usr/ports/ on a FreeBSD system:
...
lang/php56
lang/php70
lang/php71
www/apache22
www/apache24
www/nginx
...
Jails are FreeBSD's own container solution, i.e. an environment encapsulated by the host system that guarantees a consistent environment when building packages.
An example:
poudriere bulk -j 110 -p 2017Q3 -z ap24-php70 www/nginx
creates the nginx package from the Q3 2017 port tree in a container with FreeBSD 11.0 release. If necessary, package dependencies can of course also be defined and package-specific build parameters specified. For example, the Apache web server must be bound to exactly one dedicated PHP version during compilation due to mod-php. The specification of so-called sets, here ap24-php70, makes this possible.
This generates multidimensional package lists that can be made available via a web server. punkt.de hosting provides these at http://packages.pluspunkthosting.de/packages/, sorted by FreeBSD release, port tree quarter and set:
...
110-2017Q3-ap22-php56
110-2017Q3-ap24-php56
110-2017Q3-ap24-php70
110-2017Q3-ap24-php71
110-HEAD-ap24-php70
110-HEAD-ap24-php71
111-2017Q3-ap22-php56
111-2017Q3-ap24-php56
111-2017Q3-ap24-php70
111-2017Q3-ap24-php71
111-HEAD-ap24-php70
111-HEAD-ap24-php71
...
The packages are now ready. From an operator's point of view, this is a great solution because it is well-founded and maintainable. But what about the developers? How does a developer benefit from the repositories?
Developers and the Poudriere
In order for developers to install the packages of a repository, the official FreeBSD repository must be disconnected by adding the line
is entered. In addition, the file /usr/local/etc/pkg/repos/punkt.de.conf must be created and filled with the following content:
punkt.de: { url: https://packages.pluspunkthosting.de/packages/110-2017Q3-ap24-php70, enabled: yes, mirror_type: NONE }
The repository 110-2017Q3-ap24-php70 is only selected as an example and can of course be replaced accordingly. A subsequent
sudo pkg update
sudo pkg upgrade
updates the package database and the packages already installed. Done.
Conclusion
As a hoster, we see it as our responsibility to provide our customers and development teams with up-to-date and secure software. Our credo "We host what we know" applies not only to the Neos and TYPO3 applications, but also to the software packages and the underlying operating system.
By the way: The punkt.de hosting packages are publicly available. Install one of our proServer Vagrant boxes from https://boxes.hosting.punkt.de / and give it a try. All our packages are updated nightly. So you benefit from stable package versions on a professional FreeBSD platform.
