Your Neos Project on a proServer – in Five Minutes with Ansible - 2019 - Blog

tl;dr: In order to easily configure your proServer with Ansible, simply clone the Template Repository, copy the corresponding file from host_vars_examples to host_vars and customize this as well as the inventory file according to your proServer. Next, start the provisioning process with the ansible-playbook. This playbook will write all configuration parameters into a .env file which you can then access within your Neos project.

Hosting for Neos CMS Projects

The modern Neos CMS allows you to implement challenging and complex projects for the web.

Neos itself is somewhat demanding when it comes to the hosting environment. It requires a recent PHP version and needs the PHP versions for the CLI and PHP-FPM to be identical. Additionally, a fitting version of MariaDB is expected as a database backend. If your project requires the use of the whole spectrum of Neos' features, you will also need several hosting services that exceed the standard portfolio of most hosters: Redis as a nimble cacheing backend, Elasticsearch for the management and searchability of large content quantities or supervisorD in order to orchestrate background processes – just to name a few.

With the proServer, punkt.de introduced a hosting environment that is optimally tuned for Neos. Furthermore, it will continually be adjusted to match all future requirements.

"Infrastructure as Code" with Ansible

Ansible is an automation tool for the orchestration and configuration of servers. From a control node, Ansible connects to the target servers via SSH and executes all commands that are necessary for the configuration of the server. This process requires no additional software on the target server. The Ansible configuration itself consist of easily readable YAML-files that define the tasks that need to be done.

The benefits of configuring a server using Ansible code compared to doing in manually are obvious. All steps of the configuration are comprehensible and can be executed anytime to identically configure new server instances. The Ansible code, just like your project code, is versionable. This way, the infrastructure always matches the project.

What is the proServer

With the proServer, we've introduced a hosting solution that provides everything a complex web project might need. Conceptualized by our developers and continually put to the test in our own projects, the proServer is designed to support you in your daily business. It comes with a standard configuration that can be customized to meet all kinds of requirements. At the same time, the installed software is automatically updated on a regular basis.

Technically, the proServer is based on FreeBSD and Jails, a container solution. Each jail can offer the environment for a complete project or just for individual services. With our self-service platform, proServers can be instantiated at the push of a button.

Our Ansible Roles for the proServer by punkt.de

In the name of reusability, it makes sense to structure and unitize the Ansible code in a way that allows you to use it for several projects. That's exactly what we did: We published our Ansible Modules as an open source project on Github. This way, we want to enable our customers to release their projects with as little effort as possible.

Each module contains a role or a server service which it configures. Aside from standard services, such as NGINX, php-fpm or databases, we provide several other configurations that have served us well in our daily business, such as displaying configured domains upon logging in or an oAuth2 proxy for the protection of staging systems.

Configuring a proServer with Ansible

The next four steps are necessary to configure a proServer with Ansible:

1. Installing Ansible

For starters, Ansible must be installed on your local system. The Ansible installation guide documents several options for unix-like systems. On Windows machines, Ansible can run inside the Windows 10 subsystem for Linux.

2. Cloning the Project and Submodules from GitHub

The GitHub project proServer-Ansible-Template summarizes the generic roles and configurations for the project. They can be cloned using the following command:

git clone --recurse-submodules https://github.com/punktDe/proserver-ansible-template.git

3. Ansible Configuration for Your Project

The next step is configuring which proServer should be configured using which roles and services. In order to do this, two files need to be adjusted.

host_vars

We need a file that describes all specific parameters for each individual host in our inventory inside the host_vars directory.

An example of such a file can be found under host_vars_examples. For a Neos instance on a production proServer, simply copy host_vars_examples/neos/production.yaml to host_vars/production.yaml.

Inside the new file, replace all instances of 0000 by your proServer number. Then, you can customize the different services according to your projects' requirements.

inventory.ini

The Ansible inventory, managed with the inventory.ini file, contains a list of hosts (proServers) and the groups these hosts belong to. Ansible will use these groups to assign the necessary roles.

Here, one of the host definitions needs to be un-commented and assigned your vpro-number. Then, the server needs to be assigned the right groups. For a productive Neos instance, for example, you would use the line production underneath the key Neos.

host_vars Templates

We have prepared exemplary templates for the host_vars files in two variations: for Neos and for TYPO3. Templates for further services are coming soon.
The production.yaml contains the parameters for a stack consisting of NGINX with a letsencrypt SSL certificate, PHP and either mariaDB or mySQL as a database backend. Additionally, the configuration for sending e-mails and a cluster name for the optional use of Elasticsearch are already specified

The staging.yaml brings two further useful services for staging environments.
The oAuth2 proxy protects the system from unauthorized access. We've pre-configured gitlab for the authentication in the backend, so you can conveniently gain access with your gitlab login. If necessary, you're free to use a different backend – here you can find some examples.

Mailhog is the other service that is only pre-configured for staging environments. This allows you to catch all outgoing mails and display them in a web interface. An API allows access to the inbox – ideal for automatic frontend testing with codeception.

4. Ansible starten

Ansible is now ready for its first run and can be started. The following command, for instance, provisions all servers in the production environment.

ansible-playbook --ssh-extra-args=-oProxyJump=jumping@ssh-jumphost.karlsruhe.punkt.de --limit=production playbook.yaml

This concludes the server configuration for a Neos instance. The Neos project code is expected under the path /var/www/neos/current – matching the path of a project deployed using Deployer. An example configuration for a deployment of Neos with Deployer can also be found within our template project.

Bonus: Automatically Configuring Neos with .env

Not only does the Neos Ansible role take care of configuring the NGINX according to Neos' requirements. It also writes all important configuration variables like the login credentials to the database or Elasticsearch into a -env file under /usr/local/etc/neos.env. Neos can read these parameters using the .env connector.

This has several benefits. For one, checking passwords into the git repository is a thing of the past. Differentiating between staging and production inside the configuration is also obsolete. Whenever services like a database server or Elasticsearch are distributed to other hosts, the configuration is automatically adjusted without changing the project code.

The following command installs and configures the dotenv-connector via composer:

composer require helhum/dotenv-connector
composer config extra.helhum/dotenv-connector.env-file /usr/local/etc/neos.env

The configuration on the right hand side then makes sure the parameters are read from the environment.

Our template project already includes the necessary roles and examples for the automatic configuration of Neos and TYPO3 environments. Roles for further services are coming soon.

Neos reads the environment-specific configuration files directly from the .env file.

Neos:
  Flow:
    persistence:
      backendOptions:
        driver: "%env:DB_DRIVER%"
        dbname: "%env:DB_NAME%"
        user: "%env:DB_USER%"
        password: "%env:DB_PASS%"
        host: "%env:DB_HOST%"
        charset: "%env:DB_CHARSET%"

  SwiftMailer:
    transport:
      type: Swift_SmtpTransport
      options:
        host: "%env:APP_SMTP_HOST%"
        port: "%env:APP_SMTP_PORT%"

Configuration/Settings.yaml for Neos

If you have questions or need enhancements, let us know. We're sincerely looking forward to all forms of feedback – as comments here on the blog, as pull requests on GitHub or directly inside the Neos Slack channel #proserver.

Your Neos Project on a proServer – in Five Minutes with Ansible