The initial situation

As an employee of punkt.de, I work on many projects and use a wide variety of technologies to fulfill the wishes of our customers. One particular recent task was the update of a Neos/Flow application. Flow Neos is used in the customer project, among other things, for central data storage and ensures that the data is only displayed where it is allowed to be displayed (Internet or intranet). As this happens unobserved and reliably in the background, the updates were treated rather neglected, and so the project fell behind several major releases.

The target is 5.x

As we had to catch up with two major versions, the update was to be carried out in two steps. First, we want to update the application from Neos/Flow 4.3 to Neos/Flow 5.0 or 5.3 to reduce complexity.

A composer update rarely comes alone

The first step starts with customizing the composer.json. Quickly set the version to 5.0.0 and run a composer update in the terminal in the project and the first error messages appear. So take a look and see what composer is complaining about. In most cases there should be problems with the version used in the project. Especially in a project that has not been updated for so long, you will inevitably stumble across pinned outdated versions of your dependencies. Even if you rarely make such updates, composer supports you with information on possible dependency adjustments. This way, you can also make good progress with these larger updates.
For example, it turned out that an old version of our package for the error monitoring solution Sentry was still being used in our project. Version 1.2.0 was used in the project. A look at the releases page of the repository on Github showed that this package already had two newer major versions. After a look at the composer.json file of the package, you can quickly see that you can use the latest version. The following work is straightforward and you can quickly find the corresponding packages for automatically executable tests such as behat or PHPUnit and adapt them where necessary. It makes sense to remove the test packages first in order to reduce the complexity and concentrate on the essentials. For the tests, I then added the dependencies at a later stage using composer require --dev package-name in the latest version.

./flow makes (developer) life easier

As soon as composer update had updated Neos/Flow as well as all other dependencies without errors, you can call ./flow in the project directory. The following error messages are explained on the update page for Neos. At this point, it is important to know how the Neos CMS and the Neos/Flow framework relate to each other.

Neos/Flow is an MVC framework that can be used to develop any web application. It offers all the conveniences of common MVC frameworks such as dependency injection or the abstraction of the database using Doctrine. Furthermore, Neos/Flow is the substructure for the CMS Neos, similar to how ExtBase functions as a substructure for TYPO3. As the version of Neos is always one lower than the Neos/Flow version used, I found the information that was important for my update from Neos/Flow 4.3 to 5.0 in the article on updating Neos 3.3 to 4.0. Under the heading Breaking Changes, I read that the Symfony Yaml Component used in this version was stricter when parsing the YAML configuration files. This was also reflected in the error messages that I saw when calling the ./flow command in the terminal. After adjusting the configuration according to the instructions in the error messages, the ./flow command could be executed and so I was able - following the instructions on the update page - to have the database adjustments and code migrations carried out via the terminal.

Finally standard-compliant - PSR-3 LoggerInterface

Since the goal was generally Neos/Flow 5.3, I didn't hesitate for long and uploaded the version via composer. Here I also added the development dependencies for the various tests in the latest versions back to the project in order to have a stable intermediate status of the project with Neos/Flow 5.3.

A major change introduced with Neos/Flow 5.0 is the use of the PHP standard PSR-3. This standard defines a new logger interface for Neos/Flow. The use of this standard now requires a more extensive code adaptation in the project. In Neos/Flow versions <= 4.3, Neos/Flow's own logger interface for logging defined the following two functions:

public function log( $message, $severity = LOG_INFO, $additionalData = null, $packageKey = null, $className = null, $methodName = null );

public function logException(\Exception $exception, array $additionalData = []);

The PSR-3 definition shows that thelog function is now supplemented by other functions such as info, warning, debug and that these are to be used preferentially. Furthermore, thelogException function was removed from the general Neos/Flow-LoggerInterface and replaced by thelogThrowable function definition in the ThrowableStorageInterface. Based on this information, I adapted the code and replaced calls such as

$this->logger->log('my message', LOG_INFO);

through

$this->logger->info('my message', LogEnvironment::fromMethodName(__METHOD__));

The LogEnvironment::fromMethodName(__METHOD__) function returns the context information that has been automatically added by the framework in the background up to now. If the context information of the LogEnvironment is not sufficient, it can be enriched with additional information as follows, for example:

$this->logger->info( 'my message', array_merge(LogEnvironment::fromMethodName(__METHOD__), ['extraInformation' => $data]) );

A small stumbling block for me was the different behavior between the logException and logThrowable functions. Unlike logException, an exception is not logged automatically. This means that you cannot simply replace logException with logThrowable. Instead, you have to take care of logging the exception yourself:

$this->logger->error( 'Error occurred, exception follows:'. $this->throwableStorage->logThrowable($exception), LogEnvironment::fromMethodName(__METHOD__) );

In addition to these changes, the use statements and the type hints also had to be adapted:

use Psr\Log\LoggerInterface; ... /** * @var LoggerInterface * @Flow\Inject */ protected $logger;

If necessary, the configuration must be adapted if your own loggers are used in the project. The best documentation for me at this point was the Neos/Flow code itself, as this is a very special case and therefore not really documented in such detail.

The final step was to stabilize the application with the help of the various tests, which in my case cover the functionality of the application very well. Big praise to my colleagues for this. This made my work and the search for possible errors much easier.

After getting green for unit, functional and behavior acceptance tests, I could be pretty sure that the application was working as before and that I had successfully completed the update to Neos/Flow 5.3.

The final is Neos/Flow 6.x

Once the 5.3 intermediate step has been completed, the application can be updated to the latest version (Neos/Flow 6.x).

Deja Vue, or: Composer update and ./flow starts all over again

I upgraded the version of Neos/Flow to 6.0, checked the error messages from composer update, made sure that all dependencies were fulfilled and that the application could be installed via composer without errors. The errors noted by ./flow could also be quickly identified and fixed thanks to the update page.
After adjusting the configuration of the loggers in the project, I had already come quite far.

Even more standard conformity - PSR-7 HTTP Message Interface

With Neos/Flow 6.0, the PHP standard PSR-7 found its way into the world of Neos/Flow and into the Neos CMS. This standard defines the two interfaces Psr\Http\Message\RequestInterface and Psr\Http\Message\ResponseInterface and therefore how communication between a server and a client should be represented.

It was also specified that the body of a message is represented by the Psr\Http\Message\StreamInterface. The use of streams makes it possible to address the various forms of message content, such as text, images or videos, using a standardized API.

As the Neos/Flow application in my case functions as a web API for a TYPO3 instance, I set about making the application PSR-7 compatible. In Neos/Flow, the advantages provided by PSR-7 are utilized and the PSR-7-compatible implementations from the guzzlehttp package are used to represent requests to the server. In addition, the ActionRequest and ActionResponse classes are used in Neos/Flow for use in the MVC context. These function within Neos/Flow as an additional abstraction layer to the PSR-7 implementations. Equipped with this knowledge, I set about adapting the project code.

The following examples show how e.g. a file transfer from a server to a client has been realized so far and how a PSR-7 compliant implementation can look like. This code is located in an ActionController, where direct access to the ActionRequest is possible via the $request property of the controller.

PHP-native functions

header('Content-type: ' . $contentType); header('Content-Disposition: attachment; filename="' . basename($filePath)); 
header("Cache-Control: max-age=0,private,must-revalidate"); 
header("Expires: Sat, 26 Jul 1997 05:00:00 GMT"); 
ob_clean(); 
readfile($filePath); 
flush();
exit();

PSR-7 using streams

$this->response->setContentType($contentType); 
$this->response->setComponentParameter( SetHeaderComponent::class,'Content-Disposition', 'attachment; filename="' . basename($filePath) ); 
$this->response->setComponentParameter(SetHeaderComponent::class,'Cache-Control', 'max-age=0,private,must-revalidate'); 
$this->response->setComponentParameter(SetHeaderComponent::class,'Expires', 'Sat, 26 Jul 1997 05:00:00 GMT'); 
return GuzzleHttp\Psr7\stream_for(fopen($filePath, 'r+'));

Neos/Flow >= 6.0

$this->response->setHeader('Content-type', 'application/pdf'); 
$this->response->setHeader('Content-Disposition', 'attachment; filename="' . $fileName); 
$this->response->setContent($content); 
$this->response->send();

PSR-7 using streams

$this->response->setContentType('application/pdf'); 
$this->response->setComponentParameter( SetHeaderComponent::class,'Content-Disposition', 'attachment; filename="' . $fileName);
return stream_for($content);

In addition to these adjustments, which are directly related to the PSR-7 standard, indirect changes were also made, as some functions such as sendHeaders or send were removed from the previous ActionResponse implementation. The following examples show how to deal with this:

File download from Server Neos/Flow < 6.0

$this->response->setHeader('Content-type', 'application/pdf'); 
$this->response->setHeader('Content-Disposition', 'attachment; filename="' . basename($this->absoluteFilePath)); 
$this->response->sendHeaders(); 
ob_clean(); 
readfile($this->absoluteFilePath); 
flush();

Neos/Flow => 6.0

$this->response->setContentType('application/pdf'); 
$this->response->setComponentParameter( SetHeaderComponent::class, 'Content-Disposition', 'attachment; filename="' . basename($absoluteFilePath) );
return stream_for(fopen($absoluteFilePath, 'rb'));

Abort request processing and send error message in the event of an error Neos/Flow < 6.0

$this->response->setStatus($httpStatusCode); 
$this->response->setContent( json_encode( [ 'statusCode' => $internalStatusCode, 'statusMessage' => $internalStatusMessage ] ) ); 
$this->response->send(); exit();

Neos/Flow => 6.0

$this->throwStatus( $httpStatusCode, $internalStatusMessage, json_encode( [ 'statusCode' => $internalStatusCode, 'statusMessage' => $internalStatusMessage ] ) );

Parallel to these adjustments, I refactored the code and corrected errors whenever necessary in order to stabilize this update status as well.

I proceeded in the same way as with the 5.3 milestone and made extensive use of the various tests available to me in order to discover and rectify possible errors. After all the tests ran without errors, I was able to update the application to the latest Neos/Flow version 6.1 without any effort.

But the update wasn't quite finished yet. In a final step, the application was checked and linked with the help of the PHP-Linter PHP_CodeSniffer in order to update the code style as well.

Conclusion

Updating across two major versions is a big task, but it's worth it because the new version allows you to use the PRS-7 standard, making it easier for all colleagues to work together on a large project.

As Flow is central to the project, I was able to get a good overview of the overall project without having to go into too much detail. After a few exciting days, the update is done without any major pain and can be used in the project.