Blog

In this article we’ll outline an iterative deployment process for Magento 2 using git and rsync via a deployment script, and our reasons for working this way.

Magento have a help document called “Overview of deployment” which sounds promising, but it’s advice on a one-off move from development to production; it says nothing about the development to production cycle. It’s good to see that at least they say “Put all custom code in source control” though there’s little advice as to what to keep in source control and how to update it.

A bit of background – approaches to web deployment

I’m going to go into the reasons behind why we use a build-and-sync deployment approach. If this seems obvious, then feel free to jump ahead to Deployment approach for Magento 2 or even straight to the individual commands for deployment.

0. Make changes directly on the live installation

Please don’t do this.

1. Source control

Anyone who has anything more than a trivial ecommerce site knows that it’s important to try out extensions and customisations on a separate development installation from your production installation.

This then begs the question of how to get these changes from development to production when these are likely to be on different machines. Source control to the rescue!

While we’ve all gone through a phase of putting our whole site in source control, cloning it on a live server and pointing a web-server at it, it quickly becomes apparent that any slips in source control commands or problems with composer (anyone ever had to delete the vendor folder? No? OK, you’re amazing, or possibly lying) can result in a broken live site. Ecommerce site owners don’t like down-time, especially unexpected down-time.

2. Build then sync

So the second phase is the use of a deployment process that does the build separately to the live site – source control, composer updates etc. – and only then copies over changed files. This approach is pretty mature and well-adopted now – Capistrano, a deployment tool in Ruby – is 10 years old this year (happy birthday, Capistrano).

Over time the best practice has become to do all possible build and compilation so that the only thing copied over is files as this is then only a single point of failure and rsync is pretty stable, giving a general approach of:

• Build/compile site off to one side
• Sync files to live
• Minimal post-sync e.g. sending an email, cleaning up permissions

Deployment approach for Magento 2

Sadly, Magento 2 does not allow us to do a clean build and sync, as certain build steps like deploying static files requires a working installation of Magento 2. This means deploying changes and then running extra steps like deploying static files, enabling modules etc.

Maintenance Mode

Apart from the issue that additional post-sync steps add points of possible failure, if we want to minimise downtime then we need to consider how long Magento 2 needs to be in maintenance mode for during a deployment.

Copying files tends to be fast enough to not require this unless you have a very busy server, and deploying static files can be included in this category. The one step that needs maintenance mode on is when enabling a module and running setup files. Magento 2 does not have the “hit the frontend and run the setup script” problems of Magento 1, but an enabled module should not be accessed without its setup scripts run.

Greaaaaat… so that are the deployment steps then?

We have ended up with the following process:

1. Check for unexpected changes
2. Update repo
3. Composer install
4. Sync files
5. Deploy static files
6. Enable maintenance mode
   1. Enable modules
   2. Run setup scripts
   3. Compile DI
   4. Clear cache
   5. Disable maintenance mode
7. Update permissions

We’ll go through these steps one at a time in a moment. First, for reference, here is the structure we use to deploy our sites:

/var/www/vhost/somesite/
├── deploy
│   ├── deploy.conf
│   ├── .exclude-files-deploy
│   ├── log
│   └── release-candidate
│       ├── .auth.json
│       ├── .git
│       ├── composer.json
│       ├── composer.lock
│       ├── index.php
│       ├── app
│       ├── bin
│       ├── conf
│       ├── dev
│       ├── lib
│       ├── phpserver
│       ├── pub
│       ├── setup
│       ├── update
│       ├── var
│       └── vendor
└── site
.   ├── index.php
.   ├── app
.   ├── bin
.   ├── conf
.   ├── lib
.   ├── pub
.   ├── setup
.   ├── update
.   ├── var
.   └── vendor

The main thing to spot here is the parallel structure in deploy/release-candidate and site. The webserver root is pointed at /var/www/vhost/somesite/site/pub/.

We have at times dispensed with the site folder and had its contents at the level above, but this adds the complication of having to exclude having to exclude itself when copying from the deploy folder to the destination live folder – this way is cleaner and allow for other files that are not part of the website repo to be stored here such as ssl certificates.

Deployment step by step

1. Check for unexpected changes

As we have a site under version control, it’s a good idea to check that there have not been any changes between the last deployed version and the live code. This way we can detect any malign hacks, and also if the client has changed something without letting us know

A simple check:

rsync -OvrLt --delete --checksum --exclude-from='...deploy/.exclude-files-deploy' '...deploy/release-candidate/' '.../site/' --dry-run

We can then check the output of this for “sending incremental file list” or “building file list” to see if there was a difference.

2. Update repo

Pull in the latest version of your code (or a specific release candidate commit if you work this way).

From deploy/release-candidate:

git fetch
git checkout origin/production

3. Pull down with composer

A quick reminder of the two common composer commands: update and install:

update is used to pull down the latest versions of required components based on restrictions in composer.json, and this updates the composer.lock file.

install simply takes the specific versions in composer.lock and installs them locally in vendor.

Run composer install in vendor/release-candidate to populate the vendor folder in that location rather than directly on the live site:

composer install --no-dev

4. Sync files

Rsync is the perfect tool for this task as it can use file checksums and update times to avoid copying identical files:

rsync -OvzrLt --delete --checksum --exclude-from='deploy/.exclude-files-deploy' 'deploy/release-candidate/' 'site/'`

It is useful to maintain a list of files that should not be synced. For magento2 deploy/.exclude-files-deploy will contain something like:

/app/etc/env.php
/app/etc/di.xml
/app/etc/config.php
/dev
/conf
.git
.gitignore
/var
/pub
/phpserver
/auth.json

Some of these are files that will not exist in the repo that should not be deleted from the site e.g. /app/etc/env.php. Others are folders in the repo that do not need to be deployed to a production site e.g. /dev.

5. Deploy static files

In theory one should only need to run the static deployment command of the Magento CLI, but we’ve found that this does not always update all static files that have changed, so until this is fixed it is necessary to clear the static files first. If doing this it is probably best to enable maintenance before this point:

rm -Rf site/pub/static
./bin/magento setup:static-content:deploy [locales]

where [locales] is the set of locales to deploy. It defaults to just en_US so a multi-lingual site may have a locale list of en_GB en_US fr_FR es_ES

Note that this command is run from the live site/ folder. All ./bin/magento commands from this point on are run from here.

6. Enable maintenance

This can be done simply via the magento CLI:

./bin/magento maintenance:enable

6.1 Enable modules

One of the advantages of Magento 2 is that we can choose whether to enable modules or not. One could either check in app/etc/config.php so that modules are enabled in dev and then automatically enabled by syncing this file. Alternatively, one could add an option to deployment to specify which modules are enabled. In this case, we need to run the magento CLI command to enable modules:

./bin/magento module:enable [module1] ... [moduleN]

6.2 Run setup scripts

Another easy step:

./bin/magento setup:upgrade

6.3 Compile DI

As of writing there is a current issue with compiling using the standard single-tenant compiler so you must use the multi-tenant compiler even for a single store. So the command has to be:

./bin/magento setup:di:compile-multi-tenant

As of writing there is also an issue with compilation that fails due to missing a dev-only package. Our suggested workaround is to include "sjparkinson/static-review": "~4.1" in the require section of composer.json.

6.4 Clear cache

Hurray for the CLI tool!

./bin/magento cache:flush

This clearing of all caches may be over-kill, but our experience with Magento 1 is that it is better to be safe and suffer a short performance blip.

6.5 Disable maintenance mode

Surprising nobody, it’s:

./bin/magento maintenance:disable

7. Update permissions

If we’re copying files as the correct user and with sensible permissions in the repo this may not be needed, but we tend to run this to be safe on production and enforce our permission schemes.

First, as we use nginx we set the group to nginx to allow read permissions for files. This might be the apache group if you use Apache. Note that we only try to change files that are in the wrong group!

find 'site/' ( -not -group 'nginx' ) -print -exec chgrp 'nginx' {} ;

The directories are set to 750:

find 'site/' ( -type d -not -perm '2750' ) -print -exec chmod '2750' {} ;

And files to 640:

find 'site/' ( -type f -not -perm '0640' ) -print -exec chmod '0640' {} ;

Done

And that’s it! That’s currently how we deploy Magento 2 for production. We’re sure that this will evolve, and would love to hear what your thoughts are on this process.