Blog

• Do you have a Magento CE site that is pre version 1.9.2.0?
• Do you run this site using php-fpm with php version >= 5.5.22?
• Have you applied the SUPEE-6788 patch from October 2015?

Rather specific, I know, but if the answer to all these is yes, then your site is in danger of being taken down repeatedly by security scanners that are run either maliciously, or by yourself, or a security firm for PCI compliance.

Background

A quick bit of background – there have been a number of security patches for Magento this year, helping make the platform a lot safer for retailers. Most merchants will want these patches installed in short order, as attackers will start using the exploits pretty quickly and nobody wants to have their site compromised. The danger is that it can take a while for the ramifications of a change to become evident.

So in October/November we applied the SUPEE-6788 patch to our sites and pushed the change live. Shortly afterwards we started seeing some of these sites go down complaining about not being able to parse the XML in local.xml. Clearing the cache or restarting php-fpm seemed to bring them back up. On investigation we found that the sites were going down shortly after cross-site scripting (XSS) attempts were made to the sites.

This seemed odd, as one of the security improvements made by SUPEE-6788 was to disallow certain elements in the XML-RPC API that allowed XSS attacks. So having put the fix in place… the sites were now being taken down by these attempts.

The problem

The problem turns out to be caused by a two-step process:

1. A fragment of XML similar to the following being submitted to a site’s XML-RPC API:

 ]>&xxe;

2. A page that directly loads local.xml is called. This is often /index.php/install

What happens when these two steps occur (as they do when security scans are run) is that a php setting libxml_disable_entity_loader is being set to true in the first call, which breaks the loading of local.xml via simplexml_load_file() in the second call. The failure of this call, even though local.xml is readable, causes an error which will then reoccur on other page loads.

This error may not happen every time as the libxml_disable_entity_loader is set per php-fpm child so if you have 5 child processes running the error may only occur in 1 in 5 loads. Such intermittent problems are particularly hard to debug!

More on why this problem happens at the end of this post. But first…

The Fix

You’ll recall that one of my questions at the start was whether your Magento site was less than v1.9.2.0? This is because Magento have put in a fix for this issue in that version. The new file /app/bootstrap.php contains the following:

if (function_exists('libxml_disable_entity_loader')) {
libxml_disable_entity_loader(false);
}

The problem is that the libxml_disable_entity_loader setting is getting stuck on true from other calls, so running this early on in Magento calls means that libxml_disable_entity_loader will always be false when it matters.

So the fix for < v1.9.2.0 is to put this code into your index.php file just after the PHP version check (as early as possible). Normally we would not suggest modifying a core Magento file, but this is pretty safe as your next upgrade is likely to be to at least v1.9.2.0 so the loss of this fix will not be important then.

More on the causes of the problem

The libxml_disable_entity_loader setting is actually there for security. Ideally it would be set to true to counteract just the sort of XSS attack listed above. However, as the open but php bug 62577 states, when this setting is true, simplexml_load_file() simply does not work. This has not yet been addressed by PHP, so we have to make sure that libxml_disable_entity_loader is false.

…which would not be an issue where it not for the other php bug 64938 which is supposedly fixed in php 5.5.22 and 5.6.x. This bug is that the libxml_disable_entity_loader setting is not thread-safe. This means that in reentrant php workers in php-fpm, if the libxml_disable_entity_loader setting gets changed, it will not revert to its default value in the next call. Testing on php 5.5.30 has shown that this bug has not been fixed, sadly.

The incorrect marking of the latter bug as fixed is why Magento’s developers ended up causing this issue. After the SUPEE-6788 patch is applied, XML-RPC security checks for php-fpm versions < 5.5.22 and uses a fall-back heuristic approach so that libxml_disable_entity_loader is not set to true. For php >= 5.5.22 they assume (incorrectly but reasonably) that they can rely on using simplexml_load_file() and libxml_disable_entity_loader being set to true to do their checks for them.

To keep abreast of this issue, please visit the php bugs 62577 and 64938 to subscri