Too Many Open Files Error In PHP SoapClient

Whilst working on a project recently I hit upon a PHP error that I've never seen before. During a process where a soap service was calling an API the connection would fail and the program would fatal error and stop.

I had protection mechanisms in place to catch this kind of connection error, but the fatal error was caused when the program tried to throw the exception I had put in place to indicate a failed connection.

Here is the error message (with some of the detail removed). This is a Drupal site but that detail is irrelevant to the problem.

PHP message: PHP Warning:  require(/docroot/modules/custom/api_integration/src/Exception/UnableToConnectException.php): failed to open stream: Too many open files in /vendor/symfony/class-loader/ApcClassLoader.php on line 112
PHP message: PHP Fatal error:  require(): Failed opening required '/docroot/modules/custom/api_integration/src/Exception/UnableToConnectException.php' (include_path='') in /vendor/symfony/class-loader/ApcClassLoader.php on line 112

What this was telling me was that the program couldn't connect to the soap service but that it couldn't open the exception to handle this because there were 'too many open files'.

After some digging around I found people with similar problems, but nothing specifically related to the PHP SoapClient object. It turns out that this open files limit is under control from within PHP, but is part of the underlying operating system limits. Indeed, I tried out this process on a couple of different PHP versions and received the same result.

This limit (on unix systems) can be seen by using the command ulimit with the -n flag to look at the maximum number of open file descriptors. On my current system this shows that the limit is 256 files.

$ ulimit -n
256

In order to artificially hit this limit I created a little script that would open the same file over and over again, but never close the connection.

<?php

$handles = [];

for ($i = 1; $i <= 254; $i++) {
  $handles[] = fopen('/dev/null', 'w');
}

Running this script produced the following error, the final time that the script tries to open the file causes this warning. Note that I set the limit of the loop to 254. This is because the number of open files includes the PHP process and the file running the script.

PHP Warning:  fopen(/dev/null): failed to open stream: Too many open files in test.php on line 6

Looking back at the soap process I was running I could see that the soap client was connecting to the soap service and after a certain number of connections it simply couldn't connect any more and caused an error. The error regarding the inclusion of an exception file is an upstream red herring that was masking the problem.

After hours of looking through support forums talking about how to change the operating system settings (something I wasn't that happy doing) I found the solution. It turns out that the PHP SoapClient will connect to the soap service and keep the connection alive. Unfortunately, the next connection does not use this currently active connection and creates another one when the next soap request is made. During large operations this limit is quickly reached and will cause an error.

To prevent soap keeping the connections alive you need to pass in the keep_alive option when creating the SoapClient object.

$options['keep_alive'] = FALSE;
$soapClient = new \SoapClient($wsdl, $options);

This allowed me to go from a couple of hundred connections to many thousands and it allowed the soap process to complete successfully. It also meant that I had solved this problem within the code, rather than tinkering with the operating system to solve it.

Honestly, I hate using soap.

Comments

Thank you! This fixed my script-only-runs-for-so-long problem and makes perfect sense. SOAP sucks, unless you're washing your hands.

Permalink

Thank you! I had gone through an exhaustive and boring process to identify that it was actually a SOAP call that was causing the leak of filehandles (there were loads of other possibilities in my case), so it was really helpful that once I knew that, it took seconds to find this post with the solution.

Permalink

Happy to help Sam. Glad you found the solution! \o/

Name
Philip Norton
Permalink

Thanks for the time spent on this article, I have a question.

If my soap call takes a long time to complete (at least 3 minutes for response to be complete), turning the "keep_alive" to false may cause some problems.

Is there a way to close the connection manually ?

 

Permalink

Well, you can't directly close it as the client request is a blocking function. What I mean is that once the request is in motion you can't run any code to see if it's finished.

What you can do is set a timeout. From what I've read you can use the init_set() function to set the "default_socket_timeout".

ini_set("default_socket_timeout", 15);

However, that apparently doesn't effect https addresses, which makes it redundant. There are a few examples out there that extend the SoapClient class and change the __doRequest() method to use a non-blocking stream with a larger timeout, but I don't have any way of testing that these days.

Just remember that under the hood, SOAP is a http(s) request/response model. All the SoapClient does is wraps the complexity of changing arrays and objects into XML and then sends the response.

Name
Philip Norton
Permalink

Add new comment

The content of this field is kept private and will not be shown publicly.
CAPTCHA
2 + 9 =
Solve this simple math problem and enter the result. E.g. for 1+3, enter 4.
This question is for testing whether or not you are a human visitor and to prevent automated spam submissions.