httpd exited on signal 11 – solved

tl;dr

It’s probably mod_php, and try recompiling PHP without zend_multibyte. If you have WITH_MULTIBYTE=true in /var/db/ports/php5/options, change it to WITHOUT_MULTIBYTE=true and rebuild the port.

Long Story

The Apache 2.2 processes on my FreeBSD box keep dying intermittently, with messages such as:


Jul 3 08:08:37 yama kernel: pid 13526 (httpd), uid 80: exited on signal 10
Jul 3 08:25:58 yama kernel: pid 33050 (httpd), uid 80: exited on signal 11

I’ve googled around and tried various extensions.ini reordering tricks without luck. Eventually, I figured out how to get Apache to dump core. First, you need to tell Apache where to dump core by adding the CoreDumpDirectory directive to your httpd.conf. Restart apache.

Then, check to make sure that the kern.coredump and kern.sugid_coredump sysctl variables are set to 1. Because Apache calls setuid and setgid to the configured user after listening on port 80 (privileged port), FreeBSD by default will refuse to dump core on segfaults. Setting kern.sugid_coredump will make it do so.


$ sysctl kern.coredump
kern.coredump: 1
$ sysctl kern.sugid_coredump
kern.sugid_coredump: 0
$ sudo sysctl kern.sugid_coredump=1
kern.sugid_coredump: 0 -> 1

Of course, you’d also need to make sure that your ulimit doesn’t prevent it from dumping core:


$ ulimit -c
unlimited

Great! Now wait for Apache to segfault (or, if you have a reproducible test case, hit it!)

You should now find httpd.core in the directory that you configured in httpd.conf (in my case, it is /tmp.)

Do the usual gdb stuff:


$ gdb /tmp/httpd.core /usr/local/bin/httpd

In my case, I found zend_multibyte_read_script near the top of the stack, so it’s pretty clear that PHP is to blame.
Did I mention that I abhor PHP? Someday I will get rid of it entirely!

This led me to more googling and found that I may have accidentally enabled Zend Multibyte support (well, its purpose is not very well documented and I just assumed that since I do lots of i18n work, it’ll come in handy. Okay so it’s my fault after all but it still shouldn’t crash!)

Changing WITH_MULTIBYTE=true to WITHOUT_MULTIBYTE=true in /var/db/ports/php5/options and rebuilding the port made the problem go away.

Hopefully, the above will help someone in a similar situation.



5 Responses to “httpd exited on signal 11 – solved”

  1. multibyteNowDisabled Says:

    Thanks. You just saved me a ton of time.

  2. wil Says:

    nice :)

  3. Ivan Says:

    i have the same error without multibyte support

  4. dar Says:

    Me too! No idea what to do!

  5. Paul Says:

    This will help me debug a VirtualBox crash (runs with suid bit too), thanks