406 Not Acceptable and Apache mod_security

Maybe you saw my YM status rant a few days ago—406 Not Acceptable. (With “WTF! WP 2.1 sucks!” appended.) Or maybe not. Either way, I tried looking up why I was getting that error, but found no real clues except a hint at a certain .htaccess fix. And I didn’t see the connection between an HTTP response code and an .htaccess edit, especially when you read something like this, so I ignored it.

Today, I had the same unacceptable problem on a non-WordPress site and this worried me more. It’s a corporate site. If I couldn’t figure it out I’d probably die of shame and/or ignorance. Maybe because the chances of finding support for that type of problem were lower than if you were addressing a WordPress-related one, given its large community and all.

I couldn’t find anything substantial. I found a PHP bug report. I saw something on Drupal. Not helpful either. Several more email discussion lists on various topics. Now something’s fishy here. There’s no clear common denominator in any of these leads!

I gave up, did other work, checked my mail, and Googled once more. Then I found Urban Giraffe’s entry.

Some Googling later and I found information about an optional Apache module called mod_security. This is a very nice module that acts as an Apache firewall – it blocks a lot of the usual routes that people use to hack websites. In particular it scans POST requests (sent when you “save” something on a website), and displays a 406 error for anything controversial. Bingo!

Hmm. Okay. Humor me.

The reason I’m documenting these frustrating few hours of my life is in the hope that it may prove useful to someone else. It appears that mod_security, if configured aggressively, can cause a lot of problems and these may manifest themselves in Mambo, WordPress, or any piece of web software.

The solution was very simple. The following lines were added to the .htaccess file to disable mod_security:

<IfModule mod_security.c>
SecFilterEngine Off
SecFilterScanPOST Off
</IfModule>

And that is the solution I first saw a few days ago. At least things seemed fixed after I tried it.

Assuming debugging is done through trial and error and following the post hoc ergo propter hoc fallacy, the lesson here is that if you’re onto something, it’s probably correct.

Second, you can’t find everything on Google the first time around. After scouring one results page after the other, you have to modify your search phrases. If that isn’t enough, use—gasp!—other search engines.

Third and most important of all, keep on blogging. You never know whom you might end up helping. Thanks Urban Giraffe!

15 replies

  1. this isn’t really gonna be helpful after the fact, but .htaccess files can hold a lot of settings relating to your Apache webserver (that wouldn’t ordinarily be available unless you had root access to your machine and apache’s configuration file — aptly named httpd.conf).

    the most common use for it is to restrict access to a file or a folder by triggering a username/password dialog box when the specific url is accessed. other times, people use it to map friendly URLs against the querystrings that are understood by dynamic pages such as those generated by WordPress.

    i guess the other lesson here is that in the *nix world, similar names have related functions. apache’s webserver name is httpd, for example, and htaccess is the way we configure httpd’s method for accessing a given directory :)

    Reply to this

  2. Actually, I realized the Apache – .htaccess connection a bit, but when I saw the SecFilterEngine stuff, that got me asking. It’s a good thing that Urban Giraffe convinced me to try it with his explanation, as I could find no explanation for it in the previous place I had seen it. The question is, why are was I getting those errors only now, when I had been using the scripts that caused them for quite a while now. That added to the confusion. I guess I’m also wondering if SecFilterEngine will be safe to turn off. I have this feeling the webhost(s) suddenly turned them on for added security.

    Thanks very much Luis. ^_^

    Reply to this

  3. those problems occur because of invalid characters or sequence of characters in an HTTP POST. i encountered the problem myself in my wordpress blog. I simply started deleting text until I found the offending characters and simply rephrased them. However, I could not point out a trend in the offending characters.

    Reply to this

    • Finding the offending string(character) was easy in my case it was the escape sequence “%E9 ” which stands for É . after testing a few things I realized that all escape sequence from my Ajax HTTPxrequest generate the 406 error. …..

      ………………. and I am stuck here

      Reply to this

      • I found a way around the 406 error. The company hosting
        my web site would not turn down the security module
        and I need the accented character.

        I figured that % could be replaced by the word “percent”
        then I realized that the word could be a valid part of the
        string so I purposeley missspelled it. It became “pxrcent”
        from there it’a easy.

        JavaScript in the web page
        // Standard urlencoding
        Descr=escape(document.form1.Description.innerText);
        // É becomes %C9

        //replacing the % wi9th the text
        Description=Descr.replace(/%/g,”pxrcent”);
        // %C9 becomes pxrcentC9

        In the server the PHP code is:
        $Description=urldecode(str_replace(“pxrcent”,”%”,$_GET[‘Description’]));

        well you get the idea…

        Reply to this

  4. I also noticed that, but I tried getting rid of them in Notepad++ by changing the character encoding (so I could detect them) but there didn’t seem to be any change. This is also why it took a while before I could find a solution that worked. However, it’s true that the SecFilterEngine might not be the best way to go.

    Reply to this

  5. I used the SecFilterEngine method so that I could print the $_POST variables where I located the invalid characters. I cleaned up the code, removed the .htaccess modification and everything was fixed. BUT I would never have known that it was an invalid character problem had I not found your site. Thanks a lot!!

    Reply to this

  6. Pingback: Jason Friesen {dot} ca » Blog Archive » New Flock and 406 Errors

  7. Had the same problem when I created an image, uploaded it to my site and tried to load it… I tried .gif, .jpg and .png, all failed with the Not Acceptable error.

    Even more strangely, existing .png, .gif and .jpg files worked fine, so I’m not discounting that something was up with Irfanview which was corrupting the file headers somehow – but I don’t have time to investigate. The images work fine in Explorer and in Firefox, so I just applied your fix – works perfectly. Now I have time to go off and research the problem and (hopefully) fix it… Cheers!

    Reply to this

  8. Want to know something even odder? I am getting a 406 just recently as well, but here’s the kicker… Only in Firefox. My girlfriend’s IE is fine, and my Safari is fine, but Firefox is losing access to my site one directory at a time…

    Even odder, these directories exist, and not just USED to work, but alternate in availability… Sometimes they will work, sometimes they will not. Not all at once either, individually, and at random moments…

    Still Mod_Security?

    Reply to this

Reply

Your email address will not be published. Required fields are marked *

*

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <s> <strike> <strong> Anything in between < and > will be treated as HTML, so you might want to use character entities if you have to display literal markup.

*

%d bloggers like this:
Technology & Computers - Top Blogs Philippines