Apache can reject requests that identify themselves with selected bot user-agent strings. This guide explains how an administrator can apply that restriction to one website through WHM’s Include Editor.

Before you begin

Back up the include and identify the site’s actual document root. The example uses Apache 2.4 with the required authorization and SetEnvIf modules. A user-agent string can be forged, so this is not proof of a visitor’s identity or a complete defence against abusive traffic.

Add the restriction

  1. Open Service Configuration > Apache Configuration > Include Editor in WHM.

    Include Editor

  2. Under Pre Main Include, select All Versions where shown.

    Pre Main Include

  3. Adapt this example, using the actual document root and the bot names you intend to block:

    <Directory "/home/exampleuser/public_html">
        SetEnvIfNoCase User-Agent "MJ12bot|AhrefsBot|SemrushBot|Baiduspider" blocked_bot
        <RequireAll>
            Require all granted
            Require not env blocked_bot
        </RequireAll>
    </Directory>
  4. Review existing authorization directives for conflicts, save using Update and follow WHM’s configuration-check/restart controls.

Test the effect

Check that normal website requests still work and matching user-agent requests receive a forbidden response. Restore the previous include if the change causes unexpected errors. Subrequests and application-specific authorization can require additional configuration.