Bot Defender Settings
  • 14 Jun 2023
  • Dark
    Light

Bot Defender Settings

  • Dark
    Light

Article Summary

How to manage HUMAN security policies and other settings

Introduction

A policy is a set of rules that determine the behavior of HUMAN’s protection scheme.

By managing Access Control, Volume Based Rules, Risk Cookie, and Data Classification Enrichment, different levels and types of site protection can be configured.

You can create and configure policies for different parts of your site if you have more than one application.

In addition, the Bot Defender Settings section allows you to manage and set your challenge settings, custom parameters, data enrichment, and other product-related configurations.

Active / Bypass

An Application can be set in Active or Bypass mode. Setting the application to Bypass mode disables detection and set the score to zero for all requests. In Bypass mode, the JavaScript Client functionality is limited to cookie management only.

In order to set an application to Active or Bypass, go to Overview -> Policy status & Control -> [Select Application] -> set to Active / Bypass.

1542

Regular Expressions

Regular expressions are patterns used to match character combinations in strings. In JavaScript, regular expressions are also objects. HUMAN uses JavaScript regular expressions, eliminating the need to escape all characters, except for 12 characters with special functions in the regex (,, ^, $, ., |, ?, *, +, (, ), [, {)

A regular expression pattern is composed of simple characters, such as abc, or a combination of simple and special characters, such as ab*c (https://developer.mozilla.org/en-US/docs/Web/JavaScript/Guide/Regular_Expressions)

Important tips for building a safe regex

  • Escaping a special character (,, ^, $, ., |, ?, *, +, (, ), [, {) is necessary when one of the characters is used literally. To escape a special character, use \ before the character. For example, to search for a followed by * followed by b, you'd use a\*b
  • When writing the regex, try to be as explicit as possible. Catastrophic backtracking or exponential time regexes are not recommended, as they are extremely computationally heavy, and impacts effective detection by HUMAN.
  • Avoid using wildcards characters (or use sparingly), such as * or +,
  • Avoid using nested capturing groups.
  • When writing a regex for a User Agent, we recommend using a very specific User Agent that will not accidentally appear in the attacker’s User Agents.
  • Use the | to separate between User Agents.

For more information on building a regex, see:

Examples of how to build a safe regex

To create a Custom Rule to block traffic from User Agent:

PerimeterX/4.16.1 (iPhone; iOS 13.3; Scale/2.00), Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/77.0.3865.120 Safari/537.36 OPR/64.0.3417.167

All special characters must be escaped with \:

PerimeterX/4\.16\.1 \(iPhone; iOS 13\.3; Scale/2\.00\), Mozilla/5\.0 \(Windows NT 10\.0; Win64; x64\) AppleWebKit/537\.36 \(KHTML, like Gecko\) Chrome/77\.0\.3865\.120 Safari/537\.36 OPR/64\.0\.3417\.167

To create an Allowlist rule to allow any User Agent containing automation use:

.*automation.*

If you built a regex that you think is safe, and HUMAN is still not allowing it, contact HUMAN Support

Authorized Services Rate Limiting Rules

Rate Limiting Authorized (allowed) services allow you to limit the rate that authorized services send requests to your server. Once the limit is reached an HTTP 429 response (request rate limit exceeded) is served.

In the main Access Control section, you must first define which services to authorize, either as a custom rule, or as part of the Known Bots and Crawlers. All authorized services that have been added to the Allowlist are available for rate limiting. (apply your changes after adding an allowed service for the dropdown list in the rule creation process to be updated).

To add a rule:

  1. Within Bot Defender product, navigate to Product Settings > Security Policy and open the Rate Limiting Rules tab. There are no default custom rules.
  2. Click Add rate limit rule. A new line is added to the Rate Limiting Rules list.
  3. Select the service you wish to rate limit from the dropdown list (based on name/description or ID). If you cannot locate your service, you must save the changes after adding the new service to the custom list or to the HUMAN configuration and try again.
  4. Select the rate limit of the service and the time window that the service will be limited for (in seconds). You can also apply a specific lifetime duration (unlimited by default). During this time the rule will be enforced, and will be expired after the defined end time.
  5. Click Save Changes.
    1795
    Note:
    You can add multiple rules for the same service name/id for different time windows.

Volume Based Rules

Volume Based Rules allow you to define the volume of activity considered questionable by HUMAN.

1787

The panel is divided into two sub-panels:

Page Views – Intended for regular pages in your web application. Page Views are calculated against a user’s access to full pages, not including individual requests, such as XHRs.

Available Categories:

  • Page Views per Visitor - Defines the threshold for the number of page views from a single visitor.
  • Page Views per Visitor per Tab - Defines the threshold for the number of page views from a single visitor and a single browser tab.
  • Page Views per Client IP - Defines the threshold for the number of page views from a single client IP.
  • Requests per Custom Parameter - Defines the threshold for the number of requests per single custom parameter.

Requests – Intended for any type of request, but specifically useful for APIs. Requests are measured against any individual HTTP request from your application.

Available Categories:

  • Requests per Client IP - Defines the threshold for the number of server requests from a single client IP.
  • Requests per User-Agent per Client IP - Defines the threshold for the number of server requests from a single client (Client IP + User-agent).

 Available Actions:

  1. Click Add page view/ page request rule to add another rule. Each rule is evaluated independently of the others.
  2. Choose a category from the dropdown list.
  3. Enter the threshold of page views or requests
  4. Enter the time range for the rule. Volume Based Rules have a time range of 10 seconds to 86400 seconds (1 day).
  5. Click Save Changes to add the new rule.

Website Sections

Website Sections is an additional option for defining a "website section" is available in the request panel. Website sections split your app into parts, allowing you to define different rate-limiting rules for each part.

Note:
Each rate-limiting rule is evaluated separately. This makes it possible to have part of the website included in two or more sections. When a request is sent to that part of the website, it will count against all relevant rules simultaneously. The “entire app” section is counted against all requests coming to the app.
  • To add a new section, In the “Apply on” column, Click the To be applied on -> and then select an existing section or Create new section.
  • To edit a section click the pencil icon next to its name.
  • To assign a different section to a rule, click the section name and select a different one.

When adding or editing a section, the pop-up dialog allows you to choose a name and a list of URLs. Separate the URLs with commas or line breaks. Each URL must begin with a forward slash (“/”).

830

Deleting a section completely removes it, as well as all of the rules referencing that section.

The Risk Cookie is an HTTP cookie set by HUMAN on each request of a client session. The information contained in the Risk Cookie can be used to determine what actions should be taken; this includes blocking the session, issuing a challenge page, or changing the application behavior.

Note:
  • The default HUMAN policy does not apply the Risk Cookie. To use the Risk Cookie, with or without encryption, you must create a new policy other than the default policy provided by HUMAN. 
  • To enable the risk cookie, generate a cookie key. The cookie key, an identifier used by HUMAN, is used by your server to validate the Risk Cookie. To generate a new cookie key, click Generate Cookie Key.
    Note that If you generate a new cookie key, and are using the old cookie key in your application or infrastructure to perform cookie validation, the incorrect HMAC will be computed for the cookie signature. This can yield undesirable results. Take extra precautions when generating a new cookie key.

Duplicating a Policy

Duplicating a policy makes it easier to create a new policy similar to an existing policy, with minor changes.

To duplicate a policy:

  1. In the Security Policy section, select the policy you want to duplicate from the list of available policies (can be found in the “Policy Name” drop-down).
  2. Click Duplicate Policy (on the right-hand side of the screen). A new policy is created with the same name as the existing policy + copy.
  3. To use the new policy, make sure to connect the requested application to the new policy. This can be done in the *Applications connected to policy” tab, or in the “Overview” section by selecting the new policy as the “policy applied”.

Access Tokens

Access Tokens allow you to securely allow traffic that is generated by friendly applications/users.

To generate an access token, click Add and assign it a lifetime by clicking on the desired date/time.
Page requests using this token by setting HTTP Header x-px-access-token: will automatically be allowed by the Bot Defender and be considered as "Custom Allowlist" for the lifetime of the access token.

Challenge Bypass Tokens

The Challenge Bypass Token allows you to bypass the human challenge in testing environments.

The token is available in Product Settings -> Challenge Settings -> [Application Name] -> Challenge Tokens

  1. Select Add Token to generate the token then select Save Changes
  2. Add the x-px-captcha-testing header with the token as the value so it will be included on the outgoing request.
  3. When presented with a Human Challenge, refresh the page and the challenge will be bypassed.

The request will go through as if the challenge was solved and a "good" cookie will be baked. The token can be used multiple times.

Data & Configuration

In Product Settings -> Data & Configuration you can configure custom parameters and data enrichment settings. Click on the links below for further information:


Was this article helpful?

What's Next