Page tree

Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

Sitemaps are models of backend applications that can be used as a whitelist in workflows. Sitemaps can contain information to describe operations (method), paths (URI), query vars parameters (query parameters) and formdata parameter (post data parameters).

The Sitemap profiles are used in the Sitemap Validation node of the Workflow.

Sitemaps View

Sitemaps are available under the Policies pannel in the Security category. This view lists all available sitemap with the following fields:

  • Sitemap: the name of the sitemap.
  • Base Path: the base path of the sitemap.
  • Comment: a custom description of the sitemap.

This view allows to create, modify, remove, open, import and export sitemaps.

Create a Sitemap

Creating a sitemap will add a new empty sitemap. The General tab displays the following fields:

  • Name: the name of the sitemap.
  • Comment: a custom description of the sitemap.
  • Base Path: the base path of the sitemap.

The behaviours tab gives access to 3 new fields which configurate the way the sitemap will be validated when used in the Sitemap Validation node.

  • Match policy: Defines the matching behaviour of the Sitemap Validation node when static and dynamic paths exist for the same resource. The value "Any match" allows dynamic paths to be checked when static paths have not been able to match the method and path of the request. The value "Strict" allows dynamic paths only when no static path has been found to match the method and path of the request. If a static path is not able to match a request because of the parameters, the validation will not check the dynamic paths to match the request and the validation will fail.
  • Query vars policy: Defines the validation behaviour regarding query parameters. The "Defined only" value will only allow validation of query parameters defined in the sitemap used by the Sitemap Validation node. Unknown parameters will automatically make sitemap validation fails. The "Accept Undefined" value will allow undeclared query parameters to be accepted by the Sitemap Validation node.
  • FormData vars policy: Defines the validation behaviour regarding form data parameters (also called post data parameters). The "Defined only" value will allow validation of form data parameters defined in the sitemap. The "Accept Undefined" value will allow undeclared form data parameters to be accepted by the Sitemap Validation node.

 

Modify a Sitemap

Modifying a sitemap will update the fields of the selected sitemap with the following values:

  • Name: the name of the sitemap,
  • Comment: a custom description of the sitemap,
  • Base Path: the base path of the sitemap.

Remove a Sitemap

Removing the selected sitemaps will delete them from the database.

Import a Sitemap

Importing a sitemap will update the selected sitemap with a user provided file. Importation involve learning algorithm that will incorporate the sitemap from the uploaded file into the selected sitemap (i.e. methods and paths will be updated). The importation of sitemap is done with the following fields:

  • File: the file to upload and containing the data to import,
  • Type: the type of the sitemap, the importation manage the following types:
  • Operation learning: Defines if operations (methods) will be learnt from the imported file or if they will all be replaced by the wildcard method "ACCEPT-ALL".
  • Clear content: Defines if the content of the sitemap must be removed before importing the new data.

Export a Sitemap

The sitemaps can be downloaded in the swagger 2.0 json format.

Sitemap Detailed view

Double-clicking on the sitemap or clicking on the "open" button open a new tab with the content of the selected sitemap. Here is an example of a short sitemap:

A sitemap is built by adding different pieces of information to match the available pages of a websites including method and parameters. The following informations are available:

  • Paths: list of path added to the sitemap. They can be static or dynamic.
  • Operations: the HTTP methods allowed on the corresponding path.
  • Global parameters: list of parameters available everywhere in the sitemap
  • Path parameters: list of parameters available for all operations of a defined path.
  • Operation parameters: list of parameters dedicated to a specific operation and path

Build sitemap

A sitemap can be created and built directly in the GUI of DenyAll WAF. The different items used to describe a sitemap can be added and configured to create a complete sitemap matching all pages of a backend application.

Add static path

By default, sitemaps are empty. The base path defined at the creation of the sitemap is not included in the corresponding sitemap. Every paths have to be added to the sitemap to be validated by the Sitemap Validation workflow node, even the path corresponding to the base path.

The "Add" button is used to add new path to a sitemap.

Image Removed

Warning
titleAdd complete path

When adding a new path to a sitemap, the complete path with directories and sub-directories always have to be defined. It is not possible to select a directory in the sitemap and add a new path from the selected point.

Add dynamic path

Dynamic paths can be added to the sitemap to prevent creation of hundreds of path for similar resources such as images or css files. The dynamic path are added like the static path but the dynamic part of the path must be surounded by curly brackets: /{dynpath}

By default, dynamic paths will be replaced internally by the following regular expression [^/]+ which matches any string with at least one character but without slash character. Of course, it is possible to define a custom expression for a dynamic path by adding a "Path parameters" or an "Operation parameter" with attribute "Location" set to "path" (see below to get more information about Path parameters and Operation parameters).

Image Removed

Add operation

Operations are used to defined HTTP method on a selected path. The available operations are:

  • GET
  • POST
  • PUT
  • PATCH
  • DELETE
  • HEAD
  • OPTION
  • ACCEPT-ALL

The "ACCEPT-ALL" value can be used to accept any methods on a selected path. It is also possible to define custom methods.

Info
titleNone operation

When a path is added to a sitemap, no method is defined and the value "NONE" is displayd in the "Operations" column. The validation of a request will never succeed on a path with a "NONE" operation and it is mandatory to add valid operations to sitemap paths.

 

Image Removed

Operations are added by selecting a path and using the "Add" button in the "Path operations" panel available on the right of the screen.

Image Removed

Once an operation is added on a path, the name of the selected operation is displayed in the "Operations" column for the seleted path.

Image Removed

Custom methods can be created when adding an operation to a path. The custom name can be set in the pop-up displayed to add operations in upper case.

Image Removed

Custom methods are displayed and can be configured like standard methods.

Image Removed

Parameter configuration

Parameters can be added to a sitemap to match HTTP parameters in incoming requests or to define dynamic parts of paths. Parameters configuration is separated between definition part and value part. The definition part covers the name and location of the parameters with the following information:

  • Id: unique identifier only available for Global parameters. It is used to reference global parameter in other parts of the sitemap.
  • Name: name of the parameter. For query or form data parameters, it will be used to match the name of HTTP parameters.
  • Location: position of the parameter in the request. Possible values are path, query and form data. Path location is used for dynamic paths. Query location is used for HTTP query parameters located after the question mark in the requests. Form data location covers data posted in HTTP requests.
  • Required: option to define if a parameter must be present in the request or if it is optionnal. Path parameters are always required even if the option is not checked.

The value part allows configuration of the value accepted for a parameter. A list of type is available to define values of parameters and each type provides a list of attributes to define a precise set of values allowed for a parameter value:

The available data types are described below with their related attributes.

String type

The String type is the default type for parameters. This data type provides the following attributes

  • Pattern: Regular expression defining a parameter. A parameter defined with a pattern value must match the regular expression to be allowed by the Sitemap validation node. The regular expression can be tested in the GUI.
  • Max length: Maximum length of the parameter value.
  • Min length: Minimum length of the parameter value.
  • Enum: List of possible values for the parameter value. Defined with the following syntax: ["value1", "value2", "value3"].
  • Format: defines the format of the string type. The available values are "byte", "binary", "date", "date-time" and "password". The "date" and "date-time" format are defined in RFC3339. Please refer to this RFC for more details about these 2 formats.

Number

The Number type is used to define parameters with a signed decimal value. This type provides the following attributes:

  • Minimum: Minimum decimal value for the parameter value.
  • Exclusive Minimum: check box to exclude the minimum value defined by the previous field. By default, this option is unchecked.
  • Maximum: Maximum decimal value for parameter value.
  • Excluse Maximum: check box to excluse the maximum value defined by the previous field. By default, this option is unchecked.
  • Multiple of: value that will be tested to be a multiple of the parameter value. This value must be a positive number or 0.
  • Format: defines the format of the number type. The available values are "float" and "double".

Integer

The Integer type is used to define parameters with a signed integer value. This type provides the following attributes:

  • Minimum: Minimum integer value for the parameter value.
  • Exclusive Minimum: check box to exclude the minimum value defined by the previous field. By default, this option is unchecked.
  • Maximum: Maximum integer value for parameter value.
  • Excluse Maximum: check box to excluse the maximum value defined by the previous field. By default, this option is unchecked.
  • Multiple of: value that will be tested to be a multiple of the parameter value. This value must be a positive integer or 0
  • Format: defines the format of the integer type. Available values are "integer (32bits)" and "long (64bits)".

Boolean

The boolean type is used to define parameters that can take "true" or "false" values. The boolean type is case sensitive and only supports "true" and "false". If other values of "true" and "false" are required for a parameter value, the String type should be used.

Warning
titleCombined attributes

Attributes for string, number and integer types can be combined to restrict allowed values of parameters. All attributes of a parameter value are checked and must be satisfied when tested against a value in requests.

Add path parameter

Path parameters are used to defined parameters common to a set of operations (HTTP methods). They can be used to define dynamic part of the path on which they are configured and also to share parameters definition for different operations of a path.

As explained above, dynamic paths are defined with a parameter between curly brackets. The parameter can be defined by adding a path parameter to the dynamic path. When adding path parameters to defined dynamic path, the name must match the value between curly brackets and the "Location" attribute of the path parameter must be set to "path". In the following example, the dynamic path with "dynname" value is defined as a path parameter. The value of "dynname" is a string with a length between 6 and 16 characters.

Image Removed

Once added to a path, a path parameter is displayed in the "Path parameters" panel for the corresponding path. In the following example, the "dynname" parameter is common to the 3 operations (or methods) "HEAD", "GET" and "POST".

Image Removed

The other way to use path parameters is to defined query parameters or form data parameters available for all operations of a path. It helps define similar parameters for different methods only once. The definition of such path parameters is similar to the previous one but are defined with "query" or "formData" for the "Location" attribute of the path parameter. To define a query parameter for different operations, add a new path parameter to the corresponding path.

The following example shows how to add a common query parameter named "arg01" of type integer for the 3 methods "GET", "POST" and "PUT" by using only one path parameter.

Image Removed

Add operation parameter

To the contrary of path parameters which are applyed to all operations of a path, operation parameters allows dedicated parameter configurations for each operations of a path. This kind of parameters is used when a clear difference exists between all the operations of a path. This is commonly used to define parameters for GET operation differently than POST operation.

Operation parameters are added after selecting the path where they will apply and then the operation they are linked to. The operation parameters are configured in the "Operation parameters" displayed after selecting an operation of a path. In this panel, the "Add" button will display the pop-up where operation parameters are configured.

Image Removed

Since operation parameters are linked to an operation, it is possible to define operation parameters with different name or location between operations of a same path. The following example show a GET operation with only one parameter and a POST operation with 2 parameters, one located in the query and one located in form data.

Image Removed

Add global parameter

Global parameters are defined on the base path of a sitemap and can be use anywhere in the sitemap as path parameters or operation parameters. Global parameters are defined as described in the "Parameter configuration" chapter above.

The only parameter restricted to global parameter is the identified field called "Id". This field is used to identify a global parameter is the sitemap. Global parameters can be used in any part of the sitemap and need a unique identifier to link the global parameter with its definition. One of the goal of global parameters is to define parameters which appear at different paths in the sitemap and avoid adding several time the same parameters.

Image Removed

When a global parameter is added, it appears in the "Global parameters" panel available on the sitemap base path and can be used in the whole sitemap. A path parameter or operation parameter can be defined with the definition of a global parameter by using a reference to the identifier set for the global parameter.

Image Removed

Warning
titleLocation attribute of global parameters

Be careful when using reference to global parameter. The location attribute configured for the global parameter must be valid depending on which type of parameter uses the reference. A query or form Data parameter using global parameter defined with a path location will not be valid and requests on such configuration will never be accepted by the Sitemap Validation node of the workflow.

When a reference is linked to a path or an operation, it appears in the corresponding panel. References assigned to operation apply only to the operation like with operation parameters. It is also possible to combine global parameters with any other type of parameters: path parameters, operation parameters and even global parameters.

 

Learning a Sitemap

To get a more accurate sitemap of a backend application, Learning algorithm are available with Learning Logs feature.

Table of Contents