Schema-Based Testing Setup ¶

This article describes how to enable and configure Wallarm's Schema-Based Testing.

Enable¶

Schema-Based Testing is disabled by default. To enable:

Contact sales@wallarm.com to activate the Security Testing subscription plan for your account.
Go to the Security Testing → Schema-Based → Test policies tab and create at least one policy.

Prerequisites - token¶

Schema-Based Testing requires a token for authorizing data exchange between running Docker container and Wallarm Cloud. The token can be created in two ways:

Automatically - Schema-Based Testing will create it automatically and include into the docker run command on first attempt to copy Docker command from any policy. Other policies will re-use already existing token.
Manually - in Wallarm Console, go to Settings → API Tokens, click New token; on creation, set Token usage to Schema-Based Testing agent. All policies will use this token.

Configure test policies¶

You can configure test policy based on OpenAPI specification (OAS) or Postman collection. Note that one test policy is aimed at only one type of testing (either OAS or Postman-based).

OAS is more focused on input validation, injection, misconfiguration detection
Postman - on complex business logic and access control vulnerabilities

OAS-based¶

OpenAPI specification (OAS)-based test policy defines persistently:

Application's OpenAPI specification
Tests to run

Besides persistent parameters that are the same for any test run, each test policy may optionally include parameters that can be re-defined during each next test run (Runtime parameters). Re-defining the runtime parameters can be useful for embedding of Docker into the CI/CD pipelines:

Application's Target URL

(although can be redefined during each run, some initial value is required)
Authentication parameters

To configure test policy:

Go to Wallarm Console → Security Testing → Schema-Based → Test policies.
Click Add policy, attach OpenAPI specification file.
Select test types to run.
Set Target URL (can be re-defined dynamically during each test run).
Optionally, add authentication Runtime parameters.

Postman collection-based¶

With Postman-based security testing you can automate security scans alongside your regular API tests, ensuring that each API run is thoroughly tested for vulnerabilities.

API functional tests as basis

Wallarm's schema-based testing leverages your functional tests to generate security tests. The more complete your functional tests are, the broader the security coverage Wallarm can provide. More APIs, users, and requests mean richer and more effective security testing.

Pre-check of Postman collection¶

Before using Postman data for schema-based security testing:

Ensure that you collection and environment files contain:
- functional tests for API endpoints
- location of the target application
- all environment variables set
- necessary credentials to authenticate in the target application
(Recommended) Check whether the Postman collection is working at all (working). For example, this can be done by running the command:
```
newman run my_collection.json -e my_environment.json
```
This can tell you in advance whether there are any problems, for example, related to the quality of the Postman collection, the availability of the target URL, or that all the necessary variables are specified.

Valid Postman collection

If the Postman collection itself cannot work, then the schema-based security testing will not work either.

Configuring test policy in Wallarm¶

In Wallarm, Postman collection-based test policy defines:

Application's Postman collection.
Postman environment file(s) (optional if all configuration is stored in the main collection file).

Target URL and authentication

Application's Target URL and authentication parameters are defined in the Postman collection or environment files.
Test case selection is not currently supported for security testing based on Postman collections.

To configure test policy:

Go to Wallarm Console → Security Testing → Schema-Based → Test policies.
Click Add policy, set Source to Postman collection.
Attach Postman collection file.
Optionally, attach Postman environment file(s). Attaching 2 files allows running testing twice with different variable values (for example, different credentials) and then comparing results.

Business logic security testing¶

For business logic testing (OWASP API1:2023 Broken Object Level Authorization (BOLA) and OWASP API5:2023 Broken Function Level Authorization (BFLA)) Wallarm's Schema-Based Security Testing requires API traffic from at least two different authenticated users, preferably with varying privileges (e.g., admin and regular users). This diversity enables more effective business logic security tests and provides a more thorough assessment.

Business logic vulnerability	Input requirements
OWASP API1:2023 Broken Object Level Authorization (BOLA)	Testing for this vulnerability requires multiple authenticated users to demonstrate whether proper object-level authorization checks are in place.
OWASP API5:2023 Broken Function Level Authorization (BFLA)	Testing for this vulnerability requires requests from users with different privilege levels to evaluate whether function-level authorization is enforced consistently.

Example 1: Using Postman collections with requests from two users¶

Do the following:

Create a Postman collection containing requests from two authenticated users. For example, in the target application, include login and activity requests from User A and User B in the same collection.
Verify that all requests are executed correctly.
Create a test policy with the created Postman collection.

Example 2: Using Postman environments for multiple users¶

Do the following:

Create two Postman environment files, each holding the credentials for a different authenticated user, e.g.:
- env1.json for User A
- env2.json for User B
Create a test policy that uses your Postman collection and both environment files.

In this setup, the collection is executed twice - once with env1.json (User A) and once with env2.json (User B) - so each request runs under both user contexts.

Editing existing policy¶

You can edit previously created policies: while clicking policy itself opens its Docker command info, you can click the edit button to access the edit dialog:

Docker run¶

As test policy is created, it provides you with the Docker run command which allows you start tests for your application:

Go to Wallarm Console → Security Testing → Schema-Based → Test policies.
Click you policy. The policy's Docker command will be displayed.
If necessary, redefine Docker log level and format.
Copy command and run it or embed into your CI/CD pipeline. This will run security tests selected in the policy for your application.

Remember that, for OAS-based run, you can re-define the policy's Runtime parameters on each run by adding the corresponding -e parameters to the Docker run command, for example:
```
-e TARGET_URL="http://dvapi.st.wallarm.tools" 
-e AUTH_HEADER="Authorization: Bearer <VALUE>"
```
You can simplify re-defining these parameters by selecting the Rewrite authentication data and/or Rewrite target URL checkboxes that add to the command:
```
-e AUTH_HEADER="${AUTH_HEADER}" -e TARGET_URL="${TARGET_URL}"
```
View run statistics and test run results on the Test runs tab.

Deleting policies¶

You can delete a test policy. If you do so:

Information on previous test runs will remain untouched
You will not be able to run Docker's command based on the deleted policy
If policy's Docker containers are running, they will continue to do so
When policy's Docker containers stop, you will not be able to re-run them