Skip to content

API Discovery Overview

The API Discovery module of the Wallarm platform builds your application REST API inventory based on the actual API usage. The module continuously analyzes the real traffic requests and builds the API inventory based on the analysis results.

The API built inventory includes the following elements:

  • API endpoints

  • Request methods (GET, POST, and others)

  • Required and optional GET, POST, and header parameters of requests and responses including:

    • Type/format of data sent in each parameter
    • Date and time when parameter information was last updated

Availability of response parameters

Response parameters are only available when using node 4.10.1 or higher.

Issues addressed by API Discovery

Building an actual and complete API inventory is the main issue the API Discovery module is addressing.

Keeping API inventory up-to-date is a difficult task. There are multiple teams that use different APIs and it is a common case that different tools and processes are used to produce the API documentation. As a result, companies struggle in both understanding what APIs they have, what data they expose and having up-to-date API documentation.

Since the API Discovery module uses the real traffic as a data source, it helps to get up-to-date and complete API documentation by including to the API inventory all endpoints that are actually processing the requests.

As you have your API inventory discovered by Wallarm, you can:

  • Have a full visibility into the whole API estate including the list of external and internal APIs.

  • See what data is going into and out of the APIs.

  • Get a list of endpoints with the open vulnerabilities.

  • Get a list of the threats that occurred over the past 7 days per any given API endpoint.

  • Filter out only attacked APIs, sort them by number of hits.

  • Filter APIs that consume and carry sensitive data.

  • View visualized summary on your API inventory structure and problems on a handy dashboard.

  • Understand which endpoints are most likely to be an attack target.

  • Find shadow, orphan and zombie APIs.

  • Track changes in API that took place within the selected period of time.

  • Filter API endpoints by the BOLA auto protection state.

  • Provide your developers with access to the built API inventory reviewing and downloading.

How does API Discovery work?

API Discovery relies on request statistics and uses sophisticated algorithms to generate up-to-date API specs based on the actual API usage.

Traffic processing

API Discovery uses a hybrid approach to conduct analysis locally and in the Cloud. This approach enables a privacy-first process where request data and sensitive data are kept locally while using the power of the Cloud for the statistics analysis:

  1. API Discovery analyzes legitimate traffic locally. Wallarm analyzes the endpoints to which requests are made and what parameters are passed and returned.

  2. According to this data, statistics are made and sent to the Cloud.

  3. Wallarm Cloud aggregates the received statistics and builds an API description on its basis.

    Noise detection

    Rare or single requests are determined as noise and not included in the API inventory.

Noise detection

The API Discovery module bases noise detection on the two major traffic parameters:

  • Endpoint stability - at least 5 requests must be recorded within 5 minutes from the moment of the first request to the endpoint.

  • Parameter stability - the occurrence of the parameter in requests to the endpoint must be more than 1 percent.

The API inventory will display the endpoints and parameters that exceeded these limits. The time required to build the complete API inventory depends on the traffic diversity and intensity.

Also, the API Discovery performs filtering of requests relying on the other criteria:

  • Only those requests to which the server responded in the 2xx range are processed.

  • Requests that do not conform to the design principles of the REST API are not processed.

    This is done by controlling the Content-Type header of responses: if it does not contain application/json (like Content-Type: application/json;charset=utf-8), the request is considered to be a non-REST API and is not analyzed.

    If the header does not exist, API Discovery analyzes the request.

  • Standard fields such as Accept and alike are discarded.

Sensitive data detection

API Discovery detects and highlights sensitive data consumed and carried by your APIs:

  • Technical data like IP and MAC addresses

  • Login credentials like secret keys and passwords

  • Financial data like bank card numbers

  • Medical data like medical license number

  • Personally identifiable information (PII) like full name, passport number or SSN

API Discovery provides the ability to configure the detection process and add your own sensitive data patterns (requires NGINX Node 5.0.3 or Native Node 0.7.0 or higher).

Security of data uploaded to the Wallarm Cloud

API Discovery analyzes most of the traffic locally. The module sends to the Wallarm Cloud only the discovered endpoints, parameter names and various statistical data (time of arrival, their number, etc.) All data is transmitted via a secure channel: before uploading the statistics to the Wallarm Cloud, the API Discovery module hashes the values of request parameters using the SHA-256 algorithm.

On the Cloud side, hashed data is used for statistical analysis (for example, when quantifying requests with identical parameters).

Other data (endpoint values, request methods, and parameter names) is not hashed before being uploaded to the Wallarm Cloud, because hashes cannot be restored to their original state which would make building API inventory impossible.

Important

Wallarm does not send the values that are specified in the parameters to the Cloud. Only the endpoint, parameter names and statistics on them are sent.

API Discovery demo video

Watch API Discovery demo video:

Checking API Discovery in playground

To try the module even before signing up and deploying the node to your environment, explore API Discovery in Wallarm Playground.

In Playground, you can access the API Discovery view like it is filled with real data and thus learn and try out how the module works, and get some useful examples of its usage in the read-only mode.

API Discovery – Sample Data

Enabling and configuring API Discovery

To start using API Discovery, enable and configure it as described in API Discovery Setup.