# Splunk Enterprise via Logstash

These instructions provide you with the example integration of Wallarm with the Logstash data collector to further forward events to the Splunk SIEM system.

The most common logging scheme in complex systems consists of the following components:

* Data collector: accepts logs from several sources and forwards logs to the SIEM system
* [SIEM system](https://www.wallarm.com/what/siem-whats-security-information-and-event-management-technology-part-1) or log management systems: used to analyze logs and monitor the system status

![Webhook flow](https://docs.wallarm.com/images/user-guides/settings/integrations/webhook-examples/logstash/splunk-scheme.png)

## Used resources

* [Splunk Enterprise](#splunk-enterprise-configuration) with WEB URL `https://109.111.35.11:8000` and API URL `https://109.111.35.11:8088`
* [Logstash 7.7.0](#logstash-configuration) installed on Debian 11.x (bullseye) and available on `https://logstash.example.domain.com`
* Administrator access to Wallarm Console in [EU cloud](https://my.wallarm.com) to [configure the Logstash integration](#configuration-of-logstash-integration)

!!! info "Wallarm Cloud IP addresses"
    To provide Wallarm Cloud access to your system, you may need a list of its public IP addresses:

    **US Cloud:**

    ```
    34.102.90.100
    34.94.156.115
    35.235.115.105
    34.94.85.217
    34.94.51.234
    34.102.59.122
    34.94.238.72
    35.235.100.79
    34.102.45.38
    34.94.241.21
    34.94.203.193
    34.94.238.221
    34.94.9.235
    34.94.118.150
    34.94.193.9
    ```

    **EU Cloud:**

    ```
    34.141.230.156
    34.91.138.113
    34.90.114.134
    35.204.127.78
    34.90.24.155
    34.7.147.149
    ```

    **ME Cloud:**

    ```
    34.166.215.174
    ```
Since the links to the Splunk Enterprise and Logstash services are cited as examples, they do not respond.

### Splunk Enterprise configuration

Logstash logs are sent to Splunk HTTP Event Controller with the name `Wallarm Logstash logs` and other default settings:

![HTTP Event Collector Configuration](https://docs.wallarm.com/images/user-guides/settings/integrations/webhook-examples/splunk/logstash-setup.png)

To access the HTTP Event Controller, generated token `93eaeba4-97a9-46c7-abf3-4e0c545fa5cb` will be used.

A more detailed description of Splunk HTTP Event Controller setup is available in the [official Splunk documentation](https://docs.splunk.com/Documentation/Splunk/8.0.5/Data/UsetheHTTPEventCollector).

### Logstash configuration

Since Wallarm sends logs to the Logstash intermediate data collector via webhooks, the Logstash configuration should meet the following requirements:

* Accept the POST or PUT requests
* Accept HTTPS requests
* Have public URL
* Forward logs to Splunk Enterprise, this example uses the `http` plugin to forward logs

Logstash is configured in the `logstash-sample.conf` file:

* Incoming webhook processing is configured in the `input` section:
    * Traffic is sent to port 5044
    * Logstash is configured to accept only HTTPS connections
    * Logstash TLS certificate signed by a publicly trusted CA is located within the file `/etc/server.crt`
    * Private key for TLS certificate is located within the file `/etc/server.key`
* Forwarding logs to Splunk and log output are configured in the `output` section:
    * Logs are forwarded from Logstash to Splunk in the JSON format
    * All event logs are forwarded from Logstash to Splunk API endpoint `https://109.111.35.11:8088/services/collector/raw` via POST requests. To authorize requests, the HTTPS Event Collector token is used
    * Logstash logs are additionally printed on the command line (15th code line). The setting is used to verify that events are logged via Logstash

```bash linenums="1"
input {
  http { # input plugin for HTTP and HTTPS traffic
    port => 5044 # port for incoming requests
    ssl => true # HTTPS traffic processing
    ssl_certificate => "/etc/server.crt" # Logstash TLS certificate
    ssl_key => "/etc/server.key" # private key for TLS certificate
  }
}
output {
  http { # output plugin to forward logs from Logstash via HTTP/HTTPS protocol
    format => "json" # format of forwarded logs
    http_method => "post" # HTTP method used to forward logs
    url => "https://109.111.35.11:8088/services/collector/raw" # ednpoint to forward logs to
    headers => ["Authorization", "Splunk 93eaeba4-97a9-46c7-abf3-4e0c545fa5cb"] # HTTP headers to authorize requests
  }
  stdout {} # output plugin to print Logstash logs on the command line
}
```

A more detailed description of configuration files is available in the [official Logstash documentation](https://www.elastic.co/guide/en/logstash/current/configuration-file-structure.html).

!!! info "Testing Logstash configuration"
    To check that Logstash logs are created and forwarded to Splunk, the POST request can be sent to Logstash.

    **Request example:**
    ```curl
    curl -X POST 'https://logstash.example.domain.com' -H "Content-Type: application/json" -H "Authorization: Splunk 93eaeba4-97a9-46c7-abf3-4e0c545fa5cb" -d '{"key1":"value1", "key2":"value2"}'
    ```

    **Logstash logs:**
    ![Logstash logs](https://docs.wallarm.com/images/user-guides/settings/integrations/webhook-examples/logstash/splunk-curl-log.png)

    **Splunk event:**
    ![Splunk events](https://docs.wallarm.com/images/user-guides/settings/integrations/webhook-examples/splunk/logstash-curl-log.png)

### Configuration of Logstash integration

* Webhooks are sent to `https://logstash.example.domain.com`
* Webhooks are sent via POST requests
* The webhook integration has default advanced settings
* Webhooks sent to Webhook URLs are all available events: hits, system events, vulnerabilities, scope changes

![Webhook integration with Logstash](https://docs.wallarm.com/images/user-guides/settings/integrations/add-logstash-integration.png)

[More details on the Logstash integration configuration](https://docs.wallarm.com/user-guides/settings/integrations/logstash.md)

## Example testing

To test the configuration, a new user is added in Wallarm Console:

![Adding user](https://docs.wallarm.com/images/user-guides/settings/integrations/webhook-examples/adding-user.png)

Logstash will log the event as follows:

![Log about new user in Splunk from Logstash](https://docs.wallarm.com/images/user-guides/settings/integrations/webhook-examples/logstash/splunk-user-log.png)

The following entry will be displayed in Splunk events:

![New user card in Splunk from Logstash](https://docs.wallarm.com/images/user-guides/settings/integrations/webhook-examples/splunk/logstash-user.png)

## Getting events organized into a dashboard

To get Wallarm events organized into a ready-to-use dashboard in Splunk 9.0 or later, you can install the [Wallarm application for Splunk](https://splunkbase.splunk.com/app/6610).

This application provides you with a pre-configured dashboard that is automatically filled with the events received from Wallarm. In addition to that, the application enables you to proceed to detailed logs on each event and export the data from the dashboard.

![Splunk dashboard](https://docs.wallarm.com/images/user-guides/settings/integrations/splunk-dashboard-by-wallarm.png)

To install the Wallarm application for Splunk:

1. In the Splunk UI ➝ **Apps** find the `Wallarm API Security` application.
1. Click **Install** and input the Splunkbase credentials.

If some Wallarm events are already logged in Splunk, they will be displayed on the dashboard, as well as further events Wallarm will discover.

In addition, you can fully customize the ready-to-use dashboard, e.g. its view or [search strings](https://docs.splunk.com/Documentation/Splunk/latest/SearchReference/Search) used to extract data from all Splunk records.