Datadog via Fluentd/Logstash¶

Wallarm'ı, Fluentd veya Logstash ara veri toplayıcısı aracılığıyla tespit edilen olayların bildirimlerini Datadog'a gönderecek şekilde yapılandırabilirsiniz.

The most common logging scheme in complex systems consists of the following components:

• Data collector: accepts logs from several sources and forwards logs to the SIEM system

• SIEM system or log management systems: used to analyze logs and monitor the system status

Used resources¶

• Genel URL'de bulunan Fluentd veya Logstash servisi

• Genel URL'de bulunan Datadog servisi

• Wallarm Console'da EU cloud üzerinde yönetici erişimi ile Fluentd/Logstash integration yapılandırması

35.235.66.155
34.102.90.100
34.94.156.115
35.235.115.105
34.94.85.217
34.94.51.234
34.102.59.122
34.94.238.72

34.141.230.156
34.91.138.113
34.90.114.134
35.204.127.78

Requirements¶

Wallarm, webhooks aracılığıyla ara veri toplayıcısına günlük gönderdiğinden, Fluentd veya Logstash yapılandırmasının aşağıdaki gereksinimleri karşılaması gerekir:

• POST veya PUT isteklerini kabul etmek

• HTTPS isteklerini kabul etmek

• Genel bir URL'ye sahip olmak

• Günlükleri Datadog'a datadog_logs Logstash eklentisi veya fluent-plugin-datadog Fluentd eklentisi aracılığıyla iletmek

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22

input {
  http { # input plugin for HTTP and HTTPS traffic
    port => 5044 # port for incoming requests
    ssl => true # HTTPS traffic processing
    ssl_certificate => "/etc/server.crt" # Logstash TLS certificate
    ssl_key => "/etc/server.key" # private key for TLS certificate
  }
}
filter {
  mutate {
    add_field => {
        "ddsource" => "wallarm" # mutate filter adding the source field to the Datadog log record for further filtration of Wallarm logs
    }
  }
}
output {
  stdout {} # output plugin to print Logstash logs on the command line
  datadog_logs { # output plugin to forward the Logstash logs to Datadog
      api_key => "XXXX" # API key generated for the organization in Datadog
      host => "http-intake.logs.datadoghq.eu" # Datadog endpoint (depends on the registration region)
  }
}

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30

<source>
  @type http # input plugin for HTTP and HTTPS traffic
  port 9880 # port for incoming requests
  <transport tls> # configuration for connections handling
    cert_path /etc/ssl/certs/fluentd.crt
    private_key_path /etc/ssl/private/fluentd.key
  </transport>
</source>
<match datadog.**>
  @type datadog # output plugin to forward logs from Fluentd to Datadog
  @id awesome_agent
  api_key XXXX # API key generated for the organization in Datadog
  host 'http-intake.logs.datadoghq.eu' # Datadog endpoint (depends on the registration region)

  # Optional
  include_tag_key true
  tag_key 'tag'

  # Optional tags
  dd_source 'wallarm' # adding the source field to the Datadog log record for further filtration of Wallarm logs
  dd_tags 'integration:fluentd'

  <buffer>
          @type memory
          flush_thread_count 4
          flush_interval 3s
          chunk_limit_size 5m
          chunk_limit_records 500
  </buffer>
</match>

Setting up integration with Fluentd or Logstash¶

1. Wallarm Console'da Integrations → Fluentd/Logstash bölümüne giderek Datadog entegrasyon kurulumuna devam edin.

2. Entegrasyon adını girin.

3. Hedef Fluentd veya Logstash URL'sini (Webhook URL) belirtin.

4. Gerekirse, gelişmiş ayarları yapılandırın:

   • Request method: POST or PUT. By default, POST requests are sent.
   • Request header and its value if the server requires a non-standard header to execute the request. The number of headers is not limited.
   • CA certificate: certificate of CA that signed a server certificate. If CA is publicly trusted, this field is optional. If a server certificate is self-signed, this field is required and must contain a certificate of your own CA that signed a server certificate.
   • Verify TLS certificate: this setting allows to disable verification of the specified server certificate. By default, Wallarm verifies whether a server certificate is signed by a publicly trusted CA. We do not recommend disabling the verification of production server certificates. If your server uses a self-signed TLS certificate, you can add a self-signed CA certificate to allow sending requests to this server.
   • Request timeout, in seconds: if the server does not respond to the request within the specified time, the request fails. By default: 15 seconds.
   • Connection timeout, in seconds: if the connection to the server cannot be established during the specified time, the request fails. By default: 20 seconds.
   • Wallarm data format: either a JSON Array (default) or a New Line Delimited JSON (NDJSON).

5. Belirtilen URL'ye bildirim gönderimini tetikleyecek olay türlerini seçin. Olaylar seçilmezse bildirim gönderilmeyecektir.

6. Entegrasyonu test edin ve ayarların doğru olduğundan emin olun.

7. Add integration butonuna tıklayın.

Fluentd integration example:

Testing integration¶

Integration testing allows checking configuration correctness, availability of the Wallarm Cloud, and the notification format. To test the integration, you can use the button Test integration when creating or editing the integration.

The integration is tested as follows:

• Test notifications with the prefix [Test message] are sent to the selected system.

• Test notifications cover the following events (each in a single record):

  • New user in the company account
  • Newly detected hit
  • Newly discovered IP in the company scope
  • New trigger in the company account
  • Newly discovered security vulnerability

• Test notifications include test data.

Fluentd veya Logstash ara veri toplayıcısındaki test günlüğü:

[
    {
        summary:"[Test message] [Test partner(US)] New vulnerability detected",
        description:"Notification type: vuln

                    New vulnerability was detected in your system.

                    ID: 
                    Title: Test
                    Domain: example.com
                    Path: 
                    Method: 
                    Discovered by: 
                    Parameter: 
                    Type: Info
                    Threat: Medium

                    More details: https://us1.my.wallarm.com/object/555


                    Client: TestCompany
                    Cloud: US
                    ",
        details:{
            client_name:"TestCompany",
            cloud:"US",
            notification_type:"vuln",
            vuln_link:"https://us1.my.wallarm.com/object/555",
            vuln:{
                domain:"example.com",
                id:null,
                method:null,
                parameter:null,
                path:null,
                title:"Test",
                discovered_by:null,
                threat:"Medium",
                type:"Info"
            }
        }
    }
]

Test Datadog log:

Diğer kayıtlar arasında Wallarm günlüklerini bulmak için Datadog Logs hizmetinde source:wallarm_cloud arama etiketini kullanabilirsiniz.

Was this page helpful?

How can we improve this article? (Optional)

Information is unclear or confusing

Insufficient information

Outdated or incorrect information

0/240

Send