# Postanalytics Metrics of the Native Node This article describes the Postanalytics module and the service runtime metrics of the Native Node to help monitor and troubleshoot the Native Node. * The [Postanalytics module](https://docs.wallarm.com/admin-en/installation-postanalytics-en.md) uses **wstore** for local traffic processing. Its metrics are prefixed with `wallarm_wstore_*` and reflect the performance of the Postanalytics module. The available metric groups are listed [below](#blocked-requests-metrics). The exact list of metrics may vary depending on the Native Node version. Changes are reflected in the [Native Node changelog](https://docs.wallarm.com/updating-migrating/native-node/node-artifact-versions.md#0200-2025-11-25). * The **wstore** [service runtime metrics](#service-runtime-metrics) cover network activity, request processing, queue states, storage efficiency, and internal engine health. ## Requirements The Postanalytics module and the service runtime metrics of the Native Node are available [starting from version 0.20.0](https://docs.wallarm.com/updating-migrating/native-node/node-artifact-versions.md#0200-2025-11-25) for all deployment options except [Amazon Machine Image (AMI)](https://docs.wallarm.com/installation/native-node/aws-ami.md). ## Metrics endpoint By default, the Native Node provides the metrics at the following endpoint using the `tcp4` (IPv4-only) protocol: ```bash http://localhost:9001/metrics ``` This endpoint is accessible only from the server itself. !!! info "Security note" Unless you need to publicly expose the metrics endpoint (e.g., to run a Prometheus metrics scraper), we recommend using the default `localhost` listen address. You can change the default metrics host and endpoint. Follow the instructions for your deployment type: **For [All-in-one installer](https://docs.wallarm.com/installation/native-node/all-in-one.md) and [Docker image](https://docs.wallarm.com/installation/native-node/docker-image.md):** Set the following environment variables when deploying the Native Node: * `WALLARM_WSTORE__METRICS__LISTEN_ADDRESS` — specifies the host and port * `WALLARM_WSTORE__METRICS__PROTOCOL` — specifies the protocol **For [Helm chart](https://docs.wallarm.com/installation/native-node/helm-chart.md):** Configure the following settings in `values.yaml` [during or after Native Node deployment](https://docs.wallarm.com/installation/native-node/helm-chart-conf.md): * [`config.aggregation.metrics.listenAddress`](https://docs.wallarm.com/installation/native-node/helm-chart-conf.md#configaggregationmetricslistenaddress) — specifies the host and port * [`config.aggregation.metrics.protocol`](https://docs.wallarm.com/installation/native-node/helm-chart-conf.md#configaggregationmetricsprotocol) — specifies the protocol ## Blocked requests metrics --- ### `wallarm_wstore_blocked_stat_ack_missed` The number of acknowledgement attempts for blocked requests that were missed or not processed. **Type:** Counter **Labels:** None **Unit:** Count **Example:** ``` wallarm_wstore_blocked_stat_ack_missed 0 ``` --- ### `wallarm_wstore_blocked_stat_ack_success` The number of blocked requests successfully acknowledged. **Type:** Counter **Labels:** None **Unit:** Count **Example:** ``` wallarm_wstore_blocked_stat_ack_success 0 ``` --- ### `wallarm_wstore_blocked_stat_merged_new` The number of new blocked requests merged into existing blocked request records. **Type:** Counter **Labels:** None **Unit:** Count **Example:** ``` wallarm_wstore_blocked_stat_merged_new 0 ``` --- ### `wallarm_wstore_blocked_stat_merged_updates` The number of updates applied to existing blocked requests when merging new information. **Type:** Counter **Labels:** None **Unit:** Count **Example:** ``` wallarm_wstore_blocked_stat_merged_updates 0 ``` --- ### `wallarm_wstore_blocked_stat_read_items` The total number of blocked request items read from **wstore**. **Type:** Counter **Labels:** None **Unit:** Count **Example:** ``` wallarm_wstore_blocked_stat_read_items 0 ``` --- ### `wallarm_wstore_blocked_stat_register_size` The current size of the blocked requests register. **Type:** Gauge **Labels:** None **Unit:** Count **Example:** ``` wallarm_wstore_blocked_stat_register_size 0 ``` --- ### `wallarm_wstore_blocked_stat_rejected_full` The number of blocked requests rejected due to the blocked requests register being full. **Type:** Counter **Labels:** None **Unit:** Count **Example:** ``` wallarm_wstore_blocked_stat_rejected_full 0 ``` --- ### `wallarm_wstore_blocked_stat_unacked_size` The number of currently unacknowledged blocked requests. **Type:** Gauge **Labels:** None **Unit:** Count **Example:** ``` wallarm_wstore_blocked_stat_unacked_size 0 ``` ## Connections and traffic metrics --- ### `wallarm_wstore_connections_total` The total number of network connections handled by **wstore**, broken down by connection type (i.e., protocol schema like TCP or TLS). **Type**: Counter **Labels**: * `TCP` * `TLS` **Unit**: Count **Example**: ``` wallarm_wstore_connections_total{schema="TCP"} 219 ``` --- ### `wallarm_wstore_current_connections` The number of active connections currently established with **wstore**. **Type**: Gauge **Labels**: None **Unit**: Count **Example**: ``` wallarm_wstore_current_connections 9 ``` --- ### `wallarm_wstore_requests_total` The total number of requests processed, labeled by the request code and the result of the operation (`success` or `failed`). **Type**: Counter **Labels**: * `code` - type of the IPROTO request (e.g., `IPROTO_CALL`, `IPROTO_CALL_16`, `IPROTO_ID`, etc.) * `result` - result of the operation (`success` or `failed`) **Unit**: Count **Example**: ``` wallarm_wstore_requests_total{code="IPROTO_CALL_16",result="success"} 5962210 ``` --- ### `wallarm_wstore_iproto_calls_total` The total number of iproto CALL/CALL_16 requests, broken down by the function name and result. **Type**: Counter **Labels**: * `func` - name of the called function * `result` - result of the operation (`success` or `failed`) **Unit**: Count **Example**: ``` wallarm_wstore_iproto_calls_total{func="wallarm.blocked_stat.read",result="success"} 621473 ``` ## Request queue metrics --- ### `wallarm_wstore_queue_size` The current number of requests in each **wstore** queue. **Type**: Gauge **Labels**: * `engine` - e.g., `ring` * `name` - name of the **wstore** queue **Unit**: Count **Example**: ``` wallarm_wstore_queue_size{engine="ring",name="api_discovery"} 0 ``` --- ### `wallarm_wstore_queue_drops` The number of requests dropped when a **wstore** queue reaches its maximum size and begins overwriting entries in the ring buffer, broken down by queue. **Type**: Counter **Labels**: `queue` - name of the **wstore** queue **Unit**: Count **Example**: ``` wallarm_wstore_queue_drops{queue="appstructure"} 0 ``` --- ### `wallarm_wstore_queue_take_requests` The number of requests returned from the queue by the `wallarm.requests_processing.take` function, labeled by the result of the operation (`success` or `failed`). **Type**: Counter **Labels**: * `queue` - name of the **wstore** queue * `result` - result of the operation (`success` or `failed`) **Unit**: Count **Example**: ``` wallarm_wstore_queue_take_requests{queue="appstructure", result="success"} 313187 ``` --- ### `wallarm_wstore_queue_ack_drops` The number of acknowledgement attempts for requests that have already been removed from the **wstore** queue. **Type**: Counter **Labels**: `queue` - name of the **wstore** queue **Unit**: Count **Example**: ``` wallarm_wstore_queue_ack_drops{queue="appstructure"} 0 ``` --- ### `wallarm_wstore_queue_ack_return` The number of requests that were captured but not acknowledged, and were therefore returned to the queue for reprocessing. **Type**: Counter **Labels**: * `queue` - name of the **wstore** queue * `result` - result of the operation (`success` or `failed`) **Unit**: Count **Example**: ``` wallarm_wstore_queue_ack_return{queue="appstructure",result="failed"} 0 ``` --- ### `wallarm_wstore_queue_stats` The total number of `put`, `ack`, and `take` actions per queue, maintained for backward compatibility. **Type**: Counter **Labels**: none * `queue` - name of the **wstore** queue * `action` - type of the queue operation (`put`, `take`, or `ack`) **Unit**: Count **Example**: ``` wallarm_wstore_queue_stats{queue="appstructure",action="ack"} 770 ``` --- ### `wallarm_wstore_queue_throttled` The number of requests rejected due to queue throttling, broken down by queue. **Type**: Counter **Labels**: `queue` - name of the **wstore** queue **Unit**: Count **Example**: ``` wallarm_wstore_queue_throttled{queue="appstructure"} 0 ``` ## Request storage metrics --- ### `wallarm_wstore_request_storage_total_size` The total size of all stored requests in bytes. **Type**: Gauge **Labels**: None **Unit**: Bytes **Example**: ``` wallarm_wstore_request_storage_total_size 2285568 ``` --- ### `wallarm_wstore_request_storage_timeframe_size` Current time span in seconds between the oldest and newest requests stored in **wstore**. **Type**: Gauge **Labels**: None **Unit**: Seconds **Example**: ``` wallarm_wstore_request_storage_timeframe_size 308775 ``` --- ### `wallarm_wstore_request_storage_drops` The number of old requests dropped to make room for new ones when the maximum request storage size is reached. **Type**: Counter **Labels**: None **Unit**: Count **Example**: ``` wallarm_wstore_request_storage_drops 0 ``` --- ### `wallarm_wstore_request_storage_rejects` The number of incoming requests rejected because they are too large to be stored. **Type**: Counter **Labels**: None **Unit**: Count **Example**: ``` wallarm_wstore_request_storage_rejects 0 ``` ## Internal wstore engine metrics --- !!! info "Internal diagnostics (Support only)" The `wallarm_wstore_kvstore_*` metrics reflect the activity of the **wstore** internal key-value store. They are mainly for Wallarm internal diagnostics. You typically do not need to interpret them. If the Node seems to misbehave, contact [Wallarm Support](https://support.wallarm.com) and share the relevant metrics. | Metric | Type | Labels | Unit | | ------ | ---- | ------ | ---- | | `wallarm_wstore_kvstore_records_total` | Counter | None | Count | | `wallarm_wstore_kvstore_cleanups` | Counter | None | Count | | `wallarm_wstore_kvstore_errors` | Counter | `cleanup`, `drop`, `get_size`, `insert` | Count | | `wallarm_wstore_kvstore_oom_errors_total` | Counter | None | Count | | `wallarm_wstore_kvstore_insertions_total` | Counter | None | Count | | `wallarm_wstore_kvstore_lost_insertions_total` | Counter | None | Count | | `wallarm_wstore_kvstore_drops_total` | Counter | None | Count | Example: ``` wallarm_wstore_kvstore_records_total 770 wallarm_wstore_kvstore_cleanups 0 wallarm_wstore_kvstore_errors{action="cleanup"} 0 wallarm_wstore_kvstore_oom_errors_total 0 wallarm_wstore_kvstore_insertions_total 770 wallarm_wstore_kvstore_lost_insertions_total 0 wallarm_wstore_kvstore_drops_total 0 ``` ## Service runtime metrics We can divide service runtime metrics into 2 groups: * Prometheus Go metrics, prefixed with `go_*` and `process_*` Most Prometheus Go metrics are documented [here](https://demo.promlabs.com/metrics). If a metric is missing there, refer to the [official Go metrics documentation](https://pkg.go.dev/runtime/metrics). * Prometheus metrics, prefixed with `metrics_push_*`, related to its [push gateway](https://github.com/prometheus/pushgateway). See the example of service runtime metrics below. ``` go_sched_latencies_seconds_bucket{le="0"} 0 go_sched_latencies_seconds_bucket{le="2.56e-07"} 5709872 go_sched_latencies_seconds_bucket{le="4.48e-07"} 5852648 go_sched_latencies_seconds_bucket{le="7.68e-07"} 6167836 go_sched_latencies_seconds_bucket{le="1.28e-06"} 6659543 go_sched_latencies_seconds_bucket{le="2.048e-06"} 6712660 go_sched_latencies_seconds_bucket{le="3.584e-06"} 6721120 go_sched_latencies_seconds_bucket{le="6.144e-06"} 6728752 go_sched_latencies_seconds_bucket{le="1.024e-05"} 6770348 go_sched_latencies_seconds_bucket{le="1.6384e-05"} 6864636 go_sched_latencies_seconds_bucket{le="3.2768e-05"} 6945503 go_sched_latencies_seconds_bucket{le="5.7344e-05"} 7023656 go_sched_latencies_seconds_bucket{le="9.8304e-05"} 7058106 go_sched_latencies_seconds_bucket{le="0.00016384"} 7076853 go_sched_latencies_seconds_bucket{le="0.000262144"} 7081681 go_sched_latencies_seconds_bucket{le="0.000458752"} 7083229 go_sched_latencies_seconds_bucket{le="0.000786432"} 7083551 go_sched_latencies_seconds_bucket{le="0.00131072"} 7083749 go_sched_latencies_seconds_bucket{le="0.002097152"} 7083849 go_sched_latencies_seconds_bucket{le="0.003670016"} 7083899 go_sched_latencies_seconds_bucket{le="0.007340032"} 7083926 go_sched_latencies_seconds_bucket{le="0.012582912"} 7083936 go_sched_latencies_seconds_bucket{le="0.02097152"} 7083939 go_sched_latencies_seconds_bucket{le="0.033554432"} 7083940 go_sched_latencies_seconds_bucket{le="0.058720256"} 7083940 go_sched_latencies_seconds_bucket{le="0.100663296"} 7083940 go_sched_latencies_seconds_bucket{le="0.16777216"} 7083940 go_sched_latencies_seconds_bucket{le="0.268435456"} 7083940 go_sched_latencies_seconds_bucket{le="0.469762048"} 7083940 go_sched_latencies_seconds_bucket{le="0.805306368"} 7083940 go_sched_latencies_seconds_bucket{le="+Inf"} 7083940 go_mutex_wait_seconds_total 20.190952976 go_gc_mark_assist_cpu_seconds_total 1.461018963 go_gc_cpu_seconds_total 93.183776342 go_gc_pauses_seconds_bucket{le="0"} 0 go_gc_pauses_seconds_bucket{le="2.56e-07"} 0 go_gc_pauses_seconds_bucket{le="4.48e-07"} 0 go_gc_pauses_seconds_bucket{le="7.68e-07"} 0 go_gc_pauses_seconds_bucket{le="1.28e-06"} 0 go_gc_pauses_seconds_bucket{le="2.048e-06"} 0 go_gc_pauses_seconds_bucket{le="3.584e-06"} 0 go_gc_pauses_seconds_bucket{le="6.144e-06"} 5 go_gc_pauses_seconds_bucket{le="1.024e-05"} 78 go_gc_pauses_seconds_bucket{le="1.6384e-05"} 5704 go_gc_pauses_seconds_bucket{le="3.2768e-05"} 6651 go_gc_pauses_seconds_bucket{le="5.7344e-05"} 10511 go_gc_pauses_seconds_bucket{le="9.8304e-05"} 12303 go_gc_pauses_seconds_bucket{le="0.00016384"} 13390 go_gc_pauses_seconds_bucket{le="0.000262144"} 13841 go_gc_pauses_seconds_bucket{le="0.000458752"} 13974 go_gc_pauses_seconds_bucket{le="0.000786432"} 14006 go_gc_pauses_seconds_bucket{le="0.00131072"} 14020 go_gc_pauses_seconds_bucket{le="0.002097152"} 14025 go_gc_pauses_seconds_bucket{le="0.003670016"} 14031 go_gc_pauses_seconds_bucket{le="0.007340032"} 14040 go_gc_pauses_seconds_bucket{le="0.012582912"} 14041 go_gc_pauses_seconds_bucket{le="0.02097152"} 14044 go_gc_pauses_seconds_bucket{le="0.033554432"} 14044 go_gc_pauses_seconds_bucket{le="0.058720256"} 14044 go_gc_pauses_seconds_bucket{le="0.100663296"} 14044 go_gc_pauses_seconds_bucket{le="0.16777216"} 14044 go_gc_pauses_seconds_bucket{le="0.268435456"} 14044 go_gc_pauses_seconds_bucket{le="0.469762048"} 14044 go_gc_pauses_seconds_bucket{le="0.805306368"} 14044 go_gc_pauses_seconds_bucket{le="+Inf"} 14044 go_scavenge_cpu_seconds_total 0.770850166 go_memlimit_bytes 268435474 go_memstats_alloc_bytes 6909704 go_memstats_alloc_bytes_total 26189017904 go_memstats_buck_hash_sys_bytes 1687394 go_memstats_frees_total 331930366 go_memstats_gc_cpu_fraction 2.9248506362442843e-05 go_memstats_gc_sys_bytes 3411728 go_memstats_heap_alloc_bytes 6909704 go_memstats_heap_idle_bytes 7069696 go_memstats_heap_inuse_bytes 8888320 go_memstats_heap_objects 155526 go_memstats_heap_released_bytes 6832128 go_memstats_heap_sys_bytes 15958016 go_memstats_last_gc_time_seconds 1.7645698917087147e+09 go_memstats_lookups_total 0 go_memstats_mallocs_total 332085892 go_memstats_mcache_inuse_bytes 4832 go_memstats_mcache_sys_bytes 15704 go_memstats_mspan_inuse_bytes 202240 go_memstats_mspan_sys_bytes 261120 go_memstats_next_gc_bytes 11218594 go_memstats_other_sys_bytes 741694 go_memstats_stack_inuse_bytes 819200 go_memstats_stack_sys_bytes 819200 go_memstats_sys_bytes 22894856 go_cgo_calls_count 10960395 go_cpu_count 4 go_gc_duration_seconds{quantile="0"} 5.1574e-05 go_gc_duration_seconds{quantile="0.25"} 6.0257e-05 go_gc_duration_seconds{quantile="0.5"} 7.6857e-05 go_gc_duration_seconds{quantile="0.75"} 0.000138767 go_gc_duration_seconds{quantile="1"} 0.00222732 go_gc_duration_seconds_sum 0.84562571 go_gc_duration_seconds_count 7022 go_gc_forced_count 1726 go_gomaxprocs 4 go_goroutines 29 go_threads 13 go_info{version="go1.25.4"} 1 go_info_ext{compiler="gc", GOARCH="amd64", GOOS="linux", GOROOT="/usr/local/go"} 1 process_cpu_seconds_system_total 1061.14 process_cpu_seconds_total 2303.44 process_cpu_seconds_user_total 1242.3 process_major_pagefaults_total 0 process_minor_pagefaults_total 1544282 process_num_threads 12 process_resident_memory_bytes 26955776 process_start_time_seconds 1764310991 process_virtual_memory_bytes 1273065472 process_virtual_memory_peak_bytes 1273098240 process_resident_memory_peak_bytes 30531584 process_resident_memory_anon_bytes 16076800 process_resident_memory_file_bytes 10878976 process_resident_memory_shared_bytes 0 process_io_read_bytes_total 528866931 process_io_written_bytes_total 274111565 process_io_read_syscalls_total 18250793 process_io_write_syscalls_total 14300084 process_io_storage_read_bytes_total 0 process_io_storage_written_bytes_total 0 metrics_push_block_size_bytes_bucket{url="https://api.wallarm.com:443/v2/node/stat/victoriametrics",vmrange="1.136e+04...1.292e+04"} 2 metrics_push_block_size_bytes_bucket{url="https://api.wallarm.com:443/v2/node/stat/victoriametrics",vmrange="1.292e+04...1.468e+04"} 8628 metrics_push_block_size_bytes_sum{url="https://api.wallarm.com:443/v2/node/stat/victoriametrics"} 112724778 metrics_push_block_size_bytes_count{url="https://api.wallarm.com:443/v2/node/stat/victoriametrics"} 8630 metrics_push_bytes_pushed_total{url="https://api.wallarm.com:443/v2/node/stat/victoriametrics"} 112724778 metrics_push_duration_seconds_bucket{url="https://api.wallarm.com:443/v2/node/stat/victoriametrics",vmrange="1.668e-01...1.896e-01"} 28 metrics_push_duration_seconds_bucket{url="https://api.wallarm.com:443/v2/node/stat/victoriametrics",vmrange="1.896e-01...2.154e-01"} 7561 metrics_push_duration_seconds_bucket{url="https://api.wallarm.com:443/v2/node/stat/victoriametrics",vmrange="2.154e-01...2.448e-01"} 746 metrics_push_duration_seconds_bucket{url="https://api.wallarm.com:443/v2/node/stat/victoriametrics",vmrange="2.448e-01...2.783e-01"} 107 metrics_push_duration_seconds_bucket{url="https://api.wallarm.com:443/v2/node/stat/victoriametrics",vmrange="2.783e-01...3.162e-01"} 46 metrics_push_duration_seconds_bucket{url="https://api.wallarm.com:443/v2/node/stat/victoriametrics",vmrange="3.162e-01...3.594e-01"} 30 metrics_push_duration_seconds_bucket{url="https://api.wallarm.com:443/v2/node/stat/victoriametrics",vmrange="3.594e-01...4.084e-01"} 81 metrics_push_duration_seconds_bucket{url="https://api.wallarm.com:443/v2/node/stat/victoriametrics",vmrange="4.084e-01...4.642e-01"} 10 metrics_push_duration_seconds_bucket{url="https://api.wallarm.com:443/v2/node/stat/victoriametrics",vmrange="4.642e-01...5.275e-01"} 9 metrics_push_duration_seconds_bucket{url="https://api.wallarm.com:443/v2/node/stat/victoriametrics",vmrange="5.275e-01...5.995e-01"} 3 metrics_push_duration_seconds_bucket{url="https://api.wallarm.com:443/v2/node/stat/victoriametrics",vmrange="5.995e-01...6.813e-01"} 3 metrics_push_duration_seconds_bucket{url="https://api.wallarm.com:443/v2/node/stat/victoriametrics",vmrange="6.813e-01...7.743e-01"} 1 metrics_push_duration_seconds_bucket{url="https://api.wallarm.com:443/v2/node/stat/victoriametrics",vmrange="7.743e-01...8.799e-01"} 2 metrics_push_duration_seconds_bucket{url="https://api.wallarm.com:443/v2/node/stat/victoriametrics",vmrange="1.292e+00...1.468e+00"} 2 metrics_push_duration_seconds_bucket{url="https://api.wallarm.com:443/v2/node/stat/victoriametrics",vmrange="1.668e+00...1.896e+00"} 1 metrics_push_duration_seconds_sum{url="https://api.wallarm.com:443/v2/node/stat/victoriametrics"} 1801.3603247859955 metrics_push_duration_seconds_count{url="https://api.wallarm.com:443/v2/node/stat/victoriametrics"} 8630 metrics_push_errors_total{url="https://api.wallarm.com:443/v2/node/stat/victoriametrics"} 0 metrics_push_interval_seconds{url="https://api.wallarm.com:443/v2/node/stat/victoriametrics"} 30 metrics_push_total{url="https://api.wallarm.com:443/v2/node/stat/victoriametrics"} 8630 ```