> ## Documentation Index
> Fetch the complete documentation index at: https://private-7c7dfe99-fix-nav-issues.mintlify.site/llms.txt
> Use this file to discover all available pages before exploring further.

> System table containing metrics that are calculated periodically in the background. For example, the amount of RAM in use.

# system.asynchronous_metrics

<Info>
  **Querying in ClickHouse Cloud**

  The data in this system table is held locally on each node in ClickHouse Cloud. Obtaining a complete view of all data, therefore, requires the `clusterAllReplicas` function. See [here](/reference/system-tables/overview#system-tables-in-clickhouse-cloud) for further details.
</Info>

<h2 id="description">
  Description
</h2>

Contains metrics that are calculated periodically in the background. For example, the amount of RAM in use.

<h2 id="columns">
  Columns
</h2>

* `metric` ([String](/reference/data-types/string)) — Metric name.
* `value` ([Float64](/reference/data-types/float)) — Metric value.
* `description` ([String](/reference/data-types/string) - Metric description)

<h2 id="example">
  Example
</h2>

```sql theme={null}
SELECT * FROM system.asynchronous_metrics LIMIT 10
```

```text theme={null}
┌─metric──────────────────────────────────┬──────value─┬─description────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┐
│ AsynchronousMetricsCalculationTimeSpent │ 0.00179053 │ Time in seconds spent for calculation of asynchronous metrics (this is the overhead of asynchronous metrics).                                                                                                                                              │
│ NumberOfDetachedByUserParts             │          0 │ The total number of parts detached from MergeTree tables by users with the `ALTER TABLE DETACH` query (as opposed to unexpected, broken or ignored parts). The server does not care about detached parts and they can be removed.                          │
│ NumberOfDetachedParts                   │          0 │ The total number of parts detached from MergeTree tables. A part can be detached by a user with the `ALTER TABLE DETACH` query or by the server itself it the part is broken, unexpected or unneeded. The server does not care about detached parts and they can be removed. │
│ TotalRowsOfMergeTreeTables              │    2781309 │ Total amount of rows (records) stored in all tables of MergeTree family.                                                                                                                                                                                   │
│ TotalBytesOfMergeTreeTables             │    7741926 │ Total amount of bytes (compressed, including data and indices) stored in all tables of MergeTree family.                                                                                                                                                   │
│ NumberOfTables                          │         93 │ Total number of tables summed across the databases on the server, excluding the databases that cannot contain MergeTree tables. The excluded database engines are those who generate the set of tables on the fly, like `Lazy`, `MySQL`, `PostgreSQL`, `SQlite`. │
│ NumberOfDatabases                       │          6 │ Total number of databases on the server.                                                                                                                                                                                                                   │
│ MaxPartCountForPartition                │          6 │ Maximum number of parts per partition across all partitions of all tables of MergeTree family. Values larger than 300 indicates misconfiguration, overload, or massive data loading.                                                                       │
│ ReplicasSumMergesInQueue                │          0 │ Sum of merge operations in the queue (still to be applied) across Replicated tables.                                                                                                                                                                       │
│ ReplicasSumInsertsInQueue               │          0 │ Sum of INSERT operations in the queue (still to be replicated) across Replicated tables.                                                                                                                                                                   │
└─────────────────────────────────────────┴────────────┴────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┘
```

<h2 id="metric-descriptions">
  Metric descriptions
</h2>

The descriptions below are generated from the C++ source by `utils/generate-async-metrics-docs`. The single source of truth is the string literal next to each metric registration in `src/Common/AsynchronousMetrics.cpp`, `src/Interpreters/ServerAsynchronousMetrics.cpp`, and `src/Coordination/KeeperAsynchronousMetrics.cpp`. Metric names that include a variable suffix (per-disk, per-CPU, per-interface, ...) are shown with a `*name*` placeholder; the running server reports them with the concrete suffix substituted in.

<h3 id="asynchronousheavymetricscalculationtimespent">
  AsynchronousHeavyMetricsCalculationTimeSpent
</h3>

Time in seconds spent for calculation of asynchronous heavy (tables related) metrics (this is the overhead of asynchronous metrics).

<h3 id="asynchronousheavymetricsupdateinterval">
  AsynchronousHeavyMetricsUpdateInterval
</h3>

Heavy (tables related) metrics update interval

<h3 id="asynchronousmetricscalculationtimespent">
  AsynchronousMetricsCalculationTimeSpent
</h3>

Time in seconds spent for calculation of asynchronous metrics (this is the overhead of asynchronous metrics).

<h3 id="asynchronousmetricsupdateinterval">
  AsynchronousMetricsUpdateInterval
</h3>

Metrics update interval

<h3 id="asyncloggingmetric_firstqueuesize">
  AsyncLogging*metric\_first*QueueSize
</h3>

Number of async messages queued pending for logging in this channel

<h3 id="blockactivetime_name">
  BlockActiveTime\_*name*
</h3>

Time in seconds the block device had the IO requests queued. This is a system-wide metric, it includes all the processes on the host machine, not just clickhouse-server. Source: `/sys/block`. See [https://www.kernel.org/doc/Documentation/block/stat.txt](https://www.kernel.org/doc/Documentation/block/stat.txt)

<h3 id="blockactivetimeperop_name">
  BlockActiveTimePerOp\_*name*
</h3>

Similar to the `BlockActiveTime` metrics, but the value is divided to the number of IO operations to count the per-operation time.

<h3 id="blockdiscardbytes_name">
  BlockDiscardBytes\_*name*
</h3>

Number of discarded bytes on the block device. These operations are relevant for SSD. Discard operations are not used by ClickHouse, but can be used by other processes on the system. This is a system-wide metric, it includes all the processes on the host machine, not just clickhouse-server. Source: `/sys/block`. See [https://www.kernel.org/doc/Documentation/block/stat.txt](https://www.kernel.org/doc/Documentation/block/stat.txt)

<h3 id="blockdiscardmerges_name">
  BlockDiscardMerges\_*name*
</h3>

Number of discard operations requested from the block device and merged together by the OS IO scheduler. These operations are relevant for SSD. Discard operations are not used by ClickHouse, but can be used by other processes on the system. This is a system-wide metric, it includes all the processes on the host machine, not just clickhouse-server. Source: `/sys/block`. See [https://www.kernel.org/doc/Documentation/block/stat.txt](https://www.kernel.org/doc/Documentation/block/stat.txt)

<h3 id="blockdiscardops_name">
  BlockDiscardOps\_*name*
</h3>

Number of discard operations requested from the block device. These operations are relevant for SSD. Discard operations are not used by ClickHouse, but can be used by other processes on the system. This is a system-wide metric, it includes all the processes on the host machine, not just clickhouse-server. Source: `/sys/block`. See [https://www.kernel.org/doc/Documentation/block/stat.txt](https://www.kernel.org/doc/Documentation/block/stat.txt)

<h3 id="blockdiscardtime_name">
  BlockDiscardTime\_*name*
</h3>

Time in seconds spend in discard operations requested from the block device, summed across all the operations. These operations are relevant for SSD. Discard operations are not used by ClickHouse, but can be used by other processes on the system. This is a system-wide metric, it includes all the processes on the host machine, not just clickhouse-server. Source: `/sys/block`. See [https://www.kernel.org/doc/Documentation/block/stat.txt](https://www.kernel.org/doc/Documentation/block/stat.txt)

<h3 id="blockinflightops_name">
  BlockInFlightOps\_*name*
</h3>

This value counts the number of I/O requests that have been issued to the device driver but have not yet completed. It does not include IO requests that are in the queue but not yet issued to the device driver. This is a system-wide metric, it includes all the processes on the host machine, not just clickhouse-server. Source: `/sys/block`. See [https://www.kernel.org/doc/Documentation/block/stat.txt](https://www.kernel.org/doc/Documentation/block/stat.txt)

<h3 id="blockqueuetime_name">
  BlockQueueTime\_*name*
</h3>

This value counts the number of milliseconds that IO requests have waited on this block device. If there are multiple IO requests waiting, this value will increase as the product of the number of milliseconds times the number of requests waiting. This is a system-wide metric, it includes all the processes on the host machine, not just clickhouse-server. Source: `/sys/block`. See [https://www.kernel.org/doc/Documentation/block/stat.txt](https://www.kernel.org/doc/Documentation/block/stat.txt)

<h3 id="blockqueuetimeperop_name">
  BlockQueueTimePerOp\_*name*
</h3>

Similar to the `BlockQueueTime` metrics, but the value is divided to the number of IO operations to count the per-operation time.

<h3 id="blockreadbytes_name">
  BlockReadBytes\_*name*
</h3>

Number of bytes read from the block device. It can be lower than the number of bytes read from the filesystem due to the usage of the OS page cache, that saves IO. This is a system-wide metric, it includes all the processes on the host machine, not just clickhouse-server. Source: `/sys/block`. See [https://www.kernel.org/doc/Documentation/block/stat.txt](https://www.kernel.org/doc/Documentation/block/stat.txt)

<h3 id="blockreadmerges_name">
  BlockReadMerges\_*name*
</h3>

Number of read operations requested from the block device and merged together by the OS IO scheduler. This is a system-wide metric, it includes all the processes on the host machine, not just clickhouse-server. Source: `/sys/block`. See [https://www.kernel.org/doc/Documentation/block/stat.txt](https://www.kernel.org/doc/Documentation/block/stat.txt)

<h3 id="blockreadops_name">
  BlockReadOps\_*name*
</h3>

Number of read operations requested from the block device. This is a system-wide metric, it includes all the processes on the host machine, not just clickhouse-server. Source: `/sys/block`. See [https://www.kernel.org/doc/Documentation/block/stat.txt](https://www.kernel.org/doc/Documentation/block/stat.txt)

<h3 id="blockreadtime_name">
  BlockReadTime\_*name*
</h3>

Time in seconds spend in read operations requested from the block device, summed across all the operations. This is a system-wide metric, it includes all the processes on the host machine, not just clickhouse-server. Source: `/sys/block`. See [https://www.kernel.org/doc/Documentation/block/stat.txt](https://www.kernel.org/doc/Documentation/block/stat.txt)

<h3 id="blockwritebytes_name">
  BlockWriteBytes\_*name*
</h3>

Number of bytes written to the block device. It can be lower than the number of bytes written to the filesystem due to the usage of the OS page cache, that saves IO. A write to the block device may happen later than the corresponding write to the filesystem due to write-through caching. This is a system-wide metric, it includes all the processes on the host machine, not just clickhouse-server. Source: `/sys/block`. See [https://www.kernel.org/doc/Documentation/block/stat.txt](https://www.kernel.org/doc/Documentation/block/stat.txt)

<h3 id="blockwritemerges_name">
  BlockWriteMerges\_*name*
</h3>

Number of write operations requested from the block device and merged together by the OS IO scheduler. This is a system-wide metric, it includes all the processes on the host machine, not just clickhouse-server. Source: `/sys/block`. See [https://www.kernel.org/doc/Documentation/block/stat.txt](https://www.kernel.org/doc/Documentation/block/stat.txt)

<h3 id="blockwriteops_name">
  BlockWriteOps\_*name*
</h3>

Number of write operations requested from the block device. This is a system-wide metric, it includes all the processes on the host machine, not just clickhouse-server. Source: `/sys/block`. See [https://www.kernel.org/doc/Documentation/block/stat.txt](https://www.kernel.org/doc/Documentation/block/stat.txt)

<h3 id="blockwritetime_name">
  BlockWriteTime\_*name*
</h3>

Time in seconds spend in write operations requested from the block device, summed across all the operations. This is a system-wide metric, it includes all the processes on the host machine, not just clickhouse-server. Source: `/sys/block`. See [https://www.kernel.org/doc/Documentation/block/stat.txt](https://www.kernel.org/doc/Documentation/block/stat.txt)

<h3 id="cgroupmaxcpu">
  CGroupMaxCPU
</h3>

The maximum number of CPU cores according to CGroups.

<h3 id="cgroupmemorytotal">
  CGroupMemoryTotal
</h3>

The total amount of memory in cgroup, in bytes. If stated zero, the limit is the same as OSMemoryTotal.

<h3 id="cgroupmemoryused">
  CGroupMemoryUsed
</h3>

The amount of memory used in cgroup, in bytes. On cgroup v2 this is anon + sock + non-reclaimable kernel memory; on cgroup v1 this is RSS. In both cases the kernel OS page cache (file-backed cache) is excluded.

<h3 id="cgroupmemoryusedwithoutpagecache">
  CGroupMemoryUsedWithoutPageCache
</h3>

The amount of memory used in cgroup, in bytes, excluding the ClickHouse userspace page cache. This is CGroupMemoryUsed minus the userspace page cache size. When userspace page cache is disabled, this value equals CGroupMemoryUsed.

<h3 id="cgroupsystemtime">
  CGroupSystemTime
</h3>

The ratio of time the CPU core was running OS kernel (system) code.

<h3 id="cgroupsystemtimenormalized">
  CGroupSystemTimeNormalized
</h3>

The value is similar to `CGroupSystemTime` but divided by the number of available CPU cores to be measured in the \[0..1] interval regardless of the number of cores. This allows you to average the values of this metric across multiple servers in a cluster even if the number of cores is non-uniform, and still get the average resource utilization metric. If specified, the Cgroup CPU quota divided by its period can be used instead of the actual number of CPU cores, and in that case the value of this metric may exceed 1 at some moments.

<h3 id="cgroupusertime">
  CGroupUserTime
</h3>

The ratio of time the CPU core was running userspace code. This includes also the time when the CPU was under-utilized due to the reasons internal to the CPU (memory loads, pipeline stalls, branch mispredictions, running another SMT core).

<h3 id="cgroupusertimenormalized">
  CGroupUserTimeNormalized
</h3>

The value is similar to `CGroupUserTime` but divided by the number of available CPU cores to be measured in the \[0..1] interval regardless of the number of cores. This allows you to average the values of this metric across multiple servers in a cluster even if the number of cores is non-uniform, and still get the average resource utilization metric. If specified, the Cgroup CPU quota divided by its period can be used instead of the actual number of CPU cores, and in that case the value of this metric may exceed 1 at some moments.

<h3 id="cpufrequencymhz_core_id">
  CPUFrequencyMHz\_*core\_id*
</h3>

The current frequency of the CPU, in MHz. Most of the modern CPUs adjust the frequency dynamically for power saving and Turbo Boosting.

<h3 id="dictionarymaxupdatedelay">
  DictionaryMaxUpdateDelay
</h3>

The maximum delay (in seconds) of dictionary update

<h3 id="dictionarytotalfailedupdates">
  DictionaryTotalFailedUpdates
</h3>

Number of errors since last successful loading in all dictionaries.

<h3 id="diskavailable_name">
  DiskAvailable\_*name*
</h3>

Available bytes on the disk (virtual filesystem). Remote filesystems may not provide this information and can show a large value like 16 EiB.

<h3 id="diskgetobjectthrottleravailable_name">
  DiskGetObjectThrottlerAvailable\_*name*
</h3>

Number of GetObject requests that can be currently issued without hitting throttling limit on the disk (virtual filesystem). Local filesystems may not provide this information.

<h3 id="diskgetobjectthrottlerrps_name">
  DiskGetObjectThrottlerRPS\_*name*
</h3>

GetObject Request throttling limit on the disk in requests per second (virtual filesystem). Local filesystems may not provide this information.

<h3 id="diskputobjectthrottleravailable_name">
  DiskPutObjectThrottlerAvailable\_*name*
</h3>

Number of PutObject requests that can be currently issued without hitting throttling limit on the disk (virtual filesystem). Local filesystems may not provide this information.

<h3 id="diskputobjectthrottlerrps_name">
  DiskPutObjectThrottlerRPS\_*name*
</h3>

PutObject Request throttling limit on the disk in requests per second (virtual filesystem). Local filesystems may not provide this information.

<h3 id="disktotal_name">
  DiskTotal\_*name*
</h3>

The total size in bytes of the disk (virtual filesystem). Remote filesystems may not provide this information and can show a large value like 16 EiB.

<h3 id="diskunreserved_name">
  DiskUnreserved\_*name*
</h3>

Available bytes on the disk (virtual filesystem) without the reservations for merges, fetches, and moves. Remote filesystems may not provide this information and can show a large value like 16 EiB.

<h3 id="diskused_name">
  DiskUsed\_*name*
</h3>

Used bytes on the disk (virtual filesystem). Remote filesystems do not always provide this information.

<h3 id="edaci_correctable">
  EDAC*i*\_Correctable
</h3>

The number of correctable ECC memory errors. A high number of this value indicates bad RAM which has to be immediately replaced, because in presence of a high number of corrected errors, a number of silent errors may happen as well, leading to data corruption. Source: `/sys/devices/system/edac/mc/`

<h3 id="edaci_uncorrectable">
  EDAC*i*\_Uncorrectable
</h3>

The number of uncorrectable ECC memory errors. A non-zero number of this value indicates bad RAM which has to be immediately replaced, because it indicates potential data corruption. Source: `/sys/devices/system/edac/mc/`

<h3 id="filesystemcachebytes">
  FilesystemCacheBytes
</h3>

Total bytes in the `cache` virtual filesystem. This cache is hold on disk.

<h3 id="filesystemcachecapacity">
  FilesystemCacheCapacity
</h3>

Total capacity in the `cache` virtual filesystem. This cache is hold on disk.

<h3 id="filesystemcachefiles">
  FilesystemCacheFiles
</h3>

Total number of cached file segments in the `cache` virtual filesystem. This cache is hold on disk.

<h3 id="filesystemlogspathavailablebytes">
  FilesystemLogsPathAvailableBytes
</h3>

Available bytes on the volume where ClickHouse logs path is mounted. If this value approaches zero, you should tune the log rotation in the configuration file.

<h3 id="filesystemlogspathavailableinodes">
  FilesystemLogsPathAvailableINodes
</h3>

The number of available inodes on the volume where ClickHouse logs path is mounted.

<h3 id="filesystemlogspathtotalbytes">
  FilesystemLogsPathTotalBytes
</h3>

The size of the volume where ClickHouse logs path is mounted, in bytes. It's recommended to have at least 10 GB for logs.

<h3 id="filesystemlogspathtotalinodes">
  FilesystemLogsPathTotalINodes
</h3>

The total number of inodes on the volume where ClickHouse logs path is mounted.

<h3 id="filesystemlogspathusedbytes">
  FilesystemLogsPathUsedBytes
</h3>

Used bytes on the volume where ClickHouse logs path is mounted.

<h3 id="filesystemlogspathusedinodes">
  FilesystemLogsPathUsedINodes
</h3>

The number of used inodes on the volume where ClickHouse logs path is mounted.

<h3 id="filesystemmainpathavailablebytes">
  FilesystemMainPathAvailableBytes
</h3>

Available bytes on the volume where the main ClickHouse path is mounted.

<h3 id="filesystemmainpathavailableinodes">
  FilesystemMainPathAvailableINodes
</h3>

The number of available inodes on the volume where the main ClickHouse path is mounted. If it is close to zero, it indicates a misconfiguration, and you will get 'no space left on device' even when the disk is not full.

<h3 id="filesystemmainpathtotalbytes">
  FilesystemMainPathTotalBytes
</h3>

The size of the volume where the main ClickHouse path is mounted, in bytes.

<h3 id="filesystemmainpathtotalinodes">
  FilesystemMainPathTotalINodes
</h3>

The total number of inodes on the volume where the main ClickHouse path is mounted. If it is less than 25 million, it indicates a misconfiguration.

<h3 id="filesystemmainpathusedbytes">
  FilesystemMainPathUsedBytes
</h3>

Used bytes on the volume where the main ClickHouse path is mounted.

<h3 id="filesystemmainpathusedinodes">
  FilesystemMainPathUsedINodes
</h3>

The number of used inodes on the volume where the main ClickHouse path is mounted. This value mostly corresponds to the number of files.

<h3 id="grpcrejectedconnections">
  GRPCRejectedConnections
</h3>

Number of rejected connections for the GRPC protocol.

<h3 id="grpcthreads">
  GRPCThreads
</h3>

Number of threads in the server of the GRPC protocol.

<h3 id="hashtablestatscacheentries">
  HashTableStatsCacheEntries
</h3>

The number of entries in the cache of hash table sizes. The cache for hash table sizes is used for predictive optimization of GROUP BY.

<h3 id="hashtablestatscachehits">
  HashTableStatsCacheHits
</h3>

The number of times the prediction of a hash table size was correct.

<h3 id="hashtablestatscachemisses">
  HashTableStatsCacheMisses
</h3>

The number of times the prediction of a hash table size was incorrect.

<h3 id="httpconnectionpoolgroup_nametcprcvbuftotalbytes">
  HTTPConnectionPool*group\_name*TCPRcvBufTotalBytes
</h3>

Total kernel TCP receive buffer memory (sk\_rmem\_alloc) across all HTTP connection pool sockets.

<h3 id="httpconnectionpoolgroup_nametcpsndbuftotalbytes">
  HTTPConnectionPool*group\_name*TCPSndBufTotalBytes
</h3>

Total kernel TCP transmit buffer memory (sk\_wmem\_alloc) across all HTTP connection pool sockets.

<h3 id="httprejectedconnections">
  HTTPRejectedConnections
</h3>

Number of rejected connections for the HTTP interface (without TLS).

<h3 id="httpsecurerejectedconnections">
  HTTPSecureRejectedConnections
</h3>

Number of rejected connections for the HTTPS interface.

<h3 id="httpsecurethreads">
  HTTPSecureThreads
</h3>

Number of threads in the server of the HTTPS interface.

<h3 id="httpthreads">
  HTTPThreads
</h3>

Number of threads in the server of the HTTP interface (without TLS).

<h3 id="interserverrejectedconnections">
  InterserverRejectedConnections
</h3>

Number of rejected connections for the replicas communication protocol (without TLS).

<h3 id="interserversecurerejectedconnections">
  InterserverSecureRejectedConnections
</h3>

Number of rejected connections for the replicas communication protocol (with TLS).

<h3 id="interserversecurethreads">
  InterserverSecureThreads
</h3>

Number of threads in the server of the replicas communication protocol (with TLS).

<h3 id="interserverthreads">
  InterserverThreads
</h3>

Number of threads in the server of the replicas communication protocol (without TLS).

<h3 id="jemallocactive">
  jemalloc.active
</h3>

An internal metric of the low-level memory allocator (jemalloc). See [https://jemalloc.net/jemalloc.3.html](https://jemalloc.net/jemalloc.3.html)

<h3 id="jemallocallocated">
  jemalloc.allocated
</h3>

An internal metric of the low-level memory allocator (jemalloc). See [https://jemalloc.net/jemalloc.3.html](https://jemalloc.net/jemalloc.3.html)

<h3 id="jemallocarenasalldirty_purged">
  jemalloc.arenas.all.dirty\_purged
</h3>

An internal metric of the low-level memory allocator (jemalloc). See [https://jemalloc.net/jemalloc.3.html](https://jemalloc.net/jemalloc.3.html)

<h3 id="jemallocarenasallmuzzy_purged">
  jemalloc.arenas.all.muzzy\_purged
</h3>

An internal metric of the low-level memory allocator (jemalloc). See [https://jemalloc.net/jemalloc.3.html](https://jemalloc.net/jemalloc.3.html)

<h3 id="jemallocarenasallpactive">
  jemalloc.arenas.all.pactive
</h3>

An internal metric of the low-level memory allocator (jemalloc). See [https://jemalloc.net/jemalloc.3.html](https://jemalloc.net/jemalloc.3.html)

<h3 id="jemallocarenasallpdirty">
  jemalloc.arenas.all.pdirty
</h3>

An internal metric of the low-level memory allocator (jemalloc). See [https://jemalloc.net/jemalloc.3.html](https://jemalloc.net/jemalloc.3.html)

<h3 id="jemallocarenasallpmuzzy">
  jemalloc.arenas.all.pmuzzy
</h3>

An internal metric of the low-level memory allocator (jemalloc). See [https://jemalloc.net/jemalloc.3.html](https://jemalloc.net/jemalloc.3.html)

<h3 id="jemallocarenasdirty_decay_ms">
  jemalloc.arenas.dirty\_decay\_ms
</h3>

An internal metric of the low-level memory allocator (jemalloc). See [https://jemalloc.net/jemalloc.3.html](https://jemalloc.net/jemalloc.3.html)

<h3 id="jemallocbackground_threadnum_runs">
  jemalloc.background\_thread.num\_runs
</h3>

An internal metric of the low-level memory allocator (jemalloc). See [https://jemalloc.net/jemalloc.3.html](https://jemalloc.net/jemalloc.3.html)

<h3 id="jemallocbackground_threadnum_threads">
  jemalloc.background\_thread.num\_threads
</h3>

An internal metric of the low-level memory allocator (jemalloc). See [https://jemalloc.net/jemalloc.3.html](https://jemalloc.net/jemalloc.3.html)

<h3 id="jemallocbackground_threadrun_intervals">
  jemalloc.background\_thread.run\_intervals
</h3>

An internal metric of the low-level memory allocator (jemalloc). See [https://jemalloc.net/jemalloc.3.html](https://jemalloc.net/jemalloc.3.html)

<h3 id="jemalloccache_arenapactive">
  jemalloc.cache\_arena.pactive
</h3>

An internal metric of the low-level memory allocator (jemalloc). See [https://jemalloc.net/jemalloc.3.html](https://jemalloc.net/jemalloc.3.html)

<h3 id="jemalloccache_arenapdirty">
  jemalloc.cache\_arena.pdirty
</h3>

An internal metric of the low-level memory allocator (jemalloc). See [https://jemalloc.net/jemalloc.3.html](https://jemalloc.net/jemalloc.3.html)

<h3 id="jemallocepoch">
  jemalloc.epoch
</h3>

An internal incremental update number of the statistics of jemalloc (Jason Evans' memory allocator), used in all other `jemalloc` metrics.

<h3 id="jemallocmapped">
  jemalloc.mapped
</h3>

An internal metric of the low-level memory allocator (jemalloc). See [https://jemalloc.net/jemalloc.3.html](https://jemalloc.net/jemalloc.3.html)

<h3 id="jemallocmergetree_arenaactive_bytes">
  jemalloc.mergetree\_arena.active\_bytes
</h3>

Active bytes in the dedicated jemalloc MergeTree arena. Holds long-lived MergeTree heap state: per-part metadata (`NamesAndTypesList`, `SerializationInfoByName`, the `serializations` map, `column_name_to_position`, `MergeTreeDataPartChecksums` tree, the `Poco::LRUCache<String, ColumnSize>` delegates inside each `IMergeTreeDataPart`, the per-part `ColumnSize`/`IndexSize` maps, `MinMaxIndex`, `VersionMetadataOnDisk`, and the `MergeTreeDataPart{Compact,Wide}` object itself) plus per-table metadata (`StorageInMemoryMetadata` / `ColumnsDescription` / `VirtualColumnsDescription` clones set up by `setProperties`, the `serialization_hints` aggregation, and the `columns_descriptions_cache`). Active parts and outdated parts pending cleanup both contribute. Disjoint from the cache arena and JIT arena. The per-part columns `system.parts.primary_key_bytes_in_memory[_allocated]` and `system.parts.index_granularity_bytes_in_memory[_allocated]` are subsets of this metric (when their values are non-zero — they can also live in `PrimaryIndexCacheBytes` instead, which is in the cache arena and not counted here).

<h3 id="jemallocmergetree_arenadirty_bytes">
  jemalloc.mergetree\_arena.dirty\_bytes
</h3>

Dirty bytes in the MergeTree arena that are eligible for purging back to the OS.

<h3 id="jemallocmergetree_arenapactive">
  jemalloc.mergetree\_arena.pactive
</h3>

An internal metric of the low-level memory allocator (jemalloc). See [https://jemalloc.net/jemalloc.3.html](https://jemalloc.net/jemalloc.3.html)

<h3 id="jemallocmergetree_arenapdirty">
  jemalloc.mergetree\_arena.pdirty
</h3>

An internal metric of the low-level memory allocator (jemalloc). See [https://jemalloc.net/jemalloc.3.html](https://jemalloc.net/jemalloc.3.html)

<h3 id="jemallocmetadata">
  jemalloc.metadata
</h3>

An internal metric of the low-level memory allocator (jemalloc). See [https://jemalloc.net/jemalloc.3.html](https://jemalloc.net/jemalloc.3.html)

<h3 id="jemallocmetadata_thp">
  jemalloc.metadata\_thp
</h3>

An internal metric of the low-level memory allocator (jemalloc). See [https://jemalloc.net/jemalloc.3.html](https://jemalloc.net/jemalloc.3.html)

<h3 id="jemallocprofactive">
  jemalloc.prof.active
</h3>

An internal metric of the low-level memory allocator (jemalloc). See [https://jemalloc.net/jemalloc.3.html](https://jemalloc.net/jemalloc.3.html)

<h3 id="jemallocproflg_sample">
  jemalloc.prof.lg\_sample
</h3>

An internal metric of the low-level memory allocator (jemalloc). See [https://jemalloc.net/jemalloc.3.html](https://jemalloc.net/jemalloc.3.html)

<h3 id="jemallocprofthread_active_init">
  jemalloc.prof.thread\_active\_init
</h3>

An internal metric of the low-level memory allocator (jemalloc). See [https://jemalloc.net/jemalloc.3.html](https://jemalloc.net/jemalloc.3.html)

<h3 id="jemallocresident">
  jemalloc.resident
</h3>

An internal metric of the low-level memory allocator (jemalloc). See [https://jemalloc.net/jemalloc.3.html](https://jemalloc.net/jemalloc.3.html)

<h3 id="jemallocretained">
  jemalloc.retained
</h3>

An internal metric of the low-level memory allocator (jemalloc). See [https://jemalloc.net/jemalloc.3.html](https://jemalloc.net/jemalloc.3.html)

<h3 id="jitter">
  Jitter
</h3>

The difference in time the thread for calculation of the asynchronous metrics was scheduled to wake up and the time it was in fact, woken up. A proxy-indicator of overall system latency and responsiveness.

<h3 id="keeperapproximatedatasize">
  KeeperApproximateDataSize
</h3>

The approximate data size of ClickHouse Keeper, in bytes.

<h3 id="keeperavglatency">
  KeeperAvgLatency
</h3>

Average request latency of ClickHouse Keeper.

<h3 id="keepercommitlogscacheentries">
  KeeperCommitLogsCacheEntries
</h3>

Number of entries stored in the in-memory cache for next logs to be committed

<h3 id="keepercommitlogscachesize">
  KeeperCommitLogsCacheSize
</h3>

Total size of in-memory cache for next logs to be committed

<h3 id="keeperephemeralscount">
  KeeperEphemeralsCount
</h3>

The number of ephemeral nodes in ClickHouse Keeper.

<h3 id="keeperfollowers">
  KeeperFollowers
</h3>

The number of followers of ClickHouse Keeper.

<h3 id="keeperisexceedingmemorysoftlimithit">
  KeeperIsExceedingMemorySoftLimitHit
</h3>

1 if ClickHouse Keeper is exceeding the memory soft limit, 0 otherwise.

<h3 id="keeperisfollower">
  KeeperIsFollower
</h3>

1 if ClickHouse Keeper is a follower, 0 otherwise.

<h3 id="keeperisleader">
  KeeperIsLeader
</h3>

1 if ClickHouse Keeper is a leader, 0 otherwise.

<h3 id="keeperisobserver">
  KeeperIsObserver
</h3>

1 if ClickHouse Keeper is an observer, 0 otherwise.

<h3 id="keeperisstandalone">
  KeeperIsStandalone
</h3>

1 if ClickHouse Keeper is in a standalone mode, 0 otherwise.

<h3 id="keeperkeyarenasize">
  KeeperKeyArenaSize
</h3>

The size in bytes of the memory arena for keys in ClickHouse Keeper.

<h3 id="keeperlastcommittedlogidx">
  KeeperLastCommittedLogIdx
</h3>

Index of the last committed log in ClickHouse Keeper.

<h3 id="keeperlastlogidx">
  KeeperLastLogIdx
</h3>

Index of the last log stored in ClickHouse Keeper.

<h3 id="keeperlastlogterm">
  KeeperLastLogTerm
</h3>

Raft term of the last log stored in ClickHouse Keeper.

<h3 id="keeperlastsnapshotidx">
  KeeperLastSnapshotIdx
</h3>

Index of the last log present in the last created snapshot.

<h3 id="keeperlatestlogscacheentries">
  KeeperLatestLogsCacheEntries
</h3>

Number of entries stored in the in-memory cache for latest logs

<h3 id="keeperlatestlogscachesize">
  KeeperLatestLogsCacheSize
</h3>

Total size of in-memory cache for latest logs

<h3 id="keeperlatestsnapshotsize">
  KeeperLatestSnapshotSize
</h3>

The uncompressed size in bytes of the latest snapshot created by ClickHouse Keeper.

<h3 id="keepermaxfiledescriptorcount">
  KeeperMaxFileDescriptorCount
</h3>

The maximum number of open file descriptors in ClickHouse Keeper.

<h3 id="keepermaxlatency">
  KeeperMaxLatency
</h3>

Maximum request latency of ClickHouse Keeper.

<h3 id="keeperminlatency">
  KeeperMinLatency
</h3>

Minimal request latency of ClickHouse Keeper.

<h3 id="keeperopenfiledescriptorcount">
  KeeperOpenFileDescriptorCount
</h3>

The number of open file descriptors in ClickHouse Keeper.

<h3 id="keeperpacketsreceived">
  KeeperPacketsReceived
</h3>

Number of packets received by ClickHouse Keeper.

<h3 id="keeperpacketssent">
  KeeperPacketsSent
</h3>

Number of packets sent by ClickHouse Keeper.

<h3 id="keeperpathswatched">
  KeeperPathsWatched
</h3>

The number of different paths watched by the clients of ClickHouse Keeper.

<h3 id="keepersessionwithwatches">
  KeeperSessionWithWatches
</h3>

The number of client sessions of ClickHouse Keeper having watches.

<h3 id="keepersyncedfollowers">
  KeeperSyncedFollowers
</h3>

The number of followers of ClickHouse Keeper who are also in-sync.

<h3 id="keepertargetcommitlogidx">
  KeeperTargetCommitLogIdx
</h3>

Index until which logs can be committed in ClickHouse Keeper.

<h3 id="keepertcprejectedconnections">
  KeeperTCPRejectedConnections
</h3>

Number of rejected connections for the Keeper TCP protocol (without TLS).

<h3 id="keepertcpsecurerejectedconnections">
  KeeperTCPSecureRejectedConnections
</h3>

Number of rejected connections for the Keeper TCP protocol (with TLS).

<h3 id="keepertcpsecurethreads">
  KeeperTCPSecureThreads
</h3>

Number of threads in the server of the Keeper TCP protocol (with TLS).

<h3 id="keepertcpthreads">
  KeeperTCPThreads
</h3>

Number of threads in the server of the Keeper TCP protocol (without TLS).

<h3 id="keeperwatchcount">
  KeeperWatchCount
</h3>

The number of watches in ClickHouse Keeper.

<h3 id="keeperznodecount">
  KeeperZnodeCount
</h3>

The number of nodes (data entries) in ClickHouse Keeper.

<h3 id="keeperzxid">
  KeeperZxid
</h3>

The current transaction id number (zxid) in ClickHouse Keeper.

<h3 id="loadaverage1">
  LoadAverage1
</h3>

The whole system load, averaged with exponential smoothing over 1 minute. The load represents the number of threads across all the processes (the scheduling entities of the OS kernel), that are currently running by CPU or waiting for IO, or ready to run but not being scheduled at this point of time. This number includes all the processes, not only clickhouse-server. The number can be greater than the number of CPU cores, if the system is overloaded, and many processes are ready to run but waiting for CPU or IO.

<h3 id="loadaverage15">
  LoadAverage15
</h3>

The whole system load, averaged with exponential smoothing over 15 minutes. The load represents the number of threads across all the processes (the scheduling entities of the OS kernel), that are currently running by CPU or waiting for IO, or ready to run but not being scheduled at this point of time. This number includes all the processes, not only clickhouse-server. The number can be greater than the number of CPU cores, if the system is overloaded, and many processes are ready to run but waiting for CPU or IO.

<h3 id="loadaverage5">
  LoadAverage5
</h3>

The whole system load, averaged with exponential smoothing over 5 minutes. The load represents the number of threads across all the processes (the scheduling entities of the OS kernel), that are currently running by CPU or waiting for IO, or ready to run but not being scheduled at this point of time. This number includes all the processes, not only clickhouse-server. The number can be greater than the number of CPU cores, if the system is overloaded, and many processes are ready to run but waiting for CPU or IO.

<h3 id="longestrunningmerge">
  LongestRunningMerge
</h3>

Elapsed time in seconds of the longest currently running background merge.

<h3 id="maxpartcountforpartition">
  MaxPartCountForPartition
</h3>

Maximum number of parts per partition across all partitions of all tables of MergeTree family. Values larger than 300 indicates misconfiguration, overload, or massive data loading.

<h3 id="memorycode">
  MemoryCode
</h3>

The amount of virtual memory mapped for the pages of machine code of the server process, in bytes.

<h3 id="memorydataandstack">
  MemoryDataAndStack
</h3>

The amount of virtual memory mapped for the use of stack and for the allocated memory, in bytes. It is unspecified whether it includes the per-thread stacks and most of the allocated memory, that is allocated with the 'mmap' system call. This metric exists only for completeness reasons. I recommend to use the `MemoryResident` metric for monitoring.

<h3 id="memoryresident">
  MemoryResident
</h3>

The amount of physical memory used by the server process, in bytes.

<h3 id="memoryresidentmax">
  MemoryResidentMax
</h3>

Maximum amount of physical memory used by the server process, in bytes.

<h3 id="memoryresidentwithoutpagecache">
  MemoryResidentWithoutPageCache
</h3>

The amount of physical memory used by the server process, excluding userspace page cache, in bytes. This provides a more accurate view of actual memory usage when userspace page cache is utilized. When userspace page cache is disabled, this value equals MemoryResident.

<h3 id="memoryshared">
  MemoryShared
</h3>

The amount of memory used by the server process, that is also shared by another processes, in bytes. ClickHouse does not use shared memory, but some memory can be labeled by OS as shared for its own reasons. This metric does not make a lot of sense to watch, and it exists only for completeness reasons.

<h3 id="memoryvirtual">
  MemoryVirtual
</h3>

The size of the virtual address space allocated by the server process, in bytes. The size of the virtual address space is usually much greater than the physical memory consumption, and should not be used as an estimate for the memory consumption. The large values of this metric are totally normal, and makes only technical sense.

<h3 id="mysqlrejectedconnections">
  MySQLRejectedConnections
</h3>

Number of rejected connections for the MySQL compatibility protocol.

<h3 id="mysqlthreads">
  MySQLThreads
</h3>

Number of threads in the server of the MySQL compatibility protocol.

<h3 id="networkreceivebytes_interface_name">
  NetworkReceiveBytes\_*interface\_name*
</h3>

Number of bytes received via the network interface. This is a system-wide metric, it includes all the processes on the host machine, not just clickhouse-server.

<h3 id="networkreceivedrop_interface_name">
  NetworkReceiveDrop\_*interface\_name*
</h3>

Number of bytes a packet was dropped while received via the network interface. This is a system-wide metric, it includes all the processes on the host machine, not just clickhouse-server.

<h3 id="networkreceiveerrors_interface_name">
  NetworkReceiveErrors\_*interface\_name*
</h3>

Number of times error happened receiving via the network interface. This is a system-wide metric, it includes all the processes on the host machine, not just clickhouse-server.

<h3 id="networkreceivepackets_interface_name">
  NetworkReceivePackets\_*interface\_name*
</h3>

Number of network packets received via the network interface. This is a system-wide metric, it includes all the processes on the host machine, not just clickhouse-server.

<h3 id="networksendbytes_interface_name">
  NetworkSendBytes\_*interface\_name*
</h3>

Number of bytes sent via the network interface. This is a system-wide metric, it includes all the processes on the host machine, not just clickhouse-server.

<h3 id="networksenddrop_interface_name">
  NetworkSendDrop\_*interface\_name*
</h3>

Number of times a packed was dropped while sending via the network interface. This is a system-wide metric, it includes all the processes on the host machine, not just clickhouse-server.

<h3 id="networksenderrors_interface_name">
  NetworkSendErrors\_*interface\_name*
</h3>

Number of times error (e.g. TCP retransmit) happened while sending via the network interface. This is a system-wide metric, it includes all the processes on the host machine, not just clickhouse-server.

<h3 id="networksendpackets_interface_name">
  NetworkSendPackets\_*interface\_name*
</h3>

Number of network packets sent via the network interface. This is a system-wide metric, it includes all the processes on the host machine, not just clickhouse-server.

<h3 id="networktcpreceivequeue">
  NetworkTCPReceiveQueue
</h3>

Total size of receive queues of network sockets used on the server across TCPv4 and TCPv6.

<h3 id="networktcpsocketremoteaddresses">
  NetworkTCPSocketRemoteAddresses
</h3>

Total number of unique remote addresses of network sockets used on the server across TCPv4 and TCPv6.

<h3 id="networktcpsockets">
  NetworkTCPSockets
</h3>

Total number of network sockets used on the server across TCPv4 and TCPv6, in all states.

<h3 id="networktcpsockets_description">
  NetworkTCPSockets\_*description*
</h3>

Total number of network sockets in the specific state on the server across TCPv4 and TCPv6.

<h3 id="networktcptransmitqueue">
  NetworkTCPTransmitQueue
</h3>

Total size of transmit queues of network sockets used on the server across TCPv4 and TCPv6.

<h3 id="networktcpunrecoveredretransmits">
  NetworkTCPUnrecoveredRetransmits
</h3>

Total size of current retransmits (unrecovered at this moment) of network sockets used on the server across TCPv4 and TCPv6.

<h3 id="numberofdatabases">
  NumberOfDatabases
</h3>

Total number of databases on the server.

<h3 id="numberofdetachedbyuserparts">
  NumberOfDetachedByUserParts
</h3>

The total number of parts detached from MergeTree tables by users with the `ALTER TABLE DETACH` query (as opposed to unexpected, broken or ignored parts). The server does not care about detached parts and they can be removed.

<h3 id="numberofdetachedparts">
  NumberOfDetachedParts
</h3>

The total number of parts detached from MergeTree tables. A part can be detached by a user with the `ALTER TABLE DETACH` query or by the server itself it the part is broken, unexpected or unneeded. The server does not care about detached parts and they can be removed.

<h3 id="numberofpendingmutations">
  NumberOfPendingMutations
</h3>

The total number of mutations that are in left to be mutated.

<h3 id="numberofpendingmutationsoverexecutiontime">
  NumberOfPendingMutationsOverExecutionTime
</h3>

The total number of mutations which have data part left to be mutated over the specified max\_pending\_mutations\_execution\_time\_to\_warn setting.

<h3 id="numberoftables">
  NumberOfTables
</h3>

Total number of tables summed across the databases on the server, excluding the databases that cannot contain MergeTree tables. The excluded database engines are those who generate the set of tables on the fly, like `Lazy`, `MySQL`, `PostgreSQL`, `SQlite`.

<h3 id="numberoftablessystem">
  NumberOfTablesSystem
</h3>

Total number of tables in the system database on the server stored in tables of MergeTree family.

<h3 id="oscontextswitches">
  OSContextSwitches
</h3>

The number of context switches that the system underwent on the host machine. This is a system-wide metric, it includes all the processes on the host machine, not just clickhouse-server.

<h3 id="oscpuoverload">
  OSCPUOverload
</h3>

Relative CPU deficit, calculated as: how many threads are waiting for CPU relative to the number of threads, using CPU. If it is greater than zero, the server would benefit from more CPU. If it is significantly greater than zero, the server could become unresponsive. The metric is accumulated between the updates of asynchronous metrics.

<h3 id="osguestnicetimecpu_suffix">
  OSGuestNiceTime*cpu\_suffix*
</h3>

The ratio of time spent running a virtual CPU for guest operating systems under the control of the Linux kernel, when a guest was set to a higher priority (See `man procfs`). This is a system-wide metric, it includes all the processes on the host machine, not just clickhouse-server. This metric is irrelevant for ClickHouse, but still exists for completeness. The value for a single CPU core will be in the interval \[0..1]. The value for all CPU cores is calculated as a sum across them \[0..num cores].

<h3 id="osguestnicetimenormalized">
  OSGuestNiceTimeNormalized
</h3>

The value is similar to `OSGuestNiceTime` but divided to the number of CPU cores to be measured in the \[0..1] interval regardless of the number of cores. This allows you to average the values of this metric across multiple servers in a cluster even if the number of cores is non-uniform, and still get the average resource utilization metric. If specified, the Cgroup CPU quota divided by its period can be used instead of the actual number of CPU cores, and in that case the value of this metric may exceed 1 at some moments.

<h3 id="osguesttimecpu_suffix">
  OSGuestTime*cpu\_suffix*
</h3>

The ratio of time spent running a virtual CPU for guest operating systems under the control of the Linux kernel (See `man procfs`). This is a system-wide metric, it includes all the processes on the host machine, not just clickhouse-server. This metric is irrelevant for ClickHouse, but still exists for completeness. The value for a single CPU core will be in the interval \[0..1]. The value for all CPU cores is calculated as a sum across them \[0..num cores].

<h3 id="osguesttimenormalized">
  OSGuestTimeNormalized
</h3>

The value is similar to `OSGuestTime` but divided to the number of CPU cores to be measured in the \[0..1] interval regardless of the number of cores. This allows you to average the values of this metric across multiple servers in a cluster even if the number of cores is non-uniform, and still get the average resource utilization metric. If specified, the Cgroup CPU quota divided by its period can be used instead of the actual number of CPU cores, and in that case the value of this metric may exceed 1 at some moments.

<h3 id="osidletimecpu_suffix">
  OSIdleTime*cpu\_suffix*
</h3>

The ratio of time the CPU core was idle (not even ready to run a process waiting for IO) from the OS kernel standpoint. This is a system-wide metric, it includes all the processes on the host machine, not just clickhouse-server. This does not include the time when the CPU was under-utilized due to the reasons internal to the CPU (memory loads, pipeline stalls, branch mispredictions, running another SMT core). The value for a single CPU core will be in the interval \[0..1]. The value for all CPU cores is calculated as a sum across them \[0..num cores].

<h3 id="osidletimenormalized">
  OSIdleTimeNormalized
</h3>

The value is similar to `OSIdleTime` but divided to the number of CPU cores to be measured in the \[0..1] interval regardless of the number of cores. This allows you to average the values of this metric across multiple servers in a cluster even if the number of cores is non-uniform, and still get the average resource utilization metric. If specified, the Cgroup CPU quota divided by its period can be used instead of the actual number of CPU cores, and in that case the value of this metric may exceed 1 at some moments.

<h3 id="osinterrupts">
  OSInterrupts
</h3>

The number of interrupts on the host machine. This is a system-wide metric, it includes all the processes on the host machine, not just clickhouse-server.

<h3 id="osiowaittimecpu_suffix">
  OSIOWaitTime*cpu\_suffix*
</h3>

The ratio of time the CPU core was not running the code but when the OS kernel did not run any other process on this CPU as the processes were waiting for IO. This is a system-wide metric, it includes all the processes on the host machine, not just clickhouse-server. The value for a single CPU core will be in the interval \[0..1]. The value for all CPU cores is calculated as a sum across them \[0..num cores].

<h3 id="osiowaittimenormalized">
  OSIOWaitTimeNormalized
</h3>

The value is similar to `OSIOWaitTime` but divided to the number of CPU cores to be measured in the \[0..1] interval regardless of the number of cores. This allows you to average the values of this metric across multiple servers in a cluster even if the number of cores is non-uniform, and still get the average resource utilization metric. If specified, the Cgroup CPU quota divided by its period can be used instead of the actual number of CPU cores, and in that case the value of this metric may exceed 1 at some moments.

<h3 id="osirqtimecpu_suffix">
  OSIrqTime*cpu\_suffix*
</h3>

The ratio of time spent for running hardware interrupt requests on the CPU. This is a system-wide metric, it includes all the processes on the host machine, not just clickhouse-server. A high number of this metric may indicate hardware misconfiguration or a very high network load. The value for a single CPU core will be in the interval \[0..1]. The value for all CPU cores is calculated as a sum across them \[0..num cores].

<h3 id="osirqtimenormalized">
  OSIrqTimeNormalized
</h3>

The value is similar to `OSIrqTime` but divided to the number of CPU cores to be measured in the \[0..1] interval regardless of the number of cores. This allows you to average the values of this metric across multiple servers in a cluster even if the number of cores is non-uniform, and still get the average resource utilization metric. If specified, the Cgroup CPU quota divided by its period can be used instead of the actual number of CPU cores, and in that case the value of this metric may exceed 1 at some moments.

<h3 id="osmemoryavailable">
  OSMemoryAvailable
</h3>

The amount of memory available to be used by programs, in bytes. This is very similar to the `OSMemoryFreePlusCached` metric. This is a system-wide metric, it includes all the processes on the host machine, not just clickhouse-server.

<h3 id="osmemorybuffers">
  OSMemoryBuffers
</h3>

The amount of memory used by OS kernel buffers, in bytes. This should be typically small, and large values may indicate a misconfiguration of the OS. This is a system-wide metric, it includes all the processes on the host machine, not just clickhouse-server.

<h3 id="osmemorycached">
  OSMemoryCached
</h3>

The amount of memory used by the OS page cache, in bytes. Typically, almost all available memory is used by the OS page cache - high values of this metric are normal and expected. This is a system-wide metric, it includes all the processes on the host machine, not just clickhouse-server.

<h3 id="osmemoryfreepluscached">
  OSMemoryFreePlusCached
</h3>

The amount of free memory plus OS page cache memory on the host system, in bytes. This memory is available to be used by programs. The value should be very similar to `OSMemoryAvailable`. This is a system-wide metric, it includes all the processes on the host machine, not just clickhouse-server.

<h3 id="osmemoryfreewithoutcached">
  OSMemoryFreeWithoutCached
</h3>

The amount of free memory on the host system, in bytes. This does not include the memory used by the OS page cache memory, in bytes. The page cache memory is also available for usage by programs, so the value of this metric can be confusing. See the `OSMemoryAvailable` metric instead. For convenience we also provide the `OSMemoryFreePlusCached` metric, that should be somewhat similar to OSMemoryAvailable. See also [https://www.linuxatemyram.com/](https://www.linuxatemyram.com/). This is a system-wide metric, it includes all the processes on the host machine, not just clickhouse-server.

<h3 id="osmemoryswapcached">
  OSMemorySwapCached
</h3>

The amount of memory in swap that was also loaded in RAM. Swap should be disabled on production systems. If the value of this metric is large, it indicates a misconfiguration. This is a system-wide metric, it includes all the processes on the host machine, not just clickhouse-server.

<h3 id="osmemorytotal">
  OSMemoryTotal
</h3>

The total amount of memory on the host system, in bytes.

<h3 id="osnicetimecpu_suffix">
  OSNiceTime*cpu\_suffix*
</h3>

The ratio of time the CPU core was running userspace code with higher priority. This is a system-wide metric, it includes all the processes on the host machine, not just clickhouse-server. The value for a single CPU core will be in the interval \[0..1]. The value for all CPU cores is calculated as a sum across them \[0..num cores].

<h3 id="osnicetimenormalized">
  OSNiceTimeNormalized
</h3>

The value is similar to `OSNiceTime` but divided to the number of CPU cores to be measured in the \[0..1] interval regardless of the number of cores. This allows you to average the values of this metric across multiple servers in a cluster even if the number of cores is non-uniform, and still get the average resource utilization metric. If specified, the Cgroup CPU quota divided by its period can be used instead of the actual number of CPU cores, and in that case the value of this metric may exceed 1 at some moments.

<h3 id="osopenfiles">
  OSOpenFiles
</h3>

The total number of opened files on the host machine. This is a system-wide metric, it includes all the processes on the host machine, not just clickhouse-server.

<h3 id="osprocessesblocked">
  OSProcessesBlocked
</h3>

Number of threads blocked waiting for I/O to complete (`man procfs`). This is a system-wide metric, it includes all the processes on the host machine, not just clickhouse-server.

<h3 id="osprocessescreated">
  OSProcessesCreated
</h3>

The number of processes created. This is a system-wide metric, it includes all the processes on the host machine, not just clickhouse-server.

<h3 id="osprocessesrunning">
  OSProcessesRunning
</h3>

The number of runnable (running or ready to run) threads by the operating system. This is a system-wide metric, it includes all the processes on the host machine, not just clickhouse-server.

<h3 id="ossoftirqtimecpu_suffix">
  OSSoftIrqTime*cpu\_suffix*
</h3>

The ratio of time spent for running software interrupt requests on the CPU. This is a system-wide metric, it includes all the processes on the host machine, not just clickhouse-server. A high number of this metric may indicate inefficient software running on the system. The value for a single CPU core will be in the interval \[0..1]. The value for all CPU cores is calculated as a sum across them \[0..num cores].

<h3 id="ossoftirqtimenormalized">
  OSSoftIrqTimeNormalized
</h3>

The value is similar to `OSSoftIrqTime` but divided to the number of CPU cores to be measured in the \[0..1] interval regardless of the number of cores. This allows you to average the values of this metric across multiple servers in a cluster even if the number of cores is non-uniform, and still get the average resource utilization metric. If specified, the Cgroup CPU quota divided by its period can be used instead of the actual number of CPU cores, and in that case the value of this metric may exceed 1 at some moments.

<h3 id="osstealtimecpu_suffix">
  OSStealTime*cpu\_suffix*
</h3>

The ratio of time spent in other operating systems by the CPU when running in a virtualized environment. This is a system-wide metric, it includes all the processes on the host machine, not just clickhouse-server. Not every virtualized environments present this metric, and most of them don't. The value for a single CPU core will be in the interval \[0..1]. The value for all CPU cores is calculated as a sum across them \[0..num cores].

<h3 id="osstealtimenormalized">
  OSStealTimeNormalized
</h3>

The value is similar to `OSStealTime` but divided to the number of CPU cores to be measured in the \[0..1] interval regardless of the number of cores. This allows you to average the values of this metric across multiple servers in a cluster even if the number of cores is non-uniform, and still get the average resource utilization metric. If specified, the Cgroup CPU quota divided by its period can be used instead of the actual number of CPU cores, and in that case the value of this metric may exceed 1 at some moments.

<h3 id="ossystemtimecpu_suffix">
  OSSystemTime*cpu\_suffix*
</h3>

The ratio of time the CPU core was running OS kernel (system) code. This is a system-wide metric, it includes all the processes on the host machine, not just clickhouse-server. The value for a single CPU core will be in the interval \[0..1]. The value for all CPU cores is calculated as a sum across them \[0..num cores].

<h3 id="ossystemtimenormalized">
  OSSystemTimeNormalized
</h3>

The value is similar to `OSSystemTime` but divided to the number of CPU cores to be measured in the \[0..1] interval regardless of the number of cores. This allows you to average the values of this metric across multiple servers in a cluster even if the number of cores is non-uniform, and still get the average resource utilization metric. If specified, the Cgroup CPU quota divided by its period can be used instead of the actual number of CPU cores, and in that case the value of this metric may exceed 1 at some moments.

<h3 id="osthreadsrunnable">
  OSThreadsRunnable
</h3>

The total number of 'runnable' threads, as the OS kernel scheduler seeing it.

<h3 id="osthreadstotal">
  OSThreadsTotal
</h3>

The total number of threads, as the OS kernel scheduler seeing it.

<h3 id="osuptime">
  OSUptime
</h3>

The uptime of the host server (the machine where ClickHouse is running), in seconds.

<h3 id="osusertimecpu_suffix">
  OSUserTime*cpu\_suffix*
</h3>

The ratio of time the CPU core was running userspace code. This is a system-wide metric, it includes all the processes on the host machine, not just clickhouse-server. This includes also the time when the CPU was under-utilized due to the reasons internal to the CPU (memory loads, pipeline stalls, branch mispredictions, running another SMT core). The value for a single CPU core will be in the interval \[0..1]. The value for all CPU cores is calculated as a sum across them \[0..num cores].

<h3 id="osusertimenormalized">
  OSUserTimeNormalized
</h3>

The value is similar to `OSUserTime` but divided to the number of CPU cores to be measured in the \[0..1] interval regardless of the number of cores. This allows you to average the values of this metric across multiple servers in a cluster even if the number of cores is non-uniform, and still get the average resource utilization metric. If specified, the Cgroup CPU quota divided by its period can be used instead of the actual number of CPU cores, and in that case the value of this metric may exceed 1 at some moments.

<h3 id="pagecachemaxbytes">
  PageCacheMaxBytes
</h3>

Current limit on the size of userspace page cache, in bytes.

<h3 id="postgresqlrejectedconnections">
  PostgreSQLRejectedConnections
</h3>

Number of rejected connections for the PostgreSQL compatibility protocol.

<h3 id="postgresqlthreads">
  PostgreSQLThreads
</h3>

Number of threads in the server of the PostgreSQL compatibility protocol.

<h3 id="processsignalqueuelimit">
  ProcessSignalQueueLimit
</h3>

Total limit of signal queue (once it reaches ProcessSignalQueueSize, you may get CANNOT\_CREATE\_TIMER errors)

<h3 id="processsignalqueuesize">
  ProcessSignalQueueSize
</h3>

Size of signal queue (pending signals, timers for query profiling)

<h3 id="prometheusrejectedconnections">
  PrometheusRejectedConnections
</h3>

Number of rejected connections for the Prometheus endpoint. Note: prometheus endpoints can be also used via the usual HTTP/HTTPs ports.

<h3 id="prometheusthreads">
  PrometheusThreads
</h3>

Number of threads in the server of the Prometheus endpoint. Note: prometheus endpoints can be also used via the usual HTTP/HTTPs ports.

<h3 id="psi_type_stall_type">
  PSI\_*type*\_*stall\_type*
</h3>

Microseconds of stall time since last measurement.Upstream docs can be found [https://docs.kernel.org/accounting/psi.html](https://docs.kernel.org/accounting/psi.html) for the metrics and how to interpret them

<h3 id="queriesmemoryusage">
  QueriesMemoryUsage
</h3>

Total memory currently used by all running queries on the server, in bytes. Useful for attributing memory pressure to the concurrent query load.

<h3 id="queriespeakmemoryusage">
  QueriesPeakMemoryUsage
</h3>

Sum of per-user query memory peaks across all users tracked in `ProcessList`, in bytes. Each user's peak is the high-water mark of that user's memory tracker, which is reset when the user has no running queries. This is therefore an aggregate of currently-tracked per-user peaks, not a single server-wide peak of all queries since startup.

<h3 id="replicasmaxabsolutedelay">
  ReplicasMaxAbsoluteDelay
</h3>

Maximum difference in seconds between the most fresh replicated part and the most fresh data part still to be replicated, across Replicated tables. A very high value indicates a replica with no data.

<h3 id="replicasmaxinsertsinqueue">
  ReplicasMaxInsertsInQueue
</h3>

Maximum number of INSERT operations in the queue (still to be replicated) across Replicated tables.

<h3 id="replicasmaxmergesinqueue">
  ReplicasMaxMergesInQueue
</h3>

Maximum number of merge operations in the queue (still to be applied) across Replicated tables.

<h3 id="replicasmaxqueuesize">
  ReplicasMaxQueueSize
</h3>

Maximum queue size (in the number of operations like get, merge) across Replicated tables.

<h3 id="replicasmaxrelativedelay">
  ReplicasMaxRelativeDelay
</h3>

Maximum difference between the replica delay and the delay of the most up-to-date replica of the same table, across Replicated tables.

<h3 id="replicassuminsertsinqueue">
  ReplicasSumInsertsInQueue
</h3>

Sum of INSERT operations in the queue (still to be replicated) across Replicated tables.

<h3 id="replicassummergesinqueue">
  ReplicasSumMergesInQueue
</h3>

Sum of merge operations in the queue (still to be applied) across Replicated tables.

<h3 id="replicassumqueuesize">
  ReplicasSumQueueSize
</h3>

Sum queue size (in the number of operations like get, merge) across Replicated tables.

<h3 id="tcprejectedconnections">
  TCPRejectedConnections
</h3>

Number of rejected connections for the TCP protocol (without TLS).

<h3 id="tcpsecurerejectedconnections">
  TCPSecureRejectedConnections
</h3>

Number of rejected connections for the TCP protocol (with TLS).

<h3 id="tcpsecurethreads">
  TCPSecureThreads
</h3>

Number of threads in the server of the TCP protocol (with TLS).

<h3 id="tcpthreads">
  TCPThreads
</h3>

Number of threads in the server of the TCP protocol (without TLS).

<h3 id="temperaturei">
  Temperature*i*
</h3>

The temperature of the corresponding device in ℃. A sensor can return an unrealistic value. Source: `/sys/class/thermal`

<h3 id="temperature_hwmon_name">
  Temperature\_*hwmon\_name*
</h3>

The temperature reported by the corresponding hardware monitor in ℃. A sensor can return an unrealistic value. Source: `/sys/class/hwmon`

<h3 id="temperature_hwmon_name_sensor_name">
  Temperature\_*hwmon\_name*\_*sensor\_name*
</h3>

The temperature reported by the corresponding hardware monitor and the corresponding sensor in ℃. A sensor can return an unrealistic value. Source: `/sys/class/hwmon`

<h3 id="totalbytesofmergetreetables">
  TotalBytesOfMergeTreeTables
</h3>

Total amount of bytes (compressed, including data and indices) stored in all tables of MergeTree family.

<h3 id="totalbytesofmergetreetablessystem">
  TotalBytesOfMergeTreeTablesSystem
</h3>

Total amount of bytes (compressed, including data and indices) stored in tables of MergeTree family in the system database.

<h3 id="totalindexgranularitybytesinmemory">
  TotalIndexGranularityBytesInMemory
</h3>

The total amount of memory (in bytes) used by index granules (only takes active parts into account).

<h3 id="totalindexgranularitybytesinmemoryallocated">
  TotalIndexGranularityBytesInMemoryAllocated
</h3>

The total amount of memory (in bytes) reserved for index granules (only takes active parts into account).

<h3 id="totalpartsofmergetreetables">
  TotalPartsOfMergeTreeTables
</h3>

Total amount of data parts in all tables of MergeTree family. Numbers larger than 10 000 will negatively affect the server startup time and it may indicate unreasonable choice of the partition key.

<h3 id="totalpartsofmergetreetablessystem">
  TotalPartsOfMergeTreeTablesSystem
</h3>

Total amount of data parts in tables of MergeTree family in the system database.

<h3 id="totalprimarykeybytesinmemory">
  TotalPrimaryKeyBytesInMemory
</h3>

The total amount of memory (in bytes) used by primary key values (only takes active parts into account).

<h3 id="totalprimarykeybytesinmemoryallocated">
  TotalPrimaryKeyBytesInMemoryAllocated
</h3>

The total amount of memory (in bytes) reserved for primary key values (only takes active parts into account).

<h3 id="totalprojectionindexgranularitybytesinmemory">
  TotalProjectionIndexGranularityBytesInMemory
</h3>

The total amount of memory (in bytes) used by projection index granularity (only takes active parts into account).

<h3 id="totalprojectionindexgranularitybytesinmemoryallocated">
  TotalProjectionIndexGranularityBytesInMemoryAllocated
</h3>

The total amount of memory (in bytes) reserved for projection index granularity (only takes active parts into account).

<h3 id="totalprojectionprimarykeybytesinmemory">
  TotalProjectionPrimaryKeyBytesInMemory
</h3>

The total amount of memory (in bytes) used by projection primary key values (only takes active parts into account).

<h3 id="totalprojectionprimarykeybytesinmemoryallocated">
  TotalProjectionPrimaryKeyBytesInMemoryAllocated
</h3>

The total amount of memory (in bytes) reserved for projection primary key values (only takes active parts into account).

<h3 id="totalrowsofmergetreetables">
  TotalRowsOfMergeTreeTables
</h3>

Total amount of rows (records) stored in all tables of MergeTree family.

<h3 id="totalrowsofmergetreetablessystem">
  TotalRowsOfMergeTreeTablesSystem
</h3>

Total amount of rows (records) stored in tables of MergeTree family in the system database.

<h3 id="totaluncompressedbytesofmergetreetables">
  TotalUncompressedBytesOfMergeTreeTables
</h3>

Total amount of uncompressed bytes, as reported by the part checksums, stored in all tables of MergeTree family. It is the same source as the `total_bytes_uncompressed` column of `system.tables`, and it does not include files that are stored uncompressed, such as marks and primary key indices.

<h3 id="totaluncompressedbytesofmergetreetablessystem">
  TotalUncompressedBytesOfMergeTreeTablesSystem
</h3>

Total amount of uncompressed bytes, as reported by the part checksums, stored in tables of MergeTree family in the system database. It is the same source as the `total_bytes_uncompressed` column of `system.tables`, and it does not include files that are stored uncompressed, such as marks and primary key indices.

<h3 id="trackedmemory">
  TrackedMemory
</h3>

Memory tracked by ClickHouse (should be equal to MemoryTracking metric), in bytes.

<h3 id="uptime">
  Uptime
</h3>

The server uptime in seconds. It includes the time spent for server initialization before accepting connections.

<h3 id="vmmaxmapcount">
  VMMaxMapCount
</h3>

The maximum number of memory mappings a process may have (/proc/sys/vm/max\_map\_count).

<h3 id="vmnummaps">
  VMNumMaps
</h3>

The current number of memory mappings of the process (/proc/self/maps). If it is close to the maximum (VMMaxMapCount), you should increase the limit for vm.max\_map\_count in /etc/sysctl.conf

<h3 id="zookeeperclientlastzxidseen">
  ZooKeeperClientLastZXIDSeen
</h3>

The last ZXID seen by the current ZooKeeper client session. This value increases monotonically as the client observes transactions from ZooKeeper.

**See Also**

* [Monitoring](/guides/oss/deployment-and-scaling/monitoring/monitoring) — Base concepts of ClickHouse monitoring.
* [system.metrics](/reference/system-tables/metrics) — Contains instantly calculated metrics.
* [system.events](/reference/system-tables/events) — Contains a number of events that have occurred.
* [system.metric\_log](/reference/system-tables/metric_log) — Contains a history of metrics values from tables `system.metrics` and `system.events`.
