PromQL Selectors & Matchers

In our last post, we learned what PromQL is, where we use it, and what data types it returns. We ran our first queries against Node Exporter metrics in the Prometheus UI.

In this post, we will learn how to filter time series by their labels using selectors and matchers. 
Our setup is a Prometheus server scraping a Linux host via Node Exporter. 
All examples will run in the Prometheus UI at http://<our-server>:9090.

The Problem With Bare Metric Names

When you query using a metric name like “node_cpu_seconds_total”, Prometheus returns every time series that matches this metric.

We will see many results — one per CPU core per mode. Node Exporter reports CPU time broken down by mode: idle, user, system, iowait, irq, softirq, steal, and nice. In a multi-host environment, the same query returns all of those series for every server Prometheus scrapes.

We need a way to say: give me only the series I care about. That is what a selector does.

What Is a Selector?

A selector narrows a query to only the series that match specific label conditions. We write it using curly braces immediately after the metric name:

metric_name{label_condition}

The conditions inside the curly braces are called matchers. PromQL gives us four matcher operators.

The Four Matchers

= — Exact match

The = operator selects series where a label equals an exact value. We will use it to pin every query to our specific host:

node_cpu_seconds_total{instance="172.31.33.131:9100"}

The results narrow down to our host only. 

We still see all CPU modes and all cores, but every result belongs to 172.31.33.131:9100.

We can add a second matcher to filter further. Let's isolate the idle mode only:

node_cpu_seconds_total{instance="172.31.33.131:9100", mode="idle"}

On a dual-core VM, we will see two series — one for cpu="0" and one for cpu="1". Multiple matchers work as a logical AND — every condition must match.




!= — Exclude exact value

The != operator excludes series where a label equals a specific value. Let's flip the previous query — we want everything except idle:

node_cpu_seconds_total{instance="172.31.33.131:9100", mode!="idle"}

We will see all CPU modes except idle. This is useful when we want to measure active CPU work without the idle baseline inflating the result.




=~ — Regex match

The =~ operator selects series where a label matches a regular expression. Prometheus uses RE2 syntax.
Let's say we only want to see the CPU time spent in user space and kernel mode:

node_cpu_seconds_total{instance="172.31.33.131:9100", mode=~"user|system"}

The | character means "or". We will see two groups of series — one for mode="user" and one for mode="system", across all CPU cores.

We will use the same pattern to filter disk metrics. Node Exporter reports many filesystems including virtual ones. Let's match only the root filesystem and the /data mount point:

node_filesystem_avail_bytes{instance="172.31.33.131:9100", mountpoint=~"/|/data"}

Important: The regex must match the full label value, not just a substring. The pattern "sys" will not match "system". We must write "sys.*" or ".*sys.*" depending on what we need.




!~ — Regex exclusion

The !~ operator excludes series where a label matches a regular expression. This is the operator we will reach for most often when cleaning up disk and network queries.

Let's get available disk space on all real, persistent filesystems and exclude the virtual ones:

node_filesystem_avail_bytes{
  instance="172.31.33.131:9100",
  fstype!~"tmpfs|squashfs|devtmpfs"
}

We will see results only for real filesystems. The tmpfs, squashfs, and devtmpfs entries disappear from the output.

We'll apply the same idea to network interfaces. The loopback interface and virtual Ethernet devices add noise to network metrics. Let's exclude them:
promql

node_network_receive_bytes_total{
  instance="172.31.33.131:9100",
  device!~"lo|veth.*"
}

We will see receive throughput only for real network interfaces.

Querying Without a Metric Name

We can use a selector without a metric name. This selects every metric Prometheus has scraped from a target:

{instance="172.31.33.131:9100", job="node_exporter"}

We'll run this in the Prometheus UI. We will see every single metric available for our host. This is useful when we first connect to a new target and want to explore what is available before writing specific queries.

Building a Production-Ready Query

Let's put it all together. We want to monitor disk space on our MuleSoft host. We need available bytes, on real filesystems, excluding the boot partition, scoped to our specific server:
promql

node_filesystem_avail_bytes{
  instance="172.31.33.131:9100",
  fstype!~"tmpfs|squashfs|devtmpfs",
  mountpoint!="/boot"
}

We will run this and see exactly what we need — no virtual filesystems, no boot partition, only the filesystems that matter for a running application. This is the query we will use as the basis for a disk space alert later in this series.

Matcher Quick Reference

Operator	Meaning	Example
"="	Exact match	mode="idle"
!=	Exclude exact value	mode!="idle"
"=~"	Regex match	mode=~"user|system"
!~	Regex exclusion	mode!~"idle|iowait"

Multiple matchers in the same selector are combined with AND logic.

Summary

A selector filters time series by their labels. We place matchers inside curly braces after the metric name. We use = and !=for exact matches and =~ and !~ for regex matches. We stack multiple matchers with commas — all conditions must match. Every production query we write will start with instance="172.31.33.131:9100" to make sure we are looking at the right host.

In the next post, we will learn about modifiers. We will shift our queries backwards in time using offset and pin them to an exact moment using @. These tools are essential for comparing current behavior against historical baselines and for incident post-mortems.