[1.10.2] error parsing file: Too few fields in mount string

[pasztorl]

Hi,

node_filesystem_avail_bytes not reported, and I couldn't find this message in the source code so I'm stuck looking for the error. I've attached the logs. Could someone please take a look?

prometheus-prometheus-node-exporter-n4jns.log

[lakshmiyagnanandareddy]

Hi @pasztorl,

This is a configuration issue caused by a race condition reading /proc/self/mountinfo.

Fix

Add run to your mount exclusions:

--collector.filesystem.mount-points-exclude=^/(dev|proc|sys|run|var/lib/kubelet/.+)($|/)

If using Helm:

helm upgrade prometheus-node-exporter prometheus-community/prometheus-node-exporter \
  -n monitoring \
  --set 'extraArgs[0]=--collector.filesystem.mount-points-exclude=^/(dev|proc|sys|run|var/lib/kubelet/.+)($|/)' \
  --reuse-values

Verify (after ~2 min for pod restart)

# Should return 0
kubectl logs -n monitoring <pod-name> --tail=100 | grep "collector failed" | wc -l

# Should show filesystem metrics
kubectl exec -n monitoring <pod-name> -- wget -q -O- localhost:9100/metrics | grep node_filesystem_avail_bytes

---

If this doesn't resolve it, please share:

1. How you deployed Node Exporter (Helm chart or kubectl manifest)
2. The exact configuration you applied
3. Current error messages from pod logs

Let me know how it goes!

[Yzzzi]

Hi @lakshmiyagnanandareddy

Looks like I am facing exactly the same issue as OP.
Tried to apply fix you proposed but it didn't work, collector is still failing.

1. Deployed via official helm chart, version 4.49.1, into k3s cluster version v1.28.6+k3s2
2. config.yaml
3. The errors are slightly different on different nodes, although, alike
   From one node:

time=2026-02-06T19:50:13.579Z level=ERROR source=collector.go:168 msg="collector failed" name=filesystem duration_seconds=0.108221905 err="error parsing file: Too few fields in mount string: 77765 26 0:8250 / /run/k3s/containerd/io.containerd.runtime.v2.task/"

From the other one:

time=2026-02-06T19:52:41.006Z level=ERROR source=collector.go:168 msg="collector failed" name=filesystem duration_seconds=0.275330447 err="error parsing file: Too few fields in mount string: 2652 28 0:2852 / /var/lib/kubelet/pods/50e73ded-71b7-45e9-8b04-8bf3def"

Also tried to play around with regex, also tried different flags, like --collector.filesystem.mount-points-exclude, --collector.filesystem.ignored-mount-points, but it didn't help either.

Let me know if I can provide any additional information that may be helpful!

[lakshmiyagnanandareddy]

@Yzzzi The error suggests the exclusion isn't being applied. Can you verify the args reached the pods?

# Check if the exclusion flag is actually running
kubectl get daemonset -n monitoring prometheus-prometheus-node-exporter -o yaml | grep -A 10 "args:"

You should see:

args:
  - --path.procfs=/host/proc
  - --path.sysfs=/host/sys
  - --collector.filesystem.mount-points-exclude=^/(dev|proc|sys|run|var/lib/kubelet/.+)($|/)  #  Must be present

If the --collector.filesystem.mount-points-exclude line is missing, the Helm chart didn't apply your extraArgs.

If it's missing, try this explicit upgrade:

helm upgrade prometheus-prometheus-node-exporter prometheus-community/prometheus-node-exporter \
  -n monitoring \
  --version 4.49.1 \
  --set 'extraArgs[0]=--collector.filesystem.mount-points-exclude=^/(dev|proc|sys|run|var/lib/kubelet/.+)($|/)'

Then verify again and check logs for errors.

Let me know what you find!

[Yzzzi]

Hi @lakshmiyagnanandareddy

The flag is present in the args, confirmed.

➜  ~ kubectl get daemonset -n monitoring prometheus-node-exporter -o yaml | grep -A 10 "args:"           
      - args:
        - --path.procfs=/host/proc
        - --path.sysfs=/host/sys
        - --path.rootfs=/host/root
        - --path.udev.data=/host/root/run/udev/data
        - --web.listen-address=[$(HOST_IP)]:9100
        - --collector.filesystem.mount-points-exclude=^/(dev|proc|sys|run|var/lib/kubelet/.+)($|/)
        env:
        - name: HOST_IP
          value: 0.0.0.0

On top of that, I can see it being parsed in the startup logs, but the issue is still there.

time=2026-02-06T20:14:29.834Z level=INFO source=filesystem_common.go:265 msg="Parsed flag --collector.filesystem.mount-points-exclude" collector=filesystem flag=^/(dev|proc|sys|run|var/lib/kubelet/.+)($|/)
...
time=2026-02-07T04:30:22.955Z level=ERROR source=collector.go:168 msg="collector failed" name=filesystem duration_seconds=0.540097497 err="error parsing file: Too few fields in mount string: 95478 26 0:8324 / /run/k3s/containerd/io.containerd.runtime.v2.task/k8s.io/283e8c40924c0b0d"
time=2026-02-08T04:30:52.603Z level=ERROR source=collector.go:168 msg="collector failed" name=filesystem duration_seconds=0.176100708 err="error parsing file: Too few fields in mount string: 62807 26 0:9213 / /run/k3s/containerd/io.containerd.runtime.v2."

[mehrdadbn9]

Hi, I'd like to investigate this 'Too few fields' parsing error in the collector. The issue is related to file parsing. Could a maintainer please assign it to me? Thank you!

[Vedant-Mhatre]

I opened a fix for this in #3582.

I noticed @mehrdadbn9 had previously asked about working on this. Since I didn’t see an assignment or an in-progress PR, I went ahead with a fresh patch. If you already have work in progress, I’m happy to coordinate.

The current PR takes a conservative approach:

• retry transient mountinfo parse errors a small number of times
• keep existing fallback/error behavior for non-parse failures
• avoid any metric/schema changes

CI is green on the PR.

[discordianfish]

What race condition specifically? How can this happen? I'd rather not blindly retry like the PR unless we know exactly what is going on

[jtetteroo-fugro]

This happens in https://github.com/prometheus/procfs where it cannot parse a string from proc/self/mountinfo in the function parseMountInfoString():mountinfo.go.

My thinking is it should just generate a warning for broken lines and keep going, but it will completely stop in parseMountInfo()

It is possible ReadFileNoStat fails to read /proc/self/mountinfo properly in GetMounts() and you end up with malformed input.
The maximum buffer size for ReadFileNoStat in procfs is 1024Kb (internal/util/readfile.go), it might be running into this limit and chopping off the line at the wrong point.