Skip to content

SOLR-18165 Use dot separated metric names#4223

Open
janhoy wants to merge 6 commits intoapache:mainfrom
janhoy:SOLR-18165-dot-separated-metric-names
Open

SOLR-18165 Use dot separated metric names#4223
janhoy wants to merge 6 commits intoapache:mainfrom
janhoy:SOLR-18165-dot-separated-metric-names

Conversation

@janhoy
Copy link
Contributor

@janhoy janhoy commented Mar 17, 2026

Implements SOLR-18165: align Solr's OTLP metric names with the OpenTelemetry semantic conventions, which require dot-separated names.

What changed

OTLP metric names now use dots instead of underscores:

Before (v10.0) After
solr_core_requests solr.core.requests
solr_core_update_committed_ops solr.core.update.committed.ops
solr_node_executor solr.node.executor

Prometheus metric names are unchanged. The OTel Prometheus exporter automatically converts dots to underscores, so the /admin/metrics?wt=prometheus output still produces solr_core_requests_total, solr_core_update_committed_ops etc.

Metric name filtering (/admin/metrics?wt=prometheus&name=solr_core_requests) continues to work using Prometheus-format names. The FilterablePrometheusMetricReader now filters by getMetadata().getPrometheusName() (underscore format) rather than against the OTel internal name — which also fixes a pre-existing issue where metrics with unit suffixes like jvm_system_memory_bytes could not be filtered by name.

Files changed

  • 17 production files — metric name string literals updated (all "solr_core_*" and "solr_node_*" occurrences)
  • AttributedInstrumentFactory.javatoNodeMetricName() updated to use "solr.core""solr.node" substitution
  • FilterablePrometheusMetricReader.java — name-filter logic updated as described above
  • changelog/unreleased/SOLR-18165-dot-separated-metric-names.yml

Breaking change

This is a back-compat break for any OTLP consumer relying on solr_core_* / solr_node_* metric names from v10.0. Affected users can add a metricstransform processor in their OpenTelemetry Collector to rename metrics during transition.


Review guidance

All feedback welcome, but one area especially worth a second look:

Metric names that retain underscores. The convention used here is:

  • Dots separate hierarchy/category levels (e.g. solrcoreupdatecommitted.ops)
  • Underscores are kept within tokens that represent a single compound concept

The following metric names intentionally retain underscores:

Metric name Underscore rationale
solr.core.is_leader boolean flag (is_leader is a single predicate)
solr.core.ref_count compound noun
solr.core.disk_space compound noun
solr.core.update.auto_commits compound noun
solr.core.update.docs.pending_commit compound noun
solr.core.update_log.* (7 metrics) update_log is the tlog component name
solr.core.replication.is_leader, is_follower, is_enabled, is_replicating, is_polling_disabled boolean flags
solr.core.crossdc.producer.submitted.delete_by_id named operation
solr.core.crossdc.producer.submitted.delete_by_query named operation
solr.core.crossdc.producer.document_size compound noun
solr.core.crossdc.producer.doc_too_large_errors compound noun phrase
solr.core.indexsearcher.index.num_docs compound noun
solr.core.indexsearcher.index.commit_size compound noun
solr.core.indexsearcher.open.warmup_time compound noun
solr.core.replication.download_speed compound noun
solr.core.replication.downloaded_size compound noun
solr.core.replication.time_elapsed compound noun
solr.core.searcher.warming_max compound noun

Changing any of these now is cheap; once released each rename becomes another breaking change.

https://issues.apache.org/jira/browse/SOLR-18165

Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR implements SOLR-18165 by aligning Solr’s OTLP-exported metric names with OpenTelemetry’s dot-separated naming conventions, while keeping Prometheus output stable and preserving Prometheus-style name filtering.

Changes:

  • Renames OTLP metric instruments from underscore-separated to dot-separated names across core, node, and CrossDC metrics.
  • Updates FilterablePrometheusMetricReader to filter using Prometheus base names (getMetadata().getPrometheusName()) rather than OTel internal names.
  • Adds release-facing documentation: Solr 10.1 upgrade note + an unreleased changelog YAML entry describing the breaking change.

Reviewed changes

Copilot reviewed 21 out of 21 changed files in this pull request and generated 1 comment.

Show a summary per file
File Description
solr/solr-ref-guide/modules/upgrade-notes/pages/major-changes-in-solr-10.adoc Documents the OTLP metric-name breaking change and Prometheus impact/workaround.
solr/modules/cross-dc/src/java/org/apache/solr/crossdc/update/processor/ProducerMetrics.java Updates CrossDC producer OTLP metric names to dot-separated format.
solr/core/src/java/org/apache/solr/update/UpdateShardHandler.java Updates executor metric prefix to dot-separated format.
solr/core/src/java/org/apache/solr/update/UpdateLog.java Updates update log OTLP metric names to dot-separated format (keeping selected underscores).
solr/core/src/java/org/apache/solr/update/SolrIndexWriter.java Updates index writer OTLP metric names to dot-separated format.
solr/core/src/java/org/apache/solr/update/PeerSyncWithLeader.java Updates peer-sync-with-leader OTLP metric names to dot-separated format.
solr/core/src/java/org/apache/solr/update/PeerSync.java Updates peer-sync OTLP metric names to dot-separated format.
solr/core/src/java/org/apache/solr/update/DirectUpdateHandler2.java Updates update handler OTLP metric names to dot-separated format.
solr/core/src/java/org/apache/solr/search/stats/StatsCache.java Updates stats cache OTLP metric name to dot-separated format.
solr/core/src/java/org/apache/solr/search/SolrIndexSearcher.java Updates index searcher OTLP metric names/prefixes to dot-separated format.
solr/core/src/java/org/apache/solr/search/SolrFieldCacheBean.java Updates field cache OTLP metric names to dot-separated format.
solr/core/src/java/org/apache/solr/metrics/otel/instruments/AttributedInstrumentFactory.java Updates core→node metric-name prefix substitution to dot-separated form.
solr/core/src/java/org/apache/solr/metrics/otel/FilterablePrometheusMetricReader.java Switches name filtering to Prometheus base names to keep /admin/metrics?wt=prometheus&name=... working post-rename.
solr/core/src/java/org/apache/solr/handler/component/SuggestComponent.java Updates suggester OTLP metric name to dot-separated format.
solr/core/src/java/org/apache/solr/handler/component/HttpShardHandlerFactory.java Updates HTTP shard executor metric prefix to dot-separated format.
solr/core/src/java/org/apache/solr/handler/admin/CoreAdminHandler.java Updates node executor metric prefix to dot-separated format.
solr/core/src/java/org/apache/solr/handler/RequestHandlerBase.java Updates request handler OTLP metric names to dot-separated format.
solr/core/src/java/org/apache/solr/handler/ReplicationHandler.java Updates replication OTLP metric names to dot-separated format (keeping selected underscores).
solr/core/src/java/org/apache/solr/core/SolrCore.java Updates core/searcher OTLP metric names to dot-separated format.
solr/core/src/java/org/apache/solr/core/CoreContainer.java Updates node cache/executor metric prefixes to dot-separated format.
changelog/unreleased/SOLR-18165-dot-separated-metric-names.yml Adds changelog entry describing the change and its back-compat impact.
Comments suppressed due to low confidence (1)

solr/core/src/java/org/apache/solr/metrics/otel/FilterablePrometheusMetricReader.java:80

  • The suffix-stripping logic only handles _total, _sum, _bucket, _created, and _info, but Prometheus histograms/summaries also expose a *_count series. If a user filters with name=<metric>_count, it won't be sanitized to the base name and the metric won't match. Consider adding _count to the recognized suffix set (and keep the doc/examples in sync).
    // Users may filter by Prometheus-format names (e.g. "solr_core_requests") or with a
    // Prometheus type suffix (e.g. "solr_core_requests_total"). Strip any such suffix so we can
    // compare against the Prometheus base name returned by getMetadata().getPrometheusName().
    Set<String> sanitizedNames =
        includedNames.stream()
            .map(
                (name) -> {
                  for (String suffix : PROM_SUFFIXES) {
                    if (name.endsWith(suffix)) {
                      return name.substring(0, name.lastIndexOf(suffix));
                    }
                  }
                  return name;

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

You can also share your feedback on Copilot code review. Take the survey.

Comment on lines 68 to +99
@@ -84,7 +85,18 @@ public MetricSnapshots collect(
if (sanitizedNames.isEmpty()) {
snapshotsToFilter = super.collect();
} else {
snapshotsToFilter = super.collect(sanitizedNames::contains);
// We collect all metrics and filter by Prometheus name rather than using
// super.collect(Predicate) which matches on OTel internal names. This avoids a mismatch
// when OTel names use dot-separators (e.g. "solr.core.requests") but users filter by the
// Prometheus underscore-format name they see in the output (e.g. "solr_core_requests").
MetricSnapshots all = super.collect();
MetricSnapshots.Builder nameFiltered = MetricSnapshots.builder();
for (MetricSnapshot snapshot : all) {
if (sanitizedNames.contains(snapshot.getMetadata().getPrometheusName())) {
nameFiltered.metricSnapshot(snapshot);
}
}
snapshotsToFilter = nameFiltered.build();
@janhoy janhoy requested review from dsmiley and mlbiscoc March 17, 2026 21:18
Co-authored-by: Matthew Biscocho <biscocho.matthew@gmail.com>
janhoy and others added 2 commits March 20, 2026 21:36
Co-authored-by: Chan Chan <chan.dx.dev@gmail.com>
Deprecate solr.core.indexsearcher.open.warmup_time

=== OTEL metrics classified as BETA

The new OTEL metrics system, naming and dashboard should be treated as a BETA. We reserve the right to make breaking changes in this space in minor versions.
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I added a BETA declaration her as proposed.


=== Metric solr_core_indexsearcher_open_warmup_time is deprecated

If you use `solr_core_indexsearcher_open_warmup_time` in a dashboard, it duplicates `solr_core_indexsearcher_warmup_time` and is deprecated for removal in a later 10.x release.
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I did not remove the duplicate but deprecate it. With the BETA regime in place we can kill this in a minor version.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In this case, we should also then update the # HELP description in the metric to say deprecated.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

cat:index cat:metrics cat:search documentation Improvements or additions to documentation

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants