Skip to content

Commit 4c45623

Browse files
krajoramakrajorama
committed
Classic histogram and summary as complex types
Ref: prometheus/OpenMetrics#283 Signed-off-by: György Krajcsovits <[email protected]>
1 parent 3195123 commit 4c45623

File tree

1 file changed

+76
-100
lines changed

1 file changed

+76
-100
lines changed

‎docs/specs/om/open_metrics_spec_2_0.md‎

Lines changed: 76 additions & 100 deletions
Original file line numberDiff line numberDiff line change
@@ -86,9 +86,9 @@ ComplexValue MUST contain all information necessary to recreate a sample value f
8686

8787
The following Metric Types MUST use ComplexValue for Metric Values:
8888

89-
TODO: Below will switch to Histogram and Summary in the next PR.
90-
* [Histogram](#histogram) MetricFamily Type with [Native Buckets](#native-buckets).
91-
* [GaugeHistogram](#gauge-histogram) MetricFamily Type with [Native Buckets](#native-buckets).
89+
* [Histogram](#histogram) MetricFamily Type.
90+
* [GaugeHistogram](#gauge-histogram) MetricFamily Type.
91+
* [Summary](#summary) MetricFamily Type.
9292

9393
Other Metric Types MUST use Numbers.
9494

@@ -498,9 +498,9 @@ normal-char = %x00-09 / %x0B-21 / %x23-5B / %x5D-D7FF / %xE000-10FFFF
498498
start-timestamp = %d115.116 "@" timestamp
499499
500500
; Complex values
501-
complex-value = nativehistogram
501+
complex-value = native-histogram / classic-histogram / classic-summary
502502
503-
nativehistogram = nh-count "," nh-sum "," nh-schema "," nh-zero-threshold "," nh-zero-count [ "," nh-negative-spans "," nh-negative-buckets ] [ "," nh-positive-spans "," nh-positive-buckets ]
503+
native-histogram = nh-count "," nh-sum "," nh-schema "," nh-zero-threshold "," nh-zero-count [ "," nh-negative-spans "," nh-negative-buckets ] [ "," nh-positive-spans "," nh-positive-buckets ]
504504
505505
; count:x
506506
nh-count = %d99.111.117.110.116 ":" non-negative-integer
@@ -532,6 +532,32 @@ non-negative-integer = ["+"] 1*"0" / ["+"] positive-integer
532532
; Leading 0s explicitly okay.
533533
positive-integer = *"0" positive-digit *DIGIT
534534
positive-digit = "1" / "2" / "3" / "4" / "5" / "6" / "7" / "8" / "9"
535+
536+
; count:12,sum:100.0,bucket:[0.1:3,05:12,+Inf:12]
537+
classic-histogram = ch-count "," ch-sum "," ch-bucket
538+
539+
; count:x where x is a real number, not +-Inf or NaN
540+
ch-count = %d99.111.117.110.116 ":" non-negative-integer
541+
; sum:x where x is a real number or +-Inf or NaN
542+
ch-sum = %d115.117.109 ":" number
543+
; bucket:[...]
544+
ch-bucket = %d98.117.99.107.101.116 ":" "[" ch-le-counts "]"
545+
ch-le-counts = ch-pos-inf-bucket / (ch-neg-inf-bucket / ch-le-bucket) *("," ch-le-bucket) "," ch-pos-inf-bucket
546+
ch-pos-inf-bucket = "+" %d73.110.102 ":" non-negative-integer
547+
ch-neg-inf-bucket = "-" %d73.110.102 ":" non-negative-integer
548+
ch-le-bucket = realnumber ":" non-negative-integer
549+
550+
; count:12.0,sum:100.0,quantile:[0.9:2.0,0.95:3.0,0.99:20.0]
551+
classic-summary = cs-count "," cs-sum "," cs-quantile
552+
553+
; count:x where x is a real number, not +-Inf or NaN
554+
cs-count = %d99.111.117.110.116 ":" non-negative-integer
555+
; sum:x where x is a real number or +-Inf or NaN
556+
cs-sum = %d115.117.109 ":" number
557+
; quantile:[...]
558+
cs-quantile = %d113.117.97.110.116.105.108.101 ":" "[" [ cs-q-counts ] "]"
559+
cs-q-counts = cs-q-count *("," cs-q-count)
560+
cs-q-count = realnumber ":" realnumber
535561
```
536562

537563
#### Overall Structure
@@ -552,10 +578,8 @@ An example of a complete exposition:
552578
# TYPE acme_http_router_request_seconds summary
553579
# UNIT acme_http_router_request_seconds seconds
554580
# HELP acme_http_router_request_seconds Latency though all of ACME's HTTP request router.
555-
acme_http_router_request_seconds_sum{path="/api/v1",method="GET"} 9036.32 [email protected]
556-
acme_http_router_request_seconds_count{path="/api/v1",method="GET"} 807283.0 [email protected]
557-
acme_http_router_request_seconds_sum{path="/api/v2",method="POST"} 479.3 [email protected]
558-
acme_http_router_request_seconds_count{path="/api/v2",method="POST"} 34.0 [email protected]
581+
acme_http_router_request_seconds{path="/api/v1",method="GET"} {count:807283,sum:9036.32,quantile:[0.95:2,0.99:20]} [email protected]
582+
acme_http_router_request_seconds{path="/api/v2",method="GET"} {count:34,sum:479.3,quantile:[0.95:2.5,0.99:2.9]} [email protected]
559583
# TYPE go_goroutines gauge
560584
# HELP go_goroutines Number of goroutines that currently exist.
561585
go_goroutines 69
@@ -567,11 +591,7 @@ process_cpu_seconds_total 4.20072246e+06
567591
# UNIT acme_http_request_seconds seconds
568592
# HELP acme_http_request_seconds Latency histogram of all of ACME's HTTP requests.
569593
acme_http_request_seconds{path="/api/v1",method="GET"} {count:2,sum:1.2e2,schema:0,zero_threshold:1e-4,zero_count:0,positive_spans:[1:2],positive_buckets:[1,1]} [email protected]
570-
acme_http_request_seconds_count{path="/api/v1",method="GET"} 2 [email protected]
571-
acme_http_request_seconds_sum{path="/api/v1",method="GET"} 1.2e2 [email protected]
572-
acme_http_request_seconds_buckets{path="/api/v1",method="GET",le="0.5"} 1 [email protected]
573-
acme_http_request_seconds_buckets{path="/api/v1",method="GET",le="1"} 2 [email protected]
574-
acme_http_request_seconds_buckets{path="/api/v1",method="GET",le="+Inf"} 2 [email protected]
594+
acme_http_request_seconds{path="/api/v1",method="GET"} {count:2,sum:1.2e2,bucket:[0.5:1,1:2,+Inf:2]} [email protected]
575595
# TYPE "foodb.read.errors" counter
576596
# HELP "foodb.read.errors" The number of errors in the read path for fooDb.
577597
{"foodb.read.errors","service.name"="my_service"} 3482
@@ -627,32 +647,10 @@ Integer numbers MUST NOT have a decimal point. Examples are `23`, `0042`, and `1
627647

628648
Floating point numbers MUST be represented either with a decimal point or using scientific notation. Examples are `8903.123421` and `1.89e-7`. Floating point numbers MUST fit within the range of a 64-bit floating point value as defined by IEEE 754, but MAY require so many bits in the mantissa that results in lost precision. This MAY be used to encode nanosecond resolution timestamps.
629649

630-
Arbitrary integer and floating point rendering of numbers MUST NOT be used for "quantile" and "le" label values as in section "Canonical Numbers". They MAY be used anywhere else numbers are used.
631-
632650
###### ComplexValues
633651

634652
ComplexValue is represented as structured data with fields. There MUST NOT be any whitespace around fields. See the ABNF for exact details about the format and possible values.
635653

636-
###### Considerations: Canonical Numbers
637-
638-
Numbers in the "le" label values of histograms and "quantile" label values of summary metrics are special in that they're label values, and label values are intended to be opaque. As end users will likely directly interact with these string values, and as many monitoring systems lack the ability to deal with them as first-class numbers, it would be beneficial if a given number had the exact same text representation.
639-
640-
Consistency is highly desirable, but real world implementations of languages and their runtimes make mandating this impractical. The most important common quantiles are 0.5, 0.95, 0.9, 0.99, 0.999 and bucket values representing values from a millisecond up to 10.0 seconds, because those cover cases like latency SLAs and Apdex for typical web services. Powers of ten are covered to try to ensure that the switch between fixed point and exponential rendering is consistent as this varies across runtimes. The target rendering is equivalent to the default Go rendering of float64 values (i.e. %g), with a .0 appended in case there is no decimal point or exponent to make clear that they are floats.
641-
642-
Exposers MUST produce output for positive infinity as +Inf.
643-
644-
Exposers SHOULD produce output for the values 0.0 up to 10.0 in 0.001 increments in line with the following examples:
645-
0.0 0.001 0.002 0.01 0.1 0.9 0.95 0.99 0.999 1.0 1.7 10.0
646-
647-
Exposers SHOULD produce output for the values 1e-10 up to 1e+10 in powers of ten in line with the following examples:
648-
1e-10 1e-09 1e-05 0.0001 0.1 1.0 100000.0 1e+06 1e+10
649-
650-
Parsers MUST NOT reject inputs which are outside of the canonical values merely because they are not consistent with the canonical values. For example 1.1e-4 must not be rejected, even though it is not the consistent rendering of 0.00011.
651-
652-
Exposers SHOULD follow these patterns for non-canonical numbers, and the intention is by adjusting the rendering algorithm to be consistent for these values that the vast majority of other values will also have consistent rendering. Exposers using only a few particular le/quantile values could also hardcode. In languages such as C where a minimal floating point rendering algorithm such as Grisu3 is not readily available, exposers MAY use a different rendering.
653-
654-
A warning to implementers in C and other languages that share its printf implementation: The standard precision of %f, %e and %g is only six significant digits. 17 significant digits are required for full precision, e.g. `printf("%.17g", d)`.
655-
656654
##### Timestamps
657655

658656
Timestamps SHOULD NOT use exponential float rendering for timestamps if nanosecond precision is needed as rendering of a float64 does not have sufficient precision, e.g. `1604676851.123456789`.
@@ -754,38 +752,42 @@ MetricPoints MUST NOT be interleaved.
754752
A correct example where there were multiple MetricPoints and Samples within a MetricFamily would be:
755753

756754
```openmetrics-add-eof
757-
# TYPE foo_seconds summary
758-
# UNIT foo_seconds seconds
759-
foo_seconds_count{a="bb"} 0 123
760-
foo_seconds_sum{a="bb"} 0 123
761-
foo_seconds_count{a="bb"} 0 456
762-
foo_seconds_sum{a="bb"} 0 456
763-
foo_seconds_count{a="ccc"} 0 123
764-
foo_seconds_sum{a="ccc"} 0 123
765-
foo_seconds_count{a="ccc"} 0 456
766-
foo_seconds_sum{a="ccc"} 0 456
755+
# TYPE foo stateset
756+
foo{entity="controller",foo="a"} 1.0
757+
foo{entity="controller",foo="bb"} 0.0
758+
foo{entity="controller",foo="ccc"} 0.0
759+
foo{entity="replica",foo="a"} 1.0
760+
foo{entity="replica",foo="bb"} 0.0
761+
foo{entity="replica",foo="ccc"} 1.0
767762
```
768763

769764
An incorrect example where Metrics are interleaved:
770765

771-
```
772-
# TYPE foo_seconds summary
773-
# UNIT foo_seconds seconds
774-
foo_seconds_count{a="bb"} 0 123
775-
foo_seconds_count{a="ccc"} 0 123
776-
foo_seconds_count{a="bb"} 0 456
777-
foo_seconds_count{a="ccc"} 0 456
766+
```openmetrics-add-eof
767+
# TYPE foo stateset
768+
foo{entity="controller",foo="a"} 1.0
769+
foo{entity="controller",foo="bb"} 0.0
770+
foo{entity="controller",foo="ccc"} 0.0
771+
foo{entity="replica",foo="a"} 1.0
772+
foo{entity="replica",foo="bb"} 0.0
773+
foo{entity="replica",foo="ccc"} 1.0
774+
foo{entity="controller",foo="a"} 1.0
775+
foo{entity="controller",foo="bb"} 0.0
776+
foo{entity="controller",foo="ccc"} 0.0
778777
```
779778

780779
An incorrect example where MetricPoints are interleaved:
781780

782-
```
781+
```openmetrics-add-eof
783782
# TYPE foo_seconds summary
784783
# UNIT foo_seconds seconds
785-
foo_seconds_count{a="bb"} 0 123
786-
foo_seconds_count{a="bb"} 0 456
787-
foo_seconds_sum{a="bb"} 0 123
788-
foo_seconds_sum{a="bb"} 0 456
784+
# TYPE foo stateset
785+
foo{entity="controller",foo="a"} 1.0
786+
foo{entity="controller",foo="bb"} 0.0
787+
foo{entity="replica",foo="a"} 1.0
788+
foo{entity="controller",foo="ccc"} 0.0
789+
foo{entity="replica",foo="bb"} 0.0
790+
foo{entity="replica",foo="ccc"} 1.0
789791
```
790792

791793
#### Metric types
@@ -945,47 +947,35 @@ An example of a Metric with no labels and a MetricPoint with Sum, Count and Star
945947

946948
```openmetrics-add-eof
947949
# TYPE foo summary
948-
foo_count 17.0 [email protected]
949-
foo_sum 324789.3 [email protected]
950+
foo {count:17,sum:324789.3,quantile:[]} [email protected]
950951
```
951952

952953
An example of a Metric with no labels and a MetricPoint with two quantiles and Start Timestamp values:
953954

954955
```openmetrics-add-eof
955956
# TYPE foo summary
956-
foo{quantile="0.95"} 123.7 [email protected]
957-
foo{quantile="0.99"} 150.0 [email protected]
957+
foo {count:0,sum:0.0,quantile:[0.95:123.7,0.99:150]} [email protected]
958958
```
959959

960960
Quantiles MAY be in any order.
961961

962962
##### Histogram with Classic Buckets
963963

964-
The MetricPoint's Sum Value Sample MetricName MUST have the suffix `_sum`. The MetricPoint's Count Value Sample MetricName MUST have the suffix `_count`. The MetricPoint's Classic Bucket values Sample MetricNames MUST have the suffix `_bucket`.
964+
The MetricPoint's value MUST be a ComplexValue.
965+
966+
The ComplexValue MUST include the Count, Sum and Classic Bucket values as the fields `count`, `sum`, `bucket`, in this order.
965967

966968
If present the MetricPoint's Start Timestamp MUST be inlined with the Metric point with a `st@` prefix. If the value's timestamp is present, the Start Timestamp MUST be added right after it. If exemplar is present, the Start Timestamp MUST be added before it. Start Timestamp MUST be appended to all Classic Bucket values, to the MetricPoint's Sum and MetricPoint's Count.
967969

968-
Classic Buckets MUST be sorted in number increasing order of "le", and the value of the "le" label MUST follow the rules for Canonical Numbers.
970+
Classic Buckets MUST be sorted in number increasing order of their threshold.
969971

970972
All Classic Buckets MUST be present, even ones with the value 0.
971973

972-
An example of a Metric with no labels and a MetricPoint with Sum, Count, and Start Timestamp values, and with 12 Classic Buckets. A wide and atypical but valid variety of “le” values is shown on purpose:
974+
An example of a Metric with no labels and a MetricPoint with Sum, Count, and Start Timestamp values, and with 12 Classic Buckets. A wide and atypical but valid variety of bucket threshold values is shown on purpose:
973975

974976
```openmetrics-add-eof
975977
# TYPE foo histogram
976-
foo_bucket{le="0.0"} 0 [email protected]
977-
foo_bucket{le="1e-05"} 0 [email protected]
978-
foo_bucket{le="0.0001"} 5 [email protected]
979-
foo_bucket{le="0.1"} 8 [email protected]
980-
foo_bucket{le="1.0"} 10 [email protected]
981-
foo_bucket{le="10.0"} 11 [email protected]
982-
foo_bucket{le="100000.0"} 11 [email protected]
983-
foo_bucket{le="1e+06"} 15 [email protected]
984-
foo_bucket{le="1e+23"} 16 [email protected]
985-
foo_bucket{le="1.1e+23"} 17 [email protected]
986-
foo_bucket{le="+Inf"} 17 [email protected]
987-
foo_count 17 [email protected]
988-
foo_sum 324789.3 [email protected]
978+
foo {count:17,sum:324789.3,bucket:[0.0:0,1e-05:0,0.0001:5,0.1:8,1.0:10,10.0:11,100000.0:11,1e+06:15,1e+23:16,1.1e+23:17,+Inf:17]} [email protected]
989979
```
990980

991981
##### Histogram with Native Buckets
@@ -1037,11 +1027,7 @@ The order ensures that implementations can easily skip the Classic Buckets if th
10371027
# UNIT acme_http_request_seconds seconds
10381028
# HELP acme_http_request_seconds Latency histogram of all of ACME's HTTP requests.
10391029
acme_http_request_seconds{path="/api/v1",method="GET"} {count:2,sum:1.2e2,schema:0,zero_threshold:1e-4,zero_count:0,positive_spans:[1:2],positive_buckets:[1,1]}
1040-
acme_http_request_seconds_count{path="/api/v1",method="GET"} 2
1041-
acme_http_request_seconds_sum{path="/api/v1",method="GET"} 1.2e2
1042-
acme_http_request_seconds_buckets{path="/api/v1",method="GET",le="0.5"} 1
1043-
acme_http_request_seconds_buckets{path="/api/v1",method="GET",le="1"} 2
1044-
acme_http_request_seconds_buckets{path="/api/v1",method="GET",le="+Inf"} 2
1030+
acme_http_request_seconds{path="/api/v1",method="GET"} {count:2,sum:1.2e2,bucket:[0.5:1,1:2,+Inf:2]}
10451031
```
10461032

10471033
###### Exemplars
@@ -1054,33 +1040,23 @@ The "0.01" bucket has no Exemplar. The 0.1 bucket has an Exemplar with no Labels
10541040

10551041
```openmetrics-add-eof
10561042
# TYPE foo histogram
1057-
foo {count:10,sum:1.0,schema:0,zero_threshold:1e-4,zero_count:0,positive_spans:[0:2],positive_buckets:[5,5]} [email protected] # {trace_id="shaZ8oxi"} 0.67 1520879607.789 # {trace_id="ookahn0M"} 1.2 1520879608.589
1058-
foo_bucket{le="0.01"} 0 [email protected]
1059-
foo_bucket{le="0.1"} 8 [email protected] # {} 0.054
1060-
foo_bucket{le="1"} 11 [email protected] # {trace_id="KOO5S4vxi0o"} 0.67
1061-
foo_bucket{le="10"} 17 [email protected] # {trace_id="oHg5SJYRHA0"} 9.8 1520879607.789
1062-
foo_bucket{le="+Inf"} 17 [email protected]
1063-
foo_count 17 [email protected]
1064-
foo_sum 324789.3 [email protected]
1043+
foo {count:17,sum:324789.3,schema:0,zero_threshold:1e-4,zero_count:0,positive_spans:[0:2],positive_buckets:[5,12]} [email protected] # {trace_id="shaZ8oxi"} 0.67 1520879607.789 # {trace_id="ookahn0M"} 1.2 1520879608.589
1044+
foo {count:17,sum:324789.3,bucket:[0.01:0,0.1:8,1.0:11,10.0:17,+Inf:17]} [email protected] # {} 0.054 1520879607.7 # {trace_id="KOO5S4vxi0o"} 0.67 1520879602.890 # {trace_id="oHg5SJYRHA0"} 9.8 1520879607.789
10651045
```
10661046

10671047
##### GaugeHistogram with Classic Buckets
10681048

1069-
The MetricPoint's Sum Value Sample MetricName MUST have the suffix `_gsum`. The MetricPoint's Count Value Sample MetricName MUST have the suffix `_gcount`. The MetricPoint's Classic Bucket values Sample MetricNames MUST have the suffix `_bucket`.
1049+
The MetricPoint's value MUST be a ComplexValue.
1050+
1051+
The ComplexValue MUST include the Gcount, Gsum and Classic Bucket values as the fields `count`, `sum`, `bucket`, in this order.
10701052

1071-
Classic Buckets MUST be sorted in number increasing order of "le", and the value of the "le" label MUST follow the rules for Canonical Numbers.
1053+
Classic Buckets MUST be sorted in number increasing order of their threshold.
10721054

10731055
An example of a Metric with no labels, and one MetricPoint value with no Exemplar with no Exemplars in the buckets:
10741056

10751057
```openmetrics-add-eof
10761058
# TYPE foo gaugehistogram
1077-
foo_bucket{le="0.01"} 20.0
1078-
foo_bucket{le="0.1"} 25.0
1079-
foo_bucket{le="1"} 34.0
1080-
foo_bucket{le="10"} 34.0
1081-
foo_bucket{le="+Inf"} 42.0
1082-
foo_gcount 42.0
1083-
foo_gsum 3289.3
1059+
foo {count:42,sum:3289.3,bucket:[0.01:20,0.1:25,1:34,+Inf:42]}
10841060
```
10851061

10861062
##### GaugeHistogram with Native Buckets

0 commit comments

Comments
 (0)