Skip to content

Implement planned topic: 0023-buffered-metrics#218

Open
skill-temporal-developer-updater[bot] wants to merge 2 commits into
mainfrom
draft/0023-buffered-metrics
Open

Implement planned topic: 0023-buffered-metrics#218
skill-temporal-developer-updater[bot] wants to merge 2 commits into
mainfrom
draft/0023-buffered-metrics

Conversation

@skill-temporal-developer-updater
Copy link
Copy Markdown
Contributor

Validation Report — buffered-metrics

Branch: draft/0023-buffered-metrics
Authored files:

  • references/typescript/buffered-metrics.md (new, 221 lines, 29 typedoc citation tags)
  • references/typescript/observability.md (4-line cross-link block added)

Source of truth used: TypeScript SDK source (https://github.com/temporalio/sdk-typescript main branch, especially packages/common/src/metrics.ts, packages/worker/src/runtime.ts, packages/worker/src/runtime-metrics.ts, packages/worker/src/runtime-options.ts, and packages/test/src/test-runtime-buffered-metrics.ts) and the published typedoc at typescript.temporal.io. The skill's <!-- typedoc: … --> citation form requires this, since the APIs covered (MetricMeter, MetricsBuffer, etc.) do not appear in documentation/docs/.


Go / no-go

Check Threshold Result Verdict
1 — Citation audit ≥ 98% resolve 29/29 typedoc tags resolve to real symbols PASS
2 — Reverse-grep audit 0 unexplained misses 0 misses PASS
3 — Regression patterns 0 hits 0 hits (apparent matches are all in the "Common mistakes" table, correctly labelled as wrong) PASS
4 — Independent re-verification ≥ 95% of 10 sampled 10/10 sampled = 100% PASS (sample) — but one substantive out-of-sample correctness issue detected (see Check 4 findings)
5 — Integration-layout audit N/A not an integration topic; references/integrations.md untouched N/A
6 — Tone and scope audit 0 workaround disclosures (pattern 1) 0 pattern-1 findings PASS

Overall verdict: MINOR FIXES. All four required checks meet their numerical thresholds. One substantive code-correctness issue was found while auditing the setup pattern; it appears in two code blocks and is a targeted fix that does not require re-authoring. Per the template's verdict rubric (§5), the formal sample passing at ≥ 95% and Checks 1/3 clean would normally be GO, but the issue is significant enough to call out and fix before merge.


Check 1 — Citation audit

All 29 <!-- typedoc: … --> tags resolve to real symbols in the SDK source.

Tag (representative) Resolved location Notes
common.MetricMeter (and #withTags, #createUpDownCounter) packages/common/src/metrics.ts MetricMeter interface Match. Note that createUpDownCounter is declared optional (?) on the interface; the skill doesn't note this, but it's only relevant if a user implements MetricMeter themselves, which is not the documented path.
common.MetricCounter, common.MetricUpDownCounter, common.MetricGauge, common.MetricHistogram same file, individual interfaces Match. Each extends Metric and pins its own kind and (for counter/up-down-counter) valueType.
common.MetricKind type MetricKind = 'counter' | 'histogram' | 'gauge' | 'up-down-counter' Exact match.
common.MetricTags type MetricTags = Record<string, string | number | boolean> Exact match.
common.NumericMetricValueType type NumericMetricValueType = 'int' | 'float' Exact match.
common namespace#noopMetricMeter export const noopMetricMeter = new NoopMetricMeter() Exact match.
worker.MetricsBuffer (and #retrieveUpdates) packages/worker/src/runtime-metrics.ts class MetricsBuffer Match. retrieveUpdates(): ArrayIterator<BufferedMetricUpdate>.
worker.MetricsBufferOptions (and #maxBufferSize) same file Match. Defaults: maxBufferSize ?? 10000, useSecondsForDurations ?? false.
worker.BufferedMetricUpdate same file, interface with metric, value, attributes Match.
worker.Runtime (and #install, #metricMeter, #metricsBuffer) packages/worker/src/runtime.ts Match. install(options) and instance() are static methods; metricMeter is a public readonly instance property; metricsBuffer: MetricsBuffer | undefined is a public readonly instance property.
worker.RuntimeOptions, worker.PrometheusMetricsExporter, worker.OtelCollectorExporter packages/worker/src/runtime-options.ts All match.
workflow namespace#metricMeter packages/workflow/src/metrics.ts export const metricMeter: MetricMeter = … Match — it is a const, not a function.
activity namespace#metricMeter packages/activity/src/index.ts export const metricMeter: MetricMeter = { … } Match — also a const.
activity namespace#metricMeter adjacent to ActivityOutboundCallsInterceptor.getMetricTags() (line 93) The cited symbol metricMeter resolves correctly; the adjacent ActivityOutboundCallsInterceptor symbol resolves to the worker namespace (worker.ActivityOutboundCallsInterceptor), not activity. Minor: not a citation failure — the tag is for metricMeter — but a reader trying to import ActivityOutboundCallsInterceptor won't find it in @temporalio/activity. Not flagged as a finding because the tag's stated subject (metricMeter) is correct; flagged here for awareness.

Citation count: 29 typedoc tags. Unresolved citations: 0.


Check 2 — Reverse-grep audit

Every factual token from the authored file was located in the SDK source.

  • Type / interface / class names: MetricMeter, MetricCounter, MetricUpDownCounter, MetricGauge, MetricHistogram, MetricsBuffer, MetricsBufferOptions, BufferedMetricUpdate, MetricKind, MetricTags, NumericMetricValueType, Runtime, RuntimeOptions, TelemetryOptions, PrometheusMetricsExporter, OtelCollectorExporter, ActivityOutboundCallsInterceptor, Metric — all present.
  • Method names: createCounter, createUpDownCounter, createGauge, createHistogram, add, set, record, withTags, retrieveUpdates, install, instance, getMetricTags — all present, on the symbols claimed.
  • Property / field names: maxBufferSize, useSecondsForDurations, valueType, attributes, metric, value, kind, name, unit, description, metricMeter, metricsBuffer — all present.
  • String literal enums: "counter", "histogram", "gauge", "up-down-counter", "int", "float" — all present.
  • Defaults: maxBufferSize default 10000 ✓, useSecondsForDurations default false ✓.
  • Negative-existence claim: the skill says "There is no RuntimeMetricMeter exported type" — verified, no such symbol is exported.
  • Import paths: @temporalio/worker, @temporalio/workflow, @temporalio/activity, @temporalio/common — each package's index.ts re-exports the claimed symbols.

Unexplained misses: 0.


Check 3 — Regression on known bugs

No universal-regression patterns appear anywhere in the skill (no --profile flag, no TEMPORAL_TLS_CLIENT_*_PATH variants, no tcld service-account, no --output text/jsonl, no saas-api.tmprl.cloud:7233). Topic-specific regressions also clean:

  • "upDownCounter" (camelCase) appears once on line 178 — inside the "Common mistakes" table, correctly identified as the wrong literal. Not a regression hit.
  • retrieveBufferedMetrics() appears once on line 179 — inside the "Common mistakes" table, correctly identified as nonexistent. Confirmed against packages/worker/src/runtime.ts: no such method on Runtime.
  • import { MetricsBuffer } from '@temporalio/common' appears once on line 180 — inside the "Common mistakes" table, correctly identified as wrong. MetricsBuffer is in @temporalio/worker.
  • Counter / up-down-counter being called with a valueType argument — flagged correctly as wrong in the table.

Hits: 0.


Check 4 — Independent re-verification

Sampled claims (10 / 10 = 100% match)

# Authored claim Independent reading of source Match?
1 "Counters and up-down-counters always record integers (valueType: "int")" (line 35) MetricCounter extends Metric { … valueType: 'int'; } and MetricUpDownCounter extends Metric { … valueType: 'int'; } in packages/common/src/metrics.ts
2 "MetricKind is "counter" | "histogram" | "gauge" | "up-down-counter"" (line 37) export type MetricKind = 'counter' | 'histogram' | 'gauge' | 'up-down-counter' (same file)
3 createGauge(name, valueType?, unit?, description?) (line 32) createGauge(name: string, valueType?: NumericMetricValueType, unit?: string, description?: string): MetricGauge
4 "Every instrument also has withTags(tags: MetricTags): <SameInstrument>" (line 49) Each instrument interface declares its own withTags(tags: MetricTags) returning the same instrument type (e.g. MetricCounter)
5 "metricMeter … is a property, not a function" (line 53) workflow.metricMeter and activity.metricMeter are exported as const; Runtime#metricMeter is public readonly
6 "MetricsBuffer (exported from @temporalio/worker)" (line 99) export class MetricsBuffer { … } in packages/worker/src/runtime-metrics.ts, re-exported from packages/worker/src/index.ts
7 maxBufferSize default 10000 (line 120) this.maxBufferSize = options.maxBufferSize ?? 10000 in MetricsBuffer constructor
8 useSecondsForDurations default false (line 121) this.useSecondsForDurations = options.useSecondsForDurations ?? false
9 "retrieveUpdates() returns an ArrayIterator<BufferedMetricUpdate>" (line 138) public retrieveUpdates(): ArrayIterator<BufferedMetricUpdate> in MetricsBuffer
10 "If telemetry is not configured, the meter resolves to noopMetricMeter — calls are silently dropped" (line 95) noopMetricMeter is a NoopMetricMeter instance with empty-body methods (add(_value, _extraTags) {} etc.); workflow / activity export fall back to it when no runtime is installed

Substantive out-of-sample finding

The sample passed at 100%, but during the broader audit I found one claim that is substantively wrong and will affect every reader who follows the skill's setup code.

Location: references/typescript/buffered-metrics.md lines 103–113 (Setup) and 187–192 (Worked example).

Authored code:

const buffer = new MetricsBuffer({ maxBufferSize: 100_000 });

Runtime.install({
  telemetryOptions: {
    metrics: buffer,
  },
});

What the source requires: TelemetryOptions.metrics has type MetricsExporterConfig, defined in packages/worker/src/runtime-options.ts as:

export type MetricsExporterConfig = {
  temporality?: 'cumulative' | 'delta';
  metricPrefix?: string;
  globalTags?: Record<string, string>;
  attachServiceName?: boolean;
} & (PrometheusMetricsExporter | OtelCollectorExporter | BufferedMetricsExporter);

export interface BufferedMetricsExporter {
  buffer: MetricsBuffer;
}

The SDK discriminates between exporters at runtime by checking property names: 'prometheus' in metrics (Prometheus), 'otel' in metrics (OTel), 'buffer' in metrics (buffered). The skill's metrics: buffer (passing a MetricsBuffer instance directly) will (a) fail to typecheck against the union and (b) fail the runtime discriminator — none of 'prometheus', 'otel', or 'buffer' is in a MetricsBuffer instance.

Correct form (confirmed by the SDK's own test in packages/test/src/test-runtime-buffered-metrics.ts):

const runtime = Runtime.install({
  telemetryOptions: {
    metrics: {
      buffer,
    },
  },
});

Impact: every reader who copies the setup or worked-example code will produce non-compiling TypeScript and, if they bypass type checking, code that the SDK silently ignores (no metrics buffered). This is functionally a high-impact correctness defect even though it's localized.

Suggested fix: change two code blocks. Setup section: wrap metrics: buffer in metrics: { buffer }. Worked example: same. Optionally add one sentence to "Setup" noting the shape: "MetricsBuffer is wired in via the BufferedMetricsExporter shape — { buffer } — alongside the Prometheus and OTel shapes."


Check 6 — Tone and scope audit

End-to-end read of buffered-metrics.md.

  • Pattern 1 (workaround / escape-hatch disclosure): 0 findings. The "Hard constraints" section names limits and supported alternatives in single sentences ("Drain on a timer…", "Buffered metrics and Prometheus/OTel are exclusive…"). The "Common mistakes" table is Wrong | Fix form — the acceptable "don't do X, do Y" pattern, not a workaround disclosure.
  • Pattern 2 (in-the-weeds rationale): none. The file is dense reference content with no design-history or behind-the-scenes prose.
  • Pattern 3 (multi-sentence bullets): all multi-sentence bullets in "Hard constraints" use the second sentence to give operational follow-through (e.g. "Size the buffer for your drain interval"), which is action-changing for an AI reader. No finding.
  • Pattern 4 (Public Preview admonition): the skill carries the [!NOTE] admonition at the top (lines 3–4) noting the API is "experimental in the TypeScript SDK; the APIs may change", which matches the typedoc description's "experimental feature and may be subject to change". Phrasing is acceptable.

Statistics

  • Typedoc citation tags counted: 29
  • Citation tags resolving cleanly: 29 / 29 (100%)
  • Token classes audited (reverse-grep): types, methods, properties, string literals, defaults, import paths — 0 misses across ~60 distinct tokens.
  • Regression pattern hits (universal + topic-specific): 0
  • Check 4 random sample: 10 / 10 (100%)
  • Out-of-sample correctness findings: 1 (the metrics: buffer shape, in two code blocks)
  • Check 6 pattern-1 findings: 0

Suggested follow-up

A single small commit fixes the only substantive finding:

  1. references/typescript/buffered-metrics.md line ~110: change metrics: buffer, to metrics: { buffer },.
  2. references/typescript/buffered-metrics.md line ~192: same change in the worked example.
  3. Optionally, line 99 prose or the "When to use which" table: a half-sentence noting the wrapper shape.

No re-authoring pass required. After the fix, the skill should be GO.

skill-sync[bot] and others added 2 commits May 14, 2026 18:48
Adds TypeScript reference for the MetricMeter API and MetricsBuffer,
covering all four instrument types (Counter, UpDownCounter, Gauge,
Histogram), how to access the meter from worker/workflow/activity
contexts, and how to drain BufferedMetricUpdate events for custom
export. Grounded against the TS SDK typedoc since the local docs
clone is silent on these APIs.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
@skill-temporal-developer-updater skill-temporal-developer-updater Bot requested a review from a team as a code owner May 14, 2026 19:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

0 participants