feat: flatten dict values passed to set_span_attribute

[shreyas-n-harness]

Why

set_span_attribute("agent", {"action": "generate", "name": "devops"}) silently dropped the value — OpenTelemetry rejects mapping attribute values outright. The workaround was json.dumps() at the call site, which left the backend with an opaque string that cannot be queried or aggregated per field.

What

Dict (Mapping) values are now accepted by set_span_attribute / set_span_attributes and expanded into dot-notation attributes, so every leaf stays individually queryable:

set_span_attribute("agent", {"action": "generate", "model": {"name": "gemini-2.0"}})
# exported as: agent.action="generate", agent.model.name="gemini-2.0"

Nothing is serialized on the caller's thread. The dict is parked in a thread-safe registry keyed by span context, and a new FlattenDictSpanProcessor drains it at on_end and writes the flat keys onto the span. The processor sits outermost in the pipeline (wrapping GenAiPayloadScrubSpanProcessor) so payload scrubbing and attribute-based exclusion match on the flattened keys.

The registry is keyed by span context rather than object identity because Span.end() hands on_end a fresh ReadableSpan snapshot, not the recording Span the enrichment helper saw. It is also bounded (2048 spans, oldest evicted) so a span that never ends cannot leak its entry.

Flatten rules

Case	Result
str / bool / int / float leaf	kept as the native OTel type
None leaf	skipped
Any other object	str(value)
List of same-typed scalars	OTel array attribute at that key
List of dicts or mixed types	JSON string at that key
Nesting deeper than 3 levels	JSON string at the depth-3 key
Flattened key already set on the span	skipped — the explicit value wins

At most 32 leaf attributes per dict; the remainder is dropped with a debug log.

Configuration

Flattening is enabled by default. No env var is required — dict values are flattened automatically. To disable, set an explicit opt-out:

Variable	Default	Effect
HARNESS_SPAN_ATTRIBUTE_FLATTEN_ENABLED	on (unset = on)	Only false disables flattening; any other value (including empty) leaves it enabled. When disabled, OTel rejects dict values as before.
HARNESS_SPAN_ATTRIBUTE_FLATTEN_RAW_JSON	off	true additionally stores the whole dict as JSON under the original key

Legacy HA_ / AT_ / TA_ prefixes are also supported for both flags.

Changes

• new src/harness_sdk/flatten_dict_registry.py — thread-safe, bounded span-to-pending-dicts registry plus env flag helpers (opt-out semantics)
• new src/harness_sdk/flatten_dict_span_processor.py — decorator SpanProcessor that flattens at on_end
• src/harness_sdk/span_enrichment.py — accept Mapping values and route them to the registry
• src/harness_sdk/plugins/builtin/pipeline.py — wrap the chain outermost when flattening is enabled
• src/harness_sdk/agent_init.py — same wrapping for the console-exporter path
• README.md / AGENTS.md / CLAUDE.md — document behavior, rules, and env flags

Existing non-dict behavior is unchanged: scalars and sequences still go straight to span.set_attribute.

Test plan

• python -m pytest test/flatten_dict_span_processor_test.py test/span_enrichment_test.py test/plugins/builtin/test_pipeline.py -q — 32 passed
• ./scripts/run-unit-tests.sh — 270 passed; the 13 failures (openai, grpc) reproduce identically on main and are unrelated
• pylint src/harness_sdk --disable=C,R --ignore-patterns=config_pb2.py — 10.00/10
• QA smoke: span with set_span_attribute("agent", {"action": "generate", "name": "devops"}) exported flattened keys to QA ingest

[CLAassistant]

Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
_{You have signed the CLA already but the status is still pending? Let us recheck it.}

[t-santoshsahu]

this should be configurable?

[shreyas-n-harness]

Makes sense, updated