Skip to content

[dynamic control] Add pipeline config initialization from declarative config or failover to file#2766

Open
jackshirazi wants to merge 3 commits intoopen-telemetry:mainfrom
jackshirazi:policy23a
Open

[dynamic control] Add pipeline config initialization from declarative config or failover to file#2766
jackshirazi wants to merge 3 commits intoopen-telemetry:mainfrom
jackshirazi:policy23a

Conversation

@jackshirazi
Copy link
Copy Markdown
Contributor

Description:

Adding a method to read pipeline initialization from declarative config, or if not present failover to a file in yaml (or a json file).

The declarative config looks like

telemetry_policy:
  sources:
    - kind: opamp
      format: jsonkeyvalue
      location: vendor
      mappings:
        - sourceKey: sampling_rate
          policyType: trace_sampling_rate_policy

If declarative config is not present, the failover file is expected to have the node under telemetry_policy, ie starting at sources as the top-level node

Existing Issue(s):

#2546

Testing:

Added

Documentation:

Added

Outstanding items:

Wiring up the policy pipeline:

  • Generic: message -> provider -> policy -> policy handler -> implementer
  • eg change sampling rate message -> OpampPolicyProvider -> TraceSamplingRatePolicy -> PolicyStore -> TraceSamplingRatePolicyImplementer

Steps needed for the wiring:

  • configuring the pipeline
    • DONE in this PR
  • providers reading policies, eg OpampPolicyProvider, FilePolicyProvider, etc
    • DONE: OpampPolicyProvider
  • implementers applying policies, eg TraceSamplingRatePolicyImplementer
    • DONE: TraceSamplingRatePolicyImplementer
  • policy structures (eg TraceSamplingRatePolicy) that the provider converts messages into
    • DONE: TraceSamplingRatePolicy
  • registering config to policy structures (eg "trace_sampling_rate_policy" registered to TraceSamplingRatePolicy)
  • initializing policy classes (eg TraceSamplingRatePolicy needs to install a custom sampler)
    • DONE TraceSamplingRatePolicy initialization
  • activate configured providers (eg start OpampPolicyprovider reading from it's source)
  • register implementers for policies (eg a new TraceSamplingRatePolicy is applied by a TraceSamplingRatePolicyImplementer)
  • link the provider to processing policies and applying implementers

Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds initialization of dynamic-control’s telemetry policy pipeline from OpenTelemetry declarative config (preferred) with fallback to file-based YAML/JSON config.

Changes:

  • Add PolicyInitConfig loaders for declarative config (telemetry_policy.sources) and declarative-first fallback helpers (including via OpenTelemetry / ExtendedOpenTelemetry).
  • Add file-based loading from otel.java.experimental.telemetry.policy.init.yaml/json with YAML precedence and warning-on-failure behavior.
  • Add unit tests covering declarative vs file fallback selection and error cases.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 2 comments.

File Description
dynamic-control/src/main/java/io/opentelemetry/contrib/dynamic/policy/registry/PolicyInitConfig.java Implements declarative-first init config parsing and file fallback, plus reflective access to “general config” accessor.
dynamic-control/src/test/java/io/opentelemetry/contrib/dynamic/policy/registry/PolicyInitConfigTest.java Adds tests for YAML/JSON precedence, parse failures, and declarative-vs-file selection paths.

Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 2 out of 2 changed files in this pull request and generated 4 comments.

Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 3 out of 3 changed files in this pull request and generated no new comments.

Comments suppressed due to low confidence (1)

dynamic-control/src/main/java/io/opentelemetry/contrib/dynamic/policy/registry/json/JsonNodePolicyInitConfigParser.java:55

  • JSON init-config parsing now trims/normalizes optional location, but other string fields (kind, format, sourceKey, policyType) are still accepted with leading/trailing whitespace (and even whitespace-only, since requireText only checks isEmpty()). This can lead to hard-to-debug mismatches because mapping keys/policy types are used verbatim downstream. Consider trimming and validating non-blank in requireText (returning the trimmed value), and applying the same normalization to mapping fields for consistency with declarative parsing.
    JsonNode locationNode = objectNode.get("location");
    String location =
        locationNode != null && locationNode.isTextual()
            ? normalizeOptionalText(locationNode.asText())
            : null;

    JsonNode mappingsNode =
        requireArray(objectNode.get("mappings"), "Each source must define a 'mappings' array.");
    List<PolicySourceMappingConfig> mappings = new ArrayList<>();

Comment on lines +273 to +278
} catch (NoSuchMethodException
| IllegalAccessException
| InvocationTargetException
| RuntimeException ignored) {
// Fall through to backwards-compatible accessor below.
}
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should we scope this a bit more and add some logging? currently if the method exists but throws a NPE or something we'll fall through and the user might not know why

Suggested change
} catch (NoSuchMethodException
| IllegalAccessException
| InvocationTargetException
| RuntimeException ignored) {
// Fall through to backwards-compatible accessor below.
}
} catch (NoSuchMethodException ignored) {
// Method doesn't exist in this runtime version — fall through.
} catch (IllegalAccessException | InvocationTargetException e) {
logger.log(Level.WARNING, "Failed to invoke getGeneralConfig via reflection", e);
}


List<DeclarativeConfigProperties> mappingConfigs =
sourceConfig.getStructuredList(MAPPINGS_DECLARATIVE_KEY);
if (mappingConfigs == null) {
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should we also guard against empty maps?

Suggested change
if (mappingConfigs == null) {
if (mappingConfigs == null || mappingConfigs.isEmpty()) {

Comment on lines +33 to +43
static final String TELEMETRY_POLICY_DECLARATIVE_KEY = "telemetry_policy";
static final String SOURCES_DECLARATIVE_KEY = "sources";
static final String KIND_DECLARATIVE_KEY = "kind";
static final String FORMAT_DECLARATIVE_KEY = "format";
static final String LOCATION_DECLARATIVE_KEY = "location";
static final String MAPPINGS_DECLARATIVE_KEY = "mappings";
static final String SOURCE_KEY_DECLARATIVE_KEY = "sourceKey";
static final String POLICY_TYPE_DECLARATIVE_KEY = "policyType";
static final String POLICY_INIT_CONFIG_PROPERTY_JSON =
"otel.java.experimental.telemetry.policy.init.json";
static final String POLICY_INIT_CONFIG_PROPERTY_YAML =
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

any reason not to make these all private? I'm assuming ability to access them in tests but just checking

}

@Nullable
static ConfigProvider getConfigProvider(@Nullable OpenTelemetry openTelemetry) {
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
static ConfigProvider getConfigProvider(@Nullable OpenTelemetry openTelemetry) {
private static ConfigProvider getConfigProvider(@Nullable OpenTelemetry openTelemetry) {

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants