-
Notifications
You must be signed in to change notification settings - Fork 529
feat(flagd): Flagd plugin #7753
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Draft
aepfli
wants to merge
19
commits into
Flagsmith:main
Choose a base branch
from
aepfli:feat/flagd-sync-endpoint
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Draft
Changes from all commits
Commits
Show all changes
19 commits
Select commit
Hold shift + click to select a range
b904bb0
feat: add flagd-compatible HTTP sync endpoint
aepfli 2439971
docs: reframe flagd-sync as a management UI for flagd
aepfli fe2c118
chore: add local-dev compose stack (Flagsmith + flagd + Postgres)
aepfli d84fb46
fix: bootstrap_flagd_local sets a working admin password
aepfli 2b1d75f
fix: don't override the baked-in flagsmith healthcheck
aepfli 05f9c05
fix: chown flagd-config volume before bootstrap writes to it
aepfli c0fddf1
fix: wrap flagd in an alpine launcher for shell-based bootstrap
aepfli b16a8ec
fix: copy flagd from /flagd-build (upstream binary path)
aepfli ba7f8c1
fix: use flagd's native --sources + Authorization-header auth
aepfli ca6b6fd
docs: bring flagd integration guides in line with v0.13 + authHeader
aepfli b5a4005
feat: drop redundant 'off' variant from flagd output
aepfli b07cd7a
feat(flagd): per-project opt-in, admin toggle, diagnostics, scheduled…
aepfli 90710c4
feat(flagd): preserve override-value distinct from control
aepfli 2e51f54
refactor(flagd): drop 'off' variant; override value is the sole signal
aepfli 9f3546d
feat(flagd): inline single-use segments; only share via \$evaluators
aepfli a2fabcc
docs: refresh flagd guides for opt-in gate, diagnostics, new variants
aepfli c2e4d47
Merge branch 'main' of https://github.com/Flagsmith/flagsmith into fe…
aepfli 9764d42
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] e88d77e
refactor(flagd): extract helpers to satisfy ruff C901 complexity limit
aepfli File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -25,3 +25,4 @@ src/CI_COMMIT_SHA | |
| *.iml | ||
|
|
||
| AGENTS.local.md | ||
| .claude/ | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1 @@ | ||
| default_app_config = "integrations.flagd.apps.FlagdConfig" |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,6 @@ | ||
| from core.apps import BaseAppConfig | ||
|
|
||
|
|
||
| class FlagdConfig(BaseAppConfig): | ||
| name = "integrations.flagd" | ||
| default = True |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,12 @@ | ||
| FLAGD_TRANSLATOR_VERSION = "v1" | ||
| FLAGD_SCHEMA_URL = "https://flagd.dev/schema/v0/flags.json" | ||
|
|
||
| VARIANT_CONTROL = "control" | ||
|
|
||
| DEFAULT_IDENTITY_OVERRIDE_LIMIT = 100 | ||
|
|
||
| WARNING_REGEX_UNSUPPORTED = "regex_unsupported" | ||
| WARNING_UNKNOWN_OPERATOR = "unknown_operator" | ||
| WARNING_MALFORMED_VALUE = "malformed_value" | ||
| WARNING_IDENTITY_OVERRIDE_LIMIT = "identity_override_limit_exceeded" | ||
| WARNING_DISABLED_OVERRIDE_NO_OP = "disabled_override_no_op" |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,129 @@ | ||
| """ | ||
| Per-environment translation diagnostics. | ||
|
|
||
| Reuses the same translation pipeline as the sync endpoint to surface | ||
| issues that would degrade flagd consumption (type mismatches, unsupported | ||
| operators, identity-override cap overflow). Operators can curl the | ||
| diagnostics endpoint to audit an environment, alert on warnings in | ||
| their CI pipeline, or build their own dashboard against the JSON. | ||
|
|
||
| This module deliberately lives alongside ``services.py`` rather than | ||
| inside it so the diagnostic path can evolve independently of the | ||
| hot-path sync builder. | ||
| """ | ||
|
|
||
| from __future__ import annotations | ||
|
|
||
| from typing import Any | ||
|
|
||
| import structlog | ||
| from django.conf import settings | ||
|
|
||
| from integrations.flagd.constants import ( | ||
| DEFAULT_IDENTITY_OVERRIDE_LIMIT, | ||
| FLAGD_TRANSLATOR_VERSION, | ||
| ) | ||
| from integrations.flagd.translators.flag import feature_state_to_flagd_flag | ||
| from integrations.flagd.translators.segment import ( | ||
| segment_to_jsonlogic, | ||
| slugify_name, | ||
| slugify_segment_name, | ||
| ) | ||
| from integrations.flagd.translators.type_check import detect_type_mismatch | ||
| from integrations.flagd.types import TranslationWarning | ||
| from util.engine_models.environments.models import EnvironmentModel | ||
| from util.mappers.engine import map_environment_to_engine | ||
|
|
||
| logger = structlog.get_logger("flagd_sync") | ||
|
|
||
|
|
||
| def diagnose_environment(environment: Any) -> dict[str, Any]: | ||
| """ | ||
| Run the translator over ``environment`` purely to collect warnings. | ||
| Returns a structured report keyed by feature name plus an | ||
| environment-level summary. | ||
| """ | ||
| engine_environment: EnvironmentModel = map_environment_to_engine( | ||
| environment, with_integrations=False | ||
| ) | ||
| project = environment.project | ||
| identity_override_limit = getattr( | ||
| settings, | ||
| "FLAGD_SYNC_IDENTITY_OVERRIDE_LIMIT", | ||
| DEFAULT_IDENTITY_OVERRIDE_LIMIT, | ||
| ) | ||
|
|
||
| # Translate segments once so flag translation can reuse the cache. | ||
| segments = engine_environment.project.segments | ||
| segment_warnings: list[TranslationWarning] = [] | ||
| segment_keys: dict[int, str] = {} | ||
| segment_targeting: dict[int, Any] = {} | ||
| used: set[str] = set() | ||
| for segment in segments: | ||
| key = slugify_segment_name(segment.name, taken=used) | ||
| used.add(key) | ||
| segment_keys[segment.id] = key | ||
| segment_targeting[segment.id] = segment_to_jsonlogic( | ||
| segment, warnings=segment_warnings | ||
| ) | ||
|
|
||
| default_feature_states = [ | ||
| fs for fs in engine_environment.feature_states if fs.feature_segment is None | ||
| ] | ||
|
|
||
| features: list[dict[str, Any]] = [] | ||
| for fs in default_feature_states: | ||
| per_feature_warnings: list[TranslationWarning] = [] | ||
|
|
||
| # Type-consistency check — flagd's JSON Schema requires variants | ||
| # of one flag to share a type. This catches it pre-emptively so | ||
| # operators see a clear warning rather than a TYPE_MISMATCH at | ||
| # evaluation time. We include segment + identity override values | ||
| # because an override can legitimately introduce a different type | ||
| # (e.g. a number override on a string flag). | ||
| per_feature_warnings.extend( | ||
| detect_type_mismatch( | ||
| fs, | ||
| segments=segments, | ||
| identity_overrides=engine_environment.identity_overrides, | ||
| ) | ||
| ) | ||
|
|
||
| # Run the regular translator just to capture operator-level | ||
| # warnings (REGEX skipped, identity-override cap, malformed | ||
| # values, etc.). We don't keep the resulting flag here — the | ||
| # sync endpoint owns that path. | ||
| feature_state_to_flagd_flag( | ||
| fs, | ||
| feature_key=fs.feature.name, | ||
| segments=segments, | ||
| segment_targeting=segment_targeting, | ||
| segment_keys=segment_keys, | ||
| identity_overrides=engine_environment.identity_overrides, | ||
| identity_override_limit=identity_override_limit, | ||
| warnings=per_feature_warnings, | ||
| ) | ||
|
|
||
| if per_feature_warnings: | ||
| features.append( | ||
| { | ||
| "name": fs.feature.name, | ||
| "warnings": list(per_feature_warnings), | ||
| } | ||
| ) | ||
|
|
||
| flag_set_id = f"{slugify_name(project.name)}/{slugify_name(environment.name)}" | ||
| return { | ||
| "flagSetId": flag_set_id, | ||
| "translatorVersion": FLAGD_TRANSLATOR_VERSION, | ||
| "environmentWarnings": list(segment_warnings), | ||
| "features": features, | ||
| "summary": { | ||
| "featuresWithWarnings": len(features), | ||
| "totalWarnings": sum(len(f["warnings"]) for f in features) | ||
| + len(segment_warnings), | ||
| }, | ||
| } | ||
|
|
||
|
|
||
| __all__ = ("diagnose_environment",) |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,11 @@ | ||
| class FlagdTranslationError(Exception): | ||
| """Base class for translation errors.""" | ||
|
|
||
|
|
||
| class UntranslatableConditionError(FlagdTranslationError): | ||
| """Raised when a condition cannot be expressed in JsonLogic.""" | ||
|
|
||
| def __init__(self, reason: str, operator: str | None = None) -> None: | ||
| super().__init__(reason) | ||
| self.reason = reason | ||
| self.operator = operator |
Empty file.
Empty file.
155 changes: 155 additions & 0 deletions
155
api/integrations/flagd/management/commands/bootstrap_flagd_local.py
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,155 @@ | ||
| """ | ||
| Idempotent bootstrap helper for local flagd development. | ||
|
|
||
| Creates (or reuses) an Organisation, Project, Environment, and | ||
| server-side ``EnvironmentAPIKey``, then writes the resulting key to | ||
| stdout — and optionally to a shell-sourceable env file. Designed to be | ||
| run from a docker-compose init service so flagd can pick up the key | ||
| without any manual UI clicks. | ||
| """ | ||
|
|
||
| from __future__ import annotations | ||
|
|
||
| import argparse | ||
| from pathlib import Path | ||
| from typing import Any | ||
|
|
||
| from django.core.management.base import BaseCommand | ||
|
|
||
| from environments.models import Environment, EnvironmentAPIKey | ||
| from integrations.flagd.models import FlagdProjectConfiguration | ||
| from organisations.models import Organisation, OrganisationRole | ||
| from projects.models import Project | ||
| from users.models import FFAdminUser | ||
|
|
||
|
|
||
| class Command(BaseCommand): | ||
| help = "Create or reuse a local Flagsmith env and emit a server-side key for flagd." | ||
|
|
||
| def add_arguments(self, parser: argparse.ArgumentParser) -> None: | ||
| parser.add_argument( | ||
| "--organisation", | ||
| default="local-dev", | ||
| help="Organisation name (default: local-dev).", | ||
| ) | ||
| parser.add_argument( | ||
| "--project", | ||
| default="local-dev", | ||
| help="Project name (default: local-dev).", | ||
| ) | ||
| parser.add_argument( | ||
| "--environment", | ||
| default="development", | ||
| help="Environment name (default: development).", | ||
| ) | ||
| parser.add_argument( | ||
| "--api-key-name", | ||
| default="flagd-local", | ||
| help="EnvironmentAPIKey label (default: flagd-local).", | ||
| ) | ||
| parser.add_argument( | ||
| "--api-key", | ||
| default=None, | ||
| help=( | ||
| "Force the EnvironmentAPIKey value to a specific server-side " | ||
| "key (must start with `ser.`). Useful for local-dev where the " | ||
| "key must be known at compose-parse time. Default: auto-generate." | ||
| ), | ||
| ) | ||
| parser.add_argument( | ||
| "--admin-email", | ||
| default="admin@example.com", | ||
| help="Email for the local admin user (default: admin@example.com).", | ||
| ) | ||
| parser.add_argument( | ||
| "--admin-password", | ||
| default="admin", | ||
| help=( | ||
| "Password to set on the local admin user. Always overwritten " | ||
| "on each run so the local dev experience stays predictable. " | ||
| "Default: admin." | ||
| ), | ||
| ) | ||
| parser.add_argument( | ||
| "--output", | ||
| type=Path, | ||
| help=( | ||
| "If given, write `FLAGSMITH_SERVER_KEY=...` to this file. " | ||
| "Useful for sourcing from a docker-compose init service." | ||
| ), | ||
| ) | ||
|
|
||
| def handle(self, *args: Any, **options: Any) -> None: | ||
| organisation, _ = Organisation.objects.get_or_create( | ||
| name=options["organisation"] | ||
| ) | ||
| project, _ = Project.objects.get_or_create( | ||
| name=options["project"], organisation=organisation | ||
| ) | ||
| environment, _ = Environment.objects.get_or_create( | ||
| name=options["environment"], project=project | ||
| ) | ||
| # Local-dev expects flagd to be reachable out of the box. | ||
| FlagdProjectConfiguration.objects.update_or_create( | ||
| project=project, defaults={"enabled": True} | ||
| ) | ||
|
|
||
| # Create or refresh the local admin user so the operator can | ||
| # log into the Flagsmith UI without juggling password-reset | ||
| # links. The user is attached to the organisation so they land | ||
| # straight in the bootstrapped project on login. | ||
| admin_email = options["admin_email"] | ||
| admin_password = options["admin_password"] | ||
| admin = FFAdminUser.objects.filter(email=admin_email).first() | ||
| if admin is None: | ||
| admin = FFAdminUser.objects.create_superuser( # type: ignore[no-untyped-call] | ||
| email=admin_email, | ||
| is_active=True, | ||
| password=admin_password, | ||
| ) | ||
| else: | ||
| admin.set_password(admin_password) | ||
| admin.is_active = True | ||
| admin.is_staff = True | ||
| admin.is_superuser = True | ||
| admin.save() | ||
| if not admin.belongs_to(organisation.id): | ||
| admin.add_organisation(organisation, role=OrganisationRole.ADMIN) | ||
|
|
||
| forced_key: str | None = options.get("api_key") | ||
| if forced_key and not forced_key.startswith("ser."): | ||
| raise ValueError( | ||
| "--api-key must start with 'ser.' to be accepted by the flagd " | ||
| f"sync endpoint; got {forced_key!r}." | ||
| ) | ||
|
|
||
| api_key = ( | ||
| EnvironmentAPIKey.objects.filter( | ||
| environment=environment, name=options["api_key_name"] | ||
| ) | ||
| .order_by("created_at") | ||
| .first() | ||
| ) | ||
| if api_key is None: | ||
| create_kwargs: dict[str, Any] = { | ||
| "environment": environment, | ||
| "name": options["api_key_name"], | ||
| } | ||
| if forced_key: | ||
| create_kwargs["key"] = forced_key | ||
| api_key = EnvironmentAPIKey.objects.create(**create_kwargs) | ||
| elif forced_key and api_key.key != forced_key: | ||
| api_key.key = forced_key | ||
| api_key.save() | ||
|
|
||
| line = f"FLAGSMITH_SERVER_KEY={api_key.key}" | ||
| self.stdout.write(line) | ||
| self.stdout.write( | ||
| self.style.SUCCESS(f"Admin user: {admin_email} / {admin_password}") | ||
| ) | ||
|
|
||
| output: Path | None = options.get("output") | ||
| if output is not None: | ||
| output.parent.mkdir(parents=True, exist_ok=True) | ||
| output.write_text(line + "\n") | ||
| self.stdout.write(self.style.SUCCESS(f"Wrote {output}")) |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,24 @@ | ||
| import prometheus_client | ||
|
|
||
| flagsmith_flagd_sync_requests_total = prometheus_client.Counter( | ||
| "flagsmith_flagd_sync_requests_total", | ||
| "Number of flagd HTTP sync requests served by the Flagsmith API.", | ||
| ["status"], | ||
| ) | ||
|
|
||
| flagsmith_flagd_document_build_seconds = prometheus_client.Histogram( | ||
| "flagsmith_flagd_document_build_seconds", | ||
| "Wall-clock time spent translating an environment to a flagd document.", | ||
| ) | ||
|
|
||
| flagsmith_flagd_document_size_bytes = prometheus_client.Histogram( | ||
| "flagsmith_flagd_document_size_bytes", | ||
| "Size in bytes of the flagd document returned by the sync endpoint.", | ||
| buckets=(512, 4096, 32_768, 262_144, 2_097_152), | ||
| ) | ||
|
|
||
| flagsmith_flagd_translation_warnings_total = prometheus_client.Counter( | ||
| "flagsmith_flagd_translation_warnings_total", | ||
| "Translation warnings emitted while building flagd documents.", | ||
| ["reason"], | ||
| ) |
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
i feel like i need to revert this.