feat: Automate MVD deployment and E2E transfer tests

[alzcurda-i2cat]

Feature Request

Which Areas Would Be Affected?

Deployment, testing, automation scripts, and Kubernetes seed jobs.

Why Is the Feature Desired?

Currently, setting up the Minimum Viable Dataspace (MVD) locally and testing end-to-end data transfers requires running multiple manual CLI commands (KinD, Helm, kubectl) and manually interacting with the Bruno collection. This process is time-consuming, repetitive for developers, and error-prone.

Additionally, the identityhub-seed.yaml jobs sometimes fail abruptly during cluster startup if the Keycloak service takes a bit longer to be fully ready to issue tokens, causing race conditions.

Who will sponsor this feature?

N/A

Solution Proposal

We would like to contribute an automation layer to simplify the lifecycle without adding new technology stacks to the project.

Our idea is to provide:

1. Deployment automation: A mechanism to automatically orchestrate the cluster creation, docker build, and deploy steps (Traefik, Gateway API, Kustomize) in one command.
2. Transfer automation: A script to programmatically simulate a Consumer fetching the catalog, negotiating a contract, and executing an E2E data transfer.
3. Seed Job Resilience: Increase the backoffLimit in identityhub-seed.yaml and add an active retry-loop (wait mechanism) for fetching the Keycloak issuer token.

Architectural Question for the Maintainers:
Before we submit a PR, we would like to know the preferred approach:

• For deployment automation, should we wrap the setup steps in a pure Bash script or do you prefer extending build.gradle.kts with specific lifecycle tasks?
• For the E2E Transfer automation, should we write a Bash script using curl, or should we rely exclusively on the existing Java test suite in tests/end2end/?

We are happy to implement it according to your preferences!

[paullatzelsperger]

First off, as per our contribution guidelines, this really should be a discussion.

---

I'm taking the liberty of answering this one: we do not want an additional layer of complexity on top of MVD. It is already quite complex, and we want to keep it as bare-bones as possible. It should be trivial for any reasonably skilled developer to execute a few simple steps and bash commands.
In fact, we recently simplified things considerably by getting rid of Terraform/OpenTofu as deployment mechanism and a few other things because it was deemed unnecessarily complex.

Putting additional tools (bash scripts, or even python code) in to make deployment "easier" achieves the exact opposite while only moving the problem: it introduces complexity for zero gain.

MVD is not some ready-made product that you can incorporate in your application landscape, it is a playground for developers new to EDC. We explicitly want human manual interaction (i.e. running bash commands and clicking through Bruno collections) because we want people to get a feel for it.

<rant>
MVD is not an E2E test harness either, so any use case beyond "demo and playing around" requires some serious dev work, and automation would be just one aspect of it. You can automate to your heart's content, stick python and 3 other languages and frameworks on top of each other, but please do this in your own repo/fork and don't expect us to accept and maintain this in all eternity.
</rant>

---

That being said, if there is a bug such as race conditions or problems with the active-wait loop, feel free to file a focused bug report and a PR for that.

So to be blunt: we do not want any automation beyond what's already there, neither for deployment nor for data transfers, for the reasons stated above. Please to not raise any PRs to that effect.