Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 15 additions & 0 deletions .ai/.claude-plugin/plugin.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
{

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(curiosity): why is this needed?

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It enables installation via Claude's plugin management system.

claude plugin marketplace add huggingface/diffusers
claude plugin install diffusers@diffusers-skills --scope project

"name": "diffusers",
"description": "Conventions and task skills for contributing to diffusers",
"author": {
"name": "The Hugging Face team"
},
"homepage": "https://huggingface.co/docs/diffusers/main/en/conceptual/contribution#coding-with-ai-agents",
"repository": "https://github.com/huggingface/diffusers",
"license": "Apache-2.0",
"keywords": [
"diffusers",
"diffusion-models",
"contributing"
]
}
6 changes: 6 additions & 0 deletions .ai/.codex-plugin/plugin.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
{
"name": "diffusers",
"version": "0.1.0",
"description": "Conventions and task skills for contributing to diffusers",
"skills": "./skills/"
}
75 changes: 52 additions & 23 deletions .ai/AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,45 +2,74 @@

## Setup

- Local Claude Code agents: run `make claude` after cloning to wire the [skills](#skills) under `.claude/`.
- Local OpenAI Codex agents: run `make codex` after cloning to wire the [skills](#skills) under `.agents/`.
We recommend developing in a virtual environment managed by [uv](https://docs.astral.sh/uv/):

## Coding style
```bash
uv venv && source .venv/bin/activate
uv pip install -e . # provides diffusers-cli
```

Strive to write code as simple and explicit as possible.
List the available skills, and what each one is for, with:

- Prefer inlining small helper/utility functions over factoring them out — a reader should be able to follow the full flow without jumping between functions. If a private helper has only one caller, inlining it at the call site is usually the cleaner choice.
- No defensive code, unused code paths, or legacy stubs — do not add fallback paths, safety checks, or configuration options "just in case"; do not carry unused method parameters "for API consistency", backwards-compatibility aliases for names that never shipped, or deprecation shims for code that was never released. When porting from a research repo, delete training-time code paths, experimental flags, and ablation branches entirely — only keep the inference path you are actually integrating.
- Do not guess user intent and silently correct behavior. Make the expected inputs clear in the docstring, and raise a concise error for unsupported cases rather than adding complex fallback logic.
```bash
diffusers-cli skills list
```

---
Install them with:

## Code formatting
```bash
diffusers-cli skills add <skill name> # install one, or --all for every skill
# --claude / --codex / --cursor to pick one target
```

- `make style` and `make fix-copies` should be run before opening a PR
`diffusers-cli skills update` refreshes what you installed.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Where is this handled?

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@DN6 👀


Skills used to be installed through the `Makefile`, which symlinked the project-level `.claude/skills` and
`.agents/skills` at `.ai/skills` in this repo. Remove those links if they exist — otherwise the install writes through
them into `.ai/` itself:

```bash
rm -f .claude/skills .agents/skills
```

At the start of a session, confirm the setup and tell the user what you find:

```bash
python utils/check_ai.py # guides and skills are consistent

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's add check_ai.py to our CI.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it runs via make quality now.

diffusers-cli skills list # which skills are installed
```

### Copied Code
Claude Code and Codex can also install via plugins

- Many classes are kept in sync with a source via a `# Copied from ...` header comment
- Do not edit a `# Copied from` block directly — run `make fix-copies` to propagate changes from the source
- Remove the header to intentionally break the link
```bash
claude plugin marketplace add huggingface/diffusers
claude plugin install diffusers@diffusers-skills --scope project

codex plugin marketplace add huggingface/diffusers # then install from the Plugins Directory
```


## Code formatting

- `make style` and `make fix-copies` should be run before opening a PR

## Reference guides

- **Models** — see [models.md](models.md) for model conventions, attention pattern, implementation rules, dependencies, and gotchas. For adding or converting a model, use the [model-integration](./skills/model-integration/SKILL.md) skill.
- **Pipelines** — see [pipelines.md](pipelines.md) for pipeline conventions, patterns, and gotchas.
- **Modular pipelines** — see [modular.md](modular.md) for modular pipeline conventions, patterns, and gotchas.
- **Tests** — see [testing.md](testing.md) for test conventions: required test layers, tester mixins, and dummy-component rules.
- **Coding style** — see [code_style.md](references/code_style.md) for how code should read, and the `# Copied from` rules.
- **Models** — see [models.md](references/models.md) for model conventions, attention pattern, implementation rules, dependencies, and gotchas. For adding or converting a model, use the [model-integration](skills/model-integration/SKILL.md) skill.
- **Pipelines** — see [pipelines.md](references/pipelines.md) for pipeline conventions, patterns, and gotchas.
- **Modular pipelines** — see [modular.md](references/modular.md) for modular pipeline conventions, patterns, and gotchas.
- **Tests** — see [testing.md](references/testing.md) for test conventions: required test layers, tester mixins, and dummy-component rules.

## Skills

Task-specific guides live in `.ai/skills/` and are loaded on demand by AI agents. Available skills include:

- [model-integration](./skills/model-integration/SKILL.md) (adding/converting pipelines)
- [custom-blocks](./skills/custom-blocks/SKILL.md) (packaging a `ModularPipelineBlocks` subclass for the Hub)
- [diffusers-cli](./skills/diffusers-cli/SKILL.md) (running pipelines, inspecting schemas, and using the Diffusers CLI)
- [self-review](./skills/self-review/SKILL.md) (pre-PR self-review against the project rules)
- [model-integration](skills/model-integration/SKILL.md) (adding/converting pipelines)
- [custom-blocks](skills/custom-blocks/SKILL.md) (packaging a `ModularPipelineBlocks` subclass for the Hub)
- [diffusers-cli](skills/diffusers-cli/SKILL.md) (running pipelines, inspecting schemas, and using the Diffusers CLI)
- [self-review](skills/self-review/SKILL.md) (pre-PR self-review against the project rules)

## Self-review before a PR

Before opening a PR, run self-review against [review-rules.md](review-rules.md). The [self-review skill](skills/self-review/SKILL.md) runs this as the same pass the `@claude` CI reviewer uses. Share the final report on the PR (description or comment) — see the skill for details.
Before opening a PR, run self-review against [review-rules.md](references/review-rules.md). The [self-review skill](skills/self-review/SKILL.md) runs this as the same pass the `@claude` CI reviewer uses. Share the final report on the PR (description or comment) — see the skill for details.
16 changes: 16 additions & 0 deletions .ai/plugin.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
{
"$schema": "https://agent-plugins.org/schemas/1.0.0/plugin.schema.json",
"name": "diffusers",
"description": "Conventions and task skills for contributing to diffusers",
"author": {
"name": "The Hugging Face team"
},
"homepage": "https://huggingface.co/docs/diffusers/main/en/conceptual/contribution#coding-with-ai-agents",
"repository": "https://github.com/huggingface/diffusers",
"license": "Apache-2.0",
"keywords": [
"diffusers",
"diffusion-models",
"contributing"
]
}
17 changes: 17 additions & 0 deletions .ai/references/code_style.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
# Coding style and copied code

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this being factored out? Nice.


Shared reference for how diffusers code should read, and for the `# Copied from` mechanism.

## Coding style

Strive to write code as simple and explicit as possible.

- Prefer inlining small helper/utility functions over factoring them out — a reader should be able to follow the full flow without jumping between functions. If a private helper has only one caller, inlining it at the call site is usually the cleaner choice.
- No defensive code, unused code paths, or legacy stubs — do not add fallback paths, safety checks, or configuration options "just in case"; do not carry unused method parameters "for API consistency", backwards-compatibility aliases for names that never shipped, or deprecation shims for code that was never released. When porting from a research repo, delete training-time code paths, experimental flags, and ablation branches entirely — only keep the inference path you are actually integrating.
- Do not guess user intent and silently correct behavior. Make the expected inputs clear in the docstring, and raise a concise error for unsupported cases rather than adding complex fallback logic.

## Copied code

- Many classes are kept in sync with a source via a `# Copied from ...` header comment
- Do not edit a `# Copied from` block directly — run `make fix-copies` to propagate changes from the source
- Remove the header to intentionally break the link
1 change: 0 additions & 1 deletion .ai/models.md → .ai/references/models.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,6 @@
# Model conventions and rules

Shared reference for model-related conventions, patterns, and gotchas.
Linked from `AGENTS.md`, `skills/model-integration/SKILL.md`, and `review-rules.md`.

## Coding style

Expand Down
2 changes: 1 addition & 1 deletion .ai/modular.md → .ai/references/modular.md
Original file line number Diff line number Diff line change
Expand Up @@ -227,7 +227,7 @@ Preset files never import from each other: each `modular_blocks_*.py` self-assem

## InputParam / OutputParam

Use `.template("<name>")` for params with a canonical meaning (`prompt`, `negative_prompt`, `image`, `generator`, `num_inference_steps`, `latents`, `prompt_embeds`, `images`, `videos`, etc.) — the template carries a vetted description and type hint. The full registry lives in [`src/diffusers/modular_pipelines/modular_pipeline_utils.py`](../src/diffusers/modular_pipelines/modular_pipeline_utils.py) (`INPUT_PARAM_TEMPLATES`, `OUTPUT_PARAM_TEMPLATES`); read that file rather than relying on a hardcoded list here, since names get added.
Use `.template("<name>")` for params with a canonical meaning (`prompt`, `negative_prompt`, `image`, `generator`, `num_inference_steps`, `latents`, `prompt_embeds`, `images`, `videos`, etc.) — the template carries a vetted description and type hint. The full registry lives in [`src/diffusers/modular_pipelines/modular_pipeline_utils.py`](https://github.com/huggingface/diffusers/blob/main/src/diffusers/modular_pipelines/modular_pipeline_utils.py) (`INPUT_PARAM_TEMPLATES`, `OUTPUT_PARAM_TEMPLATES`); read that file rather than relying on a hardcoded list here, since names get added.

For params that don't match a template (model-specific names, custom semantics), declare the field directly:

Expand Down
3 changes: 1 addition & 2 deletions .ai/pipelines.md → .ai/references/pipelines.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,6 @@
# Pipeline conventions and rules

Shared reference for pipeline-related conventions, patterns, and gotchas.
Linked from `AGENTS.md`, `skills/model-integration/SKILL.md`, and `review-rules.md`.

> **Prefer modular for new pipelines.** [Modular Diffusers](modular.md) is the preferred way to add a new pipeline; the standard `DiffusionPipeline` covered below is still supported but is no longer the default. We prefer modular especially for models that don't fit a fixed task-based structure (e.g. modality baked into the checkpoint) or that are actively evolving. The conventions below apply when you do build or review a standard pipeline.

Expand Down Expand Up @@ -71,7 +70,7 @@ src/diffusers/pipelines/<model>/

4. **Subclassing an existing pipeline for a variant.** Don't use an existing pipeline class (e.g. `FluxPipeline`) to override another (e.g. `FluxImg2ImgPipeline`) inside the core `src/` codebase. Each pipeline lives in its own file with its own class, even if it shares 90% of `__call__` with a sibling. Convention across diffusers — flux, sdxl, wan, qwenimage — is duplicated `__call__` between img2img / text2img / inpaint variants, not subclassing. Reuse private utilities (shared schedulers, prep functions) but not the pipeline class itself.

5. **Copying a method from another pipeline without `# Copied from`.** When you reuse a method like `encode_prompt`, `prepare_latents`, `check_inputs`, or `_prepare_latent_image_ids` from another pipeline, add a `# Copied from` annotation so `make fix-copies` keeps the two in sync. Forgetting it means future refactors to the source drift away from your copy silently — and reviewers waste time spotting near-identical code that should have been linked. The annotation grammar (decorator placement, rename syntax with `with old->new`, etc.) is implemented in [`utils/check_copies.py`](../utils/check_copies.py) — read it for the exact rules.
5. **Copying a method from another pipeline without `# Copied from`.** When you reuse a method like `encode_prompt`, `prepare_latents`, `check_inputs`, or `_prepare_latent_image_ids` from another pipeline, add a `# Copied from` annotation so `make fix-copies` keeps the two in sync. Forgetting it means future refactors to the source drift away from your copy silently — and reviewers waste time spotting near-identical code that should have been linked. The annotation grammar (decorator placement, rename syntax with `with old->new`, etc.) is implemented in [`utils/check_copies.py`](https://github.com/huggingface/diffusers/blob/main/utils/check_copies.py) — read it for the exact rules.

6. **Be deliberate about methods on the pipeline.** `__call__` is the user's mental model. The methods on the class are how they navigate it. Diffusers convention (flux, sdxl, wan, qwenimage) is a flat class body of public lifecycle methods (`__init__`, `check_inputs`, `encode_prompt`, `prepare_latents`, `__call__`). Two principles, not strict rules — use judgment:
- **If a method is called from `__call__`, and it's a step in the pipeline lifecycle, make it public.** Each call from `__call__` should correspond to a step a user can identify: either a standard one (`encode_prompt`, `prepare_latents`, `set_timesteps`, …) or a pipeline-specific one (`prepare_src_latents`, `prepare_reference_audio_latents`, …). Don't gate these behind a `_`; they're part of the pipeline's API surface alongside their standard siblings.
Expand Down
File renamed without changes.
8 changes: 4 additions & 4 deletions .ai/review-rules.md → .ai/references/review-rules.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,16 +3,16 @@
Review-specific rules for Claude. Focus on correctness — style is handled by ruff.

Before reviewing, read and apply the guidelines in:
- [AGENTS.md](AGENTS.md) — coding style, copied code
- [code_style.md](code_style.md) — coding style, copied code
- [models.md](models.md) — model conventions, attention pattern, implementation rules, dependencies, gotchas
- [pipelines.md](pipelines.md) — pipeline conventions, coding style, gotchas
- [modular.md](modular.md) — modular pipeline conventions, patterns, common mistakes
- [testing.md](testing.md) — test conventions: required test layers, tester mixins, dummy-component rules. When a PR adds or changes tests, check them against this guide.
- [skills/model-integration/pitfalls.md](skills/model-integration/pitfalls.md) — known pitfalls causing numerical discrepancies between the reference implementation and the diffusers port (dtype mismatches, config assumptions, etc.)
- [pitfalls.md](pitfalls.md) — known pitfalls causing numerical discrepancies between the reference implementation and the diffusers port (dtype mismatches, config assumptions, etc.)

## Common mistakes

Common mistakes are covered in the common-mistakes / gotcha sections in [AGENTS.md](AGENTS.md), [models.md](models.md), [pipelines.md](pipelines.md), and [modular.md](modular.md). Additionally, watch for below patterns that aren't covered there:
Common mistakes are covered in the common-mistakes / gotcha sections in [code_style.md](code_style.md), [models.md](models.md), [pipelines.md](pipelines.md), and [modular.md](modular.md). Additionally, watch for below patterns that aren't covered there:

- **Ephemeral context.** Comments, docstrings, and files that only made sense to the current PR's author or reviewer don't help a future reader/user/developer. Examples: `# per reviewer comment on PR #NNNN`, `# as discussed in review`, `# TODO from offline chat`, debug printouts. Same for files: parity harnesses, comparison scripts, anything in `scripts/` with hardcoded developer paths or imports from the reference repo. State the *reason* so the comment stands alone, or drop it.

Expand All @@ -21,7 +21,7 @@ Common mistakes are covered in the common-mistakes / gotcha sections in [AGENTS.
A PR can leave existing docs stale or surface a pattern worth recording. Scan the docs related to what the PR touches and flag updates as a **suggestions / additional info** section (not blocking):

- **Usage docs.** New or changed public behavior — a new pipeline/model, a new argument, changed defaults, a renamed API — should have matching updates in `docs/`, docstrings, and examples. Flag any that now describe outdated behavior or that are missing for the new surface.
- **Agent docs.** If the review turns up a rule, pattern, or common gotcha that isn't written down yet — especially one the author got wrong or that you had to reason out — propose adding it to the relevant agent guide ([AGENTS.md](AGENTS.md), [models.md](models.md), [pipelines.md](pipelines.md), [modular.md](modular.md), a skill, or this file) so the next contributor/agent gets it for free instead of repeating the mistake. Human review comments on the PR are a good source for these: if a human reviewer pointed something out and your review missed it, that usually indicates a doc gap — figure out what's missing and propose the addition.
- **Agent docs.** If the review turns up a rule, pattern, or common gotcha that isn't written down yet — especially one the author got wrong or that you had to reason out — propose adding it to the relevant agent guide ([code_style.md](code_style.md), [models.md](models.md), [pipelines.md](pipelines.md), [modular.md](modular.md), a skill, or this file) so the next contributor/agent gets it for free instead of repeating the mistake. Human review comments on the PR are a good source for these: if a human reviewer pointed something out and your review missed it, that usually indicates a doc gap — figure out what's missing and propose the addition.

## Dead code analysis (new models)

Expand Down
File renamed without changes.
12 changes: 7 additions & 5 deletions .ai/skills/custom-blocks/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -146,12 +146,14 @@ diffusers-cli run --model my-user/my-denoise-block --trust-remote-code \

## Verifying the install

If `diffusers-cli` isn't on PATH, see the install verification section of
[`../diffusers-cli/SKILL.md`](../diffusers-cli/SKILL.md#verifying-the-cli-is-installed).
If `diffusers-cli` isn't on PATH after `pip install -e .`, reinstall with
`pip install -e . --force-reinstall --no-deps` and check `which diffusers-cli`. If the binary is missing recent
features (e.g. `unrecognized arguments: --lora`), reinstall. See the
[`diffusers-cli` skill](https://github.com/huggingface/diffusers/blob/main/.ai/skills/diffusers-cli/SKILL.md#verifying-the-cli-is-installed) for more.

## Related

- [`diffusers-cli` skill](../diffusers-cli/SKILL.md) — once your block is uploaded, `schema`/`run`
let you call it from the terminal without writing Python.
- diffusers' [modular pipelines docs](../../../docs/source/en/modular_diffusers) — for writing the block
- the [`diffusers-cli` docs](https://github.com/huggingface/diffusers/blob/main/docs/source/en/using-diffusers/cli.md) — once your block
is uploaded, `schema` and `run` call it from the terminal without writing Python.
- diffusers' [modular pipelines docs](https://huggingface.co/docs/diffusers/main/en/modular_diffusers/overview) — for writing the block
class itself.
Loading
Loading