Skip to content

Introduce TomlFile abstraction and migrate all TOML I/O callsites#6942

Closed
ryancbahan wants to merge 3 commits intomainfrom
rcb-add-toml-domain-model
Closed

Introduce TomlFile abstraction and migrate all TOML I/O callsites#6942
ryancbahan wants to merge 3 commits intomainfrom
rcb-add-toml-domain-model

Conversation

@ryancbahan
Copy link
Contributor

@ryancbahan ryancbahan commented Mar 6, 2026

Why this PR exists

TOML file reads/writes are done ad-hoc today through a handful of different functions. This creates a few problems:

  • exposes implementation details unnecessarily, making it hard to change our read/write strategy in a unified way (e.g. moving away from toml-patch would require updating many callsites)
  • there's no unified abstraction for agents or commands to hook into for all TOML needs
  • the boundary between "what is in a file" and "what is Shopify's domain representation of app config" are tightly coupled. Reads, writes, validations, and transforms are all tied together in the same mutation-heavy pipeline
  • we call a lot of thing "toml" or "toToml" when they're not actually TOML -- they're object or string representations that are trying to match TOML. The ambiguity trips up humans and agents

This PR implements a TomlFile API that formally encodes TOML file I/O and its corresponding types via a unified public interface. Its job is specifically to handle reads and writes, as well as validating whether the TOML format is valid. It does not handle Shopify app config or domain representations outside of TOML syntax validity.

Summary

  • Introduces TomlFile class in cli-kit — api for reading, patching, removing keys from, and replacing TOML files on disk
  • Migrates every TOML I/O callsite to use TomlFile, replacing setAppConfigValue/setManyAppConfigValues/unsetAppConfigValue functions and direct encodeToml/decodeToml usage
  • Extension builders now return objects instead of TOML strings — serialization happens at the write boundary
  • encodeToml/decodeToml moved to internal codec.ts (no longer publicly exported)
  • breakdown-extensions.ts diff field extraction uses Object.keys() instead of encode→regex-parse round-trip
  • writeAppConfigurationFile now goes through TomlFile (replace + transformRaw for comment injection)
  • TomlParseError wraps parse errors with file path context

Test plan

  • All 20 TomlFile unit tests pass (read/patch/remove/replace/transformRaw)
  • All 1681 app tests pass (services + models)
  • All cli-kit toml/environments/json-schema tests pass
  • Manual verification: shopify app config link, shopify app dev, shopify app deploy produce identical TOML output

🤖 Generated with Claude Code

Copy link
Contributor Author

This stack of pull requests is managed by Graphite. Learn more about stacking.

@ryancbahan ryancbahan changed the title add e2e section Introduce TomlFile abstraction and migrate all TOML I/O callsites Mar 6, 2026
Adds a general-purpose TomlFile class in cli-kit that provides a unified
interface for reading, patching, removing keys from, and replacing TOML
files on disk. Migrates all callsites to use it, replacing the scattered
setAppConfigValue/setManyAppConfigValues/unsetAppConfigValue functions
and direct encodeToml/decodeToml usage.

Key changes:
- TomlFile class with read/patch/remove/replace/transformRaw methods
- Extension builders return objects instead of TOML strings
- writeAppConfigurationFile goes through TomlFile (replace + transformRaw
  for comment injection)
- breakdown-extensions uses Object.keys() instead of encode→regex round-trip
- encodeToml/decodeToml moved to internal codec module (not publicly exported)
- TomlParseError wraps parse errors with file path context
- Removed decode parameter from loadConfigurationFileContent/parseConfigurationFile

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@ryancbahan ryancbahan force-pushed the rcb-add-toml-domain-model branch from ee1faf2 to b485b0a Compare March 6, 2026 16:19
@github-actions
Copy link
Contributor

github-actions bot commented Mar 6, 2026

Coverage report

St.
Category Percentage Covered / Total
🟡 Statements 78.85% 14494/18382
🟡 Branches 73.17% 7197/9836
🟡 Functions 79.05% 3694/4673
🟡 Lines 79.18% 13696/17298

Test suite run success

3801 tests passing in 1453 suites.

Report generated by 🧪jest coverage report action from b3b4356

…→ buildExtensionConfig

These functions now return config objects instead of TOML strings,
so the names should reflect that.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@github-actions
Copy link
Contributor

github-actions bot commented Mar 6, 2026

Differences in type declarations

We detected differences in the type declarations generated by Typescript for this branch compared to the baseline ('main' branch). Please, review them to ensure they are backward-compatible. Here are some important things to keep in mind:

  • Some seemingly private modules might be re-exported through public modules.
  • If the branch is behind main you might see odd diffs, rebase main into this branch.

New type declarations

packages/cli-kit/dist/public/node/toml/codec.d.ts
import { JsonMap } from '../../../private/common/json.js';
export type JsonMapType = JsonMap;
/**
 * Given a TOML string, it returns a JSON object.
 *
 * @param input - TOML string.
 * @returns JSON object.
 */
export declare function decodeToml(input: string): JsonMapType;
/**
 * Given a JSON object, it returns a TOML string.
 *
 * @param content - JSON object.
 * @returns TOML string.
 */
export declare function encodeToml(content: JsonMap | object): string;
packages/cli-kit/dist/public/node/toml/index.d.ts
export type { JsonMapType } from './codec.js';
packages/cli-kit/dist/public/node/toml/toml-file.d.ts
import { JsonMapType } from './codec.js';
/**
 * Thrown when a TOML file cannot be parsed. Includes the file path for context.
 */
export declare class TomlParseError extends Error {
    readonly path: string;
    constructor(path: string, cause: Error);
}
/**
 * General-purpose TOML file abstraction.
 *
 * Provides a unified interface for reading, patching, removing keys from, and replacing
 * the content of TOML files on disk.
 *
 * - `read` populates content from disk
 * - `patch` does surgical WASM-based edits (preserves comments and formatting)
 * - `remove` deletes a key by dotted path (preserves comments and formatting)
 * - `replace` does a full re-serialization (comments and formatting are NOT preserved).
 * - `transformRaw` applies a function to the raw TOML string on disk.
 */
export declare class TomlFile {
    /**
     * Read and parse a TOML file from disk. Throws if the file doesn't exist or contains invalid TOML.
     * Parse errors are wrapped in {@link TomlParseError} with the file path for context.
     *
     * @param path - Absolute path to the TOML file.
     * @returns A TomlFile instance with parsed content.
     */
    static read(path: string): Promise<TomlFile>;
    readonly path: string;
    content: JsonMapType;
    constructor(path: string, content: JsonMapType);
    /**
     * Surgically patch values in the TOML file, preserving comments and formatting.
     *
     * Accepts a nested object whose leaf values are set in the TOML. Intermediate tables are
     * created automatically. Setting a leaf to `undefined` removes it (use `remove()` for a
     * clearer API when deleting keys).
     *
     * @example
     * ```ts
     * await file.patch({build: {dev_store_url: 'my-store.myshopify.com'}})
     * await file.patch({application_url: 'https://example.com', auth: {redirect_urls: ['...']}})
     * ```
     */
    patch(changes: {
        [key: string]: unknown;
    }): Promise<void>;
    /**
     * Remove a key from the TOML file by dotted path, preserving comments and formatting.
     *
     * @param keyPath - Dotted key path to remove (e.g. 'build.include_config_on_deploy').
     * @example
     * ```ts
     * await file.remove('build.include_config_on_deploy')
     * ```
     */
    remove(keyPath: string): Promise<void>;
    /**
     * Replace the entire file content. The file is fully re-serialized — comments and formatting
     * are NOT preserved.
     *
     * @param content - The new content to write.
     * @example
     * ```ts
     * await file.replace({client_id: 'abc', name: 'My App'})
     * ```
     */
    replace(content: JsonMapType): Promise<void>;
    /**
     * Transform the raw TOML string on disk. Reads the file, applies the transform function
     * to the raw text, writes back, and re-parses to keep `content` in sync.
     *
     * Use this for text-level operations that can't be expressed as structured edits —
     * e.g. Injecting comments or positional insertion of keys in arrays-of-tables.
     * Subsequent `patch()` calls will preserve any comments added this way.
     *
     * @param transform - A function that receives the raw TOML string and returns the modified string.
     * @example
     * ```ts
     * await file.transformRaw((raw) => `# Header comment\n${raw}`)
     * ```
     */
    transformRaw(transform: (raw: string) => string): Promise<void>;
}

Existing type declarations

We found no diffs with existing type declarations

@ryancbahan
Copy link
Contributor Author

Superseded by stacked PRs: #6943#6944#6945

@ryancbahan ryancbahan closed this Mar 6, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant