pfstack

The map, the rules and the template for the pf* stack: a family of small Python libraries for doing one thing in every directory of a tree, running programs from Python with a truthful record, and building ChRIS plugins on both.

This repository holds no code to install. It holds what the stack's repositories share and should not each carry a copy of: the working rules for work in any of them, the map of which package does what and depends on which, the guide to building a ChRIS plugin on the stack with the mistakes already made, and a skeleton to cut a new plugin from.

The stack

pftree	Walk a directory tree, filter it, and run read, analyze and save stages per directory, in threads, with a record. Standard library and tqdm only.
pfdo	The frame around a per-directory task: options, file selection, run, report, command line. A task attaches by subclass or by decorator.
pftag	A template grammar, %{name|function}, for naming outputs from inputs and building commands. No dependencies.
hostlib	Run a program, stream its output while it runs, keep a record whose truth does not depend on reading the output. No dependencies.
pflog	Time a call and keep the record, as one line of JSON. Depends on hostlib.
pfdo_run	Run a templated command on every selected file of a tree. pfdo, pftag and hostlib together.
pl-shexec	pfdo_run as a ChRIS plugin, and the base a plugin wrapping any command-line program is built on through plugin_build.

Dependencies run one way, down the table. Each package re-exports the records of the one below it through its models module, so a program imports from one package and never reaches past it. docs/stack.adoc is the full map.

What is here

docs/rules.adoc

The working rules: code style, layout, documentation, releasing, and how to work with the operator. They live here once; a stack repository points to them and carries no copy. Check this repository out at ~/src/pfstack on every machine.

docs/stack.adoc

The map: each package's job, what it is not for, its public surface, what it re-exports, and the dependency graph.

docs/plugin.adoc

How to build a ChRIS plugin on the stack, both shapes, with every gotcha pl-shexec paid for.

docs/history.adoc

The 2026 revitalization: why each package is what it is, in one place.

docs/status.adoc and docs/plan.adoc

Where work in flight stands, how to continue it, and the plan for it. They live in the working copy only and are not published, because they name private hosts and spending.

docs/chell_handling.adoc

A working document on what a chell /bin run should do with its operands, inside and outside the feeds tree: each case as a chell line, what should happen, what happens now, and the decisions taken and open.

template/plugin-on-shexec/

The skeleton for a plugin that wraps a command-line program: it depends on pl-shexec and contributes a spec, the default command, and the image. pl-freesurfer is cut from this one. Copy it, rename plname, set COMMAND and the image.

template/plugin/

The skeleton for a plugin whose work is Python, one function per directory on pfdo's frame: package, tests, pyproject.toml and setup.cfg, Dockerfile, CI that tests before it builds and uploads on a tag, and the docs trio. Copy it, rename plname, write the one function.

Using it

git clone git@github.com:FNNDSC/pfstack.git ~/src/pfstack

For a new plugin:

cp -r ~/src/pfstack/template/plugin-on-shexec ~/src/pl-mytool   # wraps a program
cp -r ~/src/pfstack/template/plugin ~/src/pl-mytool             # or: Python per directory
cd ~/src/pl-mytool && grep -rl plname . | xargs sed -i 's/plname/mytool/g' && mv plname mytool

Then follow docs/plugin.adoc.