Assembly workflow by KateSakharova · Pull Request #26 · nf-core/seqsubmit

KateSakharova · 2025-11-11T10:21:27Z

Resolving #4

This pull request introduces support for a new "metagenomic_assemblies" submission mode to the pipeline, alongside significant improvements to documentation and input schema validation.

Major new features and improvements:

Support for metagenomic assemblies:

Added the metagenomic_assemblies mode, routed to a new ASSEMBLYSUBMIT workflow, with its own input schema (assets/schema_input_assembly.json) and example samplesheet (assets/samplesheet_assembly.csv). The schema enforces required fields and validates that either coverage or reads are provided for each assembly.
Updated the main workflow (main.nf) to dispatch to GENOMESUBMIT or ASSEMBLYSUBMIT based on the --mode parameter, and to handle multiqc report output accordingly.

Documentation and usability:

Rewrote and expanded the README.md to clearly describe the three supported submission modes, required/optional input columns, command-line usage, and output structure. Added example samplesheets and clarified parameter documentation.

Input validation and schemas:

Renamed the genome submission input schema to schema_input_genome.json
Added new schema schema_input_assembly.json

Configuration and output structure:

Output directories for manifests, webin_cli reports, and multiqc reports are now grouped by mode (e.g., outdir/mags/upload/manifests) for better separation of results.
Added dedicated test config files for genome and assembly workflows. FIXME: test.config for now left empty.

Required improvements:
**- Schema mentions are removed from nextflow_schema.json, to enable usage of --input for different samplesheets, that enables automatic validation. validateSamplesheet function has to be added to pipeline inizialisation.

Tests on GitHub CI fail due to "missing secrets" error.**

PR checklist

github-actions · 2025-11-11T10:23:24Z

`nf-core pipelines lint` overall result: Passed ✅ ⚠️

Posted for pipeline commit bd9a67f

+| ✅ 200 tests passed       |+
#| ❔   4 tests were ignored |#
#| ❔   1 tests had warnings |#
!| ❗  23 tests had warnings |!

Details

❗ Test warnings:

readme - README contains the placeholder zenodo.XXXXXXX. This should be replaced with the zenodo doi (after the first release).
pipeline_todos - TODO string in README.md: Add citation for pipeline after first release. Uncomment lines below and update Zenodo doi and badge at the top of this file.
pipeline_todos - TODO string in README.md: Add bibliography of tools and data used in your pipeline
pipeline_todos - TODO string in nextflow.config: Optionally, you can add a pipeline-specific nf-core config at https://github.com/nf-core/configs
pipeline_todos - TODO string in nextflow.config: Update the field with the details of the contributors to your pipeline. New with Nextflow version 24.10.0
pipeline_todos - TODO string in methods_description_template.yml: #Update the HTML below to your preferred methods description, e.g. add publication citation for this pipeline
pipeline_todos - TODO string in nextflow.config: Specify any additional parameters here
pipeline_todos - TODO string in usage.md: Add documentation about anything specific to running your pipeline. For general topics, please point to (and add to) the main nf-core website.
pipeline_todos - TODO string in output.md: Write this documentation describing your workflow's output
pipeline_todos - TODO string in awsfulltest.yml: You can customise AWS full pipeline tests as required
pipeline_todos - TODO string in base.config: Check the defaults for all processes
pipeline_todos - TODO string in base.config: Customise requirements for specific processes.
pipeline_todos - TODO string in test_assembly.config: Specify the paths to your test data on nf-core/test-datasets
pipeline_todos - TODO string in test_assembly.config: Give any required params for the test so that command line flags are not needed
pipeline_todos - TODO string in test_genome.config: Specify the paths to your test data on nf-core/test-datasets
pipeline_todos - TODO string in test_genome.config: Give any required params for the test so that command line flags are not needed
pipeline_todos - TODO string in test_full.config: Specify the paths to your full test data ( on nf-core/test-datasets or directly in repositories, e.g. SRA)
pipeline_todos - TODO string in test_full.config: Give any required params for the test so that command line flags are not needed
pipeline_todos - TODO string in environment.yml: List required Conda package(s).
pipeline_todos - TODO string in main.nf.test: Once you have added the required tests, please run the following command to build this file:
pipeline_todos - TODO string in main.nf: Optionally add in-text citation tools to this list.
pipeline_todos - TODO string in main.nf: Optionally add bibliographic entries to this list.
pipeline_todos - TODO string in main.nf: Only uncomment below if logic in toolCitationText/toolBibliographyText has been filled!

❔ Tests ignored:

files_exist - File is ignored: conf/igenomes.config
files_exist - File is ignored: conf/igenomes_ignored.config
nextflow_config - Config variable ignored: params.input
files_unchanged - File ignored due to lint config: .github/PULL_REQUEST_TEMPLATE.md

❔ Tests fixed:

rocrate_readme_sync - Mismatch fixed: RO-Crate description updated from README.md.

✅ Tests passed:

files_exist - File found: .gitattributes
files_exist - File found: .gitignore
files_exist - File found: .nf-core.yml
files_exist - File found: .prettierignore
files_exist - File found: .prettierrc.yml
files_exist - File found: CHANGELOG.md
files_exist - File found: CITATIONS.md
files_exist - File found: CODE_OF_CONDUCT.md
files_exist - File found: LICENSE or LICENSE.md or LICENCE or LICENCE.md
files_exist - File found: nextflow_schema.json
files_exist - File found: nextflow.config
files_exist - File found: README.md
files_exist - File found: .github/.dockstore.yml
files_exist - File found: .github/CONTRIBUTING.md
files_exist - File found: .github/ISSUE_TEMPLATE/bug_report.yml
files_exist - File found: .github/ISSUE_TEMPLATE/config.yml
files_exist - File found: .github/ISSUE_TEMPLATE/feature_request.yml
files_exist - File found: .github/PULL_REQUEST_TEMPLATE.md
files_exist - File found: .github/workflows/branch.yml
files_exist - File found: .github/workflows/nf-test.yml
files_exist - File found: .github/actions/get-shards/action.yml
files_exist - File found: .github/actions/nf-test/action.yml
files_exist - File found: .github/workflows/linting_comment.yml
files_exist - File found: .github/workflows/linting.yml
files_exist - File found: assets/email_template.html
files_exist - File found: assets/email_template.txt
files_exist - File found: assets/sendmail_template.txt
files_exist - File found: assets/nf-core-seqsubmit_logo_light.png
files_exist - File found: conf/modules.config
files_exist - File found: conf/test.config
files_exist - File found: conf/test_full.config
files_exist - File found: docs/images/nf-core-seqsubmit_logo_light.png
files_exist - File found: docs/images/nf-core-seqsubmit_logo_dark.png
files_exist - File found: docs/output.md
files_exist - File found: docs/README.md
files_exist - File found: docs/README.md
files_exist - File found: docs/usage.md
files_exist - File found: nf-test.config
files_exist - File found: tests/default.nf.test
files_exist - File found: main.nf
files_exist - File found: assets/multiqc_config.yml
files_exist - File found: conf/base.config
files_exist - File found: .github/workflows/awstest.yml
files_exist - File found: .github/workflows/awsfulltest.yml
files_exist - File found: modules.json
files_exist - File found: ro-crate-metadata.json
files_exist - File not found check: .github/ISSUE_TEMPLATE/bug_report.md
files_exist - File not found check: .github/ISSUE_TEMPLATE/feature_request.md
files_exist - File not found check: .github/workflows/push_dockerhub.yml
files_exist - File not found check: .markdownlint.yml
files_exist - File not found check: .nf-core.yaml
files_exist - File not found check: .yamllint.yml
files_exist - File not found check: bin/markdown_to_html.r
files_exist - File not found check: conf/aws.config
files_exist - File not found check: docs/images/nf-core-seqsubmit_logo.png
files_exist - File not found check: lib/Checks.groovy
files_exist - File not found check: lib/Completion.groovy
files_exist - File not found check: lib/NfcoreTemplate.groovy
files_exist - File not found check: lib/Utils.groovy
files_exist - File not found check: lib/Workflow.groovy
files_exist - File not found check: lib/WorkflowMain.groovy
files_exist - File not found check: lib/WorkflowSeqsubmit.groovy
files_exist - File not found check: parameters.settings.json
files_exist - File not found check: pipeline_template.yml
files_exist - File not found check: Singularity
files_exist - File not found check: lib/nfcore_external_java_deps.jar
files_exist - File not found check: .travis.yml
nextflow_config - Found nf-schema plugin
nextflow_config - Config variable found: manifest.name
nextflow_config - Config variable found: manifest.nextflowVersion
nextflow_config - Config variable found: manifest.description
nextflow_config - Config variable found: manifest.version
nextflow_config - Config variable found: manifest.homePage
nextflow_config - Config variable found: timeline.enabled
nextflow_config - Config variable found: trace.enabled
nextflow_config - Config variable found: report.enabled
nextflow_config - Config variable found: dag.enabled
nextflow_config - Config variable found: process.cpus
nextflow_config - Config variable found: process.memory
nextflow_config - Config variable found: process.time
nextflow_config - Config variable found: params.outdir
nextflow_config - Config variable found: manifest.mainScript
nextflow_config - Config variable found: timeline.file
nextflow_config - Config variable found: trace.file
nextflow_config - Config variable found: report.file
nextflow_config - Config variable found: dag.file
nextflow_config - Config variable (correctly) not found: params.nf_required_version
nextflow_config - Config variable (correctly) not found: params.container
nextflow_config - Config variable (correctly) not found: params.singleEnd
nextflow_config - Config variable (correctly) not found: params.igenomesIgnore
nextflow_config - Config variable (correctly) not found: params.name
nextflow_config - Config variable (correctly) not found: params.enable_conda
nextflow_config - Config variable (correctly) not found: params.max_cpus
nextflow_config - Config variable (correctly) not found: params.max_memory
nextflow_config - Config variable (correctly) not found: params.max_time
nextflow_config - Config variable (correctly) not found: params.validationFailUnrecognisedParams
nextflow_config - Config variable (correctly) not found: params.validationLenientMode
nextflow_config - Config variable (correctly) not found: params.validationSchemaIgnoreParams
nextflow_config - Config variable (correctly) not found: params.validationShowHiddenParams
nextflow_config - Config variable (correctly) not found: validation.failUnrecognisedParams
nextflow_config - Config variable (correctly) not found: validation.failUnrecognisedHeaders
nextflow_config - Config timeline.enabled had correct value: true
nextflow_config - Config report.enabled had correct value: true
nextflow_config - Config trace.enabled had correct value: true
nextflow_config - Config dag.enabled had correct value: true
nextflow_config - Config manifest.name began with nf-core/
nextflow_config - Config variable manifest.homePage began with https://github.com/nf-core/
nextflow_config - Config dag.file ended with .html
nextflow_config - Config variable manifest.nextflowVersion started with >= or !>=
nextflow_config - Config manifest.version ends in dev: 1.0.0dev
nextflow_config - Config params.custom_config_version is set to master
nextflow_config - Config params.custom_config_base is set to https://raw.githubusercontent.com/nf-core/configs/master
nextflow_config - Lines for loading custom profiles found
nextflow_config - nextflow.config contains configuration profile test
nextflow_config - Config default value correct: params.custom_config_version= master
nextflow_config - Config default value correct: params.custom_config_base= https://raw.githubusercontent.com/nf-core/configs/master
nextflow_config - Config default value correct: params.publish_dir_mode= copy
nextflow_config - Config default value correct: params.max_multiqc_email_size= 25.MB
nextflow_config - Config default value correct: params.validate_params= true
nextflow_config - Config default value correct: params.pipelines_testdata_base_path= https://raw.githubusercontent.com/nf-core/test-datasets/
nextflow_config - Config default value correct: params.upload_tpa= false
nextflow_config - Config default value correct: params.test_upload= true
nextflow_config - Config default value correct: params.upload_force= true
nextflow_config - Config default value correct: params.webincli_submit= true
nf_test_content - 'tests/default.nf.test' contains outdir parameter
nf_test_content - 'tests/default.nf.test' snapshots a 'versions.yml' file
nf_test_content - 'tests/default.nf.test' snapshots a 'versions.yml' file
nf_test_content - 'tests/nextflow.config' contains modules_testdata_base_path
nf_test_content - 'tests/nextflow.config' contains pipelines_testdata_base_path
nf_test_content - 'nf-test.config' sets a testsDir
nf_test_content - 'nf-test.config' sets a workDir
nf_test_content - 'nf-test.config' sets a configFile
files_unchanged - .gitattributes matches the template
files_unchanged - .prettierrc.yml matches the template
files_unchanged - CODE_OF_CONDUCT.md matches the template
files_unchanged - LICENSE matches the template
files_unchanged - .github/.dockstore.yml matches the template
files_unchanged - .github/CONTRIBUTING.md matches the template
files_unchanged - .github/ISSUE_TEMPLATE/bug_report.yml matches the template
files_unchanged - .github/ISSUE_TEMPLATE/config.yml matches the template
files_unchanged - .github/ISSUE_TEMPLATE/feature_request.yml matches the template
files_unchanged - .github/workflows/branch.yml matches the template
files_unchanged - .github/workflows/linting_comment.yml matches the template
files_unchanged - .github/workflows/linting.yml matches the template
files_unchanged - assets/email_template.html matches the template
files_unchanged - assets/email_template.txt matches the template
files_unchanged - assets/sendmail_template.txt matches the template
files_unchanged - assets/nf-core-seqsubmit_logo_light.png matches the template
files_unchanged - docs/images/nf-core-seqsubmit_logo_light.png matches the template
files_unchanged - docs/images/nf-core-seqsubmit_logo_dark.png matches the template
files_unchanged - docs/README.md matches the template
files_unchanged - .gitignore matches the template
files_unchanged - .prettierignore matches the template
actions_nf_test - '.github/workflows/nf-test.yml' is triggered on expected events
actions_nf_test - '.github/workflows/nf-test.yml' checks minimum NF version
actions_awstest - '.github/workflows/awstest.yml' is triggered correctly
actions_awsfulltest - .github/workflows/awsfulltest.yml is triggered correctly
actions_awsfulltest - .github/workflows/awsfulltest.yml does not use -profile test
readme - README Nextflow minimum version badge matched config. Badge: 25.04.0, Config: 25.04.0
readme - README nf-core template version badge found.
pipeline_if_empty_null - No ifEmpty(null) strings found
plugin_includes - No wrong validation plugin imports have been found
pipeline_name_conventions - Name adheres to nf-core convention
template_strings - Did not find any Jinja template strings (0 files)
schema_lint - Schema lint passed
schema_lint - Schema title + description lint passed
schema_lint - Input mimetype lint passed: 'text/csv'
schema_params - Schema matched params returned from nextflow config
system_exit - No System.exit calls found
actions_schema_validation - Workflow validation passed: branch.yml
actions_schema_validation - Workflow validation passed: fix_linting.yml
actions_schema_validation - Workflow validation passed: clean-up.yml
actions_schema_validation - Workflow validation passed: linting.yml
actions_schema_validation - Workflow validation passed: awsfulltest.yml
actions_schema_validation - Workflow validation passed: template-version-comment.yml
actions_schema_validation - Workflow validation passed: nf-test.yml
actions_schema_validation - Workflow validation passed: download_pipeline.yml
actions_schema_validation - Workflow validation passed: awstest.yml
actions_schema_validation - Workflow validation passed: release-announcements.yml
actions_schema_validation - Workflow validation passed: linting_comment.yml
merge_markers - No merge markers found in pipeline files
modules_json - Only installed modules found in modules.json
multiqc_config - assets/multiqc_config.yml found and not ignored.
multiqc_config - assets/multiqc_config.yml contains report_section_order
multiqc_config - assets/multiqc_config.yml contains export_plots
multiqc_config - assets/multiqc_config.yml contains report_comment
multiqc_config - assets/multiqc_config.yml follows the ordering scheme of the minimally required plugins.
multiqc_config - assets/multiqc_config.yml contains a matching 'report_comment'.
multiqc_config - assets/multiqc_config.yml contains 'export_plots: true'.
modules_structure - modules directory structure is correct 'modules/nf-core/TOOL/SUBTOOL'
local_component_structure - local subworkflows directory structure is correct 'subworkflows/local/TOOL/SUBTOOL'
base_config - conf/base.config found and not ignored.
modules_config - conf/modules.config found and not ignored.
modules_config - GENOME_UPLOAD found in conf/modules.config and Nextflow scripts.
modules_config - ENA_WEBIN_CLI found in conf/modules.config and Nextflow scripts.
modules_config - MULTIQC found in conf/modules.config and Nextflow scripts.
modules_config - GENERATE_ASSEMBLY_MANIFEST found in conf/modules.config and Nextflow scripts.
nfcore_yml - Repository type in .nf-core.yml is valid: pipeline
nfcore_yml - nf-core version in .nf-core.yml is set to the latest version: 3.5.1
rocrate_readme_sync - RO-Crate description matches the README.md.

Run details

nf-core/tools version 3.5.1
Run at 2026-03-09 12:01:30

nf-core-bot · 2026-03-05T15:40:12Z

Warning

Newer version of the nf-core template is available.

Your pipeline is using an old version of the nf-core template: 3.5.1.
Please update your pipeline to the latest version.

For more documentation on how to update your pipeline, please see the nf-core documentation and Synchronisation documentation.

…study not provided

…ow schema

…module

…input schema

…diagram

ochkalova

There are still some things to fix, but it starts to look like a pipeline 😄

FernandoDuarteF and others added 23 commits March 6, 2026 11:43

Added validation

a87ff76

Added test for each subworkflow

465c1d8

Added coverM module

ca095c7

Added assembly submit workflow

c7ae940

Added params.input to .nf-core.yml

d857e8a

Fixed linting

8184c72

Updated assembly submit workflow

5d8b240

Fixed nextflow.config

8058ebd

Updated assembly submit workflow

2569abe

add draft assembly workflow (not yet tested)

ce8b84e

update schema for assembly input validation

87c30dc

update assembly_uploader version in GENERATE_ASSEMBLY_MANIFEST process

f4c9757

update assembly_uploader version in REGISTERSTUDY process

72ffa5d

ASSEMBLYSUBMIT workflow fixes after tests

03f797d

add ENA_WEBIN and ENA_WEBIN_PASSWORD to env in nextflow.config

3e49ae3

input schema fixes after tests

eaa3d7b

append --test flag if params.test_upload = true

8fd8509

linter fixes in REGISTERSTUDY

4964c4d

update assembly_uploader to the latest version to fix long alias issue

1321d69

cleanup, add more TODOs

92de4fb

use "metagenomic_assemblies" instead of "assemblies" for clarity

11b7ae6

update column name in the samplesheet

ba49cb8

clean up test configs a little

26c6296

ochkalova force-pushed the assembly_workflow branch from e09443f to 26c6296 Compare March 6, 2026 11:54

ochkalova added 4 commits March 6, 2026 12:02

linter fixes

d068775

update nextflow schema with new parameters

1c895ed

add TODO to organise tests

2e97ea5

rename ena_genome_study_accession to submission_study, only

b855e46

ochkalova added 14 commits March 6, 2026 12:52

refactor: only register new study in assembly workflow if submission_…

ce6e5be

…study not provided

refactor: update Channel references to use lowercase 'channel'

ac58fff

fix: include TPA parameter in assembly manifest generation

43ef432

fix: add 'input' to required fields and update descriptions in nextfl…

4522b71

…ow schema

fix: set upload_tpa to false and mark for removal in genome_uploader …

c19953d

…module

fix: update FASTA file pattern to enforce gzipped format in assembly …

710d00f

…input schema

docs: enhance README with detailed submission modes and add workflow …

38ab8e7

…diagram

fix: add '--outdir' option to README for output directory specification

a11a9d7

fix: rename test_assembly.conf to test.conf to satisfy nf-core linter

5803b9a

fix: patch fastavalidator module

39d7414

fix: bugfixes revealed by testing

6e31241

add more test cases for assemblysubmit

27b75b2

to be fixed: create empty test.config to satisfy linter

38e68d6

fix: update README for ENA submission examples

bd9a67f

ochkalova self-requested a review March 9, 2026 16:22

ochkalova approved these changes Mar 9, 2026

View reviewed changes

ochkalova marked this pull request as ready for review March 9, 2026 16:24

ochkalova merged commit da12288 into dev Mar 9, 2026
11 of 17 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Assembly workflow#26

Assembly workflow#26
ochkalova merged 41 commits intodevfrom
assembly_workflow

KateSakharova commented Nov 11, 2025 •

edited by ochkalova

Loading

Uh oh!

github-actions bot commented Nov 11, 2025 •

edited

Loading

❗ Test warnings:

❔ Tests ignored:

❔ Tests fixed:

✅ Tests passed:

Run details

Uh oh!

nf-core-bot commented Mar 5, 2026 •

edited

Loading

Uh oh!

ochkalova left a comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Conversation

KateSakharova commented Nov 11, 2025 • edited by ochkalova Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

PR checklist

Uh oh!

github-actions bot commented Nov 11, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

nf-core pipelines lint overall result: Passed ✅ ⚠️

❗ Test warnings:

❔ Tests ignored:

❔ Tests fixed:

✅ Tests passed:

Run details

Uh oh!

nf-core-bot commented Mar 5, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ochkalova left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

KateSakharova commented Nov 11, 2025 •

edited by ochkalova

Loading

github-actions bot commented Nov 11, 2025 •

edited

Loading

`nf-core pipelines lint` overall result: Passed ✅ ⚠️

nf-core-bot commented Mar 5, 2026 •

edited

Loading