Skip to content

[Pyomo.DoE] Add simultaneous design of multiple experiments#3866

Open
smondal13 wants to merge 190 commits into
Pyomo:mainfrom
smondal13:add-multiexperiment
Open

[Pyomo.DoE] Add simultaneous design of multiple experiments#3866
smondal13 wants to merge 190 commits into
Pyomo:mainfrom
smondal13:add-multiexperiment

Conversation

@smondal13
Copy link
Copy Markdown
Contributor

@smondal13 smondal13 commented Mar 2, 2026

Fixes # .

Summary/Motivation:

This PR adds a new DesignOfExperiments.optimize_experiments() API in pyomo/contrib/doe/doe.py to support simultaneous optimization of multiple experiments in one workflow. The motivation is to provide a multi-experiment DoE interface with stronger initialization options, clearer mode handling (template vs. user-initialized experiments), and richer diagnostics/results than the existing single-experiment path.

Changes proposed in this PR:

  • Added API optimize_experiments() for multi-experiment DoE optimization.
  • Added Cholesky-based D- and A-optimality objective and Greybox-based D-, A-, E-, and ME- optimality objective
  • Implemented two operating modes:
    • Template mode: pass one experiment and set n_exp.
    • User-initialized mode: pass a list of experiments; n_exp is inferred/validated.
  • Added optional LHS-based initialization (initialization_method="lhs") with controls for:
    • sample count, seed, candidate evaluation parallelism, combination fim metric scoring parallelism,
    • worker count, chunk size, parallel threshold, and optional wall-clock budget.
  • Added symmetry-breaking constraints for multi-experiment solves:
    • supports user-specified variable through sym_break_cons suffix,
    • falls back to the first experiment input with a diagnostic warning when not provided.
  • Expanded output for this API:
    • per-scenario and per-experiment results (designs, outputs, measurement errors, FIM/sensitivities),
    • aggregated FIM metrics, timings, settings, names, diagnostics, and structured run_info.
  • Added JSON-safe serialization via _DoEResultsJSONEncoder for numpy/Pyomo-enum values when writing results_file.

Note:

  • Added new documentation.md which describes the API. This documentation is to help the reviewers to understand the API and will not be merged into Pyomo:main

Remove before merging

  • documentation.md
  • rb_multi.py

Legal Acknowledgement

By contributing to this software project, I have read the contribution guide and agree to the following terms and conditions for my contribution:

  1. I agree my contributions are submitted under the BSD license.
  2. I represent I am authorized to make the contributions and grant the license. If my employer has rights to intellectual property that includes these contributions, I represent that I have received permission to make contributions and grant the required license on behalf of that employer.

…d replace `self.experiment` with `self.experiment_list[0]`. `doe/reactor_example.py` runs successfully.
…sing the `doe/reactor_multi_experiment.py`
…nt for `optimize_experiments()` and `compute_FIM()`
… both of sensitivity and optimize_experiments()
…A-opt gave different result for grid and optimization
@smondal13
Copy link
Copy Markdown
Contributor Author

@blnicho @mrmundt All the tests are passing except win/3.11 with

INFO: The following extensions were downloaded:
    [FAIL]  gjh
    [ OK ]  mcpp

Comment thread pyomo/contrib/doe/tests/test_doe_errors.py Outdated
smondal13 and others added 2 commits May 21, 2026 11:24
Co-authored-by: Miranda Mundt <55767766+mrmundt@users.noreply.github.com>
Comment thread pyomo/contrib/doe/tests/test_greybox.py Outdated
@smondal13
Copy link
Copy Markdown
Contributor Author

@mrmundt and @blnicho , all the comments are addressed, and all the tests are passing now.

@slilonfe5
Copy link
Copy Markdown
Member

@mrmundt @blnicho, Shuvo has addressed all the comments. This looks pretty good to me. Any final review would be appreciated; otherwise, I think we are good to go.

@codecov
Copy link
Copy Markdown

codecov Bot commented May 27, 2026

Codecov Report

❌ Patch coverage is 91.66667% with 1 line in your changes missing coverage. Please review.
✅ Project coverage is 90.15%. Comparing base (3603b1e) to head (a20061b).
⚠️ Report is 6 commits behind head on main.

Files with missing lines Patch % Lines
pyomo/contrib/doe/grey_box_utilities.py 91.66% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #3866      +/-   ##
==========================================
+ Coverage   90.11%   90.15%   +0.03%     
==========================================
  Files         909      909              
  Lines      108580   109236     +656     
==========================================
+ Hits        97847    98477     +630     
- Misses      10733    10759      +26     
Flag Coverage Δ
builders 28.98% <91.66%> (-0.12%) ⬇️
default 86.17% <91.66%> (?)
expensive 34.97% <91.66%> (?)
linux 87.68% <91.66%> (-1.93%) ⬇️
linux_other 87.68% <91.66%> (+0.05%) ⬆️
oldsolvers 27.90% <8.33%> (-0.15%) ⬇️
osx 83.09% <8.33%> (+0.03%) ⬆️
win 85.44% <91.66%> (+0.07%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@adowling2
Copy link
Copy Markdown
Member

@smondal13 This PR is now showing merge conflicts.

Copy link
Copy Markdown
Member

@adowling2 adowling2 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this is practically ready. I requested a few small comments. Also, there is a merge conflict (I think).

Comment thread pyomo/contrib/doe/tests/test_doe_build.py Outdated
Comment thread pyomo/contrib/doe/tests/test_doe_build.py
Comment thread pyomo/contrib/doe/tests/test_doe_build.py
@smondal13
Copy link
Copy Markdown
Contributor Author

smondal13 commented May 28, 2026

I think this is practically ready. I requested a few small comments. Also, there is a merge conflict (I think).

@adowling2 I have addressed your comments. @adowling2 @mrmundt, I have resolved the conflicts. The conflicts were from @adowling2's recent PR #3867 merge into Pyomo:main.

@smondal13
Copy link
Copy Markdown
Contributor Author

@mrmundt @blnicho, the following two tests are failing. I think these two are not part of my PR

=========================== short test summary info ============================
FAILED pyomo/contrib/iis/tests/test_iis.py::TestIIS::test_write_iis_any_solver - SystemError: <method 'write' of '_io.FileIO' objects> returned a result with an exception set
FAILED pyomo/contrib/iis/tests/test_iis.py::TestIIS::test_write_iis_cplex - SystemError: <method 'write' of '_io.FileIO' objects> returned a result with an exception set

@blnicho
Copy link
Copy Markdown
Member

blnicho commented May 28, 2026

@smondal13 those failures were resolved in #3965 and I merged main into your PR to pick up the fix.

@smondal13
Copy link
Copy Markdown
Contributor Author

@blnicho @mrmundt this one is failing due to download issue

downloaded:
[FAIL] gjh
[ OK ] mcpp
Error: Process completed with exit code 1.

@smondal13 smondal13 moved this from Ready for design review to Ready for final review in ParmEst & Pyomo.DoE Development May 31, 2026
@smondal13
Copy link
Copy Markdown
Contributor Author

@blnicho @mrmundt Two tests are failing in this PR, which are not related to the PR.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

Status: Ready for final review

Development

Successfully merging this pull request may close these issues.

7 participants