Skip to content

Generate reference config from previous logs#17

Merged
alexandraBara merged 8 commits intodevelopmentfrom
alex_diff_logs
Jul 16, 2025
Merged

Generate reference config from previous logs#17
alexandraBara merged 8 commits intodevelopmentfrom
alex_diff_logs

Conversation

@alexandraBara
Copy link
Copy Markdown
Collaborator

@alexandraBara alexandraBara commented Jul 7, 2025

Scope: Use previous node-scraper logs to generate a reference config by parsing the log files.
node-scraper --gen-reference-config-from-logs <path_to_previous_logs>

Sample run:

node-scraper --gen-reference-config-from-logs scraper_logs_TheraC56_2025_07_07-11_11_08_AM/

Sample output:

$ node-scraper --gen-reference-config-from-logs scraper_logs_TheraC56_2025_07_07-11_11_08_AM/
  2025-07-07 14:13:41 CDT       INFO               nodescraper | Log path: ./scraper_logs_2025_07_07-02_13_41_PM
  2025-07-07 14:13:41 CDT       INFO               nodescraper | Reference config written to: /home/alexbara/node-scraper/reference_config.json

reference_config.json:

{
  "global_args": {},
  "plugins": {
    "BiosPlugin": {
      "analysis_args": {
        "exp_bios_version": [
          "M17"
        ],
        "regex_match": false
      }
    },
    "OsPlugin": {
      "analysis_args": {
        "exp_os": [
          "8.10"
        ],
        "exact_match": true
      }
    }
  },
  "result_collators": {}

README.md updates to follow

Base automatically changed from alex_diff to development July 9, 2025 19:49
@bargajda-amd
Copy link
Copy Markdown
Collaborator

bargajda-amd commented Jul 10, 2025

@alexandraBara have you checked if reference config file generated directly (--gen-reference-config) is identical to one generated from logs, assuming same "input" node?

@alexandraBara
Copy link
Copy Markdown
Collaborator Author

alexandraBara commented Jul 10, 2025

@alexandraBara have you checked if reference config file generated directly (--gen-reference-config) is identical to one generated from logs, assuming same "input" node?

I have not, let me investigate

@alexandraBara
Copy link
Copy Markdown
Collaborator Author

alexandraBara commented Jul 10, 2025

@alexandraBara have you checked if reference config file generated directly (--gen-reference-config) is identical to one generated from logs, assuming same "input" node?

I have not, let me investigate

The end files are the same. We will be adding this test to functional testing in the future to make sure it stays that way.

landrews-amd
landrews-amd requested changes Jul 11, 2025
View reviewed changes
Copy link
Copy Markdown
Collaborator

@landrews-amd landrews-amd left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good! added a couple suggestions

Comment thread nodescraper/cli/cli.py Outdated
Comment thread nodescraper/cli/cli.py Outdated
Comment thread test/unit/framework/test_helper.py
Comment thread nodescraper/models/taskresult.py Outdated
Comment thread nodescraper/cli/helper.py Outdated
Comment thread nodescraper/cli/helper.py Outdated
landrews-amd
landrews-amd approved these changes Jul 15, 2025
View reviewed changes
Comment thread nodescraper/cli/cli.py Outdated
"--gen-reference-config-from-logs",
dest="reference_config_from_logs",
type=log_path_arg,
help="Generate reference config from previous run logfiles. Writes to ./reference_config.json.",
Copy link
Copy Markdown
Collaborator

@landrews-amd landrews-amd Jul 15, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should remove the 'Writes to ./reference_config.json.' part since we are now leveraging the output path arg

@alexandraBara alexandraBara merged commit a498e7c into development Jul 16, 2025
5 checks passed
@alexandraBara alexandraBara deleted the alex_diff_logs branch July 16, 2025 14:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@landrews-amd landrews-amd landrews-amd approved these changes

@bargajda-amd bargajda-amd bargajda-amd approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@alexandraBara @bargajda-amd @landrews-amd