Skip to content

Conversation

@alexandraBara
Copy link
Collaborator

Scope: Use previous node-scraper logs to generate a reference config by parsing the log files.
node-scraper --gen-reference-config-from-logs <path_to_previous_logs>

Sample run:

node-scraper --gen-reference-config-from-logs scraper_logs_TheraC56_2025_07_07-11_11_08_AM/

Sample output:

$ node-scraper --gen-reference-config-from-logs scraper_logs_TheraC56_2025_07_07-11_11_08_AM/
  2025-07-07 14:13:41 CDT       INFO               nodescraper | Log path: ./scraper_logs_2025_07_07-02_13_41_PM
  2025-07-07 14:13:41 CDT       INFO               nodescraper | Reference config written to: /home/alexbara/node-scraper/reference_config.json

reference_config.json:

{
  "global_args": {},
  "plugins": {
    "BiosPlugin": {
      "analysis_args": {
        "exp_bios_version": [
          "M17"
        ],
        "regex_match": false
      }
    },
    "OsPlugin": {
      "analysis_args": {
        "exp_os": [
          "8.10"
        ],
        "exact_match": true
      }
    }
  },
  "result_collators": {}

README.md updates to follow

Base automatically changed from alex_diff to development July 9, 2025 19:49
@bargajda-amd
Copy link
Collaborator

@alexandraBara have you checked if reference config file generated directly (--gen-reference-config) is identical to one generated from logs, assuming same "input" node?

@alexandraBara
Copy link
Collaborator Author

@alexandraBara have you checked if reference config file generated directly (--gen-reference-config) is identical to one generated from logs, assuming same "input" node?

I have not, let me investigate

@alexandraBara
Copy link
Collaborator Author

@alexandraBara have you checked if reference config file generated directly (--gen-reference-config) is identical to one generated from logs, assuming same "input" node?

I have not, let me investigate

The end files are the same. We will be adding this test to functional testing in the future to make sure it stays that way.

Copy link
Collaborator

@landrews-amd landrews-amd left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good! added a couple suggestions

"--gen-reference-config-from-logs",
dest="reference_config_from_logs",
type=log_path_arg,
help="Generate reference config from previous run logfiles. Writes to ./reference_config.json.",
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should remove the 'Writes to ./reference_config.json.' part since we are now leveraging the output path arg

@alexandraBara alexandraBara merged commit a498e7c into development Jul 16, 2025
5 checks passed
@alexandraBara alexandraBara deleted the alex_diff_logs branch July 16, 2025 14:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants