Create reference config from system settings #15

New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
alexandraBara merged 22 commits into development from alex_diff
Jul 9, 2025
README.md
            
                      Original file line number
                      Diff line number
                      Diff line change
                  
    @@ -17,8 +17,10 @@ The Node Scraper CLI can be used to run Node Scraper plugins on a target system.
  
    options are available:

    ```sh

    usage: node-scraper [-h] [--sys-name STRING] [--sys-location {LOCAL,REMOTE}] [--sys-interaction-level {PASSIVE,INTERACTIVE,DISRUPTIVE}] [--sys-sku STRING] [--sys-platform STRING] [--plugin-configs [STRING ...]]

                        [--system-config STRING] [--connection-config STRING] [--log-path STRING] [--log-level {CRITICAL,FATAL,ERROR,WARN,WARNING,INFO,DEBUG,NOTSET}]

    usage: node-scraper [-h] [--sys-name STRING] [--sys-location {LOCAL,REMOTE}]

                        [--sys-interaction-level {PASSIVE,INTERACTIVE,DISRUPTIVE}] [--sys-sku STRING] [--sys-platform STRING]

                        [--plugin-configs [STRING ...]] [--system-config STRING] [--connection-config STRING] [--log-path STRING]

                        [--log-level {CRITICAL,FATAL,ERROR,WARN,WARNING,INFO,DEBUG,NOTSET}] [--gen-reference-config]

                        {run-plugins,describe,gen-plugin-config} ...

    node scraper CLI

    @@ -32,120 +34,60 @@ positional arguments:
  
    options:

      -h, --help            show this help message and exit

      --sys-name STRING     System name (default: TheraC55)

      --sys-name STRING     System name (default: <my_system_name>)

      --sys-location {LOCAL,REMOTE}

                            Location of target system (default: LOCAL)

      --sys-interaction-level {PASSIVE,INTERACTIVE,DISRUPTIVE}

                            Specify system interaction level, used to determine the type of actions that plugins can perform (default: INTERACTIVE)

                            Specify system interaction level, used to determine the type of actions that plugins can perform (default:

                            INTERACTIVE)

      --sys-sku STRING      Manually specify SKU of system (default: None)

      --sys-platform STRING

                            Specify system platform (default: None)

      --plugin-configs [STRING ...]

                            built-in config names or paths to plugin config JSONs. Available built-in configs: NodeStatus (default: None)

                            built-in config names or paths to plugin config JSONs. Available built-in configs: NodeStatus (default:

                            None)

      --system-config STRING

                            Path to system config json (default: None)

      --connection-config STRING

                            Path to connection config json (default: None)

      --log-path STRING     Specifies local path for node scraper logs, use 'None' to disable logging (default: .)

      --log-level {CRITICAL,FATAL,ERROR,WARN,WARNING,INFO,DEBUG,NOTSET}

                            Change python log level (default: INFO)

      --gen-reference-config

                            Generate reference config. File will be written to ./reference_config.json. (default: False)

    ```

    The plugins to run can be specified in two ways, using a plugin JSON config file or using the

    ### Subcommmands

    Plugins to run can be specified in two ways, using a plugin JSON config file or using the

    'run-plugins' sub command. These two options are not mutually exclusive and can be used together.

    ---

    ### Describing Built-in Configs and Plugins

    1. **'describe' subcommand**

    You can use the `describe` subcommand to display details about built-in configs or plugins.

    #### List all built-in configs:

    List all built-in configs:

    ```sh

    node-scraper describe config

    ```

    #### Show details for a specific built-in config:

    Show details for a specific built-in config

    ```sh

    node-scraper describe config <config-name>

    ```

    #### List all available plugins:

    List all available plugins**

    ```sh

    node-scraper describe plugin

    ```

    #### Show details for a specific plugin:

    Show details for a specific plugin

    ```sh

    node-scraper describe plugin <plugin-name>

    ```

    ---

    ### Plugin Configs

    A plugin JSON config should follow the structure of the plugin config model defined here.

    The globals field is a dictionary of global key-value pairs; values in globals will be passed to

    any plugin that supports the corresponding key. The plugins field should be a dictionary mapping

    plugin names to sub-dictionaries of plugin arguments. Lastly, the result_collators attribute is

    used to define result collator classes that will be run on the plugin results. By default, the CLI

    adds the TableSummary result collator, which prints a summary of each plugin’s results in a

    tabular format to the console.

    ```json

    {

        "globals_args": {},

        "plugins": {

            "BiosPlugin": {

                "analysis_args": {

                    "exp_bios_version": "TestBios123"

                }

            },

            "RocmPlugin": {

                "analysis_args": {

                    "exp_rocm_version": "TestRocm123"

                }

            }

        }

    }

    ```

    ### 'gen-plugin-config' sub command

    The 'gen-plugin-config' sub command can be used to generate a plugin config JSON file for a plugin

    or list of plugins that can then be customized. Plugin arguments which have default values will be

    prepopulated in the JSON file, arguments without default values will have a value of 'null'.

    #### 'gen-plugin-config' Examples

    Generate a config for the DmesgPlugin:

    ```sh

    node-scraper gen-plugin-config --plugins DmesgPlugin

    ```

    This would produce the following config:

    ```json

    {

      "global_args": {},

      "plugins": {

        "DmesgPlugin": {

          "collection": true,

          "analysis": true,

          "system_interaction_level": "INTERACTIVE",

          "data": null,

          "analysis_args": {

            "analysis_range_start": null,

            "analysis_range_end": null,

            "check_unknown_dmesg_errors": true,

            "exclude_category": null

          }

        }

      },

      "result_collators": {}

    }

    ```

    ### 'run-plugins' sub command

    2. **'run-plugins' sub command**

    The plugins to run and their associated arguments can also be specified directly on the CLI using

    the 'run-plugins' sub-command. Using this sub-command you can specify a plugin name followed by

    the arguments for that particular plugin. Multiple plugins can be specified at once.

    @@ -167,7 +109,7 @@ options:
  
    ```

    #### 'run-plugins' Examples

    Examples

    Run a single plugin

    ```sh

    @@ -191,8 +133,69 @@ Use plugin configs and 'run-plugins'
  
    node-scraper run-plugins BiosPlugin

    ```

    3. **'gen-plugin-config' sub command**

    The 'gen-plugin-config' sub command can be used to generate a plugin config JSON file for a plugin

    or list of plugins that can then be customized. Plugin arguments which have default values will be

    prepopulated in the JSON file, arguments without default values will have a value of 'null'.

    Examples

    Generate a config for the DmesgPlugin:

    ```sh

    node-scraper gen-plugin-config --plugins DmesgPlugin

    ```

    This would produce the following config:

    ```json

    {

      "global_args": {},

      "plugins": {

        "DmesgPlugin": {

          "collection": true,

          "analysis": true,

          "system_interaction_level": "INTERACTIVE",

          "data": null,

          "analysis_args": {

            "analysis_range_start": null,

            "analysis_range_end": null,

            "check_unknown_dmesg_errors": true,

            "exclude_category": null

          }

        }

      },

      "result_collators": {}

    }

    ```

    ### Plugin Configs

    A plugin JSON config should follow the structure of the plugin config model defined here.

    The globals field is a dictionary of global key-value pairs; values in globals will be passed to

    any plugin that supports the corresponding key. The plugins field should be a dictionary mapping

    plugin names to sub-dictionaries of plugin arguments. Lastly, the result_collators attribute is

    used to define result collator classes that will be run on the plugin results. By default, the CLI

    adds the TableSummary result collator, which prints a summary of each plugin’s results in a

    tabular format to the console.

    ```json

    {

        "globals_args": {},

        "plugins": {

            "BiosPlugin": {

                "analysis_args": {

                    "exp_bios_version": "TestBios123"

                }

            },

            "RocmPlugin": {

                "analysis_args": {

                    "exp_rocm_version": "TestRocm123"

                }

            }

        }

    }

    ```

    ### '--plugin-configs' example

    1. **'--plugin-configs' command**

    A plugin config can be used to compare the system data against the config specifications:

    ```sh

    node-scraper --plugin-configs plugin_config.json

    @@ -249,3 +252,42 @@ Here is an example of a comprehensive plugin config that specifies analyzer args
  
      "desc": "My golden config"

    }

    ```

    2. **'gen-reference-config' command**

    This command can be used generate a reference config that is populated with current system

    configurations. The plugins that use analyzer args, where applied, will be populated with system

    data.

    Sample command:

    ```sh

    node-scraper --gen-reference-config run-plugins BiosPlugin OsPlugin

    ```

    This will generate the following config:

    ```json

    {

      "global_args": {},

      "plugins": {

        "BiosPlugin": {

          "analysis_args": {

            "exp_bios_version": [

              "M17"

            ],

            "regex_match": false

          }

        },

        "OsPlugin": {

          "analysis_args": {

            "exp_os": [

              "8.10"

            ],

            "exact_match": true

          }

        }

      },

      "result_collators": {}

    ```

    This can be later used on a different platform for comparison, using the steps at #2:

    ```sh

    node-scraper --plugin-configs reference_config.json

    ```
Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Create reference config from system settings #15

Uh oh!

Diff view

Diff view

There are no files selected for viewing

Uh oh!
