response_integration_structure.md

Deep Dive: Response Integration Structure

Integration Structure Guide

This document outlines the standard folder structure and file contents for creating a new integration. Integrations are located in one of two directories:

• content/response_integrations/third_party/community for community-contributed integrations.
• content/response_integrations/third_party/partner for partner-supported integrations.

All integrations, whether community or partner, follow the same internal structure described below.

Note: All file names must be in snake_case

Folder Structure

An integration is placed inside either the community or partner directory. The integration_name folder itself has the following structure:

integration_name/
├── actions/
│   ├── __init__.py
│   ├── action1.py
│   ├── action1.yaml
│   ├── action2.py
│   └── action2.yaml
│
├── core/
│   ├── __init__.py
│   ├── integration_client.py
│   └── data_models/
│       ├── data_model_1.py
│       └── data_model_2.py
│
├── connectors/
│   ├── __init__.py
│   ├── connector1.py
│   ├── connector1.yaml
│   ├── connector2.py
│   └── connector2.yaml
│
├── widgets/
│   ├── widget1.html
│   ├── widget1.yaml
│   ├── widget2.html
│   └── widget2.yaml
│
├── jobs/
│   ├── __init__.py
│   ├── job1.py
│   ├── job1.yaml
│   ├── job2.py
│   └── job2.yaml
│
├── resources/
│   ├── action1_JsonResult_example.json
│   ├── image.png
│   └── logo.svg
│
├── tests/
│   ├── __init__.py
│   ├── common.py
│   ├── conftest.py
│   ├── core/
│   │   ├── __init__.py
│   │   ├── session.py
│   │   └── product.py
│   │
│   └── test_defaults/
│   │    ├── __init__.py
│   │    └── test_imports.py
│   │
│   └── test_actions/
│       ├── __init__.py
│       ├── test_action1.py
│       └── test_action2.py
│
├── __init__.py
├── .python-version
├── ontology_mapping.yaml
├── definition.yaml
├── pyproject.toml
├── release_notes.yaml
└── uv.lock

File Contents and Purpose

Root Level Files

• __init__.py: Empty file that marks the directory as a Python package.

• .python-version: Specifies the Python version required for the integration (typically 3.11).

• ontology_mapping.yaml: Specifies the default mapping rules of events to SOAR entities. A default configuration must exist if the integration has at least one connector.

• definition.yaml: Contains metadata about the integration, including:

  • API version
  • Integration identifier and display name
  • Configuration schema (API keys, endpoints, etc.)
  • Author information
  • Categories/tags

• pyproject.toml: Defines project dependencies and build settings:

  • Name
  • Description
  • Version number
  • Python version (typically ">=3.11,<3.12")
  • Development dependencies (pytest, mypy, integration_testing, etc.)
  • Production dependencies
  • Tool configurations (pytest, mypy, etc.)
  • Project metadata

• release_notes.yaml: Documents changes for each version:

  • Version numbers
  • Release dates (in YYYY-MM-DD format)
  • Description of changes
  • Bug fixes
  • New features
  • Optional boolean flags can be added and will default to false if omitted.

  Boolean Flags Explained

  new : Set to true for a new component or for the initial 1.0 release. In the latter case, it will be displayed as new integration in the Content Hub.

  deprecated : Set to true if a feature is being phased out but is still functional.

  removed : Set to true when a feature has been completely removed.

  regressive : Set to true if the change is breaking existing functionalities.

  # Example: Adding a new action to an existing integration.
  - description: Added a new action 'action example'.
    integration_version: 2.0
    item_name: action example
    item_type: Action
    new: true
    publish_time: '2025-10-15'

• uv.lock: Lock file generated by the UV package manager, listing exact versions of all dependencies.

actions/ Directory

The actions/ directory contains all the actions that the integration provides:

• __init__.py: Empty file to mark the directory as a Python package.
• action_name*.py: Python implementation files containing action classes:
  • Each action should be a standalone script/class
  • Should include proper type hints
  • Should handle errors gracefully
  • Should document inputs/outputs with Google-style docstrings
  • It is advised to use base classes from tip-common for out-of-the-box error handling, logging, and sdk handling of scripts
• action_name*.yaml: YAML definition files for each action, including:
  • Input parameters with types, descriptions, and examples
  • Output schema
  • Display information (name, description, category)
  • Required permissions

Typical actions include:

• ping.py/ping.yaml: Basic connectivity test to verify API access
• Service-specific actions (e.g., check_ip_reputation.py, fetch_alerts.py)

core/ Directory

The core/ directory contains the core implementation of the integration:

• __init__.py: Empty file to mark the directory as a Python package.
• Include any other modules that are common code for your scripts.
• Include a class that is responsible for authenticating with the third party product
• Include an api_client class that receives an authenticated session and uses it to make requests to the product
• Modules for constants, utility functions, data models, and so on
• This folder can contain subfolders, all python files must have different names, even if in nested folders

tests/ Directory

The tests/ directory contains all test files for the integration:

• __init__.py: Empty file to mark the directory as a Python package.
• common.py: Common test utilities and fixtures:
  • Mock response data
  • Test constants
  • Helper functions
• conftest.py: Common test utilities and fixtures:
  • Shared fixtures
  • Pytest configuration
• test_defaults/: Directory containing default test cases:
  • __init__.py: Empty file to mark the directory as a Python package
  • test_imports.py: Tests that using the integration’s virtual environment, all python components in the integration do not have import issues
• test_actions/: Directory containing all action test modules

Best Practices

1. Modular Design: Keep the code modular with clear separation of concerns.
2. Type Hints: Use Python type hints throughout the codebase for better IDE support and static type checking.
3. Pydantic Models: Use Pydantic for data validation and serialization/deserialization.
4. Error Handling: Implement comprehensive error handling with meaningful error messages.
5. Documentation: Document all classes, methods, and functions with Google-style docstrings.
6. Testing: Write comprehensive tests for all functionality, including:
   • Unit tests for individual components
   • Integration tests with mocked API responses
   • Edge case testing (error conditions, unusual inputs)
7. Configuration: Ensure all configurable aspects of the integration are defined in definition.yaml.
8. Version Control: Follow semantic versioning for releases.
9. Code Style: Adhere to the project's code style guidelines (using ruff for linting and formatting).
10. Security: Never hardcode sensitive information such as API keys or passwords.

Required Implementation

At minimum, every integration should implement:

1. A core client class that handles API communication
2. A Ping action to verify connectivity
3. At least one service-specific action that demonstrates value
4. Comprehensive tests for all components
5. Proper error handling for API failures
6. Type hints throughout the codebase

Developing a New Integration

When creating a new integration:

1. Start by setting up the basic folder structure
2. Define the configuration schema in definition.yaml
3. Create a new pyproject.toml file using uv init and add dependencies like requests and tip-common using uv add.
4. Implement the core client with authentication
5. Create the Ping action for basic connectivity testing
6. Add service-specific actions one by one
7. Write tests as you implement each component
8. Document everything thoroughly
9. Run linting and type checking before submitting