Skip to content

[FEATURE] Constrain codebase to statically typed and region-based lifetimes Python subset #1839

@oberstet

Description

@oberstet

Summary

Constrain the autobahn-python codebase to a statically typed Python subset, enabling:

  1. All objects have a static type — either automatically inferred by the type checker or explicitly annotated
  2. Implicit Any is forbidden — must be eliminated or explicitly justified
  3. Public API surface is fully typed and safely inferable by tooling
  4. Alignment with modern (2025) Python typing best practices and PEP-compliant conventions

This constraint is a prerequisite for future work toward SLSA Level 4 — compiling typed Python to WASM components for reproducible, verifiable builds.

See also:


Related: PR #1838

This issue supersedes and extends the work started in PR #1838 (@bblommers).

PR #1838 adds type hints to a subset of the codebase:

  • autobahn/twisted/websocket.py
  • autobahn/websocket/protocol.py
  • autobahn/wamp/message.py
  • autobahn/wamp/types.py

This issue establishes the comprehensive style guide and acceptance criteria that PR #1838 and all future typing work should align with.

See PR #1838 Style Analysis below for detailed comparison.


Strategic Context

This issue is part of a broader initiative to enable deterministic, reproducible compilation of the WAMP Python ecosystem (txaio, autobahn-python, zlmdb, cfxdb, wamp-xbr, crossbar) to WebAssembly.

The key architectural principle is:

Python is treated as a source language, not a runtime platform.

Type checking is not merely a linting step — it is the first stage of compilation. To use type tools as a compiler frontend, we must ensure every symbol's type is statically known.

See design documents:


Why autobahn-python?

autobahn-python is our core WAMP client library for Python, and fundamental to crossbar as our WAMP router as well:

  • Implements WAMP protocol (RPC, PubSub) over WebSocket and RawSocket transports
  • Supports both Twisted and asyncio
  • Large codebase with significant public API surface
  • Foundation for crossbar and downstream applications

The codebase currently has:

  • Partial type annotations (inconsistent coverage)
  • Mixed typing styles (legacy Optional, Union alongside modern syntax)
  • Unannotated functions, especially in older modules
  • Implicit attribute creation patterns

This must be addressed systematically across all modules.


PR #1838 Style Analysis

PR #1838 by @bblommers is a valuable contribution that adds type hints to several files. Below is an analysis of the style choices made and their alignment with this project's style guide.

Files Modified in PR #1838

File Scope
autobahn/twisted/websocket.py Twisted WebSocket adapter
autobahn/websocket/protocol.py Core WebSocket protocol
autobahn/wamp/message.py WAMP message types
autobahn/wamp/types.py WAMP type definitions

Style Choices: Aligned with Style Guide

Choice Example from PR Status
Union syntax X | Y str | None, int | str Aligned (PEP 604)
Lowercase generics dict[str, Any], tuple[str, dict], list[bytes] Aligned (PEP 585)
@overload decorator Used in check_or_raise_uri(), Timings.diff() Aligned (PEP 484)
Literal for overloads Literal[True], Literal[False] Aligned (PEP 586)
Remove :type: from docstrings Removed redundant type annotations Aligned
Return type annotations -> None, -> str, -> tuple[...] Aligned

Style Choices: Divergent from Style Guide

Issue Current in PR Required by Style Guide Action Required
from __future__ import annotations Not present Required in every module Add to all files
Legacy typing imports from typing import Optional still present Should be removed (use X | None) Remove and migrate
Mixed Optional usage Optional[str] in some places Use str | None everywhere Migrate all
Incomplete parameter types payload untyped in several methods All parameters must be typed Add missing types

Specific Examples of Divergence

1. Missing from __future__ import annotations:

# CURRENT (autobahn/twisted/websocket.py line 28)
from typing import Any, Optional

# REQUIRED
from __future__ import annotations

from typing import Any  # Optional no longer needed

2. Legacy Optional still used:

# CURRENT (autobahn/twisted/websocket.py)
peer: Optional[str] = None
is_server: Optional[bool] = None

# REQUIRED
peer: str | None = None
is_server: bool | None = None

3. Untyped parameters:

# CURRENT (autobahn/websocket/protocol.py)
def _onMessageFrameData(self, payload) -> None:  # payload untyped
def _onMessageFrame(self, payload) -> None:      # payload untyped
def _onPing(self, payload) -> None:              # payload untyped

# REQUIRED
def _onMessageFrameData(self, payload: bytes) -> None:
def _onMessageFrame(self, payload: bytes) -> None:
def _onPing(self, payload: bytes) -> None:

4. Partially typed return in __iter__:

# CURRENT (autobahn/websocket/protocol.py)
def __iter__(self) -> Iterable[str]:
    return self._timings.__iter__()

# BETTER (more precise)
def __iter__(self) -> Iterator[str]:
    return iter(self._timings)

Coverage Gap

PR #1838 covers only 4 files out of the full autobahn-python codebase. Major modules still requiring typing:

Module Status Priority
autobahn/wamp/protocol.py Untyped High
autobahn/wamp/component.py Untyped High
autobahn/wamp/serializer.py Untyped Medium
autobahn/wamp/auth.py Untyped Medium
autobahn/wamp/cryptosign.py Untyped Medium
autobahn/asyncio/websocket.py Untyped High
autobahn/asyncio/wamp.py Untyped High
autobahn/twisted/wamp.py Untyped High
autobahn/websocket/compress*.py Untyped Low
autobahn/rawsocket/*.py Untyped Medium

Recommendation for PR #1838

To align PR #1838 with this style guide:

  1. Add from __future__ import annotations to all modified files
  2. Replace all Optional[X] with X | None
  3. Remove unused typing imports (Optional, Union, Dict, Tuple)
  4. Add missing parameter types (especially payload: bytes)
  5. Run ruff check --select ANN,UP and fix violations

What "Statically Typed Subset" Means

Required Typing Discipline

All public functions and methods must have:

  • Parameter type annotations
  • Explicit return type annotation
async def call(
    self,
    procedure: str,
    *args: Any,
    **kwargs: Any,
) -> Any:
    ...

Every class must declare instance attributes upfront:

class Session:
    _transport: ITransport | None
    _session_id: int | None
    _realm: str | None
    _authid: str | None
    _authrole: str | None

    def __init__(self) -> None:
        self._transport = None
        self._session_id = None
        ...

All module-level globals must be typed:

_log: Logger = make_logger()
WAMP_SERIALIZERS: Final[dict[str, type[Serializer]]] = {...}

All containers must have explicit type parameters:

_subscriptions: dict[int, Subscription] = {}
_registrations: dict[int, Registration] = {}
_pending_calls: dict[int, Future[Any]] = {}

Forbidden Patterns

Implicit Any:

# BAD
items = []

# GOOD
items: list[Message] = []

Dynamic attribute creation after __init__:

# BAD
self.foo = 1

# GOOD
class Bar:
    foo: int
    def __init__(self) -> None:
        self.foo = 1

Runtime type hacks:

# FORBIDDEN
eval("...")
exec("...")
getattr(obj, dynamic_name)  # where dynamic_name is not a literal

Python Typing Style Guide

This project follows modern Python 3.11+ typing conventions aligned with official PEPs.

Minimum Python Version

Python 3.11+ is required. This enables:

  • Self type (PEP 673)
  • Required / NotRequired for TypedDict (PEP 655)
  • ExceptionGroup and except* syntax
  • Native union syntax without from __future__ import annotations

Required Import

Every module must begin with:

from __future__ import annotations

This enables:

  • Forward references without quotes (PEP 563)
  • Consistent annotation behavior
  • Future compatibility with PEP 649 (Python 3.14+)

Union Types (PEP 604)

Use X | Y syntax, not Union[X, Y] or Optional[X]:

# GOOD
def process(value: str | None) -> int | str:
    ...

# BAD
def process(value: Optional[str]) -> Union[int, str]:
    ...

Built-in Generic Types (PEP 585)

Use lowercase built-in generics, not typing module equivalents:

# GOOD
items: list[int] = []
mapping: dict[str, bytes] = {}
pair: tuple[int, str] = (1, "a")

# BAD
items: List[int] = []
mapping: Dict[str, bytes] = {}

Type Aliases

# GOOD
Callback: TypeAlias = Callable[[int], None]

# For Python 3.12+, prefer PEP 695 syntax:
type Callback = Callable[[int], None]

TypeVar and Generics

from typing import TypeVar

T = TypeVar("T")

def identity(x: T) -> T:
    return x

Protocol for Structural Typing (PEP 544)

Prefer Protocol over ad-hoc duck typing:

from typing import Protocol

class ITransport(Protocol):
    def send(self, data: bytes) -> None: ...
    def close(self) -> None: ...

Constants with Final (PEP 591)

from typing import Final

WAMP_VERSION: Final[int] = 2
DEFAULT_REALM: Final[str] = "realm1"

Literal Types (PEP 586)

from typing import Literal

def set_mode(mode: Literal["json", "msgpack", "cbor"]) -> None:
    ...

Overloads for Conditional Return Types (PEP 484)

from typing import Literal, overload

@overload
def get_session(create: Literal[True]) -> Session: ...
@overload
def get_session(create: Literal[False]) -> Session | None: ...

def get_session(create: bool = False) -> Session | None:
    ...

Self Type (PEP 673)

from typing import Self

class ComponentConfig:
    def with_realm(self, realm: str) -> Self:
        self._realm = realm
        return self

Avoiding Any

Any defeats static analysis and must be avoided.

If unavoidable:

  1. Explicitly justify in a comment
  2. Isolate to minimal scope
  3. Wrap in a typed facade if possible
# ACCEPTABLE: WAMP payload can be any JSON-serializable value
# This is fundamental to the protocol design
payload: Any

Docstrings

Remove redundant :type: and :rtype: annotations when type hints are present:

# GOOD
def connect(host: str, port: int) -> Connection:
    """
    Establish a connection.

    :param host: The hostname or IP address.
    :param port: The port number.
    :returns: An established connection.
    """

# BAD (redundant)
def connect(host: str, port: int) -> Connection:
    """
    :param host: The hostname.
    :type host: str  # REMOVE
    :rtype: Connection  # REMOVE
    """

Type Checking Configuration

Primary Tool: ty (strict mode)

ty is the authoritative type checker for this project.

Configuration in pyproject.toml:

[tool.ty]
python-version = "3.11"

Strict mode invocation:

ty check --warn any-type

Linting: ruff

[tool.ruff]
target-version = "py311"
line-length = 120

[tool.ruff.lint]
select = [
    "ANN",  # flake8-annotations
    "I",    # isort
    "E",    # pycodestyle errors
    "F",    # pyflakes
    "W",    # pycodestyle warnings
    "UP",   # pyupgrade (modernize syntax)
    "TCH",  # flake8-type-checking (imports)
]

[tool.ruff.lint.flake8-annotations]
mypy-init-return = true
suppress-none-returning = false
allow-star-arg-any = false

[tool.ruff.lint.isort]
required-imports = ["from __future__ import annotations"]

Implementation Workflow

Phase 1: Configuration

  1. Update pyproject.toml with ruff and ty configuration
  2. Add from __future__ import annotations to all modules
  3. Update justfile with type checking recipes

Phase 2: Align PR #1838

  1. Update PR Autobahn WebSocket Protocol - Improve typing #1838 to match style guide
  2. Merge PR Autobahn WebSocket Protocol - Improve typing #1838 as foundation

Phase 3: Progressive Typing

Priority order:

  1. Core WAMP protocol (wamp/protocol.py, wamp/component.py)
  2. Asyncio bindings (asyncio/websocket.py, asyncio/wamp.py)
  3. Twisted bindings (twisted/wamp.py)
  4. Serializers and auth (wamp/serializer.py, wamp/auth.py)
  5. WebSocket internals (websocket/*.py)
  6. RawSocket (rawsocket/*.py)

Phase 4: CI Integration

  1. Add type checking to CI workflow
  2. Gate PRs on passing type checks
  3. Document typed subset contract

Scope and Constraints

In scope:

  • Add type annotations to all public APIs
  • Declare class-level attributes
  • Type all containers explicitly
  • Add from __future__ import annotations to all files
  • Migrate legacy typing syntax (OptionalX | None, etc.)

Out of scope (for this issue):

  • Runtime behavior changes
  • Logic rewrites (unless required for stable types)
  • Test typing (tracked separately)

Acceptance Criteria

  • All public functions/methods have parameter and return type annotations
  • All classes declare instance attributes at class level
  • All containers have explicit type parameters
  • No implicit Any (explicit Any only with justification)
  • from __future__ import annotations in every module
  • No legacy Optional, Union, Dict, List, Tuple imports
  • ty check --warn any-type passes with zero errors
  • ruff check . passes with zero errors (ANN rules enabled)
  • Type checking added to CI
  • PR Autobahn WebSocket Protocol - Improve typing #1838 aligned and merged

Related Work

  • PR Autobahn WebSocket Protocol - Improve typing #1838: Initial type hints contribution by @bblommers — to be aligned with this style guide
  • txaio typed subset: Foundation library, being typed in parallel
  • SLSA Level 3 implementation: Current focus on provenance; typed subset enables future Level 4
  • WASM compilation roadmap: Typed subset is prerequisite for Python → WASM compiler frontend

References

PEPs

PEP Title Relevance
PEP 484 Type Hints Foundation
PEP 526 Variable Annotations Class attributes
PEP 544 Protocols: Structural subtyping Interface typing
PEP 563 Postponed Evaluation of Annotations from __future__ import annotations
PEP 585 Type Hinting Generics In Standard Collections list[T] vs List[T]
PEP 586 Literal Types Literal["a", "b"]
PEP 591 Adding a final qualifier Final[T]
PEP 604 Union Operators X | Y syntax
PEP 612 Parameter Specification Variables ParamSpec
PEP 655 Required and NotRequired for TypedDict TypedDict fields
PEP 673 Self Type Self return type
PEP 695 Type Parameter Syntax Python 3.12+ type statement

Tools

  • ty — Astral's type checker (strict mode)
  • ruff — Fast Python linter with annotation rules
  • pyright — Alternative type checker (reference)

Checklist

  • I have searched existing issues to avoid duplicates
  • I have described the problem clearly
  • I have provided use cases
  • I have considered alternatives
  • I have assessed impact and breaking changes

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions