Architecture

Module layout

xferry/
  __main__.py          # python -m xferry entry point
  cli.py               # public console-script wrapper
  server.py            # public server exports

src/
  __main__.py          # compatibility module entry point
  cli.py               # argparse
  server.py            # XFerryServer: socket lifecycle + WS helpers
  settings.py          # INI/env/CLI settings normalization and validation
  runtime_posture.py   # typed, redacted effective launch posture
  extensions.py        # explicit plugin API dataclasses and plugin loading
  metrics.py           # MetricsCollector (thread-safe counters)
  notepad_service.py   # NOTE domain logic shared by HTTP and WebSocket
  request_pipeline.py  # auth/dispatch/send orchestration
  features.py          # typed built-in method policy registry + projections
  config.py            # constants: hidden files, status map
  websocket.py         # RFC 6455 frame parser and handshake
  http/
    request.py         # HTTPRequest parser
    response.py        # HTTPResponse builder
    io.py              # socket reader with Content-Length enforcement
    utils.py           # path helpers + shared descendant resolver
  handlers/
    base.py            # BaseHandler (shared utilities)
    context.py         # narrow storage/metrics/SMUGGLE runtime context
    registry.py        # HandlerRegistry (method -> callable)
    files.py           # GET/POST/PUT/DELETE/FETCH/NONE
    info.py            # INFO, PING
    notepad.py         # NOTE HTTP handlers
    advanced_upload.py # advanced upload transports
    smuggle.py         # SMUGGLE (HTML smuggling demo)
  security/
    auth.py            # Basic Auth + rate limiter
    crypto.py          # XOR/HMAC helpers
    keys.py            # ECDH P-256 for Secure Notepad
    tls.py             # cert generation helpers + built-in ACME/sslip.io integration
    tls_manager.py     # TLSManager: SSL context lifecycle + temp files
  utils/
    captcha.py         # PIL-free CAPTCHA renderer
    smuggling.py       # HTML smuggling template

Request flow

sequenceDiagram
    Client->>Server: TCP connect (maybe TLS)
    Server->>TLSManager: wrap_socket (if TLS)
    Server->>IO: receive_request()
    IO-->>Server: raw bytes
    Server->>RequestPipeline: process()
    RequestPipeline->>HTTPRequest: parse
    RequestPipeline->>Auth: _authenticate_request()
    RequestPipeline->>WS: check_websocket_upgrade() for /notes/ws
    RequestPipeline->>RequestPipeline: _check_payload_size()
    RequestPipeline->>HandlerMixin: dispatch
    HandlerMixin->>HandlerRegistry: lookup
    HandlerRegistry-->>Handler: core or explicitly registered plugin method
    Handler->>NotepadService: NOTE/WS note operations (when applicable)
    Handler-->>RequestPipeline: HTTPResponse
    RequestPipeline->>MetricsCollector: record()
    RequestPipeline-->>Client: response bytes

Public package compatibility

xferry is the current product, distribution, CLI, and public import namespace. New public APIs and examples use xferry.

The implementation package src remains installed and supported throughout the 2.x release line as a deprecated compatibility surface. Its planned removal boundary is 3.0. Consumers should migrate public imports and module invocations to their xferry equivalents during 2.x; see ADR-011.

Built-in handler runtime context

HandlerRuntimeContext is the narrow dependency boundary for the first extracted built-in handler slice. It exposes exactly the upload directory, UploadStorageService, optional MetricsCollector, and a SmuggleTempCoordinator. It is distinct from the public plugin extensions.HandlerContext, whose compatibility contract is unchanged.

The server still creates the underlying registered-path set and lock so existing construction and test hosts retain the same object lifetime. SmuggleTempCoordinator is their sole operational owner: Files asks it whether a generated artifact is registered and unregisters streamed one-shot responses; SMUGGLE performs creation, quota/retention cleanup and usage scans inside coordinator transactions. Neither handler reaches server lock or set fields directly. Metrics and shutdown cleanup use the same context, preserving synchronization and avoiding a second source of truth.

Other handlers continue using the established mixin host during this incremental extraction. New Files/SMUGGLE dependencies belong in HandlerRuntimeContext; do not add another direct server-field dependency.

Concurrency

See ADR-005. One accept loop, ThreadPoolExecutor pool (10 workers by default), keep-alive per worker.

Runtime persistence

Runtime file state is split across a small number of explicit directories:

• --dir / root_dir is the operator-owned content root. The server creates uploads/ for user-visible file operations and notes/ for Secure Notepad ciphertext and plaintext note metadata.
• Temporary self-signed TLS certificates are created under the platform temp directory and removed on process exit.
• ACME state is stored under Path.home() / ".xferry" / "acme", including account keys, domain private keys, and live/<domain>/fullchain.pem plus privkey.pem. TLSManager also reads the legacy Path.home() / ".xferry" / "letsencrypt" cache when the new ACME cache is empty.
• In the Docker image the runtime user is xferry, so ACME state lives under /home/xferry/.xferry. The ACME Compose profile mounts that path as a dedicated named volume; treat it as certificate secret material.

Security layers

1. Transport — TLS 1.2+ via TLSManager.
2. Authentication — Basic Auth with PBKDF2-SHA256 (BasicAuthenticator), rate-limited (AuthRateLimiter).
3. Authorisation — path containment via src.http.utils.resolve_descendant_path() under BaseHandler._get_file_path() / _resolve_safe_path(), plus HIDDEN_FILES checks in handler policy.
4. Upload scope — all user-visible file reads and writes are constrained to <root>/uploads/; the built-in UI and /static/... are served read-only package resources.
5. Upload dispatch — POST/PUT/PATCH/NONE normally use Basic multipart/raw profiles. The exact /_xferry/advanced-routing control route owns process-local prefix/decoder state; a matching standard-method prefix routes directly to AdvancedUploadHandlersMixin with no Basic fallback. Unknown non-standard methods carrying payload retain the Advanced fallback. Advanced carriers cover body formats, headers, query, cookies, and marked paths.
6. Ephemeral session diagnostics — complete ECDH session IDs remain protocol data; logs use only a stable sidfp:<12-hex> fingerprint and unknown-session exceptions are generic.

See the threat model for what each layer buys you.

Operational observability and scan policy

MetricsCollector owns thread-safe counters and gauges. Storage services and handlers report only closed low-cardinality scopes/reasons: uploads, encrypted notes, generated SMUGGLE artifacts, configured quota denials, advanced decode rejections, and five scan scopes. User paths, filenames, note titles, session IDs, arbitrary methods/encodings, and exception messages are never labels.

Existing filesystem traversals report their own item counts and elapsed time. No metric adds a second hot-path traversal. An explicit PING or GET /metrics snapshot refreshes exact upload, note, and SMUGGLE usage and records those three traversals as storage_snapshot; this makes the snapshot O(n) in stored entries.

INFO retains exact total_items, so pagination bounds response size but still sorts/scans the directory in O(n). Aggregate upload quota checks and Notepad usage/listing are likewise exact (the latter is bounded by the default 1,000-note policy). The reproducible local baseline measured INFO at 12.569/214.355 ms and one upload-quota check at 41.902/260.961 ms for 1k/10k files; a 1,000-note list measured 289.664 ms. These timings are observations, not portable SLA thresholds.

The current decision is to keep exact scans and add no cache/index. A correctness-safe index would require startup reconciliation, external filesystem mutation rules, and a fallback scan. Revisit only when representative storage.scans.*.avg_ms remains above 500 ms or an agreed cardinality/throughput target is missed.

Configuration and extension boundaries

src.features.CoreMethodSpec is the single owner for each built-in method's name, handler binding, mutation flag, exact/wildcard CORS eligibility, primary UI group, and exposure note. Handler registration, CORS, browser-mutation checks, and PING projections derive from this registry. supported_methods remains the runtime availability contract; additive method_groups is presentation metadata, while plugin methods remain a separate policy surface.

src.settings.ServerSettings is the operator-facing configuration boundary. It resolves built-in defaults, optional launch-preset defaults, INI files, XFERRY_* environment variables, and explicit CLI flags before the CLI constructs XFerryServer. Preset selection follows file < environment < CLI, but every explicit setting remains above every preset default. Sparse INI provenance retains explicit false, 0, empty nullable values, and values equal to legacy defaults.

src.runtime_posture.RuntimePosture derives the same secret-free URL, exposure, persistence/data paths, TLS/Auth modes, validation state, effective limits, and warning codes for startup, --check-config, and --print-config. The public-direct preset engages strict validation in this layer so services and containers can run xferry --config <file> --check-config before startup.

src.extensions / xferry.extensions is the only supported external plugin surface. Plugins are not auto-discovered; an operator must explicitly list modules in configuration or pass PluginSpec objects when embedding the server. Plugin methods are registered after core methods, cannot override core methods by default, and carry policy metadata for CORS and browser mutation checks. Core method names stay reserved in the single full mode; a plugin cannot claim SMUGGLE or NOTE unless the operator explicitly enables plugins_override_core. Advanced prefix activation is rejected if any plugin owns POST, PUT, PATCH, or NONE; the exact control route remains service-owned even when / is the active prefix.