Sprint 1A: Vision foundation and Vision Lab

.github/workflows/build.yml

@@ -31,6 +31,21 @@ jobs:
       - name: Run unit tests
         run: ./gradlew test --stacktrace
 
+      - name: Summarize unit tests
+        run: |
+          total=0; failures=0; errors=0; skipped=0
+          for f in app/build/test-results/test*/TEST-*.xml; do
+            [ -e "$f" ] || continue
+            t=$(grep -o 'tests="[0-9]*"' "$f" | head -1 | grep -o '[0-9]*')
+            fl=$(grep -o 'failures="[0-9]*"' "$f" | head -1 | grep -o '[0-9]*')
+            er=$(grep -o 'errors="[0-9]*"' "$f" | head -1 | grep -o '[0-9]*')
+            sk=$(grep -o 'skipped="[0-9]*"' "$f" | head -1 | grep -o '[0-9]*')
+            total=$((total + ${t:-0})); failures=$((failures + ${fl:-0}))
+            errors=$((errors + ${er:-0})); skipped=$((skipped + ${sk:-0}))
+          done
+          echo "unit tests: $total, failures: $failures, errors: $errors, skipped: $skipped"
+          if [ "$failures" -ne 0 ] || [ "$errors" -ne 0 ]; then exit 1; fi
+
       - name: Compile instrumentation tests
         run: ./gradlew assembleAndroidTest --stacktrace
 

.gitignore

@@ -5,3 +5,6 @@ local.properties
 *.iml
 .idea/
 .DS_Store
+
+# Superpowers scratch workspace (ledger, briefs, review packages)
+.superpowers/

README.md

@@ -1,20 +1,37 @@
-# DraftLens — Sprint 0.1 Capture Hardening
+# DraftLens — Sprint 1A Vision Foundation
 
-Native Android app that assists MLBB Draft Mode players. Sprint 0.1 hardens the previously working Foundation: **live screen capture with user consent, in a foreground service, with real-time debug preview, strict lifecycle accounting and bounded/stale-free state handling.**
+Native Android app that assists MLBB Draft Mode players. Sprint 1A turns the previously
+working capture path into a real **image-bearing** foundation: frames carry pixels, geometry
+is tested, preprocessing validates before it renders, and a Vision Lab lets you place manual
+slot annotations and save calibration profiles.
 
-**Maintainer-reported device smoke:** Sprint 0 capture and preview were used on a physical Android device and confirmed working. Device/version details and expanded matrix results are listed in [docs/TESTING_PHYSICAL_DEVICE.md](docs/TESTING_PHYSICAL_DEVICE.md) as pending.**
+**Maintainer-reported device smoke:** the Sprint 0 capture path was used on a physical Android
+device and confirmed working. Sprint 1A itself has **not** been verified on a physical device —
+see [docs/TESTING_PHYSICAL_DEVICE.md](docs/TESTING_PHYSICAL_DEVICE.md) and
+[docs/VISION_TESTING.md](docs/VISION_TESTING.md).
 
-## What works in Sprint 0
+## What works in Sprint 1A
 - MediaProjection consent flow (`ActivityResultContracts.CreateScreenCaptureIntent`)
 - Foreground service (`mediaProjection` type) with persistent notification + Stop action
 - Frame acquisition at 2 FPS default, configurable 334–2000 ms (hard cap 3 FPS)
 - Live frame preview in the debug screen, frame count / resolution / latency / timestamp
 - Pause / resume, rotation & resize handling, clean resource teardown on stop/revoke
-- Explicit `NotImplemented` detection seams — no fake results anywhere
+- Non-null frame contract: `FramePayload`, `OriginalFrame`, `ProcessedFrame`, `SlotImage`
+- Rotation-aware geometry with round-trip tests (`FrameTransform`)
+- Validated preprocessing with honest failure codes (`PreprocessingMath`,
+  `PixelArrayFramePreprocessor`, `AndroidFramePreprocessor`)
+- Manual slot annotations with stable ids, bounds/duplicate/overlap validation and a strict
+  JSON codec
+- Versioned calibration profiles: 1 % aspect-ratio tolerance, exact orientation match,
+  unsupported-schema refusal, delete-as-escape-hatch
+- Vision Lab tab: frame preview, freeze/resume, explicit processing, interactive annotation
+  canvas, text slot editor, profile controls
 
 ## What is deliberately NOT here (later milestones)
 ML hero recognition, recommendation engine, counter/synergy/role scoring, floating overlays,
-API sync, player profiling, builds. See `tasks/plan.md` and `docs/ARCHITECTURE.md`.
+API sync, player profiling, builds. The Vision Lab says **"Detection: not implemented"** —
+manual slots are calibration data, never detections. See `tasks/plan.md` and
+`docs/ARCHITECTURE.md`.
 
 ## Build
 Requires JDK 17 and Android SDK platform 35, build-tools 35.
@@ -25,19 +42,30 @@ Requires JDK 17 and Android SDK platform 35, build-tools 35.
 ```
 GitHub Actions (`.github/workflows/build.yml`) runs exactly those steps on push/PR and uploads
 the APK artifact. **This authoring environment is aarch64 and the SDK's aapt2/build-tools are
-x86_64-only, so no APK was assembled locally — the CI run is the build of record.** See
-Implementation Report in this repo's history / final message for what was and was not executed.
+x86_64-only, so nothing that touches resources can be assembled locally — the CI run is the
+build of record.** See `docs/VISION_TESTING.md` for exactly what ran.
 
 ## Run
-1. Install APK on a physical device (emulator validates compile, not MediaProjection reliability).
-2. Tap **Start Capture** → grant the system screen-capture consent dialog.
-3. MLBB (or any app) — the debug screen shows live frames.
-4. **Stop Capture**, or the notification's **Stop**, or revoke consent in Settings to exercise teardown.
+1. Install the APK from the `draftlens-debug` CI artifact on a physical device (the emulator
+   validates compilation, not MediaProjection reliability).
+2. Open the app; the **Debug Capture** tab is first.
+3. Tap **Start Capture** → grant the system screen-capture consent dialog.
+4. MLBB (or any app) — the debug screen shows live frames.
+5. Switch to the **Vision Lab** tab. Capture keeps running while you are there.
+6. Freeze a frame, drag out a slot rectangle, then edit it in the text editor below the
+   canvas. Press **Process Frame** to run the preprocessing engine on the selected frame.
+7. Save the annotations as a calibration profile; apply it later on a matching frame.
+8. **Stop Capture**, or the notification's **Stop**, or revoke consent in Settings to
+   exercise teardown.
 
 ## Docs
-- `docs/ARCHITECTURE.md` — module layout, state machine, data flow
+- `docs/ARCHITECTURE.md` — module layout, state machine, data flow, Vision boundary
+- `docs/VISION_ARCHITECTURE.md` — Vision packages, ownership, processing rules
+- `docs/VISION_FRAME_CONTRACT.md` — payload contract, ownership, original/processed/slot frames
+- `docs/VISION_COORDINATES.md` — normalized coordinates, rotation tables, crop semantics
+- `docs/VISION_LAB.md` — user-facing controls, freeze behavior, slot editing, profile rules
+- `docs/VISION_TESTING.md` — test inventory, CI gates, what was and was not run
 - `docs/SCREEN_CAPTURE.md` — MediaProjection lifecycle, Android 14+ rules, limits
 - `docs/DATA_SOURCES.md` — Rone Arena vs MLBBDex audit (verified live 2026-11-04)
-- `docs/PRIVACY.md` — consent, no-upload, game-internals rules
 - `docs/decisions/ADR-001-android-native.md` — why native, single-module, and version pins
 - `docs/IMPLEMENTATION_REPORT.md` — executed verification, limitations, next tasks

app/build.gradle.kts

@@ -1,6 +1,7 @@
 plugins {
     alias(libs.plugins.android.application)
     alias(libs.plugins.kotlin.android)
+    alias(libs.plugins.kotlin.serialization)
     alias(libs.plugins.composeCompiler)
 }
 
@@ -66,6 +67,7 @@ dependencies {
     implementation(libs.androidx.ui.tooling.preview)
     implementation(libs.androidx.material3)
     implementation(libs.kotlinx.coroutines.android)
+    implementation(libs.kotlinx.serialization.json)
     // Room deferred until first persisted entity is defined
     implementation(libs.androidx.datastore.preferences)
     debugImplementation(libs.androidx.ui.tooling)

app/src/main/kotlin/com/draftlens/DraftLensApp.kt

@@ -1,10 +1,30 @@
 package com.draftlens
 
 import android.app.Application
+import com.draftlens.capture.CaptureVisionFrameSource
 import com.draftlens.datastore.AppPreferencesRepository
+import com.draftlens.datastore.CalibrationProfileDataStore
+import com.draftlens.vision.calibration.CalibrationRepository
+import com.draftlens.vision.model.VisionFrameSource
+import kotlinx.coroutines.CoroutineScope
+import kotlinx.coroutines.Dispatchers
+import kotlinx.coroutines.SupervisorJob
 
 class DraftLensApp : Application() {
     val preferences: AppPreferencesRepository by lazy {
         AppPreferencesRepository(this)
     }
+
+    /** Calibration metadata only; never screenshots or pixels. */
+    val calibrationProfiles: CalibrationRepository by lazy {
+        CalibrationProfileDataStore(this)
+    }
+
+    /**
+     * The capture-to-vision adapter, owned by the process rather than by a screen, so a
+     * retained Vision Lab keeps receiving frames when the Activity is recreated.
+     */
+    val visionSource: VisionFrameSource by lazy {
+        CaptureVisionFrameSource.fromHolder(CoroutineScope(SupervisorJob() + Dispatchers.Default))
+    }
 }

app/src/main/kotlin/com/draftlens/MainActivity.kt

@@ -4,18 +4,61 @@ import android.os.Bundle
 import androidx.activity.ComponentActivity
 import androidx.activity.compose.setContent
 import androidx.activity.enableEdgeToEdge
+import androidx.compose.foundation.layout.Column
+import androidx.compose.foundation.layout.fillMaxSize
+import androidx.compose.foundation.layout.fillMaxWidth
+import androidx.compose.foundation.layout.statusBarsPadding
+import androidx.compose.material3.Tab
+import androidx.compose.material3.TabRow
+import androidx.compose.material3.Text
+import androidx.compose.runtime.getValue
+import androidx.compose.runtime.mutableIntStateOf
+import androidx.compose.runtime.saveable.rememberSaveable
+import androidx.compose.runtime.setValue
+import androidx.compose.ui.Modifier
 import com.draftlens.feature.home.DebugCaptureScreen
+import com.draftlens.feature.visionlab.VisionLabScreen
 
+/**
+ * Two destinations: the capture debug screen and the Vision Lab.
+ *
+ * Capture lives in a foreground service, so switching tabs never stops it. The vision
+ * source is remembered here so it keeps observing capture while the other tab is open.
+ */
 class MainActivity : ComponentActivity() {
     override fun onCreate(savedInstanceState: Bundle?) {
         super.onCreate(savedInstanceState)
         enableEdgeToEdge()
         val app = application as DraftLensApp
         setContent {
-            DebugCaptureScreen(
-                preferences = app.preferences,
-                activity = this
-            )
+            var selectedTab by rememberSaveable { mutableIntStateOf(0) }
+
+            Column(modifier = Modifier.fillMaxSize()) {
+                TabRow(
+                    selectedTabIndex = selectedTab,
+                    modifier = Modifier
+                        .fillMaxWidth()
+                        .statusBarsPadding(),
+                ) {
+                    Tab(
+                        selected = selectedTab == 0,
+                        onClick = { selectedTab = 0 },
+                        text = { Text("Debug Capture") },
+                    )
+                    Tab(
+                        selected = selectedTab == 1,
+                        onClick = { selectedTab = 1 },
+                        text = { Text("Vision Lab") },
+                    )
+                }
+                when (selectedTab) {
+                    0 -> DebugCaptureScreen(preferences = app.preferences, activity = this@MainActivity)
+                    else -> VisionLabScreen(
+                        calibrationRepository = app.calibrationProfiles,
+                        source = app.visionSource,
+                    )
+                }
+            }
         }
     }
 }

app/src/main/kotlin/com/draftlens/capture/CaptureSessionService.kt

@@ -17,12 +17,15 @@ import android.os.Build
 import android.os.IBinder
 import android.os.SystemClock
 import android.util.Log
+import android.view.Display
+import android.view.Surface
 import android.view.WindowManager
 import androidx.core.app.NotificationCompat
 import androidx.core.app.ServiceCompat
 import com.draftlens.MainActivity
 import com.draftlens.R
 import com.draftlens.models.CaptureStatus
+import com.draftlens.vision.model.FrameRotation
 import kotlinx.coroutines.CoroutineScope
 import kotlinx.coroutines.Dispatchers
 import kotlinx.coroutines.Job
@@ -50,6 +53,7 @@ class CaptureSessionService : Service() {
 
     private var projection: MediaProjection? = null
     private var virtualDisplay: VirtualDisplay? = null
+    private var displayListener: DisplayManager.DisplayListener? = null
     private var captureJob: Job? = null
     private var frameJob: Job? = null
     private var foregroundPromoted = false
@@ -159,9 +163,11 @@ class CaptureSessionService : Service() {
                         val wm = getSystemService(Context.WINDOW_SERVICE) as WindowManager
                         val (captureWidth, captureHeight) = captureDimensions(wm)
                         val dpi = resources.configuration.densityDpi
+                        val displayRotation = currentDisplayRotation()
 
-                        pipeline.start(captureWidth, captureHeight, dpi)
+                        pipeline.start(captureWidth, captureHeight, dpi, displayRotation)
                         createVirtualDisplay(proj, captureWidth, captureHeight, dpi)
+                        registerDisplayListener()
                         CaptureStateHolder.beginCapture(captureWidth, captureHeight, dpi)
                         collectFrames()
                     }
@@ -187,6 +193,31 @@ class CaptureSessionService : Service() {
         return point.x to point.y
     }
 
+    /** Device orientation at acquisition; recorded separately from pixel correction. */
+    @Suppress("DEPRECATION")
+    private fun currentDisplayRotation(): FrameRotation {
+        return try {
+            val display = (getSystemService(Context.DISPLAY_SERVICE) as DisplayManager)
+                .getDisplay(Display.DEFAULT_DISPLAY)
+            if (display == null) {
+                Log.w(TAG, "Default display unavailable; assuming display rotation 0")
+                FrameRotation.ROTATION_0
+            } else {
+                display.rotation.toFrameRotation()
+            }
+        } catch (e: Exception) {
+            Log.w(TAG, "Display rotation unavailable; assuming display rotation 0", e)
+            FrameRotation.ROTATION_0
+        }
+    }
+
+    private fun Int.toFrameRotation(): FrameRotation = when (this) {
+        Surface.ROTATION_90 -> FrameRotation.ROTATION_90
+        Surface.ROTATION_180 -> FrameRotation.ROTATION_180
+        Surface.ROTATION_270 -> FrameRotation.ROTATION_270
+        else -> FrameRotation.ROTATION_0
+    }
+
     private fun promoteToForegroundMediaProjection(): Boolean {
         if (foregroundPromoted) {
             Log.i(TAG, "Foreground service already promoted with mediaProjection type (id=$NOTIFICATION_ID)")
@@ -259,23 +290,79 @@ class CaptureSessionService : Service() {
             return
         }
         if (CaptureStateHolder.state.value.status != CaptureStatus.CAPTURING) return
-        if (width == CaptureStateHolder.state.value.displayWidth &&
-            height == CaptureStateHolder.state.value.displayHeight
-        ) {
-            return
+
+        val displayRotation = currentDisplayRotation()
+        val change = DisplayChangePolicy.classify(
+            CaptureStateHolder.state.value.displayWidth,
+            CaptureStateHolder.state.value.displayHeight,
+            width,
+            height,
+        )
+        when (change) {
+            is DisplayChange.RotationOnly -> {
+                // The capture size is unchanged: only the rotation metadata needs updating.
+                updateDisplayRotation(displayRotation)
+                Log.i(TAG, "Display rotation updated to $displayRotation without resizing")
+            }
+
+            is DisplayChange.Resize -> {
+                try {
+                    val replacement = pipeline.prepareResize(width, height, densityDpi, displayRotation) ?: return
+                    // Caller owns cleanup after detach; VirtualDisplay must not retain a released Surface.
+                    display.setSurface(null)
+                    replacement.closePrevious()
+                    display.setSurface(replacement.surface)
+                    display.resize(width, height, densityDpi)
+                    CaptureStateHolder.updateDisplay(width, height, densityDpi)
+                    Log.i(TAG, "VirtualDisplay resized (${width}x${height}, ${densityDpi}dpi)")
+                } catch (e: Exception) {
+                    failProjectionSession("Failed to resize VirtualDisplay", e)
+                }
+            }
+        }
+    }
+
+    /**
+     * Records a new device orientation on the pipeline, independently of any resize.
+     * The pixel correction stays [FrameRotation.ROTATION_0]: MediaProjection buffers are
+     * already display-oriented, so the rotation is metadata, never a transform.
+     */
+    private fun updateDisplayRotation(rotation: FrameRotation) {
+        pipeline.updateDisplayRotation(rotation)
+    }
+
+    /**
+     * Watches the default display so an orientation change is picked up even when the
+     * captured content keeps its size and MediaProjection reports no resize.
+     */
+    private fun registerDisplayListener() {
+        val manager = getSystemService(Context.DISPLAY_SERVICE) as? DisplayManager ?: return
+        val listener = object : DisplayManager.DisplayListener {
+            override fun onDisplayAdded(displayId: Int) = Unit
+            override fun onDisplayRemoved(displayId: Int) = Unit
+
+            override fun onDisplayChanged(displayId: Int) {
+                if (displayId == Display.DEFAULT_DISPLAY) {
+                    updateDisplayRotation(currentDisplayRotation())
+                }
+            }
         }
+        try {
+            manager.registerDisplayListener(listener, null)
+            displayListener = listener
+        } catch (e: Exception) {
+            Log.w(TAG, "Display listener registration failed", e)
+        }
+    }
 
+    private fun unregisterDisplayListener() {
+        val listener = displayListener ?: return
+        displayListener = null
         try {
-            val replacement = pipeline.prepareResize(width, height, densityDpi) ?: return
-            // Caller owns cleanup after detach; VirtualDisplay must not retain a released Surface.
-            display.setSurface(null)
-            replacement.closePrevious()
-            display.setSurface(replacement.surface)
-            display.resize(width, height, densityDpi)
-            CaptureStateHolder.updateDisplay(width, height, densityDpi)
-            Log.i(TAG, "VirtualDisplay resized (${width}x${height}, ${densityDpi}dpi)")
+            (getSystemService(Context.DISPLAY_SERVICE) as? DisplayManager)
+                ?.unregisterDisplayListener(listener)
         } catch (e: Exception) {
-            failProjectionSession("Failed to resize VirtualDisplay", e)
+            Log.w(TAG, "Display listener unregister failed", e)
         }
     }
 
@@ -301,6 +388,7 @@ class CaptureSessionService : Service() {
         frameJob = null
         try { virtualDisplay?.release() } catch (e: Exception) { Log.w(TAG, "VirtualDisplay release failed", e) }
         virtualDisplay = null
+        unregisterDisplayListener()
         try {
             projection?.unregisterCallback(projectionCallback)
             projection?.stop()