|
13 | 13 |
|
14 | 14 | ## Table of Contents |
15 | 15 |
|
| 16 | +- [What's new (2026-06-17)](#whats-new-2026-06-17) |
16 | 17 | - [What's new (2026-06)](#whats-new-2026-06) |
17 | 18 | - [What's new (2026-05)](#whats-new-2026-05) |
18 | 19 | - [Features](#features) |
|
56 | 57 |
|
57 | 58 | --- |
58 | 59 |
|
| 60 | +## What's new (2026-06-17) |
| 61 | + |
| 62 | +Thirty-plus automation primitives across input realism, vision, flow |
| 63 | +control, triggers, window management, and file security — plus recoverable |
| 64 | +deletion and an editor undo. Each ships with a headless API, an `AC_*` |
| 65 | +executor command, and a visual Script Builder entry; vision and window |
| 66 | +features keep their geometry / IO operations injectable so the logic is |
| 67 | +fully unit-tested. Full reference page: |
| 68 | +[`docs/source/Eng/doc/new_features/v4_features_doc.rst`](docs/source/Eng/doc/new_features/v4_features_doc.rst). |
| 69 | + |
| 70 | +**Human-like input** |
| 71 | +- **Human-like mouse motion** — `move_mouse_humanized` walks an eased, bowed cubic-Bezier path with optional overshoot + jitter, deterministic by `seed` (`AC_human_move`). |
| 72 | +- **Human-like typing** — `type_text_humanized` types character by character with a jittered per-key delay and optional "thinking" pauses, seedable (`AC_human_type`). |
| 73 | + |
| 74 | +**Vision** |
| 75 | +- **VLM natural-language assertion** — `assert_by_description` asks a vision-language model whether the screen matches a description; the `verify()` companion to `locate_by_description` (`AC_assert_vlm`). |
| 76 | +- **Scroll-to-find** — `scroll_until_visible` scrolls a direction until a template image or OCR text appears, or the budget runs out (`AC_scroll_to_find`). |
| 77 | +- **Region colour stats** — `region_color_stats` reports a region's average + dominant colour and that colour's pixel fraction (`AC_region_color_stats`). |
| 78 | +- **QR reading** — `read_qr_codes` decodes QR codes in a screen region via OpenCV's `QRCodeDetector` (no new dependency) (`AC_read_qr`). |
| 79 | + |
| 80 | +**Flow control & variables** |
| 81 | +- **Reusable macros** — `AC_define_macro` / `AC_call_macro`: define a named, parameterised action sub-routine once and call it with `${arg}` bindings. |
| 82 | +- **In-process parallel** — `AC_parallel` runs branch action lists concurrently, each on an isolated executor so branches never race on shared variables. |
| 83 | +- **Performance-budget assertion** — `assert_duration` / `AC_assert_duration` fails a block that takes longer than a millisecond budget. |
| 84 | +- **Read into a variable** — `AC_ocr_to_var`, `AC_shell_to_var`, `AC_read_file_to_var`, `AC_http_to_var` (body or dotted JSON path), `AC_now_to_var` (strftime), `AC_random_to_var` (seeded int / float / choice). |
| 85 | +- **Transform a variable** — `AC_transform_var`: upper / lower / strip / title / replace / regex-extract / slice, in place or into a new variable. |
| 86 | +- **Assert a variable** — `assert_variable` / `AC_assert_var`: eq / ne / lt / gt / contains / regex through the assertion DSL. |
| 87 | + |
| 88 | +**Triggers & smart waits** |
| 89 | +- **Composite triggers** — `AllOfTrigger` / `AnyOfTrigger` / `SequenceTrigger` combine any existing trigger by boolean AND / OR / ordered sequence. |
| 90 | +- **Cron trigger** — `CronTrigger` fires on a five-field cron expression, composing with the boolean triggers (e.g. "at 09:00 *and* only if the image is on screen"). |
| 91 | +- **More smart waits** — `wait_until_clipboard_changes` (`AC_wait_clipboard_change`) and `wait_until_window_closed` (`AC_wait_window_closed`). |
| 92 | + |
| 93 | +**Window management** |
| 94 | +- **Per-window capture** — `capture_window` screenshots exactly a window's bounds by title (`AC_capture_window`). |
| 95 | +- **Layout save / restore** — `save_window_layout` / `restore_window_layout` snapshot every window's position to JSON and move them all back later (`AC_save_window_layout` / `AC_restore_window_layout`). |
| 96 | +- **Snap / tile** — `snap_window` moves a window to a screen half, quarter, or maximize (`AC_snap_window`). |
| 97 | + |
| 98 | +**File security & safety** |
| 99 | +- **Action-file signing** — `sign_action_file` / `verify_action_file` (HMAC-SHA256 sidecar); `execute_files` can require signatures via `JE_AUTOCONTROL_REQUIRE_SIGNED_ACTIONS` (`AC_sign_action_file` / `AC_verify_action_file`). |
| 100 | +- **Action-file encryption** — `encrypt_action_file` / `decrypt_action_file` (Fernet, AES-128-CBC + HMAC) (`AC_encrypt_action_file` / `AC_decrypt_action_file`). |
| 101 | +- **Recoverable deletion** — `move_to_trash` sends a file to the OS recycle bin (Win32 `SHFileOperation` undo flag / macOS Trash / Linux XDG trash, preferring `send2trash`) (`AC_move_to_trash`). |
| 102 | + |
| 103 | +**Reporting & notifications** |
| 104 | +- **Screenshot annotation** — `annotate_screenshot` draws labelled boxes / translucent highlights / arrows / text onto a capture (`AC_annotate_screenshot`). |
| 105 | +- **Desktop notifications** — `notify` shows a cross-platform toast (notify-send / osascript / PowerShell), injection-safe (`AC_notify`). |
| 106 | + |
| 107 | +**GUI** |
| 108 | +- **Recording Editor undo** — every edit is snapshotted; **Ctrl+Z** (and an Undo button) restore the prior state. |
| 109 | +- **Triggers tab** — "Combine selected" wraps chosen triggers into a composite; new **Cron** trigger type. |
| 110 | +- **Assertions tab** — new **VLM** ("screen matches description") assertion kind. |
| 111 | +- Every new `AC_*` command appears in the visual **Script Builder**. |
| 112 | + |
| 113 | +**Fixes** — repaired the USB-passthrough approval-prompt crash on PySide6 6.11.1 (`Q_ARG(object)` → a Qt signal), eight stale / broken GUI + USB tests, two lost exception chains, and brought thirteen functions back under the cyclomatic-complexity gate. |
| 114 | + |
59 | 115 | ## What's new (2026-06) |
60 | 116 |
|
61 | 117 | Nine additions that turn the automation primitives into a full **QA / test |
@@ -142,6 +198,7 @@ sense) a Qt GUI tab. Full reference page: |
142 | 198 | ## Features |
143 | 199 |
|
144 | 200 | - **QA / Test Framework** — assertion DSL (`assert_text` / `_image` / `_pixel` / `_window` + audio/video assertions), data-driven execution (CSV / JSON / SQLite / Excel → `AC_for_each_row`), a scored `run_suite` with setup/teardown/tags, JUnit + Allure report output, flaky-test detection with auto-quarantine, accessibility / i18n auditing (missing labels, WCAG contrast, truncation), and a parallel mobile device matrix. See [What's new (2026-06)](#whats-new-2026-06) |
| 201 | +- **Automation toolkit** — human-like mouse motion + typing, VLM / variable / duration assertions, reusable macros + in-process parallel blocks, composite + cron triggers, read-into-a-variable commands (OCR / shell / file / HTTP / time / random), variable transforms, scroll-to-find, region colour stats, QR reading, per-window capture / layout save-restore / snap, screenshot annotation, desktop notifications, action-file signing + encryption, recoverable (recycle-bin) deletion, and Recording-Editor undo. See [What's new (2026-06-17)](#whats-new-2026-06-17) |
145 | 202 | - **Mouse Automation** — move, click, press, release, drag, and scroll with precise coordinate control |
146 | 203 | - **Keyboard Automation** — press/release individual keys, type strings, hotkey combinations, key state detection |
147 | 204 | - **Image Recognition** — locate UI elements on screen using OpenCV template matching with configurable threshold |
|
0 commit comments