fix: permit clear() when create_index=False, and drop its FT.INFO dependency

docs/user_guide/installation.md

@@ -196,7 +196,8 @@ The command-to-category mapping below was measured against live servers rather t
 |---|---|---|---|
 | `index.query()`, `index.search()`, `index.aggregate()` | `FT.SEARCH`, `FT.AGGREGATE` | Yes | Yes |
 | `index.load()` | `HSET` or `JSON.SET` (needs key access) | Yes | Yes |
-| `index.exists()`, `index.info()`, `index.clear()`, `SearchIndex.from_existing()`, `rvl index info`, `rvl stats` | `FT.INFO` | Yes | **No** |
+| `index.clear()` | `FT.SEARCH`, then `DEL` per batch | Yes | Yes |
+| `index.exists()`, `index.info()`, `SearchIndex.from_existing()`, `rvl index info`, `rvl stats` | `FT.INFO` | Yes | **No** |
 | `index.create()` | `FT.CREATE` | Yes | **No** |
 | `index.delete()`, `rvl index delete`, `rvl index destroy` | `FT.DROPINDEX` | Yes | Yes |
 | Enumerating indexes (see below) | `FT._LIST` | **No** | **No** |
@@ -240,7 +241,7 @@ cache = SemanticCache(
 
 `create_index=False` is available on `SemanticCache`, `MessageHistory`, `SemanticMessageHistory` and `SemanticRouter`. It skips the existence check, the comparison of your schema against the live index, and index creation — the constructor issues no index command at all. Pass it when the index is managed externally, or when the credential cannot run `FT.INFO`. It cannot be combined with `overwrite=True`, which asks for the opposite.
 
-A `SearchIndex` used directly needs nothing special: build it with `from_dict()` or `from_yaml()`, then load and query. Two of its methods stay unavailable, because both read index metadata: `from_existing()`, which reconstructs a schema out of Redis, and `clear()`, which starts by calling `info()`.
+A `SearchIndex` used directly needs nothing special: build it with `from_dict()` or `from_yaml()`, then load and query. The methods that stay unavailable are the ones that read index metadata — `exists()`, `info()`, and `from_existing()`, which reconstructs a schema out of Redis. `clear()` is not among them: it enumerates with `FT.SEARCH` and deletes in batches, so it needs no more than querying does.
 
 The flag also skips the SVS-VAMANA capability probe described above, since that runs inside `create()`.
 
@@ -264,7 +265,27 @@ With `create_index=False` nothing verifies that the live index matches the schem
 
 For the silent cases the tell is `FT.INFO`'s `key_type`, `prefixes` and `attributes` — not `hash_indexing_failures`, which stays `0` because those keys were never indexing candidates. Diagnosing it therefore needs a credential that can run `FT.INFO`.
 
-An extension constructed with `create_index=False` refuses index-wide `delete()` and `clear()` operations (and their async cache equivalents). This protects an externally managed index — including an index reached through an alias — from being destroyed through an attach-only instance. Targeted operations such as dropping a specific cache entry or message remain available. Perform lifecycle-wide destructive operations through the privileged provisioning path that owns the index.
+### What an attach-only instance may still do
+
+Removing *entries* is available on every path, and is how a caller invalidates an externally managed cache without holding the provisioning credential: `clear()` (plus `SemanticCache.aclear()`), and targeted removal of a specific cache entry, message or route. None of it removes the index, and all of it runs under `+@read +@write`.
+
+What `create_index=False` refuses is `delete()` (and `SemanticCache.adelete()`), because that drops the index. Refusing it protects an externally managed index — including one reached through an alias — from being destroyed through an attach-only instance. Drop the index through the privileged provisioning path that owns it.
+
+The two kinds of `clear()` decide *which keys go* differently, and neither choice is verified against the live index under this flag:
+
+| Method | Deletes | Chooses keys by |
+|---|---|---|
+| `SemanticCache.clear()`, `aclear()` | every key under `{name}:` | `SCAN`/`DEL` on the prefix this instance declares — no index command at all |
+| `MessageHistory.clear()`, `SemanticMessageHistory.clear()`, `SemanticRouter.clear()` | every document the live index covers | `FT.SEARCH` paging via `SearchIndex.clear()` |
+
+`FT.SEARCH` is in `@read` as well as `@search`, so a `+@read +@write` credential is granted it — unlike `FT.INFO`, which is in neither and is what made these three unavailable before. Note that `FT.SEARCH` additionally requires the credential's key patterns to be a superset of the index prefixes, the same rule described under [Key permissions](#key-permissions).
+
+Because the two enumerate differently, they fail differently, and the section above is what decides which failure you get. Both are silent:
+
+- **Prefix-based clearing deletes too much, or nothing.** `SCAN`/`DEL` is blind to the index and to the key type, so it removes every key under `{name}:` — another writer's entries, and unrelated application data sharing that namespace root. And if the live index covers a *different* prefix, or is an alias onto one, `clear()` deletes only what this instance itself wrote and leaves every served entry in place: it reports success and the cache still returns the stale hits you called it to invalidate.
+- **Index-based clearing deletes documents you never wrote.** `SearchIndex.clear()` deletes what the live index covers, so against an index on a different prefix — or a multi-`PREFIX` index, or an alias — it removes another application's documents while leaving this instance's own unindexed entries behind.
+
+Diagnosing either needs `FT.INFO`, which is the command an attach-only credential does not have. If the index is provisioned for you, get its `prefixes` and `key_type` from whoever provisions it and make your extension's name match, rather than inferring it from a successful query.
 
 ### Key permissions
 

redisvl/extensions/cache/llm/semantic.py

@@ -14,7 +14,7 @@
     CACHE_VECTOR_FIELD_NAME,
     CREATE_INDEX_OVERWRITE_CONFLICT,
     ENTRY_ID_FIELD_NAME,
-    EXTERNAL_INDEX_LIFECYCLE_CONFLICT,
+    EXTERNAL_INDEX_DROP_CONFLICT,
     INSERTED_AT_FIELD_NAME,
     METADATA_FIELD_NAME,
     PROMPT_FIELD_NAME,
@@ -309,28 +309,46 @@ def set_threshold(self, distance_threshold: float) -> None:
         self._distance_threshold = float(distance_threshold)
 
     def delete(self) -> None:
-        """Delete the cache and its index entirely."""
+        """Delete the cache and its index entirely.
+
+        Raises:
+            ValueError: If ``create_index=False``. Use :meth:`clear` to
+                empty the cache and leave the index standing.
+        """
         if not self._create_index:
-            raise ValueError(EXTERNAL_INDEX_LIFECYCLE_CONFLICT)
+            raise ValueError(EXTERNAL_INDEX_DROP_CONFLICT)
         self._index.delete(drop=True)
 
     async def adelete(self) -> None:
-        """Async delete the cache and its index entirely."""
+        """Async delete the cache and its index entirely.
+
+        Raises:
+            ValueError: If ``create_index=False``. See :meth:`delete`.
+        """
         if not self._create_index:
-            raise ValueError(EXTERNAL_INDEX_LIFECYCLE_CONFLICT)
+            raise ValueError(EXTERNAL_INDEX_DROP_CONFLICT)
         aindex = await self._get_async_index()
         await aindex.delete(drop=True)
 
     def clear(self) -> None:
-        """Clear all cache keys when RedisVL manages the index lifecycle."""
-        if not self._create_index:
-            raise ValueError(EXTERNAL_INDEX_LIFECYCLE_CONFLICT)
+        """Delete every cache entry, leaving the index in place.
+
+        Clears by key prefix, not by index membership, so it removes every key
+        under ``{name}:`` and nothing outside it. Available under
+        ``create_index=False``; dropping the index is :meth:`delete`.
+
+        Warning:
+            Under ``create_index=False`` the prefix is unverified, so this can
+            delete keys the index never covered and miss entries it does. See
+            :doc:`/user_guide/installation`.
+        """
         super().clear()
 
     async def aclear(self) -> None:
-        """Async clear all cache keys when RedisVL manages the index lifecycle."""
-        if not self._create_index:
-            raise ValueError(EXTERNAL_INDEX_LIFECYCLE_CONFLICT)
+        """Async delete every cache entry, leaving the index in place.
+
+        See :meth:`clear` for the caveats, which apply identically here.
+        """
         await super().aclear()
 
     def drop(self, ids: list[str] | None = None, keys: list[str] | None = None) -> None:

redisvl/extensions/constants.py

@@ -42,9 +42,11 @@
 )
 
 # Raised when an extension attached to an externally managed index is asked to
-# perform an index-wide destructive operation.
-EXTERNAL_INDEX_LIFECYCLE_CONFLICT: str = (
-    "Cannot delete or clear an index when create_index=False because RedisVL "
-    "does not manage that index's lifecycle. Use the externally managed "
-    "provisioning path to perform index-wide destructive operations."
+# drop that index. Removing entries is deliberately not covered: `clear()`
+# leaves the index in place, so it is not a lifecycle operation.
+EXTERNAL_INDEX_DROP_CONFLICT: str = (
+    "Cannot delete the index when create_index=False because RedisVL does not "
+    "manage that index's lifecycle. Use the externally managed provisioning "
+    "path to drop it. To remove every entry while leaving the index in place, "
+    "use clear()."
 )

redisvl/extensions/message_history/message_history.py

@@ -4,7 +4,7 @@
 
 from redisvl.extensions.constants import (
     CONTENT_FIELD_NAME,
-    EXTERNAL_INDEX_LIFECYCLE_CONFLICT,
+    EXTERNAL_INDEX_DROP_CONFLICT,
     ID_FIELD_NAME,
     METADATA_FIELD_NAME,
     ROLE_FIELD_NAME,
@@ -95,15 +95,28 @@ def __repr__(self) -> str:
         return f"MessageHistory(name={self._name!r}, session_tag={self._session_tag!r})"
 
     def clear(self) -> None:
-        """Clears the conversation message history."""
-        if not self._create_index:
-            raise ValueError(EXTERNAL_INDEX_LIFECYCLE_CONFLICT)
+        """Delete every message, leaving the index in place.
+
+        Clears by index membership, so it removes the documents the live index
+        covers. Available under ``create_index=False``; dropping the index is
+        :meth:`delete`.
+
+        Warning:
+            Under ``create_index=False`` the live index is unverified, so if its
+            prefix differs from this instance's it removes documents this
+            instance never wrote. See :doc:`/user_guide/installation`.
+        """
         self._index.clear()
 
     def delete(self) -> None:
-        """Clear all conversation keys and remove the search index."""
+        """Remove every message and drop the search index.
+
+        Raises:
+            ValueError: If ``create_index=False``. Use :meth:`clear` to
+                remove the messages and leave the index standing.
+        """
         if not self._create_index:
-            raise ValueError(EXTERNAL_INDEX_LIFECYCLE_CONFLICT)
+            raise ValueError(EXTERNAL_INDEX_DROP_CONFLICT)
         self._index.delete(drop=True)
 
     def drop(self, id: str | None = None) -> None:

redisvl/extensions/message_history/semantic_history.py

@@ -5,7 +5,7 @@
 from redisvl.extensions.constants import (
     CONTENT_FIELD_NAME,
     CREATE_INDEX_OVERWRITE_CONFLICT,
-    EXTERNAL_INDEX_LIFECYCLE_CONFLICT,
+    EXTERNAL_INDEX_DROP_CONFLICT,
     ID_FIELD_NAME,
     MESSAGE_VECTOR_FIELD_NAME,
     METADATA_FIELD_NAME,
@@ -156,15 +156,28 @@ def __repr__(self) -> str:
         )
 
     def clear(self) -> None:
-        """Clears the message history."""
-        if not self._create_index:
-            raise ValueError(EXTERNAL_INDEX_LIFECYCLE_CONFLICT)
+        """Delete every message, leaving the index in place.
+
+        Clears by index membership, so it removes the documents the live index
+        covers. Available under ``create_index=False``; dropping the index is
+        :meth:`delete`.
+
+        Warning:
+            Under ``create_index=False`` the live index is unverified, so if its
+            prefix differs from this instance's it removes documents this
+            instance never wrote. See :doc:`/user_guide/installation`.
+        """
         self._index.clear()
 
     def delete(self) -> None:
-        """Clear all message keys and remove the search index."""
+        """Remove every message and drop the search index.
+
+        Raises:
+            ValueError: If ``create_index=False``. Use :meth:`clear` to
+                remove the messages and leave the index standing.
+        """
         if not self._create_index:
-            raise ValueError(EXTERNAL_INDEX_LIFECYCLE_CONFLICT)
+            raise ValueError(EXTERNAL_INDEX_DROP_CONFLICT)
         self._index.delete(drop=True)
 
     def drop(self, id: str | None = None) -> None:

redisvl/extensions/router/semantic.py

@@ -9,7 +9,7 @@
 
 from redisvl.extensions.constants import (
     CREATE_INDEX_OVERWRITE_CONFLICT,
-    EXTERNAL_INDEX_LIFECYCLE_CONFLICT,
+    EXTERNAL_INDEX_DROP_CONFLICT,
     ROUTE_VECTOR_FIELD_NAME,
 )
 from redisvl.extensions.router.schema import (
@@ -654,6 +654,11 @@ def add_route(self, route: Route) -> str:
     def remove_route(self, route_name: str) -> None:
         """Remove a route and all references from the semantic router.
 
+        Like :meth:`add_route`, this replaces the router's stored config with
+        this instance's route list, so removing one route from a router holding
+        only a subset drops the rest from the config :meth:`from_existing`
+        reads.
+
         Args:
             route_name (str): Name of the route to remove.
         """
@@ -671,18 +676,31 @@ def remove_route(self, route_name: str) -> None:
             self._update_router_state()
 
     def delete(self) -> None:
-        """Delete the semantic router index and its persisted route config."""
+        """Delete the semantic router index and its persisted route config.
+
+        Raises:
+            ValueError: If ``create_index=False``. Use :meth:`clear` to
+                remove the route references and leave the index standing.
+        """
         if not self._create_index:
-            raise ValueError(EXTERNAL_INDEX_LIFECYCLE_CONFLICT)
+            raise ValueError(EXTERNAL_INDEX_DROP_CONFLICT)
         self._index.delete(drop=True)
         # The route config is stored as a standalone JSON key that is not
         # tracked by the search index, so it must be removed explicitly.
         self._index._redis_client.delete(f"{self.name}:route_config")
 
     def clear(self) -> None:
-        """Flush all routes from the semantic router index."""
-        if not self._create_index:
-            raise ValueError(EXTERNAL_INDEX_LIFECYCLE_CONFLICT)
+        """Delete every route reference, leaving the index in place.
+
+        Clears by index membership. Available under ``create_index=False``;
+        dropping the index is :meth:`delete`.
+
+        Warning:
+            The stored ``route_config`` is left as it was, here and on the
+            default path. A separate process calling :meth:`from_existing`
+            afterwards will report routes whose reference vectors are gone.
+            :meth:`remove_route` keeps the two in step.
+        """
         self._index.clear()
         self.routes = []