[SPARK-55964] Cache coherence: clear function registry on DROP DATABASE

[srielau]

• Add dropFunctionsInDatabase(db) to FunctionRegistryBase and implementations
• SessionCatalog.dropDatabase clears scalar and table function registry cache for the dropped database so resolution does not see stale entries
• Add SessionCatalogSuite tests for cache coherence

What changes were proposed in this pull request?

We now delete functions from a a schema dropped within the session from the sesssion function registry
design-cache-coherence.md

Why are the changes needed?

Without this change the session coudl keep resolving fucntions from a schema it had dropped.

Does this PR introduce any user-facing change?

It fixes a bug that can be observed.

How was this patch tested?

Added new testcases

Was this patch authored or co-authored using generative AI tooling?

Claude Opus4.6

[cloud-fan]

Review Summary

Prior state and problem: When a database is dropped via SessionCatalog.dropDatabase, the session-level function registries (functionRegistry and tableFunctionRegistry) can retain stale entries for functions that belonged to the dropped database. Subsequent function resolution could then resolve these phantom functions.

Design approach: Add a dropFunctionsInDatabase method to FunctionRegistryBase, with a scan-and-remove implementation in SimpleFunctionRegistryBase and a no-op in EmptyFunctionRegistryBase. Call both registries from SessionCatalog.dropDatabase.

Implementation: Changes span FunctionRegistry.scala (trait + two implementations) and SessionCatalog.scala (5-line addition in dropDatabase). Two new tests in SessionCatalogSuite verify the clearing for scalar and table function registries.