Skip to content

test(crac/Autoprime): Add warm-up tests that auto-cover the generated provider of every service module#7188

Open
joviegas wants to merge 1 commit into
feature/master/crac_auto_priming_supportfrom
joviegas/warmup_all_module_tests
Open

test(crac/Autoprime): Add warm-up tests that auto-cover the generated provider of every service module#7188
joviegas wants to merge 1 commit into
feature/master/crac_auto_priming_supportfrom
joviegas/warmup_all_module_tests

Conversation

@joviegas

@joviegas joviegas commented Jul 23, 2026

Copy link
Copy Markdown
Contributor

Motivation and Context

CRaC warm-up codegen emits a <Service>WarmUpProvider for each service module, but no test ran those generated providers. The existing warmup-tests module covered the SdkWarmUp orchestrator and HTTP-client warm-up with hand-written providers.

This change adds tests that run the generated provider of every service. The tests put all services on the classpath through the aws-sdk-java aggregate artifact. The aggregate picks up each new service, so the tests cover a new service with no change to this module. A completeness test turns a new service that ships without a registered provider into a build failure instead of a silent gap.

Modifications

Added to test/warmup-tests:

  • AllServicesWarmUpTest: parameterized over every generated provider that ServiceLoader finds, one sync case and one async case per provider. Each case asserts that warm-up completes without throwing, that the selected warm-up operation runs (OperationRecordingInterceptor records it), and that SDK loggers emit no warn or error logs (LogCaptor captures them, filtered to SDK loggers). KNOWN_NO_OP_PROVIDERS lists services whose selector picked no operation because every operation is streaming, event-streaming, or deprecated. The test asserts these providers record no operation, and an entry fails once codegen starts selecting an operation for that service.

  • SdkWarmUpPrimeAllServicesTest: runs the customer entry point SdkWarmUp.prime() with all services on the classpath and asserts it logs no warm-up failure for any generated service provider. The test scopes the assertion to service-provider warnings because the HTTP-client warm-up step logs the same message when an HTTP client fails to build.

  • WarmUpProviderCompletenessTest: scans the classpath jars and asserts every service jar registers a SdkWarmUpProvider in META-INF/services. The ServiceLoader-driven tests generate no case for an unregistered service, so this test reads the jars directly. The test enumerates jars with ClassLoader.getResources instead of java.class.path because surefire runs the test JVM with a manifest-only booter jar.

  • OperationRecordingInterceptor and its execution.interceptors registration: records the name of the operation that warm-up invokes, so the tests confirm the selected operation ran.

  • pom.xml: adds test-scope dependencies on the aws-sdk-java aggregate (all services on the classpath), test-utils (LogCaptor), the log4j2 binding (log4j-core, log4j-slf4j-impl), and http-auth-aws-crt (sigv4a signer, required by services whose endpoint resolves to sigv4a). Disables JaCoCo for this module: it loads every service client, and instrumenting the two largest generated clients (SSM, Glue) pushes them past the JVM class-size limit.

Testing

License

  • I confirm that this pull request can be released under the Apache 2 license

@joviegas
joviegas requested a review from a team as a code owner July 23, 2026 22:35
@joviegas joviegas changed the title Add warm-up tests that auto-cover the generated provider of every service module test(crac/Autoprome): Add warm-up tests that auto-cover the generated provider of every service module Jul 24, 2026
@joviegas joviegas changed the title test(crac/Autoprome): Add warm-up tests that auto-cover the generated provider of every service module test(crac/Autoprime): Add warm-up tests that auto-cover the generated provider of every service module Jul 24, 2026
@joviegas
joviegas force-pushed the joviegas/warmup_all_module_tests branch 3 times, most recently from d558d8b to 396b4c3 Compare July 24, 2026 16:35
@joviegas
joviegas force-pushed the joviegas/warmup_all_module_tests branch from 396b4c3 to 4778e7e Compare July 24, 2026 16:40
* JVM with a manifest-only booter jar, so {@code java.class.path} lists only {@code surefirebooter*.jar} and scanning
* it would find nothing.
*/
class WarmUpProviderCompletenessTest {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this test really necessary? What potential issue would this test catch?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This test catches a service jar that ships without a valid SdkWarmUpProvider registration in META-INF/services. The other two tests discover providers through ServiceLoader, and ServiceLoader only sees registered providers, so an unregistered service simply produces no test case and the suite passes while that service silently loses warm-up coverage. This test reads the classpath jars directly, so the same defect becomes a build failure.

This can happen because codegen writes the registration as a plain text file (WarmUpProviderRegistrationTask). A codegen regression can skip or corrupt that file if there is customized code for a module and without any unit test failing, and a hand-written META-INF/services file added to a module later would overwrite the generated one during packaging. This test confirms every service jar has its registration, and that is what gives the other two tests their input: they only execute providers that are registered.

Comment thread test/warmup-tests/pom.xml
<description>Centralized tests for the CRaC HTTP-client warm-up, run against every sync HTTP client so each client does not
need its own warm-up test. This is a leaf test module: nothing depends on it, so depending on the HTTP clients here
does not create a dependency cycle.</description>
<description>Centralized tests for the CRaC warm-up feature. Covers the SdkWarmUp orchestrator and HTTP-client warm-up

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: technically, this feature/test doesn't directly interact with CRaC, let's do a scan and remove the wording. Can be done in a separate PR

Comment thread test/warmup-tests/pom.xml
<version>${awsjavasdk.version}</version>
<scope>test</scope>
</dependency>
<!-- Every service module, so the generated SdkWarmUpProviders of all services are on the test classpath.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Question: any reason we want to run tests for all services? How long does it take to run all tests?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, we run all services because the warm-up operation is selected per service from each service's model; only executing every generated provider proves each one works. The full module runs offline (Canned Req/Response).

These test guard the customer path: a customer with all services on the classpath calls SdkWarmUp.prime(), and a broken provider logs a WARN and skips that service's warm-up. These tests turn that customer-visible WARN into a build failure here.

New services are covered automatically, the aggregate dependency picks up each new service, and the completeness test fails the build if a service ships without a registered provider

@joviegas joviegas Jul 24, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I will get the exact time once I merge the PR 7183. The issues mentioned in this PR were caught by these tests

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants