VED-1017: Generate JSON ACK file

[JamesW1-NHS]

Summary

PR #1177 with the e2e_automation test changes merged manually. This one (not #1177) is to be merged to master.

The PRs are identical, except that #1168 has no unverified commits.
For confidence, you can verify that the branches are identical by fetching them locally and doing a diff i.e.
git fetch origin
git diff VED-1017-test-merge-e2e VED-1017-json-ack-file

• Routine Change
• ❗ Breaking Change

Acceptance Criteria

A structured JSON ACK is generated when batch processing completes (EOF received / completion confirmed).

The ACK includes:

• Batch-level metadata (e.g. system, version, generated date, filename, provider)
• A summary section showing total, successful, and failed record counts
• A failures section listing only failed records with relevant error details
• No per-record success acknowledgements are included
• The ACK file is generated only once batch completion is confirmed (via EOF)

Conforms to the Design document: https://nhsd-confluence.digital.nhs.uk/pages/viewpage.action?pageId=1187373499&spaceKey=Vacc&title=Immunisation%2BFHIR%2BAPI%2B-%2BACK%2BFile%2BManagement#addcomment

Note: I have marked this as a breaking change as it requires changes to the E2E Automation Tests. These are ongoing. This currently builds successfully in master because the tests requiring modification have been temporarily patched.

Note to Testers

• Changes are to ack_backend only. No change to API or other lambdas.
• E2E Automation tests require modification (see above).

Reviews Required

• Dev
• Test
• Tech Author
• Product Owner

Review Checklist

ℹ️ This section is to be filled in by the reviewer.

• I have reviewed the changes in this PR and they fill all of the acceptance criteria of the ticket.
• If there were infrastructure, operational, or build changes, I have made sure there is sufficient evidence that the changes will work.
• If there were changes that are outside of the regular release processes e.g. account infrastructure to setup, manual setup for external API integrations, secrets to set, then I have checked that the developer has flagged this to the Tech Lead as release steps.
• I have checked that no Personal Identifiable Data (PID) is logged as part of the changes.

[github-actions]

This branch is working on a ticket in the NHS England VED JIRA Project. Here's a handy link to the ticket:

VED-1017

[sonarqubecloud]

Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
97.1% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud