Skip to content

Support WAN SciTag markers for remote HTTP(S) TPC transfers#8042

Merged
kofemann merged 1 commit intodCache:masterfrom
ShawnMcKee:pr/scitags-wan-core-now
Mar 16, 2026
Merged

Support WAN SciTag markers for remote HTTP(S) TPC transfers#8042
kofemann merged 1 commit intodCache:masterfrom
ShawnMcKee:pr/scitags-wan-core-now

Conversation

@ShawnMcKee
Copy link
Contributor

@ShawnMcKee ShawnMcKee commented Mar 16, 2026

Summary

This PR implements the agreed Patch 1-3 scope for WAN SciTag marker support.

Patch 1: Pool marker protocol allowlist

  • Add remotehttpdatatransfer and remotehttpsdatatransfer to TransferLifeCycle.needMarker(...).
  • Keep WAN protocol scope as: xrootd, http, https, remotehttpdatatransfer, remotehttpsdatatransfer.

Patch 2: WebDAV TPC transfer-tag plumbing for remote HTTP(S)

  • Capture SciTag carriers from COPY request in CopyFilter (SciTag, TransferHeaderSciTag).
  • Pass selected transfer tag into RemoteTransferHandler.acceptRequest(...) and RemoteTransfer.
  • Set transfer tag on both remote HTTP and remote HTTPS protocol-info objects.
  • Add serialized transfer-tag storage and getter support to RemoteHttpDataTransferProtocolInfo (used by HTTPS subclass as well).

Patch 3: Access-log observability

  • Add tpc.scitag and tpc.transferheaderscitag fields in WebDAV LoggingHandler access logs.

Validation Evidence

Validated on umfs19 with dcdum02 TPC traffic:

  • pool protocol observed: remotehttpsdatatransfer
  • exact tag matches confirmed (pool transferTag equals door tpc.scitag) for multiple pnfsids, including:
    • 00004C3CC39D92B94DF2B02B4A870AF09DA8 (144)
    • 00004677DBAD412943CAB37A32E479D3C103 (143)
    • 0000E1813ED9C11149D19137F03ABE1CC628 (144)
    • 00000FDBDB1E8DEA4FC7B2FD25FDC4C2361D (144)

Out of Scope

  • No gftp, dcap, or nfs4 protocol expansion in this PR.
  • No optional GSIFTP extension in this PR.

Copy link
Member

@kofemann kofemann left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @ShawnMcKee, thanks for the contribution. I have some minor comments. The code looks good. Please annotate commits with Signed -off-by: tags.


@Override
public String getTransferTag() {
return transferTag == null ? "" : transferTag;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The extra null check is redundant, as initial value and setter alwast set "" for nulls

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch. I removed the redundant null check in getTransferTag(). The value is normalized to "" by the setter and by deserialization fallback, so returning the field directly is sufficient. Included in commit 5b4768f.

}

private String sanitiseHeaderValue(String value) {
if (value == null) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wish there was a comment saying that "-" is a magic placeholder to indicate nulls.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added an explicit sentinel constant and comment to document the magic placeholder: MISSING_HEADER_VALUE = "-" now represents missing/blank SciTag header values. I also replaced the literal "-" checks with that constant for readability. Included in commit 5b4768f.

@@ -192,6 +197,14 @@ public static URI getTpcDestination(HttpServletRequest request) {
return (URI) request.getAttribute(TPC_DESTINATION_ATTRIBUTE);
}

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's good practice to have javadoc for public methods

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed. I added Javadocs for the public accessor methods introduced for request attributes (getTpcSource, getTpcDestination, getTpcSciTag, and getTpcTransferHeaderSciTag). Included in commit 5b4768f.

@ShawnMcKee ShawnMcKee force-pushed the pr/scitags-wan-core-now branch from d2f44bd to 5b4768f Compare March 16, 2026 18:03
@ShawnMcKee
Copy link
Contributor Author

ShawnMcKee commented Mar 16, 2026

Note on validation with all three SciTag PRs (8039, 8040, 8042):

To test the full SciTag stack together, I created branch combo/8039-8040-8042.

Why this was needed:

So a direct master-only test branch would miss #8040. The combo branch starts from the 11.2 side (to include #8040) and layers the corresponding #8039/#8042 changes for integrated validation.

Signed-off-by: Shawn McKee <smckee@umich.edu>
@ShawnMcKee ShawnMcKee force-pushed the pr/scitags-wan-core-now branch from 5b4768f to 88a9be1 Compare March 16, 2026 18:14
@kofemann
Copy link
Member

Thanks for the update.I have merges #8040 into the master too. In general, please issue PR against master. We will do the backports.

@kofemann kofemann merged commit 916669b into dCache:master Mar 16, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants