celery: allow using links instead of child spans for task execution

[jlc-christie]

Description

Duplicate of closed (stale, lack of reviews) PR #3779 since I lack the permissions to re-open.

As described in #3002, the current default behaviour results in all celery tasks that execute are child spans of the code that pushed it on to the broker, which conflicts with the semantic conventions.

This PR doesn't change the default behaviour since this may be undesirable for consumers expecting the behaviour to be consistent.

Fixes #3002 by providing an optional parameter to the .instrument() method to allow for optionally using span links, e.g.

CeleryInstrumentor().instrument(use_links=True)

Type of change

• New feature (non-breaking change which adds functionality)

How Has This Been Tested?

Please describe the tests that you ran to verify your changes. Provide instructions so we can reproduce. Please also list any relevant details for your test configuration

• Unit test added
• Tested E2E locally with jaeger with use_links=False and also use_links=True and observed the expected behaviour: when the flag is False (default) all task executions of a fan-out job are child spans of the code that enqueued it and when True they appeared as linked spans.

Does This PR Require a Core Repo Change?

• No.

Checklist:

See contributing.md for styleguide, changelog guidelines, and more.

• Followed the style guidelines of this project
• Changelogs have been updated
• Unit tests have been added
• Documentation has been updated

[jlc-christie]

@tammy-baylis-swi @MikeGoldsmith would either of you be able to take a look at this if you get some spare time please? I've noticed you've both been active with celery instrumentation reviews recently. It's been previously approved by @Kludex on #3779 but was closed due to no second review a while ago.

[emdneto]

Thanks for the PR!

Just a heads-up: we no longer update CHANGELOG.md directly. The changelog is now generated from changelog fragments using Towncrier.

Please add the appropriate changelog fragment for this change instead of editing CHANGELOG.md manually. You can find the instructions and expected format in CONTRIBUTING.md.

[tammy-baylis-swi]

Thank you for your patience @jlc-christie . I think having this as an opt-in to better meet semantic conventions makes sense.

If it's not too much trouble it would be nice if the docker tests also tested this feature. Else I think this looks good after addressing the towncrier generation.