diff --git a/docs/assets/datadog-observability.png b/docs/assets/datadog-observability.png new file mode 100644 index 000000000..a3729a95f Binary files /dev/null and b/docs/assets/datadog-observability.png differ diff --git a/docs/observability/datadog.md b/docs/observability/datadog.md new file mode 100644 index 000000000..cbe854211 --- /dev/null +++ b/docs/observability/datadog.md @@ -0,0 +1,85 @@ +# Agent Observability with Datadog + +[Datadog LLM +Observability](https://www.datadoghq.com/product/llm-observability/) helps AI +engineers, data scientists, and application developers quickly develop, +evaluate, and monitor LLM applications. Confidently improve output quality, +performance, costs, and overall risk with structured experiments, end-to-end +tracing across AI agents, and evaluations. + +## Overview + +Datadog LLM Observability can [automatically instrument and trace your agents +built on Google +ADK](https://docs.datadoghq.com/llm_observability/instrumentation/auto_instrumentation?tab=python#google-adk), +allowing you to: + +- **Observe agent executions and interactions** - Automatically capture every + agent run, tool call, and code execution within your agents +- **Capture LLM calls and responses** made with the underlying Google GenAI SDK +- **Debug issues** by providing error rates, token usage and cost, and + out-of-the-box evaluations on your LLM calls and tool usage + +## Prerequisites + +Sign up for a [Datadog account](https://www.datadoghq.com/) if you do not have +one and [get your API +key](https://docs.datadoghq.com/account_management/api-app-keys/#api-keys). + +## Installation + +Install the required packages: + +```bash +pip install ddtrace +``` + +## Setup + +### Configure Environment Variables + +You will also need to specify an ML Application name in the following +environment variables. An ML Application is a grouping of LLM Observability +traces associated with a specific LLM-based application. See [ML Application +Naming +Guidelines](https://docs.datadoghq.com/llm_observability/instrumentation/sdk?tab=python#application-naming-guidelines) +for more information on limitations with ML Application names. + +```shell +export DD_API_KEY= +export DD_SITE= +export DD_LLMOBS_ENABLED=true +export DD_LLMOBS_ML_APP= +export DD_LLMOBS_AGENTLESS_ENABLED=true +export DD_APM_TRACING_ENABLED=false # Only set this if you are not using Datadog APM +``` + +Additionally, configure any LLM provider API keys: + +```shell +export GEMINI_API_KEY= +# or +export GOOGLE_API_KEY= +... +``` + +### Run Your Application + +Once you have configured your environment variables, you can run your +application and start observing your LLM-based applications. + +```shell +ddtrace-run python your_application.py +``` + +## Observe + +Navigate to the [Datadog LLM Observability Traces +View](https://app.datadoghq.com/llm/traces) to see the traces generated by your +application. + +![datadog-observability.png](../assets/datadog-observability.png) + +## Support and Resources +- [Datadog LLM Observability](https://www.datadoghq.com/product/llm-observability/) +- [Datadog Support](https://docs.datadoghq.com/help/) diff --git a/mkdocs.yml b/mkdocs.yml index 5251d25d1..ad5c55349 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -262,6 +262,7 @@ nav: - BigQuery Agent Analytics: observability/bigquery-agent-analytics.md - AgentOps: observability/agentops.md - Arize AX: observability/arize-ax.md + - Datadog: observability/datadog.md - Freeplay: observability/freeplay.md - MLflow: observability/mlflow.md - Monocle: observability/monocle.md