Agent Skills

dt-obs-ext-monitors

3rd-party test and monitor result ingestion into Dynatrace Grail via the platform events ingest API (platform/ingest/custom/events/). Use when sending external synthetic test outcomes, CI monitor data, or third-party check results to Dynatrace. Covers token scope, full event schema for external_test_run and external_test_step (including dt.security_context, ci.*, trace correlation, and pipeline-added fields), curl and Java DTO examples, and DQL verification. Triggers: "ingest test results", "sen

Install

npx skills add https://github.com/dynatrace/dynatrace-for-ai --skill dt-obs-ext-monitors
SKILL.md

External Monitor Ingestion

Send 3rd-party test and monitor results to Dynatrace Grail using the events ingest API. This is the canonical replacement for the deprecated POST /api/v1/synthetic/ext/tests endpoint.

Overview

Events posted to /platform/ingest/custom/events/{endpoint} land in Grail and are processed by OpenPipeline, which:

  • Extracts metrics (external.test.availability, external.test.duration) for alerting and SLOs
  • Registers each unique test.id as an EXT_TEST Smartscape node — enabling Davis Problems to attach to a named entity ("External test X went down") rather than floating without topology context
  • Adds result.status.category to step events (SUCCESS / SKIPPED / FAIL) for dashboard filtering

Two event types form a test result:

Type Purpose
external_test_run Overall pass/fail result for one test execution
external_test_step One step within that run (optional; enables step-level metrics)

Authentication

Two token types are accepted:

Token type Scope
Classic Api-Token openpipeline.events.custom
Platform Token / OAuth openpipeline:events.custom:ingest

Common wrong guess that does NOT work: events.ingest.

Quick Start

Prerequisite: The ingest endpoint must be created in OpenPipeline before sending events. The external.tests endpoint is provisioned automatically by the default Dynatrace 3rd-party monitors Monaco bundle. See references/event-ingestion.md for setup details and custom endpoint creation.

Send one minimal test result (replace external.tests with your configured endpoint name):

curl -X POST "https://{env-id}.live.dynatrace.com/platform/ingest/custom/events/external.tests" \
  -H "Authorization: Api-Token {token}" \
  -H "Content-Type: application/json" \
  -d '[{
    "event.kind": "EXTERNAL_TEST_EVENT",
    "event.type": "external_test_run",
    "test.id": "my-api-health-check",
    "test.run.id": "a1b2c3d4-e5f6-7890-abcd-ef1234567890",
    "test.name": "My API Health Check",
    "test.type": "api",
    "test.run.status": "passed",
    "test.run.availability": 1,
    "test.run.duration_ms": 245,
    "test.run.location": "us-east-1",
    "dt.security_context": "team-checkout",
    "timestamp": "2024-01-15T10:30:00Z"
  }]'

Expected response: HTTP 200 (empty body).

Full event schema, step events, extended examples with error fields and CI metadata, Java DTO shapes, and DQL queries: references/event-ingestion.md

Verify in Grail

After sending, confirm the event appears:

fetch events, from:now()-1h
| filter event.type == "external_test_run"
| fields timestamp, test.id, test.name, test.run.status, test.run.duration_ms, test.run.location
| sort timestamp desc
| limit 20

Key Constraints

  • Body must be a JSON array ([{...}]), not an object wrapper ({"events":[...]}).
  • A single request can mix run events and step events in the same array.
  • timestamp must be ISO 8601 UTC (e.g. "2024-01-15T10:30:00Z").
  • test.run.availability must be integer 1 or 0, not a string.
  • All step events in a run must share the same test.run.id and test.id as the parent run event.
  • Step events must include test.step.id (unique within the run) in addition to test.run.id.
  • event.kind: "EXTERNAL_TEST_EVENT" must be present — used by Grail for event classification. Routing to the correct pipeline is driven by event.type, not event.kind.
  • Do not send dt.smartscape.ext_test or result.status.category — these are written by the pipeline after ingestion and will be overwritten if included.
  • dt.security_context controls data access policies; it’s recommended to set it on every event to enable per-team access control and cost attribution in multi-team tenants.

Multi-Location Tests

Send the same test.id from multiple locations — each with a different test.run.location value — to build a multi-location test. OpenPipeline creates one EXT_TEST node per test.id and one availability metric timeseries per (test.id, location) pair.

This enables two alerting tiers out of the box:

Alert type Fires when
Local outage A single location's availability drops (per-location timeseries)
Global outage Average across all locations drops below threshold (e.g. majority failing)

Threshold maths for a 3-location test: 1 location failing → avg 0.67 (no alert at 0.5 threshold); 2 failing → avg 0.33 (fires); all 3 failing → avg ≈ 0 (fires immediately).

Keep test.id stable across all locations — changing it creates a new Smartscape node and breaks metric history.

Related Skills

  • dt-dql-essentials — DQL syntax for querying ingested events and building analysis queries
  • dt-obs-frontends — Link test runs to frontend entities via dt.smartscape.frontend to draw EXT_TEST → FRONTEND dependency edges in Smartscape

Related skills

azure-diagnosticsmicrosoft608KDebug Azure production issues on Azure using AppLens, Azure Monitor, resource health, and safe triage. WHEN: debug production issues, troubleshoot app service, app service high CPU, app service deployment failure, troubleshoot container apps, troubleshoot functions, troubleshoot AKS, VM RDP, Linux SSH, VM black screen, can't connect to VM, reset VM password, NSG or firewall blocking, kubectl cannot connect, kube-system/CoreDNS failures, pod pending, crashloop, node not ready, upgrade failures, aazure-preparemicrosoft608KPrepare azd-based Azure projects for deployment: generates azure.yaml, infrastructure (Bicep/Terraform), and Dockerfiles for the Azure Developer CLI (azd) workflow. USE ONLY when the user explicitly wants to use azd as the deployment tool, or the project already has an azure.yaml file. DO NOT USE FOR: non-azd deployments, Python App Service code-only deploys (use python-appservice-deploy), or cross-cloud migration (use azure-cloud-migrate). WHEN: prepare app for azd, create azure.yaml, set up azazure-aimicrosoft608KUse for Azure AI: Search, Speech, OpenAI, Document Intelligence. Helps with search, vector/hybrid search, speech-to-text, text-to-speech, transcription, OCR. WHEN: AI Search, query search, vector search, hybrid search, semantic search, speech-to-text, text-to-speech, transcribe, OCR, convert text to speech.azure-deploymicrosoft607KExecute Azure deployments for ALREADY-PREPARED applications that have existing .azure/deployment-plan.md and infrastructure files. DO NOT use this skill when the user asks to CREATE a new application — use azure-prepare instead. This skill runs azd up, azd deploy, terraform apply, and az deployment commands with built-in error recovery. Requires .azure/deployment-plan.md from azure-prepare and validated status from azure-validate. WHEN: \"run azd up\", \"run azd deploy\", \"execute deployment\",

Search skills and MCP servers

Fuzzy search across 23,137 skills and servers