On this page
The short answer
Evaluate an observability export by reconstructing a known task outside the original dashboard. Check identifiers, parent relationships, timestamps, labels, versions, and usage provenance. Downloading records is not the same as preserving useful evidence, especially when exported spans lose the links that distinguish retries from separate user tasks.
What to take away
- Reconstruct a complete synthetic task from the export.
- Test labels and relationships, not only field presence.
- Distinguish missing usage from measured zero.
Sources checked . Research synthesis and Rubrex recommendations. Examples and local experiments are labeled; they are not client results.
Define the evidence that must survive migration
Phoenix documents tracing through OpenTelemetry and OpenInference. Rubrex recommends using an application-level export contract in addition to those instrumentation concepts. Name the task identifier, trace and parent identifiers, prompt and application versions, final status, review labels, and cost provenance needed for future analysis.
Create a synthetic task with one retrieval, a failed model attempt, a retry, and a final reviewed answer. Give every event a known identifier. Preserve a small reference ledger outside the observability tool so the export can be compared with the intended task rather than with another view of the same potentially incomplete data.
Evidence: What is Arize Phoenix? [1]
Export through the path the team will retain
Use the actual planned export mechanism and file format in an isolated trial. Include pagination or time-window boundaries if the interface uses them. Check whether a task spanning the boundary is split, omitted, or duplicated, and whether later updates such as review labels appear in the retained record.
Open the result with a separate reader or parser. Reconstruct the parent-child graph and chronological sequence. A flat list of spans can contain all text while losing the relationship that explains which attempt produced the answer. Record manual repairs required rather than silently correcting the exported data before evaluation.
Check semantic differences and access scope
Compare time zones or timestamp precision, enumerated statuses, absent fields, and estimated versus reported usage. A missing value converted to zero changes the meaning of the evidence. Preserve review rubric versions so an old score is not compared with a new definition under the same label.
Include tenant-scoped and restricted-field cases if the product exposes exports to users. An export should not widen access merely because the dashboard hides a field. Report omissions, duplicate records, broken relationships, and unauthorized fields separately, with examples from the synthetic reference ledger.
| Case | Evidence to retain | Acceptance condition |
|---|---|---|
| Task crosses export window | Reference and exported IDs | No silent omission or duplication |
| Retry relationship | Parent graph | Final attempt identifiable |
| Unknown usage | Exported null and provenance | Unknown not converted to zero |
Accept portability for a named use case
Release the export path when a reviewer can reconstruct the test task, identify the final accepted attempt, and interpret labels and usage without undocumented assumptions. State whether the artifact supports migration, offline analysis, incident review, or evaluation replay; these purposes require different details.
Retain the reference ledger, export schema, parser assumptions, and unresolved gaps. Re-run after backend or SDK changes. Portability is a tested property of the chosen fields and workflow, not a guarantee that two observability products share feature parity or that every historical record can be migrated without loss.
Limits of the evidence
The export and task ledger are proposed synthetic fixtures. No vendor migration was executed. Feature support and schemas vary; this method does not establish interoperability, complete historical recovery, or equivalent access controls across products.
Common questions
Is a JSON download enough?
Only if it preserves the relationships and meanings required for the intended later use.
Should old scores retain rubric versions?
Yes. A score without its definition can become misleading after the evaluator changes.
Sources & further reading
Primary sources behind this briefing. A source’s findings apply to its own study conditions; publication on arXiv does not establish peer review.
- What is Arize Phoenix? Arize AI · Living reference · Maintained documentationReviewed: documented OpenTelemetry/OpenInference tracing, evaluations, and experiments; no hands-on comparison. Accessed October 16, 2026.
Questions or corrections? Write to Rubrex. Read our editorial approach.