Skip to main content
Read the same metrics and console logs shown on a reservation’s Observability tab from your own tooling. Use a physical node ID from List reserved GPU nodes, or the ID of a VM in one of your reservations.

What you can read

  • Your reservation window only. Reads are clamped to the reservation’s start and end. Samples and log lines outside it are omitted, and a cancelled reservation returns 404. An empty result means no data in your window; it does not mean the machine is down.
  • Customer metrics. cpu_utilization, cpu_temperature, gpu_utilization, gpu_temperature, gpu_power_w, gpu_fan_speed_percent, component_temperature_max, memory_utilization, gpu_memory_utilization, storage_utilization, and ethernet_latency_ms. A metric appears only when the machine reports it.
  • String values. Latest-sample values are strings; parse numbers yourself. Physical nodes can also report gpus, a JSON-encoded array of per-GPU readings.
  • Staleness. Check observed_at. A stored sample is not proof that the machine is reachable now.
A machine outside your organization’s reservations returns 404 with {"error": "not_found"}.

Read the latest sample and recent logs

ornn check combines lifecycle, health, reachability, metrics, and logs for one machine or cluster.
For a VM, use the same requests under /v1/observability/virtual-machines/{virtual_machine_id}/. Per-GPU history (gpu_uuid) is available only for physical nodes, over at most 24 hours.

History resolution

Query telemetry history returns one window per metric and time bucket. Ornn picks the tier from the span and max_points: full returns raw samples (each window’s start equals its end), while hour, day, and month summarize each bucket with mean, median, std, min, and max. Request a shorter span or more points for finer detail; max_points accepts 1 to 500.

Console log pages

Console log queries return the newest page first by default. Keep start, end, and order fixed and pass next_cursor to read further back; null means no more lines in the range. Streams are kernel and serial. Each line has a severity of error, warning, or info.

Dashboards across many machines

Get latest telemetry for many machines accepts up to 5,000 node and VM IDs in one request. If any ID is outside your organization’s reservations, the whole request returns 404, so batch one reservation’s machines at a time. For larger sets, send the chunks in parallel with a small random delay rather than one after another. Compare observed_at between polls to detect a new sample. Back off on 429 and 5xx responses.