Every validation run above cost 0 tokens: correctness is decided by scraping the artifact and diffing against the IR, never by a model. Models are used only to read docs, generate the artifact, and repair it.
Pipeline
Stage
Executed by
Model
Cost
Ingest docs
deterministic code
—
2,789 chars normalized
Extract surface
standard tier
sonnet
1 call(s) · 4.9K in / 2.3K out
Design IR
standard tier
sonnet
1 call(s) · 7.5K in / 4.6K out
Generate code
strong tier
opus
1 call(s) · 9.6K in / 4.5K out
Run + validate
deterministic code
—
1 run(s), 0 tokens
Validation diff · expected (IR) vs observed (scrape)
Metric
Type
Observed
Series
Sample
Labels
Note
✓
rabbitmq_messages_published_total
counter
counter
1
118,681
—
✓
rabbitmq_messages_delivered_total
counter
counter
1
114,346
—
✓
rabbitmq_messages_acknowledged_total
counter
counter
1
114,326
—
✓
rabbitmq_messages_confirmed_total
counter
counter
1
118,636
—
✓
rabbitmq_messages
gauge
gauge
1
335
—
✓
rabbitmq_messages_ready
gauge
gauge
1
320
—
✓
rabbitmq_messages_unacknowledged
gauge
gauge
1
15
—
✓
rabbitmq_connections
gauge
gauge
1
4
—
✓
rabbitmq_channels
gauge
gauge
1
8
—
✓
rabbitmq_exchanges
gauge
gauge
1
12
—
✓
rabbitmq_queues
gauge
gauge
1
3
—
✓
rabbitmq_consumers
gauge
gauge
1
5
—
✓
rabbitmq_queue_messages
gauge
gauge
3
120
queue, vhost
✓
rabbitmq_queue_messages_ready
gauge
gauge
3
118
queue, vhost
✓
rabbitmq_queue_messages_unacknowledged
gauge
gauge
3
2
queue, vhost
✓
rabbitmq_queue_consumers
gauge
gauge
3
2
queue, vhost
✓
rabbitmq_queue_memory_bytes
gauge
gauge
3
187,312
queue, vhost
✓
rabbitmq_queue_messages_published_total
counter
counter
3
59,340
queue, vhost
✓
rabbitmq_node_memory_used_bytes
gauge
gauge
1
142,606,336
node
✓
rabbitmq_node_fd_used
gauge
gauge
1
42
node
✓
rabbitmq_node_fd_total
gauge
gauge
1
1,048,576
node
✓
rabbitmq_node_sockets_used
gauge
gauge
1
6
node
✓
rabbitmq_node_sockets_total
gauge
gauge
1
943,626
node
✓
rabbitmq_node_processes_used
gauge
gauge
1
512
node
✓
rabbitmq_node_processes_total
gauge
gauge
1
1,048,576
node
✓
rabbitmq_node_uptime_milliseconds
gauge
gauge
1
326,726
node
✓
rabbitmq_node_disk_free_bytes
gauge
gauge
1
53,687,091,200
node
✓
rabbitmq_node_running
gauge
gauge
1
1
node
Cost ledger
Model
Calls
Tokens in
Tokens out
opus
1
9.6K
4.5K
sonnet
2
12.4K
6.9K
total
3
22.0K
11.4K
validation (1 run(s))
0
0
0
Artifacts
exporter.py — Generated exporter · 320 lines
#!/usr/bin/env python3
"""Prometheus exporter for the RabbitMQ Management HTTP API.
Usage:
python exporter.py --port PORT --target BASE_URL [--username U] [--password P]
"""
import argparse
import sys
import time
import requests
from prometheus_client import start_http_server
from prometheus_client.core import (
CollectorRegistry,
CounterMetricFamily,
GaugeMetricFamily,
)
REQUEST_TIMEOUT = 5
QUEUE_LABELS = ["queue", "vhost"]
NODE_LABELS = ["node"]
# (metric name, help, JSON path within /api/overview)
OVERVIEW_COUNTERS = [
("rabbitmq_messages_published_total",
"Total number of messages published across the broker (cumulative).",
("message_stats", "publish")),
("rabbitmq_messages_delivered_total",
"Total messages delivered to consumers in acknowledgement mode plus basic.get (cumulative).",
("message_stats", "deliver_get")),
("rabbitmq_messages_acknowledged_total",
"Total number of messages acknowledged by consumers (cumulative).",
("message_stats", "ack")),
("rabbitmq_messages_confirmed_total",
"Total number of messages confirmed to publishers (cumulative).",
("message_stats", "confirm")),
]
OVERVIEW_GAUGES = [
("rabbitmq_messages",
"Total number of messages in all queues.",
("queue_totals", "messages")),
("rabbitmq_messages_ready",
"Number of messages ready for delivery across all queues.",
("queue_totals", "messages_ready")),
("rabbitmq_messages_unacknowledged",
"Number of messages delivered but not yet acknowledged across all queues.",
("queue_totals", "messages_unacknowledged")),
("rabbitmq_connections",
"Current number of connections.",
("object_totals", "connections")),
("rabbitmq_channels",
"Current number of channels.",
("object_totals", "channels")),
("rabbitmq_exchanges",
"Current number of exchanges.",
("object_totals", "exchanges")),
("rabbitmq_queues",
"Current number of queues.",
("object_totals", "queues")),
("rabbitmq_consumers",
"Current number of consumers.",
("object_totals", "consumers")),
]
QUEUE_GAUGES = [
("rabbitmq_queue_messages",
"Total number of messages in the queue (ready plus unacknowledged).",
("messages",)),
("rabbitmq_queue_messages_ready",
"Number of messages in the queue ready for delivery.",
("messages_ready",)),
("rabbitmq_queue_messages_unacknowledged",
"Number of messages in the queue delivered to consumers and awaiting acknowledgement.",
("messages_unacknowledged",)),
("rabbitmq_queue_consumers",
"Number of consumers on the queue.",
("consumers",)),
("rabbitmq_queue_memory_bytes",
"Bytes of memory used by the queue process.",
("memory",)),
]
QUEUE_COUNTERS = [
("rabbitmq_queue_messages_published_total",
"Total number of messages published to the queue (cumulative).",
("message_stats", "publish")),
]
NODE_GAUGES = [
("rabbitmq_node_memory_used_bytes",
"Memory used by the node, in bytes.",
("mem_used",)),
("rabbitmq_node_fd_used",
"Number of file descriptors in use by the node.",
("fd_used",)),
("rabbitmq_node_fd_total",
"File descriptor limit of the node.",
("fd_total",)),
("rabbitmq_node_sockets_used",
"Number of sockets in use by the node.",
("sockets_used",)),
("rabbitmq_node_sockets_total",
"Socket limit of the node.",
("sockets_total",)),
("rabbitmq_node_processes_used",
"Number of Erlang processes in use on the node.",
("proc_used",)),
("rabbitmq_node_processes_total",
"Erlang process limit of the node.",
("proc_total",)),
("rabbitmq_node_uptime_milliseconds",
"Milliseconds since the node started.",
("uptime",)),
("rabbitmq_node_disk_free_bytes",
"Free disk space on the node, in bytes.",
("disk_free",)),
]
NODE_RUNNING_NAME = "rabbitmq_node_running"
NODE_RUNNING_HELP = (
"Whether the node is running (1) or not (0). "
"Converted from the boolean 'running' field."
)
def log(msg):
print(f"[rabbitmq-exporter] {msg}", file=sys.stderr, flush=True)
def dig(obj, path):
"""Walk a nested dict by key path; return None if any step is missing."""
cur = obj
for key in path:
if not isinstance(cur, dict) or key not in cur:
return None
cur = cur[key]
return cur
def to_number(value):
"""Convert a JSON value to float; return None for non-numeric values."""
if value is None or isinstance(value, bool):
return None
if isinstance(value, (int, float)):
return float(value)
if isinstance(value, str):
try:
return float(value)
except ValueError:
return None
return None
def to_bool_number(value):
if isinstance(value, bool):
return 1.0 if value else 0.0
if isinstance(value, (int, float)):
return 1.0 if value else 0.0
if isinstance(value, str):
lowered = value.strip().lower()
if lowered in ("true", "1", "yes"):
return 1.0
if lowered in ("false", "0", "no"):
return 0.0
return None
class RabbitMQCollector:
def __init__(self, target, username=None, password=None):
self.target = target.rstrip("/")
self.session = requests.Session()
if username is not None or password is not None:
self.session.auth = (username or "", password or "")
self.session.headers.update({"Accept": "application/json"})
def _get(self, path):
url = f"{self.target}{path}"
try:
resp = self.session.get(url, timeout=REQUEST_TIMEOUT)
resp.raise_for_status()
return resp.json()
except requests.RequestException as exc:
log(f"request to {url} failed: {exc}")
except ValueError as exc:
log(f"invalid JSON from {url}: {exc}")
return None
def describe(self):
# Avoid triggering a scrape of the target at registration time.
return []
def collect(self):
yield from self._collect_overview()
yield from self._collect_queues()
yield from self._collect_nodes()
def _collect_overview(self):
data = self._get("/api/overview")
if not isinstance(data, dict):
if data is not None:
log("unexpected /api/overview payload type; skipping")
return
for name, help_text, path in OVERVIEW_COUNTERS:
value = to_number(dig(data, path))
if value is None:
continue
fam = CounterMetricFamily(name, help_text)
fam.add_metric([], value)
yield fam
for name, help_text, path in OVERVIEW_GAUGES:
value = to_number(dig(data, path))
if value is None:
continue
fam = GaugeMetricFamily(name, help_text)
fam.add_metric([], value)
yield fam
def _collect_queues(self):
data = self._get("/api/queues")
if not isinstance(data, list):
if data is not None:
log("unexpected /api/queues payload type; skipping")
return
gauges = {
name: GaugeMetricFamily(name, help_text, labels=QUEUE_LABELS)
for name, help_text, _ in QUEUE_GAUGES
}
counters = {
name: CounterMetricFamily(name, help_text, labels=QUEUE_LABELS)
for name, help_text, _ in QUEUE_COUNTERS
}
for item in data:
if not isinstance(item, dict):
continue
qname = item.get("name")
vhost = item.get("vhost")
if qname is None or vhost is None:
continue
labels = [str(qname), str(vhost)]
for name, _, path in QUEUE_GAUGES:
value = to_number(dig(item, path))
if value is not None:
gauges[name].add_metric(labels, value)
for name, _, path in QUEUE_COUNTERS:
value = to_number(dig(item, path))
if value is not None:
counters[name].add_metric(labels, value)
for name, _, _ in QUEUE_GAUGES:
if gauges[name].samples:
yield gauges[name]
for name, _, _ in QUEUE_COUNTERS:
if counters[name].samples:
yield counters[name]
def _collect_nodes(self):
data = self._get("/api/nodes")
if not isinstance(data, list):
if data is not None:
log("unexpected /api/nodes payload type; skipping")
return
gauges = {
name: GaugeMetricFamily(name, help_text, labels=NODE_LABELS)
for name, help_text, _ in NODE_GAUGES
}
running = GaugeMetricFamily(
NODE_RUNNING_NAME, NODE_RUNNING_HELP, labels=NODE_LABELS
)
for item in data:
if not isinstance(item, dict):
continue
node = item.get("name")
if node is None:
continue
labels = [str(node)]
for name, _, path in NODE_GAUGES:
value = to_number(dig(item, path))
if value is not None:
gauges[name].add_metric(labels, value)
run_val = to_bool_number(item.get("running"))
if run_val is not None:
running.add_metric(labels, run_val)
for name, _, _ in NODE_GAUGES:
if gauges[name].samples:
yield gauges[name]
if running.samples:
yield running
def parse_args(argv=None):
parser = argparse.ArgumentParser(
description="Prometheus exporter for the RabbitMQ Management HTTP API."
)
parser.add_argument("--port", type=int, required=True,
help="Port to serve /metrics on.")
parser.add_argument("--target", required=True,
help="Base URL of the RabbitMQ management API, e.g. http://127.0.0.1:15672")
parser.add_argument("--username", default=None, help="Basic auth username.")
parser.add_argument("--password", default=None, help="Basic auth password.")
return parser.parse_args(argv)
def main(argv=None):
args = parse_args(argv)
registry = CollectorRegistry()
registry.register(RabbitMQCollector(args.target, args.username, args.password))
start_http_server(args.port, registry=registry)
log(f"serving metrics on :{args.port}/metrics for target {args.target}")
try:
while True:
time.sleep(3600)
except KeyboardInterrupt:
pass
if __name__ == "__main__":
main()
spec.json — Metric IR (validation contract) · 545 lines
{
"service": "rabbitmq",
"kind": "python_exporter",
"endpoints": [
{
"url": "/api/overview",
"method": "GET",
"auth": "basic",
"auth_detail": "HTTP Basic authentication; default user guest / password guest (localhost only)",
"response_format": "json",
"notes": "System-wide overview: message stats, queue totals, object totals."
},
{
"url": "/api/queues",
"method": "GET",
"auth": "basic",
"auth_detail": "HTTP Basic authentication; default user guest / password guest (localhost only)",
"response_format": "json",
"notes": "JSON array of all queues across all vhosts; one element per queue."
},
{
"url": "/api/nodes",
"method": "GET",
"auth": "basic",
"auth_detail": "HTTP Basic authentication; default user guest / password guest (localhost only)",
"response_format": "json",
"notes": "JSON array of cluster nodes; one element per node."
}
],
"metrics": [
{
"name": "rabbitmq_messages_published_total",
"type": "counter",
"help": "Total number of messages published across the broker (cumulative).",
"unit": "",
"labels": [],
"value": {
"min": 0.0,
"max": null
},
"source": "GET /api/overview -> message_stats.publish",
"required": false
},
{
"name": "rabbitmq_messages_delivered_total",
"type": "counter",
"help": "Total messages delivered to consumers in acknowledgement mode plus basic.get (cumulative).",
"unit": "",
"labels": [],
"value": {
"min": 0.0,
"max": null
},
"source": "GET /api/overview -> message_stats.deliver_get",
"required": false
},
{
"name": "rabbitmq_messages_acknowledged_total",
"type": "counter",
"help": "Total number of messages acknowledged by consumers (cumulative).",
"unit": "",
"labels": [],
"value": {
"min": 0.0,
"max": null
},
"source": "GET /api/overview -> message_stats.ack",
"required": false
},
{
"name": "rabbitmq_messages_confirmed_total",
"type": "counter",
"help": "Total number of messages confirmed to publishers (cumulative).",
"unit": "",
"labels": [],
"value": {
"min": 0.0,
"max": null
},
"source": "GET /api/overview -> message_stats.confirm",
"required": false
},
{
"name": "rabbitmq_messages",
"type": "gauge",
"help": "Total number of messages in all queues.",
"unit": "",
"labels": [],
"value": {
"min": 0.0,
"max": null
},
"source": "GET /api/overview -> queue_totals.messages",
"required": true
},
{
"name": "rabbitmq_messages_ready",
"type": "gauge",
"help": "Number of messages ready for delivery across all queues.",
"unit": "",
"labels": [],
"value": {
"min": 0.0,
"max": null
},
"source": "GET /api/overview -> queue_totals.messages_ready",
"required": true
},
{
"name": "rabbitmq_messages_unacknowledged",
"type": "gauge",
"help": "Number of messages delivered but not yet acknowledged across all queues.",
"unit": "",
"labels": [],
"value": {
"min": 0.0,
"max": null
},
"source": "GET /api/overview -> queue_totals.messages_unacknowledged",
"required": true
},
{
"name": "rabbitmq_connections",
"type": "gauge",
"help": "Current number of connections.",
"unit": "",
"labels": [],
"value": {
"min": 0.0,
"max": null
},
"source": "GET /api/overview -> object_totals.connections",
"required": true
},
{
"name": "rabbitmq_channels",
"type": "gauge",
"help": "Current number of channels.",
"unit": "",
"labels": [],
"value": {
"min": 0.0,
"max": null
},
"source": "GET /api/overview -> object_totals.channels",
"required": true
},
{
"name": "rabbitmq_exchanges",
"type": "gauge",
"help": "Current number of exchanges.",
"unit": "",
"labels": [],
"value": {
"min": 0.0,
"max": null
},
"source": "GET /api/overview -> object_totals.exchanges",
"required": true
},
{
"name": "rabbitmq_queues",
"type": "gauge",
"help": "Current number of queues.",
"unit": "",
"labels": [],
"value": {
"min": 0.0,
"max": null
},
"source": "GET /api/overview -> object_totals.queues",
"required": true
},
{
"name": "rabbitmq_consumers",
"type": "gauge",
"help": "Current number of consumers.",
"unit": "",
"labels": [],
"value": {
"min": 0.0,
"max": null
},
"source": "GET /api/overview -> object_totals.consumers",
"required": true
},
{
"name": "rabbitmq_queue_messages",
"type": "gauge",
"help": "Total number of messages in the queue (ready plus unacknowledged).",
"unit": "",
"labels": [
{
"name": "queue",
"description": "Queue name (JSON field 'name').",
"allowed_values": null,
"required": true
},
{
"name": "vhost",
"description": "Virtual host the queue belongs to.",
"allowed_values": null,
"required": true
}
],
"value": {
"min": 0.0,
"max": null
},
"source": "GET /api/queues -> [].messages",
"required": true
},
{
"name": "rabbitmq_queue_messages_ready",
"type": "gauge",
"help": "Number of messages in the queue ready for delivery.",
"unit": "",
"labels": [
{
"name": "queue",
"description": "Queue name (JSON field 'name').",
"allowed_values": null,
"required": true
},
{
"name": "vhost",
"description": "Virtual host the queue belongs to.",
"allowed_values": null,
"required": true
}
],
"value": {
"min": 0.0,
"max": null
},
"source": "GET /api/queues -> [].messages_ready",
"required": true
},
{
"name": "rabbitmq_queue_messages_unacknowledged",
"type": "gauge",
"help": "Number of messages in the queue delivered to consumers and awaiting acknowledgement.",
"unit": "",
"labels": [
{
"name": "queue",
"description": "Queue name (JSON field 'name').",
"allowed_values": null,
"required": true
},
{
"name": "vhost",
"description": "Virtual host the queue belongs to.",
"allowed_values": null,
"required": true
}
],
"value": {
"min": 0.0,
"max": null
},
"source": "GET /api/queues -> [].messages_unacknowledged",
"required": true
},
{
"name": "rabbitmq_queue_consumers",
"type": "gauge",
"help": "Number of consumers on the queue.",
"unit": "",
"labels": [
{
"name": "queue",
"description": "Queue name (JSON field 'name').",
"allowed_values": null,
"required": true
},
{
"name": "vhost",
"description": "Virtual host the queue belongs to.",
"allowed_values": null,
"required": true
}
],
"value": {
"min": 0.0,
"max": null
},
"source": "GET /api/queues -> [].consumers",
"required": true
},
{
"name": "rabbitmq_queue_memory_bytes",
"type": "gauge",
"help": "Bytes of memory used by the queue process.",
"unit": "bytes",
"labels": [
{
"name": "queue",
"description": "Queue name (JSON field 'name').",
"allowed_values": null,
"required": true
},
{
"name": "vhost",
"description": "Virtual host the queue belongs to.",
"allowed_values": null,
"required": true
}
],
"value": {
"min": 0.0,
"max": null
},
"source": "GET /api/queues -> [].memory",
"required": true
},
{
"name": "rabbitmq_queue_messages_published_total",
"type": "counter",
"help": "Total number of messages published to the queue (cumulative).",
"unit": "",
"labels": [
{
"name": "queue",
"description": "Queue name (JSON field 'name').",
"allowed_values": null,
"required": true
},
{
"name": "vhost",
"description": "Virtual host the queue belongs to.",
"allowed_values": null,
"required": true
}
],
"value": {
"min": 0.0,
"max": null
},
"source": "GET /api/queues -> [].message_stats.publish",
"required": false
},
{
"name": "rabbitmq_node_memory_used_bytes",
"type": "gauge",
"help": "Memory used by the node, in bytes.",
"unit": "bytes",
"labels": [
{
"name": "node",
"description": "Cluster node name (JSON field 'name').",
"allowed_values": null,
"required": true
}
],
"value": {
"min": 0.0,
"max": null
},
"source": "GET /api/nodes -> [].mem_used",
"required": true
},
{
"name": "rabbitmq_node_fd_used",
"type": "gauge",
"help": "Number of file descriptors in use by the node.",
"unit": "",
"labels": [
{
"name": "node",
"description": "Cluster node name (JSON field 'name').",
"allowed_values": null,
"required": true
}
],
"value": {
"min": 0.0,
"max": null
},
"source": "GET /api/nodes -> [].fd_used",
"required": true
},
{
"name": "rabbitmq_node_fd_total",
"type": "gauge",
"help": "File descriptor limit of the node.",
"unit": "",
"labels": [
{
"name": "node",
"description": "Cluster node name (JSON field 'name').",
"allowed_values": null,
"required": true
}
],
"value": {
"min": 0.0,
"max": null
},
"source": "GET /api/nodes -> [].fd_total",
"required": true
},
{
"name": "rabbitmq_node_sockets_used",
"type": "gauge",
"help": "Number of sockets in use by the node.",
"unit": "",
"labels": [
{
"name": "node",
"description": "Cluster node name (JSON field 'name').",
"allowed_values": null,
"required": true
}
],
"value": {
"min": 0.0,
"max": null
},
"source": "GET /api/nodes -> [].sockets_used",
"required": true
},
{
"name": "rabbitmq_node_sockets_total",
"type": "gauge",
"help": "Socket limit of the node.",
"unit": "",
"labels": [
{
"name": "node",
"description": "Cluster node name (JSON field 'name').",
"allowed_values": null,
"required": true
}
],
"value": {
"min": 0.0,
"max": null
},
"source": "GET /api/nodes -> [].sockets_total",
"required": true
},
{
"name": "rabbitmq_node_processes_used",
"type": "gauge",
"help": "Number of Erlang processes in use on the node.",
"unit": "",
"labels": [
{
"name": "node",
"description": "Cluster node name (JSON field 'name').",
"allowed_values": null,
"required": true
}
],
"value": {
"min": 0.0,
"max": null
},
"source": "GET /api/nodes -> [].proc_used",
"required": true
},
{
"name": "rabbitmq_node_processes_total",
"type": "gauge",
"help": "Erlang process limit of the node.",
"unit": "",
"labels": [
{
"name": "node",
"description": "Cluster node name (JSON field 'name').",
"allowed_values": null,
"required": true
}
],
"value": {
"min": 0.0,
"max": null
},
"source": "GET /api/nodes -> [].proc_total",
"required": true
},
{
"name": "rabbitmq_node_uptime_milliseconds",
"type": "gauge",
"help": "Milliseconds since the node started.",
"unit": "milliseconds",
"labels": [
{
"name": "node",
"description": "Cluster node name (JSON field 'name').",
"allowed_values": null,
"required": true
}
],
"value": {
"min": 0.0,
"max": null
},
"source": "GET /api/nodes -> [].uptime",
"required": true
},
{
"name": "rabbitmq_node_disk_free_bytes",
"type": "gauge",
"help": "Free disk space on the node, in bytes.",
"unit": "bytes",
"labels": [
{
"name": "node",
"description": "Cluster node name (JSON field 'name').",
"allowed_values": null,
"required": true
}
],
"value": {
"min": 0.0,
"max": null
},
"source": "GET /api/nodes -> [].disk_free",
"required": true
},
{
"name": "rabbitmq_node_running",
"type": "gauge",
"help": "Whether the node is running (1) or not (0). Converted from the boolean 'running' field.",
"unit": "",
"labels": [
{
"name": "node",
"description": "Cluster node name (JSON field 'name').",
"allowed_values": null,
"required": true
}
],
"value": {
"min": 0.0,
"max": 1.0
},
"source": "GET /api/nodes -> [].running (boolean mapped to 0/1)",
"required": true
}
],
"notes": "All endpoints use HTTP Basic auth on port 15672. /api/queues and /api/nodes return arrays; each element yields a series labelled by queue+vhost or node respectively. message_stats may be absent on objects with no activity yet, so message_stats-derived metrics are required=false. Node 'running' is boolean and is converted to 0/1. Node uptime is exposed as a gauge in milliseconds as reported by the API (docs do not state it is a counter). String fields (rabbitmq_version, erlang_version, state) are not exported as metrics."
}