Every validation run above cost 0 tokens: correctness is decided by scraping the artifact and diffing against the IR, never by a model. Models are used only to read docs, generate the artifact, and repair it.
Pipeline
Stage
Executed by
Model
Cost
Ingest docs
deterministic code
—
2,793 chars normalized
Run + validate
deterministic code
—
1 run(s), 0 tokens
Per-stage attribution was added after this run; only totals are available here.
Validation diff · expected (IR) vs observed (scrape)
Metric
Type
Observed
Series
Sample
Labels
Note
✓
rabbitmq_messages_published_total
counter
counter
1
103,458
—
✓
rabbitmq_messages_delivered_total
counter
counter
1
99,123
—
✓
rabbitmq_messages_acknowledged_total
counter
counter
1
99,103
—
✓
rabbitmq_messages_confirmed_total
counter
counter
1
103,413
—
✓
rabbitmq_queue_totals_messages
gauge
gauge
1
335
—
✓
rabbitmq_queue_totals_messages_ready
gauge
gauge
1
320
—
✓
rabbitmq_queue_totals_messages_unacknowledged
gauge
gauge
1
15
—
✓
rabbitmq_object_totals_connections
gauge
gauge
1
4
—
✓
rabbitmq_object_totals_channels
gauge
gauge
1
8
—
✓
rabbitmq_object_totals_exchanges
gauge
gauge
1
12
—
✓
rabbitmq_object_totals_queues
gauge
gauge
1
3
—
✓
rabbitmq_object_totals_consumers
gauge
gauge
1
5
—
✓
rabbitmq_queue_messages
gauge
gauge
3
120
queue, vhost
✓
rabbitmq_queue_messages_ready
gauge
gauge
3
118
queue, vhost
✓
rabbitmq_queue_messages_unacknowledged
gauge
gauge
3
2
queue, vhost
✓
rabbitmq_queue_consumers
gauge
gauge
3
2
queue, vhost
✓
rabbitmq_queue_memory_bytes
gauge
gauge
3
187,312
queue, vhost
✓
rabbitmq_queue_messages_published_total
counter
counter
3
51,729
queue, vhost
✓
rabbitmq_node_mem_used_bytes
gauge
gauge
1
142,606,336
node
✓
rabbitmq_node_fd_used
gauge
gauge
1
42
node
✓
rabbitmq_node_fd_total
gauge
gauge
1
1,048,576
node
✓
rabbitmq_node_sockets_used
gauge
gauge
1
6
node
✓
rabbitmq_node_sockets_total
gauge
gauge
1
943,626
node
✓
rabbitmq_node_processes_used
gauge
gauge
1
512
node
✓
rabbitmq_node_processes_total
gauge
gauge
1
1,048,576
node
✓
rabbitmq_node_uptime_milliseconds
gauge
gauge
1
22,267
node
✓
rabbitmq_node_disk_free_bytes
gauge
gauge
1
53,687,091,200
node
Cost ledger
Model
Calls
Tokens in
Tokens out
opus
1
7.4K
3.8K
sonnet
2
8.4K
9.6K
total
3
15.8K
13.4K
validation (1 run(s))
0
0
0
Artifacts
exporter.py — Generated exporter · 265 lines
#!/usr/bin/env python3
"""Prometheus exporter for the RabbitMQ Management HTTP API."""
import argparse
import logging
import sys
import time
import requests
from prometheus_client import start_http_server
from prometheus_client.core import (
REGISTRY,
CounterMetricFamily,
GaugeMetricFamily,
)
LOG = logging.getLogger("rabbitmq_exporter")
REQUEST_TIMEOUT = 5
# (metric name without _total, help, JSON path)
OVERVIEW_COUNTERS = [
("rabbitmq_messages_published",
"Total number of messages published to the cluster since node start.",
("message_stats", "publish")),
("rabbitmq_messages_delivered",
"Total number of messages delivered to consumers (with ack) plus basic.get calls since node start.",
("message_stats", "deliver_get")),
("rabbitmq_messages_acknowledged",
"Total number of messages acknowledged by consumers since node start.",
("message_stats", "ack")),
("rabbitmq_messages_confirmed",
"Total number of messages confirmed by the broker since node start.",
("message_stats", "confirm")),
]
OVERVIEW_GAUGES = [
("rabbitmq_queue_totals_messages",
"Total number of messages (ready plus unacknowledged) across all queues in the cluster.",
("queue_totals", "messages")),
("rabbitmq_queue_totals_messages_ready",
"Total number of messages ready for delivery across all queues in the cluster.",
("queue_totals", "messages_ready")),
("rabbitmq_queue_totals_messages_unacknowledged",
"Total number of messages delivered but not yet acknowledged across all queues in the cluster.",
("queue_totals", "messages_unacknowledged")),
("rabbitmq_object_totals_connections",
"Current number of open connections in the cluster.",
("object_totals", "connections")),
("rabbitmq_object_totals_channels",
"Current number of open channels in the cluster.",
("object_totals", "channels")),
("rabbitmq_object_totals_exchanges",
"Current number of exchanges in the cluster.",
("object_totals", "exchanges")),
("rabbitmq_object_totals_queues",
"Current number of queues in the cluster.",
("object_totals", "queues")),
("rabbitmq_object_totals_consumers",
"Current number of consumers in the cluster.",
("object_totals", "consumers")),
]
QUEUE_GAUGES = [
("rabbitmq_queue_messages",
"Total number of messages (ready plus unacknowledged) in the queue.",
("messages",)),
("rabbitmq_queue_messages_ready",
"Number of messages ready for delivery in the queue.",
("messages_ready",)),
("rabbitmq_queue_messages_unacknowledged",
"Number of messages delivered but awaiting acknowledgement in the queue.",
("messages_unacknowledged",)),
("rabbitmq_queue_consumers",
"Number of consumers attached to the queue.",
("consumers",)),
("rabbitmq_queue_memory_bytes",
"Bytes of memory used by the queue's Erlang process.",
("memory",)),
]
QUEUE_COUNTERS = [
("rabbitmq_queue_messages_published",
"Total number of messages published to this queue since it was declared.",
("message_stats", "publish")),
]
NODE_GAUGES = [
("rabbitmq_node_mem_used_bytes",
"Bytes of memory used by the node's Erlang runtime.",
("mem_used",)),
("rabbitmq_node_fd_used",
"Number of file descriptors currently used by the node.",
("fd_used",)),
("rabbitmq_node_fd_total",
"Total number of file descriptors available to the node.",
("fd_total",)),
("rabbitmq_node_sockets_used",
"Number of file descriptors used as sockets by the node.",
("sockets_used",)),
("rabbitmq_node_sockets_total",
"Total number of file descriptors available for use as sockets by the node.",
("sockets_total",)),
("rabbitmq_node_processes_used",
"Number of Erlang processes currently used on the node.",
("proc_used",)),
("rabbitmq_node_processes_total",
"Total number of Erlang processes available on the node.",
("proc_total",)),
("rabbitmq_node_uptime_milliseconds",
"Milliseconds since the node was started.",
("uptime",)),
("rabbitmq_node_disk_free_bytes",
"Bytes of free disk space available on the node's disk monitor partition.",
("disk_free",)),
]
def get_path(obj, path):
"""Walk a nested dict path and return a float, or None if absent/non-numeric."""
cur = obj
for key in path:
if not isinstance(cur, dict) or key not in cur:
return None
cur = cur[key]
if isinstance(cur, bool) or cur is None:
return None
if isinstance(cur, (int, float)):
return float(cur)
if isinstance(cur, str):
try:
return float(cur)
except ValueError:
return None
return None
class RabbitMQCollector:
def __init__(self, target, username=None, password=None):
self.target = target.rstrip("/")
self.session = requests.Session()
if username is not None:
self.session.auth = (username, password if password is not None else "")
self.session.headers.update({"Accept": "application/json"})
def _fetch(self, path):
url = self.target + path
try:
resp = self.session.get(url, timeout=REQUEST_TIMEOUT)
resp.raise_for_status()
return resp.json()
except requests.RequestException as exc:
LOG.error("Request to %s failed: %s", url, exc)
except ValueError as exc:
LOG.error("Invalid JSON from %s: %s", url, exc)
return None
def describe(self):
# Avoid polling the target at registration time.
return []
def collect(self):
yield from self._collect_overview()
yield from self._collect_queues()
yield from self._collect_nodes()
def _collect_overview(self):
data = self._fetch("/api/overview")
if not isinstance(data, dict):
if data is not None:
LOG.error("Unexpected /api/overview payload type: %s", type(data).__name__)
return
for name, help_text, path in OVERVIEW_COUNTERS:
value = get_path(data, path)
if value is None:
continue
fam = CounterMetricFamily(name, help_text)
fam.add_metric([], value)
yield fam
for name, help_text, path in OVERVIEW_GAUGES:
value = get_path(data, path)
if value is None:
continue
fam = GaugeMetricFamily(name, help_text)
fam.add_metric([], value)
yield fam
def _collect_queues(self):
data = self._fetch("/api/queues")
if not isinstance(data, list):
if data is not None:
LOG.error("Unexpected /api/queues payload type: %s", type(data).__name__)
return
labels = ["vhost", "queue"]
gauges = {n: GaugeMetricFamily(n, h, labels=labels) for n, h, _ in QUEUE_GAUGES}
counters = {n: CounterMetricFamily(n, h, labels=labels) for n, h, _ in QUEUE_COUNTERS}
for q in data:
if not isinstance(q, dict):
continue
qname = q.get("name")
vhost = q.get("vhost")
if qname is None or vhost is None:
continue
label_values = [str(vhost), str(qname)]
for name, _, path in QUEUE_GAUGES:
value = get_path(q, path)
if value is not None:
gauges[name].add_metric(label_values, value)
for name, _, path in QUEUE_COUNTERS:
value = get_path(q, path)
if value is not None:
counters[name].add_metric(label_values, value)
for fam in list(gauges.values()) + list(counters.values()):
if fam.samples:
yield fam
def _collect_nodes(self):
data = self._fetch("/api/nodes")
if not isinstance(data, list):
if data is not None:
LOG.error("Unexpected /api/nodes payload type: %s", type(data).__name__)
return
gauges = {n: GaugeMetricFamily(n, h, labels=["node"]) for n, h, _ in NODE_GAUGES}
for node in data:
if not isinstance(node, dict):
continue
node_name = node.get("name")
if node_name is None:
continue
for name, _, path in NODE_GAUGES:
value = get_path(node, path)
if value is not None:
gauges[name].add_metric([str(node_name)], value)
for fam in gauges.values():
if fam.samples:
yield fam
def parse_args(argv=None):
parser = argparse.ArgumentParser(description="RabbitMQ Prometheus exporter")
parser.add_argument("--port", type=int, required=True, help="Port to serve /metrics on")
parser.add_argument("--target", required=True,
help="Base URL of the RabbitMQ management API, e.g. http://localhost:15672")
parser.add_argument("--username", default=None, help="Basic auth username")
parser.add_argument("--password", default=None, help="Basic auth password")
return parser.parse_args(argv)
def main():
logging.basicConfig(
stream=sys.stderr,
level=logging.INFO,
format="%(asctime)s %(levelname)s %(name)s: %(message)s",
)
args = parse_args()
collector = RabbitMQCollector(args.target, args.username, args.password)
REGISTRY.register(collector)
start_http_server(args.port)
LOG.info("Serving metrics on :%d for target %s", args.port, args.target)
while True:
time.sleep(3600)
if __name__ == "__main__":
main()
spec.json — Metric IR (validation contract) · 525 lines
{
"service": "rabbitmq",
"kind": "python_exporter",
"endpoints": [
{
"url": "/api/overview",
"method": "GET",
"auth": "basic",
"auth_detail": "Default guest/guest, only usable from localhost",
"response_format": "json",
"notes": "Whole-system summary including message stats, queue totals, and object totals"
},
{
"url": "/api/queues",
"method": "GET",
"auth": "basic",
"auth_detail": "Default guest/guest, only usable from localhost",
"response_format": "json",
"notes": "List of all queues across all vhosts"
},
{
"url": "/api/nodes",
"method": "GET",
"auth": "basic",
"auth_detail": "Default guest/guest, only usable from localhost",
"response_format": "json",
"notes": "List of nodes in the cluster"
}
],
"metrics": [
{
"name": "rabbitmq_messages_published_total",
"type": "counter",
"help": "Total number of messages published to the cluster since node start.",
"unit": "",
"labels": [],
"value": {
"min": 0.0,
"max": null
},
"source": "/api/overview#message_stats.publish",
"required": false
},
{
"name": "rabbitmq_messages_delivered_total",
"type": "counter",
"help": "Total number of messages delivered to consumers (with ack) plus basic.get calls since node start.",
"unit": "",
"labels": [],
"value": {
"min": 0.0,
"max": null
},
"source": "/api/overview#message_stats.deliver_get",
"required": false
},
{
"name": "rabbitmq_messages_acknowledged_total",
"type": "counter",
"help": "Total number of messages acknowledged by consumers since node start.",
"unit": "",
"labels": [],
"value": {
"min": 0.0,
"max": null
},
"source": "/api/overview#message_stats.ack",
"required": false
},
{
"name": "rabbitmq_messages_confirmed_total",
"type": "counter",
"help": "Total number of messages confirmed by the broker since node start.",
"unit": "",
"labels": [],
"value": {
"min": 0.0,
"max": null
},
"source": "/api/overview#message_stats.confirm",
"required": false
},
{
"name": "rabbitmq_queue_totals_messages",
"type": "gauge",
"help": "Total number of messages (ready plus unacknowledged) across all queues in the cluster.",
"unit": "",
"labels": [],
"value": {
"min": 0.0,
"max": null
},
"source": "/api/overview#queue_totals.messages",
"required": true
},
{
"name": "rabbitmq_queue_totals_messages_ready",
"type": "gauge",
"help": "Total number of messages ready for delivery across all queues in the cluster.",
"unit": "",
"labels": [],
"value": {
"min": 0.0,
"max": null
},
"source": "/api/overview#queue_totals.messages_ready",
"required": true
},
{
"name": "rabbitmq_queue_totals_messages_unacknowledged",
"type": "gauge",
"help": "Total number of messages delivered but not yet acknowledged across all queues in the cluster.",
"unit": "",
"labels": [],
"value": {
"min": 0.0,
"max": null
},
"source": "/api/overview#queue_totals.messages_unacknowledged",
"required": true
},
{
"name": "rabbitmq_object_totals_connections",
"type": "gauge",
"help": "Current number of open connections in the cluster.",
"unit": "",
"labels": [],
"value": {
"min": 0.0,
"max": null
},
"source": "/api/overview#object_totals.connections",
"required": true
},
{
"name": "rabbitmq_object_totals_channels",
"type": "gauge",
"help": "Current number of open channels in the cluster.",
"unit": "",
"labels": [],
"value": {
"min": 0.0,
"max": null
},
"source": "/api/overview#object_totals.channels",
"required": true
},
{
"name": "rabbitmq_object_totals_exchanges",
"type": "gauge",
"help": "Current number of exchanges in the cluster.",
"unit": "",
"labels": [],
"value": {
"min": 0.0,
"max": null
},
"source": "/api/overview#object_totals.exchanges",
"required": true
},
{
"name": "rabbitmq_object_totals_queues",
"type": "gauge",
"help": "Current number of queues in the cluster.",
"unit": "",
"labels": [],
"value": {
"min": 0.0,
"max": null
},
"source": "/api/overview#object_totals.queues",
"required": true
},
{
"name": "rabbitmq_object_totals_consumers",
"type": "gauge",
"help": "Current number of consumers in the cluster.",
"unit": "",
"labels": [],
"value": {
"min": 0.0,
"max": null
},
"source": "/api/overview#object_totals.consumers",
"required": true
},
{
"name": "rabbitmq_queue_messages",
"type": "gauge",
"help": "Total number of messages (ready plus unacknowledged) in the queue.",
"unit": "",
"labels": [
{
"name": "vhost",
"description": "Virtual host the queue belongs to.",
"allowed_values": null,
"required": true
},
{
"name": "queue",
"description": "Name of the queue.",
"allowed_values": null,
"required": true
}
],
"value": {
"min": 0.0,
"max": null
},
"source": "/api/queues[]#messages",
"required": true
},
{
"name": "rabbitmq_queue_messages_ready",
"type": "gauge",
"help": "Number of messages ready for delivery in the queue.",
"unit": "",
"labels": [
{
"name": "vhost",
"description": "Virtual host the queue belongs to.",
"allowed_values": null,
"required": true
},
{
"name": "queue",
"description": "Name of the queue.",
"allowed_values": null,
"required": true
}
],
"value": {
"min": 0.0,
"max": null
},
"source": "/api/queues[]#messages_ready",
"required": true
},
{
"name": "rabbitmq_queue_messages_unacknowledged",
"type": "gauge",
"help": "Number of messages delivered but awaiting acknowledgement in the queue.",
"unit": "",
"labels": [
{
"name": "vhost",
"description": "Virtual host the queue belongs to.",
"allowed_values": null,
"required": true
},
{
"name": "queue",
"description": "Name of the queue.",
"allowed_values": null,
"required": true
}
],
"value": {
"min": 0.0,
"max": null
},
"source": "/api/queues[]#messages_unacknowledged",
"required": true
},
{
"name": "rabbitmq_queue_consumers",
"type": "gauge",
"help": "Number of consumers attached to the queue.",
"unit": "",
"labels": [
{
"name": "vhost",
"description": "Virtual host the queue belongs to.",
"allowed_values": null,
"required": true
},
{
"name": "queue",
"description": "Name of the queue.",
"allowed_values": null,
"required": true
}
],
"value": {
"min": 0.0,
"max": null
},
"source": "/api/queues[]#consumers",
"required": true
},
{
"name": "rabbitmq_queue_memory_bytes",
"type": "gauge",
"help": "Bytes of memory used by the queue's Erlang process.",
"unit": "bytes",
"labels": [
{
"name": "vhost",
"description": "Virtual host the queue belongs to.",
"allowed_values": null,
"required": true
},
{
"name": "queue",
"description": "Name of the queue.",
"allowed_values": null,
"required": true
}
],
"value": {
"min": 0.0,
"max": null
},
"source": "/api/queues[]#memory",
"required": true
},
{
"name": "rabbitmq_queue_messages_published_total",
"type": "counter",
"help": "Total number of messages published to this queue since it was declared.",
"unit": "",
"labels": [
{
"name": "vhost",
"description": "Virtual host the queue belongs to.",
"allowed_values": null,
"required": true
},
{
"name": "queue",
"description": "Name of the queue.",
"allowed_values": null,
"required": true
}
],
"value": {
"min": 0.0,
"max": null
},
"source": "/api/queues[]#message_stats.publish",
"required": false
},
{
"name": "rabbitmq_node_mem_used_bytes",
"type": "gauge",
"help": "Bytes of memory used by the node's Erlang runtime.",
"unit": "bytes",
"labels": [
{
"name": "node",
"description": "Name of the RabbitMQ node.",
"allowed_values": null,
"required": true
}
],
"value": {
"min": 0.0,
"max": null
},
"source": "/api/nodes[]#mem_used",
"required": true
},
{
"name": "rabbitmq_node_fd_used",
"type": "gauge",
"help": "Number of file descriptors currently used by the node.",
"unit": "",
"labels": [
{
"name": "node",
"description": "Name of the RabbitMQ node.",
"allowed_values": null,
"required": true
}
],
"value": {
"min": 0.0,
"max": null
},
"source": "/api/nodes[]#fd_used",
"required": true
},
{
"name": "rabbitmq_node_fd_total",
"type": "gauge",
"help": "Total number of file descriptors available to the node.",
"unit": "",
"labels": [
{
"name": "node",
"description": "Name of the RabbitMQ node.",
"allowed_values": null,
"required": true
}
],
"value": {
"min": 0.0,
"max": null
},
"source": "/api/nodes[]#fd_total",
"required": true
},
{
"name": "rabbitmq_node_sockets_used",
"type": "gauge",
"help": "Number of file descriptors used as sockets by the node.",
"unit": "",
"labels": [
{
"name": "node",
"description": "Name of the RabbitMQ node.",
"allowed_values": null,
"required": true
}
],
"value": {
"min": 0.0,
"max": null
},
"source": "/api/nodes[]#sockets_used",
"required": true
},
{
"name": "rabbitmq_node_sockets_total",
"type": "gauge",
"help": "Total number of file descriptors available for use as sockets by the node.",
"unit": "",
"labels": [
{
"name": "node",
"description": "Name of the RabbitMQ node.",
"allowed_values": null,
"required": true
}
],
"value": {
"min": 0.0,
"max": null
},
"source": "/api/nodes[]#sockets_total",
"required": true
},
{
"name": "rabbitmq_node_processes_used",
"type": "gauge",
"help": "Number of Erlang processes currently used on the node.",
"unit": "",
"labels": [
{
"name": "node",
"description": "Name of the RabbitMQ node.",
"allowed_values": null,
"required": true
}
],
"value": {
"min": 0.0,
"max": null
},
"source": "/api/nodes[]#proc_used",
"required": true
},
{
"name": "rabbitmq_node_processes_total",
"type": "gauge",
"help": "Total number of Erlang processes available on the node.",
"unit": "",
"labels": [
{
"name": "node",
"description": "Name of the RabbitMQ node.",
"allowed_values": null,
"required": true
}
],
"value": {
"min": 0.0,
"max": null
},
"source": "/api/nodes[]#proc_total",
"required": true
},
{
"name": "rabbitmq_node_uptime_milliseconds",
"type": "gauge",
"help": "Milliseconds since the node was started.",
"unit": "milliseconds",
"labels": [
{
"name": "node",
"description": "Name of the RabbitMQ node.",
"allowed_values": null,
"required": true
}
],
"value": {
"min": 0.0,
"max": null
},
"source": "/api/nodes[]#uptime",
"required": true
},
{
"name": "rabbitmq_node_disk_free_bytes",
"type": "gauge",
"help": "Bytes of free disk space available on the node's disk monitor partition.",
"unit": "bytes",
"labels": [
{
"name": "node",
"description": "Name of the RabbitMQ node.",
"allowed_values": null,
"required": true
}
],
"value": {
"min": 0.0,
"max": null
},
"source": "/api/nodes[]#disk_free",
"required": true
}
],
"notes": "Cluster-wide totals from /api/overview use no labels and are distinct metric names from the per-queue series under /api/queues to keep label dimensionality consistent per metric name. message_stats.* fields are absent until the corresponding activity has occurred on the node/queue, so they are marked required=false. Per-queue and per-node metrics carry the identifying labels (vhost+queue, node) from their source objects; no unbounded or ID-like labels (e.g. connection/channel IDs, timestamps) are used."
}