Custom metrics

A program run by the robot can publish its own figures: parts counted, cycle time, last error, a flag. These are custom metrics. The program declares them once, then assigns them like attributes, and every assignment is stored on the robot and streamed to whoever listens. Another client reads them with get_metrics(), during the run or after it.

Quick start

from datetime import datetime, timedelta

from pyniryo_nate import Metric, Nate
from pyniryo_nate.exceptions import ClientError
from pyniryo_nate.models import Joints


class CycleMetrics:
    parts = Metric("parts", 0, int)
    cycle_time = Metric("cycle_time", timedelta(0), timedelta)
    last_error = Metric("last_error", "", str)
    running = Metric("running", False, bool)


with Nate() as nate:
    metrics = CycleMetrics()
    nate.metrics.register_metrics(metrics)
    metrics.running = True

    for _ in range(10):
        started = datetime.now()
        try:
            nate.robot.move(Joints(0, 0, 0, 0, 0, 0)).wait()
            nate.robot.move(Joints(0.5, 0, 0, 0, 0, 0)).wait()
        except ClientError as error:
            metrics.last_error = str(error)
            break
        metrics.parts += 1
        metrics.cycle_time = datetime.now() - started

    metrics.running = False

Declaring

A metric is a Metric placed on a class, with the name under which it is published, its initial value and its type. The type is one of str, int, float, bool, datetime or timedelta, and the initial value must be of that type: anything else raises TypeError when the class is defined.

The metrics come into existence when an instance of the class is passed to register_metrics(). The call declares them to the robot with their initial values and binds the instance to the client.

Warning

Assign a metric only after register_metrics(). Before that, the assignment raises RuntimeError, as the client has nowhere to publish it.

Using

An assignment publishes the new value, and nothing else is needed. Reading the attribute gives the last value assigned, so metrics.parts += 1 reads, adds and publishes.

Closing the client, which the with block does on exit, waits for the pending updates to go through. A program that ends without closing the client may lose its last assignments. How an assignment turns into a message is covered in Under the hood.

Advanced

What identifies the metrics

The metrics of a client are keyed by a metrics id. In a program run by the robot, that id is the execution id of the run: the robot sets it in the environment of the program, next to the execution token described in Connecting to a robot. Two runs of the same program get two sets of metrics, and a run is looked up afterwards by its execution id.

Run from your own machine, the client draws a random id. To read the metrics of such a run, choose the id yourself:

export NATE_CORRELATION_ID=bench-run-1

The id is at most 36 characters and must not contain /, +, #, whitespace or a control character, as it is one level of an MQTT topic. register_metrics() raises DataValidationError otherwise.

Under the hood

A Metric is a Python descriptor: an object that lives on the class and defines __get__ and __set__. When an attribute of an instance is backed by a descriptor on its class, Python routes every read and every assignment through those two methods instead of the instance dictionary. That is what makes a plain-looking assignment do more than store a value:

metrics.parts = 1       # runs CycleMetrics.parts.__set__(metrics, 1)
print(metrics.parts)    # runs CycleMetrics.parts.__get__(metrics, CycleMetrics)

__set__ does two things. It stores the value in a private slot of the instance, _parts for the attribute parts, so that the next read returns it. Then it puts the metric on the publish queue of the client. __get__ returns the slot, or the initial value when nothing has been assigned yet.

The queue is not on the class. register_metrics() attaches it to the instance, once the metrics are declared to the robot. Before that, __set__ finds no queue and raises RuntimeError. Registration also reads the metrics from the class, not from the instance, which is why they are declared as class attributes.

The client starts a daemon thread when it is built. The thread blocks on the queue and, for each metric it takes, sends two things: an MQTT message on device/{device_id}/custom-metrics/{metrics_id} for whoever listens live, and a PATCH on /api/custom-metrics/{metrics_id} so the robot stores the latest value for as long as it keeps its database. The assignment waits for neither, which is why it never slows the move it follows. A failure to send is logged and does not reach the program.

Closing the client puts a stop marker on the queue and waits until everything queued before it has been sent. That is what the with block does on exit. A program that ends without closing loses whatever is still queued, since the thread is a daemon and dies with the process.

Both messages carry the name of the metric, its value as a string and the declared type:

Type	Carried as
str, int, float	str(value)
bool	"True" or "False"
datetime	ISO 8601, value.isoformat()
timedelta	Seconds, value.total_seconds()

Reading the metrics

From a remote host, start the program and read its metrics by execution id. get_metrics() returns the latest value of each metric as a Metric, with the value as a string and its Python type:

from datetime import datetime, timedelta

from pyniryo_nate import Nate


def parse(metric):
    if metric.type is bool:
        return metric.value == "True"
    if metric.type is datetime:
        return datetime.fromisoformat(metric.value)
    if metric.type is timedelta:
        return timedelta(seconds=float(metric.value))
    return metric.type(metric.value)


with Nate(hostname="10.10.10.10", login=("user", "secret")) as nate:
    execution = nate.programs.execute(program_id="...")
    execution.wait()

    for metric in nate.metrics.get_metrics(execution.execution_id):
        print(metric.name, parse(metric))

Past runs are listed by get_executions(), and each execution id is a metrics id.