Algorithms

Objects like pydfmux.Bolometer (and its subclasses) are convenient when describing the object model and linkages between them. It is not convenient to mix this “plumbing” code with the higher-level code that perform “science” tasks (e.g. tuning bolometers) within the experiment.

In this section, we describe how sequences of “science” code acting on objects can be structured. We divide this type of code into three categories:

1. Quick “throw-aways”, running on each object in a collection,
2. Calls running on each object in a collection using @macro(), and
3. Calls running on a collection of objects, using @algorithm().

In all cases, the goal is a succinct coding style that does not clutter “science” code with “plumbing” concerns (such as parallelizing the same code across a large collection of objects.)

In the following sections, we describe these three categories. Finally, we describe “registering”, a way to make @macros() and @algorithms() very convenient to access and invoke.

Throw-Aways

Consider the following Python code:

>>> # Grab a Dfmux reference, for convenience
>>> d = hwm.query(pydfmux.Dfmux).first()

>>> # Grab all readout channels
>>> cs = hwm.query(pydfmux.ReadoutChannel)

>>> # Zero their frequencies, and amplitudes
>>> cs.set_amplitude(0, d.UNITS.NORMALIZED, d.TARGET.CARRIER)
>>> cs.set_amplitude(0, d.UNITS.NORMALIZED, d.TARGET.NULLER)
>>> cs.set_frequency(0, d.UNITS.HZ, d.TARGET.CARRIER)
>>> cs.set_frequency(0, d.UNITS.HZ, d.TARGET.DEMOD)

Even with only a single Dfmux board, this snippet results in 2,048 command interactions (4 methods on 8 modules with 64 channels each.) In between each of the four calls, we must wait for the slowest command to complete.

The following call accomplishes exactly the same task using call_with, and involves only a single synchronization point at the end of the call.

>>> def zero(channel):
...     channel.set_amplitude(0, d.UNITS.NORMALIZED, d.TARGET.CARRIER)
...     channel.set_amplitude(0, d.UNITS.NORMALIZED, d.TARGET.NULLER)
...     channel.set_frequency(0, d.UNITS.HZ, d.TARGET.CARRIER)
...     channel.set_frequency(0, d.UNITS.HZ, d.TARGET.DEMOD)

>>> cs.call_with(zero)

This pattern is typically used with “throw-away” code that’s intended to run once from within a confined piece of Python code. The call_with() function dispatches the zero() function once per entry in the query. These calls are dispatched in parallel, ensuring good performance even with a very large number of objects in a query.

Macros

Macros are similar to Throw-Aways, but are intended to be less transient. At their simplest, macros are throw-aways with a little extra annotation:

>>> @macro(pydfmux.ReadoutChannel)
... def zero(channel):
...     channel.set_amplitude(0, d.UNITS.NORMALIZED, d.TARGET.CARRIER)
...     channel.set_amplitude(0, d.UNITS.NORMALIZED, d.TARGET.NULLER)
...     channel.set_frequency(0, d.UNITS.HZ, d.TARGET.CARRIER)
...     channel.set_frequency(0, d.UNITS.HZ, d.TARGET.DEMOD)

The @macro() decorator adds typechecking code to zero(), ensuring any code that calls it passes in only ReadoutChannel objects. You can then invoke it as before:

>>> hwm.query(ReadoutChannel).call_with(zero)

Algorithms

Above, the throwaway or macro functions all took a single Bolometer argument. The call_with() invocation was responsible for parallelizing the call across a number of objects retrieved from a HWM query.

Sometimes, the algorithm code needs to be smarter about how and when it parallelizes. For example, maybe Bolometer objects are a natural input to an algorithm (like the drop_bolos algorithm), but the algorithm needs to do set-up and teardown at a ReadoutModule level. A macro cannot perform these extra steps, because it’s invoked in parallel on each object in a collection.

Instead, we use the @algorithm() macro:

>>> @algorithm(pydfmux.Bolometer)
... def drop_bolos(bolos, ...):
...
...     modules = hwm.query(_dfmux.ReadoutModule) \
...         .join(_dfmux.ReadoutChannel) \
...         .join(_dfmux.ChannelMapping) \
...         .join(bolos_hwm.subquery())
...
...     [...]

In this case, the drop_bolos function receives the entire query object and does not parallelize automatically. Typically, @algorithm() functions include a @macro() definition they invoke when they’re ready to parallelize.

Algorithms can be invoked the same way as throwaways and macros:

>>> hwm.query(pydfmux.Bolometer).call_with(drop_bolos)

Registered Macros/Algorithms

If a macro or algorithm is useful in a wide scope of situations, the loose coupling between objects (e.g. Bolometer) and algorithms/macros (e.g. drop_bolos) can seem cumbersome. By registering macros or algorithms, we provide the means to call them directly, just like an ordinary method associated with a class:

>>> @macro(pydfmux.ReadoutChannel, register=True)
... def zero(channel):
...     [...]

>>> @algorithm(pydfmux.Bolometer, register=True)
... def drop_bolos(bolos, ...):
...     [...]

When declared with the register=True argument, macros and algorithms can be invoked directly:

>>> hwm.query(ReadoutChannel).zero()

>>> bolos = hwm.query(pydfmux.Bolometer)
>>> bolos.drop_bolos()

Again, @macro() parallelizes a function accepting individual objects; @algorithm() applies to functions accepting a collection of objects.

Query Filtering

When working with systems that contain multiple IceBoards, network or hardware problems can occasionally result in one or two IceBoards dropping offline. To avoid having to restart and adjust tuning scripts to exclude the offline boards, it can be useful to have algorithms automatically filter the query with which they’re called. This feature is enabled by declaring the algorithm with the filter_query=True argument.

>>> @algorithm(pydfmux.Bolometer, register=True, filter_query=True)
... def drop_bolos(bolos, ...):
...
...    [...]

To disable or enable this feature when the algorithm is called, the filter_query argument can be added as a keyword.

>>> bolos.drop_bolos(..., filter_query=False)

The filter is applied only to the query input to the top-most algorithm, so algorithms that are called from within other algorithms will not filter their queries again.

The online state of each IceBoard is stored in its online attribute, and can be queried using the check_online() method. Once an IceBoard goes offline, its online state remains False. Once an offline board is recovered, its state can be updated by manually calling the check_online() method with the argument reset=True.