Example Hardware Map Queries¶
Table of Contents
The correct syntax when using HWM query objects is not necessarily intuitive, since it tries to bridge python-like syntax with SQL-like queries.
Once you understand the mechanics though, it because pretty trivial to construct arbitrarily complex queries. Below are some examples demonstrating most of what people will generally be trying to do. If none of these covers your circumstance, let me know (Joshua). This page in particular should be a resource that grows if need be.
What is a Query?¶
A query is a method of hardware map (hwm) objects. It can also refer to the
object that method returns:
In [1]: import pydfmux
In [2]: session = pydfmux.load_session(open('mcgill/hardware_maps/mcgill_hwm_004/mcgill_hwm_complete.yaml','r'))
In [3]: hwm = session['hardware_map']
In [4]: bolos = hwm.query(pydfmux.Bolometer)
In [5]: bolos
Out[5]: <pydfmux.core.hardware_map.HWMQuery at 0x1104a42d0>
That query object is iterable, like a list of returns, but also has some special properties: in particular it knows about all of the methods of the objects contained.
In [6]: bolos.all()
Out[6]:
[Wafer(u'arg1a').Bolometer(u'1A.1.X'),
Wafer(u'arg1a').Bolometer(u'1A.1.Y'),
Wafer(u'arg1a').Bolometer(u'1A.2.X'),
Wafer(u'arg1a').Bolometer(u'1A.2.Y'),
Wafer(u'arg1a').Bolometer(u'1A.3.X'),
Wafer(u'arg1a').Bolometer(u'1A.3.Y'),
Wafer(u'arg1a').Bolometer(u'1A.4.X'),
Wafer(u'arg1a').Bolometer(u'1A.4.Y'),
Wafer(u'arg1a').Bolometer(u'1A.5.X'),
Wafer(u'arg1a').Bolometer(u'1A.5.Y'),
Wafer(u'arg1a').Bolometer(u'1A.6.X'),
Wafer(u'arg1a').Bolometer(u'1A.6.Y'),
Wafer(u'arg1a').Bolometer(u'1A.7.X'),
Wafer(u'arg1a').Bolometer(u'1A.7.Y'),
Wafer(u'arg1a').Bolometer(u'1A.8.X'),
Wafer(u'arg1a').Bolometer(u'1A.8.Y')]
In [7]: bolos[0]
Out[7]: Wafer(u'arg1a').Bolometer(u'1A.1.X')
In [8]: bolos[0].<tab>
bolos[0].channel_map bolos[0].name bolos[0].state
bolos[0].hwm bolos[0].overbias bolos[0].tune
bolos[0].iceboard bolos[0].readout_channel bolos[0].wafer
bolos[0].lc_channel bolos[0].rfrac
bolos[0].metadata bolos[0].squid
In [9]: bolos.lc_channel
Out[9]:
[LCBoard(u'LC001_A').LCChannel(1),
LCBoard(u'LC001_A').LCChannel(2),
LCBoard(u'LC001_A').LCChannel(3),
LCBoard(u'LC001_A').LCChannel(4),
LCBoard(u'LC001_B').LCChannel(1),
LCBoard(u'LC001_B').LCChannel(2),
LCBoard(u'LC001_B').LCChannel(3),
LCBoard(u'LC001_B').LCChannel(4),
LCBoard(u'LC001_C').LCChannel(1),
LCBoard(u'LC001_C').LCChannel(2),
LCBoard(u'LC001_C').LCChannel(3),
LCBoard(u'LC001_C').LCChannel(4),
LCBoard(u'LC001_D').LCChannel(1),
LCBoard(u'LC001_D').LCChannel(2),
LCBoard(u'LC001_D').LCChannel(3)]
In [10]: bolos.overbias_and_null()
2015-08-21 15:45:15 | INFO | overbias_and_null.py | overbias_and_null | STARTING: overbiasing on 1 modules
.
.
.
All of pydfmux “runs on” query objects, in the sense that the core algorithms act
using query objects. Often you will see the algorithms executed by calling
methods of the object contained within query on the set of all objects contained
in the query (bolos.overbias_and_null()).
So, a query object is the central object of pydfmux, it is important to know how to get and manipulate these objects.
Elements of Querying¶
When formulating a query, there are three primary ingredients:
query(pydfmux.OBJECT)specifies the type of object to be contained in the query returned.filter(. . .)filters the contents based on attributes of the object, or any other joined object.join(pydfmux.ANOTHEROBJECT)walks along the ORM map (Objects in a Dfmux Hardware Map.) to “join” objects together in order to filter by joined objects in addition to returned objects (covered in detail at the bottom of this section).
Filters¶
A query for Bolometers, filtered based on whether the “tuned” in the hardware map is “True”:
In [11]: bolos_all = hwm.query(pydfmux.Bolometer)
In [12]: all(bolos_all.tune)
Out[12]: False
In [13]: bolos_tune = hwm.query(pydfmux.Bolometer).filter(pydfmux.Bolometer.tune==True)
In [14]: all(bolos_tune.tune)
Out[14]: True
Note here that .filter is a method of the query object, so additional
filters can be performed on the same query object:
In [15]: bolos_tune_trimmed = bolos_tune.filter(pydfmux.Bolometer.rfrac<0.9)
These could also have been done simultaneously as separate arguments to filter:
In [16]: bolos_tune_trimmed = hwm.query(pydfmux.Bolometer).filter(pydfmux.Bolometer.tune==True,
pydfmux.Bolometer.rfrac<0.9)
Instead of justing using comparators in the filter (<, >, ==, etc) it can also be used to check for inclusion. Given a list of Bolometer names that I wish to tune separately, I query for bolometer whose names are in that list:
In [17]: special_bolos = hwm.query(pydfmux.Bolometer).filter(pydfmux.Bolometer.name.in_(['1A.5.Y', '1A.6.X']))
In [18]: special_bolos.name
Out[18]: [u'1A.5.Y', u'1A.6.X']
Note that the “in” is NOT a separate word (as you might expect from pure python). There is no space between “Bolometer.name.” and “in_”. The underscore is also important.
If, alternatively, I have a list of bolometers I wish to exclude, I can perform
a negation by prepending the filter line with a tidle (~).
In [19]: good_bolos = hwm.query(pydfmux.Bolometer).filter(~pydfmux.Bolometer.name.in_(['1A.5.Y', '1A.6.X']))
In [20]: '1A.5.Y' in good_bolos.name
Out[20]: False
Warning
This is different from python and not intuitive, so I want to emphasize it. To negate a filtering parameter, instead of using “not” as you would in python, it should be prepended with a tilde (~).
Joins¶
What if you want to query for Bolometers, but filter based on the transimpedance of the SQUIDs they are attached to?
This will not work:
In [21]: bolos = hwm.query(pydfmux.Bolometer).filter(pydfmux.SQUID.transimpedance>300) # Bad Query
Because “transimpedance” is an attribute of Bolometers, and this hasn’t shown the
query how to get from Bolometers to SQUIDs. That is the role of join.
In order to get from SQUID to Bolometer we need to step through the ChannelMapping object (see Objects in a Dfmux Hardware Map.).
A correct query will look like:
In [22]: bolos = y['hardware_map'].query(pydfmux.Bolometer).join(pydfmux.ChannelMapping,
pydfmux.SQUID)\
.filter(pydfmux.SQUID.transimpedance>200)
Another example: To query SQUIDs that are controlled by Mezzanine 014, there are two different paths from the SQUID object to the MGMEZZ04 object:
In [23]: squids = y['hardware_map'].query(pydfmux.SQUID).join(pydfmux.SQUIDModule,
pydfmux.SQUIDController,
pydfmux.MGMEZZ04)\
.filter(pydfmux.MGMEZZ04.serial=='008')
In [24]: squids = y['hardware_map'].query(pydfmux.SQUID).join(pydfmux.ChannelMapping,
pydfmux.ReadoutChannel,
pydfmux.ReadoutModule,
pydfmux.MGMEZZ04)\
.filter(pydfmux.MGMEZZ04.serial=='008')
The second query will only work if there actually exist channel mappings between that SQUID and a readout channel. This happens in the CSV file that is part of the hardware map, and is therefore not guaranteed to be present for every SQUID. The first query, on the other hand, makes no such assumptions, and is thus the safer choice.
Note that, once items have been joined, any of them may be filtered on. So, in the example above, I can query for SQUIDs attached to Mezzanine 008, Module 1 like so:
In [25]: squids = y['hardware_map'].query(pydfmux.SQUID).join(pydfmux.SQUIDModule,
pydfmux.SQUIDController,
pydfmux.MGMEZZ04)\
.filter(pydfmux.MGMEZZ04.serial=='008',
pydfmux.SQUIDModule.module==1)
ORM Plot Resource¶
The interconnections between these structures are shown in Objects in a Dfmux Hardware Map..
Paths to walk along the ORM.