Auto-generating Hardware Maps (SPT3G)

Introduction

Writing a hardware map manually for more than one or two LC combs becomes challenging due to the large amount of bookkeeping involved in tracking bolometer names and the readout channels mapped to them. This results particularly from the fact that the hardware map files contain a significant amount of redundant information that can be automatically reconstructed from knowledge of the connections between the warm and cold electronics and the wiring on the wafer itself. To streamline the generation of hardware maps for the SPT3G wafer layout, a set of scripts exists in pydfmux to auto-generate SPT3G hardware maps from the minimal specification of the electronics connections. This specification can be provided as a spreadsheet that is usually small enough that it can be realistically hand-edited in a few minutes for a single wafer. Note that this guide currently only applies to SPT3G wafers and readout.

End users in the lab will probably only find it necessary to use the bash script pydfmux/spt3g/make_hwm_lab_template, so stick to that when getting started. The basic flow of this script and of the auto-generating tools mirrors the steps taken in setting up a new wafer:

  • Generate a skeleton HWM (just a YAML file) containing just IceBoards, which is useful for taking network analyses.
  • Match LC channels to peaks/frequencies in network analyses.
  • Generate lists of channels to exclude from HWM.
  • Build the full HWM.

Each of these stages can be run one at a time by toggling flags at the top of the script, and usage is described in the section below titled “Stages of Setting up a Hardware Map”. In addition to this script, you will also need to supply a list of the electronics connections, called a meta-hardware map. The meta-HWM is basically a stripped down tab-separated spreadsheet that specifies which IceBoard is connected to which SQUIDs are connected to which LCs, etc. This is described in the next section “Writing Meta-Hardware Maps”.

Writing Meta-Hardware Maps

Due to the variety of possible hardware configurations and nomenclature schemes, the format of meta-HWMs have a fair amount of flexibility that has the potential to generate some confusion. So we will stick to the formatting that is likely to be useful for the vast majority of testing cases in the lab. An example of a meta-HWM that one might use in the lab is shown in the table below.

year wafer iceboard squid_board squid lc_chip side flex_cable overbias_amp type sptpol_LC_side resistor_board
2017 W170 0028 07-08 7 LC68.v3.b5.c16 2 7,8 30 3G    
2017 W170 0028 07-08 8 LC68.v3.b5.c15 2 5,6 30 3G    
2017 W170 0028 07-08 4 LC68.v3.b5.c8 1 7,8 30 3G    
2017 W170 0028 07-08 1 LC68.v3.b5.c7 1 5,6 30 3G    
2017 W170 0028 07-08 3 LC68.v3.b2.c16   resistors 30 3G   b1
2017 W170 0028 07-08 2 LC68.v3.b2.c15 1 3,4 30 3G    
2017 W170 0136 B33 5 LC12.sptpol.006.5 1 1 30 sptpol 1  
2017 W170 0136 B33 6 LC12.sptpol.006.6 1 1 30 sptpol 1  
2017 W170 0136 B33 7 LC12.sptpol.006.7 1 1 30 sptpol 1  
2017 W170 0136 B33 8 LC12.sptpol.006.8 1 1 30 sptpol 1  
2017 W170 0136 B33 1 LC12.sptpol.006.1 1 2 30 sptpol 2  
2017 W170 0136 B33 2 LC12.sptpol.006.2 1 2 30 sptpol 2  
2017 W170 0136 B33 3 LC12.sptpol.006.3 1 2 30 sptpol 2  
2017 W170 0136 B33 4 LC12.sptpol.006.4 1 2 30 sptpol 2  

You must then save this as a tab-separated file that will look something like this:

year        wafer   iceboard        squid_board     squid   lc_chip side    flex_cable      overbias_amp    type    sptpol_LC_side  resistor_board
2017        W170    0028    07-08   7       LC68.v3.b5.c16  2       7,8     30      3G
2017        W170    0028    07-08   8       LC68.v3.b5.c15  2       5,6     30      3G
2017        W170    0028    07-08   4       LC68.v3.b5.c8   1       7,8     30      3G
2017        W170    0028    07-08   1       LC68.v3.b5.c7   1       5,6     30      3G
2017        W170    0028    07-08   3       LC68.v3.b2.c16          resistors       30      3G              b1
2017        W170    0028    07-08   2       LC68.v3.b2.c15  1       3,4     30      3G
2017        W170    0136    B33     5       LC12.sptpol.006.5       1       1       30      sptpol  1
2017        W170    0136    B33     6       LC12.sptpol.006.6       1       1       30      sptpol  1
2017        W170    0136    B33     7       LC12.sptpol.006.7       1       1       30      sptpol  1
2017        W170    0136    B33     8       LC12.sptpol.006.8       1       1       30      sptpol  1
2017        W170    0136    B33     1       LC12.sptpol.006.1       1       2       30      sptpol  2
2017        W170    0136    B33     2       LC12.sptpol.006.2       1       2       30      sptpol  2
2017        W170    0136    B33     3       LC12.sptpol.006.3       1       2       30      sptpol  2
2017        W170    0136    B33     4       LC12.sptpol.006.4       1       2       30      sptpol  2

The column titles and values are essentially self-explanatory, but a few points are worth clarifying:

  • The file should be tab-separated, similar to the mapping, wafer, and LC files used in the hardware map.

  • The overbias_amp parameter is specified in units of pA. This number ends up being the default overbias power.

  • The fields sptpol_LC_side and resistor_board are not mandatory unless you are mapping SPTpol LC boards or resistor boards:

    • Handling SPTpol LCs: specify sptpol in the type field, and the specify which side of the LC board is connected to each flex cable using the sptpol_LC_side field.
    • Handling resistor boards: Specify resistors in the flex_cable field, leave side blank, and give the resistor board a name in the resistor_board field.

  • Specifying the year may seem rather odd. In order to maintain compatibility with the conventions at Pole, bolometer names are designed to be independent of channel matching so that they are invariant under corrections to the channel matching. That convention includes the year information so that data from separate seasons is treated as coming from separate detectors by default.

Stages of Setting up a Hardware Map

Once the meta-HWM is written, one can use pydfmux/spt3g/make_hwm_lab_template to go through each of the stages of building the hardware map. These are described in the following subsections. It is advisable to create a new directory to hold the hardware map files and copy the make_hwm_lab_template script to a subdirectory in that location:

mkdir my_hwm
cd my_hwm
mkdir build
cp /path/to/pydfmux/spt3g/make_hwm_lab_template build

It is good practice to keep all files needed to generate the hardware map, such as the meta-HWM file, in the build subdirectory, while the hardware map files themselves reside one level up.

Before getting started running this script, you will need to edit the variables in the section of the script called ##### USER SETTINGS #####. The meaning of each of these variables should be documented internally in the comments in the script.

Basic YAML File for Netanals

Before taking network analyses (or really doing anything at all), one needs a skeleton hardware map that contains all IceBoards and readout modules. This can be created simply by generating a YAML file containing all the IceBoards specified in your meta-hardware map. This is easy to do by hand, but it can also be performed by editing make_hwm_lab_template to use the settings:

DO_SQUID_ONLY_HWM=1
DO_LC_MATCH=0
DO_BUILD_EXCLUDE_FILE=0
DO_BUILD_HWM=0

In addition, check that all paths declared in the body of the script are set to something reasonable. This will generate the skeleton YAML file that you can use with pydfmux to take network analyses.

Then run the script:

source make_hwm_lab_template

LC Matching

After taking network analyses and, if you didn’t set use_mesh=True when taking your network anlyses, fitting them to extract the bias frequencies, we will match the resonant frequencies to bolometers. This can be done with the template script by setting:

DO_SQUID_ONLY_HWM=0
DO_LC_MATCH=1
DO_BUILD_EXCLUDE_FILE=0
DO_BUILD_HWM=0

Be sure to also point the variable NETANAL_DIR to the data directory containing the network analysis output (or fits). The script should be able to identify which network analysis files to use, but note that it uses data from all readout modules in the directory. If there are bad netanals that you don’t want to include, you should delete them, or copy only your good data to a new directory. The script also needs to know whether the network analysis data was taken using the use_mesh=True setting in the master tuning script. If so, set IS_MESH=true, otherwise set IS_MESH=false. Finally, be sure that LCMATCH_FILE is set to the full path and name of the pickle file that you want to create to store the LC matching data produced by the script. After making changes, again run:

source make_hwm_lab_template

This may take a few seconds, after which several new files will be created. There will be some basic plots that show which channels were matched to which resonances, and there will be a pickle file that contains a summary of the matching information to be used later when writing the hardware map.

Generating Exclude Lists

Before writing the hardware map, we can optionally specify a list of detectors that we will include in the wafer files, but which will be set to not be overbiased or tuned. These lists are called exclude lists and can be generated in two ways:

  • From wafer pinout: The warm pinout information from each wafer (as generated from the automatic wafer pinout board) can be read automatically and used to generate an exclude list file. This information is usually available from Fermilab, which typically bonds and does the warm pinout. Look on the wiki for the data, or contact Donna, Adam, or Sasha at Fermilab. Opens and TES-TES shorts are the defects that are excluded.

  • Manually: Exclude lists can be manually written as text files. These are tab-separated files that can use four different types of column headers for specifying channels:

    wafer    side    flex_cable    zif_odd
# or
LC_ind    lc_chip
# or
wafer    physical_name
# or
crate    slot    squid    LC_ind

    Depending on what data are used to identify the channels to cut, different formats are more or less natural.

Setting Custom Bolometer Properties

One occasionally wants to set custom properties for bolometers in the wafer files. Perhaps you want to set all of the overbias powers bolometer-by-bolometer to correspond to the measured saturation power of the bolometer, plus a margin of a few picowatts. There is a feature that allows any field in the standard wafer files to be replaced with values specified in a user-supplied tab-separated CSV file. The format of this CSV file is:

wafer    name    myproperty1    myproperty2
# or
wafer    physical_name    myproperty1    myproperty2

To tell the hardware map builder to use this file, specify the path and name to the file in the following line of the master template script:

CUSTOM_BOLO_PROPS=/path/to/custom/properties.csv

Bolometers that match the name and wafer in the custom CSV file will have their properties replaced in the respective wafer file.

Writing the Final Hardware Map

After setting any exclude lists, we’re finally ready to build the full hardware map. To do this, set the following in the template script:

DO_SQUID_ONLY_HWM=0
DO_LC_MATCH=0
DO_BUILD_EXCLUDE_FILE=0
DO_BUILD_HWM=1

After making changes, once again run:

source make_hwm_lab_template

This should generate all the output files you expect to see in a hardware map.

Advanced Usage

The following python modules and scripts are involved in the hardware map generation:

  • pydfmux/spt3g/make_hwm_lab_template : Bash script wrapper for make_hwm.py, with flags that can be set to generate all pieces of a hardware map. This script is particularly useful as a way to keep track of what flags are being used in the hardware map generation. This version of the template is appropriate for users performing tests in the lab.
  • pydfmux/spt3g/make_hwm_template : Similar to the previous Bash script, but with defaults more appropriate for the setup at Pole.
  • pydfmux/spt3g/make_hwm.py : command-line Python script with user-level features that perform each stage of the channel matching and hardware map generation
  • pydfmux/spt3g/hwm_tools.py : module containing low-level functions that handle various aspects of the channel mapping and hardware map writing
  • pydfmux/spt3g/lc_tools.py : module containing low-level functions that perform the channel matching of the LC network analyses

In principle, end users should only ever need to interact with make_hwm_lab_template, but the command line documentation for make_hwm.py may be useful for making modifications to the template script:

python make_hwm.py -h

Similarly, the python modules may be useful for developers wanting to extend the functionality of the HWM tools.