Daily Observing Tasks

1. Quick Links
2. Daily Startup Instructions
   1. Quickie version (for experienced observers):
   2. Complete version for novices
3. Daily Shutdown Instructions
   1. Quickie version (for experienced observers):
   2. Complete version for novices
4. Cryogen Fill Instructions
5. IV Curve Skydip Instructions
6. What to check while observing
7. Revision History

Quick Links

Daily Startup Instructions

Quickie version (for experienced observers):

• Open dome and start UIP.  Check that Bolocam pointing setup is being used, focus parameters and mode are ok, that the chopper is OFF, and that the tertiary is in the correct position.
  
• Check that fridge cycle is done.  Do cryo fills if necessary.
• Check that optical path is clear.
• If rotator position is not what you want, change it.
• Check cables and hoses at dewar.
  
• Start slewing to first source.  (Watch out for the sun if it's still up!)
• Prep DAS computer (Andante): open (but do not start) the DAS program, time sync, cable check.
• Run start_tel_util on allegro via login from kilauea.
• Start DAS.
• Start orrery.
• Start observing.
• Start merging and automated analysis software.
• Check last night's raw data backups.
  

Complete version for novices

1. If they are still at the summit, check with day crew to see if it is ok to start moving telescope.
   
   
2. Pull all the red telescope STOP buttons so that you can move the telescope and open the dome.
   
   
3. Open the dome ~1 hr before you would like to start observing to cool the dish.  If necessary, coordinate this with the day crew.  Make sure you are pointed in a safe direction. You only need to open the dome to ~50%. Note in the observing log the time that you open the dome and the fraction.  Make sure there is no sun on the primary or secondary.  When you open fully later, do not allow the dome to open all the way; it can sometimes jam open in that position.  Open all the way and then close it a little bit if you don't know how far to open it.
   
   
4. Log into kilauea as bolocam.  (Use the first desktop of the right screen -- this will keep things organized nicely.)  The password is in the white Bolocam Manual binder. 
   
   At the prompt, type UIP to start UIP. Accept the defaults for the login questions. At the UIP prompt (UIP>) type INSTRUMENT BOLOCAM and immediately check that the antenna computer monitor shows POINTING BOLOCAM and the appropriate focus parameters.  Current values (as of 2010/02):
   
   
   • FOCUS OFFSET = 0.25 mm
     
     
   • Y OFFSET = -0.35 mm
   
   Also, check that the FOCUS MODE is STEALTHY.  If it is not, you can fix this with the command
   
   
   UIP> FOCUS /STEALTHY
   
   
   
5. Check that the secondary chopper is not on.  On the antenna display, look for the BSW field and make sure it says NOT CHOPPING.  If it is still on, type in UIP the command
   
   
   UIP> SECONDARY /STOP
   
   If you see a CHOPPER ERROR message on the antenna display, then you need to reset the chopper controller in the sidecab.  Go into the sidecab (don't forget to ground yourself).  In the instrument rack at the far wall (picture), you will find the chopper controller (picture).  There are two buttons labeled STOP and GO (picture).  Press STOP first, then GO.  The CHOPPER ERROR message should go away.
   
   If after stopping the chopper the Antenna Computer screen does not update the chopper information to NOT CHOPPING, it is likely that the monitor process on the antenna computer died. From the UIP window issue the command
   
   
   UIP> ant/restart=3/nosync
   
   
   to reset the display (cf. the CSO troubleshooting page ).
   
   
6. Check that the tertiary is in out of the way and/or use the UIP command
   
   
   UIP> TERTIARY /CASS
   
   
   to move it out of the way.
   
   
7. Check that the fridge cycle has completed or is near completion (UC Fridge GRT stabilized at < 300 mK; refer to example cycle).  You may not move the telescope from the ZA = 55 fridge cycle position until the 3He pump is < 10K.  Note: if pumping on LHe bath replace steps 5 and 6 with Pumping on the LHe Bath.
   
   
8. If necessary, do cryogen fills.  (The day crew will already have done this for most observers.)  Don't forget to press a red telescope STOP button before starting the fill, and to pull the STOP button after you are finished.
   
   
9. Check that the mirrors are free of obstructions, and the tertiary mirror is out of the way.   This is easiest to do at low ZA, ZA = 10 for example.  In UIP type
   
   
   UIP> ZA 10
   
   
   to move the telescope to ZA = 10.
   
   You can see inside the optics box by removing one of the side covers using the thumb screws; see the picture of the access covers.  This is what the unobstructed mirrors look like: first flat, second flat, and ellipsoidal tertiary.  Wipe away any dust and debris that has accumulated with a kimwipe and alcohol if necessary.  (Kimwipes and ethyl alcohol are available in the AOS lab if not already on the alidade platform.)
   
   If you stick your head into the box, you should be able to see the dewar window.  Be careful to avoid knocking the mirrors while doing this.  If you cannot see the white foam window, you should see it covered by a large aluminum cap.  Remove the cap by simply pulling it off (it is held on by spring loaded pins).  If you take the cap off, leave it outside on the alidade platform so it does not warm up!  
   
   If the eccosorb shield is completely in place, you can check that the flip mirror is out of the way by sticking your head inside one of the optics box access ports and looking out along the optical path, to the first flat mirror and up through the optics box input port to the secondary.  If you can't see the secondary, then the flip mirror may be in the way.  Pull off one of the front panels of the eccosorb shield to check (see the pictures on the Eccosorb Shield page for details on what the shield looks like partially deconstructed).
   
   If the eccosorb shield is not in place, or you have removed one of the panels to check, you can see directly whether the flip mirror is in the way.  Here is what the flip mirror looks like when it is out of the way: front, back.  Once you  have confirmed the flip mirror is out of the beam, see the Eccosorb Shield page for details on setting up the eccosorb shield.
   
   
10. Rotator:
    
    
    • If you will be using the rotator, no setup should be necessary -- the software will automatically send it to the appropriate angle for your observations.  Though it is suggested that you do a test rotation using interactive as explained in the following steps so you can find and get around any problems before you start observing.  It doesn't matter what angle you leave the rotator at after your test, the software will move it to the correct angle when you start observing.  Do not use the rotator unless you have direct confirmation from the Bolocam support person that it is fully functional.
      
      
    • If you will not be using the dewar rotator, then it can be left in a fixed position. You should check with the Bolocam support person to see whether it is necessary to move it from its current position. If it is not necessary to move it, then no action is needed.
      
      If for some reason you do need to reset or change the rotator angle, you may do the following.  You should already be at ZA = 10, if not do so by going into UIP and issuing the command
      
      
      UIP> ZA 10
      
      
      If you will be observing a single field over a large range of parallactic angle and will be scanning in equatorial coordinates, then the rotator position is not very important: sky rotation will average it out.  You should just set the rotator angle to 0 deg using the interactive program.  The details of the rotator control are explained elsewhere, but you can just follow these simple instructions:
      
      
      1. SSH to allegro.submm.caltech.edu (from any terminal) as observer, password in the white Bolocam Manual binder.
         
         
      2. Type interactive at the shell prompt.
         
         
      3. Type 888 to get the current angle.
         
         
      4. Check that the current angle is a reasonable value (inside the rotation range given in the startup message).  If it already 0, then you don't need to do anything.  If it is not zero (e.g., another observer used the rotator and forgot to reset it to 0 deg), type 0 at the prompt to tell the rotator to go to 0 deg.  Go outside and check that, after rotating, the homing tab lines up with the homing sensor as indicated here.
         
         If the current angle is not reasonable, then the dewar needs to be re-homed, see the instructions for doing this.  This will set the rotator to 0 deg also.  Again, check the alignment of the homing tab and homing sensor.  Do a test rotation by using interactive to rotate to 5 deg and then back to 0 deg.
         
         
      5. Exit interactive by typing 999.
         
      
      If you will be doing azimuth scans (e.g., because you are chopping), then the rotator should be set to 13.18 degrees, which is FIDUCIAL_ANGLE - (scan_angle_azel + SCAN_OFFSET_ANGLE) = 23.88 deg - (0 deg + 10.7 deg) = 13.18 deg.  Use the interactive program as explained above to do this, setting the rotator to 13.18 deg instead of 0 deg.
      
      
    • If you will not be using the dewar rotator but will be observing over a small range of parallactic angle and using a raster or dither mode (as opposed to lissajous), then you probably should be using the dewar rotator!
      
      
11. Make sure no cables/hoses have come loose or are binding anywhere.  Some pictures of how the cables and dewar typically look are provided here, here, and here.
    
    
12. Start slewing over to your first source and finish opening the dome -- this will take a long time, it can be done while the remaining steps are happening. You can use the OBSERVE command to do this, or just type in the AZ and ZA commands to get you close. Leave one person dedicated to ensuring no sun problems. Sun should not be allowed to hit the back of the primary or the secondary mirror. Sun is allowed to shine on the back part of the secondary structure (the white part), though be careful because the sun may move DOWN the structure as you observe if the source is rising too quickly.  In step 15, you will start the webcam utility, which will let you monitor the sun exposure from the control room.
    
    
13. DAS computer prep.  The DAS computer sits in the black Bolocam racks on the third floor.  Here is a picture of it.  (Note that you can access the DAS computer remotely using Remote Desktop Connection if you prefer; see the instructions; you can use ahi.submm.caltech.edu, the PC in the computer room, or you can use a remote desktop client on kilauea or any of the other linux machines).  If you see a login prompt, log in as bolocam; the password is in the white Bolocam Manual binder.
    
    
    • Create the directory D:\das_data\YYYYMMDD where YYYYMMDD is the UT day of the data you are about to take.
      
      
    • Delete enough old data such that >= 3 GB are free.
      
      
    • If there is not already an open LabView window titled BCAM_DAS_YYYYMMDD (where YYYYMMDD are a year, month, and day), you will find on the Desktop an icon whose name is of this form.  Double-click on it to open the DAS program.   Change the output directory to the above one you just created.  Make sure Trigger switch is UP.  The Trigger switch as at the bottom of the front panel, you may need to scroll down to see it.  Do not start the DAS yet!
      
      
    • Do a quick check of front of DAS for loose cables; see these pictures for what you should see: jpg1, jpg2, jpg3.
      
      
    • Time-sync the DAS computer:
      
      
    • Open the Windows clock control panel so you can monitor the time on the DAS computer to the nearest second.  Do this by double-clicking on the time display in the lower-right corner of the Windows desktop.  You should see the Date and Time Properties dialog box appear, with a time display with seconds in the Date and Time tab.
      
      
    • Monitor the time on allegro by logging in via SSH: There should be a shortcut to SSH Secure Shell or some similar thing on the Windows Desktop.  Double-click on it.  Click on Quick Connect.  Enter allegro.submm.caltech.edu for the Host Name and observer for the User Name and hit the Connect button.  You may see a dialog box asking about host keys; click Yes to accept any new host keys.  You will prompted for the password (which is in the white Bolocam Manual binder in the control room), after which you should see an allegro shell prompt appear.
      
      You can have allegro provide a continuous time display by typing the following at the shell prompt:
      
      
      > das_sync
      
      
      at which point you should see a time display appear, updating every 0.5 seconds.
      
      
    • Note how far apart the two clocks are.  If you time sync'd the night before and the difference is more than 5 seconds, inform the Bolocam support person, but continue with the rest of the startup procedure.
      
      
    • To synchronize the DAS computer, click on the Internet Time tab in the Date and Time Properties dialog box you opened earlier.  Automatic time synchronization to hau.submm.caltech.edu should already be enabled.  Click the Update Now button.  You should see a message appear that The time has been successfully synchronized with ...
      
      
    • Switch back to the Date and Time tab in the Date and Time Properties dialog box and check that the DAS computer's clock and allegro are sync'd to better than 1-2 seconds.  If they do not become synchronized, inform the Bolocam support person, but continue with the rest of the startup procedure.
      
      
    • Close all the windows you opened (related to the time sync), clicking OK or logging out as necessary.
      
      
    • Do not start the DAS yet!
      
      
14. Start the various programs on allegro (rotator controller, encoder log writer, DAS file copier, pointing log copier):
    
    
    • Log in to kilauea (the computer with the dual screens) as bolocam, password is in the white Bolocam Manual binder.
      
      
    • Start a terminal window in the left screen (the DISPLAY variable for this screen is :0.0).  From this window, SSH to allegro as observer.  The password is in the white Bolocam Manual binder.  You will have to leave this SSH client open because it provides the tunnel through which the X connections of the windows about to be started run.  Be sure to not kill this window later on.  If you accidentally kill it, it's not a tragedy -- none of the processes dies, these windows are just displaying some log files.  You ought to restart these monitor windows by starting a new SSH client to allegro and following the instructions on the Troubleshooting page.
      
      
    • At the allegro shell prompt, type
      
      
      > start_tel_util YYYYMMDD R E 1
      
      
      where
      
      
      • YYYYMMDD = UT date (see the antenna computer display)
        
        
      • R indicates whether you want to use the dewar rotator or not (R = 1 means use the rotator -- it will rotate to the optimal angle for each observation).  You should not use the rotator unless you have direct confirmation from the Bolocam support person that it is working.
        
        
      • E indicates whether you want to read the rotator encoder.  You should always set E = 0 if you are not using the rotator and have no direct confirmation from the Bolocam support person that the encoder is connected and working.
        
        
      The pros and cons of using the rotator were discussed on the PreparingForObserving page.  If you are not using the rotator, make sure you have confirmation from the Bolocam support person that it is either already in the correct location or that you should set the rotator to its nominal angle as indicated above. 
       
      Five windows will pop up: rotator, write_log, dirsync, plogd, and rpcd.  Wait until you see some text in all the windows to make sure that nothing is going wrong.  You won't see much in the rotator window or the dirsync window yet, especially if you are not using the rotator, but you should see at least one line of text in all the windows.
      
      
    • Note: if you are splitting nights and you need to change the rotator mode midway through the night:
      e.g., the first observer wants to use the rotator but the second one does not, or vice versa.  Do the following:
      
      
      1. At a shell prompt on allegro you started above, type
         
         
         > check_tel_util
         
         
         
      2. Get a listing of process id's (PIDs) by typing
         
         
         > ps x
         
         
         Find the PID for the rotator process (will include rotator in the last column) and kill that process by typing
         
         
         > kill -9 PID
         
         
         
      3. Reset the rotator angle as if it were the start of the night as explained above.
         
         
      4. Restart rotator in the mode you want by typing
         
         
         > rotator R >>& /data00/encdir/rotator_YYYYMMDD.log &
         
         
         where R = 0 or R = 1 is the desired mode.
         
         
      5. You should see a message appear in the rotator window indicating the rotator has been restarted in the appropriate mode.
         
      
      
15. Go back upstairs to the DAS computer (or do this using remote access).  Start the DAS by clicking on the square button with an arrow in it that is right under the BCAM_DAS_YYYYMMDD menu bar (you know, File, Edit, etc.).  Wait until you see data (will require up to 1 minute).  You can select the channel to be plotted on each graphs using the menu.  As a reference the DC lockin channels should be ~few volts and bit-limited.  These are channels 32-55 (hex 1), 96-119 (hex 2) and so on, the channels whose mod 64 is 32 to 55.  The bias monitor DC lockin channels are also good to look at because they won't drift due to base temperature or loading changes or due to sky noise.  They are channels 377-382.  The instantaneous bolometer signals for all the working bolometers are also plotted in map form in the two square plots on the right.  After your checks are done - unless you were remotely connected to the DAS computer - turn off the monitor, all lights inside the dome, and go back to the control room.
    
    
16. If you are observing during daylight hours, you will want to use the webcam utility.  It shows the webcams that view the secondary and the back of the primary.  It updates more quickly than the webcam display on the CSO web page.  To start it, type webcam & in the SSH client to allegro that you started above.
    
    
17. Start the orrery so you can see easily where your sources are and where you are in local coordinates.
    
    
    • Start a new terminal window on kilauea in the second desktop of the right display.  Don't forget to use the second desktop of the right display -- otherwise you will suffer from window chaos.
      
      
    • If you want the orrery displayed on knoppix (the screen to the right of the antenna computer display, shown here), type at the shell prompt on kilauea
      
      
      > orrery -display knoppix:0.0
      
      
      If knoppix.submm.caltech.edu is not running (blank screen where orrery should appear, ping knoppix.submm.caltech.edu yields nothing), or if you want the orrery displayed on kilauea, instead type
      
      
      > orrery &
    
    
18. Go to UIP and start your observing.  Sheets for logging your observations are available from the main BolocamWebPage.
    
    
19. Start merging, slicing, and automated analysis:
    
    
    1. Start a new terminal window on kilauea in the third desktop of the right display.  Don't forget to use the third desktop -- otherwise you will suffer from window chaos. 
       
       
    2. Type start_merge YYYYMMDD in the terminal window you opened in the third desktop of the right display.  A window labeled merge should appear and you should see it start updating as it merges the pointing, rotator, and DAS files.
       
       
    3. Type start_autos in the terminal window you opened in the third desktop of the right display.  Eight (yes, eight!) windows should appear.  In each window, IDL will start up and a program will start running.  These programs are:
       
       
    • slicing of merged files into single-observation files:  run_auto_slice_files
      
      
    • cleaning of pointing files: run_auto_clean_files_ptg
      
      
    • mapping of pointing files: run_auto_map_files_ptg
      
      
    • centroiding of pointing maps: run_auto_centroid_files
      
      
    • cleaning of science field files: run_auto_clean_files_blankfields
      
      
    • mapping of science field files: run_auto_map_files_blankfields
      
      
    • diagnostics on science field cleaned files: run_auto_diag_clean_files_blankfields
      
      
    • diagnostics on science field map files: run_auto_diag_map_files_blankfields
      
      
    • For instructions on monitoring these routines, see the section below.
      
      
20. Check the status of the previous night's backups and do the second backup if necessary.  See the DataArchiving page for instructions.
    
    
21. Try to minimize the number of additional windows you open on kilauea; the X server does not appear to be very robust and can crash just from having too many windows open.
    

Daily Shutdown Instructions

Quickie version (for experienced observers):

• Stop DAS.
• Begin to close dome.
• Do cryo fills if necessary and put window cap on dewar.
• Start fridge cycle if you are responsible for it.
• Kill data copying processes.
• Reset rotator angle if necessary.
• STOW the telescope (but be sure dome is sufficiently closed to avoid sun danger!) and go to ZA = 55 if the fridge will be cycled.
  
• Kill automated analysis routines when caught up to end of night's data.
• Start raw data backup.
• Start allegro:/home/observer backup.
• Check that dome is closed, telescope is in desired position, and gas exhaust from dewar is reasonable.
• Exit UIP, log out, turn off the lights, hit the STOP button, lock up, and leave.
  

Complete version for novices

1. Go upstairs and stop the DAS (or use remote access) by pressing the large red STOP button in the right half of the screen.
   
   
2. Start closing the dome and tip back to ZA = 25 for cryo fills. Be careful of the sun during tip back and dome close.
   
   
3. If the day crew is not doing cryogen fills for you, do the following:
   
   
   • Fill LHe and LN (cryogen fills). You should push one of the red telescope STOP buttons while doing a cryo fill. This of course means that you have to wait until the dome is sufficiently closed that there's no sun danger.  Note: if pumping on LHe bath replace step 3 with Pumping on the LHe Bath. 
     
     
   • After the fill is done and the baths are capped, make sure the bath caps are on and the appropriate vent lines attached and that the vent lines are unobstructed and have good gas outflow.
     
     
   • Pull STOP button.
     
     
   • Restart dome closing if not already closed.
     
     
4. Put the window cap on the dewar.  It is spring-loaded and snaps on.
   
   
5. If the day crew is not doing cryogen fills, start the fridge cycle, possibly with a start delay.  You can also ask the day crew to do this for you, just let them know when you want to start observing.
   
   
• Push the red STOP button on the Fridge Monitoring LabView VI. It may take up to a minute for the VI to stop.
  
  
• Click on the startFRIDGE shortcut on the DAS computer desktop. 
  
  
• Set the mode appropriately -- you will in general just want CYCLE.
  
  
• You will be asked for a start delay.  Remember that it is a delay, not the absolute start time!  You want the fridge to be cold in time to start observing, and the cycle takes 2-2.5 hours including stabilization at the end. (If you are pumping, you can delay it more -- the observers can start pumping before the fridge has fully equilibrated.)  You should in general set the cycle to start as late as possible so you are sure to have a cold fridge all night; typical fridge cycle start times are 3-4 pm.  Certainly, schedule the cycle so the LHe bath will be topped off before your cycle starts. 
  
  
• Hit OK.  The fridge monitoring/cycling LabView program Fridge_cycle.vi will appear.  You can check at the top of the screen that the Cycle Enabled button is green, that the Cycle Delay, Output Filename, Start Date, Start Time, and Elapsed Time make sense (remember, start time is in UT).
  
  
• Advise the day crew to leave telescope at ZA = 55 during the cycle, let them know when the cycle will run.
  
  
6. Kill the data file copying processes:
   
   
   • At your ssh client on allegro, issue the command
     
     
     > kill_tel_util
     
     
     Make sure you see an explicit message on the terminal in which you type this command that each of the four processes, rotator, write_log, dirsync, plogd, and rpcd have been killed.  You won't see any message in the monitor windows themselves.  It is especially important that plogd and rpcd be killed to prevent the pointing files from being overwritten by the next day's files.
     
     
   • Explicitly kill the 5 windows with the names rotator,  write_log,  dirsync, plogd, and rpcd using the X button in the windows' title bars.
     
     
7. If you used the rotator, or for some reason set the rotator to a non-standard angle (i.e., neither of values specified above), you need to set it back to a standard angle for the fridge cycle.  Follow the instructions given above for doing this.  The fridge cycle will fail if the rotator angle is set to a value < 0 deg!
   
   
8. Once the dome is far enough closed that there is no Sun danger, type STOW at the UIP prompt to move the telescope to the stow position in azimuth.  You will have to override the ZA command from STOW if you want to leave the telescope at ZA different from 30 degrees; just type IDLE and then issue explicit AZ and ZA commands using the AZ the STOW command was using.  Remember to set ZA = 55 if you want to do a fridge cycle!
   
   
9. If merging is caught up to the end of the day, kill the merge window.  You will likely see an error message on the last file, something to the effect of
   
        Now crashing, satisfied...
        Happily aborting with error
   
   This indicates the program has reached the last (arbitrarily sized) file of the day.  In this case, the error message should be ignored. If the minute on which merge has crashed is before the last minute of data taken, there was a problem and you should look at the merging section of the Troubleshooting page.
   
   
10. If the slicing is caught up to the end of the day (all but the last observation sliced), then hit c in the IDL session running the slicing to have it slice the last file.  Quit IDL and close the window once the last file is sliced.
    
    
11. Once the other analysis routines are caught up, hit q in each of the windows to stop them.  Type exit to exit each of the IDL sessions, then kill the windows.  If problems resulting in the analysis being far from done, you should quit the sessions in the same way regardless and restart all these routines from the office at HP (you can just start them up as you do at the start of the day (see above) -- they will figure out which files have not yet been analyzed).
    
    
12. Start a backup of the raw data. See the instructions on the DataArchiving page.
    
    
13. Back up the /home/observer directory by logging in to allegro as observer and issuing the command
    
    
    > backup_home_disk /data00/backup
    
    
    which should create the file /data00/backup/backup_home_observer_YYYYMMDD.tar.gz.
    


14. Make sure the dome is closed, the telescope is at the desired ZA, and the dewar cryogen baths are capped and have unobstructed vent lines with outflowing gas.
    
    
15. Exit UIP, log out of any open consoles, PUSH A STOP BUTTON, turn off all the lights, lock the door, and look forward to a healthy and delicious breakfast at HP followed by a restful sleep in their fine accommodations.
    

Cryogen Fill Instructions

• Tip to ZA = 25.
  
  
• Push a red telescope STOP button.
  
  
• Use the large safety ladder with the platform step to get to the top of the dewar if possible, it is much safer than the standard ladder.
  
  
• Defrost the inlets of the baths that need fills.  When defrosting, be sure to heat the metal, not the rubber hoses or the RTV at the bottom of the inlets!  A loss of vacuum could really ruin your day.
  
  
• REMEMBER, THE LN BATH IS THE CENTER BATH!
  
  
• Fill LHe from the storage dewar on the alidade platform. Use the longest flexible LHe transfer line you can find; it is possible to fill at this high elevation with the correct line.  Fill LN either using a funnel or from a storage dewar  on the third floor using a long fill line. Do the fills simultaneously to save time.
  
  
• Defrost the bath inlets (heat the metal of the inlet) and replace the bath caps.
  
  
• Check that the vent lines are free of obstructions and gas is flowing out.
  
  
• GET THE LADDER OUT OF THE WAY, ESPECIALLY IF USING THE SAFETY LADDER.  When the telescope tips back to near zenith, the ladder could be crushed, or, worse, the ladder could damage the dewar.
  
  
• Pull the red telescope STOP afterward.

IV Curve Skydip Instructions

1. Slew to whatever azimuth you want to do the skydip at.  Presumably you are doing it in conjunction with observations of some sort of calibrator source, so you should do it at the same azimuth.
   
   
2. Execute the macro by issuing the command
   
   
   UIP> EXEC BOLOCAM_SKYDIP_LOADCURVE.MAC
   
   
3. Wait while the telescope slews to ZA = 5 degrees and until you see the lines appear on the screen
   
   
   C OBSERVER SHOULD GO OUTSIDE AND PREPARE FOR IV CURVE SKYDIP
   
   
4. Go to the alidade platform, open the back of the electronics box so you can see the boards, and flip the switches on the bias board as indicated below, leaving the e-box open after flipping the switches.  The bias board is the 2nd board from the left, it has lots of switches and knobs.  Note that the instructions differ depending on whether you are using a Rev. 2 or a Rev. 3 bias board!  There is a diagram of the Rev. 2 board front panel on the outside of the electronics box for reference (since the front panel is unlabeled!).  You have 2 minutes to do this, don't dilly-dally!
   
   
• Rev. 2 bias board:
  
  
  1. AC/DC switch (TOP switch, right underneath the 9-pin D connector output) should flip to DC mode (DOWN position)
     
     
  2. EXT/INT switch (second switch below the AC/DC switch) to EXT (UP position)
     
     
• Rev. 3 bias board: Set rotary switch to "0V"
  
  
5. Go back inside and wait for the lines to appear on the screen
   
   
   C OBSERVER SHOULD NOW GO OUTSIDE AND ENABLE THE TRIANGLE WAVE
   
   
6. Go back outside and turn on the triangle wave as follows and close the e-box.  Again, you have 2 minutes to do this.
   
   
   • Rev. 2 bias board: flip the AC/DC switch to AC (UP position)
     
     
   • Rev. 3 bias board: set the rotary switch to triangle wave mode (image of a triangle wave)
     
   
   
7. Wait until the macro runs through all the elevations and the FLSIGNAL 128 /RESET command to appears on the screen, followed by the comment line
   
   
   C OBSERVER SHOULD GO OUTSIDE AND RETURN THE BOARD TO ITS NOMINAL STATE
   
   
   and the bell indicating the macro is finished.
   
   
8. Go back outside, open the e-box, put the bias board back into its normal state as follows, and close the e-box:
   
   
   • Rev. 2 bias board: flip the EXT/INT switch to INT mode (DOWN position)
     
     
   • Rev. 3 bias board: set the rotary switch to "0-5V"
     
   
   
9. Continue with observing as before.
   

• antenna computer display and puuoo screen with orrery and webcam
  
  
  
  
• UIP, data stream programs, and (possibly) gbolostrip.
  THIS PICTURE IS OUT OF DATE.  You will not see gbolostrip. and the data stream program windows look different now.
  
  
  
  
• merging and data processing screens
  
  
  

• Make sure that the antenna computer is tracking the intended source, and that ZA < 60.  See here for more details on what all the variables mean and what they should be doing.
  
  
• Check that write_log, dirsync.py, plogd, and rpcd are updating.  These are very important: if they are not updating, then we may be losing data (something may have crashed).  If the rotator is enabled, check that you are seeing dewar rotation messages in the rotator window; certainly, make sure that the rotator window is still up.
  
  
• If running gbolostrip, watch to see if the plots in gbolostrip look as they should.  If you are not running gbolostrip, you can check most of these things by looking at typical channels on the DAS screen (which is accessible in the control room by Remote Desktop Connection, see the instructions):
  
  
• AC broadband noise ~.02 Volts (i.e., above and beyond common-mode sky noise).  Signal names are BOLO_A1, etc.
  
  
• DC level of a few Volts.  Signal names are DCB_A1, etc.
  
  
• Bias monitors functioning and with decent noise.  Signal names are A1_ACBIAS, etc. for AC lockin outputs, A1_DCBIAS for DC lockin outputs.  AC lockin signals should have tens of mV broadband noise.  DC lockin outputs should be a few volts.
  
  
• RGRT = - (resistance/200kohms) x 10 Volts (you can read the resistance bridge directly -- it is the Linear Research LR750 module mounted in the rack with the lockins).  Signal name is RGRT.
  
  
• If you are chopping, check that the chopper encoder signal is being read in correctly and is of the right frequency and amplitude.  Calibration: voltage offset is 0.388 V, scale is 88.1 arcsec/volt.  Signal name is CHOP_ENC.
  
  
• observation number updating (observation number is indicated on the left side of the screen)
  
  
• reasonable values for rotator angle, tau, temperature, humidity (also on the left side of the screen)
  
  
• If you want to look at other signals, you can find their names in
  
  
  allegro:~/quickplot2/config-2003-October/CS.bolocam
  
  
  You will have to look in a different file if the configuration directory you used is different.
  
  
• If gbolostrip is not working, then you can monitor the DAS directly using remote access.  See the instructions given earlier for which signals are on which channels.  Refer to the above information about gbolostrip for indications of what you should see.  Note that you only get access to the bolometer timestreams, no pointing or housekeeping information, so you will be flying somehat blind if you use the DAS only.  Keep an eye to make sure the timestreams are updating and the DAS has not died.
  

• Make sure that the merging, slicing, cleaning, mapping, centroiding, and diagnostic plotting windows are all updating.  The cleaning or mapping may stop if a source is encountered that is not in their params files, which will result in an error message indicating this.  See the Troubleshooting page for instructions on dealing with this. 
  
  
• Keep an eye on the diagnostic outputs.  There are two kinds:
  
  
• Pointing sources: in kilauea:~/data/centroid/, there are directories for each source.  Those directories will have .ctr files that provide centroid information for each bolometer for the pointing observations.  Make sure new files are appearing for the sources you have observed.
  
  
• Science fields: Diagnostic plots for each observation are saved to kilauea:~/data/psd_plot/ and kilauea:~/data/map_sum_plot/, with subdirectories for each source.  For each observation, there will be 10 files in the psd_plot directory and 2 files in the map_sum_plot directory, listed below.  Check that these files are behaving reasonably.  Everything will degrade if the weather gets worse, but make sure things do not change too much (< factor of 2 in sensitivity and PSDs).  (Example plots will be POSTED here at a later date).
  
  
• psd_plot/object_name/YYMMDD_OOO_clean1_psd.eps:
  
  
  • The first 6 pages are bolometer noise plots, one page for each hextant, with point source profile (green) and optimally filtered PSD (red) overlaid.  Should asymptote to 1-2 mV/√Hz above a few Hz, then roll off above about 20 Hz, with some spectral lines between 5 and 20 Hz.  The region below 1-2 Hz will vary with conditions but should in general remain below 5 mV/√Hz.
    
    
  • The last page is for the 6 hextant bias monitors.  The noise levels should be about 1.5 mV/√Hz (Rev. 2) or 0.5 mV/√Hz (Rev. 3) and white, with maybe a spectral line near 10 Hz.
  
  
• psd_plot/object_name/YYMMDD_OOO_clean1_sens.eps: Optimally filtered point-source sensitivity based on above PSDs.
  
  
• psd_plot/object_name/YYMMDD_OOO_clean1_pca_evals.eps: Scan-by-scan histograms of the eigenvalues from PCA cleaning, with the eigenvalue cut displayed (we discard PCA eigenvectors with eigenvalues above the cut).
  
  
• psd_plot/object_name/YYMMDD_OOO_clean1_pca_sum.eps: Summary histograms of the number of PCA eigenvectors that are cut, the value of the cut on the eigenvalue distribution, median of the eigenvalue distribution, and the width of the eigenvalue distribution.  The histograms have one entry for each scan in the observation.
  
  
• map_sum_plot/object_name/YYMMDD_OOO_clean1_map_diag_hist.eps: Histograms of the maps for the observation:
  
  
• histogram of the coverage map (all pixels in naive and optimally filtered maps, good coverage region of naive and optimally filtered maps).  Should have a nice peak.
  
  
• cumulative distribution for the coverage histograms
  
  
• histograms of pixel values (again, full naive map, full optimally filtered map, good coverage region of naive map, good coverage region of optimally filtered map)
  
  
• histograms of sensitivity ( = pixel value x √time in pixel ) (all four versions)
  
  
• map_sum_plot/object_name/YYMMDD_OOO_clean1_map_diag_3d.eps: Maps:
  
  
• naive map (square indicates good coverage region)
  
  
• naive coverage map (square indicates good coverage region)
  
  
• PSD of naive map (good coverage region) (greyscale, surface, and contour plots)
  
  
• optimal filter in freequency space
  
  
• optimally filtered map
  
  
• optimally filtered coverage map
  
  
• PSD of optimally filtered map
  
  
• In the afternoon and morning - be sure the sun is not hitting the primary or secondary.
  
  
• If pumping on the LHe bath, watch for ice plugs; see Pumping on the LHe Bath.
  

• 2003/11/02 JS/SG
  First web version
  
  
• 2003/11/04 SG
  Add Troubleshooting section.
  
  
• 2003/12/07 SG
  Move Troubleshooting section to a separate page.
  
  
• 2003/12/08 SG
  Move Pumping on the LHe Bath section to a separate page.  Move tape backup instructions to a separate page
  
  
• 2004/01/31 SG
  Add link back to main page, add instructions for IV curve skydip.
  
  
• 2004/02/03 SG
  Change instructions to assume analysis is run on kilauea instead of champinux
  
  
• 2004/02/04 SG
  Add link to cabling instructions for DC IV curve skydips, formatting changes, add warning about tertiary flip mirror and secondary chopper and jamming open of dome, add reboot of fridge computer to shutdown procedure, give details on QuickLook signal names.
  
  
• 2004/02/09 SG
  Add check of andante's time synchronization on daily basis, correct directory names (now kilauea:~/data rather than /data10/YYYYMM)
  
  
• 2004/02/14 SG
  Use w32tm to do DAS time sync rather than by hand.
  
  
• 2004/02/26 SG
  Modify start_tel_util instructions for new version of start_tel_util, add setenv TERM vt100 command instructions prior to logging in to alpha1.  More detail on starting DAS.
  
  
• 2004/02/27 SG
  Modify shutdown instruction to use kill_tel_util script.
  
  
• 2004/04/27 SG
  Clarifications and updates, add instructions for skydip IV curves with Rev. 3 bias board.
  
  
• 2004/04/28 SG
  Add link to Eccosorb Shield page, update links to DataArchiving page and remove all tape instructions from this page.
  
  
• 2004/05/05 SG
  Update start_tel_util for new syntax.
  
  
• 2004/05/13 SG
  Add lots of pictures, minor updates.
  
  
• 2004/05/19 SG
  Instructions for window cap, move fridge instructions outside of cryo fill instructions.
  
  
• 2004/05/22 SG
  Minor clarification on options for start_tel_util, add warnings about killing the SSH window.
  
  
• 2004/05/23 SG
  Change instructions to require check of flip mirror even if eccosorb shield is in place, move to ZA = 10 to check flip mirror, add instructions for starting orrery.
  
  
• 2004/05/24 SG
  Detailed instructions for setting rotator to the correct angle at start and end of night, and for changing rotator modes midway through the night.
  
  
• 2004/05/27 SG, PR, DJH
  Add pictures of computer screens with standard windows.
  
  
• 2004/10/02 SG
  Add warning to avoid opening too many X windows on kilauea.  Add quickie checklist for startup and shutdown.  Add link to example fridge cycle.  Clarify where DAS STOP button is and what state it should be in.  Specifically indicate that start_merge and start_autos should be run in right display.
  
  
• 2004/10/09 SG
  Add expected gbolostrip messages.
  
  
• 2004/10/31 SG
  Add link to antenna computer display summary.
  
  
• 2004/11/14 SG
  Emphasize need to stow at ZA = 55 for fridge cycle.
  
  
• 2005/01/25 SG
  Minor updates to docs -- add instructions for resetting chopper, finding PIDs, other minor typos.
  
  
• 2005/02/21 SG
  Drop gbolostrip until it is fixed.
  
  
• 2004/02/23 SG
  Add Quick Links section to jump to troubleshooting pages, specific directions for monitoring DAS if gbolostrip not available.
  
  
• 2005/03/08 SG
  Update DAS computer time-syncing commands.
  
  
• 2005/06/03 SG
  Minor corrections.
  
  
• 2005/09/11 PR
  misc updates (DAS vi related, sync command, 'not chopping' display error, ref to Pika (RDC), ref to Puuoo (orrery), and various pics)
  
  
• 2007/02/04 SG
  Miscellaneous updates to reflect changes to procedures.
  


• 2007/02/21 SG
  Update FOCUS OFFSET to reflect current value.
  
  
• 2007/03/24 SG
  Add note about making sure FOCUS MODE is correct.
  
  
• 2010/02/26 SG
  Update for UIP on kilauea.
  


• 2010/12/09 JS
  Remove references to window cap, which is no longer used.