Troubleshooting

This page explains various antenna computer related problems and how to fix them. Most of the advices also apply to the AOS backend computer. Please refer to the official Troubleshooting Guide for details and other generic problems.

1. Antenna computer seems dead.

1.1. Symptom: ANTENNA command hangs.

A process other than UIP on the VMS computer may be holding an exclusive lock for a resource or a disk file required by UIP. Find an offending process and terminate it if there is any.

If that is not the case, check if the antenna computer can be reached over the network (i.e., try ping). If you can ping, try logging in using ssh2. If either ping or ssh2 fails, the problem may be caused by the antenna computer. Follow the instructions below.

1.2. Symptom: Can't login.

Are you trying to logon to the antenna computer using telnet or rsh? For security reasons, those network services have been disabled. You have to use a Secure Shell version 2 client (ssh2) to make a remote connection, or you have to login from the console.

Can you ping the antenna computer? If so, either the secure shell server or the operating system is dead. If ping fails, the problem is likely caused by the network driver or the OS crash. In either case, follow the instructions below.

1.3. Symptom: Can't ping.

Can you still login from the console? If so, login as either root or cso and reboot the computer. If you can not login even from the console, the antenna computer is really dead. Reset the computer by pressing the reset switch. Please report this incident to me.

1.4. Symptom: Can't boot.

This is a very serious problem. If the attached hard disk drive is mechanically dead, or the root file system is badly corrupt and unrepairable by fsck, you have to replace it with a spare drive.

2. Antenna computer is doing odd things.

2.1. Symptom: Time is wrong.

If one of the timing modules on the antenna computer, which is marked as "WWV," differs by nearly a second or more from the real WWVH clock, you have to resynchronize them. The easiest way is to reboot the antenna computer twice.

2.2. Symptom: Pointing is way off.

If the telescope pointing (values of FAZO and FZAO) differs by more than several arcseconds from previous nights, the secondary mirror position, the Sidecab mirror positions (when the heterodyne receivers are being used), and/or the time on the timing module(s) may be wrong. For troubleshooting stepper motor related issues, see this page. For timing problem, see above.

2.3. Symptom: The telescope makes a long slew in the middle of integration.

Your source's azimuth angle may be passing -90 degrees from north or +360 degrees from west. In either case, you are hitting the azimuth limit, and the telescope behavior is normal (you could have avoided it, though). Or, your source may be passing through (or within several degrees of) the zenith.

If the telescope is trying to point below the horizon, or time (and/or date) on the antenna display is apparently wrong, it is likely caused by a glitch of the reference clock (currently WWVH) or the timing module on the antenna computer. If a glitch is more than one second long, you should get a warning message from UIP. If the glitch disappears after a few seconds, you may discard the currrent integration and continue. If the clock error remains and you keep getting messages from UIP, you probably need to reboot the antenna computer to resynchronize the system clock with NTP and WWVH.

2.4. UIP command has no effect.

If you are receiving messages like "FROM ANT: 1" from UIP, the communication link between the VMS computer (alpha1) and the antenna computer is out of sync. This can happen if an UIP command is interrupted by ^Y or a sub-process running UIP commands is stopped by the KILL command. To recover from this state, you have to reload the antenna computer by the ANTENNA command:

UIP> ANTENNA /RESTART /NOSYNCHRONIZE

Check the integrity of information displayed on the status monitor after reload, since some of the score board variables, which are sent to the antenna computer as part of the reload procedure, may be corrupt.

2.5. ANTENNA command times out.

The ANTENNA command will time out if the computer that controls the beam switch and the AOS5 is down. Try to ping the computer rtlinux (128.171.86.14). If it is not responding, reboot it. The computer is in the 4-8 GHz IF processor rack in the AOS room.