Due to the lapse in federal government funding, NASA is not updating this website. We sincerely regret this inconvenience.
// SSH, PIV, and other Connectivity Issues
If you are having issues logging in to an NCCS system, it could be due to one of the following reasons:
- Check the Systems Status page to see if any of the Systems are down.
- Verify you are using the correct RSA Passcode, or if the token has been locked due to multiple failed login attempts. If this is the case, open a ticket with the NCCS User Services Group, or contact the Enterprise Service Desk – (301) 286-3100; National Toll Free: 1(877) 677-2123.
- If you have forgotten your NCCS LDAP Password, if it locked after 5 failed attempts, or if it expired after 60 days, open a ticket with the NCCS User Services Group to have it reset over the phone.
- Extensive shell profiles that execute programs like module commands or output text to STDOUT could cause problems with login and file transfer. It is advisable to keep profiles to a minimum.
Resolving PIV SSH Issues
PIV SSH is only possible if you have a NASA provided PIV card and PIV card reader. Most PIV SSH issues can be resolved by checking your client configuration. Please ensure you have set up your ~/.ssh/config according to the examples on the PIV SSH bastion host documentation.
Additional "direct mode" configuration is required for scp, rsync, or sftp to work.
Additional PIV Troubleshooting
Sometimes ssh or the ssh agent is having issues reading the card or the card reader isn't functioning correctly. If that happens, login.nccs.nasa.gov will fallback to an RSA Token (PASSCODE) because many users don't have PIV access. Usually this can be resolved by re-seating the reader after physically pulling the card out or, when using the ssh agent with PKCS11Provider, refreshing the cached credentials.
Windows users with a NASA provided laptop can run the "PIV Cache Cleaner" application to clear the certificates on their smartcard. If you do not have a NASA provided laptop, see KB0014334 on esd.nasa.gov to do this manually or call the ESD at 1-877-677-2123.
MacOS users can unseat and reseat the ssh-keychain for their PIV card to clear the cached credentials. Linux users may be able to do this too, but they will need to provide the proper path to their PKCS11 software:# To remove and re-add the ssh-keychain. When asked for a passphrase, enter your 6-digit PIN.
$ ssh-add -e /usr/lib/ssh-keychain.dylib ; ssh-add -s /usr/lib/ssh-keychain.dylib
#To confirm the ssh-agent has cached your ssh credentials:
$ ssh-add -l
If you get a "Could not remove card" error on removal, that is normal. It indicates the agent hadn't been using the ssh-keychain prior. If the message is received after entering the PIN for the PIV card, that indicates that either the card isn't inserted or ssh is unable to read the card, in which case, the issue should be directed to the ESD.
If the agent is refusing to re-add the ssh-keychain, then the physical card may need to be removed and reinserted before running the above commands.
If everything works correctly, the ssh-agent will cache the ssh credentials from your PIV card.
Note: If you are having a problem with the PIV card itself, open a ticket with the ESD or call 1-877-677-2123. The NCCS does not manage or distribute NASA PIV cards.
FAQ and common SSH errors
Refer the user to the PowerPoint slides linked on this page of the File Transfer Docs (just below the WinSCP video).
Users doing X11 forwarding may come across the message "/home/userid/.Xauthority does not exist" on their login. This message is simply a warning that the file is not there and it will be regenerated. Often, this message comes from login.nccs.nasa.gov NOT Discover or ADAPT. It can safely be ignored and users should be able to run their typical X11 functions.
When you are logging into Discover via ssh you're actually making two connections. The first is to the "bastion host" login.nccs.nasa.gov which will prompt you for a PASSCODE and then a host name. The second connection is to a Discover login node which prompts for your NCCS Password. There are several "login.nccs.nasa.gov" systems with temporary /home directories because they exist solely as a bastion to direct users to Discover, ADAPT, etc. The .Xauthority file may not be found because the /home directory on login.nccs.nasa.gov is regularly cleared when there are no active connections.
If a user removed the .Xauthority file from their Discover /home manually they would see this message twice. First, the benign one from login.nccs.nasa.gov, then one from Discover. The .Xauthority file gets created in part of the ssh connection called "xauth" and passes information from a local X11 server to the remote connection. The .Xauthority file is meant to only hold entries for current connections. Sometimes, if your connection dies, it will leave old info around from previous sessions. This usually isn't an issue as long as xauth is able to update an entry for a new connection, but other times you just need to remove the file and let it get regenerated.
This is a bug specific to MacOS version 13.4. The error occurs when the PKCS11Provider is defined in more than one "host" block in your ~/.ssh/config file or under a "Host *" block. Because this is defined more than once ssh executes a helper application called ssh-apple-pkcs11 and when it is called again it creates an infinite loop causing the connection to abort. You can see this in action by using the -v flag with your ssh command.
The workaround is to write aliases to run ssh-apple-pkcs11 directly and bypass the reexec. Note: this is specific to MacOS 13.4 and Apple may have changed/fixed this in other releases. Do the following:
# Create the file
touch ~/.ssh/ssh_alias_workaround
# Add these aliases to the file
alias ssh=/usr/libexec/ssh-apple-pkcs11
alias scp="scp -S /usr/libexec/ssh-apple-pkcs11"
alias sftp="sftp -S /usr/libexec/ssh-apple-pkcs11"
alias rsync="rsync -e /usr/libexec/ssh-apple-pkcs11"
# You may "source" the file interactively, or add it to your shell init (.bashrc/.cshrc/.zshrc)
source ~/.ssh/ssh_alias_workaround
To confirm the aliases are present in the shell, run "alias", and they should be listed. Retry your ssh/scp/sftp/rsync command again and it should be working. This workaround will allow you to continue using your existing configuration that defines PKCS11Provider in multiple host blocks. The NCCS currently does not have an expected fix date from Apple, and it may be resolved in future releases.
See PIV Troubleshooting above. Physically remove and reinsert the smartcard from the reader before refreshing your cached credentials.
Changes due to PIV-M requirements have led to disabled password access between Discover login nodes. Users must set up an ssh key pair and put the public key in their ~/.ssh/authorized_keys file to move between login nodes:
# Use the default name/file and the default of no passphrase.
$ ssh-keygen -t ecdsa
$ cat ~/.ssh/id_ecdsa.pub >> ~/.ssh/authorized_keys
This setup only needs to be done once, though keys can be replaced or others can be added. Just make sure the .pub key is added to the authorized_keys file.
A host key verification error indicates that you have a host key cached in your ~/.ssh/known_hosts file that doesn't match what the host/server on the other end returns. This can be resolved by removing the offending key from the line it appears in the known_hosts file. Example error output:
Add correct host key in /home/userid/.ssh/known_hosts to get rid of this message.
Offending RSA key in /home/userid/.ssh/known_hosts:12 # <-- This number is the line it appears.
It can also be removed with the following command:
ssh-keygen -R <hostname or IP> -f ~/.ssh/known_hosts
# The -R flag removes all keys belonging to the hostname and the -f flag specifies the filename of the key file
A user may verify an SSH Fingerprint from the NCCS on this page.
Generate an ssh or ecdsa (not "dsa") ssh key pair without a password (on Discover or ADAPT login node) and then copy the ".pub" key to $HOME/.ssh/authorized_keys file. This is only allowed within the individual system (keys from outside of Discover or ADAPT are not allowed).
Example on Discover
nacks@discover14:~/.ssh> ssh-keygen -t rsa
Generating public/private rsa key pair.
Enter file in which to save the key (/home/nacks/.ssh/id_rsa):
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in /home/nacks/.ssh/id_rsa.
Your public key has been saved in /home/nacks/.ssh/id_rsa.pub.
The key fingerprint is:
SHA256:qvXiEcKueLOGzmVKxgf0OO4lL+YVLXV+f0PtQR2yNeQ nacks@discover14
The key's randomart image is:
+---[RSA 2048]----+
| ..= |
| = +|
| . . . . E.|
|. o.o o o |
| + +o...S. . o |
|o o.o. o. . . . .|
| B *. + . o . |
|*+&. o.o . . |
|=O++..... |
+----[SHA256]-----+
nacks@discover14:~/.ssh> cat $HOME/.ssh/id_rsa.pub > $HOME/.ssh/authorized_keys
Ensure permissions on $HOME and .ssh do not have group/world write and $HOME/.ssh
does not have group or world access:
chmod go-w $HOME
chmod go-rwx $HOME/.ssh