Troubleshooting

By default, the eG agent uses secure shell (SSH) to connect to an IBM HMC server and pull out the 'outside view' metrics related to the AIX LPARs running on the IBM pSeries servers managed by that HMC server. Likewise, the eG agent also uses SSH to obtain the 'inside-view' of the LPARs. Password Authentication is the default method for SSH connections in eG Enterprise. If the eG agent fails to obtain the 'outside view' and/or the 'inside view' of one/more AIX LPARs, it could imply either/both of the following:

1. The problematic AIX LPARs and/or the IBM HMC server managing those LPARs do not support SSH;

2. Password authentication is not supported by the SSH daemon running on the AIX LPAR and/or the IBM HMC server. Under such circumstances, you can perform either of the following:

   • Enable Password Authentication in the SSH daemon on the problematic AIX LPARs and/or on the IBM HMC server via which the eG agent pulls out the 'outside view' metrics of the LPARs; or,
   • Implement Key-Based Authentication between the eG agent and the SSH daemon of the AIX LPAR and/or the IBM HMC server.

If you pick option (1), then follow the steps given below to enable password authentication:

1. Login to the AIX LPAR / IBM HMC server with which the eG agent is unable to establish an SSH connection.
2. Edit the sshd_config file in the /etc/ssh directory.
3. Check whether the Password Authentication flag in the sshd_config file is set to no. If so, set it to yes.
4. Then, save the file and restart/signal the SSH daemon (eg., using kill -1 <sshd_config PID>).

On the contrary, if you choose to enable key-based authentication [i.e, option (2)], then you will have to generate a public/private key pair. A public/private key pair is available in the <EG_INSTALL_DIR>\agent\sshkeys directory (on Windows; on Unix, this will be /opt/egurkha/agent/sshkeys) of the eG agent. While the private key is available in the file named id_rsa, the public key is contained within the file authorized_keys. You now have the option to proceed with the default keys or generate a different key pair. If you decide to go with the keys bundled with the eG agent, do the following:

1. To enable key-based authentication, the private key should remain in the <EG_INSTALL_DIR>\agent\sshkeys directory (on Windows; on Unix, this will be /opt/egurkha/agent/sshkeys), and the public key should be copied to each of the problematic AIX LPARs and/or the IBM HMC server. To achieve this, first login to the problem target (AIX LPAR/IBM HMC server) as the eG user.
2. Create a directory named .ssh in the <USER_HOME_DIR> on the AIX LPAR/IBM HMC server, using the command: mkdir ~/.ssh.

3. Next, copy the authorized_keys file from the <EG_INSTALL_DIR>\agent\sshkeys directory (on Windows; on Unix, this will be /opt/egurkha/agent/sshkeys) on the eG remote agent host to the <USER_HOME_DIR>/.ssh directory on the AIX LPAR/IBM HMC server.

   Make sure that the permission of the .ssh directory and the authorized_keys file is 700. 

4. Finally, on the eG manager host, edit the <EG_INSTALL_DIR>\manager\config\eg_tests.ini file. Against the EgJavaSSHKeyFile parameter, enter: agent/sshkeys/id_rsa.pub, and save the file.

On the other hand, if you want to generate a new key pair, then do the following:

1. Login to any AIX/Linux host in your environment (even an AIX LPAR) as an eG user.

2. From the <USER_HOME_DIR>, execute the command: ssh-keygen -t rsa. Upon executing the command, you will be requested to specify the full path to the file to which the key is to be saved. By default, a directory named .ssh will be created in the <USER_HOME_DIR>, to which the key pair will be saved. To go with the default location, simply press Enter.

   Generating public/private rsa key pair.
   Enter file in which to save the key (/home/egurkha/.ssh/id_rsa):

3. Next, you will be prompted to provide a pass phrase. Provide any pass phrase of your choice.

   Enter passphrase (empty for no passphrase): eginnovations
   Enter same passphrase again: eginnovations

4. If the key pair is created successfully, then the following messages will appear:

   Your identification has been saved in /hom

   e/egurkha/.ssh/id_rsa.

   Your public key has been saved in /home/egurkha/.ssh/id_rsa.pub.

   The key fingerprint is:

   09:f4:02:3f:7d:00:4a:b4:6d:b9:2f:c1:cb:cf:0e:e1 dclements@sde4.freshwater.com

5. The messages indicate that the private key has been saved to a file named id_rsa in the <USER_HOME_DIR>/.ssh, and the public key has been saved to a file named id_rsa.pub in the same directory. Now, to enable key-based authentication, do the following;
6. Login to the AIX LPAR and/or the IBM HMC server as the eG user.
7. Create a directory named .ssh in the <USER_HOME_DIR> on the AIX LPAR and/or the IBM HMC server, using the command: mkdir ~/.ssh.
8. Next, copy the id_rsa.pub file from the <USER_HOME_DIR>/.ssh directory on the AIX/Linux host to the <USER_HOME_DIR>/.ssh directory on the AIX LPAR and/or the IBM HMC server.
9. Ensure that the id_rsa.pub file on the AIX LPAR and/or the IBM HMC server is renamed as authorized_keys.
10. Repeat this procedure on every AIX LPAR to be monitored.

11. Then, lock the file permissions down to prevent other users from being able to read the key pair data, using the following commands:

    chmod go-w ~/

    chmod 700 ~/.ssh

    chmod go-rwx ~/.ssh/*

12. Finally, on the eG manager host, edit the <EG_INSTALL_DIR>\manager\config\eg_tests.ini file. Against the EgJavaSSHKeyFile parameter, enter: agent/sshkeys/id_rsa.pub, and save the file.

Instead of choosing between the authentication modes (Password or Key-based), you can also disable the usage of the Java SSH client, and use plink to connect to AIX LPARs and the IBM HMC server. To achieve this, follow the steps given below:

1. Edit the eg_tests.ini file in the /opt/egurkha/manager/config directory (on Unix; on Windows, this will be <EG_INSTALL_DIR>\manager\config directory).
2. Set the JavaSSHForVm flag in the [agent_settings] section of the file to false; by default, this is set to true indicating that the eG agent uses Java SSH by default. By setting the flag to false, you can ensure that the eG agent does not use Java SSH, and instead uses the plink command to connect to AIX LPARs and the IBM HMC server.

3. The plink command exists in the <EG_INSTALL_DIR>\lib\vmgfiles directory (on Windows; on Unix, this will be /opt/egurkha/lib/vmgfiles) of the eG agent. To use the plink command, you need to explicitly configure the SSH keys, so that the eG agent is able to communicate with the AIX LPARs and the IBM HMC server using SSH. To do this, follow the steps given below:

   • Go to the command prompt and switch to the directory containing the plink command.

   • Then, execute the plink command to connect to any of the AIX LPARs on the IBM pSeries server and to the IBM HMC server. The syntax for the plink command is as follows:

     plink -ssh <user>@<IP_of_target_host> <command>

     For example, assume that you want to connect to the AIX LPAR, 192.168.10.7, as user john with password john, to know its hostname. The syntax of the plink command in this case will be:

     plink -ssh john@192.168.10.7 hostname, where hostname is the command to be executed on the remote host for extracting its hostname.

   • To ensure that you do not connect to an imposter host, ssh2.x presents you with a unique host key fingerprint for that host, and requests your confirmation to save the displayed host key to the cache.

     The server's host key is not cached in the registry. You have no guarantee that the server is the computer you think it is.

     The server's rsa2 key fingerprint is:<host key>

     If you trust this host, enter "y" to add the key to PuTTY's cache and carry on connecting.

     If you want to carry on connecting just once, without adding the key to the cache, enter "n".

     If you do not trust this host, press Return to abandon the connection.

     Store key in cache? (y/n) y

Once you confirm the host key storage and provide the user's password to connect to the AIX LPAR and/or the IBM HMC server, this message will not appear during your subsequent attempts to connect to an AIX LPAR and/or an IBM HMC server. In other words, the eG agent will be able to execute tests on all AIX LPARs and the IBM HMC server without any interruption. Therefore, press y to confirm key storage.