SSH Agent forwarding on Ubuntu 14.04

Use Case: You have got some server in cloud that are not exposed to internet directly. You have one server called (bastion host) by which you can login to the back end servers.

You can login to the bastion host and from there, you can login to the rest of the server that are not exposed to internet. In this way you need to keep the files on bastion host that is not good.

With the help of agent forwarding you don't need to store private key on bastion host, key will be on your local machine and via ssh agent-forwarding the same key would be used to login to the back end server.

I believe that you already know how to login to linux server with key authentication. If not then you can find it here

I assume that you already have setup login via private key on destination server.

I assume that you got host1(client) bastion(middle Server, bastion host) and server1 (Destination Server). With the help of SSH Agent forwarding I can login from host1>bastion>server1 without storing any private key on bastion.
From a network perspective, you ideally want your private servers only accessible via a bastion host or other intermediary host that doesn't contain any of private key.

On Server: You don't need to do anything if you have setup login via private/public key authentication

On Client:
Add your key to your ssh deamon
$ ssh-add "/home/keys/private.pem"
You will receive the below out put
$ ssh-add /home/keys/private.pem
Identity added: /home/keys/private.pem (/home/keys/private.pem)

You may allow Agent forwarding in sshd_config file or can issue in command as well
You can specify -A in ssh command to use agent forwarding

SSH Client configuration
The SSH clients should be configured to allow agent forwarding. The following entry should be added in/etc/ssh/ssh_config file to apply this change for all users in the server, or /home/user/.ssh/config file to apply this change only to specific user.

Host *
    ForwardAgent yes
The above enable agent forwarding from all hosts. It is an aggressive setting. You may want to restrict it to specific hosts depending on your use case.

ssh -At bastion.example.com ssh server1.example.com

This will take you to server1.example.com

How to Setup and Secure Linux SSH Logins to use Private PEM Keys

SSH logins are susceptible to brute force attacks. A thousand things can go wrong which could give someone unauthorized access to your server. The best way to secure your SSH login is to use Public/Private PEM keys. This is default login type for Amazon EC2 servers. Unfortunately Amazon’s interface only created a single account. This tutorial will show you how to setup additional PEM keys for other users.
Once you’ve logged into your server, do the following:
Step 1: New Account setup
Here we will create the new account, and add them to the sudoers group.
sudo adduser user1
sudo su
passwd user1
visudo
Optional: Add the user to Sudoers
visudo
#add this to the last line
1 user1   ALL = (ALL)    ALL
Step 2: Generate the Public/Private key files
Now we will create the public and private key files for user1
su user1
#Enter the password
cd ~/
ssh-keygen -b 2048 -t rsa -f user1
mkdir .ssh
chmod 700 .ssh
cat user1.pub >> .ssh/authorized_keys
chmod 600 .ssh/authorized_keys
chown 600 user1 .ssh
chown user1 .ssh/authorized_keys

Step 3: Download your private key
If you key is not in .pem format you can change it via below command
openssl rsa -in user1 -outform PEM -out user1.pem
You will now have to download, or copy the contents of your private pem file.
If you are going to copy the contents of the file to a key file on your local system, just copy and paste the data into a new file.
Before using your key, make sure to change the permissions to 600.
chmod 600 user1.pem
Step 4: Test your SSH Login
Now let’s test our password-less login to make sure the private pem files are working.
ssh -i /path/to/file/user1.pem user1@server1.exampledomain.com
That should do it! Hope you find this tutorial workable.