Changing SSH Port is often recommended, in order to disables simple scanners

Prerequisite : choose a new port number that is currenlty not used on your server. Prefers an unusual port number rarely used, or used for another function that you do not use and do not plan to use. Wikipedia give a list of TCP/UDP ports.
Check that it is free on your server with any port checker from the internet, like http://www.yougetsignal.com/tools/open-ports/

I will name it xxx below.

For Ubuntu 12.04

1) Open the port you will use in your firewall :

2) Edit /etc/ssh/sshd_config, search for the line "Port 22" and change it to Port xxx. Save and quit

Alternate method : in console, copy and paste this command-line

find '/etc/ssh' -name 'sshd_config' | xargs -d "\n" perl -pi -e 's|#Port 22|Port xxx|g'

3) restart sshd service :

4) Open a new connection with the new ssh port to ensure it works.the curent port in use will become blocked after 5)

5) Close the port 22 that is no longer in use :