Check if there is a file called maintenance.flag in your magento root. If so Delete it.
EDIT: In Magento 2.* the maintenance.flag file is now located in the var folder
When Magento is performing certain tasks it temporarily creates this file. Magento checks for its existence and if it’s there will send users to the page you described.
It’s supposed to automatically delete this file when done processing whatever task it was doing, but I’ve experienced occasions where something went wrong and it failed to delete it.