One who enters in your site will always leave it. You should care about the bounce rate only.
Alastair, that is definitely true. However, the exit rate can show you which pages are losing your audience the most. If you have a big discrepancy in the exit % on a specific page as compared to the other pages, it might be worth it to look into that page to identify if it is a cause for visitors to leave the website prematurely. Of course this variable may be more or less relevant depending on your business.
Hope this helps