A bounce is when a visitor to your site leaves without ever visiting more that the page they arrived at.
So, if you have a high bounce rate it COULD mean that the person found the information they wanted and left.
OR it COULD mean that you had a visit from someone and they thought that you page/site was not relevant and so left without looking any further.
You need to look into which pages they are bouncing from, how they are getting to the page, and what you expected them to do.
After that you need to decide if the bounce rate is reasonable or not.
If not, then you are probably getting the wrong people to your site (bad page titles/descriptions, wrong keywords targetted in PPC or for SEO etc) and you need to decide how you can get the right people to your site.