Joomla (an OS CMS) is not my favorite piece of software to work with – and out of the box runs like a dog.
I have been working on speeding up on of our clients sites that runs off Joomla – and found that.
1 Turning on cacheing in site->global config
2 Turn off stats again on the site->global config menu.
3 For all of your modules enable caching (if the module suports it)
4 Use image ready(or other graphics software) to reduce the size of any images
you can also try turning on Gzip page compression ( this may help it depends where the bottle neck is)
I am going to look at the database as well but is probaly the usual dogs breakfast that most os mysql databases are so I am not hopefull of finding any speed ups there.