It's a bit more than two hours drive, but on the advice of member here we stopped into South West Rocks on our last trip and have to say that it was great we stayed at the Top Tourist Park on the edge of town (dog freindly) really good site with nice staff and the weather was great although a bit cold at night when we were there.....
The town itself was really laid back with plenty of beaches and places to see, although I thought Trial Bay Gaol was a bit exxy to get into.
Someone said they could easily see themselves retiring there and I could see why, saying that, I would go sooner rather than later because when the new sections of the Pacific Coast Highway reach the area I suspect it will turn into the new Byron Bay......