I grew up there, so is my old stomping ground.....
Agree, the caravan park there is really nice, spacious and very convenient, given that purpose of your visit is to Drifta (and that won't be a quick visit there!!- Luke has a lot to see there). On the edge of the town sporting park complex- really pretty and probably one of the best in NSW.
Another option is just beyond Barrington, just over the bridge (heading away from Gloucester- about 6km out of town), and on the right, on the banks of the Barrington River (Barrington Reserve)- not big, but enough for an overnight stay.