I find once you get out of the metro area, most not too remote places have taps at parks, tourist info centres, sporting ovals etc. Just ask a local.
Never been a problem for us finding water.
I carry a B.E.S.T. filter. It removes most tastes, even from that SA water.
Cheers Glen