Have been to both, and while more vans seem to go to Diamond, it isn't really a van park as such, just a grassed campground, which I don't think is as nice as Kylies, but is closer to the beach.....as in 50m away compared to 2 or 300m for Kylies..
On any given weekend ( except for holidays/long weekends ) Kylies is a lot quieter than Diamond ( except for the Kangas,monitors and occasional Koala )
Bring your own firewood or buy it at Diamond ( was $5 a bag last time I was there...very cheap )