I think it defeats the purpose of having a swag
Only if the purpose of having a swag is to be as uncomfortable as possible.
The extra time involved in putting my stretcher up is about a minute, and as others have said, it's way more comfortable than being on the ground.
It's still a quick and easy way to set up a 'bed' in the bush.
Btw, for the record, my setup is all Darche. A 1000mm wide stretcher, an 1100mm wide swag and sleeping bag, and a 900mm wide self inflating mattress rather than the one that came with the swag. The mattress sits neatly between the stretcher sides, and the swag overlaps them by a couple of inches both side, which means any rain sheds off rather than going between the swag base and stretcher. It's as comfy as my bed at home.