I used to smoke, always respected others and moved away when I would light up. However, if they came and joined me to have a chat, that was their decision and I did whatever I wanted. Whether they were down wind of me or near me, I made the move away first for their health and consideration.
Good on you firefox for speaking up! My SWMBO still smokes and respects others at camping sights. She does go for a walk to get away from the kids. At night though when it is me and her, I don't mind it, but I have never been tempted to start again.