I used to wear a rash guard under my scuba diving wetsuit for this. Problem went away when I replaced it with a better fitting suit. Not sure how good idea this is for a long swim though. Maybe give it a try on your next open water swim? Also, what about something like Body Glide?
