Your experience is normal for many swimmers. If I take a break for a few days (or longer) from swimming, the first few times back I have the same type of congestion. I usually hear the next couple days everyone asking "do you have a cold?". For me it clears once I'm in a regular swimming routine again. I do have pollen/ragweed allergies for which I take Allegra. I do believe I've heard several comments in this forum that some people do have allergic reactions to pool water (I guess chlorine). Not sure I've helped you unless it clears after a few days like it usually does for me. I do remember someone commenting in this forum who uses the inhaler allergy stuff before swimming as a cure to being cleared for a swim.
Welcome to swimming!