You swim La Jolla so you'll be used to cold spring temps and avoiding spring storms. If you want protected harbor, try Bayshore in Long Beach (check the water quality if it has rained, true everywhere but here in particular).
Other good spots south to north (actually east to west)... Malaga Cove/South Torrance Beach, between palos verdes and Redondo Pier; Hermosa Pier to Manhattan Pier is roughly 2 miles. Can swim up in Playa del Rey, but mnot alot of infrastructure, start getting close to marina del rey so I'd suggest staying south of that.
North of that, the Ketch street beach and venice Pier area.
After that probably have to leapfrog up to Gladstone's area and north, Malibu. Above that, Zuma.