Unless your endurance is poor to begin with, I think speed focused training will not improve your endurance.
Speed focused training (fast swimming, long rest, broken swims, strength, etc.) will not cause your aerobic fitness to improve. Your heart and lungs will not spend enough time at the levels required to improve.
At least that's my theory!