I suppose if you swab for about 30,000 years, some of the metal will wear off making the instrument a little flatter. Other than that, I see no harm in swabbing daily. I sometimes do it twice if I've put the instrument away and then taken it out to play again later. It's always wet after playing.
After I've been playing i use to clean and make my ready sax ready for next seance. It just take a few minutes. I make a "roll" of strong kitchen paper, Once a week I use to damp the paper with vinegar. Once a months I rinse the neck with luke warm water c 30°. I use the brush for this.
...... use to be a pond down in the bow and around the palm keys are also lots of water.. Make two paper rolls. Make sure one comes down to the bow and one reach the palm keys. Let it soak up the water for some minutes. (I use to dry the neck while I'm waiting) . I tilt the sax to get most of water out from the sax and remove the paper rolls before I do next step.
..... dry every pad with a strong kitchen paper, pad paper (no powder) or some kind of pad dryer, The closed pads are important. Add a some oil and cork grease if necessary. No more sticking pad, "snafs" (dirt) in the neck, tube or on the pads. The sax is ready to be used. And your pads will last longer as well.