Transmission phase measurements of many-electron quantum dots (small mean level spacing \delta) revealed universal phase lapses by \pi between consecutive resonances. In contrast, for dots with only a few electrons (large \delta) the appearance or not of a phase lapse depends on the dot parameters. We show that a model of a multi-level quantum dot with local Coulomb correlations and arbitrary level-lead couplings reproduces the generic features of the observed behavior. The universal behavior at small follows from Fano-type antiresonances of pairs of renormalized single-particle levels.