Passivity is a fundamental requirement for rational models to guarantee stable behavior when included in general time domain simulations. This paper introduces a new variant of one passivity enforcement scheme that is based on residue matrix perturbation. The constrained least squares problem associated with the passivity condition is transformed into a least distance problem which is solved as a non-negative least squares problem while utilizing the sparsity pattern that arises with multi-terminal problems. The resulting method is very fast and requires only a small amount of computer memory, thereby being applicable to models with many terminals and high orders. The method is demonstrated to be suitable for frequency-dependent modeling of subnetworks, transformer winding branch impedances, and measured transformer admittance data.