The measurement of 2D blood flow velocities using an angle-independent speckle tracking (ST) approach has shown promise as a clinical tool for quantification of cardiovascular deficiencies. However, the ST estimator can be highly corrupted by noise in regions of reduced signal-to-noise ratio (SNR). This work proposes a model-based blood velocity estimation technique which combines ST measurements and color flow imaging (CFI) measurements with a blood flow model based on the Navier-Stokes equations for fluid flow, yielding improved velocity estimates by adaptively weighting the measurements and correcting for estimator artifacts such as Doppler aliasing. Validation with simulated measurements from a computational fluid dynamics (CFD) simulated flow field show an overall improvement in root mean square (RMS) error compared to the ST and CFI measurements, also with spatiotemporal averaging. In vivo examples are included showing that the model-based filter is able to provide robust angle-independent velocity estimates with less need for spatiotemporal averaging, preserving spatial details of the flow field.