Differential Drive robot car using cart-caster vs. omnidirectional ball caster

I am building a differential drive robot based on Linorobot. I thought I would be clever and use TWO shopping-cart-style casters (more stable) instead of the ONE usual omnidirectional ball caster.

A shopping card caster (also seen for example on office chairs) has a wheel on a rotating base. The wheel is directional but the base rotates. The thing is that (as we know from shopping carts) when you start going in a new direction it takes a moment for the base to rotate during which time it resists motion or tries to steer somewhere else.

Now that I am running it I am almost positive that the kinematics are totally different and in fact very hard to model. The more I think of it the more I believe that the default kinematics for a differential drive won’t work and the vehicle will not go in the intended direction.

Yet for example the ubiquity robotics Magi uses just such a setup. Do you know a way to handle the kinematics or should I just go back to the same omnidirectional wheel caster that everyone always seems to use? Thank YOU!