This fixes some strange bugs where deterministic position updates lead
to strange artifacts.