I spent so long looking for why we drifted. Turns out we were just walking, slowly, in different directions.