Do I need to return the handle to the base in the same direction I removed it?

No. The dumbbell, barbell, and curl bar features a symmetrical handle and weight plates. As long as the weight is set equally on both sides, you can return it to the base in either direction.