We perform supercurrent and tunneling spectroscopy measurements on gate-tunable InAs/Al Josephson junctions (JJs) in an in-plane magnetic field and report on phase shifts in the current-phase relation measured with respect to an absolute phase reference. The impact of orbital effects is investigated by studying multiple devices with different superconducting lead sizes. At low fields, we observe gate-dependent phase shifts of up to φ0 = 0.5π, which are consistent with a Zeeman field coupling to highly transmissive Andreev bound states via Rashba spin-orbit interaction. A distinct phase shift emerges at larger fields, concomitant with a switching current minimum and the closing and reopening of the superconducting gap. These signatures of an induced phase transition, which might resemble a topological transition, scale with the superconducting lead size, demonstrating the crucial role of orbital effects. Our results elucidate the interplay of Zeeman, spin-orbit, and orbital effects in InAs/Al JJs, giving improved understanding of phase transitions in hybrid JJs and their applications in quantum computing and superconducting electronics.