Whether the zonal structure of a polar vortex matters for surface climate is an open question, with much observational work showing a role but with limited modeling work and demonstration of a causal influence. Here, we isolate this influence using a moist general circulation model. We find that the surface responses differ qualitatively depending on the zonal asymmetries of the shifted polar vortex and concurrently occurring wave reflection events, and provide a mechanistic explanation for why.