How do you keep off-the-shoulder tops in place?

I recently bought a top like the one pictured and I love the way it shapes my body, but the folded part always rides up and turns into a cowl neck every time I lift my arms a bit too high. I don’t want to return it but if it’s not functional and requires adjusting with every movement, I might just have to.

Is there any way to stop it from moving that much?