There's no reason that it's wrong, because it's not wrong. Control of your orgasms, and of your touching yourself, is just one more thing that can be negotiated by both you and your partner. Some people enjoy orgasm control, or the control/humiliation of having the right to touch certain body parts or behave in certain ways taken away, but it's not for everyone.
If that negotiation something that your partner is contemptuous of, and if masturbation is important to you but he is not willing to budge on it, then you should stop fingering yourself for just long enough to arrange the finger for vertical display, and should then return to your regularly scheduled programming.