Yes. Patience only lasts so long. Insecurity is devilish because it makes you say things that become self-fulfilling prophecy. "You don't love me," "You're going to leave me," "Why don't you like me?" These all put seeds in a person's head, and every time they are repeated it's like giving water and sunlight to the idea that you're not good enough, that there's something wrong with you, that the relationship won't work out.

That said, I don't know your personal situation at all. I don't know what you've been through. Does your partner? Have you had serious conversations about your past experiences, your mental state, and your problems? IF this person really is "the one" then it's probably worth at least the attempt to bare your soul, share your challenges, and try to come up with a way forward that works for you both.

But what do I know? Maybe your partner is abusive toward you but you want him to be The One. Or maybe you are not capable of a trusting relationship. If you feel like you are too damaged to have a real relationship then back off for now. End it, take the time to fix yourself, and then try again. If on the other hand you think you're ready, and you think he's Good, and you think he will be healthful for you, then be painfully honest, talk it out, and try to find a way forward.