My dietitian explained the reason the diet progresses so slow is because there are a lot of sutures holding your stomach (and intenstines) together and you want those raw edges to heal together without the obstruction of bits of food getting in their way irritating them. You want nice, smoothly healed suture lines. I don't think you've done irreversible damage from going off plan, but I agree with the others, I'd examine why you felt you needed to go off plan, and I'd return to it. If you feel you've healed faster than others and should advance, call your team and talk to them about it. They might let you advance sooner, or they might tell you to stay the course.
This surgery and the aftermath take a lot of willpower. We can choose to stay with the guidelines we've been given from our team, or we can do our own thing, but there will be consequences for each choice. If you bend the rules for earlier food progression, will you bend the rules when it comes to what you should eat as you begin to lose weight? Will you go off course throughout your weight loss phase because you don't feel like adhering to it? We all have those moments, it is understandable, we are human. But we have to be willing to confront ourselves when we do have those moments and look deeper into what is going on there that caused us to make those choices. Because one choice is one choice, but one choice made repeatedly is a pattern, and a pattern can either heal or hinder us.