I don't think you have to give back your vows so you can focus on sutra. I don't think it is really that hard to keep your samayas. Especially given the fact that unlike the vows samayas can be repaired. Yes just like the story of Atisha I think says it it is almost impossible to keep them and you have to purify everyday, but that is not that horrible. Also it is really hard to really .. ef up if I am correct. It is very much enough to do Guru Yoga and Vajrasattva meditation daily and do your best during the post meditation. It can take together I dont know ... 45 minutes both practices? It can be quite a lot, but it still isn't hours and it is very possible to do. Plus it pushes you forward. I understand the feeling that one wants to focus on sutra. I always wanted to be a zennie, but here I am trying to do my ngöndro and lovin' it. Sutra can be way simpler than vajrayana. In vajrayana there is a huge amount of practices and bilions of ways how to go wrong. But at the same time I do not think that it cannot be simple. One can do just guruyoga and it can lead to enlightenment in a single life. That is very simple. If you want it simpler then keep samayas as pure as possible and you get enlightenment in 16 lifetimes. It can get restrictive but restriction is what brings some of the best pieces of art... I know that one size doesn't fit all but I find myself thriving in this restriction.
But you do you, man. I do not know your reasons or your story Sorry for this long ramble. I just wanna say that finding ways how to keep samayas can be easier than giving them back. The process of keeping them is also pushing you forward. And most importantly the point of this whole thing is do not throw away the baby with the water and to paraphrase a certain wise young lady named after an US state: you can get the best of both worlds, chill it out and take it slow.