In my own experience it is more difficult for someone who has been at a high level of fitness to completely let go. As much as I would have wanted to allow this to happen, the physiology of my body being pushed to such physical limits on a daily basis made this way harder. I experienced quite a few exercise withdrawal symptoms so to speak. But my advice would be maybe reduce your activity and eat more. However do not stop completely. Also there are quite a few articles about the effects of suddenly stopping exercise among athletes. That being said this is based on my own experience and you may find it works differently for you.