I would imagine after he started showing signs of heat on the path of application.
Well if the oral tradition is correct it would have been a time analogous to when Milarepa was joined by Tseringma and her four sisters.
Peter Alan Roberts notes that the later Kagyu tradition scrubbed the names of Milarepa's consorts. He also points out that sexual yogas were such an important part of Milarepa's transmission that Rechungpa refused to teach monks.
Rechungpa's lineage which includes this stuff when to Drukpa Kagyu, AFAIK. You can get these docs and read them. Known as "Rechungpa's Hearing Lineage."
The problem I see with the path of desire is lineage. I don't think folks have lineage that combines the path of desire with the lineage of blessings and the lineage of ultimate realization. At least, not in the Kagyu/Gelukpa/Sakya world. IMHO. My information is that the Jonanpa's have kept all these lineages alive in their Kalachakra tradition.
Many people don't realize that just because cool practice X is done somewhere (which is just the teaching lineage), doesn't mean that teaching lineage is combined with the lineages of practice blessings and ultimate realization. These three have to be combined in one teacher or the practice has no power any more.
For the most part, the sarma path of desire is a dead lineage. The Dzogchen practice with sexual contact is a different animal. IMHO. The sarma lineages of blessing and ultimate realization are a dying animal. You can only look to Drukpa, Drikung and to some extent Karma Kagyu to get a real full fledged sarma lineage of transmission, blessing and realization. This cannot be overstated enough. Again, Dzogchen is a different animal.
But... What the Kagyus have done with the path of liberation is most excellent indeed. The traditions of instruction, blessing and realization are very fresh and high potency. Especially with respect to Drukpa and Drikung, they are first and foremost retreat lineages. There are so many Milarepa-like yogis over there who don't have much scholarly knowledge, but whose realization is very high level and pure.
Look what happened to Rechungpa's lineage. It's basically dead too. Rechungpa did not have the karmic power to create something vast and long lasting, even though he spent his whole life with Milarepa. Gampopa who was with Mila for only one year, went on to establish enormous lineages which is awesome. So the Kagyu position makes sense, that Mila foresaw that Rechungpa wasn't going to be a big guru, and gave his lineage of blessing and ultimate realization to Gampopa who was peerless in his morality, trust and faith. Because he was a monk, that base gave him the credibility in the world make Kagyu huge, while also preserving the power the makes Buddhas. It all makes sense because Kagyu was connected always with ex-Nalanda abbots. It's only natural for that institutional energy to carry on.
Having said that, if I were to look to the path of desire I would go Dzogchen, because all the lineages are complete there. Guru Rinpoche is the real guru of the path of desire and his method of hiding treasures for his reincarnated disciples to find in the future is an amazing way to keep it all going. Guru Rinpoche, mahasiddhas etc. were out in the wild. Their totally mind-blowing systems flow from that origin.
Besides, the path of liberation is the best path anyway. It really is the most secure, fast and easy way to get to the end.