I’d gently lift an eyebrow toward characterizations of magnetic transformers as “inefficient” but could easily understand how they might permit ugliness on the power grid to pass through and cause harm to the relatively fragile electronics in a drop-in LED replacement lamp.
If I’m interpreting correctly (not always the case) you’re speaking here of deleting the voltage conversion at the fixture level, performing the 120->12 conversion between the AC line and your existing switches, and repurposing the existing 120V wiring between switch and fixture as low-voltage wiring.
Could such a scheme work? Likely so. Would you run afoul of local regulations and electrical codes in so doing? Probably. Could you do it anyway without gross risk to persons and property? Maybe. The devil’s in the details.