Greetings,
First off, FET current ratings don’t mean what they might seem to mean. This post walks through that. Your immediate issues probably don’t lie in that area, but once you solve those immediate issues, it’s a topic you which you may find yourself returning.
This sounds like a classic “put it all together, flipped the switch, and it went poof” situation. It’s a common milestone in the skills development journey, and most folks usually need to repeat the lesson a number of times before learning that one should expect any new design to have faults.
Sometimes these faults are in construction; a rogue solder blob somewhere, wrong component placed, part placed backwards, etc.
Sometimes they’re simple oversights of implementation; lack of attention to polygon fills for example, have resulted in unintentional connections between nets on more than one occasion. A person can also make unintended connections by bad routing choices; the segment of the layout highlighted below seems rather suspect…
And other times the faults are a result of poor design, which is an inevitable part of producing good (or better) ones.
Things to consider here along the latter lines would be protection for the FET gates against over-voltage, and whether the drive waveforms are suitable for the task. R(1,4,5) are shown in the schematic having a rather high value of 100Ω; that’s going to slow the high-side transition times substantially, possibly to an extent that would allow for shoot-through on any given leg of the bridge. The gate drive traces also seem rather scrawny. A person can get away with that sort of thing with flea-power FETs, but as scale and speed increase the impedance of the gate drive network needs to be kept low, in order to provide adequate current to switch the FET in a timely manner.