The problem with ion thrusters is not just that they require an absurd amount of power, but they require electrical power, and equipment to transfer that power from electrical machinery to the exhaust plasma. So you not only need ~40 gigawatts of electrical power to produce a single SpaceX Raptor's worth of thrust, you need to convert that power from whatever your power source is with many gigawatts of waste heat from conversion losses, and transmit it all to and through your engines via electrical conductors and more power conversion/regulation machinery, then the engines themselves are only around 60% efficient so you've got another 16 or so gigawatts of waste heat to get rid of.
So, your ion thruster ship is evaporating due to the waste heat it produces long before it's producing true continuous-acceleration torchship levels of thrust.
The specifics are of course unknown, but a real fusion torch drive would operate much more like a conventional chemical engine, generating the power by fusion within the plasma being used as reaction mass and using magnetic fields to direct that plasma in the desired direction for thrust without making any effort to convert or handle it in any other form. The machinery would instead be designed to absorb as little of the power output as possible, and may incorporate regenerative cooling similar to chemical engines, which are already able to operate at quite extreme power outputs.