Electronic – Confused about the XIP (eXecute In Place) function of QSPI FLASH

I vote for DMA. It's really flexible in Cortex-M3 and up - and you can do all kind of crazy things like automatically getting data from one place and outputing into another with specified rate or at some events without spending ANY CPU cycles. DMA is much more reliable.

But it might be quite hard to understand in details.

Another option is soft-cores on FPGA with hardware implementation of these tight things.

Another answer: Stop using interrupts.

People jump to use interrupts too easily. Personally, I rarely use them because they actually waste a lot of time, as you are discovering.

It's often possible to write a main loop which polls everything so rapidly that's it's latency is within spec, and very little time is wasted.

loop
{
    if (serial_bit_ready)
    {
        // shift serial bit into a byte
    }

    if (serial_byte_ready)
    {
        // decode serial data
    }

    if (enough_serial_bytes_available)
    {
        // more decoding
    }        

    if (usb_queue_not_empty)
    {
        // handle USB data
    }        
}

There might be some things in the loop which happen far more often than others. Perhaps the incoming bits for example, in which case, add more of those tests, so that more of the processor is dedicated to that task.

loop
{
    if (serial_bit_ready)
    {
        // shift serial bit into a byte
    }

    if (serial_byte_ready)
    {
        // decode serial data
    }

    if (serial_bit_ready)
    {
        // shift serial bit into a byte
    }

    if (enough_serial_bytes_available)
    {
        // more decoding
    }        

    if (serial_bit_ready)
    {
        // shift serial bit into a byte
    }

    if (usb_queue_not_empty)
    {
        // handle USB data
    }        
}

There might be some events for which the latency of this approach is too high. For example, you might need a very accurately timed event. In which case, have that event on interrupt, and have everything else in the loop.