Electronic – TriState Connection to Atmega328 and FT232R

I have a custom circuit board containing an ATMEGA328 and an FT232R for bus interface via the RX/TX pins on the ATMEGA. I also have an off board serial connection that I would like to access the ATMEGA328. Basically, I want to be able to specify using either the off board serial interface or the FT232 serial interface to the ATMEGA. These would never attempt to access the ATMEGA RX/TX pins at the same time.

The solution I came up with was to use a tristate buffer to either connect or disconnect the off board serial connections. The FT232R is always connected to the ATMEGA328.

Here's the problem. Everything works fine if I don't add the tristate buffer to the board. As soon as I add the tristate buffer, the ATMEGA heats up and the RX/TX interface to the FT232 becomes inoperable. This occurs even with the off board RX/TX connections being left floating and with the tristate enable pins pulled high through a 10K resistor (which should set the NL27WZ125 dual tristate buffer outputs to a high Z state).

I figure I am missing something about correct use of the tristate buffer, but I don't know what it is. When the enable pins are pulled high, the tristate outputs should be in a high impedance state and shouldn't affect the circuit at all.

I have triple checked all on board connections, and again, everything works fine until I physically solder the tristate buffer chip to the board.

Here's a simplified schematic (wish I had enough points to post the actual schematic as a picture).

^{simulate this circuit – Schematic created using CircuitLab}

Here's the initial Eagle Schematic (sorry for the incorrect schematic from earlier):

I also realized that the RX buffer was connected in the wrong direction, so I think it should go this way:

However, I don't see how the initial Eagle Schematic should cause the problem described in the original question.