Ethernet – Flow control in networks

Of course you can directly interact with the IP layer. The question is not about "why you can't", but about "why you wouldn't want/need to".

Since you mention higher level languages, the reason why they exist is the same as the reason the transport layer exists. A higher level language makes things easier by providing abstraction. You can code your application in an assembly language, but you will be spending a lot of effort on problems that have been hidden by higher level languages.

Most applications will benefit from a network which provides features like process identification, reliable communication, in order delivery or flow control. So it makes sense implementing these features once in a standardized protocol, instead of having to include custom solutions in every application.

There are situations in which designing your own solution can be beneficial or even necessary, but in most cases using TCP or UDP will just work and save you a lot of hassle. Doing it yourself can also be very educational and make you appreciate the work that has been done by others.

Is this a fluke?

Probably.

does it mean that I need to increase my buffers?

You disabled flow-control, so this would've made the problem worse if it were a buffer overflow issue. Error counters should tell you if there were any issues.

Flow Control is a negotiated function. If the switch has it "off" (send and receive), then the connected devices should disable it as well. Of course, this assumes your nics, drivers, and switches are braindamaged.