USB-C – Replacing 4pin USB with USB-C Works Only in One Direction

I have an AR1100 with a usb mini-B female connector :
https://github.com/adafruit/Adafruit-AR1100-Resistive-Touch-Controller-PCB

Basically 4 usual USB pins : GND, D+, D-, +5V

I want to replace it with a usb-c connector.
At first I put a pull-down 5.1K resistor to GND on CC1 (A5) and CC2 (B5) pins. It didn't work at all.

I understood my mystake thanks to this topic, which explains if I want to make USB-C to USB-A device then I have to put a 5.1K pull-down resistor, but if I want to make a Type-A HOST (AR1100 IC : GND, +5V, D+ and D- signals) to Type-C device (my USB-C connector) I have to put a 56K pull-up resistor:
USB-A <-> USB-C adaptor

Then I replace the 5.1K pull down resistor with a 56K pull up to VBus and verified my schematic thanks to this topic: usbC_to_usbA. This time it works but only in on direction. Not the reverse.

After some thinking I came out with this :In one direction, D+ is gonna be connected to DP1 (A6) and D- to DN1 (A7).
But if I reverse my cable what's happen ? Then D+ is gonna be connected to DP2 (B6) and D- is gonna be connected to DN2 (B7) ? And because I left them unconnected that's why my signal doesn't work on reverse side ?

Here is my USB-C my board layout if needed:

EDIT:

See EDIT2, This assumption is probably wrong.
@Finbarr Vbus would be more correct if I'm doing something like this ?
Schema is coming from the official AR1100 schematic with original mini-b USB connector. If not would you please provide me correct layout for Vbus as I would like to use it in my project. I still have huge doubts as I'm still connecting Vbus to 5V here.

EDIT2:

This webpage Guide to USB-C Pinout and Features mention "The VBUS and GND pins are power and the return paths for the signals. The default VBUS voltage is 5 V but the standard allows the devices to negotiate and choose a VBUS voltage other than the default value. The Power Delivery allows VBUS to have a voltage up to 20 V. "
So I'm unsure how should I set up my VBUS pins.

This topic : Vbus line dfp clearly says "By Type-C specifications, a Type-C port should not supply any power until CC pins signal any connection, and power roles of connected partners are identified via Rp/Rd sense mechanism.You need a high-side power switch to turn VBUS on, which should be controlled by this "ID" pin".

So it seems the way, add a high-side power switch to turn VBUS on controlled by ID pin.