There is nothing in the USB spec that forces a host to identify itself, and there is no standard way a device can request such information. Without a cooperating driver in the operating system, there is no way for a ordinary device to know details about the host, like what operating system it is running, what processor it is using, etc.
I have, however, noticed that different operating systems tend to have their own quirks about what they request in what order during enumeration. You might be able to do some testing and find a way to distinguish a few of the major operating systems. However, that would not be guaranteed, could easily change at minor revisions, and can certainly be deliberately spoofed.
If you want a device to know it is connected to a trusted host, you will have to previously load software on that host. The device could, for example, send out a randomly generated string of bits when asked the right way by your software on the host. The host must then respond with a reply that is hashed from the original bits in a particular way envolving a password. Only a host with the right algorithm and password could reply correctly.
Of course that is just simple scheme. It can get a lot more complicated depending on what level of attacker you want to protect youself from. For example, the simple scheme I described above would be no good against a man in the middle attack.
I think you should be able to detect it quite reliably by knowing the electrical properties of the pins. USB hosts pull the DP and DN data lines low with 15kOhm resistance. USB devices pull the DP line (DN line if it's a slow speed device) to 3 volts with a 1.5K resistance. If the device at the other end pulls up and also you pull up, nothing would happen because the device doesn't react to the extra pull-up. So you could use this to check if the other end starts to send packets to the bus.
I suggest a procedure something like this:
You could start by pulling the DN line weakly high (with something like 200K to 1M resistor). You could detect a pull-down on the DN line to give a first hint that there's a host at the other side.
Then measure the VBUS, if it's high, it's a further indication that there is a host at the other side. If so, activate a 1.5k pull-up (to 3 volts) on the DP line and listen for incoming packets. If there actually is a host on the other side, it will issue a bus reset and send a GET_DESCRIPTOR packet to the bus. If this happens, you know that the other chip is a host. I suggest to do it in this order as if there's a device at the other side, it will not react to the DP pull-up.
If nothing happens, there may be a device at the other side. Release your DP pull-up and see if it's still pulled up at the other side. If that is so, then there is probably a self-powered device at the other side. If not, then start driving the VBUS line to 5 volts. At this time at the latest you should see the device's 1.5k pull-up tp 3 volts on the DP line. If you see instead 3 volts in the DN line, then there's a slow speed device such as a keyboard or a mouse at the other side.
Now act as a host and issue a BUS RESET and GET DESCRIPTOR : DEVICE DESCRIPTOR to address 0, endpoint 0. The device should now answer and finally at this point you know for sure that there is a usb device at the other end. If you haven't enabled your 5V drive to VBUS so far, I suggest that you enable it now. Test with all the devices you intend to support.
Ok, this is a hack, and the official bus doesn't work this way because if both ends would work like this then it would be a matter of luck (timing) if it detects correctly or not. But if the other end is a well-behaving USB entity, then I think that it should work. I've done a fair bit of work on the USB, but not this exact thing.
Best Answer
USB host can't talk to another USB host.
You seem to have a design constraint to have a Type-A receptacle. Understandable, to read a USB key. But change the data role in your design will be problematic then.
You can start with your design having Type-A receptacle and set the OTG_FS_ID low. It will be a legitimate USB host.
However, for your design to be a USB device, you can't use Type-A receptacle, it must be Type-B receptacle.
What you can do is to add another Type-B connector, and de-multiplex USB data to it using a dedicated IC USB switch, because the D+/D- data lines are already muxed inside the MCU.
Since Type-A nor Type-B standard connectors don't have ID pin, the switch can be done by manually selecting the mode (set OTG_FS_ID low or high). The direction of data switch should correspond to the position of manual OTG switch.
Alternatively, you can use VBUS=high signal on Type-B port to make this switch for you. You will have then a priority of device function over host function if a stadard B - A cable is connected to a PC host. This all might be a heck of confusion to customers, but this is the price to go with old USB connectors.