I'd like to clarify how the configuration address space in PCI and PCIe works.
Namely, a PCI peripheral on a bus is addressed with device:function pair (the bus or domain:bus are not included, since only 1 bus is considered).
How is device address determined? What sets it?
According to the answer in this question on superuser and other texts the device address is actually address of the PCI slot, and thus it should be wired in the hardware during manufacturing. Is it correct?
Then, how is this address defined in PCIe case?
From the same answer:
each device has its own individual point-to-point serial connection to its upstream device
— thus, in PCIe each slot is connected to upstream switch device in star topology. And addresses of the slots are re-assigned by the switch on reset? (So, on power-up the switch calls for devices in each slot and if there is a response the switch assigns device address to the slot?) As in the same answer:
(hence each bridge, including the top-level "root complex", tells each device what its device ID will be)
Or they are also resolved in hardware/firmware of the switch? (And the switch always has a device address assigned to those wires going to a slot.)
It seems for easier hot-plug-ability each slot should have permanent address on the bus (on the "network" of the PCIe switch).
Best Answer
This is rather different between PCI and PCI express. However, the bus and device numbers are not explicitly configured. For PCI, the device number is determined by which AD line is tied to the IDSEL input. Hence the device number is determined by the physical slot in which the card is installed, determined by the board layout. See https://superuser.com/questions/1057421/how-is-the-device-determined-in-pci-enumeration-bus-device-function for more information.
For PCIe, what happens is that host software configures the switch ports to route traffic based on the bus and device numbers, then the target device captures the bus and device numbers from the PCIe configuration request TLPs. The switches themselves are dumb - they rely on host software to configure everything, and then route traffic based on the base/limit and bus number register settings. Each link is point-to-point and gets its own bus number. The device at the far end of a link always has device number 0. Inside of PCIe switches there is an emulated PCI bus, and each switch port will have its own device number. So the ports that correspond to each slot can have different device numbers, fixed in silicon by the switch manufacturer, but the devices installed in those slots will all have device number 0.
Since the device number always being zero is a bit of a waste, ARI was created for PCIe. When ARI is enabled, the device number gets rolled into the function number, enabling a single PCIe device to support up to 256 functions. This goes hand in hand with SR-IOV.
For hot-plugging, it is up to software to set aside both address space and bus numbers to support new devices in the hierarchy. PCIe is designed to be an extension of PCI and as a result hot-plug support is poor.