Electronic – Has anyone seen a Micro-USB socket which installs with the socket perpendicular to the PCB

The Sparkfun Eagle Library (pre-6.0, 6.0+) has one. It's called "USBPCB".

Here's a project using it.

If this is for production access only, then here's a trick I used once. The PCB (standard 0.062" thickness) was designed with SMT pads along one edge (both sides) so that a 2×7 0.100" JTAG header could be soldered to it, with the pins sticking out from the edge. This was useful for debugging prototypes, which needed more or less constant access to the JTAG (and weren't in cases).

In production, the connector is omitted, and I used a slightly modified DIP clip to access the pads for the connector. I just needed to bend the contacts inward a bit to get adequate friction and contact pressure.

Instead of the DIP clip, you could use a standard card edge connector as well.