Success in a similar endeavor with USB
At my work, we did a very similar thing.
We have a camera outputting serialized video data over a differential pair. Some of our cameras use custom flex PCB or flat flex cables for the connection. Flat flex cables can be shielded, and flex PCBs can be designed so that the traces have whatever impedance you need, and this worked well for us. We also had 3.3V and 1.8V power nets and an I2C bus running over the connection, so a custom cable was practically mandatory. Note that 3.3V power and raw serialized video data are not a USB bus.
However, in a newer revision, we had the challenge of routing the cable partially out of the enclosure (where FFC and flex PCBs aren't possible for durability reasons) and had an opportunity to use a new SERDES chip that embedded the I2C with the video data. We also added a SOT-6 1.8V LDO to the camera module, so we only needed to send 3.3V across the cable. Suddenly, (OK, so this was really planned) USB cables and connectors were ideal.
USB cables and connectors (like HDMI cables and connectors) are manufactured in enormous quantities. Economies of scale make them far cheaper than custom cabling and high-speed connectors. Our product passed FCC review, was approved by our legal department, met automotive qualifications, and has been deployed in tens of thousands of vehicles worldwide. It may be important to note that the USB cable is inaccessible to the consumer behind a screwed-down case, and is not removable is therefore invisible; there can be no argument that we're claiming compliance with the USB standard.
We're a major American company supplying all the major automotive companies; this was all above-board. There are other products (as BarsMonster mentioned) which use logo and trademark-sensitive interfaces for other purposes. You may be able to use this as evidence that it's acceptable in the absence of a lawyer's input.
Check with your legal team
Even given the existence of other products which do this, you should probably have your legal team or lawyer review the situation. The issue that will trip you up is probably (IANAL) trademark compliance. The letters "HMDI" and the logo:
(note: I believe this is fair use!) are trademarked. You can't manufacture products which use these logos and descriptions unless you're approved by the organization which holds the trademark. If your product isn't sold, or if the trademark is not visible, you're also exempt. The trademark usage guidelines are described on the HDMI website:
http://www.hdmi.org/learningcenter/trademark_logo_pub.aspx
and you'll be using the trademark on a non-compliant product as a non-member of the organization. For our USB system, no trademark was visible. For yours, the Name and Label Requirements for HDMI® Cables say:
All cables must be labeled, both on the cable itself and on the front of the cable packaging, with the appropriate Cable Name Logo.
There is some further description of labeling on the cable itself (which would make the trademark permanently visible on your product; it wasn't on ours), labeling with a wrap-around sticker (which you probably wouldn't be allowed to remove) and labeling on the connector moldings. If you get cables with labels only on the connector molding, you may be able to hide this inside your product, but then it can't be unplugged. I'm not sure if that will work for you.
The final answer in all questions like this is always "Ask a lawyer".
from http://www.hdmi.org/learningcenter/faq.aspx#94
Q. How will HDMI change the way we interface with our entertainment
systems? The most tangible and immediate way that HDMI changes the way
we interface with our components is in the set-up. One cable replaces
up to 11 analog cables, highly simplifying the setting up of a home
theater as well as supporting the aesthetics of new component design
with cable simplification.
Next, when the consumer turns on the HDMI-connected system, the video
is of higher quality since the signal has been neither compressed nor
converted from digital to analog and back.
Lastly, because of the two-way communication capabilities of HDMI,
components that are connected via HDMI constantly talk to each other
in the background, exchanging key profile information so that content
is sent in the best format without the user having to scroll through
set-up menus. The HDMI specification also includes the option for
manufacturers to include CEC functionality (Consumer Electronics
Control), a set of commands that utilizes HDMI’s two- way
communication to allow for single remote control of any CEC-enabled
devices connected with HDMI. For example, CEC includes one-touch play,
so that one touch of play on the DVD will trigger the necessary
commands over HDMI for the entire system to power on and
auto-configure itself to respond to the command. CEC has a variety of
common commands as part of its command set, and manufacturers who
implement CEC must do so in a way that ensures that these common
command sets interoperate amongst all devices, regardless of
manufacturer.
It's pretty clear that in the standard, they were trying to make sure that you would ALWAYS have the right cable when trying to assemble consumer-grade AV systems, and one way to do that is to have one cable style. Also, they want to play up the two-way communication features of HDMI.
Of course, that's speculation, but I can honestly assert that I've never cursed at HDMI connections anywhere near as loudly as I've cursed at USB connections.
Best Answer
High speed cables use shielding, twisted pairs and separation etc to mitigate intereference. Each standard has a maximum length that helps ensure that it can deal with exepected background noise over that distance. So, assuming that you are not using long cable runs, the connector you choose is a low loss type and you try and maintain as much of the original shielding/twists etc( eg do not expose 30cm of cable at your join) it will probably be ok. But until you make the connector you are probably not going to know.