I've brewed up a device loosely based on USBKEY, and it works. Mostly. In other words not well enough.
Linux' driver treats it as CDC-ACM, which it is, gives me a proper device node, and I can to a few tens of millions of request-response operations, perfect. Except for when I try to plug it into an SMSC2602 hub, then the descriptors cannot be read. With SMSC2502 it only works on the first port. With a NEC (also USB2-HS-TT) hub it works perfectly on all four ports, and it works in the 4 onboard USB2 ports.
Once in a while it works on the 2602 hub - there are three modes: perfect, intermittent packet loss, and no comms at all.
In Windows XP it mostly works in all the onboard ports - though once every ~300k* exchanges a frame gets queued and sent on the USB side, only it looks like usbser.sys throws it away, it is never passed to userspace (the frame is not full so I do not send a null frame after it).
If I use the NEC hub that works so well in Linux, one in 16k packets are lost. With the SMSC hubs it's every twentieth packet.
Okay, so it must be my firmware or layout, I think. If I take a virgin USBKEY and plug it into the 2602 hub, then try to copy the files off the USBKEY, after a few seconds the leds flash red and the USBKEY disconnects.
I've inspected the USB signal lines for poor SI, tried extra pulldown on the wire that shouldn't need one, added 47pF caps to see if the extra load would change anything, with no results.
Does the above make anyone think 'haha, I had the exact same problem and it's soo easy to fix' - then I'm willing to beg, and provide cold beer if we ever meet in person.
* per 5 million if the machine is idle, per 300k when it's showing nasatv