Announcement: New open-source USB library for STM32F103xx MCUs

[thanks4opensource]

I have just released a new 1.2.0 update to my open-source USB library for STM32F103xx MCUs:

https://github.com/thanks4opensource/papoon_usb

papoon_usb is a lightweight, efficient, cleanly-designed library for the USB Device peripheral in the STMF103xx (and similar) MCUs. Its API makes USB very easy to use from application code. It comes with four USB device class implementations -- USB-CDC, USB-MIDI, USB-HID, and a custom minimal data class -- and new USB classes are similarly easy to implement (basically just add the appropriate USB descriptors).

The source code is released under the GNU General Public License (GPL).

The library is written in C++ using updated versions of my "regbits" and "regbits_stm" development frameworks, which are included in the papoon_usb repository. There are no external dependencies. Usage of regbits in the library is transparent, and applications need not use it (although they would benefit from doing so). Additionally, examples of C wrappers to the library are included in case linkage with pure C code is needed for some reason.

Fair notice: The README documentation contains a large number of highly critical, opinionated statements regarding the design of the USB standards and of ST's hardware and particularly software (HAL) implementations of them. Reading, much less agreeing with, them isn't required for use of the library, but they are probably unavoidable if browsing the document to understand how to use the library. Those easily offended may want to stick with the documentation in the header files and the example application source code provided, although the README does contain important library usage and design background information.

Feedback in the form of comments, bug reports, improvements, and disagreements with the opinions expressed above are welcome, either here on community.st or as "Issue" reports on GitHub. (Note that this statement should not be considered a guarantee of support or fixes.) At some point in the future I may write a similar library for the very different USB peripheral implementations in the STM32F7, etc. product lines -- again, not a promise. Administrators: Feel free to move this post to a different sub-forum as appropriate, or remove it entirely given the descriptions of criticisms described above.

[waclawek.jan]

Thanks.

Yes, I wouldn't use it because of C++ (I won't use anyway it as I have mine =) ).

efficient, cleanly-designed library

Oh yes, sure. Each one is... ;)

A few remarks commenting your rants. But, don't get me wrong, I genuinely do hate USB and do agree it's a mess. (You for example haven't even mentioned terminology used in USB realm, which is at too many places inconsistent and too often incomprehensible. For example, the commonly used term "device" in fact means a group of "not-host" items, and that consists of "hub" and "function". Linux world tops this with "gadget").

Nonetheless, I tend to understand, why - it's not all that simple as it appears from the viewpoint of status; it was certainly a laborious and complex process to get to it. Surely, detailed explanation of many of the features of USB would make much more sense would they be given a historical perspective. Yes, the result is nasty, but in all honesty, I personally wouldn't probably make it much better (see, I have a high opinion on my abilities). And at least the hardware and lowest-level protocol definitions appear to be sound and solid (those are normally screened from the user by hardware which I did not have to design, so I may be wrong).

As for the details:

Inconsistencies and strange design decisions abound in USB, above and beyond the lack of layering. The basic design sends device description to the host via a small "Device Descriptor" and a large "Configuration Descriptor" containing multiple sub-descriptors (plus optional standalone "String Descriptors"). This is already two mechanisms devoted to the same task -- the host retrieves descriptors by type and index, but also by parsing the Configuration Descriptor for its components. (Possibly this was done for efficiency.)

No. the idea is layering here too. You can have several configurations in a single device, and the mechanism to select from them is built into the protocol. I know of no host/OS which would have a ready-made default implementation for this (of course custom drivers/applications can do that). This is not a nonsense - for example, you may want to have your STLink in one configuration providing your debug and virtual-serial and virtual-disk interfaces; and in other configuration providing one DFU interface for update (I don't know how STLink actually implements this).

Actually, for HS-capable devices, you have to have two configurations for HS and for FS - they may be implemented as the same configuration, but they may be different, too. See USB2.0 9.6.4 Other_Speed_Configuration. The selection between them is implicit, though (given by the connection speed detected by hardware), in contrast with the multiple configurations I mentioned above.

The fact, glossed over in the USB-IF standard and completely omitted in ST's documentation, is that setting the device's address must be delayed until the next transfer on the bus occurs. (Trust me. I spent many fine debugging hours discovering this fact.) It is these types of crucial details that make USB so difficult.

I feel your pain and I've struggled with this, too. However, this falls into the unfortunate Oh, we all know this, it goes without saying, it follows from the usual going of things and Es ist Selbstverstandlich cathegory of things. The issue is, that (and I won't use the official terminology as I've forgotten it merrily and won't look it up again now) the Control channel's "setup process" consists of two or three transactions (and this means quite a couple of packets going forth and back) (yes, this is weird in some way, too, and it's not documented in the cleanest possible way either) and, quite understandably, all of them have to contain the same address. As you receive the new address in the middle of the "setup process", you have to use the "old address" until this one "setup process"'s last "acknowledge" transaction ends.

There is no reason for the hub to communicate anything on the cable other than information intended for that device, so why is an "address" needed?

Again, historical perspective would help here. The very first hubs were barely more than passive redrivers. It means, that all devices on the tree see all the communication, and that requires to have the addressing in place. The original specs was only Full Speed, Low Speed (which genuinely requires logic in the hub, albeit still just quite rudimentary, see PRE PID in the specs) came as an afterthought. And even then, should hubs steer traffic to individual ports fully selectively, they would need to perform on-line protocol decoding and some buffering/retransmission, and that's quite a bit of logic. Remember, roots of USB go back to the early 90s and prices were not in microcents per transistor as they are now.

I won't defend the ST's USB module, but:

...primitive and lacking in features/capabilities... Supporting evidence for these claims is the fact that ST started using a vastly improved USB peripheral in MCUs following the STM32F10x series.

Ah, you'd wish.

There are 2 USB incarnations in the STM32 - the device-only (in 'F100-103, 'F0, 'F3, 'L0, 'L1, lower-end 'L4), which you've encountered; and an OTG (i.e. both Host and Device capable, in 'F105-107, higher-end 'L4, 'F2, 'F4, 'F7, 'H7), which is an IP purchased from Synopsys and can be found in chips of various makes around (keyword is Designware 2, and one of the incarnations is to be found in the older RaspberryPis, among others).

And, that one is a pure hell, I can assure you.

Casual research seems to indicate that it was carried over from the smaller STM8 8-bit MCUs.

I'm not sure. I am not familiar with older ST mcu offerings, but I'd say this is a 16-bit module in its roots; in particular because of this.

"Toggle-only" bits

Yes, this is an idiotic design; however, I believe they can be handled safely. We may discuss this elsewhere.

JW

[thanks4opensource]

Thanks, JW. Even though I disagree with you on some points, I really appreciate that you obviously read, thought about, and intelligently replied to my "rants" (I hadn't thought about them in those terms, but, yes, I'll accept the label. ;) )

> efficient, cleanly-designed library

> Oh yes, sure. Each one is... ;)

We all love our children. ;)

> You for example haven't even mentioned terminology used in USB realm, which is at too many places inconsistent and

> too often incomprehensible.

Oh, yes. :( Even the host-centric "IN" and "OUT" monikers which require continuous mental jui-jitsu when working on the "device" side. (Why not "UP" and "DOWN". Except for the disaster that's OTG these would be crystal clear when though about from either end.)

> Yes, the result is nasty, but in all honesty, I personally wouldn't probably make it much better (see, I have a high opinion

> on my abilities).

https://github.com/thanks4opensource/tri2b-quad4me -- tri2b and quad4me: clockless arbitrated bit-level serial protocols

If you have time to waste read my rants there, including the fact that the code exists only because nobody I've found, ST in particular, actually implements I2C multi-master arbitration correctly.

USB? Asymmetric host/hub vs device architecture? Polled? Minimum worst-case latency of 1ms for full speed, 125 us for high? I'd give anything to dump it, but it's unavoidable for getting data into and out of a PC (WiFi/Bluetooth are worse, and ethernet's PHY requirements are too demanding for low-end SOCs).

> And at least the hardware and lowest-level protocol definitions appear to be sound and solid (those are normally screened

> from the user by hardware which I did not have to design, so I may be wrong).

I'm not as qualified to comment on them as I am on the software layer. They do seem fine from what I've investigated, and as you say the ST peripheral does largely do a good job of shielding software from them.

> No. the idea is layering here too.

I have no problem with multiple configurations -- in fact I think it's a great idea. The inclusion of both multiple interfaces and multiple configurations is a bit of overkill, as evidenced by the fact that (I've read) very few devices implement multiple interfaces.

My objection is in how the protocol walks all over the layering. CDC-ACM's SET_LINE_CODING and GET_LINE_CODING are obviously class-specific, yet they come in as STANDARD, not CLASS requests. And on the control endpoint instead of the out-of-band ACM endpoint which, as near as I observed, isn't used for anything.

Many other such examples, including the various descriptor layouts, but my brain turned to mush long ago just accepting and coding around them, so I'll have to come up with some later.

I still contend that a rational design would build up from simple capabilities and layer the more complex requirements on top of them. C++ base and derived classes, but you say you don't like C++. ;) If you can't do "hello, world" in a design I don't trust it to implement an operating system.

> the Control channel's "setup process" consists of two or three transactions

Yes, your very clear explanation is what I eventually came to understand. Where were you when I was beating my head against the wall trying to implement this? ;)

And thanks for the history lesson on dumb early hubs. I'd suspected something similar. Doesn't change how much I hate the design. Even after online translation I'm not sure I understand Es ist Selbstverstandlich but I long ago realized "you can't fight City Hall" when it comes to USB. Just deal with it.

> There are 2 USB incarnations in the STM32 - the device-only (in 'F100-103, 'F0, 'F3, 'L0, 'L1, lower-end 'L4), which you've

> encountered; and an OTG (i.e. both Host and Device capable, in 'F105-107, higher-end 'L4, 'F2, 'F4, 'F7, 'H7), which is an

> IP purchased from Synopsys and can be found in chips of various makes around (keyword is Designware 2, and one of

> the incarnations is to be found in the older RaspberryPis, among others). And, that one is a pure hell, I can assure you.

Oh, lovely. :( I've glanced at the F7 Reference Manual and saw that the USB was completely different, that there was no hope of "porting" papoon_usb to it. I'd hoped that with the FIFOs, etc. it meant that more of the protocol had been moved into hardware and the software driver would in turn be easier. Not so? Maybe I'll have to find another solution instead of spending more months writing code when the time comes for the greater processing power the newer chips provide.

Was it you who I've seen posts from before regarding ST's purchase of Synopsys IP? I very early on made up my mind that the older ST USB library, and a large part of the newer HAL one, were ported from third-party code by junior-level developers who never understood it and were given minimal schedule time to "just make it work" by management. Once again, I never wanted to write any of this code, but I looked at implementing my simple custom communication class (I don't need CDC-ACM baud rate and parity bits control to send bytes between a PC and an STM MCU -- layering, again) in the existing libraries and threw up my hands in frustration.

> I'm not sure. I am not familiar with older ST mcu offerings, but I'd say this is a 16-bit module in its roots; in particular because of this.

Not important, but your link is broken. Not your fault: It's almost impossible to correctly format a link (or anything else) on this site's crappy forum software.

Thanks again for taking the time to comment on my work.

[thanks4opensource]

> You for example haven't even mentioned terminology used in USB realm, which is at too many places inconsistent and

> too often incomprehensible.

Forgot my favorite example: Can USB endpoints be bidirectional (other than the default control endpoint zero which is implicitly so) or are they only/always unidirectional?

I don't have exact statistics, but I'm convinced that online documentation is split 50-50 on the topic, with half definitively stating they're only unidirectional and the other half bidirectional, each with equal conviction. And it's kind of an important distinction to get right. After much confusion, experimentation, and reverse engineering I finally realized it's mostly a matter of semantics: If a USB descriptor has one endpoint with a bEndpointAddress byte of 0x84, i.e. endpoint number 4 with the high bit set indicating it's an IN endpoint, and another 0x04 (OUT endpoint, also number 4), are those two different unidirectional endpoints or one bidirectional endpoint?

Note that ST's RM008 documentation uses the bidirectional nomenclature, which make more sense and matches the hardware. A single USB_EPnR register, containing a single 4-bit-wide endpoint number in its EA field can have one or both of its STAT_RX and STAT_TX fields set to non-DISABLED thus making it an OUT/RX (here we go again, IN/OUT/RX/TX, choose your terms) endpoint or an IN/TX ,or both. USB is complex enough already, but idiocy like this (it's either idiotic or purposely obfuscated, take your pick) makes it even more difficult.

And although I've confirmed that bidirectional (aka double unidirectional) endpoints work, I've seen a large performance difference between them and having two different unidirectional endpoint numbers, on for IN and one for OUT. I don't know if this is a false result of my testing, a problem with my library, the fault of the ST USB hardware, an inherent limitation of the USB, standard, or what. If you or anyone else has insights to share I'd appreciate hearing them.

[waclawek.jan]

Mark,

When it comes to USB it's hard to avoid ranting, but let's try to stick to the technicalities.

> "IN" and "OUT" monikers which require continuous mental jui-jitsu

+1

OTOH, Up and Down *is* used, although in a slightly different way, as "upstream/downstream facing ports" in the HUB section. It denotes the hierarchy relationship in the tree, rather than the momentarily flow of data.

(But I do use Up and Down, as well as Rx and Tx, liberally, as I see fit given circumstances).

> [better]

Oh yes, of course. Yes we need to stick to standards as interoperability is a key to future. Yes the problem is the standard is defined poorly, so all parties struggle in implementing it interoperably. It's usually the devices implementers who have to match quirks of given host implementations (usually in "big name" OS).

In many respects, at the higher levels, USB (and that includes the "well-known" classes) is only a very loosely and widely defined framework, of which implementers (mostly OS implementers) inevitably have to pick cherries to implement. That then creates another level of "defacto" standards, which is even "poorlier" defined and even harder to match. (A good example is the Audio Class, which was one of the very first classes to be implemented (which is visible on some idiosyncracies in the details which carried over from those early times), was given quite a bit of care, brought quite some features into USB itself; yet to bring OS/host and device implementers somewhat more together, they needed to amend it with a "Basic Audio Devices" document...)

> [Synopsys OTG IP]

> more of the protocol had been moved into hardware

Yes.

> and the software driver would in turn be easier

No.

It is a nightmare. Yes, there are layers, but they are not defined very well, and they have many internal crossings. There are many obscurities, and many historical issues. You don't want to depart from implementations known to work, as you are likely to trigger some internal quirk in the hardware, which may quite well be unknown to the authors (and is most certainly unknown to ST).

"Documentation" is an ST-adjusted Synopsys boilerplate (you can find it across other makers who use the same module, sometimes even in the rough unmodified form, where you can see some of the options that can be set when instatiating the IP, which in turn explains some of the otherwise unexplicable wording in ST's version, which attempted to screen out the configurable stuff, but that's obviously hard to do perfectly), which again has its historical issues, was probably never adequate and just got worse during times. ST uses several different versions of the IP in the various STM32 models and ST does not acknowledge that, does not provide guidelines and difference documents, ST even does not acknowledge the existence of version register, even if ST uses it in its code.

Would we have a real forum software here, you could've easily traced which part of it was I working on, simply by following my rants here. At ST, I am already known as a ranter, and particularly Synopsys OTG ranter (e.g. "I remember your post about USB documentation" here).

But maybe it's just me. Obviously there are several software implementations using the module out there, free and paid (for the various chips, including Linux); and they all coped.

> CDC-ACM's SET_LINE_CODING and GET_LINE_CODING are obviously class-specific, yet they come in as STANDARD, not CLASS requests.

> And on the control endpoint instead of the out-of-band ACM endpoint which, as near as I observed, isn't used for anything.

CDC ACM is meant for modems. That's in itself a very wide topic, but in its rudimentary form, it means two data streams (in "standard RS232" parlance it would be Rx and Tx (and btw., that again is the ever-confusing Up/Down paradigm which can easily blow one's mind if it comes to the intermediaries as modems are) and two control streams.

The control endpoint is used to convey upstream control signals - in modem terminology, CTS, DSR and RI.

The downstream control signals - RTS and DTR - are conveyed through Endpoint 0, i.e. the Control endpoint. That may be confusing, but the reason is to spare endpoints (see below). The upstream endpoint can't be spared, given asymmetric polled, as EP0 is not suitable to be polled (it's bus bandwidth- and software-expensive).

The painful task to implement requests/handling for these signals even if you don't need them may seem to be superfluous for the now usual usage of CDC ACM as a generic data stream. But that again boils down to the very wide definition in the USB Class document, the choices the Host/OS implementers made (they may have simply chosen to omit the requirement for the control signals - maybe that wouldn't be a completely conforming implementation but I believe this could be solved in a working manner), and the fact that the humble device implementer has to match whatever these choices were, unless he wants to go the even more painful way of custom drivers.

> [bidirectional endpoints]

No, endpoints in USB are strictly unidirectional. Yes, this is nowhere said out loudly enough, AFAIK, and there's too much confusing information out there. The implementation of device-only ST IP is an example of such a false hint.

Now again there's an exception, EP0 is in fact bidirectional.

As there's quite a lot of hardware to handle individual endpoints, they are considered expensive, witnessed by the fact that even today in the era of supercheap transistors, the full 16/16 implementation is far from being commonplace (and I know of no such in STM32 - maybe in H7? I didn't check). That's why protocols where there's a low-latency low-bandwidth requirement for downstream signaling tend to wrap these into EP0's requests.

> [broken link]

I doubt anybody who can't use such a broken link would get USB working... ;)

But I fixed it (for those who wonder: it started with "http://https//" but the rest was OK).

> [papoon]

On a local discussion board, besides C++, GPL was also mentioned as the main drawback.

Jan

[S.Ma]

And GPL license vs MIT, BSD or similar maybe a detterrent for most commercial projects.