Running STM32CubeMX without GUI

[ANath.1]

Hi All,
I am trying to setup a build pipeline for the STM32 platform to be able to compile projects in an automated fashion. So far, I have an .ioc file which I use the to generate the project using the STM32CubeMX commands which are invoked via the script in the cli mode. The flow works on an Ubuntu machine which supports GUI. However, when I try to containerize the flow into a docker container, the build breaks as the STM32CubeMX tries to render a progress bar for the Loading IOC step followed by a Rendering UI progress bar. It is not possible to load display components on the docker and hence STM32CubeMX code generation fails. Is there any way to launch STM32CubeMX in a headless non-gui mode so that we can run this on headless machines?

This is the UI that pops up.
The following is the contents of my script file

config load <filename>.ioc
project name app
project toolchain STM32CubeIDE
project path <path>
SetCopyLibrary copy all
SetStructure Advanced
project generate
exit

 Command to invoke STM32CubeMX:

java -jar <path_to_CubeMX>/STM32CubeMX -q <scriptfile>

Thanks,

Abhijit Nathwani

[Ghofrane GSOURI]

Hello @ANath.1 

Thanks a lot for having reported.

Issue has been raised to our dev team in order to fix the issue.

Internal ticket number:   183448 (This is an internal tracking number and is not accessible or usable by customers).

I will keep you posted with updates

THX

Ghofrane

[ANath.1]

Hi @Ghofrane GSOURI 
Thank you for the response. Request you to kindly keep updating the thread here with updates to the issue.

Thanks,

Abhijit Nathwani

[BarryWhit]

@ANath.1 , please see discussion in Project generation from command line, headless for a possible solution.

[ANath.1]

Hi @BarryWhit ,
The current workaround I implemented is using xvfb display as discussed in the thread. However, in a *headless* build tool, it should not be required, as documented in the user guide. So this is a definite bug that needs fixing.

Thanks,

Abhijit Nathwani

[BarryWhit]

I guess what you're saying is technically correct. I think what ST actually means by headless is more like "batch-mode"/"non-interactive"/scriptable, no GUI interaction required. But by all means, you guys get them to fix it if you can. There's also the issue where if something goes wrong during batch-mode generation, you get a GUI message box and user interaction is required instead of cube exiting with an error code - which is not cool (and andrew said as much in the other thread).

[ANath.1]

Hi @BarryWhit 
The main reason to raise a concern is because the behavior of the CubeMX tool contradicts the user manual (UM1718). 

It explicitly says to run CubeMX "without UI", use this command. However, the said command pops-up a window which is not expected. 

---

There's also the issue where if something goes wrong during batch-mode generation, you get a GUI message box and user interaction is required instead of cube exiting with an error code - which is not cool (and andrew said as much in the other thread).

---

I agree, the tool should exit gracefully with error codes instead of popping up windows when invoked in non-GUI mode. Hopefully the issues are resolved soon :) We can always try !

Thanks,

Abhijit

[Pavel A.]

Already several times people ask for scriptable CubeMX code generation for CI. I really cannot understand why they think they *need* this, sorry. It looks rather like a X/Y issue. Maybe what they actually need is configuring the projects to *link* to Cube libraries located somewhere else, rather than *copying* the static library files into their projects. (looking at the statement "SetCopyLibrary copy all" in the OP). 

In my experience, generation or re-generation of a project in Cube always requires a review and build validation by a human. This makes all generated files "precious" and justifies keeping them in version control. Especially files that contain "user code" sections. The "static" Cube library files with this approach are not copied into the project and not included into the project version control repo at all - they are in a separate "library" repo or module (or directly pulled from the official ST github repo).

So there's no real need to involve the CubeMX and its databases into CI automation. The future removal of the CubeMX part from CubeIDE will help to put the latter in a container.

The only reason to need (*) CubeMX generation in CI pipeline is IMHO only when you develop a CubeMX add-on or work for ST and test the CubeMX itself.

(*) difference between need and want

 

[andrew_parlane]

---

@Pavel A. wrote:

I really cannot understand why they think they *need* this, sorry. It looks rather like a X/Y issue.

---

We don't need it, but It's not an X/Y issue. It's just a nice to have that's a very common feature and fits into a sensible build flow, allowing  continuous integration, automatic builds, etc... We can work around this not technically being supported, but it's being asked for a lot because it's a pretty basic feature these days.

 

---

@Pavel A. wrote:

Maybe what they actually need is configuring the projects to *link* to Cube libraries located somewhere else, rather than *copying* the static library files into their projects. (looking at the statement "SetCopyLibrary copy all" in the OP). 

---

That would work for some cases but not all.

1. When there are bugs / missing features you sometimes want to patch the provided code.
2. When you actually want auto-generated code, rather than just copied files. For example how main() is auto-generated to initialise your peripherals. Or those same peripheral initialisation functions.

---

@Pavel A. wrote:

In my experience, generation or re-generation of a project in Cube always requires a review and build validation by a human. This makes all generated files "precious" and justifies keeping them in version control. Especially files that contain "user code" sections. The "static" Cube library files with this approach are not copied into the project and not included into the project version control repo at all - they are in a separate "library" repo or module (or directly pulled from the official ST github repo).

---

I would disagree. Generation should be a repeatable process. If I re-generate the same project without any changes to the configuration file / script the output should be consistent. Files with "user code" in them would clearly need to be checked in, but that's something I've (and I assume many others who are asking for this) have explicitly avoided so we don't have to check these files in.

The problem with checking these auto-generated files in are:

• They often don't match coding standards, while not a problem in it's own right tools may have to be configured to avoid automatic checks on these files when checking them in.
• A minor change to a configuration file can cause a handful of unrelated changes in auto-generated files which make the code review process harder, and clutter up the commit. For example I added a new peripheral which then got initialised in main(), this caused two other initialisations to swap locations.
• Text editors that automatically format code / remove whitespace / line endings could make accidental changes to these files which you then have to constantly revert so they don't end up cluttering up your git status.
• If you modify the code in anyway (bug fix, feature addition), you have to re-apply the fix manually after every regeneration. By not checking in those changes you can add a patch to your generation script to do this automatically. If you update the FW package you may have to update the patch, but simply regenerating the project (as long as you use your script) will always work.

These are all problems I've had over the years working with various different tools, and this has led me and the company I work for to have a policy to not check in auto-generated code where possible. Other companies may have different policies, and I'm not saying they are wrong, there's no perfect solution here. I would however like the option to do this without jumping through a bunch of hoops.

I would also like to clarify that the STM32 tools are been pretty decent quality compared to some tools I've worked with. This feature is so far my only real complaint.

Andrew

[BarryWhit]

> They often don't match coding standards

but the style guide should executable and automated. The pretty printer can applied automatically when checking in, with a git hook.

 

> A minor change to a configuration file can cause a handful of unrelated changes in auto-generated files which make the code review process harder,

I think this is a good point. We use code-generation to reduce the cognitive overhead of boilerplate. Putting the boilerplate in commits, arguably, runs counter to that. OTOH, there's also a strong trend of vendoring every dependency you can, to insulate your build from the whims of some 12-year old who wrote a nodejs module in 2014, which you didn't even know was a transitive dependency of your project way way down in there. So there's

also an argument for keeping everything you need to reproduce your build, if github has a meltdown tomorrow for example. or in 10 years if ST changes the generator in incompatible ways, and the version you use now is no longer available for download, but your biggest customer is paying you $$$ to make a change. etc'.

 

> Text editors that automatically format code / remove whitespace / line endings

can be configured not to do so.

 

> so they don't end up cluttering up your git status.

Just FYI, git (via .gitoptions) can be configured to ignore whitespace changes IIRC, and also to control how crlf it managed in working dir vs. commits (windows/linux convention).

 

> If you modify the code in anyway (bug fix, feature addition), you have to re-apply the fix manually after every regeneration. By not checking in those changes you can add a patch to your generation script to do this automatically.

And if you can automate the latter case, you can also automate the former. Whenever you regenerate after a change, you also recompile and run tests. Just put the patch-check/apply in your makefile. Or again, in a git hook.

 

Not to mention, you could customize the templates CubeMX uses to generate the code in the first place.

 

So yeah, everything can be worked around, Like you said, What I don't understand is what about the current (admittedly wonky) situation is preventing you from doing any of the things you described. It's just the wonkiness itself that's not great right?

 

As an aside, are you aware that ST has a project going centered around providing a parallel development flow using VSCode? I'm not sure how the code generation is worked in (Update: the user uses the CubeMX GUI to generate CMake projects which VS Code/the extension can work with), but you should have a look - There's a thread about it around here somewhere.

 

 

[andrew_parlane]

---

@BarryWhit wrote:

Not to mention, you could customize the templates CubeMX uses to generate the code in the first place.

This is interesting, I've not see reference to this before. Can you point me at how you would do this?

 

---

@BarryWhit wrote:

So yeah, everything can be worked around, Like you said, What I don't understand is what about the current (admittedly wonky) situation is preventing you from doing any of the things you described. It's just the wonkiness itself that's not great right?

---

Indeed, there are multiple ways to achieve my end goal. The workarounds that you and others have provided are sufficient for that. To that end I'm not tearing my hair out over this. It's a nice to have and I'm trying to raise awareness of why it would be useful. I am not really expecting it to happen any time soon, but if nobody raised awareness / complained about this it would never get fixed.

 

---

@BarryWhit wrote:

As an aside, are you aware that ST has a project going centered around providing a parallel development flow using VSCode? I'm not sure how the code generation is worked in (maybe it isn't), but you should have a look - There's a thread about it around here somewhere.

---

I'll look it up, thanks.

Andrew

[BarryWhit]

@BarryWhit wrote:

Not to mention, you could customize the templates CubeMX uses to generate the code in the first place.

This is interesting, I've not see reference to this before. Can you point me at how you would do this?

I've not seen any official docs about it, but I haven't really looked either.

 

If you look in CubeMX GUI under the Project Manager/Code Generator/Template Settings,  you'll see some   things configurables that suggest ST made some consideration for the case of users using custom templates.

 

CubeMX uses the Apache FreeMarker Java Template Engine. The template files are quite dense, but it's not impossible to figure out if you're motivated.  The default location for template files (just text files) is:

STM32CubeIDE\plugins\com.st.stm32cube.common.mx_6.10.0.202311202037\db

 

There are also a handful of user-created CubeMX packs out there which integrate with CubeMX and its code-generator, so it has been done. One example I've used is I-CUBE-USBD-COMPOSITE , which lets you create composite devices with the ST USB Device library for example. Find it on github.

 

UPDATE: If you think about it, the .ioc file must be primarily the context object for the template engine. In theory, you could take the ioc, and the template files bundled with your version, and create an independent CLI java project based on Apache FreeMarker that would generate byte-for-byte the same output. But this seems like a rabbit hole.

 

Also note that you have the option of generating .c/.h file per peripheral instead of the default, which may (or may not) make your life easier with the post-generation patch flow you said you favored.

 

> if nobody raised awareness / complained about this it would never get fixed.

 

fair. But also possible that a few people ask for it and it would never get fixed. No harm in trying though.

 

It just seems that you're asking for a major time investment by ST for relatively little gain by either ST or their customers, so it'd be hard to justify for them. This is precisely because the current situation does do the job, it's just clunky. It's always an uphill struggle to make a business case for investing resources in replacing something that works, with something that works but is also pretty. Even if the ST team member that responds shares your aesthetics and accepts your arguments, they have to sell the required resource investment to their boss - and they probably can't, because it already works kinda.

Asking them to open-source the code-generation plugin component of CubeMX so you, and the group of highly committed hardcore super-developers which you can prove you represent , can take it from there - that'd be a far more attractive ask IMO.

 

[Pavel A.]

>  but if nobody raised awareness / complained about this it would never get fixed.

This was my point... Nobody raised awareness - because almost nobody (outside ST at least) runs CubeMX in CI environment. Those who do are cut off by Occam blade ))

 

[andrew_parlane]

---

@Pavel A. wrote:

>  but if nobody raised awareness / complained about this it would never get fixed.

This was my point... Nobody raised awareness - because almost nobody (outside ST at least) runs CubeMX in CI environment. Those who do are cut off by Occam blade ))

---

While searching for how to resolve my issues with this I found a handful of posts from over the last 4 or 5 years asking this same question. It's not exactly a hot topic, but it is something that multiple people have asked for, including both OP and myself this month. Then how many others haven't bothered to create an account and post because they found one of those older threads and decided the workarounds were good enough, if a bit wonky.

 

[BarryWhit]

Late Update:

UM1718: Custom code generation 

 

(I really should google before I claim there are no docs)