getting clangd to work in a multiroot project

[sb_st]

Hi, I am really struggling to get clangd to work correctly in a setup I am trying to get going. I'm new to this VSCode workflow, and am finding VSCode seems to have a lot of persnickety behaviors that really require a lot of specificity to get things working right. 

Here is what I am trying to do:

• I start with a fresh workspace, and ONLY the STM32Cube vscode extension pack installed. 
• I add two folders to my workspace:
  • an stm32 project folder, as created by cubeMX
  • a shared_code folder, which contains multiple subfolders, each of which contain .c/.h files. Nothing particularly fancy here. 

I can get this setup compiling fine, but clangd is complaining about various #include files in my shared code area. I understand that the reason for this is because there is no .clangd file in my shared_code area, so clangd can't "see" it from my project area. Intrepid internet searching has led me to adding this to my code-workspace file:

{
	"folders": [
		{
			"name": "stm32_project",
			"path": "."
		},
		{
			"name": "shared_code",
			"path": "../shared_code"
		}
	],
	"settings": {
		"stm32cube-ide-clangd.arguments": [
			"starm-clangd",
			"--query-driver=${env:CUBE_BUNDLE_PATH}/gnu-tools-for-stm32/13.3.1+st.9/bin/arm-none-eabi-gcc",
			"--query-driver=${env:CUBE_BUNDLE_PATH}/gnu-tools-for-stm32/13.3.1+st.9/bin/arm-none-eabi-g++",
			"--compile-commands-dir=${workspaceFolder:stm32_project}/build/Debug"
		],
		"stm32cube-ide-clangd.path": "cube"
	}
}

It is my understanding that this *should* work. What I find, though, is that I need instead to either change that --compile-commands-dir to:

"--compile-commands-dir=${workspaceFolder}/build/Debug"

OR provide an actual absolute path to my project folder. If I do either of those things, clangd works fine.

However, I notice that if I reorder the folders in the Explorer view like so:

{
	"folders": [
		{
			"name": "shared_code",
			"path": "../shared_code"
		},
		{
			"name": "stm32_project",
			"path": "."
		}
	],
	"settings": {
		"stm32cube-ide-clangd.arguments": [
			"starm-clangd",
			"--query-driver=${env:CUBE_BUNDLE_PATH}/gnu-tools-for-stm32/13.3.1+st.9/bin/arm-none-eabi-gcc",
			"--query-driver=${env:CUBE_BUNDLE_PATH}/gnu-tools-for-stm32/13.3.1+st.9/bin/arm-none-eabi-g++",
			"--compile-commands-dir=${workspaceFolder}/build/Debug"
		],
		"stm32cube-ide-clangd.path": "cube"
	}
}

clangd breaks again. 

It's my understanding that the way around this is to use the "workspaceFolder:stm32_project" syntax, to be specific about what folder my compile_commands.json file lives in. 

Further, if I try to save my workspace alongside my two folders - rather than inside my project folder - that the ${workspaceFolder} syntax doesn't seem to work at all (it is not obvious to me how to check the value of what this 'variable' is resolving to, so I'm not sure how to troubleshoot this). 

// This seems to work, sort of
my_shared_code/
my_project/
 my_workspace.workspace

// I can't get this to work (and it's more annoying to have the 
// workspace file outside of the project anyway)
my_shared_code/
my_project/
my_workspace.workspace

All of this makes me feel as though I am using vscode "wrong", and certainly makes it clear that the order of things in the Explorer pane and the location of workspace files aren't as arbitrary as I presumed they were. 

At this point, I feel compelled to ask some questions:

• Am I approaching this the right way?
• Where does ST expect VScode workspace files to be saved?
• Why doesn't ${workspaceFolder:stm32_project} work in the above example?

 

[Cartu38 OpenDev]

@sb_st 
Works for me or am I missing something specific you're trying to achieve ?

What is key is compile_commands.json file yes this one is driving fully clangd. If your project complie fine this file should be fine too so all should work as a charm.

Have you any CMake project as part of your shared folder ... if so another story please confirm if so.

[Cartu38 OpenDev]

Works the very same if relocating somewhere else ...

 

[sb_st]

Hm. I am not sure what I'm doing differently, but this is definitely not my experience - none of this is "working like a charm". 

I have created an example project structure here that exemplifies this problem - I am attaching a zip file of it here. 

Here shows the structure of what I'm trying to do:

 So you can see my directory structure (which differs slightly from your example - I am using a series of nested subfolders in my shared code area). We can see that this project compiles fine - so cmake itself appears to be working. 

(note: I have tried structuring the subfolders in my shared code area as separate Cmake libraries. So, each component gets its own CMakeLists.txt file. This produces some complaints from the ST extension, but it compiles fine. It does not solve my issue with clangd, however). 

So, I try saving my workspace, and adding this line to my workspace file:

 It's my understanding that the above SHOULD work, but it does not. If I modify it slightly though, we can see it seems to make clangd happier:

 However, if I reorder my folders in the Explorer pane, it breaks again:

I agree that all this *seems* like it looks much like your example. However, it sure doesn't seem to be working, and I can't understand what I'm doing wrong. 

[Cartu38 OpenDev]

@sb_st 

I've pain to get back your attached material  

But anyway from my project trying to mimic your snapshots I'm getting

So works the same ok

Maybe this is not ok to you ?

 

 

[Cartu38 OpenDev]

@sb_st 
I would be agree about "if nok to you" in real ...
I guess I've understood what issue is ... have to find proper solution then.

Moving to exact same as you my indexign experience is bad too and issue is reported Thanks clangd output channel trace:

Means because editing a file as part of shared area, clangd integration because multi root workspace is doing kind of project auto switch. No compile database is part of this shared project root so the stuff is blind not being able to index.

Maybe adding some CMake material as part of shared project is the key. Possibly https://cmake.org/cmake/help/latest/module/ExternalProject.html is helpful.

[sb_st]

Thanks @Cartu38 OpenDev, yeah your solution of using:

#include "../params/myAdderParam.h"

instead of:

#include "myAdderParam.h"

seems like more of a workaround rather than a 'real' solution (though I certainly appreciate you digging in to help me out with this!)

It does seem that this issue might be related to header-only shared code files. I don't seem to have problems with components in my shared code area that have .c files, because those become proper build targets, which seems like what clangd wants to see?

[Cartu38 OpenDev]

@sb_st 
Source files (.c) are working fine because explicitly pointed as part of your project CmakeLists.txt. Header files (.h) are not ... reason why.
In real from clangd perspective none of them are processed right because project context switch ...

[ankes]

The virus scan of your attachment completed and I managed to obtain a copy. After extracting and creating a workspace on the two folders I can reproduce your problem, and I believe I have an idea how to go about it.

When you use a multi-root workspace (that's the official term btw) then CMake Tools "reconfigures" itself based on the file you have open in the editor, and the location of that file in the overall scheme of things.

As an example, if I open the "sysmem.c" file and look at the CMake tab, I can see this:

If I swap to "my_driver.c" the extension's state changes accordingly:

As you can see, the "configure" section of the second file shows "__unspec__" in the selected kit row which indicates that CMake is in an "unconfigured" state. This alreay hints us at what's going awry: the second workspace root ("my_shared_code") does not contain any CMakeLists.txt or CMakePresets.json files. In short, CMake doesn't have any work to do.

This setup that you are trying to build is a bit exotic to say the least. It looks like you want to build some kind of a library from the shared code but what kind, exactly? If it is an ordinary static library then it needs the headers from CMSIS and HAL to compile. If it is a dynamic library then it will also need the source files from HAL. You can also create a CMake "interface library" like the CubeMX does for HAL but that's a bit outside my forte so I can't help you there.

However, I take a stab at guessing that you want a static library. To facilitate this I have patched through the example you provided and added the necessary glue bits. Extract the files to a directory and diff the directories to get the big picture, but here are the key pointers:

• Upgraded CMake version requirement to 4.0.1 (STM32 extension provides this so it's no problem)
• CMakeLists.txt in the "my_shared_code" defines a static library that is then imported in "my_stm32f4_project"
• CMakePresets.json (version 10) in the "my_shared_code" links to the actual file in "my_stm32f4_project" side
• In "my_stm32f4_project" side, the CMakePresets.json is modified to use "fileDir" in the toolchain path; this allows the include statement to find the right file, instead of looking inside "my_shared_code"

You can take the archive, and turn the workspace folders in whichever order you want. Everything still works.

[sb_st]

Hm, when I download this, and open the setup, and move the my_shared_code folder above the my_stm32f4_project, I still see errors in the my_driver.c file:

 

However, I'd like to focus on this:

---

This setup that you are trying to build is a bit exotic to say the least.

---

It's not my intent to deliberately build something exotic or difficult - I'm a hobbyist who's spent a few years coming up to speed with the stm32 ecosystem (coming from an arduino context), and learning mostly through cubeIDE. I have an f4 development board I made myself, and a few breakout boards that I've made as well, and a handful of projects that are all coded to work with this board ("project", in this context, is a cubeMX-generated stm32 project).

I often want to reuse things like drivers and constants and my own math libraries and such in these projects. It seemed like duplicating this code into each project wasn't so efficient, so I found that if I created a centralized folder that contained them, I was able to configure my "project settings" in cubeIDE such that I could bring them in and they would compile fine. I didn't realize I was doing something highly unorthodox by working this way, and I suppose it is to great credit to cubeIDE that I was apparently able to do this with very little complication and have it all "just work" for me. 

I'm now trying to adapt to vscode, and I'm fully aware that this carries with it the need to learn new workflows. But...I'm inexperienced with vscode beyond just using it as a text editor, and I am not a highly-opinionated, mature, or super-experienced embedded software engineer, so I don't really know what I don't know here, or what constitutes "exotic" in terms of how I organize my code. Cmake is new to me, Clangd is new to me, vscode with this toolchain stuff is new to me. It's all new to me, and I'm just trying to come up to speed with it. 

I wouldn't say that my opinions or needs are hardened enough to demand that ST's vscode development effort should bend to my setup, but I WOULD really love to know how I might better do this, or how ST imagines I SHOULD be doing it, or how others might do this differently. I'm very open to learning that. I'm eager, I just don't know where to look. 

[ankes]

To begin with, let me apologize for any potential harshness of tone in my previous reply. I did not mean that there's anything inherently wrong in your setup, I just found it a bit strange at first. From a software architecture perspective there are two kinds of "shared libraries" in the very broadest sense:

• Ones which are fully independent, and provide an API or similar to use it
• Ones which are not independent, and place constraints on the consumers

An example of the first one would be the STM32 USB Device Middleware library, or even a general-purpose math library. The STM32 USB library defines things like "USB_OK", using them instead of "HAL_OK" (and thus taking a dependency on a specific HAL library which provides these definitions). In this sense, it is "independent" from the HAL, and could theoretically be used with completely different, non-STM32 devices as well. It exports several "glue functions" that the consumer is expected to call in order to inform the library about specific events happening on the USB side. The "tinyusb" project is a precisely similar construct. There's a lot of "general purpose" code in that library, and then a thin wrapper which makes it "device-specific".

An example of the second would be how CubeMX generates the "cmake/stm32cubemx/CMakeLists.txt" file in any given STM32 project. This library is not truly independent; it actually presets the consumer to use a specific STM32F4xx variant without allowing any deviation (this happens with the "MX_Defines_Syms" section, near the top of the file). It also directly includes the HAL header files and even the source files. The consumer of this library gets a very tailored set of functionality from the HAL.

The "exotic" portion comes from the way your files were organized in the example archive. The shared library code referred to the HAL library without actually enforcing the use of any specific STM32F4xx variant. At the same time, "my_driver.c" still wanted to import a key header file from the library. This was a bit backwards because in order to use STM32F4xx HAL, you must define the exact chip that you want to use, very much like the "MX_Defines_Syms" section does.

Now that you've explained how you want to use the shared driver code I believe what you want is the second option. In short, you'd like to share code that's specific to an exact STM32F4xx variant, and only usable in projects that use this specific chip. Assuming this is the case, then later today I will try to adapt the example I provided to be more along these lines, so that you can see the difference in practise.

As for the error you're experiencing now, could you provide the error message that appears when you hover over the red squiggles? Also, did you run CMake's "configure" for both projects? I cannot see the "build" folders that are generated as part of this process in your screenshot.

To run CMake's configure for both projects open "my_driver.c", the CMake's status window, and then re-configure the project. Refer to my screenshot for an example (although in here I have just opened a random file in my own project).

Do the same for "sysmem.c" in the other project. Note how the "folder" in the CMake status window should change as a result. If it does not change then a setting which allows this to happen is off somewhere in your user side preferences file. If so, just click the edit icon next to the first row under "Folder" (hover over to see it) and choose the other folder.

If you manage to configure both projects, do the red squiggles go away?

[ankes]

You're welcome.

I did take a look into moving HAL to the shared folder but unfortunately it failed in the end. The problem is that the HAL library requires a define statement which chooses the exact chip, and this define must be present when the sources are compiled.

This means that either the CMake "shared library" that compiles HAL sources chooses it -- which then makes the "my_shared_code" entirely specific to this chip, and prevents using it in other chips of the same family -- or alternatively all code in the shared side uses CMake's INTERFACE keyword so that the chip can be chosen in the project side which uses it. However, this second approach results in no compilation, no compile_commands.json and thus no clangd support when looking at the source files living inside "my_shared_code".

Remember to mark one of the answers as "accepted answer" so that this thread gets closed.