STM32MP151aac custom board kernel not starting

[michiel.tjampens]

I'm trying to get to know stm32mp1 while making a board for this thing.

While using the default image provided by manufacturer seems to work. The kernel used is old compared to the current ST BSP and support from them isn't exactly great so I'm trying to migrate to the latest BSP. This isn't going so well.

Following instructions found in the wiki. I've managed to compile it but when trying to run, it just stops at 'starting kernel'. Recompiling with LL_DEBUG and EARLY_PRINTK, resulted in some more info.

The lines that seems to show an issue are:

[ 1.599531] i2c_dev: i2c /dev entries driver [ 1.606005] syscon-reboot: probe of reboot failed with error -22 [ 1.733737] SMCCC: SOC_ID: ARCH_SOC_ID not implemented, skipping .... [ 2.031116] irq: no irq domain found for gpio@54004000 ! [ 2.036565] gpio-keys joystick: Found button without gpio or irq [ 2.057386] gpio-keys: probe of joystick failed with error -22 [ 3.458283] Warning: unable to open an initial console. [ 3.465913] Freeing unused kernel image (initmem) memory: 1024K [ 3.487792] Run /init as init process 
[ 3.866045] udevd[133]: Starting version 244
[ 3.893045] udevd[134]: /lib/udev/rules.d/50-udev-default.rules:18 Unknown group 'tty', ignoring
[ 3.917955] udevd[134]: /lib/udev/rules.d/50-udev-default.rules:19 Unknown group 'tty', ignoring
[ 3.926963] udevd[134]: /lib/udev/rules.d/50-udev-default.rules:20 Unknown group 'tty', ignoring
[ 3.947840] udevd[134]: /lib/udev/rules.d/50-udev-default.rules:21 Unknown group 'tty', ignoring
[ 3.956848] udevd[134]: /lib/udev/rules.d/50-udev-default.rules:22 Unknown group 'tty', ignoring
[ 3.988188] udevd[134]: /lib/udev/rules.d/50-udev-default.rules:23 Unknown group 'tty', ignoring
[ 3.997197] udevd[134]: /lib/udev/rules.d/50-udev-default.rules:24 Unknown group 'tty', ignoring
[ 4.028175] udevd[134]: /lib/udev/rules.d/50-udev-default.rules:25 Unknown group 'dialout', ignoring
[ 4.047834] udevd[134]: /lib/udev/rules.d/50-udev-default.rules:27 Unknown group 'kmem', ignoring
[ 13.991414] arm-scmi firmware:scmi-0: Invalid channel descriptor for 'scmi-0'
[ 13.998713] arm-scmi: probe of firmware:scmi-0 failed with error -22
[ 14.011959] platform wifi-pwrseq: deferred probe pending
[ 14.017331] amba 48004000.sdmmc: deferred probe pending
[ 14.022685] amba 58005000.sdmmc: deferred probe pending
[ 14.027978] amba 58007000.sdmmc: deferred probe pending
[ 14.033239] platform led: deferred probe pending
[ 14.037911] platform 4c001000.mailbox: deferred probe pending
[ 14.043695] platform mlahb:m4@10000000: deferred probe pending
[ 14.049587] platform 48002000.dma-router: deferred probe pending
[ 14.055633] platform 4c000000.hwspinlock: deferred probe pending
[ 14.061701] platform 50000000.rcc: deferred probe pending
[ 14.067138] platform 50001020.pwr: deferred probe pending
[ 14.072595] platform 5000d000.interrupt-controller: deferred probe pending
[ 14.079536] platform soc:pin-controller@50002000: deferred probe pending
[ 14.086282] platform soc:pin-controller-z@54004000: deferred probe pending
[ 14.093221] platform 48000000.dma: deferred probe pending
[ 14.098692] platform 48001000.dma: deferred probe pending
[ 14.104128] platform 58000000.dma: deferred probe pending
[ 14.109585] platform 5a006000.usbphyc: deferred probe pending
[ 14.115368] platform 50001000.pwr: deferred probe pending
[ 14.120822] platform 4000f000.serial: deferred probe pending
[ 14.126520] platform 40010000.serial: deferred probe pending
[ 14.132237] platform 40018000.serial: deferred probe pending
[ 14.137954] platform 5c000000.serial: deferred probe pending
[ 14.143652] platform 54003000.rng: deferred probe pending
[ 14.149122] platform 58002000.nand-controller: deferred probe pending
[ 14.155606] platform 5800a000.ethernet: deferred probe pending
[ 14.161559] platform 49000000.usb-otg: deferred probe pending
[ 14.167348] platform 5800d000.usbh-ehci: deferred probe pending
[ 14.173332] platform 5c004000.rtc: deferred probe pending
[ 14.178788] platform 40013000.i2c: deferred probe pending
[ 14.184223] platform 5c002000.i2c: deferred probe pending
[ 14.189677] platform 50028000.thermal: deferred probe pending
[ 14.195463] platform 5a002000.watchdog: deferred probe pending
[ 14.201356] platform cpufreq-dt: deferred probe pending

Based on that I found another post that might be related, so I checked the dts files. No mention of mco1.

Next step was compiling the original device tree files with the same BSP mentioned earlier...

The complains raised by the compiler were that it couldn't find the sram and optee group. Have those been removed from stm32mp151.dtsi? According to another post the sram was renamed to sram1 but can't find that either. If so, is there a changelog somewhere that helps with migrating from an old bsp version?

[Olivier GALLIEN]

Hi @michiel.tjampens ,

Please confirm that latest BSP is the ecosystem V5 right STM32 MPU ecosystem release note - stm32mpu ?  What's the current BSP provided by MYIR ? 

Change can be quite huge and you have to adopt a strategy similar to a new bring-up. 

Import ioc or generate equivalent in last CubeMX version, generate device tree and complete them carefully. 

Following articles may help : 

FAQ : STM32MP1 – How to migrate STM32CubeMx project from an old ecosystem to a new one?

FAQ : STM32MP1 - How to create a device tree adapted to your design with STM32CubeMX?

FAQ: STM32MP1 How to bring up STM32MP1

Olivier 

[michiel.tjampens]

Hi Olivier,

Thanks for the links, I'll look into them.

I think Myir uses BSP 3.1 because that's the version number on the files in the sdk folder...

The kernel is 5.4.31 and that has been around since april 2020. The cubeprogrammer version used is 2.7.0.

So seems like i'll have to start pretty much from scratch.

 

[Olivier GALLIEN]

Hi @michiel.tjampens ,

Version you gave are even DV2.x., so very old. 

I don't know if you succeed to bring DV5.0 by our own but don't hesitate to ask MYIR to provide a more recent baseline. 

Olivier 

 

[michiel.tjampens]

Hi @Olivier GALLIEN ,

Well I've asked them, but so far only a broken english reply. I think they asked what errors I got but unsure (so replied as such).

So went ahead and followed the second link you gave. But the kernel device tree generated after following it works even worse than the old one (atleast that gave error messages), it just stops after loading the device tree and then reboots. Maybe it's best to start completely from 0 instead of mixing two things (new kernel with dtb, old uboot etc).

Michiel

[Olivier GALLIEN]

Hi @michiel.tjampens ,

 

Yes, ST does not ensure compatibility of new kernel on top of old BSP.  Avoid any mix and use all components from the same release. 

 

Olivier 

[michiel.tjampens]

Started from scratch (empty CubeMX project) to an actually booting board. To get to the same symptom (not booting kernel) but a completely different cause (i think). Below is the output after compiling the kernel according to the README_HOW_TO in the distr or dev package. With the only change being 'enable low level debug'.

 

[ 0.199871] cpuidle: using governor menu
[ 0.210280] hw-breakpoint: found 5 (+1 reserved) breakpoint and 4 watchpoint registers.
[ 0.218341] hw-breakpoint: maximum watchpoint size is 8 bytes.
[ 0.224779] Serial: AMBA PL011 UART driver
[ 0.231370] stm32-pm-domain pm_domain: domain core-ret-power-domain registered
[ 0.238744] stm32-pm-domain pm_domain: subdomain core-power-domain registered
[ 0.245927] stm32-pm-domain pm_domain: domains probed
[ 0.259010] 8<--- cut here ---
[ 0.262103] Unhandled fault: imprecise external abort (0x1c06) at 0xae4a437e
[ 0.269191] [ae4a437e] *pgd=00000000
[ 0.272801] Internal error: : 1c06 [#1] PREEMPT SMP ARM

 

Not sure if probing the domains caused the crash or the thing it does after that.

Any ideas/tips on how to debug this?

[Olivier GALLIEN]

Hi @michiel.tjampens ,

 

Could you please provide the complete boot log from the very first trace of TF-A ? 

Thanks,

 

Olivier 

[michiel.tjampens]

Hi @Olivier GALLIEN ,

Attached it for reference. Managed to resolve that issue. Forgot to update the kernel dts for yocto after all the issues, copy pasted the last u-boot one and now it gets passed that point. After comparing the two, scmi was missing from the kernel one. (attached the dts)

But that brought me to another issue. Attached is full bootlog2.txt.

 

[ 0.402919] Driver 'scmi-optee' was unable to register with bus_type 'tee' because the bus was not initialized.
...
[ 1.665708] (NULL device *): TA_CMD_GET_ENTROPY invoke err: ffff000a
...
Starting systemd-udevd version 253.1^
[ 10.725045] optee-rng optee-ta-ab7a617c-b8e7-4d8f-8301-d09b61036b64: TA_CMD_GET_ENTROPY invoke err: ffff000a
root '/dev/disk/by-partuuid/491f6117-415d-4f53-88c9-6e0de54deac6' doesn't exist or does not contain a /dev.

 

Checked in U-boot and that uuid is indeed the rootfs and does contain /dev.

And another issue:

[ 3.365707] stpmic1 0-0033: Failed to get main IRQ: -22
[ 3.369655] stpmic1: probe of 0-0033 failed with error -22
[ 3.384419] stm32f7-i2c 5c002000.i2c: STM32F7 I2C-0 bus adapter

Is it normal that a stm32mp151 uses stm32f7 drivers?

 

[michiel.tjampens]

Didn't get any further with the /dev issue, but did fix the entropy one. Turns out that on the stm32mp157 the RNG module is selected by default but not on the stm32mp151, so I overlooked it when Lab15 stated to leave it as default.

More questions raised (that might lead to something):

• Before the kernel boots, the console shows "Retrieving file: /st-image-resize-initrd", but that resize doesn't happen. Is resizing the fs's the only thing this does? Tried manually resizing ,thought maybe the reason /dev wasn't filled in was lack of space, that wasn't it.
• Checked /dev and it's there but empty, what should normally populate this?
• Is it normal that the PER clock is at 0? Double checked with the Lab and everything is copied.
  • Clocks:
    - MPU : 650 MHz
    - MCU : 200 MHz
    - AXI : 266.500 MHz
    - PER : 0 MHz
    - DDR : 533 MHz

Attached bootlog from after fixing entropy etc.

[debugging]

Perhaps you could hit a key to stop the autoboot to debug further from the u-boot command line,

I had the same question about PE it's zero. but I even can't get to the the step of ... loading the environment after the DDR 533Mhz....  because after clock #151 (UART4)  enabled the boot freezes..

[michiel.tjampens]

I'm doing this as a hobby and the problem is that I probably lack the knowledge, don't know what I could do from the u-boot cli for this issue. The kernel doesn't find the rootfs but u-boot 'mmc' cmds clearly show it's there.

So far I've managed to solve various issues, but this time I haven't even got an idea where/how to start looking.

PE being zero might mean something but might totally be irrelevant. I do know that with the original image from the manufacturer PE wasn't 0. But the only peripherials used (i think) are the uart4 and i2c4, both use the same clksrc and the former seems to be working fine but the latter isn't (pmic issue in logs).

[debugging]

I know exactly how you feel, It takes a lot  persistence and passion to try to get your board up and running where you invested so much time in to design, while having to workaround the lack of support (sources and knowledge...) to reach that goal for the software. I am learning along the way facing many roadblocks

I found that DTS generated with CubeMX set CLK_CKPER_DISABLED  for OPP-TEE and TF-A instead of CLK_CKPER_HSE in DK boards while it's the same processor and almost all same peripherals although I only enabled UART4 and USB. After the change it became 24MHz, Don't know why CubeMX disabled PERI.  MX also set the RTC clock divider to 0 while for DK1 boards it is 23.

Clocks:
- MPU : 650 MHz
id=32 clock = bebc200 : 200000 kHz
- MCU : 200 MHz
- AXI : 266.500 MHz
id=30 clock = 16e3600 : 24000 kHz
- PER : 24 MHz
- DDR : 533 MHz

no matter what it did not fix my UART4 clock 151 issue. There is a big difference in clock configuration between DTS generated for a custom board from MX compared to the DTS from ST's own boards. Perhaps ST tuned the DT's for the boards and for a custom board it make it very difficult to understand where a problem could be

[michiel.tjampens]

Kind of a let down expecting two replies only to get a proper one and a useless ai summary...

PER probably 0 because it's used for (q)spi,sai and lptim. So if not using those, no need for DT to activate it. So one possible cause i can scratch of the list.