STM32MP157C-DK2 u-boot SPL debug timeout

[Giacomo]

Hi,

I start working with the new STM32MP157C-DK2 board and after testing the demo i restart from the bottom, compiling just uboot+SPL (no TF-A). I compiled with the last Linaro toolchain 'gcc-linaro-7.4.1-2019.02-x86_64_arm-linux-gnueabihf' and prepared the sd card with minimum just to boot (fsbl1 - fsbl2 -sspl partitions).

All works fine: i reach the u-boot command line with no errors.

Now i'm trying to debug the SPL: i'v never debugged a MPU and i wonna be sure i'm able to do this when i'll work on my own hardware. I follow the guide https://wiki.st.com/stm32mpu/wiki/GDB and i successfully connect the debugger, set some breakpoints (ex. on spl_init function) and contine throuth them. Despite all is very slow (it takes nearly 5s to continue until a break just in the next line) it seams to work. Grate !

Problems starts when i try stepping: it doesn't work and i recieve a "timeout waiting for target halt" error.

I bypassed gdb and i connect directly to the openocd Telnet server on localhost:4444, that's the result:

shell> telnet -r localhost 4444
Trying 127.0.0.1...
Connected to localhost.
Escape character is '^]'.
Open On-Chip Debugger
> reset halt
ap 0 selected, csw 0x10006000
SRST line asserted
SRST line released
stlink_connect(connect)
SWD DPIDR 0x6ba02477
stm32mp15x.cpu0 rev 5, partnum c07, arch f, variant 0, implementor 41
target halted in ARM state due to breakpoint, current mode: Supervisor
cpsr: 0x000001d3 pc: 0x2ffc2690
MMU: disabled, D-Cache: disabled, I-Cache: disabled
ap 0 selected, csw 0x10006000
pc (/32): 0x2FFC2690
pc (/32): 0x2FFC2694
stm32mp15x.cpu1 rev 5, partnum c07, arch f, variant 0, implementor 41
target halted in Thumb state due to debug-request, current mode: Supervisor
cpsr: 0x800001f3 pc: 0x0000060e
MMU: disabled, D-Cache: disabled, I-Cache: enabled
Deferring arp_examine of stm32mp15x.cpu2
Use arp_examine command to examine it manually!
Deferring arp_examine of stm32mp15x.ap2
Use arp_examine command to examine it manually!
> step
stm32mp15x.cpu0 rev 5, partnum c07, arch f, variant 0, implementor 41
stm32mp15x.cpu1 rev 5, partnum c07, arch f, variant 0, implementor 41
target halted in Thumb state due to debug-request, current mode: Supervisor
cpsr: 0x600001f3 pc: 0x0000060e
MMU: disabled, D-Cache: disabled, I-Cache: enabled
target halted in ARM state due to breakpoint, current mode: Supervisor
cpsr: 0x000001d3 pc: 0x2ffc2544
MMU: disabled, D-Cache: disabled, I-Cache: disabled
timeout waiting for target halt
./share/openocd/scripts//target/stm32mp15x.cfg:158: Error: 
at file "./share/openocd/scripts//target/stm32mp15x.cfg", line 158
>

Just the commands 'reset halt' + 'step' produce the error.

I'm using the openocd version provided with the sdk en.SDK-x86_64-stm32mp1-openstlinux-4.19-thud-mp1-19-02-20.tar.xz (here https://wiki.st.com/stm32mpu/wiki/STM32MP1_Developer_Package) and that's its output:

<mysdkpath>/x86_64-openstlinux_weston_sdk-linux/usr$ ./bin/openocd -s ./share/openocd/scripts/ -f board/stm32mp15x_dk2.cfg
Open On-Chip Debugger 0.10.0+dev-00546-g1afec4f-dirty (2019-02-01-13:33)
Licensed under GNU GPL v2
For bug reports, read
	http://openocd.org/doc/doxygen/bugs.html
cortex_a interrupt mask on
cortex_a interrupt mask on
cortex_a domain access control fixup on
cortex_a domain access control fixup on
3333
none separate
adapter speed: 5000 kHz
adapter_nsrst_assert_width: 200
adapter_nsrst_delay: 200
none srst_pulls_trst
srst_only srst_pulls_trst srst_gates_jtag srst_open_drain connect_deassert_srst
Info : Listening on port 6666 for tcl connections
Info : Listening on port 4444 for telnet connections
Info : STLINK V2J33S25 (API v2) VID:PID 0483:3752
Info : using stlink api v2
Info : Target voltage: 3.223311
Info : clock speed 5000 kHz
Info : SRST line released
Info : stlink_connect(connect)
Info : SWD DPIDR 0x6ba02477
Info : stm32mp15x.cpu0: hardware has 6 breakpoints, 4 watchpoints
Info : stm32mp15x.cpu0 rev 5, partnum c07, arch f, variant 0, implementor 41
Info : stm32mp15x.cpu0 cluster 0 core 0 multi core
Info : stm32mp15x.cpu1: hardware has 6 breakpoints, 4 watchpoints
Info : stm32mp15x.cpu1 rev 5, partnum c07, arch f, variant 0, implementor 41
Info : stm32mp15x.cpu1 cluster 0 core 1 multi core
Info : Listening on port 3334 for gdb connections
Info : Listening on port 3333 for gdb connections
srst_only srst_pulls_trst srst_gates_jtag srst_open_drain connect_deassert_srst
Info : accepting 'telnet' connection on tcp/4444
ap 0 selected, csw 0x10006000
Info : SRST line asserted
Info : SRST line released
Info : stlink_connect(connect)
Info : SWD DPIDR 0x6ba02477
Info : stm32mp15x.cpu0 rev 5, partnum c07, arch f, variant 0, implementor 41
target halted in ARM state due to breakpoint, current mode: Supervisor
cpsr: 0x000001d3 pc: 0x2ffc2690
MMU: disabled, D-Cache: disabled, I-Cache: disabled
ap 0 selected, csw 0x10006000
pc (/32): 0x2FFC2690
pc (/32): 0x2FFC2694
Info : stm32mp15x.cpu1 rev 5, partnum c07, arch f, variant 0, implementor 41
target halted in Thumb state due to debug-request, current mode: Supervisor
cpsr: 0x800001f3 pc: 0x0000060e
MMU: disabled, D-Cache: disabled, I-Cache: enabled
Info : Deferring arp_examine of stm32mp15x.cpu2
Info : Use arp_examine command to examine it manually!
Info : Deferring arp_examine of stm32mp15x.ap2
Info : Use arp_examine command to examine it manually!
Info : stm32mp15x.cpu0 rev 5, partnum c07, arch f, variant 0, implementor 41
Info : stm32mp15x.cpu1 rev 5, partnum c07, arch f, variant 0, implementor 41
target halted in Thumb state due to debug-request, current mode: Supervisor
cpsr: 0x600001f3 pc: 0x0000060e
MMU: disabled, D-Cache: disabled, I-Cache: enabled
target halted in ARM state due to breakpoint, current mode: Supervisor
cpsr: 0x000001d3 pc: 0x2ffc2544
MMU: disabled, D-Cache: disabled, I-Cache: disabled
Error: timeout waiting for target halt
./share/openocd/scripts//target/stm32mp15x.cfg:158: Error: 
at file "./share/openocd/scripts//target/stm32mp15x.cfg", line 158
 

As you can see (line 19) i'v the latest STLINK firmware. Looking inside the openocd configuration files i can't find something evident but i didn't go into it.

About the hardware i can say that the mcu doesnt restart the execution after the error, it seams to be still connected with the stlink and halted, indeed i can print registers with 'reg' command:

> reg
===== ARM registers
(0) r0 (/32): 0x2FFC0078 (dirty)
(1) r1 (/32): 0x2FFC2500 (dirty)
(2) r2 (/32): 0x00000000
(3) r3 (/32): 0x00000000
(4) r4 (/32): 0x2FFC2400
(5) r5 (/32): 0x0001D6EB
(6) r6 (/32): 0x2FFC0118
(7) r7 (/32): 0x00000005
(8) r8 (/32): 0x2FFC2500
(9) r9 (/32): 0x00000000
(10) r10 (/32): 0x02004002
(11) r11 (/32): 0x00888E45
(12) r12 (/32): 0x04591E45
(13) sp_usr (/32)
(14) lr_usr (/32)
(15) pc (/32): 0x2FFC2544
(16) r8_fiq (/32)
(17) r9_fiq (/32)
(18) r10_fiq (/32)
(19) r11_fiq (/32)
(20) r12_fiq (/32)
(21) sp_fiq (/32)
(22) lr_fiq (/32)
(23) sp_irq (/32)
(24) lr_irq (/32)
(25) sp_svc (/32): 0x2FFC1BC8
(26) lr_svc (/32): 0x0000F28D
(27) sp_abt (/32)
(28) lr_abt (/32)
(29) sp_und (/32)
(30) lr_und (/32)
(31) cpsr (/32): 0x000001D3
(32) spsr_fiq (/32)
(33) spsr_irq (/32)
(34) spsr_svc (/32): 0x000001D3
(35) spsr_abt (/32)
(36) spsr_und (/32)
(37) sp (/32)
(38) lr (/32)
(39) sp_mon (/32)
(40) lr_mon (/32)
(41) spsr_mon (/32)
(42) d0 (/64)
(43) d1 (/64)
(44) d2 (/64)
(45) d3 (/64)
(46) d4 (/64)
(47) d5 (/64)
(48) d6 (/64)
(49) d7 (/64)
(50) d8 (/64)
(51) d9 (/64)
(52) d10 (/64)
(53) d11 (/64)
(54) d12 (/64)
(55) d13 (/64)
(56) d14 (/64)
(57) d15 (/64)
(58) d16 (/64)
(59) d17 (/64)
(60) d18 (/64)
(61) d19 (/64)
(62) d20 (/64)
(63) d21 (/64)
(64) d22 (/64)
(65) d23 (/64)
(66) d24 (/64)
(67) d25 (/64)
(68) d26 (/64)
(69) d27 (/64)
(70) d28 (/64)
(71) d29 (/64)
(72) d30 (/64)
(73) d31 (/64)
(74) fpscr (/32)

Someone have suggestions ? Am i doing something wrong ?

Thanks in advice for the halp,

I congratulate ST for this new product and the new wiki.st.com: grate documentation, grate work !

P.S. Forgive my english

[AntonioST]

Buongiorno Giacomo,

I succeed to replicate your issue but only after I set ST-Link at extremely low speed, using the OpenOCD command "adapter_khz 1".

In such conditions I get, as in your case, both:

• 3~5 second delay for the "step" command
• timeout error.

I'm surprise you get such issues while your OpenOCD reports "adapter speed: 5000 kHz".

The only explanation I have is that your ST-Link is connected to the PC on a USB interface that is extremely busy. Maybe connected through some USB HUB, shared with other devices that consume all the USB bandwidth.

In such case, try to use a dedicated USB port for ST-Link.

Or maybe you are running Linux in a virtual machine and the USB speed is compromised by an incorrect configuration.

[Giacomo]

Grazie Antonio,

As you suspected the problem is caused by the USB: i'm working with VirtualBox and changing this setting solve.

I had a 3.0 USB hardware hub but selected 2.0 Controller. I wasn't aware about such a slow speed with 2.0 selected.

Now the debugger is much more faster and seams to work. I experimented some strange behaviours but i think i only need to deepen the tool and have a better overall understanding. More tests during the week.

Thanks for the help !

[AntonioST]

Perfect! Thanks for your confirmation.

If you need to "tune" the USB configuration, you can use the same tool OpenOCD to get some feedback.

For ST-Link performance it's important the "round-trip-time" on the USB interface, so you can measure the number of elementary transfers that occur per second.

In a telnet console you can run:

for {set i 0; set t [expr [ms] + 5000]} {$t > [ms]} {incr i} {stm32mp15x.ap1 mem2array v 32 0xe0080000 1}; echo [expr $i / 5]

This command will loop for 5 seconds, count the number of memory read (to the debug romtable) and print the reads/second.

On my old PC I know I have different performances on two different USB ports. Using the command above I get respectively 1479 read/s on the slow port and 2515 read/s on the fast one.

This is not the best you can get, but if you get much much lower values, then you have room to improve.

Testing different USB configurations and launching every time OpenOCD plus telnet plus command plus quit ... can be annoying.

For testing purpose, you can consider using the single Linux command that does everything in one shot:

./bin/openocd -s ./share/openocd/scripts/ -f board/stm32mp15x_dk2.cfg -c 'init; for {set i 0; set t [expr [ms] + 5000]} {$t > [ms]} {incr i} {stm32mp15x.ap1 mem2array v 32 0xe0080000 1}; echo [expr $i / 5]; shutdown'

[Giacomo]

Hi, i'm here with news

First of all i have to say the result i get from the test you suggested. With my virtual box configuration i get a score about 1200, sometimes 950. Because i wasn't sure if it was enough i set up an hdd with a full Ubuntu 16.04 installation. Grate improvement: now i get near 4000. Pretty sure it's not gonna be an usb speed issue anymore.

After playing around with SPL debugging, two new issue comes out:

Those are my gdb init files:

setup.gdb=

# Disables confirmation requests
set confirm off
 
# Connection to the host gdbserver port for Cortex-A7 SMP
target remote localhost:3334
 
# Configure GDB for OpenOCD
set remote hardware-breakpoint-limit 6
set remote hardware-watchpoint-limit 4
 
# Switch to Core0
monitor cortex_a smp_gdb 0
stepi
monitor cortex_a smp_gdb -1
# No SMP, only core 0 for the moment. We'll re-enable it in kernel
monitor cortex_a smp_off
 
source reset.gdb

reset.gdb=

# Reset the system and halt in bootrom in case of attach at boot
monitor reset
monitor sleep 2000
monitor reset halt
monitor gdb_sync
stepi
 
# Switch to Core0 after reset halt
monitor cortex_a smp_gdb 0
stepi
monitor cortex_a smp_gdb -1
# No SMP, only core 0 for the moment. We'll re-enable it in kernel
monitor cortex_a smp_off
 
symbol-file ./build/u-boot/spl/u-boot-spl
 
# Run to the program start
thbreak board_init_r
c

1) I get an error during the first 'reset.gdb' run that seams to be harmless but i'm not sure. You can see it on line 70-71 of the openocd output. I don't get the error if i source again reset.gdb inside the gdb session.

2) When i try to break on an instruction that isn't 4byte aligned i get an 'Alignment fault' by openocd (deducted by dfsr). I'm compiling uboot adding those cflags '-fno-schedule-insns -fno-schedule-insns2' (as explaned here https://www.denx.de/wiki/view/DULG/DebuggingTricks) to prevent GDB from jumping around (optimization unchanged -Os). Below an example when i try to break and run to spl_init call.

openocd output=

giacomo@giacomo-ubuntu:~/Scrivania/workspace/stm32mp1$ source spt-run-openocd.shOpen On-Chip Debugger 0.10.0+dev-00546-g1afec4f-dirty (2019-02-01-13:33)
Licensed under GNU GPL v2
For bug reports, read
	http://openocd.org/doc/doxygen/bugs.html
cortex_a interrupt mask on
cortex_a interrupt mask on
cortex_a domain access control fixup on
cortex_a domain access control fixup on
3333
none separate
adapter speed: 5000 kHz
adapter_nsrst_assert_width: 200
adapter_nsrst_delay: 200
none srst_pulls_trst
srst_only srst_pulls_trst srst_gates_jtag srst_open_drain connect_deassert_srst
Info : Listening on port 6666 for tcl connections
Info : Listening on port 4444 for telnet connections
Info : STLINK V2J33S25 (API v2) VID:PID 0483:3752
Info : using stlink api v2
Info : Target voltage: 3.234506
Info : clock speed 5000 kHz
Info : SRST line released
Info : stlink_connect(connect)
Info : SWD DPIDR 0x6ba02477
Info : stm32mp15x.cpu0: hardware has 6 breakpoints, 4 watchpoints
Info : stm32mp15x.cpu1: hardware has 6 breakpoints, 4 watchpoints
Info : stm32mp15x.cpu1 rev 5, partnum c07, arch f, variant 0, implementor 41
Info : stm32mp15x.cpu1 cluster 0 core 1 multi core
Info : Listening on port 3334 for gdb connections
Info : Listening on port 3333 for gdb connections
srst_only srst_pulls_trst srst_gates_jtag srst_open_drain connect_deassert_srst
Info : accepting 'gdb' connection on tcp/3334
Info : stm32mp15x.cpu0 rev 5, partnum c07, arch f, variant 0, implementor 41
Info : stm32mp15x.cpu0 cluster 0 core 0 multi core
target halted in Thumb state due to debug-request, current mode: Supervisor
cpsr: 0x600001f3 pc: 0xdfc8e900
MMU: enabled, D-Cache: enabled, I-Cache: enabled
Info : New GDB Connection: 1, Target stm32mp15x.cpu0, state: halted
gdb coreid 0 -> 0
gdb coreid 0 -> -1
Info : SRST line asserted
Info : SRST line released
Info : stlink_connect(connect)
Info : SWD DPIDR 0x6ba02477
Info : Deferring arp_examine of stm32mp15x.cpu2
Info : Use arp_examine command to examine it manually!
Info : Deferring arp_examine of stm32mp15x.ap2
Info : Use arp_examine command to examine it manually!
ap 0 selected, csw 0x10006000
Info : SRST line asserted
Info : SRST line released
Info : stlink_connect(connect)
Info : SWD DPIDR 0x6ba02477
Info : stm32mp15x.cpu0 rev 5, partnum c07, arch f, variant 0, implementor 41
target halted in ARM state due to breakpoint, current mode: Supervisor
cpsr: 0x000001d3 pc: 0x2ffc2690
MMU: disabled, D-Cache: disabled, I-Cache: disabled
ap 0 selected, csw 0x10006000
pc (/32): 0x2FFC2690
pc (/32): 0x2FFC2694
Info : stm32mp15x.cpu1 rev 5, partnum c07, arch f, variant 0, implementor 41
target halted in Thumb state due to debug-request, current mode: Supervisor
cpsr: 0x800001f3 pc: 0x0000060e
MMU: disabled, D-Cache: disabled, I-Cache: enabled
Info : Deferring arp_examine of stm32mp15x.cpu2
Info : Use arp_examine command to examine it manually!
Info : Deferring arp_examine of stm32mp15x.ap2
Info : Use arp_examine command to examine it manually!
gdb coreid 0 -> 0
Error: data abort at 0x00888750, dfsr = 0x00000008
Error: data abort at 0x00888750, dfsr = 0x00000008
gdb coreid 0 -> -1
Info : stm32mp15x.cpu0 rev 5, partnum c07, arch f, variant 0, implementor 41
Error: data abort at 0x2ffc3086, dfsr = 0x00000001
Error: can't add breakpoint: unknown reason

gdb output=

giacomo@giacomo-ubuntu:~/Scrivania/workspace/stm32mp1$ ./gcc-linaro-7.4.1-2019.02-x86_64_arm-linux-gnueabihf/bin/arm-linux-gnueabihf-gdb -x setup.gdb
GNU gdb (Linaro_GDB-2019.02) 8.2.1.20190122-git
Copyright (C) 2018 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Type "show copying" and "show warranty" for details.
This GDB was configured as "--host=x86_64-unknown-linux-gnu --target=arm-linux-gnueabihf".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
 <http://www.gnu.org/software/gdb/documentation/>.
 
For help, type "help".
Type "apropos word" to search for commands related to "word".
warning: No executable has been specified and target does not support
determining executable automatically. Try using the "file" command.
0xdfc8e900 in ?? ()
gdb coreid 0 -> 0
0xdfc8e900 in ?? ()
gdb coreid 0 -> -1
SRST line asserted
SRST line released
stlink_connect(connect)
SWD DPIDR 0x6ba02477
Deferring arp_examine of stm32mp15x.cpu2
Use arp_examine command to examine it manually!
Deferring arp_examine of stm32mp15x.ap2
Use arp_examine command to examine it manually!
ap 0 selected, csw 0x10006000
SRST line asserted
SRST line released
stlink_connect(connect)
SWD DPIDR 0x6ba02477
stm32mp15x.cpu0 rev 5, partnum c07, arch f, variant 0, implementor 41
target halted in ARM state due to breakpoint, current mode: Supervisor
cpsr: 0x000001d3 pc: 0x2ffc2690
MMU: disabled, D-Cache: disabled, I-Cache: disabled
ap 0 selected, csw 0x10006000
pc (/32): 0x2FFC2690
pc (/32): 0x2FFC2694
stm32mp15x.cpu1 rev 5, partnum c07, arch f, variant 0, implementor 41
target halted in Thumb state due to debug-request, current mode: Supervisor
cpsr: 0x800001f3 pc: 0x0000060e
MMU: disabled, D-Cache: disabled, I-Cache: enabled
Deferring arp_examine of stm32mp15x.cpu2
Use arp_examine command to examine it manually!
Deferring arp_examine of stm32mp15x.ap2
Use arp_examine command to examine it manually!
 
Program received signal SIGINT, Interrupt.
0x2ffc2694 in ?? ()
gdb coreid 0 -> 0
0x2ffc2694 in ?? ()
gdb coreid 0 -> -1
Hardware assisted breakpoint 1 at 0x2ffc3050: file /home/giacomo/Scrivania/works--Type <RET> for more, q to quit, c to continue without paging--
pace/stm32mp1/u-boot/common/spl/spl.c, line 468.
stm32mp15x.cpu0 rev 5, partnum c07, arch f, variant 0, implementor 41
 
Temporary breakpoint 1, board_init_r (dummy1=0x2fffcf20, dummy2=0)
 at /home/giacomo/Scrivania/workspace/stm32mp1/u-boot/common/spl/spl.c:468
468	{
(gdb) break spl.c:492
Breakpoint 2 at 0x2ffc3086: file /home/giacomo/Scrivania/workspace/stm32mp1/u-boot/common/spl/spl.c, line 492.
(gdb) c
Continuing.
Warning:
Cannot insert breakpoint 2.
Cannot access memory at address 0x2ffc3086
 
Command aborted.
(gdb)

Who could be responsible for this error ? maybe openocd stlink backend or a bad configuration ?

I really would like to have a reliable debugging tool.

Thanks

[AntonioST]

Buongiorno Giacomo,

the first issue you have found is caused by GDB.

Every time GDB halts the targets, it checks if R11 points to a stack-frame. But R11 is used as general purpose register and its value is not always pointing to a valid memory address.

The issue can be mitigated by adding the command

       set arm abi AAPCS

in the GDB script, just before the command

       target remote :3334

The sample GDB script in ST wiki has also been updated to include this command.

The second issue you have found is a bug in OpenOCD.

There is a fix sent to the OpenOCD community in

       http://openocd.zylin.com/5138

ST will include this fix in next release of openstlinux.

In mean time, as temporarily workaround, I suggest you to prefer using HW breakpoints instead of SW breakpoints. This can be easily achieved adding the command

       monitor gdb_breakpoint_override hard

in your GDB script, after the command

       target remote :3334

and before you set any breakpoint. Be aware that you can only set 6 HW breakpoints at the same time.

Alternatively, you can re-build OpenOCD adding the patch above. How to rebuild an application in openstlinux is explained in

       https://wiki.st.com/stm32mpu/wiki/OpenEmbedded_-_devtool#modify_an_existing_application_or_library_managed_by_a_recipe_on_which_you_have_the_ownership