ROM Region Merging is not padding correctly #11140
Comments
DrynnBavis
changed the title
|
I can't comment on the padding issue at the moment -- can you post both of your 'mbed_app.json' files for the bootloader and the application? Also post the 'custom_target.json' if you have one. As for See this diagram below showing the reset vector format on Cortex M4 devices:
So in your second case, the bootloader/application merging seems to be working -- the The reason your program is hanging is because the initial program counter pointer is unaligned -- that is the value Unfortunately this kind of bug pops up from time to time, so it's good to know the root cause of this issue. This is certainly a problem with the online compiler tools... Until someone from Mbed addresses this issue I would recommend trying to get set up with Mbed-CLI and building offline, that way you have a lot more control over the build and various tool/software versions. |
@AGlass0fMilk I've edited my OP to include the 3 files you asked for. Just as a reminder, the I had a feeling that To print this, I was c-style casting it to a char pointer like so Maybe I'm just printing this wrong? Or does the logic for
I am indeed using Mbed-CLI and building offline. What do you recommend I do to fix this? |
|
I recall ST parts having memory regions all over the place versus 1 contiguous flash region.
Gut feel is you should confirm those sectors that seem to be similar. |
@loverdeg-ep tagged you on the other issue with an update to my understanding of the sectors for this chip. |
|
To verify the actual value programmed into the device for the initial program counter/reset vector you can use a hex editor to view the binary. I recently had a situation where the build tools were putting an illegal address in the reset vector location... Try opening the binary up with something like https://hexed.it/?hl=en and take a screenshot. |
What address is this exactly? Sorry. I've got the file open in HexEd.it just not sure what address you want to see. |
|
The beginning of the application. So whatever that address is that the bootloader jumps to. |
|
|
@AGlass0fMilk Something else interesting, I've changed the target on my bootloader and main application to an older MCU I had on hand that I previously had success with, and this was the Again, here the program counter is pointing to an odd number, and thus seemingly unaligned. However the program does seem to be working fine. My blinky application successfully booted from the bootloader. Nothing changed but the target. So my hypothesis is that these addresses might just be random garbage at the point of time when I'm printing them. This odd |
|
Just a bump for the new week
How would I go about confirming that vtor is indeed pointing to the correct address of the vector table? I guess I'm asking how would I check the value in vtor after the bootloader boots, and then where would I find what the expected address of the vector table should be? |
|
@DrynnBavis The vector table is at address 0 by default. Where it gets relocated to must be aligned correctly. As I recall Mbed OS 5 generally ends up relocating the vector table to somewhere in SRAM See the following from http://infocenter.arm.com/help/topic/com.arm.doc.dui0553b/DUI0553.pdf
|
|
@kjbracey-arm may or may not be able to cut to the heart of the topic for you, upon returning from leave. |
|
@loverdeg-ep thanks for the response. I was looking at this datasheet yesterday, and I'm pretty convinced it is indeed being relocated to SRAM. I'm just frustrated in trying to figure out how to debug this properly. What I've learned is that the bootloader has it's own v table and after finishing (i.e. I call What exactly is expected to happen between my calling of |
|
@DrynnBavis are you able to step debug? If so, have you loaded the .elf files for both your bootloader and application? Edit: have you loaded the .elf's into gdb |
|
@40Grit I haven't tried yet. For some reason when I build with the debug profile I get this crash from |
|
Sounds like it’s trying to write debug/crash dump info to the `stdout` file handle during a critical section.
On Aug 7, 2019, at 6:30 PM, Brynn Davis <[email protected]<mailto:[email protected]>> wrote:
@40Grit<https://github.com/40Grit> I haven't tried yet. For some reason when I build with the debug profile I get this crash from __disable_irq() within mbed_start_application() from mbed_application.c:
++ MbedOS Error Info ++
Error Status: 0x8001012F Code: 303 Module: 1
Error Message: Error - writing to a file in an ISR or critical section
Location: 0x8006BE5
Error Value: 0x1
Current Thread: main Id: 0x200012B0 Entry: 0x8007AFF StackSize: 0x1000 StackMem: 0x20001C18 SP: 0x200028F8
For more info, visit: https://mbed.com/s/error?error=0x8001012F&tgt=TRACK_PILOT2
-- MbedOS Error Info --
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub<#11140?email_source=notifications&email_token=AACVSJF5XRX4XNZT25DNFMTQDMBEPA5CNFSM4IILZH32YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD3ZEQXY#issuecomment-519194719>, or mute the thread<https://github.com/notifications/unsubscribe-auth/AACVSJGBSQBQ3FWTX3GNZG3QDMBEPANCNFSM4IILZH3Q>.
|
|
Hey so I removed the print statements and as you suspected it solved the crash I was seeing on the debug profile. Finally, I was able to backtrace the crash and I found this: Not at all sure if there's anything helpful here, but I feel that this line: |
|
Can you check the debug serial output and post that here? That stack trace is just showing there was a call to the Mbed fault handler that prints out debug information.
On Aug 7, 2019, at 7:31 PM, Brynn Davis <[email protected]<mailto:[email protected]>> wrote:
Hey so I removed the print statements and as you suspected it solved the crash I was seeing on the debug profile. Finally, I was able to backtrace the crash and I found this:
(gdb) bt
#0 0x08006ce4 in write (fildes=2, buf=0x2004fec8, length=0) at libs/mbed-os/platform/mbed_retarget.cpp:677
#1 0x0800602c in mbed_error_puts (str=0x2004fec8 "\n++ MbedOS Fault Handler ++\n\nFaultType: ")
at libs/mbed-os/platform/mbed_board.c:97
#2 0x08006010 in mbed_error_vprintf (format=0x8018de4 "\n++ MbedOS Fault Handler ++\n\nFaultType: ", arg=...)
at libs/mbed-os/platform/mbed_board.c:71
#3 0x08005fca in mbed_error_printf (format=0x8018de4 "\n++ MbedOS Fault Handler ++\n\nFaultType: ")
at libs/mbed-os/platform/mbed_board.c:55
#4 0x08005aee in mbed_fault_handler (fault_type=16, mbed_fault_context_in=0x8018de0 <mbed_fault_context>)
at libs/mbed-os/platform/TARGET_CORTEX_M/mbed_fault_handler.c:48
#5 0x08005ad6 in Fault_Handler () at except.S:187
Backtrace stopped: previous frame identical to this frame (corrupt stack?)
Not at all sure if there's anything helpful here, but I feel that this line:
0x08005aee in mbed_fault_handler (fault_type=16, mbed_fault_context_in=0x8018de0 <mbed_fault_context>) might help give us some context in what's going on here. I just don't know how to interpret it.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub<#11140?email_source=notifications&email_token=AACVSJEHNT2VRINTKZSEEL3QDMIGNA5CNFSM4IILZH32YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD3ZJ6TI#issuecomment-519216973>, or mute the thread<https://github.com/notifications/unsubscribe-auth/AACVSJDRV2LBVP7FXF2U3OLQDMIGNANCNFSM4IILZH3Q>.
|
|
@DrynnBavis Are you printing to uart, SWO, or semihosting? |
|
@DrynnBavis Also, if you can step debug down at the assembly level I would see if there is a particular instruction that your fault is occurring on. |
|
@AGlass0fMilk The fault error never actually prints to serial. @loverdeg-ep I'm using UART. |
|
Just stepping through |
|
Check value of VTOR as well. |
|
@loverdeg-ep I read VTOR while I was stepping through |
|
@AGlass0fMilk @loverdeg-ep after stepping one more time I get the crash: It seems to be this assembly code that's faulting: How does one go about stepping through these line by line? |
|
Are you using command line GDB? |
yes |
|
I don't know off hand but you'll need the disassembly. Might need to make it an output of your build. |
|
Took a while but here it is @40Grit: Before crashing -> After crashing -> And NVIC also changed. After crashing -> |
|
Can you do the same for the working part. I guess just dump the peripherals that seem relevant. it would be good to see a side by side diff of the dumps using a program like win merge or kdiff etc. |
Assuming you're referring to the disco board, unfortunately I don't have a JTAG header soldered onto this board so I can't do that right now. Put an order in for one though. |
|
disco has onboard st-link i thought. you should be able to use pyocd, or some st tool to do the dump. |
|
Just in case cc @ARMmbed/team-st-mcd |
|
@40Grit Got GDB going on the disco board finally. So this is the same spot we were crashing previously with the register info of the disco board (not crashing) |
|
can you show it as a side by side diff? |
|
screenshot of, kdiff, git diff, winmerge. MS paint with highlighted differences. any of the above |
|
@40Grit |
|
you said these are the same parts except for package? |
@40Grit Jut realised I was on line 16 rather than 6 in the disco debugging when I printed the registers, see comment above with new pic |
|
same exact binary flashed to both parts?!? |
No, compiled for and flash to different targets (my custom and the disco) but both with the exact same chip. Technically they have different packages (disco uses STM32F413ZH, our target uses STM32F413RH), however both target definitions have |
|
I assume they share datasheets? |
I certainly hope so. The one I've been using -> https://www.st.com/content/ccc/resource/technical/document/reference_manual/group0/81/ea/88/1f/97/9e/4a/d0/DM00305666/files/DM00305666.pdf/jcr:content/translations/en.DM00305666.pdf |
|
that is the reference manual. wanted datasheet to see the packages etc. we should move this offline at this point. contact me at my work email. |
|
@40Grit Link to pdf is here: https://www.st.com/resource/en/datasheet/stm32f413rh.pdf EDIT: fixed link |
|
Wrong datasheet linked but I got the right one. I'd need to sit down and re-evaluate all the information at this point. I recommend pulling all the data you have collected together in an organized fashion and updating your main post. |
Hey @40Grit just found something really interesting. I've gone and stripped the main.cpp of my bootloader to this: And this is actually working, remarkably. To be explicit, I mean I've successfully booted from bootloader and into my main application (I know this because I see a prints from my main application in my terminal). So now I've started adding back the other components of my bootloader and I've found two things are causing the previous hanging bootloader behaviour once added back in:
Results from debugging around
This is what I mean when I say I believe the bootloader has finished and booted to the main app: I'm starting to think there's something in the main app that's actually causing the faults here. But this conflicts with the fact that it's my changes in the bootloader that are causing this to fail. I'd like to know (or if someone could direct me to some good literature) what is going on between |
|
I'd need to dig in and simmer at this point. Is this a standard bootloader based on mbed-bootloader? Also, if you want to see past the bootloader in gdb, you should be able to load the .elf of your application into gdb via command line alongside the one already loaded for the bootloader. You may or may not need to set a source location for the application as well. |
Not sure if there's a strict definition of an "mbed-bootloader" but I'd say it's pretty standard. I'm using restrict size and I built it from their example here .
Do you mean this in the context of loading a second elf for my blinky (main) application? Or is this unrelated to that, maybe an mbed_app target setting? Also if I haven't said it yet, thank you very much for you help this far @40Grit. |
|
may need to set source location when loading 2nd elf |
Will each of these lines individually cause the error, or do they both need to be present? You might need to de-initialize the peripherals before booting into the main app. What happens if you call |
each, no need for both, but both will also error. Currently drilling into step debugging with a block device present. Using the tip from @40Grit about the second elf file, I've been able to read what happens after mbed start. It's failing in
Yeah so I was thinking of this earlier, but the thing is that I never had to do this earlier. What would change about it now? Additionally, even without actually calling |
|
@40Grit one more thing you might be able to help me with here. I've got it down to a fault being caused within an assembly source I'm guessing I can't just load an assembly source file into GDB like I could with the elf files previously... I did find this specific assembly source file here, but I'm cautious of the line numbers. How does assembly handle whitespace/comments? Do they count as line numbers or no? |
|
Your build should be able to generate an assembly listing with your build. I don't know for sure but I would think you can use it to step through the assembly and it would retain symbols. |
|
Closing this issue as it's moved pretty far from the original issue I identified about incorrect padding. Taking the summary from all this debugging and effort and moving it to a new issue with a better title: #11205 |
|
Also to address your comment @40Grit I was unable to get gdb to go into the source files. I even tried adding them using |
Uh oh!
There was an error while loading. Please reload this page.
Description
I've got a managed bootloader application of which I've set the
target.restrict_sizeto a value of0x40000. When I use this binary as a bootloader for my main application, I get the following in my build logs:This is problematic because the bootloader calls
mbed_app_start(POST_APPLICATION_ADDR)which has a value of0x8040000. Essentially, the linker is placing my main program at0x8010000while the bootloader is actually booting into0x8040000. The behaviour of this is my application just "hangs" after the bootloader completes.Note that I've also printed the address, sp, and pc, before booting the assembly code within
start_new_application()is called, and I got the following results:and after I set
target.restrict_sizeback down to something smaller like0x1000, I see something of more expected behaviour (but still seems to hang):Is this a bug on mbed's side? Or do I need to do something in my main program's config to pad out until
0x8040000?As requested these are contents of your requested files @AGlass0fMilk
bootloader's mbed_app.json:
main app's mbed_app.json:
target definition:
Issue request type
Target: custom MCU with STM32F413RH processor
Toolchain: GCC_ARM 8.2.1
Tool: mbed-cli
Vers: cfa7938 (HEAD, tag: mbed-os-5.12.2)
The text was updated successfully, but these errors were encountered: