CubeMX LoraWAN support

[Niclas Hedhman]

Is there any explanation to how to use production DevEUI, JoinEUI, AppKey and NetwKey, rather than the hardcoded se-identity.h?

I have searched all over, application notes, youtube and even studied examples and the actual code, but unable to find out how it is intended to be put together.

I tried to populate the LoRaMacNvmData_t passed in OnRestoreContextRequest in lora_app.c, but it is some "BackupContexts" being passed, that seems to be ignored (for this purpose).

Am I expected to rely on  

FLASH_IF_Read(nvm, LORAWAN_NVM_BASE_ADDRESS, nvm_size);

and flash from an external computer, rather than generate it internal to the MCU and report it to external system during test/calibration?

[Issamos]

Hello @Niclas Hedhman 

My advice is to use the end_node exemple and try a deep debug on it to now how it exactly works.

Also this application note and this video may give some help.

Best regards.

II

[Niclas Hedhman]

@Issamos, thanks but I am WAY beyond those stages. The endnode sample, all videos I have found and the application note all assume that you are experimenting and having the keys in se-identity.h. That is not a doable if you are planning of making thousands of devices to "other people".

My question is more along the lines of; "Has ST forgotten about how to make this commercially viable, with the LoraWAN CubeMX middleware?" and/or ignored the question all together. And since the structure is very different from the Semtech stack (which has no CubeMX at all, and its own device driver libraries and everything) it is not really that viable to try to merge Semtech on every revision, and therefor I have not dug too deep into Semtech's stuff.

From my PoV, hooks are either missing, deeply hidden and/or not documented. And I am at wit's end on how progress from "prototype" to "production".

[Andrew Larkin]

I am hitting the same conundrum.

On a previous project (LoRaWAN 1.0.2), I reserved a page of flash to store DevEUI, keys, etc that could be programmed during production and just picked up the values.  This is a bit "hacky" but worked.

If KMS is not enabled, the keys from se-identity.h are placed in seNvmInit{}, which is then linked into memory section defined by 
SOFT_SE_PLACE_IN_NVM_START as __attribute__((section(".USER_embedded_Keys")))

However, this memory section isn't defined in the .ld file, so I can't predict where it will be placed.

A part of the solution is to modify the .ld file to reserve space for the USER_embedded_Keys section.

I am using the STM32WLE5CCUX_FLASH.ld file here.

I changed the /* Memories definition */ to reduce the FLASH section length and add a KEYS section

/* Memories definition */
MEMORY
{
RAM (xrw) : ORIGIN = 0x20000000, LENGTH = 64K
RAM2 (xrw) : ORIGIN = 0x10000000, LENGTH = 32K
FLASH (rx) : ORIGIN = 0x08000000, LENGTH = 252K
KEYS (rx) : ORIGIN = 0x0803F000, LENGTH = 4K
}

Then added thisto the end of the ld file, just before /DISCARD/
/* USER Embedded Keys data into "KEYS" Rom type memory */
.embeddedKeys :
{
. = ALIGN(4);
*(.USER_embedded_Keys)
. = ALIGN(4);
} >KEYS

This at least gets the seNvmInit structure (containing the keys) into a known location, but I have still to figure out which bytes are what.

One thing that bothers me though is prospect of being clobbered by the Context Management support. The default implementations of OnStoreContextRequest() and OnRestoreContextRequest() assume ownership of flash memory at LORAWAN_NVM_BASE_ADDRESS which is

/**
* @brief LoRaWAN NVM Flash address
* @note last 2 sector of a 128kBytes device
*/
#define LORAWAN_NVM_BASE_ADDRESS ((void *)0x0803F000UL)

This, to me, is bad coding behaviour as the address/memory allocation for LORAWAN_NVM_BASE_ADDRESS should also be defined within the ld file

I am dredding what all this might do to a future task which is to figure out how flash is managed for FUOTA.

[Andrew Larkin]

I should also have mentioned that on the previous project we were using stm32cubeprogrammer to program the devices and using command line options to overwrite DevEUI and Key values directly into the target memory locations.

[Niclas Hedhman]

So, I am setting up linker script as @Andrew Larkin suggested and going to use an external tool to populate the Flash.

MEMORY
{
    RAM (xrw) : ORIGIN = 0x20000000, LENGTH = 64K
    FLASH (rx) : ORIGIN = 0x08000000, LENGTH = 160K
    FLASH_FORTH (rwx) : ORIGIN = 0x08028000, LENGTH = 92K
    FLASH_NVM (rwx) : ORIGIN = 0x0803F000, LENGTH = 2K
    LORAWAN_CONTEXT (rwx) : ORIGIN = 0x0803F800, LENGTH = 2K
}

.embeddedKeys :
{
    *(.USER_embedded_Keys)
    LORAWAN_NVM_BASE_ADDRESS = .;
    FILL(0xFFFFFFFF);
    . = ORIGIN(LORAWAN_CONTEXT) + LENGTH(LORAWAN_CONTEXT) - 1;
    BYTE(0xFF)
} >LORAWAN_CONTEXT

 

That also takes care of the LORAWAN_NVM_BASE_ADDRESS to be from linker script rather than buried in lora-app.c.

 

And then I need to get hold of the memory layout, so this printAddresses() was addded;

 

void printAddresses(const SecureElementNvmData_t *ctx)
{
 MW_LOG( TS_OFF, VLEVEL_M, "###### seNvmInit: 0x%X\r\n", ctx );
 MW_LOG( TS_OFF, VLEVEL_M, "###### DevEUI: 0x%X\r\n", ((uint32_t)&ctx->SeNvmDevJoinKey.DevEui) - ((uint32_t)ctx) );
 MW_LOG( TS_OFF, VLEVEL_M, "###### JoinEui: 0x%X\r\n", ((uint32_t)&ctx->SeNvmDevJoinKey.JoinEui) - ((uint32_t)ctx) );
 MW_LOG( TS_OFF, VLEVEL_M, "###### DevAddrOTAA: 0x%X\r\n", ((uint32_t)&ctx->SeNvmDevJoinKey.DevAddrOTAA) - ((uint32_t)ctx) );
 MW_LOG( TS_OFF, VLEVEL_M, "###### DevAddrABP: 0x%X\r\n", ((uint32_t)&ctx->SeNvmDevJoinKey.DevAddrABP) - ((uint32_t)ctx) );
 for( int i=0; i<NUM_OF_KEYS; i++ )
 {
 MW_LOG(TS_OFF, VLEVEL_M, "###### KeyList[%02d].KeyID: 0x%X\r\n", i, ((uint32_t) &ctx->KeyList[i].KeyID) - ((uint32_t) ctx));
 MW_LOG(TS_OFF, VLEVEL_M, "###### KeyList[%02d].KeyValue: 0x%X\r\n", i, ((uint32_t) &ctx->KeyList[i].KeyValue) - ((uint32_t) ctx));
 }
 MW_LOG( TS_OFF, VLEVEL_M, "###### Crc32: 0x%X\r\n", ((uint32_t)&ctx->Crc32) - ((uint32_t)ctx));
}

 

 

And that gave me;

######            seNvmInit: 0x803F800
######               DevEUI: 0x0
######              JoinEui: 0x8
######          DevAddrOTAA: 0x10
######           DevAddrABP: 0x14
######    KeyList[00].KeyID: 0x18
###### KeyList[00].KeyValue: 0x19
######    KeyList[01].KeyID: 0x29
###### KeyList[01].KeyValue: 0x2A
######    KeyList[02].KeyID: 0x3A
###### KeyList[02].KeyValue: 0x3B
######    KeyList[03].KeyID: 0x4B
###### KeyList[03].KeyValue: 0x4C
######    KeyList[04].KeyID: 0x5C
###### KeyList[04].KeyValue: 0x5D
######    KeyList[05].KeyID: 0x6D
###### KeyList[05].KeyValue: 0x6E
######    KeyList[06].KeyID: 0x7E
###### KeyList[06].KeyValue: 0x7F
######    KeyList[07].KeyID: 0x8F
###### KeyList[07].KeyValue: 0x90
######    KeyList[08].KeyID: 0xA0
###### KeyList[08].KeyValue: 0xA1
######    KeyList[09].KeyID: 0xB1
###### KeyList[09].KeyValue: 0xB2
######    KeyList[10].KeyID: 0xC2
###### KeyList[10].KeyValue: 0xC3
######                Crc32: 0xD4

Now I will continue to create tool for Linux to create, flash and document keys.

Or should I do that in main(), before initializing LoraWAN?? Hmmm...

[Andrew Larkin]

I am baffled as to why this works. I tested it and it does seem to work, but it is black magic to me how a definition in the ld linker script file can override a pre-processor #define constant.

[Niclas Hedhman]

linker script; I removed the #define and replaced it with a "extern" declaration. Not ideal, I know.

[Niclas Hedhman]

That basic/original question remain; @stmicro hasn't provided any guidance that I have been able to find, and I think it needs to be improved.

[Andrew Larkin]

Maybe I don't understand the question.

The chip provides an IEEE 802-2001-compliant UID64 to use as DevEUI.

Are you expecting the chip/library to also generate the AppKey/NwkKey as well?

I suppose it should be possible - you have two sources of chip-specific entropy to derive keys from: the UID64 and the 96-bit unique die identifier. Mangle these together in some way to form a 128-bit key.

You said you tried to use SecureElementSetXXX() functions under /* USER CODE BEGIN LoRaWAN_Init_2 */

I was able to set the key successfully in that section with

uint8_t newNwkKey[] = {0x00,0x11,0x22,0x33,0x44,0x55,0x66,0x77,0x88,0x99,0xAA,0xBB,0xCC,0xDD,0xEE,0xFF};

LmHandlerSetKey( NWK_KEY, newNwkKey );

[Niclas Hedhman]

The question was "How is it INTENDED to generate EUIs and Keys at the device?", and "Is that documented somewhere I can't find?"

I don't expect the library to generate it, but to provide hook(s) to where they intended that such things are done.

SecureElementSetXXX(); Strange... I concluded that the "struct updated" was different addresses than "struct used", and wasn't copied. Perhaps the context management has saved another struct during my many experiments, and using that. And this is where good library design would have "signaled" what is going on. For instance; if there was a RequestInitialKeys() and similar callback, and those are not called, then I would know immediately that some saved values was used.

 

[Andrew Larkin]

Digging through all this to better understand what is going on...

When CONTEXT_MANAGEMENT_ENABLED == 1

There are four areas of memory that we care about:

1. seNvmInit [section(".USER_embedded_Keys") in Flash]
2. Nvm [section(".bss.LW_NVM_RAM") uninitialised in RAM]
3. NvmBackup [section(".bss.LW_NVM_BACKUP_RAM") uninitialised in RAM]
4. Nvm Context [(@0x0803f000) in Flash]

The DevEUI, JoinEUI, AppKey/NwkKey values (the "Keys") we care about for production programming of the device are located in seNvmInit.

If the linker script is "original" (as generated by the ioc file), the location in flash of the seNvmInit is undefined because the .USER_embedded_Keys memory section is not defined.

When the firmware first starts up, MX_LoRaWAN_Init() calls LoRaWAN_Init() calls LmHandlerConfigure() calls LoRaMacInitialization() calls SecureElementInit() which copies the keys from flash (seNvmInit) to uninitialised RAM (Nvm)

Nvm structure in ram is the working copy of the Keys.

On start up, it looks like a check is done to see if Nvm is valid. If a chip reset has happened, this preserves the context.

If Nvm is not valid, a copy is made from the NvmBackup RAM. This backup only seems to be updated as a part of LoRaMacHalt(). I am guessing this is like a "last known good" configuration.

If the NvmBackup is not valid, the firmware tries to restore from a context saved in flash. It seems the operation to store the Keys in Nvm context flash is a deliberate act by the application with a call to StoreContext().

If everything else has failed, then the Nvm is initialised from the seNvmInit original set of Keys.

All this seems to be pretty reasonable.

The problems though are in the management of the memory allocations.

1. The location of seNvmCtx is undefined so it is necessary to add an explicit allocation mapping in the linker script for .USER_embedded_Keys. Once this is done, precise memory locations for the keys can be determined to allow them to be overwritten during production.

2. The location of the application-stored context is improperly defined as an absolute memory location (0x0803f000) instead of being properly mapped via the linker script. This means there is no protection from conflicting use of the memory. The presumption that the last couple of pages of flash are "available" is inherently unsafe.

This conflict is NOT theoretical.

If the additional LoRaWAN packages functionality is enabled, then the Fragmented Data Block Transport package is enabled. This package also explicit uses the same region of flash (0x0803F000) (see frag_decoder_if.h)

Amending a previous comment I made about generating a CRC32 value, it is not necessary for the seNvmInit data as the CRC field is not used there - it only seems to be relevant in the ram Nvm copy, backup ram Nvm and saved context in flash as an integrity check.

[Niclas Hedhman]

Nice overview! And yes, I have defined 0x0803f000 and 0x0803f800 as two regions in the linker script, and removed the seNvmInit declaration i soft-se.c, and removed LORAWAN_NVM_BASE_ADDRESS and introduced LORAWAN_NVM_CONTEXT, which (with the definitions below) is a variable (declared as an array will get the address when used). 

extern uint8_t LORAWAN_NVM_CONTEXT[2048];

 

But I now have a system I can't use CubeMX on, as it will reset some of the changes I made. But, the way RAKwireless "low level development" (I use RAK3172) provides a driver is to overwrite the radio(-if) files, which are also re-generated by CubeMX. So the whole point of using ST supported LoraWAN is negated to a large degree right now. :\

@Andrew Larkin Thanks a lot. Without your engagement, I would probably have reverted to an STM32L1 + SX1276 and the Semtech stack. I hope ST can improve the integration, even if compatibility can't be maintained.

[Andrew Larkin]

Happy to help as I said at the start: I am hitting these very same issues right now.

I am also using RAK3172 but have started from scratch so have a "clean" project that will survive regeneration from the IOC file.

I have implemented a version of your fix for the context that I think you will like...

In lora_app.c:

/*---------------------------------------------------------------------------*/
/* LoRaWAN NVM configuration */
/*---------------------------------------------------------------------------*/
/**
* @brief LoRaWAN NVM Flash address
* @note last 2 sector of a 128kBytes device
*/
#define LORAWAN_NVM_BASE_ADDRESS ((void *)0x0803F000UL)

/* USER CODE BEGIN PD */
#undef LORAWAN_NVM_BASE_ADDRESS
extern uint8_t LORAWAN_NVM_BASE_ADDRESS[4096];
/* USER CODE END PD */

This solves the regeneration issue.

On the second point of the factory settings, the code already tags the seNvmInit with the USER_embedded_Keys segment, so I think the cleaner way to handle this is to locate the segment where you need it.

This solution also remains "clean" to code regeneration.

I chose a different memory arrangement in the ld linker script as follows:

/* Memories definition */
MEMORY
{
RAM (xrw) : ORIGIN = 0x20000000, LENGTH = 64K
RAM2 (xrw) : ORIGIN = 0x10000000, LENGTH = 32K
FLASH (rx) : ORIGIN = 0x08000000, LENGTH = 250K
LORAWAN_FACTORY (rx) : ORIGIN = 0x0803E800, LENGTH = 2K
LORAWAN_CONTEXT (rx) : ORIGIN = 0x0803F000, LENGTH = 4K
}

/* USER Embedded Keys data into "FACTORY" Rom type memory */
.embeddedKeys :
{
. = ALIGN(4);
*(.USER_embedded_Keys)
. = ALIGN(4);
} >LORAWAN_FACTORY

.context :
{
LORAWAN_NVM_BASE_ADDRESS = .;
} >LORAWAN_CONTEXT

I know you were trying to avoid unneeded the use of an extra 2k page, but I wanted to keep the memory structure the same for two reasons:
1. There is a good chance I will be using the fragmented data block support and I definitely didn't want it to be overwriting the seNvmInit data!

2. Support for KMS looks like it does require both pages of memory, if I ever want to turn this on as well.

I think both issues will come into effect if I start playing with FUOTA which I very much want to have running.

I note you were tagging the additional memory areas with (rwx). I don't know if it works to close a door to attack or not, but I choose to leave off the "x" is this memory shouldn't be executing code.

For details of how I generated the "clean" project for the RAK3172, see https://github.com/danak6jq/RAK3172/issues/12#issuecomment-1704666372