Output data from LSM6DS3 appnote doesn't make sense

[IDEngineer]

We're evaluating the LSM6DS3. Its appnote (D027415 Rev 5) has table 20 on page 39, "Output data registers content vs. angular rate (FS_G = +/- 250dps)". Zero is zero, no problem there. The other examples are 100 and 200 dps. Let's use the positive values for some sanity checks.

For +100 dps when full (positive) scale is 250 dps, you would expect the output value to be (100/250=) 40% of positive full scale. 40% of 32768 = 13107 decimal which is 0x3333. But table 20 indicates the output data will be 0x2CA4 or 11428 decimal, which is just 35% of positive full scale.

Looking at +200 dps, table 20 says the output data will be 0x5949 which is 22857 decimal, 70% of positive full scale. But if positive full scale is supposed to be 250 dps, 200 dps should be (200/250=) 80% or roughly 0x6666.

We're still doing datasheet evaluation so we don't have physical parts here to test ourselves. Which is correct - the FS_G setting of +/-250 dps, or the appnote data which is some larger full scale range?

[IDEngineer]

2+ weeks later and no responses. Hmmm....

[Eleon BORLINI]

hi, the datasheet (p.20) reports 8.75 mdps/LSB for +-250dps, which is different from +-FS/2^bit = 500dps/65536 = 7.63mdps/digit.

Coherently with the LSM6DS3 app note, the 8.75 mdps value is the minimum detectable sensitivity, and is the value that must be used to convert the gyro output to get the measured dps.

This means that the +250dps will be less than 0xFFFF theoretical value.

sorry for the delay

Link: https://www.st.com/resource/en/datasheet/lsm6ds3.pdf

[IDEngineer]

Thank you for getting back to me!

So basically the appnote is wrong and the spec sheet is right. That's fair. Presumably the appnote author was working from invalid data and it hasn't been reviewed.

Again, thank you for the confirmation!

[Eleon BORLINI]

uhm... the app note reports, for FS_G = ±250 dps, an output of 0x2CA4 for 100 dps.

So 0x2CA4 = dec11428 and, in dps, 11428*8.75 mdps/LSB = 100 dps

So the ds and the app note are coherent (the 0x7FFF will be higher than the FS, and your data out will be around 0x6F9C)...

[IDEngineer]

Bottom line is that the "full scale" value isn't the real full scale value. Hence, as you said, "0x7FFF will be higher than the FS". It boils down to which piece of information is accurate... the "full scale value" or the incremental measurement, which yields a different "full scale value" than the 100, 250, etc. These yield different results, as my original post pointed out, so they cannot both be accurate.

I appreciate you clarifying that it is the incremental measurement (the value of the LSB, if you prefer) that should be considered accurate, while the full scale values are most definitely not accurate.