TFLM output does not match TfLite output for Conv OP

[AIWintermuteAI]

First found during manual testing and confirmed with layer_by_layer_debugger script.
Output:

AssertionError: 
Arrays are not equal


TFLM output does not match TfLite output.
 Subgraph number is 0 
 Layer number is 2 
 The Tensor Index where this output does not match is 46 



Mismatched elements: 80 / 51200 (0.156%)
Max absolute difference among violations: 1
Max relative difference among violations: 1.
 ACTUAL: array([[[[   7,   14,    6,   17,    4,   -5,  -20,    8],
         [   1,  -34,   15,  -12,   15,   15,    7,    6],
         [   2,   15,   -5,   47,    6,   -6,   -3,    0],...
 DESIRED: array([[[[   7,   14,    6,   17,    4,   -5,  -20,    8],
         [   1,  -34,   15,  -12,   15,   15,    7,    6],
         [   2,   15,   -5,   47,    6,   -6,   -3,    0],...

The model in question is attached:

tflite_learn_977.tflite.zip

The difference is between optimized Conv kernel used in full tflite and reference kernel used in tflite micro.

Potentially related:
#2629

[dansitu]

Any further information on this issue?

[ddavis-2015]

@AIWintermuteAI @dansitu @ArmRyan @Unbinilium

There is no difference in CONV kernel outputs when both TfLite and TFLM are using reference kernels.

This turns out to be an issue with comparing the outputs of TfLite optimized kernels against the outputs of reference kernels. The TFLM CONV kernel uses the reference CONV operator here. This reference operator is imported into TFLM from the TfLite source code, and is thus identical to TfLite.

However, the default TfLite interpreter uses an optimized CONV kernel which in turn uses this CONV operator. As you can see, these are very different code paths. The result is an off-by-one difference among some elements of the output tensor. As to why there is a difference in the TfLite reference and optimized CONV operator outputs, you would need to ask that question in the TfLite repository, as a new issue.

Here is a colab example showing TfLite reference vs. optimized kernel vs. TFLM reference kernel for the CONV operator: conv_mismatch

This issue will be resolved in a future PR for the TFLM layer-by-layer debugger, such that the default OpResolver for TfLite uses reference kernels.

[ddavis-2015]

[internal: add additional command line options to the layer-by-layer debugger, allowing setting of the arena size, and which TfLite OpResolver to use]

[AIWintermuteAI]

Hi, @ddavis-2015 !
Thanks for putting the time into investigating.
I was aware that this is in fact the difference between optimized and reference kernels - see my original post:

The difference is between optimized Conv kernel used in full tflite and reference kernel used in tflite micro.

I'll create an issue.
Btw,

This issue will be resolved in a future PR for the TFLM layer-by-layer debugger, such that the default OpResolver for TfLite uses reference kernels.

This does not seem to be wise, if optimized tflite kernels are used by default. The only thing that will achieve is more confusion, when users see disparity then cannot reproduce it with TFLM layer-by-layer debugger.
There should no disparities between ANY kernels, optimized or otherwise.