Test frameworks included on targets that can't run tests

[Patater]

Description of defect

As discovered in #11594, our test frameworks still attempt to build for a target even when that target can't run the test frameworks. For example, on platforms that have a non-secure image (NSPE) and a secure image (SPE), SPE targets should be buildable even without serial, to save flash space within the secure image. All testing of the SPE can be done in the NSPE build, which does include serial.

Target(s) affected by this defect ?

Any targets that don't have serial, for instance. Specifically, NU_PFM_M2351_NPSA_S.

Toolchain(s) (name and version) displaying this defect ?

All toolchains

What version of Mbed-os are you using (tag or sha) ?

Mbed OS 5.14

What version(s) of tools are you using. List all that apply (E.g. mbed-cli)

Mbed OS 5.14

How is this defect reproduced ?

Attempt to build NU_PFM_M2351_NPSA_S, which by default (via targets.json) doesn't enable serial for code size reasons. The build should succeed, but it fails because the test frameworks depend on serial being present.

[0xc0170]

This is currently blocking #11594

@ARMmbed/mbed-os-test Please review

[0xc0170]

Checking the files, some files in utest are protected by #if DEVICE_SERIAL but not features/frameworks/utest/source/utest_harness.cpp . This file pulls default handlers but they are not defined as there is no serial.

Same for the file utest_greentea_handlers.cpp and features/frameworks/unity/unity/unity_config.h

[0xc0170]

I can look at this tomorrow, will test one target without serial if it builds

[0xc0170]

@ARMmbed/mbed-os-test On another hand, all our tests use greentea setup and other functionality depending on serial. Without serial we won't be able to build these.

How shall we fix this one ? Excluding the target from building? As this would be known issue - SPE target by default is not tested?

[Patater]

Excluding the target from building? As this would be known issue - SPE target by default is not tested?

I don't believe it makes sense to test the SPE target standalone. The functionality the SPE provides should be tested through tests running on the NSPE.

[ccli8]

Not resolved? How about pick one from #11594 (comment)?

[ccli8]

Any resolution? @cyliangtw @MarceloSalazar

[0xc0170]

@VeliMattiLahtela Please review

[ciarmcom]

@Patater thank you for raising this issue.Please take a look at the following comments:

We cannot automatically identify a release based on the version of Mbed OS that you have provided.
Please provide either a single valid sha of the form #abcde12 or #3b8265d70af32261311a06e423ca33434d8d80de
or a single valid release tag of the form mbed-os-x.y.z .
E.g. '5.14' has not been matched as a valid tag or sha.
NOTE: If there are fields which are not applicable then please just add 'n/a' or 'None'.This indicates to us that at least all the fields have been considered.
Please update the issue header with the missing information, the issue will not be mirroredto our internal defect tracking system or investigated until this has been fully resolved.