CMake design doc: restructure and update to the latest mbed-tools updates

Add mbed-tools - how an application is built with mbed-tools and CMake
Refactor Mbed OS CMake scripts section - details about mbed-os/cmake scripts

Author: 0xc0170

docs/design-documents/tools/cmake.md

@@ -1,101 +1,118 @@
 # CMake Mbed OS
 
+Requirements:
+- CMake 3.13 and higher
+- mbed-tools (python 3.6 and higher)
+
 Two steps approach:
 
 - Mbed OS core CMake
-- Application CMake
-
-All what can be defined should be in CMake files and configuration (app/mbed .json files). Our build system would parse the configuration, create rules and generate top level CMake file that would include others (application CMake files plus also Mbed OS core CMake).
+- Building an application with mbed-tools
 
-To stay backward compatible, we create common rules as follows.
+Definitions and configurations would be defined in CMake files, Mbed app configuration file (`/path/to/app/mbed_app.json`) and Mbed library configuration files (`/path/to/mbed-os/<LIBRARY_NAME>/mbed_lib.json`). The Mbed build system would parse the Mbed configuration files and generate a `CMakeLists.txt` configuration file.
 
-We replace current `TOOLCHAIN_`, `FEATURE_`, `TARGET_` by providing `add_subdirectory` macros. There would be `if_target()`, `if_feature()`, `_if_toolchain()` macros for better readability.
+The following rules must be respected for backward compatibility.
 
-Example, to add stm_directory only for STM targets (current rule TARGET_STM):
+Target labels, components, and features defined in `/path/to/mbed-os/targets/targets.json` are used to help the build system determine which directories contain sources/include files to use in the build process. This is accomplished through a custom CMake function (`mbed_add_cmake_directory_if_labels`) to append a prefix (TARGET_, COMPONENT_ and FEATURE_) and add matching directories to the list of directories to process.
 
-```
-add_subdirectory_if_target(STM stm_directory)
+An example, to add `TARGET_STM` in the folder `targets` where we have folders like TARGET_NXP, TARGET_STM, etc:
 
 ```
-
-Application example:
-
-```
-./TARGET_STM/qspi/driver_stm_qspi.cpp
-./TARGET_STM/peripheral_specific_to_stm/stm_specific_driver.cpp
-```
-
-As result, the top level application CMake:
-
+mbed_add_cmake_directory_if_labels("TARGET")
 ```
-add_subdirectory_if_target(STM TARGET_STM/qspi)
-add_subdirectory_if_target(STM TARGET_STM/peripheral_specific_to_stm)
 
-```
+If a user selects for example target `NUCLEO_F411RE`, the target defines the label `STM`. As result, the target folder STM is included.
 
-There would be static CMakes in the `./TARGET_STM/qspi/` and `./TARGET_STM/peripheral_specific_to_stm` that would define what files should be included or specific settins for these modules.
+The same could be applied to other labels like features or components.
 
 To migrate to the new build system, we can provide auto scanning of the module and generate CMake based on what we find or use the way as described above. In both cases, we could stay backward compatible.
 
 ## Mbed OS Core (Mbed OS repository)
 
-There are couple of CMakes in the tree.
+There are numerous CMake files in the Mbed OS repository tree:
+
+* A `CMakeLists.txt` entry point in the Mbed OS root, describing the top level build specification for the Mbed OS source tree.
+* `CMakeLists.txt` entry points in each Mbed OS module subdirectory, describing the build specification for a module or component
 
-1. Mbed OS boiler plate defined in Mbed OS root (provides way for build system to overwrite/add configuration)
-2. Mbed OS Toolchain settings (toolchain.cmake) that would get generated based on the toolchain selected
-3. Each module has own CMake (describing the module - what files are there, target/feature/component selection based on target, etc)
+A number of CMake scripts are contained in the `mbed-os/cmake` directory:
+* `toolchain.cmake` - selects the toolchain script from the `cmake/toolchains` directory, based on the value of the `MBED_TOOLCHAIN` variable
+* `profile.cmake` - selects the profile script from the `cmake/profiles` directory, based on the value of the `MBED_PROFILE` variable
+* `core.cmake` - selects the core script from the `cmake/cores` directory, based on the value of the `MBED_CPU_CORE` variable
+* `util.cmake` - custom CMake helper functions and macros
+* `app.cmake` - contains part of the build specification for an application
 
 The next sections will describe static CMake files within Mbed OS Core repository.
 
-### 1. Boilerplate CMake
+### 1. Mbed OS `CMakeLists.txt` Entry Point
 
-The main CMake file in Mbes OS repository provides just boilerplate for Mbed OS to be built. It described the Mbed OS tree, provides all the options we have in Mbed OS. This will be autogenerated by the tools.
+The `CMakeLists.txt` entry point in the root of the Mbed OS repository contains the top level build specification for Mbed OS. This file also includes the auto generated `mbed_config.cmake` script, which is created by `mbed-tools`.
 
-### 2. Toolchain CMake
+This is not intended to be included by an application.
 
-There are 3 toolchains, we provide the template that tools could fill in. The information is already in the build tools, we just need to extract that info and make it CMake compatible.
+### 2. Toolchain CMake Scripts
 
-This toolchain CMake is included by the Mbed OS CMake.
+All the toolchain settings are defined in the scripts found in `cmake/toolchains/`.
 
-### 3. Module CMake
+### 3. Profile CMake Scripts
 
-This file statically defines the structure of the module within Mbed OS. It's conditionally config based. We use regular CMake expressions plus extend it with own macros to conditionally include/exclude directories. The thumb of the rule, do not expose headers that are internal. We would like to avoid having everything in the include paths as we do now.
+The build profiles such as release or debug are defined in the scripts found in `cmake/profiles/`.
 
+### 4. MCU Core CMake Scripts
 
-`add_subdirectory` always adds the directory to the main CMake.
+The MCU core definitions are defined in the scripts found in `cmake/cores/`.
 
-```
-add_subdirectory(drivers)
-```
+### 5. Utilities CMake Scripts
 
-Conditionally directories addition based on the config:
+Custom functions/macros used within Mbed OS.
 
-```
-add_subdirectory_if_config(CONFIG_REQUIRES_RTOS rtos)
-```
-
-Conditionally directories addition based on the target/toolchain/feature.
+### 6. Application CMake
 
+The CMake script that must be included by all applications using:
 
 ```
-add_subdirectory_if_target(STM targets\TARGET_STM)
-add_subdirectory_if_toolchain(GCC_ARM cmsis\TARGET_GCC_ARM)
-add_subdirectory_if_feature(BLE ble\FEATURE_BLE)
+include(${MBED_ROOT}/cmake/app.cmake)
 ```
 
+### 7. Component `CMakeLists.txt` Entry Point
 
-## Application CMake
+This file statically defines the build specification of an Mbed OS component. It contains conditional statements that depend on the configuration parameters generated by `mbed-tools`. The component entry points make use of functions/macros defined in `util.cmake` to conditionally include or exclude directories.
+The rule of thumb is to not expose header files that are internal. We would like to avoid having everything in the include paths as we do now.
 
-We should provide application CMake functionality with  our own configuration. There are couple of approaches we could take. Statically defined CMake but then this disconnectes config and CMake - as CMake contains configuration for a project (like includes, sources, etc). Our build tool would need to parse CMake to get all paths used in the project or Mbed OS to find out where to look for configuration file. Therefore the build system has a knowledge as it is currently. We use `requires` to include/exclude modules.
+## Building an Application
 
-By default, baremetal would be selected - requires set to hal, platform, drivers and cmsis. If an app needs anything else, would use `requires` in the config to include - BLE/networking/etc.
+`mbed-tools` is the next generation of command line tooling for Mbed OS. `mbed-tools` replaces `mbed-cli` and the Python modules in the `mbed-os/tools` directory.
+
+`mbed-tools` consolidates all of the required modules to build Mbed OS, along with the command line interface, into a single Python package which can be installed using standard Python packaging tools.
 
 A user create own CMake file to configure an application, also with `mbed_app.json` configuration file. The building of an app would look like:
 
 1. Parse the arguments provided to build command
-2. Parse the application configuration
-3. Get the target configuration
-4. Get the Mbed OS configuration (select what modules we need and get their config, paths, etc)
-5. Create the toolchain.cmake file
-6. Inject all previous steps to the Mbed OS Core CMake
-7. Build an application
+1. Parse the application configuration
+1. Get the target configuration
+1. Get the Mbed OS configuration (select what modules we need and get their config, paths, etc)
+1. Create .mbedbuild/mbed_config.cmake
+1. Build an application
+
+### Configuration
+
+The main purpose of `mbed-tools` is to parse the Mbed configuration system's JSON files (`mbed_lib.json`, `mbed_app.json` and `targets.json`). The tool outputs a single CMake configuration script, which is included by `app.cmake` and `mbed-os/CMakeLists.txt`.
+
+To generate the CMake config script (named `mbed_config.cmake`) the user can run the `configure` command:
+
+`mbedtools configure -t <toolchain> -m <target>`
+
+This will output `mbed_config.cmake` in a directory named `.mbedbuild` at the root of the program tree.
+
+`mbed_config.cmake` contains several variable definitions used to select the toolchain, core and profile CMake scripts to be used in the build system generation:
+* `MBED_TOOLCHAIN`
+* `MBED_TARGET`
+* `MBED_CPU_CORE`
+* `MBED_PROFILE`
+
+The tools also generate an `MBED_TARGET_LABELS` variable, containing the labels, components and feature definitions from `targets.json`, used to select the required Mbed OS components to be built.
+
+The macro definitions parsed from the Mbed OS configuration system are also included in `mbed_config.cmake`, so it is no longer necessary for the tools to generate any `mbed_config.h` file to define macros.
+
+### mbedignore 
+
+With a build system that can understand dependencies between applications, features, components, and targets, `mbedignore` is no longer needed. CMake creates a list of exactly what's needed for an application build, rather than do what the legacy tools did and include everything by default, leaving the application to specify what is not needed. It's far more natural to express what you need rather than what you don't need, not to mention more future proof should some new feature appear in Mbed OS which your application would need to ignore.