Mbed OS CMake Improvements

[rwalton-arm]

Overview

This post identifies some problems with the current CMake architecture in Mbed OS, and proposes some improvements we could make to the design.

The idea is to generate a discussion and see if we can improve the architecture of our build system as a group.

The current architecture makes it difficult for Mbed OS to integrate with other CMake aware build systems. Mbed OS assumes it is always the top level "driver" of the build process. This assumption has lead to us implementing a circular dependency chain between the consuming application or middleware and Mbed OS. This circular dependency means Mbed OS can't be built without the consumer setting various build properties. Our reliance on external tooling means we haven't factored the Mbed CMake libraries properly in some areas. A consequence of the poorly factored library targets means we largely rely on externally set global variables to decide which source files to add to the mbed-core target. This mechanism of relying on global variables reduces composability and leaves us highly coupled to our custom tooling. For similar reasons it is also rather difficult to build parts of Mbed OS using "host" toolchains, making unit testing difficult. This all means at the moment it would be very difficult to extract a "component" or "feature" from Mbed OS and integrate it with another RTOS, for example. Attempting to follow CMake best practices, and attempting to avoid falling back on additional tooling, should allow us to more easily integrate with other OSs, middleware and applications.

I've identified several areas where we could make improvements that would move us toward a more "CMake-native" architecture.

Break dependencies on global variables set by mbed-tools

Our CMake system has been designed to mirror the old build system. This means it relies on externally set global "target labels" variables to know which components and features to include. This certainly made sense when the main goal was to deprecate the old build tools and get something working as quickly as possible. Now that we have the working foundations of our CMake system implemented, it has become more clear that our over reliance on the old configuration system has led to a rigid architecture which is not easily composable.

One of the main barriers to interoperability is our requirement for the user to run custom tooling to parse a set of JSON configuration files in a pre-build step. mbed-tools parses the config files and creates the list of target labels variables the build system relies on. The set of labels produced is specific to the combination of target, toolchain and set of optional Mbed components specified in the application configuration. If the user wants to build for a different target or toolchain, or add a new optional component to their project, the config variables must be regenerated, requiring them to run the mbed-tools configure step again.

CMake provides us a reasonable alternative for target labels: transitive usage requirements. This mechanism allows us to create dependency chains amongst sets of CMake targets without relying on externally set global variables to tell us which source files, include directories or compile definitions to include.

We can attempt to eliminate the need for the external config variables in an incremental fashion. The first step would be breaking our dependency on targets.json's targets, features and components labels. With a view to moving the other sections of our config to either be set in CMake only, or use another, less policy ridden, off-the-shelf config system.

Actions

• Use CMake targets to add components and features where possible
• Remove references to MBED_TARGET_LABELS
• (Future) Move other config definitions to CMake or another off the shelf config system.

Pros

• Build system issues are more easily "googleable" as we're using standard CMake where feasible
• Ease of adoption and ease of integration is increased significantly

Improve toolchain handling

All settings required by the toolchain should be encapsulated in toolchain files. We should ideally keep logic and exported functions out of toolchain files. Toolchain files should not have dependencies, and the build system should not have a hard dependency on non-standard variables or functions defined in toolchain files. This allows us to swap toolchains easily and enables Mbed OS to interoperate with many standard toolchain files a user might define.

Currently we include our toolchain files rather than passing them in on the command line. This means we enforce the order in which Mbed OS is added to the project at the top level. We also export various functions from our toolchain files that the user must call to enable certain features, such as minimal-printf and the "small" c-lib. Generally these functions will just add extra compile options. Again, part of the solution is to define interface libraries for minimal-printf and c-lib which forward their required flags. We should then look at removing the toolchain.cmake layer and make our actual toolchain files self contained so they can be passed on the command line.

Actions

• Remove functions to set minimal-printf and small clib from toolchain files
• Create INTERFACE libraries for those optional features
• Remove toolchain.cmake and use CMAKE_TOOLCHAIN_FILE to pass in the toolchain file instead of including it
• Create library targets for "cores"

Pros

• Easier to change between native and cross-compiles
• Easier to build unit tests and develop mbed-os
• Removes requirement for specific ordering of CMake directives in top level application (toolchain file must be processed before compiler language is enabled)
• Easier to add new toolchain support (all we would need is another toolchain file)

Cons

• User has to pass in toolchain file (mbed-tools could automate that)

Python Dependencies

Ideally we wouldn't require Python to be installed just to build Mbed OS. Python should only be needed for optional "porcelain" (mbed-tools, memap, and friends). The porcelain should always be "opt in" for a user.

If any of our Python scripts make use of third party packages, we should package them up into Python packages and distribute them on PyPI.

Actions

• If we have Python scripts with 3rd party deps, package them
• Set CMake options to run scripts not required for building, e.g memap, default OFF
• (Future) Remove the need for targets.json and mbed_lib/app.json, making mbed-tools optional (See above)

Pros

• Increases ease of adoption for developers
• We don't need to check indirect dependencies in Mbed OS CMake (prettytable et al)
• Easier to distribute Mbed OS as we have fewer hard dependencies
• Fewer CMake workarounds needed to stop the build failing if dependencies are missing

Cons

• Initial effort to package up our various Python scripts
• Burden on user to opt in to certain features, makes CMake command line invocation more complex

Remove requirement for apps to call our custom functions (or move the custom functions out of mbed-os/CMakeLists)

If we must provide custom CMake functions they should be made visible through include statements at the top-level and not as a side-effect of add_subdirectory.

Actions

• Remove requirement for app to call custom functions (if possible)
• If that's not possible (e.g to run objcopy) move the API functions to .cmake modules so they're self contained and dependencies are more easily visible

Export "public" targets

If we want to integrate nicely with other CMake middleware we should export our public CMake targets. find_package and add_subdirectory integration should ideally "look the same" to the consumer.

Actions

• Export our targets in standard CMake way using install and export macros so we can be discovered using find_package
• Create an alias namespaced target for the add_subdirectory case so the target linking code doesn't need to change

Don't force users to build specific library type

Ideally we wouldn't hard code OBJECT, STATIC or shared libs. We should use default add_library calls where possible and let the consumer specify which type of lib to build.

Blockers

• Weak linking
• Cyclic dependencies

Thanks for taking the time to read this proposal. All of your comments and feedback are welcome.

[ladislas]

Thanks a lot @rwalton-arm for putting that together and starting the discussion.

The recent improvements and discussions around CMake in mbed are really pushing the new build system forward and I'm really happy to see that happening.

I agree with everything said there and I think the changes mentioned are not only needed now, they will also improve the overall mbed experience for new comers and more seasoned developers.

I'm particularly sensible to the idea of removing the need for external tooling such as python as much as possible. I'm not sure it will be possible to use only CMake but ideally the mbed tools should only be a nice opinionated wrapper around vanilla CMake, thus making it easy for beginners and deeply customizable for experts.

I think a lot of people, me included, are willing to help knowing that it will change and improve our day to day development experience.

The question is "where should we start"? :)

[0xc0170]

The question is "where should we start"? :)

Would it help to start for each one a new discussion thread where we can start discussing? I for instance already looked at Break dependencies on global variables set by mbed-tools so got some feedback but no solution yet. Tackle one problem at the time.

Besides discussion, I would like to have these as issues (we have component: tools in Mbed OS , we can rename that to CMake as tools are not tools anymore and start grouping related issue together).

Note please, we are still working on providing functionality to build/test targets and documentation (drop in replacement for old tools - this is our priority to migrate to the new system as soon as we can).

[nat45928]

I've been using mbed-os for about a year and a half now. I came into it with lots of professional experience with embedded development, C++, and python. The build system was 100% the biggest issue I had (and continue to have) with mbed-os.

All of the listed changes sound excellent. Pulling the underlying build system more in line with standard C++/CMake and allowing mbed-tools to become porcelain around it sounds like a great way to go. I'm fully with @ladislas, where do we start? :-)

[multiplemonomials]

Just checking, you've seen mbed-cmake right? It provides a pretty much feature complete build system you can use right now.

[nat45928]

Yep, I've seen that project and it's great! However, it's fantastic to see the official tooling and build system moving towards a more standardized solution.

[multiplemonomials]

I support almost everything in here, and I think you've made great progress identifying areas that need some improvement!

However, I have to disagree with this:

We should then look at removing the toolchain.cmake layer and make our actual toolchain files self contained so they can be passed on the command line.

Personally I hate CMake's system of toolchain files because it's easy to mess up: it forces the user to play an active role (pass -DCMAKE_TOOLCHAIN_FILE) in configuring the compiler when that should be the build system's responsibility. And if they don't pass that argument, everything will explode.

I understand that the current automatic toolchain file requires special handling if you are trying to integrate mbed os as a non-top-level thing into another application. However, Mbed OS and the application have to be built for the same target, right? So when would having Mbed OS automatically set up the compiler and flags be a bad thing?

Pros:
Easier to build unit tests and develop mbed-os
Easier to change between native and cross-compiles

Just curious, in what way would changing the toolchain file setup contribute to these?

[boraozgen]

How would you use a native compiler with the current configuration? (e.g. for unit testing on the host system, building parts of the project to run on a simulator on the host)

[multiplemonomials]

For mbed-cmake, I just added a thing that doesn't load the toolchain file and exports fake/simulated functions for the stuff handled by the toolchain file.

[ladislas]

For mbed-cmake, I just added a thing that doesn't load the toolchain file and exports fake/simulated functions for the stuff handled by the toolchain file.

I've never managed to get it working this way. I ended up with writing a new main CMakeLists.txt file to handle unit tests. It was much simpler that I thought and it works perfectly, we'll publish it soon.

[ladislas]

We should keep the discussion moving forward:

Besides discussion, I would like to have these as issues (we have component: tools in Mbed OS , we can rename that to CMake as tools are not tools anymore and start grouping related issue together).

@0xc0170 yes that sounds good! Could we add another templates for issues that are not issues (description of defect) but that can be idea to implement to keep track of them. Not everything is something that doesn't work, sometimes it's an improvement over something already working or a new feature.

Note please, we are still working on providing functionality to build/test targets and documentation (drop in replacement for old tools - this is our priority to migrate to the new system as soon as we can).

@0xc0170 how far in the process are you? do you have an ETA? just to know when we can start pushing stuff for review without overflowing you guys.

[0xc0170]

Feature requests: we are discussing to update our Github issues. This is currently only for technical bugs. We will update the community with the plan, once done, we should be able to request features. The goal is t have better visibility.

We need around one month to get testing sorted, fix issues found during testing and docs. We are already working on docs (https://github.com/ARMmbed/mbed-os-5-docs/tree/feature-cmake-docs - it is bigger refactor as we need 2 build systems to describe). Teams will also look at dependencies in the tree and possibly fix them.

[ladislas]

👍 gotcha!

another feature request: can we change the name of the docs repo to mbed-os-docs to avoid confusion with the version number? :)

[boraozgen]

One of the best proposals I read about Mbed since a long time. These are exactly the features needed power users who want to craft modern and maintainable software.

I would like to link my issue on the toolchain files, which would be solved by the "Improve toolchain handling" section: ARMmbed/mbed-os#14446

[hugueskamba]

Great proposal!
Regarding Python Dependencies, some Mbed boards require additional operations to be performed on the artefacts. These post-build operations are implemented as Python scripts and some of these scripts require third-party packages. The Python dependency for these Mbed boards is not optional.

[ladislas]

How many boards? Which ones?

Would it be easier to check for the deps and improve the documentation only for these boards?

So that the vast majority can fly away from python and only those who need it can do what they need?

[hugueskamba]

In targets/targets.json search for instances of post_binary_hook to identify the boards that require it.
I believe the Mbed boards that require the third-party Python package are the Cypress boards.

[ladislas]

That's 8 targets.

IMHO, people using the rest of the targets and the tools should not have to install python dependencies they don't use.

People using those target need to install the deps and read the documentation. Checks can be added for those targets only to make sure the python deps are installed.

[hugueskamba]

It is not exactly 8. The boards that require post-build operations are: LPC1114, LPC1768, ARCH_PRO, LPC54114, LPC546XX, FF_LPC546XX, CY8CKIT064B0S2_4343W, CYTFM_064B0S2_4343W, CYSBSYSKIT_01, ARM_MUSCA_B1, ARM_MUSCA_S1.
However, I still share your opinion.

[multiplemonomials]

I noticed that "VS Code Integration" is on the roadmap for later on. I wanted to point out that I already implemented that for mbed-cmake, both as a setup guide, and a config generator that creates launch.json automatically. Feel free to reuse this! I'm not a heavy VS Code user but I'm happy with how it works so far (except for glitches with the Native Debug extension).