.. SPDX-License-Identifier: CC-BY-SA-4.0

.. Copyright (C) 2017 Chris Johns <chrisj@rtems.org>

RTEMS BSP Builder
=================

.. index:: Tools, rtems-bsp-builder

The RTEMS BSP Builder is an RTEMS developer tool to build RTEMS in ways users
do not to test changes to RTEMS. RTEMS has large number of architectures, board
support packages and configuration options. This tool provides a standard way
to test a change.

Developer Workflows
-------------------

There are a number of RTEMS developers each with a different view or expertise
in RTEMS. Developers can work in the generic areas such as scheduling, file
systems or the shell, or users can become developers adding a new BSP, or even
a new port to a new architecture. A common approach for all these developers is
to select a BSP and to work with that BSP. Developers working in a generic
areas of RTEMS tend to select a BSP that has good simulator support with good
debugging such as QEMU, while developers of a new BSP or a new port tend to
work on target hardware. This type of development does not check the other
architectures, BSP, and build options and a change may change the number of
warnings or introduce build errors. It is important for the RTEMS project to
have developers fix these issues before pushing the changes to the master
repository to avoid breaking the code for other developers. It is best for a
developer to resolve as many issues as they work on changes because comming
back to a problem often proves difficult.

The RTEMS BSP Builder forms part of a developers workflow where patches are
tested before being pushed to the repository.

Build Characteristics
---------------------

Build characteristic are the various parts of a build that can varied changing
what is built. RTEMS can vary builds based on:

#. Architecture

#. Board Support Package (BSP)

#. Build options

#. BSP Options

The BSP Builder provides a template of builds to try and reduce the possble
combinations to something manageable. It is not realistic to build all possible
combinations on a single machine in reasonible time.

The RTEMS BSP Builder specifies it builds in terms of:

#. Profiles

#. Architectures

#. BSPs

#. Builds

The RTEMS BSP Builder builds are created by user options that vary these parameters.

Profiles
^^^^^^^^

A profile is named collection of architectures and board support packages. When
the RTEMS BSP Builder is asked to build a specific profile it builds the BSPs
in the specified architectures.

The default configuration provides standard profiles for :ref:`Tiers`. They are:

#. ``tier-1`` (default)

#. ``tier-2``

#. ``tier-3``

#. ``tier-4``

The ``everythings`` profile allows all BSPs to be built.

Builds
^^^^^^

A build is a list of builds or a build set and each BSP in a profile,
architecture of BSP is built with.

The default configuration provides standard builds based around the commonly
varied configure options.

The builds are:

#. ``all`` (default)

#. ``tests``

#. ``standard``, also ``no-tests``

#. ``debug``

#. ``profiling``

#. ``smp``

#. ``smp-debug``

#. ``posix``

#. ``no-posix``

#. ``posix-debug``

#. ``posix-profiling``

#. ``network``

#. ``no-network``

#. ``network-debug``

#. ``smp-network``

#. ``smp-network-debug``

All Build
~~~~~~~~~

The ``all`` build is:

- ``debug``
- ``profiling``
- ``smp``
- ``smp-debug``
- ``posix``
- ``no-posix``
- ``posix-debug``
- ``posix-profiling``
- ``network``
- ``no-network``
- ``network-debug``
- ``smp-network``
- ``smp-network-debug``

A build maps to specific configuration options. The mappings are:

 +-----------------------+-----------------------------------------------------+
 | ``debug``             | ``config:base``, ``config:debug``                   |
 +-----------------------+-----------------------------------------------------+
 | ``profiling``         | ``config:base``, ``config:profiling``               |
 +-----------------------+-----------------------------------------------------+
 | ``smp``               | ``config:base``, ``config:smp``                     |
 +-----------------------+-----------------------------------------------------+
 | ``smp-debug``         | ``config:base``, ``config:smp``, ``config:debug``   |
 +-----------------------+-----------------------------------------------------+
 | ``posix``             | ``config:base``, ``config:posix``                   |
 +-----------------------+-----------------------------------------------------+
 | ``no-posix``          | ``config:base``, ``config:no-posix``                |
 +-----------------------+-----------------------------------------------------+
 | ``posix-debug``       | ``config:base``, ``config:posix``, ``config:debug`` |
 +-----------------------+-----------------------------------------------------+
 | ``posix-profiling``   | ``config:base``, ``config:posix``,                  |
 |                       | ``config:profiling``                                |
 +-----------------------+-----------------------------------------------------+
 | ``network``           | ``config:base``, ``config:network``                 |
 +-----------------------+-----------------------------------------------------+
 | ``no-network``        | ``config:base``, ``config:no-network``              |
 +-----------------------+-----------------------------------------------------+
 | ``network-debug``     | ``config:base``, ``config:network``,                |
 |                       | ``config:debug``                                    |
 +-----------------------+-----------------------------------------------------+
 | ``smp-network``       | ``config:base``, ``config:smp``, ``config:network`` |
 +-----------------------+-----------------------------------------------------+
 | ``smp-network-debug`` | ``config:base``, ``config:smp``,                    |
 |                       | ``config:network``, ``config:debug``                |
 +-----------------------+-----------------------------------------------------+

Build Configurations
--------------------

Build configurations are ``configure`` options. These are mapped to the various
builds. The configurations are:

 +------------------+----------------------------------------------------------+
 | ``base``         | ``--target=@ARCH@-rtems@RTEMS_VERSION@``                 |
 |                  | ``--enable-rtemsbsp=@BSP@``                              |
 |                  | ``--prefix=@PREFIX@``                                    |
 +------------------+----------------------------------------------------------+
 | ``tests``        | ``--enable-tests``                                       |
 +------------------+----------------------------------------------------------+
 | ``debug``        | ``--enable-debug``                                       |
 +------------------+----------------------------------------------------------+
 | ``no-debug``     | ``--disable-debug``                                      |
 +------------------+----------------------------------------------------------+
 | ``profiling``    | ``--enable-profiling``                                   |
 +------------------+----------------------------------------------------------+
 | ``no-profiling`` | ``--disable-profiling``                                  |
 +------------------+----------------------------------------------------------+
 | ``smp``          | ``--enable-smp``                                         |
 +------------------+----------------------------------------------------------+
 | ``no-smp``       | ``--disable-smp``                                        |
 +------------------+----------------------------------------------------------+
 | ``posix``        | ``--enable-posix``                                       |
 +------------------+----------------------------------------------------------+
 | ``no-posix``     | ``--disable-posix``                                      |
 +------------------+----------------------------------------------------------+
 | ``network``      | ``--enable-networking``                                  |
 +------------------+----------------------------------------------------------+
 | ``no-network``   | ``--disable-networking``                                 |
 +------------------+----------------------------------------------------------+

Performance
-----------

The RTEMS BSP Builder is designed to extract the maximum performance from your
hardware when building RTEMS. The RTEMS build system is based on ``autoconf``,
``automake`` and GNU ``make``. Building consists of two phases:

#. Configuring

#. Building

The Configuring phase and the start of the Build phase runs autoconf's
``configure`` scripts. These execute as a single linear process and are not run
in parallel even if you specify more than one job to ``make``. The configure
part of a build is approximately 30% of the total time and higher if building
the tests. Performing a single build at a time will not fully utilized a
multi-core machine because of the large amount of time the system is idle.

The RTEMS BSP Builder can run more than one build in parallel. A build can also
request ``make`` run its build with more than one job. The ``--jobs`` option
lets a user specify the number of build jobs to run at once and the number of
``make`` jobs each build runs with. Together these options can fully load a
system and can overload a machine.

Tuning the best ratio of buld jobs to make jobs requires running some builds
and observing the system's performance. If the build job count is too low the
system will show idle periods and if you have too many build jobs with too many
make jobs the system will have too many processing running and the operating
system's overheads in administting too processes at once lowers the overall
performance.

A fast eight core machine where the operating system shows sixteen cores can
support a build option of ``--jobs=5/10``. The machine will be fully loaded the
average build time is around 18 seconds.

The type of build selected effects the optimum jobs option. For example
building the tests changes the percentage of time spent configuring copmared to
bulding so the make jobs parameter becomes a dominant factor. Lowering the make
jobs value avoids having too many active processes running at once.

Command
-------

:program:`rtems-bsp-builder` [options]

.. option:: -?

   Display a compact help.

.. option:: -h, --help

   Display the full help.

.. option:: --prefix

   Prefix to pass to configure when building a BSP.

.. option:: --rtems-tools

   The path the RTEMS tools such as the C compiler. This option avoid polluting
   your path. This path is to the tool's prefix used to build and install the
   tools and not exact path to an executable.

.. option:: --rtems

   The path the RTEMS source tree to build.

.. option:: --build-path

   The path to build the BSP and place the build output. This can be any path
   and away from your current directory or the RTEMS source code. The storage
   does not need to be fast like an SSD.

.. option:: --log

   The log file.

.. option:: --config-report

   Print a configuration report and exit.

.. option:: --warnings-report

   Create a warnings report once all builds have finished.

.. option:: --stop-on-error

   Stop the build on an error. The default is to build all the builds for a
   profile.

.. option:: --no-clean

   Do not remove the build once finished. This option lets you inspect the
   built output. The amount of output can be large and disks can fill with this
   option.

.. option:: --profiles

   Build the comma separated list of profiles. The default is ``tier-1``.

.. option:: --arch

   A comma separated list of architecures to build using the selected build.

.. option:: --bsp

   A comma separated list of BSPs to build where a BSP is of the format
   ``arch/bsp`` using the selected build.

.. option:: --build

   The build to be used. The default is ``all``. See ``--config-report`` for a
   list of vlaid builds.

.. option:: --jobs

   The jobs options where the format is ``build-jobs/make-jobs``. The default
   is ``1/num-cores`` where ``num-cores`` is the operating system reported
   number of cores.

.. option:: --dry-run

   Do not do the actual builds just show what would be built.

Examples
^^^^^^^^

The following is a *tier-1* profile build of *all* on a machine where all the
source and tools are located on fast SSD disks and the build happens on a
spinning disk mounted under `build`. The build uses a development source tree
that is bootstrapped and ready to build. The source can have local patches that
need to be regression tested:

.. code-block:: none

  $ /opt/rtems/5/bin/rtems-bsp-builder --build-path=/build/rtems \
            --rtems-tools=/opt/work/rtems/5 \
            --rtems=/opt/work/chris/rtems/kernel/rtems.git \
            --profiles=tier-1 \
            --jobs=5/10
  RTEMS Tools Project - RTEMS Kernel BSP Builder, 5.not_released
  Profile(s): tier-1
  Cleaning: bsp-builds
  [  1/655] arm/altcycv_devkit (debug)                         Start
  [  1/655] arm/altcycv_devkit (debug)                         Creating: bsp-builds/arm/altcycv_devkit.debug
  [  2/655] arm/altcycv_devkit (no-posix)                      Start
  [  2/655] arm/altcycv_devkit (no-posix)                      Creating: bsp-builds/arm/altcycv_devkit.no-posix
  [  3/655] arm/altcycv_devkit (posix)                         Start
  [  1/655] arm/altcycv_devkit (debug)                         Configuring
  [  3/655] arm/altcycv_devkit (posix)                         Creating: bsp-builds/arm/altcycv_devkit.posix
  [  2/655] arm/altcycv_devkit (no-posix)                      Configuring
  [  4/655] arm/altcycv_devkit (posix-debug)                   Start
  [  1/655] arm/altcycv_devkit (debug)                         Building
  [  3/655] arm/altcycv_devkit (posix)                         Configuring
  [  4/655] arm/altcycv_devkit (posix-debug)                   Creating: bsp-builds/arm/altcycv_devkit.posix-debug
  [  2/655] arm/altcycv_devkit (no-posix)                      Building
  [  5/655] arm/altcycv_devkit (posix-profiling)               Start
  [  4/655] arm/altcycv_devkit (posix-debug)                   Configuring
  [  3/655] arm/altcycv_devkit (posix)                         Building
   ....
  [654/655] sparc/ngmp (posix-profiling)                       PASS
  [654/655] sparc/ngmp (posix-profiling)                       Warnings:0  exes:0  objs:0  libs:0
  [654/655] sparc/ngmp (posix-profiling)                       Finished (duration:0:01:49.002189)
  [654/655] sparc/ngmp (posix-profiling)                       Status: Pass:  655  Fail:    0 (configure:0 build:0)
  [655/655] sparc/ngmp (profiling)                             PASS
  [655/655] sparc/ngmp (profiling)                             Warnings:0  exes:0  objs:0  libs:0
  [655/655] sparc/ngmp (profiling)                             Finished (duration:0:01:260.002098)
  [655/655] sparc/ngmp (profiling)                             Status: Pass:  655  Fail:    0 (configure:0 build:0)
  [651/655] sparc/ngmp (no-posix)                              Cleaning: bsp-builds/sparc/ngmp.no-posix
  [652/655] sparc/ngmp (posix)                                 Cleaning: bsp-builds/sparc/ngmp.posix
  [653/655] sparc/ngmp (posix-debug)                           Cleaning: bsp-builds/sparc/ngmp.posix-debug
  [654/655] sparc/ngmp (posix-profiling)                       Cleaning: bsp-builds/sparc/ngmp.posix-profiling
  [655/655] sparc/ngmp (profiling)                             Cleaning: bsp-builds/sparc/ngmp.profiling
  Total: Warnings:31689  exes:6291  objs:793839  libs:37897
  Failures:
   No failure(s)
  Average BSP Build Time: 0:00:18.165000
  Total Time 3:41:48.075006
  Passes: 655   Failures: 0

To build a couple of BSPs you are interested in with tests:

.. code-block:: none

  $ /opt/rtems/5/bin/rtems-bsp-builder --build-path=/build/rtems \
            --rtems-tools=/opt/work/rtems/5 \
            --rtems=/opt/work/chris/rtems/kernel/rtems.git \
            ----log=lpc-log \
            --bsp=arm/lpc2362,arm/lpc23xx_tli800 \
            --build=tests \
            --jobs=5/12
  RTEMS Tools Project - RTEMS Kernel BSP Builder, 5.not_released
  BSPS(s): arm/lpc2362, arm/lpc23xx_tli800
  Cleaning: bsp-builds
  [1/2] arm/lpc2362 (tests)        Start
  [1/2] arm/lpc2362 (tests)        Creating: bsp-builds/arm/lpc2362.tests
  [2/2] arm/lpc23xx_tli800 (tests) Start
  [2/2] arm/lpc23xx_tli800 (tests) Creating: bsp-builds/arm/lpc23xx_tli800.tests
  [1/2] arm/lpc2362 (tests)        Configuring
  [2/2] arm/lpc23xx_tli800 (tests) Configuring
  [1/2] arm/lpc2362 (tests)        Building
  [2/2] arm/lpc23xx_tli800 (tests) Building
  [1/2] arm/lpc2362 (tests)        FAIL
  [1/2] arm/lpc2362 (tests)        Warnings:74  exes:58  objs:1645  libs:74
  [1/2] arm/lpc2362 (tests)        Finished (duration:0:01:31.708252)
  [1/2] arm/lpc2362 (tests)        Status: Pass:    0  Fail:    2 (configure:0 build:2)
  [2/2] arm/lpc23xx_tli800 (tests) FAIL
  [2/2] arm/lpc23xx_tli800 (tests) Warnings:74  exes:51  objs:1632  libs:74
  [2/2] arm/lpc23xx_tli800 (tests) Finished (duration:0:01:31.747582)
  [2/2] arm/lpc23xx_tli800 (tests) Status: Pass:    0  Fail:    2 (configure:0 build:2)
  [1/2] arm/lpc2362 (tests)        Cleaning: bsp-builds/arm/lpc2362.tests
  [2/2] arm/lpc23xx_tli800 (tests) Cleaning: bsp-builds/arm/lpc23xx_tli800.tests
  Total: Warnings:74  exes:109  objs:3277  libs:148
  Failures:
     1 tests arm/lpc2362 build:
        configure: /opt/work/chris/rtems/kernel/rtems.git/configure --target\
        =arm-rtems5 --enable-rtemsbsp=lpc2362 --prefix=/opt/rtems/5\
        --enable-tests
       error: ld/collect2:0 error: math.exe section '.rodata' will not fit
              in region 'ROM_INT'; region 'ROM_INT' overflowed by 7284 bytes

     2 tests arm/lpc23xx_tli800 build:
        configure: /opt/work/chris/rtems/kernel/rtems.git/configure --target\
        =arm-rtems5 --enable-rtemsbsp=lpc23xx_tli800\
        --prefix=/opt/rtems/5 --enable-tests
       error: ld/collect2:0 error: math.exe section '.text' will not fit in
              region 'ROM_INT'; region 'ROM_INT' overflowed by 13972 bytes

  Average BSP Build Time: 0:00:46.658257
  Total Time 0:01:33.316514
  Passes: 0   Failures: 2

The summary report printed shows both BSP builds failed with the error detail
shown. In this case both are linker related errors where the test do not fit
into the target's available resources.
