RTEMS Software Engineering (5.0.0-m2005-1)

Copyrights and License

© 2018, 2019 embedded brains GmbH
© 2018, 2019 Sebastian Huber
© 1988, 2015 On-Line Applications Research Corporation (OAR)

This document is available under the Creative Commons Attribution-ShareAlike 4.0 International Public License.

The authors have used their best efforts in preparing this material. These efforts include the development, research, and testing of the theories and programs to determine their effectiveness. No warranty of any kind, expressed or implied, with regard to the software or the material contained in this document is provided. No liability arising out of the application or use of any product described in this document is assumed. The authors reserve the right to revise this material and to make changes from time to time in the content hereof without obligation to notify anyone of such revision or changes.

The RTEMS Project is hosted at https://www.rtems.org. Any inquiries concerning RTEMS, its related support components, or its documentation should be directed to the RTEMS Project community.

1. Preface

This manual aims to guide the development of RTEMS itself. You should read this document if you want to participate in the development of RTEMS. Users of RTEMS may find background information in this manual. Please refer to the RTEMS User Manual and RTEMS Classic API Guide if you want to know how the RTEMS development environment is set up and how you can develop applications using RTEMS.

2. RTEMS Project Mission Statement

RTEMS development done under the umbrella of the RTEMS Project aims to provide a free and open real-time operating system targeted towards deeply embedded systems which is competitive with proprietary products. The RTEMS Project encourages the support and use of standard APIs in order to promote application portability and ease porting other packages to the RTEMS environment.

The RTEMS development effort uses an open development environment in which all users collaborate to improve RTEMS. The RTEMS cross development tool suite is based upon the free GNU tools and the open source standard C library newlib. RTEMS supports many host platforms and target architectures.

2.1. Free Software Project

The free software goals of the project are:

  • RTEMS and supporting components are available under various free licenses with copyrights being held by individual authors.

  • All software which executes on the target will not place undue restrictions on embedded applications. See also Licensing Requirements.

  • Patches must be legally acceptable for inclusion into the RTEMS Project or the specific project being used.

2.2. Design and Development Goals

  • Source based development with all users building from source

  • Any suitable host should be supported

  • Open testing, tests and test results

  • Ports to new architectures and CPU models

  • Addition of Board Support Packages for available hardware

  • Improved runtime libraries

  • Faster debug cycle

  • Various other infrastructure improvements

2.3. Open Development Environment

  • Encourage cooperation and communication between developers

  • Work more closely with “consumers”

  • Code available to everyone at any time, and everyone is welcome to participate in development

  • Patches will be considered equally based on their technical merits

  • All individuals and companies are welcome to contribute as long as they accept the ground rules

  • Open mailing lists

  • Developer friendly tools and procedures with a focus on keeping them current

  • Conflicts of interest exist for many RTEMS developers. The developers contributing to the RTEMS Project must put the interests of the RTEMS Project first.

3. RTEMS Stakeholders

You are a potential RTEMS stakeholder. RTEMS is a community based free and open source project. All users are treated as stakeholders. It is hoped that as stakeholders, users will contribute to the project, sponsor core developers, and help fund the infrastructure required to host and manage the project. Please have a look at the Support and Contributing chapter of the ERROR: :r:url:`user`.

4. Introduction to Pre-Qualification

RTEMS has a long history of being used to support critical applications. In some of these application domains, there are standards (e.g., DO-178C, NPR 7150.2) which define the expectations for the processes used to develop software and the associated artifacts. These standards typically do not specify software functionality but address topics like requirements definition, traceability, having a documented change process, coding style, testing requirements, and a user’s manual. During system test, these standards call for a review - usually by an independent entity - that the standard has been adhered too. These reviews cover a broad variety of topics and activities, but the process is generally referred to as qualification, verification, or auditing against the specific standard in use. The RTEMS Project will use the term “qualification” independent of the standard.

The goal of the RTEMS Qualification Project is to make RTEMS easier to review regardless of the standard chosen. Quite specifically, the RTEMS Qualification effort will NOT produce a directly qualified product or artifacts in the format dictated by a specific organization or standard. The goal is to make RTEMS itself, documentation, testing infrastructure, etc. more closely align with the information requirements of these high integrity qualification standards. In addition to improving the items that a mature, high quality open source project will have, there are additional artifacts needed for a qualification effort that no known open source project possesses. Specifically, requirements and the associated traceability to source code, tests, and documentation are needed.

The RTEMS Qualification Project is technically “pre-qualification.” True qualification must be performed on the project’s target hardware in a system context. The FAA has provided guidance for Reusable Software Components (FAA-AC20-148) and this effort should follow that guidance. The open RTEMS Project, with the assistance of domain experts, will possess and maintain the master technical information needed in a qualification effort. Consultants will provide the services required to tailor the master information, perform testing on specific system hardware, and to guide end users in using the master technical data in the context of a particular standard.

The RTEMS Qualification Project will broadly address two areas. The first area is suggesting areas of improvement for automated project infrastructure and the master technical data that has traditionally been provided by the RTEMS Project. For example, the RTEMS Qualification could suggest specific improvements to code coverage reports. The teams focused on qualification should be able to provide resources for improving the automated project infrastructure and master technical data for RTEMS. The term “resources” is often used by open source projects to refer to volunteer code contributions or funding. Although code contributions in this area are important and always welcome, funding is also important. At a minimum, ongoing funding is needed for maintenance and upgrades of the RTEMS Project server infrastructure, addition of services to those servers, and core contributors to review submissions

The second area is the creation and maintenance of master technical data that has traditionally not been owned or maintained by the RTEMS Project. The most obvious example of this is a requirements set with proper infrastructure for tracing requirements through code to test and documentation. It is expected that these will be maintained by the RTEMS Qualification Project. They will be evaluated for adoption by the main RTEMS Project but the additional maintenance burden imposed will be a strong factor in this consideration. It behooves the RTEMS Qualification Project to limit dependence on manual checks and ensure that automation and ongoing support for that automation is contributed to the RTEMS Project.

It is expected that the RTEMS Qualification Project will create and maintain maps from the RTEMS master technical data to the various qualification standards. It will maintain “scorecards” which identify how the RTEMS Project is currently doing when reviewed per each standard. These will be maintained in the open as community resources which will guide the community in improving its infrastructure.

4.1. Stakeholder Involvement

Qualification of RTEMS is a specialized activity and only specific users of RTEMS will complete a formal qualification activity. The RTEMS Project cannot self-fund this entire activity and requires stakeholder to invest in an ongoing basis to ensure that any investment they make is maintained and viable in an ongoing basis. The RTEMS core developers view steady support of the qualification effort as necessary to continue to lower the overall costs of qualifying RTEMS.

5. Software Requirements Engineering

Software engineering standards for critical software such as ECSS-E-ST-40C demand that software requirements for a software product are collected in a software requirements specification (technical specification in ECSS-E-ST-40C terms). They are usually derived from system requirements (requirements baseline in ECSS-E-ST-40C terms). RTEMS is designed as a reusable software product which can be utilized by application designers to ease the development of their applications. The requirements of the end system (system requirements) using RTEMS are only known to the application designer. RTEMS itself is developed by the RTEMS maintainers and they do not know the requirements of a particular end system in general. RTEMS is designed as a real-time operating system to meet typical system requirements for a wide range of applications. Its suitability for a particular application must be determined by the application designer based on the technical specification provided by RTEMS accompanied with performance data for a particular target platform.

Currently, no technical specification of RTEMS exists in the form of a dedicated document. Since the beginning of the RTEMS evolution in the late 1980s it was developed iteratively. It was never developed in a waterfall model. During initial development the RTEID [Mot88] and later the ORKID [VIT90] draft specifications were used as requirements. These were evolving during the development and an iterative approach was followed often using simple algorithms and coming back to optimise. In 1993 and 1994 a subset of pthreads sufficient to support GNAT was added as requirements. At this time the Ada tasking was defined, however, not implemented in GNAT, so this involved guessing during the development. Later some adjustments were made when Ada tasking was actually implemented. So, it was consciously iterative with the specifications evolving and feedback from performance analysis. Benchmarks published from other real time operating systems were used for comparison. Optimizations were carried out until the results were comparable. Development was done with distinct contractual phases and tasks for development, optimization, and the addition of priority inheritance and rate monotonic scheduling. The pthreads requirement has grown to be as much POSIX as possible.

Portability from FreeBSD to use its network stack, USB stack, SD/MMC card stack and device drivers resulted in another set of requirements. The addition of support for symmetric multiprocessing (SMP) was a huge driver for change. It was developed step by step and sponsored by several independent users with completely different applications and target platforms in mind. The high performance OpenMP support introduced the Futex as a new synchronization primitive.

Guidance

A key success element of RTEMS is the ability to accept changes driven by user needs and still keep the operating system stable enough for production systems. Procedures that place a high burden on changes are doomed to be discarded by the RTEMS Project. We have to keep this in mind when we introduce a requirements management work flow which should be followed by RTEMS community members and new contributors.

We have to put in some effort first into the reconstruction of software requirements through reverse engineering using the RTEMS documentation, test cases, sources, standard references, mailing list archives, etc. as input. Writing a technical specification for the complete RTEMS code base is probably a job of several person-years. We have to get started with a moderate feature set (e.g. subset of the Classic API) and extend it based on user demands step by step.

The development of the technical specification will take place in two phases. The first phase tries to establish an initial technical specification for an initial feature set. This technical specification will be integrated into RTEMS as a big chunk. In the second phase the technical specification is modified through arranged procedures. There will be procedures

  • to modify existing requirements,

  • add new requirements, and

  • mark requirements as obsolete.

All procedures should be based on a peer review principles.

5.1. Requirements for Requirements

5.1.1. Identification

Each requirement shall have a unique identifier (UID). The question is in which scope should it be unique? Ideally, it should be universally unique. As a best effort approach, the name RTEMS shall be used as a part in all requirement identifiers. This should ensure that the RTEMS requirements can be referenced easily in larger systems. The standard ECSS-E-ST-10-06C recommends in section 8.2.6 that the identifier should reflect the type of the requirement and the life profile situation. Other standards may have other recommendations. To avoid a bias of RTEMS in the direction of ECSS, this recommendation will not be followed.

Doorstop

The UID of an item (requirement) is defined by its file name without the extension. An UID consists of two parts, the prefix and a name or a number. The parts are divided by an optional separator. The prefix is determined by the document.

The UID scheme for RTEMS requirements is the concatenation of RTEMS, one or more component names, and a name. Each part is separated by a “-” character. For example, the UID RTEMS-CLASSIC-TASK-CREATERRINVADDR may specify that the rtems_task_create() directive shall return a status of RTEMS_INVALID_ADDRESS if the id parameter is NULL.

Doorstop

Doorstop uses documents to create namespaces (named a prefix in Doorstop). For the example above, you can create the documents like this:

doorstop create -s - RTEMS-CLASSIC -p RTEMS spec/classic
doorstop create -s - RTEMS-CLASSIC-TASK -p RTEMS-CLASSIC spec/classic/task
doorstop create -s - RTEMS-CLASSIC-SEMAPHORE -p RTEMS-CLASSIC spec/classic/semaphore

The requirement files are organized in directories.

A initial requirement item hierarchy could be this:

  • RTEMS

    • BUILD (building RTEMS BSPs and libraries)

    • CONFIG (application configuration)

    • CLASSIC

      • TASK

        • CREAT* (requirements for rtems_task_create())

        • DELT* (requirements for rtems_task_delete())

        • EXIT* (requirements for rtems_task_exit())

        • GETAFF* (requirements for rtems_task_get_affinity())

        • GETPRI* (requirements for rtems_task_get_priority())

        • GETSHD* (requirements for rtems_task_get_scheduler())

        • IDNT* (requirements for rtems_task_ident())

        • ISSUSP* (requirements for rtems_task_is_suspended())

        • ITER* (requirements for rtems_task_iterate())

        • MODE* (requirements for rtems_task_mode())

        • RESTRT* (requirements for rtems_task_restart())

        • RESUME* (requirements for rtems_task_resume())

        • SELF* (requirements for rtems_task_self())

        • SETAFF* (requirements for rtems_task_set_affinity())

        • SETPRI* (requirements for rtems_task_set_priority())

        • SETSHD* (requirements for rtems_task_set_scheduler())

        • START* (requirements for rtems_task_start())

        • SUSP* (requirements for rtems_task_suspend())

        • WKAFT* (requirements for rtems_task_wake_after())

        • WKWHN* (requirements for rtems_task_wake_when())

      • Semaphore

    • POSIX

A more detailed naming scheme and guidelines should be established. We have to find the right balance between the length of UIDs and self-descriptive UIDs. A clear scheme for all Classic API managers may help to keep the UIDs short and descriptive.

The specification of the validation of requirements should be maintained also by Doorstop. For each requirement document there should be a validation child Doorstop document with a TEST component name, e.g. RTEMS-CLASSIC-TASK-TEST. A test document may contain also validations by analysis, by inspection, and by design, see Requirement Validation.

5.1.2. Level of Requirements

The level of a requirement shall be expressed with one of the verbal forms listed below and nothing else. The level of requirements are derived from RFC 2119 [Bra97] and ECSS-E-ST-10-06C [ECS09].

5.1.2.1. Absolute Requirements

Absolute requirements shall be expressed with the verbal form shall and no other terms.

5.1.2.2. Absolute Prohibitions

Absolute prohibitions shall be expressed with the verbal form shall not and no other terms.

Warning

Absolute prohibitions may be difficult to validate. They should not be used.

5.1.2.3. Recommendations

Recommendations shall be expressed with the verbal forms should and should not and no other terms with guidance from RFC 2119:

SHOULD This word, or the adjective “RECOMMENDED”, mean that there may exist valid reasons in particular circumstances to ignore a particular item, but the full implications must be understood and carefully weighed before choosing a different course.

SHOULD NOT This phrase, or the phrase “NOT RECOMMENDED” mean that there may exist valid reasons in particular circumstances when the particular behavior is acceptable or even useful, but the full implications should be understood and the case carefully weighed before implementing any behavior described with this label.

5.1.2.4. Permissions

Permissions shall be expressed with the verbal form may and no other terms with guidance from RFC 2119:

MAY This word, or the adjective “OPTIONAL”, mean that an item is truly optional. One vendor may choose to include the item because a particular marketplace requires it or because the vendor feels that it enhances the product while another vendor may omit the same item. An implementation which does not include a particular option MUST be prepared to interoperate with another implementation which does include the option, though perhaps with reduced functionality. In the same vein an implementation which does include a particular option MUST be prepared to interoperate with another implementation which does not include the option (except, of course, for the feature the option provides.)

5.1.2.5. Possibilities and Capabilities

Possibilities and capabilities shall be expressed with the verbal form can and no other terms.

5.1.3. Syntax

Use the Easy Approach to Requirements Syntax (EARS) to formulate requirements. A recommended reading list to get familiar with this approach is [MWHN09], [MW10], and [MWGU16]. Please also have a look at the EARS quick reference sheet [Uus12]. The sentence types are:

  • Ubiquitous

    The <system name> shall <system response>.

  • Event-driven

    When <optional preconditions> <trigger>, the <system name> shall <system response>.

  • State-driven

    While <in state>, the <system name> shall <system response>.

  • Unwanted behaviour

    If <optional preconditions> <trigger>, then the <system name> shall <system response>.

  • Optional

    Where <feature>, the <system name> shall <system response>.

The optional sentence type should be only used for application configuration options. The goal is to use the enabled-by attribute to enable or disable requirements based on configuration parameters that define the RTEMS artefacts used to build an application executable (header files, libraries, linker command files). Such configuration parameters are for example the architecture, the platform, CPU port options, and build configuration options (e.g. uniprocessor vs. SMP).

5.1.4. Wording Restrictions

To prevent the expression of imprecise requirements, the following terms shall not be used in requirement formulations:

  • “acceptable”

  • “adequate”

  • “almost always”

  • “and/or”

  • “appropriate”

  • “approximately”

  • “as far as possible”

  • “as much as practicable”

  • “best”

  • “best possible”

  • “easy”

  • “efficient”

  • “e.g.”

  • “enable”

  • “enough”

  • “etc.”

  • “few”

  • “first rate”

  • “flexible”

  • “generally”

  • “goal”

  • “graceful”

  • “great”

  • “greatest”

  • “ideally”

  • “i.e.”

  • “if possible”

  • “in most cases”

  • “large”

  • “many”

  • “maximize”

  • “minimize”

  • “most”

  • “multiple”

  • “necessary”

  • “numerous”

  • “optimize”

  • “ought to”

  • “probably”

  • “quick”

  • “rapid”

  • “reasonably”

  • “relevant”

  • “robust”

  • “satisfactory”

  • “several”

  • “shall be included but not limited to”

  • “simple”

  • “small”

  • “some”

  • “state-of-the-art”.

  • “sufficient”

  • “suitable”

  • “support”

  • “systematically”

  • “transparent”

  • “typical”

  • “user-friendly”

  • “usually”

  • “versatile”

  • “when necessary”

For guidelines to avoid these terms see Table 11-2, “Some ambiguous terms to avoid in requirements” in [WB13]. There should be some means to enforce that these terms are not used, e.g. through a client-side pre-commit Git hook, a server-side pre-receive Git hook, or some scripts run by special build commands.

5.1.5. Separate Requirements

Requirements shall be stated separately. A bad example is:

RTEMS-CLASSIC-TASK-CRAT

The task create directive shall evaluate the parameters, allocate a task object and initialize it.

To make this a better example, it should be split into separate requirements:

RTEMS-CLASSIC-TASK-CRAT

When the task create directive is called with valid parameters and a free task object exists, the task create directive shall assign the identifier of an initialized task object to the id parameter and return the RTEMS_SUCCESSFUL status.

RTEMS-CLASSIC-TASK-CRATERRTOOMANY

If no free task objects exists, the task create directive shall return the RTEMS_TOO_MANY status.

RTEMS-CLASSIC-TASK-CRATERRINVADDR

If the id parameter is NULL, the task create directive shall return the RTEMS_INVALID_ADDRESS status.

RTEMS-CLASSIC-TASK-CRATERRINVNAME

If the name parameter is not valid, the task create directive shall return the RTEMS_INVALID_NAME status.

5.1.6. Conflict Free Requirements

Requirements shall not be in conflict with each other inside a specification. A bad example is:

RTEMS-CLASSIC-SEMAPHORE-MTXOBWAIT

If a mutex is not available, the mutex obtain directive shall enqueue the calling thread on the wait queue of the mutex.

RTEMS-CLASSIC-SEMAPHORE-MTXOBERRUNSAT

If a mutex is not available, the mutex obtain directive shall return the RTEMS_UNSATISFIED status.

To resolve this conflict, a condition may be added:

RTEMS-CLASSIC-SEMAPHORE-MTXOBWAIT

If a mutex is not available, when the RTEMS_WAIT option is set, the mutex obtain directive shall enqueue the calling thread on the wait queue of the mutex.

RTEMS-CLASSIC-SEMAPHORE-MTXOBERRUNSAT

If a mutex is not available, when the RTEMS_WAIT option is not set, the mutex obtain directive shall return the RTEMS_UNSATISFIED status.

5.1.7. Use of Project-Specific Terms and Abbreviations

All project-specific terms and abbreviations used to formulate requirements shall be defined in the project glossary.

5.1.8. Justification of Requirements

Each requirement shall have a rationale or justification recorded in a dedicated section of the requirement file.

Doorstop

See rationale attribute for Specification Items.

5.2. Specification Items

The technical specification of RTEMS will contain requirements, specializations of requirements, test procedures, test suites, test cases, and requirement validations. These things will be called specification items or just items if it is clear from the context.

Doorstop

Doorstop maintains items which are included in documents. The normal use case is to store a requirement with meta-data in an item. However, items can be also used to store other things like test procedures, test suites, test cases, and requirement validations. Items contain key-value pairs called attributes. Specializations of requirements may contain extra attributes, e.g. interface, build, configuration requirements. All items have the following standard Doorstop attributes:

active

A boolean value which indicates if the requirement is active or not. The value is included in the fingerprint via a document configuration option.

derived

A boolean value which indicates if the requirement is derived or not. For the definition of derived. see the Doorstop documentation. For RTEMS, this value shall be false for all requirements. The value is not included in the fingerprint.

normative

A boolean value which indicates if the requirement is normative or not. For the definition of normative. see the Doorstop documentation. For RTEMS, this value shall be true for all requirements. The value is not included in the fingerprint.

level

Indicates the presentation order within a document. For RTEMS, this value shall be unused. The value is not included in the fingerprint.

header

A header for an item. For RTEMS, this value shall be the empty string. The value is not included in the fingerprint.

reviewed

The fingerprint of the item. Maintain this attribute with the doorstop clear command.

links

The links from this item to parent items. Maintain this attribute with the doorstop link command. The value is included in the fingerprint.

ref

References to files and directories. See #365, The value is included in the fingerprint.

text

The item text. The value is included in the fingerprint.

All specification items shall have the following extended attributes:

type:

A list of item types. The value is not included in the fingerprint.

enabled-by:

The value is a list of expressions. The value is included in the fingerprint. An expression is an operator or an option. An option evaluates to true if it is included the set of enabled options of the configuration. An operator is a dictionary with exactly one key and a value. Valid keys are and, or, and not:

  • The value of the and operator shall be a list of expressions. It evaluates to the logical and of all outcomes of the expressions in the list.

  • The value of the or operator shall be a list of expressions. It evaluates to the logical or of all outcomes of the expressions in the list.

  • The value of the not operator shall be an expression. It negates the outcome of its expression.

The outcome of a list of expressions without an operator is the logical or of all outcomes of the expressions in the list. An empty list evaluates to true. Examples:

enabled-by:
- RTEMS_SMP
enabled-by:
- and:
  - RTEMS_NETWORKING
  - not: RTEMS_SMP
enabled-by:
- and:
  - not: TEST_DEBUGGER01_EXCLUDE
  - or:
    - arm
    - i386

5.2.1. Item Types

Specification items can have all sorts of types. The selection of types and the level of detail depends on a particular standard and product model. We need enough flexibility to be in line with ECSS-E-ST-10-06 and possible future applications of other standards. Each item may have a list of types. Valid types are listed below. This list may change over time. If new types are added, then a mapping between types should be specified. The item types and their definition is work in progress. A list of types follows:

  • requirement

    • functional - Functional requirements shall describe the behaviour of the software product under specific conditions.

      • capability

      • dependability-function

      • function

      • operational - Operational requirements shall

        • define the operation modes (e.g. initialization, multitasking, termination),

        • describe the operation modes, and

        • describe the operation mode transitions.

      • safety-function

    • non-functional

      • build-configuration

      • constraint

      • design

      • interface

      • interface-requirement

      • maintainability

      • performance

      • portability

      • quality

      • reliability

      • resource

      • safety

  • test-procedure

  • test-suite

  • test-case

  • validation-by-analysis

  • validation-by-inspection

  • validation-by-review-of-design

  • validation-platform

_images/req-spec-items.png

5.2.2. Requirements

Doorstop

All requirement specification items shall have the following extended attribute:

rationale:

The rationale or justification of the specification item. The value is not included in the fingerprint.

5.2.3. Build Configuration

Build configuration requirements define what needs to be built (libraries, object files, test executables, etc.) and how (configuration option header files, compiler flags, linker flags, etc.). The goal is to generate build files (Makefile or waf) and content for the Software Configuration File (SCF) from it. A YAML scheme needs to be defined for this purpose.

5.2.4. Interface Requirement

Interface requirements shall describe the high level aspects of interfaces. The item type shall be interface-requirement.

5.2.5. Interface

Interface items shall specify the interface of the software product to other software products and the hardware. The item type shall be interface. The interface items shall have an interface-category which is one of the following and nothing else:

  • application: Application interface items shall describe the interface between the software product and the application (API). The goal is to generate header files with Doxygen markup and user manual documentation parts from the application interface specification.

  • application-configuration: Application configuration items shall define parameters of the software product which can be set by the application at link-time. The goal is to generate user manual documentation parts and test cases from the application configuration specification.

  • architecture: Architecture interface items shall define the interface between the software product and the processor architecture (ABI).

  • gcc: GCC interface items shall define the interface between the software product and GCC components such as libgcc.a, libatomic.a, libgomp.a, libstdc++.a, etc.

  • hardware: Hardware interface items shall define the interface between the software product and the hardware.

  • newlib: Newlib interface items shall define the interface between the software product and the Newlib (libc.a).

The interface items shall have an interface-type which is one of the following and nothing else:

  • configuration-option

  • define

  • enum

  • function

  • header

  • macro

  • register-block

  • structure

  • typedef

  • union

  • variable

5.2.6. Interface - Application Configuration

Doorstop

The application configuration items shall have the following attribute specializations:

type

The type value shall be interface.

interface-category

The interface category value shall be application-configuration.

interface-type

The interface type value shall be configuration-option.

link

There shall be a link to a higher level requirement.

text

The application configuration requirement.

configuration-category:

A category to which this application configuration option belongs.

define:

The name of the configuration define.

data-type:

The data type of the configuration define.

value-range:

The range of valid values.

default-value:

The default value.

description:

The description of the application configuration. The description should focus on the average use-case. It should not cover potential problems, constraints, obscure use-cases, corner cases and everything which makes matters complicated.

note:

Notes for this application configuration. The notes should explain everything which was omitted from the description. It should cover all the details.

5.2.7. Test Procedure

Test procedures shall be executed by the Qualification Toolchain.

Doorstop

The test procedure items shall have the following attribute specializations:

type

The type value shall be test-procedure.

text

The purpose of this test procedure.

platform

There shall be links to validation platform requirements.

steps

The test procedure steps. Could be a list of key-value pairs. The key is the step name and the value is a description of the actions performed in this step.

5.2.8. Test Suite

Test suites shall use the RTEMS Test Framework.

Doorstop

The test suite items shall have the following attribute specializations:

type

The type value shall be test-suite.

text

The test suite description.

5.2.9. Test Case

Test cases shall use the RTEMS Test Framework.

Doorstop

The test case items shall have the following attribute specializations:

type

The type value shall be test-case.

link

The link to the requirement validated by this test case or no links if this is a unit or integration test case.

ref

If this is a unit test case, then a reference to the software item under test shall be provided. If this is an integration test case, then a reference to the integration under test shall be provided. The integration is identified by its Doxygen group name.

text

A short description of the test case.

inputs

The inputs to execute the test case.

outputs

The expected outputs.

The test case code may be also contained in the test case specification item in a code attribute. This is subject to discussion on the RTEMS mailing list. Alternatively, the test code could be placed directly in source files. A method is required to find the test case specification of a test case code and vice versa.

5.2.10. Resources and Performance

Normally, resource and performance requirements are formulated like this:

  • The resource U shall need less than V storage units.

  • The operation Y shall complete within X time units.

Such statements are difficult to make for a software product like RTEMS which runs on many different target platforms in various configurations. So, the performance requirements of RTEMS shall be stated in terms of benchmarks. The benchmarks are run on the project-specific target platform and configuration. The results obtained by the benchmark runs are reported in a human readable presentation. The application designer can then use the benchmark results to determine if its system performance requirements are met. The benchmarks shall be executed under different environment conditions, e.g. varying cache states (dirty, empty, valid) and system bus load generated by other processors. The application designer shall have the ability to add additional environment conditions, e.g. system bus load by DMA engines or different system bus arbitration schemes.

To catch resource and performance regressions via test suite runs there shall be a means to specify threshold values for the measured quantities. The threshold values should be provided for each validation platform. How this can be done and if the threshold values are maintained by the RTEMS Project is subject to discussion.

5.3. Traceability of Specification Items

The standard ECSS-E-ST-10-06C demands that requirements shall be under configuration management, backwards-traceable and forward-traceable. Requirements are a specialization of specification items in RTEMS.

5.3.1. History of Specification Items

The RTEMS specification items should placed in the RTEMS sources using Git for version control. The history of specification items can be traced with Git. Special commit procedures for changes in specification item files should be established. For example, it should be allowed to change only one specification item per commit. A dedicated Git commit message format may be used as well, e.g. use of Approved-by: or Reviewed-by: lines which indicate an agreed statement (similar to the Linux kernel patch submission guidelines). Git commit procedures may be ensured through a server-side pre-receive hook. The history of requirements may be also added to the specification items directly in a revision attribute. This would make it possible to generate the history information for documents without having the Git repository available, e.g. from an RTEMS source release archive.

5.3.2. Backward Traceability of Specification Items

Providing backward traceability of specification items means that we must be able to find the corresponding higher level specification item for each refined specification item. This is a standard Doorstop feature.

5.3.3. Forward Traceability of Specification Items

Providing forward traceability of specification items means that we must be able to find all the refined specification items for each higher level specification item. This is a standard Doorstop feature. The links from parent to child specification items are implicitly defined by links from a child item to a parent item.

5.3.4. Traceability between Software Requirements, Architecture and Design

The software requirements are implemented in Doorstop compatible YAML files. The software architecture and design is written in Doxygen markup. Doxygen markup is used throughout all header and source files. A Doxygen filter program may be provided to place Doxygen markup in assembler files. The software architecture is documented via Doxygen groups. Each Doxygen group name should have a project-specific name and the name should be unique within the project, e.g. RTEMSTopLevelMidLevelLowLevel. The link from a Doxygen group to its parent group is realized through the @ingroup special command. The link from a Doxygen group or software component to the corresponding requirement is realized through a @satisfy{req} custom command which needs the identifier of the requirement as its one and only parameter. Only links to parents are explicitly given in the Doxygen markup. The links from a parent to its children are only implicitly specified via the link from a child to its parent. So, a tool must process all files to get the complete hierarchy of software requirements, architecture and design. Links from a software component to another software component are realized through automatic Doxygen references or the @ref and @see special commands.

5.4. Requirement Validation

The validation of each requirement shall be accomplished by one or more of the following methods and nothing else:

  • By test: A Test Case specification item is provided to demonstrate that the requirement is satisfied when the software product is executed on the target platform.

  • By analysis: A statement is provided how the requirement is met, by analysing static properties of the software product.

  • By inspection: A statement is provided how the requirement is met, by inspection of the source code.

  • By review of design: A rationale is provided to demonstrate how the qualification requirement is satisfied implicitly by the software design.

Validation by test is strongly recommended. The choice of any other validation method shall be strongly justified. The requirements author is obligated to provide the means to validate the requirement with detailed instructions.

Doorstop

For an item in a parent document it is checked that at least one item in a child document has a link to it. For example a child document could contain validation items. With this feature you can check that all requirements are covered by at least one validation item.

The requirement validation by analysis, by inspection, and by design specification items shall have the following attribute specializations:

type

The type attribute value shall be validation-by-analysis, validation-by-inspection, or validation-by-review-of-design.

link

There shall be exactly one link to the validated requirement.

text

The statement or rational of the requirement validation.

5.5. Requirement Management

5.5.1. Change Control Board

Working with requirements usually involves a Change Control Board (CCB). The CCB of the RTEMS Project is the RTEMS developer mailing list.

There are the following actors involved:

  • RTEMS users: Everyone using the RTEMS real-time operating system to design, develop and build an application on top of it.

  • RTEMS developers: The persons developing and maintaining RTEMS. They write patches to add or modify code, requirements, tests and documentation.

  • RTEMS maintainers: They are listed in the MAINTAINERS file and have write access to the project repositories.

Adding and changing requirements follows the normal patch review process. The normal patch review process is described in the RTEMS User Manual. Reviews and comments may be submitted by anyone, but a maintainer review is required to approve significant changes. In addition for significant changes, there should be at least one reviewer with a sufficient independence from the author which proposes a new requirement or a change of an existing requirement. Working in another company on different projects is sufficiently independent. RTEMS maintainers do not know all the details, so they trust in general people with experience on a certain platform. Sometimes no review comments may appear in a reasonable time frame, then an implicit agreement to the proposed changes is assumed. Patches can be sent at anytime, so controlling changes in RTEMS requires a permanent involvement on the RTEMS developer mailing list.

For a qualification of RTEMS according to certain standards, the requirements may be approved by an RTEMS user. The approval by RTEMS users is not the concern of the RTEMS Project, however, the RTEMS Project should enable RTEMS users to manage the approval of requirements easily. This information may be also used by a independent authority which comes into play with an Independent Software Verification and Validation (ISVV). It could be used to select a subset of requirements, e.g. look only at the ones approved by a certain user. RTEMS users should be able to reference the determinative content of requirements, test procedures, test cases and justification reports in their own documentation. Changes in the determinative content should invalidate all references to previous versions.

5.5.2. Add a Requirement

_images/req-add.png

5.5.3. Modify a Requirement

_images/req-modify.png

5.5.4. Mark a Requirement as Obsolete

Requirements shall be never removed. They shall be marked as obsolete. This ensures that requirement identifiers are not reused. The procedure to obsolete a requirement is the same as the one to modify a requirement.

5.6. Tooling

5.6.1. Tool Requirements

To manage requirements some tool support is helpful. Here is a list of requirements for the tool:

  • The tool shall be open source.

  • The tool should be actively maintained during the initial phase of the RTEMS requirements specification.

  • The tool shall use plain text storage (no binary formats, no database).

  • The tool shall support version control via Git.

  • The tool should export the requirements in a human readable form using the Sphinx documentation framework.

  • The tool shall support traceability of requirements to items external to the tool.

  • The tool shall support traceability between requirements.

  • The tool shall support custom requirement attributes.

  • The tool should ensure that there are no cyclic dependencies between requirements.

  • The tool should provide an export to ReqIF.

5.6.2. Tool Evaluation

During an evaluation phase the following tools were considered:

The tools aNimble, OSRMT and Requirement Heap were not selected since they use a database. The tools Papyrus, ProR and ReqIF are Eclipse based and use complex XML files for data storage. They were difficult to use and lack good documentation/tutorials. The tools rmToo and Doorstop turned out to be the best candidates to manage requirements in the RTEMS Project. The Doorstop tool was selected as the first candidate mainly due a recommendation by an RTEMS user.

5.6.3. Selected Tool - Doorstop

Doorstop is a requirements management tool. It has a modern, object-oriented and well-structured implementation in Python 3.6 under the LGPLv3 license. It uses a continuous integration build with style checkers, static analysis, documentation checks, code coverage, unit test and integration tests. In 2019, the project was actively maintained. Pull requests for minor improvements and new features were reviewed and integrated within days. Each requirement is contained in a single file in YAML format. Requirements are organized in documents and can be linked to each other [BA14].

Doorstop consists of three main parts

  • a stateless command line tool doorstop,

  • a file format with a pre-defined set of attributes (YAML), and

  • a primitive GUI tool (not intended to be used).

For RTEMS, its scope will be extended to manage specifications in general. The primary reason for selecting Doorstop as the requirements management tool for the RTEMS Project is its data format which allows a high degree of customization. Doorstop uses a directed, acyclic graph of items. The items are files in YAML format. Each item has a set of standard attributes (key-value pairs).

The use case for the standard attributes is requirements management. However, Doorstop is capable to manage custom attributes as well. We will heavily use custom attributes for the specification items. Enabling Doorstop to effectively use custom attributes was done specifically for the RTEMS Project in several patch sets.

A key feature of Doorstop is the fingerprint of items. For the RTEMS Project, the fingerprint hash algorithm was changed from MD5 to SHA256. In 2019, it can be considered cryptographically secure. The fingerprint should cover the normative values of an item, e.g. comments etc. are not included. The fingerprint helps RTEMS users to track the significant changes in the requirements (in contrast to all the changes visible in Git). As an example use case, a user may want to assign a project-specific status to specification items. This can be done with a table which contains columns for

  1. the UID of the item,

  2. the fingerprint, and

  3. the project-specific status.

Given the source code of RTEMS (which includes the specification items) and this table, it can be determined which items are unchanged and which have another status (e.g. unknown, changed, etc.).

6. Software Development Management

6.1. Software Development (Git Users)

6.1.1. Browse the Git Repository Online

You can browse all available repositories online by accessing https://git.rtems.org/.

6.1.2. Using the Git Repository

The following examples demonstrate how to use the RTEMS’ Git repos. These examples are provided for the main rtems module, but they are also valid for the other modules.

First, we need to obtain our own local copy of the RTEMS Git repository:

git clone git://git.rtems.org/rtems.git rtems

This command will create a folder named rtems in the current directory. This folder will contain a full-featured RTEMS’ Git repository and the current HEAD revision checked out. Since all the history is available we can check out any release of RTEMS. Major RTEMS releases are available as separate branches in the repo.

To see all available remote branches issue the following command:

git branch -r

We can check out one of those remote branches (e.g. rtems-4.10 branch) using the command:

git checkout -b rtems410 origin/4.10

This will create a local branch named “rtems410”, containing the rtems-4.10 release, that will track the remote branch “rtems-4-10-branch” in origin (git://git.rtems.org/rtems.git). The git branch command prints a list of the current local branches, indicating the one currently checked out.

If you want to switch between local branches:

git checkout <branch-name>

With time your local repository will diverge from the main RTEMS repository. To keep your local copy up to date you need to issue:

git pull origin

This command will update all your local branches with any new code revisions available on the central repository.

6.1.3. Making Changes

Git allows you to make changes in the RTEMS source tree and track those changes locally. We recommend you make all your changes in local branches. If you are working on a few different changes or a progression of changes it is best to use a local branch for each change.

A branch for each change lets your repo’s master branch track the upstream RTEMS’ master branch without interacting with any of the changes you are working on. A completed change is emailed to the developer’s list for review and this can take time. While this is happening the upstream’s master branch may be updated and you may need to rebase your work and test again if you are required to change or update your patch. A local branch isolates a specific change from others and helps you manage the process.

First, you need to clone the repository:

git clone git://git.rtems.org/rtems.git rtems

Or if you already cloned it before, then you might want to update to the latest version before making your changes:

cd rtems
git pull

Create a local branch to make your changes in, in this example, the change is faster-context-switch:

git checkout -b faster-context-switch

Next, make your changes to files. If you add, delete ormove/rename files you need to inform Git

git add /some/new/file
git rm /some/old/file
git mv /some/old/file /some/new/file

When you’re satisfied with the changes you made, commit them (locally)

git commit -a

The -a flag commits all the changes that were made, but you can also control which changes to commit by individually adding files as you modify them by using. You can also specify other options to commit, such as a message with the -m flag.

git add /some/changed/files
git commit

Create a patch from your branch, in this case, we have two commits we want to send for review:

 git format-patch -2

There are new changes pushed to the RTEMS' master branch and our local branch
needs to be updated:
git checkout master
git pull
git checkout faster-context-switch
git rebase master

6.1.4. Working with Branches

Branches facilitate trying out new code and creating patches.

The previous releases of RTEMS are available through remote branches. To check out a remote branch, first query the Git repository for the list of branches:

git branch -r

Then check out the desired remote branch, for example:

git checkout -b rtems410 origin/4.10

Or if you have previously checked out the remote branch then you should see it in your local branches:

git branch

You can change to an existing local branch easily:

git checkout rtems410

You can also create a new branch and switch to it:

git branch temporary
git checkout temporary

Or more concisely:

git checkout -b temporary

If you forget which branch you are on

git branch

shows you by placing a * next to the current one.

When a branch is no longer useful you can delete it.

git checkout master
git branch -d temporary

If you have unmerged changes in the old branch Git complains and you need to use -D instead of -d.

6.1.5. Viewing Changes

To view all changes since the last commit:

git diff HEAD

To view all changes between the current branch and another branch, say master:

git diff master..HEAD

To view descriptions of committed changes:

git log

Or view the changeset for some file (or directory):

git log /some/file

To view the changesets made between two branches:

git log master..HEAD

Or for a more brief description use shortlog:

git shortlog master..HEAD

6.1.6. Reverting Changes

To remove all (uncommitted) changes on a branch

git checkout -f

Or to selectively revert (uncommited) files, for example if you accidentally deleted ./some/file

git checkout -- ./some/file

or

git checkout HEAD ./some/file

To remove commits there are two useful options, reset and revert. git reset should only be used on local branches that no one else is accessing remotely. git revert is cleaner and is the right way to revert changes that have already been pushed/pulled remotely.

6.1.7. git reset

git reset is a powerful and tricky command that should only be used on local (un-pushed) branches): A good description of what it enables to do can be found here. The following are a few useful examples. Note that adding a ~ after HEAD refers to the most recent commit, and you can add a number after the ~ to refer to commits even further back; HEAD by itself refers to the current working directory (changes since the last commit).

git reset HEAD~

Will undo the last commit and unstage those changes. Your working directory will remain the same, therefore a git status will yield any changes you made plus the changes made in your last commit. This can be used to fix the last commit. You will need to add the files again.

git reset --soft HEAD~

Will just undo the last commit. The changes from the last commit will still be staged (just as if you finished git adding them). This can be used to amend the last commit (e.g. You forgot to add a file to the last commit).

git reset --hard HEAD~

Will revert everything, including the working directory, to the previous commit. This is dangerous and can lead to you losing all your changes; the --hard flag ignores errors.

git reset HEAD

Will unstage any change. This is used to revert a wrong git add. (e.g. You added a file that shouldn’t be there, but you haven’t ‘committed’)

Will revert your working directory to a HEAD state. You will lose any change you made to files after the last commit. This is used when you just want to destroy all changes you made since the last commit.

6.1.8. git revert

git revert does the same as reset but creates a new commit with the reverted changes instead of modifying the local repository directly.

git revert HEAD

This will create a new commit which undoes the change in HEAD. You will be given a chance to edit the commit message for the new commit.

6.1.9. Merging Changes

Suppose you commit changes in two different branches, branch1 and branch2, and want to create a new branch containing both sets of changes:

git checkout -b merged
git merge branch1
git merge branch2

Or you might want to bring the changes in one branch into the other:

git checkout branch1
git merge branch2

And now that branch2 is merged you might get rid of it:

git branch -d branch2

If you have done work on a branch, say branch1, and have gone out-of-sync with the remote repository, you can pull the changes from the remote repo and then merge them into your branch:

git checkout master
git pull
git checkout branch1
git merge master

If all goes well the new commits you pulled into your master branch will be merged into your branch1, which will now be up-to-date. However, if branch1 has not been pushed remotely then rebasing might be a good alternative to merging because the merge generates a commit.

6.1.10. Rebasing

An alternative to the merge command is rebase, which replays the changes (commits) on one branch onto another. git rebase finds the common ancestor of the two branches, stores each commit of the branch you are on to temporary files and applies each commit in order.

For example

git checkout branch1
git rebase master

or more concisely

git rebase master branch1

will bring the changes of master into branch1, and then you can fast-forward master to include branch1 quite easily

git checkout master
git merge branch1

Rebasing makes a cleaner history than merging; the log of a rebased branch looks like a linear history as if the work was done serially rather than in parallel. A primary reason to rebase is to ensure commits apply cleanly on a remote branch, e.g. when submitting patches to RTEMS that you create by working on a branch in a personal repository. Using rebase to merge your work with the remote branch eliminates most integration work for the committer/maintainer.

There is one caveat to using rebase: Do not rebase commits that you have pushed to a public repository. Rebase abandons existing commits and creates new ones that are similar but different. If you push commits that others pull down, and then you rewrite those commits with git rebase and push them up again, the others will have to re-merge their work and trying to integrate their work into yours can become messy.

6.1.11. Accessing a developer’s repository

RTEMS developers with Git commit access have personal repositories on https://git.rtems.org/ that can be cloned to view cutting-edge development work shared there.

6.1.12. Creating a Patch

Before submitting a patch read about Contributing to RTEMS and the Commit Message formatting we require.

The recommended way to create a patch is to branch the Git repository master and use one commit for each logical change. Then you can use git format-patch to turn your commits into patches and easily submit them.

git format-patch master

Creates a separate patch for each commit that has been made between the master branch and the current branch and writes them in the current directory. Use the -o flag to redirect the files to a different directory.

If you are re-submitting a patch that has previously been reviewed, you should specify a version number for your patch, for example, use

git format-patch -v2 ...

to indicate the second version of a patch, -v3 for a third, and so forth.

Patches created using git format-patch are formatted so they can be emailed and rely on having Git configured with your name and email address, for example

git config --global user.name "Your Name"
git config --global user.email name@domain.com

Please use a real name, we do not allow pseudonyms or anonymous contributions.

6.1.13. Submitting a Patch

Using git send-email you can easily contribute your patches. You will need to install git send-email first:

sudo yum install git-email

or

sudo dnf install git-email

or

sudo apt install git-email

Then you will need to configure an SMTP server. You could install one on your localhost, or you can connect to a mail server such as Gmail.

6.1.14. Configuring git send-email to use Gmail

Configure Git to use Gmail:

git config --global sendemail.smtpserver smtp.gmail.com
git config --global sendemail.smtpserverport 587
git config --global sendemail.smtpencryption tls
git config --global sendemail.smtpuser your_email@gmail.com

It will ask for your password each time you use git send-email. Optionally you can also put it in your git config:

git config --global sendemail.smtppass your_password

6.1.15. Sending Email

To send your patches just

git send-email /path/to/patch --to devel@rtems.org

To send multiple related patches (if you have more than one commit in your branch) specify a path to a directory containing all of the patches created by git format-patch. git send-email has some useful options such as:

  • --annotate to show/edit your patch

  • --cover-letter to prepend a summary

  • --cc=<address> to cc someone

You can configure the to address:

git config --global sendemail.to devel@rtems.org

So all you need is:

git send-email /path/to/patch

6.1.16. Troubleshooting

Some restrictive corporate firewalls block access through the Git protocol (git://). If you are unable to reach the server git://git.rtems.org/ you can try accessing through http. To clone the rtems repository using the http protocol use the following command:

git clone http://git.rtems.org/rtems/ rtems

This access through http is slower (way slower!) than through the git protocol, therefore, the Git protocol is preferred.

6.1.17. Manage Your Code

You may prefer to keep your application and development work in a Git repository for all the good reasons that come with version control. For public repositories, you may like to try GitHub or BitBucket. RTEMS maintains mirrors on GitHub which can make synchronizing with upstream changes relatively simple. If you need to keep your work private, you can use one of those services with private repositories or manage your own server. The details of setting up a server are outside the scope of this document, but if you have a server with SSH access you should be able to find instructions on how to set up Git access. Once you have git configured on the server, adding repositories is a snap.

6.1.18. Private Servers

In the following, replace @USER@ with your username on your server, @REPO@ with the name of your repository, and @SERVER@ with your server’s name or address.

To push a mirror to your private server, first create a bare repository on your server.

cd /home/@USER@
mkdir git
mkdir git/@REPO@.git
cd git/@REPO@.git
git --bare init

Now from your client machine (e.g. your work laptop/desktop), push a git, perhaps one you cloned from elsewhere, or one that you made locally with git init, by adding a remote and pushing:

git remote add @SERVER@ ssh://@SERVER@/home/@USER@/git/@REPO@.git
git push @SERVER@ master

You can replace the @SERVER@ with another name for your remote if you like. And now you can push other branches that you might have created. Now you can push and pull between your client and your server. Use SSH keys to authenticate with your server if you want to save on password typing; remember to put a passphrase on your SSH key if there is a risk the private key file might get compromised.

The following is an example scenario that might be useful for RTEMS users that uses a slightly different approach than the one just outlined:

ssh @SERVER@
mkdir git
git clone --mirror git://git.rtems.org/rtems.git
## Add your ssh key to ~/.ssh/authorized_keys
exit
git clone ssh://@SERVER@/home/@USER@/git/rtems.git
cd rtems
git remote add upstream git://git.rtems.org/rtems.git
git fetch upstream
git pull upstream master
git push
## If you want to track RTEMS on your personal master branch,
## you should only push changes to origin/master that you pull
## from upstream. The basic workflow should look something like:
git checkout master
git pull upstream master
git push
git checkout -b anewbranch
## Repeat: do work, git commit -a
git push origin anewbranch

## delete a remote branch
git push origin :anewbranch
## delete a local branch
git branch -d anewbranch

6.1.19. Learn more about Git

Links to the sites with good Git information:

6.2. Software Development (Git Writers)

6.2.1. SSH Access

Currently all committer’s should have an ssh account on the main git server, dispatch.rtems.org. If you have been granted commit access and do have an account on dispatch.rtems.org one should be requested on the devel@ list. SSH access for git uses key logins instead of passwords. The key should be at least 1024 bits in length.

The public repositories can by cloned with

git clone ssh://user@dispatch.rtems.org/data/git/rtems.git

Or replace rtems.git with another repo to clone another one.

6.2.2. Personal Repository

Personal repositories keep the clutter away from the master repository. A user with a personal repository can make commits, create and delete branches, plus more without interfering with the master repository. Commits to the master repository generate email to the vc@ list and development type commits by a developer would only add noise and lessen the effectiveness of the commit list

A committer should maintain a personal clone of the RTEMS repository through which all changes merged into the RTEMS head are sent. The personal repository is also a good place for committers to push branches that contain works in progress. The following instructions show how to setup a personal repositor that by default causes commits to go to your private local repository and pushes to go to your publicly visible personal repository. The RTEMS head is configured as a remote repository named ‘upstream’ to which you can push changes that have been approved for merging into RTEMS.

Branches aren’t automatically pushed until you tell git to do the initial push after which the branch is pushed automatically. In order to keep code private just put it on a branch in your local clone and do not push the branch.

6.2.3. Create a personal repository

Set up the server side repository. In the following substitute user with your username.

# ssh git.rtems.org
[user@git ~]$ ln -s /data/git/user git
[user@git ~]$ ls -l
lrwxrwxrwx 1 user rtems 16 Feb  1 11:52 git -> /data/git/user
[user@git ~]$ cd git
[user@git git]$ git clone --mirror /data/git/rtems.git

Provide a description for the repository, for example “Clone of master repository.”

[user@git git]$ echo "Clone of master repository." > rtems.git/description
[user@git git]$ logout

Clone the repository on your local machine

# git clone ssh://user@dispatch.rtems.org/home/user/git/rtems.git
# cd rtems

Add the RTEMS repository as a remote repository and get the remote tags and branches

# git remote add upstream ssh://user@dispatch.rtems.org/data/git/rtems.git
# git fetch upstream

After a little while you should be able to see your personal repo at https://git.rtems.org/@USER@/rtems.git/ and you can create other repositories in your git directory that will propagate to https://git.rtems.org/@USER@/ if you need. For example, joel’s personal repos appear at https://git.rtems.org/joel/.

Git Personal Repositories
6.2.3.1. Check your setup
git remote show origin

Should print something similar to

* remote origin
  Fetch URL: ssh://user@dispatch.rtems.org/home/user/git/rtems.git
  Push  URL: ssh://user@dispatch.rtems.org/home/user/git/rtems.git
  HEAD branch: master
  Remote branches:
    4.10   tracked
    4.8    tracked
    4.9    tracked
    master tracked
  Local branch configured for 'git pull':
    master merges with remote master
  Local ref configured for 'git push':
    master pushes to master (up to date)
6.2.3.2. Push commits to personal repo master from local master
# git push
6.2.3.3. Push a branch onto personal repo
# git push origin branchname
6.2.3.4. Update from upstream master (RTEMS head)

When you have committed changes on a branch that is private (hasn’t been pushed to your personal repo) then you can use rebase to obtain a linear history and avoid merge commit messages.

# git checkout new_features
# git pull --rebase upstream master

If you cannot do a fast-forward merge then you could use the --no-commit flag to prevent merge from issuing an automatic merge commit message.

When you have committed changes on a branch that is public/shared with another developer you should not rebase that branch.

6.2.4. GIT Push Configuration

People with write access to the main repository should make sure that they push the right branch with the git push command. The above setup ensures that git push will not touch the main repository, which is identified as upstream, unless you specify the upstream (by git push upstream master).

Lets suppose we have a test branch intended for integration into the master branch of the main repository.

# git branch
  master
 *  test

There are two options for pushing with the branch. First,

# git push origin test

Will push the test branch to the personal repository. To delete the remote branch

# git push origin :test

You’ll still need to delete your local branch if you are done with it.

If you are going to work exclusively with one branch for a while, you might want to configure git to automatically push that branch when you use git push. By default git push will use the local master branch, but you can use the test branch as the source of your changes:

# git config remote.origin.push test:master

Now git push will merge into your master branch on your personal repository. You can also setup a remote branch:

# git config remote.origin.push test:test

You can see what branch is configured for pushing with

# git remote show origin

And reset to the default

# git config remote.origin.push master

6.2.5. Pull a Developer’s Repo

The procedures for creating personal repositories ensure that every developer can post branches that anyone else can review. To pull a developer’s personal repository into your local RTEMS git clone, just add a new remote repo:

# git remote add devname git://dispatch.rtems.org/devname/rtems.git
# git fetch devname
# git remote show devname
# git branch -a

Replace devname with the developer’s user name on git, which you can see by accessing https://git.rtems.org. Now you can switch to the branches for this developer.

Use a tracking branch if the developer’s branch is changing:

# git branch --track new_feature devname/new_feature

6.2.6. Committing

6.2.6.1. Ticket Updates

Our trac instance supports updating a related ticket with the commit message.

Any references to a ticket for example #1234 will insert the message into he ticket as an ‘update’. No command is required.

Closing a ticket can be done by prefixing the ticket number with any of the following commands:

close, closed, closes, fix, fixed, or fixes

For example:

closes #1234

This is a random update it closes #1234 and updates #5678

6.2.6.2. Commands

When merging someone’s work, whether your own or otherwise, we have some suggested procedures to follow.

  • Never work in the master branch. Checkout a new branch and apply patches/commits to it.

  • Before pushing upstream: - Update master by fetching from the server - Rebase the working branch against the updated master - Push the working branch to the server master

The basic workflow looks like

# git checkout -b somebranch upstream/master
# patch .. git add/rm/etc
# git commit ...
# git pull --rebase upstream master
# git push upstream somebranch:master

If someone pushed since you updated the server rejects your push until you are up to date.

For example a workflow where you will commit a series of patches from ../patches/am/ directory:

# git checkout -b am
# git am ../patches/am*
# git pull --rebase upstream master
# git push upstream am:master
# git checkout master
# git pull upstream master
# git log
# git branch -d am
# git push

The git log stage will show your newly pushed patches if everything worked properly, and you can delete the am branch created. The git push at the end will push the changes up to your personal repository.

Another way to do this which pushes directly to the upstream is shown here in an example which simply (and quickly) applies a patch to the branch:

git checkout -b rtems4.10 --track remotes/upstream/4.10
cat /tmp/sp.diff | patch
vi sparc.t
git add sparc.t
git commit -m "sparc.t: Correct for V8/V9"
git push upstream rtems4.10:4.10
git checkout master
git log
git branch -d rtems4.10

6.2.7. Pushing Multiple Commits

A push with more than one commit results in Trac missing them. Please use the following script to push a single commit at a time:

#! /bin/sh
commits=$(git log --format='%h' origin/master..HEAD | tail -r)
for c in $commits
do
  cmd=$(echo $c | sed 's%\(.*\)%git push origin \1:master%')
  echo $cmd
$cmd
done

6.2.8. Ooops!

So you pushed something upstream and broke the repository. First things first: stop what you’re doing and notify devel@… so that (1) you can get help and (2) no one pulls from the broken repo. For an extended outage also notify users@…. Now, breathe easy and let’s figure out what happened. One thing that might work is to just undo the push. To get an idea of what you did, run git reflog, which might be useful for getting assistance in undoing whatever badness was done.

6.3. Coding Standards

TBD - Write introduction, re-order, identify missing content

6.3.1. Coding Conventions

The style of RTEMS is generally consistent in the core areas. This page attempts to capture generally accepted practices. When in doubt, consult the code around you or look in cpukit/rtems. See the sister page Doxygen Recommendations. for examples that illustrate style rules and Doxygen usage.

6.3.1.1. Source Documentation
  • Use Doxygen according to our Doxygen Recommendations..

  • Start each file with a brief description followed by a license. See Boilerplate File Header..

  • Use /* */ comments.

  • Use comments wisely within function bodies, to explain or draw attention without being verbose.

  • Use English prose and strive for good grammar, spelling, and punctuation.

  • Use TODO: with a comment to indicate code that needs improvement. Make it clear what there is to do.

  • Use XXX or FIXME to indicate an error/bug/broken code.

6.3.1.2. Licenses

The RTEMS Project has strict requirements on the types of software licenses that apply to software it includes and distributes. Submissions will be summarily rejected that do not follow the correct license or file header requirements.

6.3.1.3. Language and Compiler
  • Use C99.

  • Treat warnings as errors: eliminate them.

  • Favor C, but when assembly language is required use inline assembly if possible.

  • Do not use compiler extensions.

  • Use the RTEMS_macros defined in score/basedefs.h for abstracting compiler-specific features.

  • Use NULL for the null pointer, and prefer to use explicit checks against NULL, e.g.,

    if ( ptr != NULL )
    

    instead of

    if ( !ptr )
    
  • Use explicit checks for bits in variables.
    • Example 1: Use
      if ( XBITS == (var & XBITS) )
      

      to check for a set of defined bits.

    • Example 2: Use
      if ( (var & X_FLAGS) != 0) )
      
      instead of
      if ( !!(var & X_FLAGS) )
      

      to check for at least 1 defined bit in a set.

  • Use ‘(void) unused;’ to mark unused parameters and set-but-unused variables immediately after being set.

  • Do not put function prototypes in C source files, any global functions should have a prototype in a header file and any private function should be declared static.

  • Declare global variables in exactly one header file. Define global variables in at most one source file. Include the header file declaring the global variable as the first include file if possible to make sure that the compiler checks the declaration and definition and that the header file is self-contained.

  • Do not cast arguments to any printf() or printk() variant. Use <inttypes.h> PRI constants for the types supported there. Use <rtems/inttypes.h> for the other POSIX and RTEMS types that have PRI constants defined there. This increases the portability of the printf() format.

  • Do not use the register keyword. It is deprecated since C++14.

6.3.1.4. Formatting
  • Use spaces instead of tabs.

  • Use two spaces for indentation, four spaces for hanging indentation.

  • Adhere to a limit of 80 characters per line..

  • Put function return types and names on one line if they fit.

  • Put function calls on one line if they fit.

  • No space between a function name or function-like macro and the opening parens.

  • Put braces on the same line as and one space after the conditional expression ends.

  • Put the opening brace of a function definition one line after the closing parenthesis of its prototype.

  • Put a single space inside and outside of each parenthesis of a conditional expression. * Exception: never put a space before a closing semi-colon.

  • Put a single space before and after ternary operators.

  • Put a single space before and after binary operators.

  • Put no space between unary operators (e.g. *, &, !, ~, ++, –) and their operands.

  • No spaces around dereferencing operators (-> and .).

  • Do not use more than one blank line in a row.

  • Do not use trailing whitespace at the end of a line.

6.3.1.5. Readability
  • Understand and follow the naming rules..

  • Use typedef to remove ‘struct’, but do not use typedef to hide pointers or arrays. * Exception: typedef can be used to simplify function pointer types.

  • Do not mix variable declarations and code.

  • Declare variables at the start of a block.

  • Only use primitive initialization of variables at their declarations. Avoid complex initializations or function calls in variable declarations.

  • Do not put unrelated functions or data in a single file.

  • Do not declare functions inside functions.

  • Avoid deep nesting by using early exits e.g. return, break, continue. * Parameter checking should be done first with early error returns. * Avoid allocation and critical sections until error checking is done. * For error checks that require locking, do the checks early after acquiring locks. * Use of ‘goto’ requires good reason and justification.

  • Test and action should stay close together.

  • Avoid complex logic in conditional and loop statements.

  • Put conditional and loop statements on the line after the expression.

  • Favor inline functions to hide compile-time feature-conditioned compilation..

  • Define non-inline functions in a .c source file.

  • Declare all global (non-static) functions in a .h header file.

  • Declare and define inline functions in one place. Usually, this is a *impl.h header file.

  • Declare and define static functions in one place. Usually, this is toward the start of a .c file. Minimize forward declarations of static functions.

  • Function declarations should include variable names.

  • Avoid excess parentheses. Learn the operator precedence. rules.

  • Always use parentheses with sizeof. This is an exception to the rule about excess parentheses.

6.3.1.6. Robustness
  • Check all return statuses.

  • Validate input parameters.

  • Use debug assertions (assert).

  • Use const when appropriate for read-only function parameters and compile-time constant values.

  • Do not hard code limits such as maximum instances into your code.

  • Prefer to use sizeof(variable) instead of sizeof(type).

  • Favor C automatic variables over global or static variables.

  • Use global variables only when necessary and ensure atomicity of operations.

  • Do not shadow variables.

  • Avoid declaring large buffers or structures on the stack.

  • Avoid using zero (0) as a valid value. Memory often defaults to being zero.

  • Favor mutual exclusion primitives over disabling preemption.

  • Avoid unnecessary dependencies, such as by not calling ‘’printf()’’ on error paths.

  • Avoid inline functions and macros with complicated logic and decision points.

  • Prefer inline functions, enum, and const variables instead of CPP macros.

  • CPP macros should use a leading underscore for parameter names and avoid macro pitfalls..

6.3.1.7. Portability
  • Think portable! RTEMS supports a lot of target hardware.

  • For integer primitives, prefer to use precise-width integer types from C99 stdint.h.

  • Write code that is 16-bit, 32-bit, and 64-bit friendly.

6.3.1.8. Maintainability
  • Minimize modifications to third-party code..

  • Keep it simple! Simple code is easier to debug and easier to read than clever code.

  • Share code with other architectures, CPUs, and BSPs where possible.

  • Do not duplicate standard OS or C Library routines.

6.3.1.9. Performance
  • Prefer algorithms with the lowest order of time and space. for fast, deterministic execution times with small memory footprints.

  • Understand the constraints of real-time programming.. Limit execution times in interrupt contexts and critical sections, such as Interrupt and Timer Service Routines (TSRs).

  • Functions used only through function pointers should be declared ‘static inline’ (RTEMS_INLINE_ROUTINE)

  • Prefer to ++preincrement instead of postincrement++.

  • Avoid using floating point except where absolutely necessary.

6.3.1.10. Miscellaneous
  • If you need to temporarily change the execution mode of a task/thread, restore it.

  • If adding code to ‘’cpukit’’ be sure the filename is unique since all files under that directory get merged into a single library.

6.3.1.11. Layering
6.3.1.12. Exceptions to the Rules
  • Minimize reformatting existing code in RTEMS unless the file undergoes substantial non-style changes.

  • Third-party code. should not be reformatted to fit RTEMS style. Exception: unmaintained third-party code adopted and maintained by RTEMS may be reformatted, subject to the above rules.

6.3.1.13. Tools

Some of the above can be assisted by tool support. Feel free to add more tools, configurations, etc here.

6.3.2. Eighty Character Line Limit

If you find yourself with code longer than 80 characters, first ask yourself whether the nesting level is too deep, names too long, compound expressions too complicated, or if some other guideline for improving readability can help to shrink the line length. Refactoring nested blocks into functions can help to alleviate code width problems while improving code readability. Making names descriptive yet terse can also improve readability. If absolutely necessary to have a long line, follow the rules on this page to break the line up to adhere to the 80 characters per line rule.

6.3.2.1. Breaking long lines

if, while, and for loops have their condition expressions aligned and broken on separate lines. When the conditions have to be broken, none go on the first line with the if, while, or for statement, and none go on the last line with the closing parenthesis and (optional) curly brace. Long statements are broken up and indented at operators, with an operator always being the last token on a line. No blank spaces should be left at the end of any line. Here is an example with a for loop.

for ( initialization = statement; a + really + long + statement + that + evaluates + to < a + boolean; another + statement++ ) {
  z = a + really + long + statement + that + needs + two + lines + gets + indented + four + more + spaces + on + the + second + and + subsequent + lines + and + broken + up + at + operators;
}

Should be replaced with

for (
  initialization = statement;
  a + really + long + statement + that + evaluates + to <
  a + boolean;
  another + statement++
) {
  z = a + really + long + statement + that + needs +
      two + lines + gets + indented + four + more +
      spaces + on + the + second + and + subsequent +
      lines + and + broken + up + at + operators;
}

Note that indentations should add 2 nesting levels (4 space characters, not tabs).

Similarly,

if ( this + that < those && this + these < that && this + those < these && this < those && those < that ) {

should be broken up like

if (
  this + that < those &&
  this + these < that &&
  this + those < these &&
  this < those &&
  those < that
) {

Note that each expression that resolves to a boolean goes on its own line. Where you place the boolean operator is a matter of choice.

When a line is long because of a comment at the end, move the comment to just before the line, for example

#define A_LONG_MACRO_NAME (AND + EXPANSION) /* Plus + a + really + long + comment */

can be replaced with

/* Plus + a + really + long + comment */
#define A_LONG_MACRO_NAME (AND + EXPANSION)

C Preprocessor macros need to be broken up with some care, because the preprocessor does not understand that it should eat newline characters. So

#define A_LONG_MACRO_NAME (AND + EXCESSIVELY + LONG + EXPANSION + WITH + LOTS + OF + EXTRA + STUFF + DEFINED)

would become

#define A_LONG_MACRO_NAME ( \
  AND + EXCESSIVELY + LONG + EXPANSION + WITH + LOTS + OF + EXTRA + STUFF + \
  DEFINED \
)

Notice that each line is terminated by a backslash then the carriage return. The backslash tells the preprocessor to eat the newline. Of course, if you have such a long macro, you should consider not using a macro.

Function declarations can be broken up at each argument, for example

int this_is_a_function( int arg1, int arg2, int arg3, int arg4, int arg5, int arg6, int arg7, int arg8, int arg9 );

would be broken up as

int this_is_a_function(
  int arg1,
  int arg2,
  int arg3,
  int arg4,
  int arg5,
  int arg6,
  int arg7,
  int arg8,
  int arg9
);

Excessively long comments should be broken up at a word boundary or somewhere that makes sense, for example

/* Excessively long comments should be broken up at a word boundary or somewhere that makes sense, for example */

would be

/* Excessively long comments should be broken up at a word boundary or
 * somewhere that makes sense, for example */

Note that multiline comments have a single asterisk aligned with the asterisk in the opening /*. The closing */ should go at the end of the last line.

6.3.3. Deprectating Interfaces

TBD - Convert the following to Rest and insert into this file TBD - https://devel.rtems.org/wiki/Developer/Coding/Deprecating

6.3.4. Doxygen Guidelines

6.3.4.1. Group Names

Doxygen group names shall use CamelCase. In the RTEMS source code, CamelCase is rarely used, so this makes it easier to search and replace Doxygen groups. It avoids ambiguous references to functions, types, defines, macros, and groups. All groups shall have an RTEMS prefix. This makes it possible to include the RTEMS files with Doxygen comments in a larger project without name conflicts.

/**
 * @defgroup RTEMSScoreThread
 *
 * @ingrop RTEMSScore
 *
 * ...
 */
6.3.4.2. Use Groups

Every file, function declaration, type definition, typedef, define, macro and global variable declaration shall belong to at least one Doxygen group. Use @defgroup and @addtogroup with @{ and @} brackets to add members to a group. A group shall be defined at most once. Each group shall be documented with an @brief description and an optional detailed description. The @brief description shall use Title Case. Use grammatically correct sentences for the detailed descriptions.

/**
 * @defgroup RTEMSScoreThread
 *
 * @ingrop RTEMSScore
 *
 * @brief Thread Handler
 *
 * ...
 *
 * @{
 */

 ... declarations, defines ...

 /** @} */
/**
 * @addtogroup RTEMSScoreThread
 *
 * @{
 */

 ... declarations, defines ...

 /** @} */
6.3.4.3. Files

Each source or header file shall have an @file block at the top of the file. The @file block should precede the license header separated by one blank line. This placement reduces the chance of merge conflicts in imported third-party code. The @file block shall be put into a group with @ingroup GroupName. The @file block should have an @brief description and a detailed description if it is considered helpful. Use @brief @copybrief GroupName as a default to copy the @brief description from the corresponding group and omit the detailed description.

/**
 * @file
 *
 * @ingroup RTEMSScoreThread
 *
 * @brief @copybrief RTEMSScoreThread
 */
/**
 * @file
 *
 * @ingroup RTEMSScoreThread
 *
 * @brief Some helpful brief description.
 *
 * Some helpful detailed description.
 */
6.3.4.4. Type Definitions

Each type defined in a header file shall be documented with an @brief description and an optional detailed description. Each type member shall be documented with an @brief description and an optional detailed description. Use grammatically correct sentences for the detailed descriptions.

/**
 * @brief The information structure used to manage each API class of objects.
 *
 * If objects for the API class are configured, an instance of this structure
 * is statically allocated and pre-initialized by OBJECTS_INFORMATION_DEFINE()
 * through <rtems/confdefs.h>.  The RTEMS library contains a statically
 * allocated and pre-initialized instance for each API class providing zero
 * objects, see OBJECTS_INFORMATION_DEFINE_ZERO().
 */
typedef struct {
  /**
   * @brief This is the maximum valid ID of this object API class.
   *
   * This member is statically initialized and provides also the object API,
   * class and multiprocessing node information.
   *
   * It is used by _Objects_Get() to validate an object ID.
   */
  Objects_Id maximum_id;

  ... more members ...
} Objects_Information;
6.3.4.5. Function Declarations

Each function declaration or function-like macros in a header file shall be documented with an @brief description and an optional detailed description. Use grammatically correct sentences for the brief and detailed descriptions. Each parameter shall be documented with an @param entry. List the @param entries in the order of the function parameters. For non-const pointer parameters

  • use @param[out], if the referenced object is modified by the function, or

  • use @param[in, out], if the referenced object is read and modified by the function.

For other parameters (e.g. const pointer and scalar parameters) do not use the [in], [out] or [in, out] parameter specifiers. Each return value or return value range shall be documented with an @retval entry. Document the most common return value first. Use a placeholder name for value ranges, e.g. pointer in the _Workspace_Allocate() example below. In case the function returns only one value, then use @return, e.g. use @retval only if there are at least two return values or return value ranges. Use grammatically correct sentences for the parameter and return value descriptions.

/**
 * @brief Sends a message to the message queue.
 *
 * This directive sends the message buffer to the message queue indicated by
 * ID.  If one or more tasks is blocked waiting to receive a message from this
 * message queue, then one will receive the message.  The task selected to
 * receive the message is based on the task queue discipline algorithm in use
 * by this particular message queue.  If no tasks are waiting, then the message
 * buffer will be placed at the rear of the chain of pending messages for this
 * message queue.
 *
 * @param id The message queue ID.
 * @param buffer The message content buffer.
 * @param size The size of the message.
 *
 * @retval RTEMS_SUCCESSFUL Successful operation.
 * @retval RTEMS_INVALID_ID Invalid message queue ID.
 * @retval RTEMS_INVALID_ADDRESS The message buffer pointer is @c NULL.
 * @retval RTEMS_INVALID_SIZE The message size is larger than the maximum
 *   message size of the message queue.
 * @retval RTEMS_TOO_MANY The new message would exceed the message queue limit
 *   for pending messages.
 */
rtems_status_code rtems_message_queue_send(
  rtems_id    id,
  const void *buffer,
  size_t      size
);
/**
 * @brief Receives a message from the message queue
 *
 * This directive is invoked when the calling task wishes to receive a message
 * from the message queue indicated by ID. The received message is to be placed
 * in the buffer. If no messages are outstanding and the option set indicates
 * that the task is willing to block, then the task will be blocked until a
 * message arrives or until, optionally, timeout clock ticks have passed.
 *
 * @param id The message queue ID.
 * @param[out] buffer The buffer for the message content.  The buffer must be
 *   large enough to store maximum size messages of this message queue.
 * @param[out] size The size of the message.
 * @param option_set The option set, e.g. RTEMS_NO_WAIT or RTEMS_WAIT.
 * @param timeout The number of ticks to wait if the RTEMS_WAIT is set.  Use
 *   RTEMS_NO_TIMEOUT to wait indefinitely.
 *
 * @retval RTEMS_SUCCESSFUL Successful operation.
 * @retval RTEMS_INVALID_ID Invalid message queue ID.
 * @retval RTEMS_INVALID_ADDRESS The message buffer pointer or the message size
 *   pointer is @c NULL.
 * @retval RTEMS_TIMEOUT A timeout occurred and no message was received.
 */
rtems_status_code rtems_message_queue_receive(
  rtems_id        id,
  void           *buffer,
  size_t         *size,
  rtems_option    option_set,
  rtems_interval  timeout
);
/**
 * @brief Allocates a memory block of the specified size from the workspace.
 *
 * @param size The size of the memory block.
 *
 * @retval pointer The pointer to the memory block.  The pointer is at least
 *   aligned by CPU_HEAP_ALIGNMENT.
 * @retval NULL No memory block with the requested size is available in the
 *   workspace.
 */
void *_Workspace_Allocate( size_t size );
/**
 * @brief Rebalances the red-black tree after insertion of the node.
 *
 * @param[in, out] the_rbtree The red-black tree control.
 * @param[in, out] the_node The most recently inserted node.
 */
void _RBTree_Insert_color(
  RBTree_Control *the_rbtree,
  RBTree_Node    *the_node
);
/**
 * @brief Builds an object ID from its components.
 *
 * @param the_api The object API.
 * @param the_class The object API class.
 * @param node The object node.
 * @param index The object index.
 *
 * @return Returns the object ID constructed from the arguments.
 */
#define _Objects_Build_id( the_api, the_class, node, index )
6.3.4.6. Header File Examples

The <rtems/score/thread.h> and <rtems/score/threadimpl.h> header files are a good example of how header files should be documented.

6.3.5. File Templates

Every source code file shall have a copyright and license block. Corresponding to the license, every file shall have an SPDX License Identifier in the first possible line of the file. C/C++ files should have a Doxygen file comment block.

The preferred license for source code is BSD-2-Clause. The preferred license for documentation is CC-BY-SA-4.0.

6.3.5.2. C/C++ Header File Template

Use the following guidelines and template for C and C++ header files (here <foo/bar/baz.h>):

  • Place the SPDX License Identifier in the first line of the file.

  • Add a Doxygen file documentation block.

  • Place the copyright and license comment block after the documentation block.

  • For the <FIRST YEAR>, <LAST YEAR>, and <COPYRIGHT HOLDER> placeholders see Copyright and License Block.

  • Separate comment blocks by exactly one blank line.

  • Separate the Doxygen comment block from the copyright and license, so that the copyright and license information is not included in the Doxygen output.

  • For C++ header files discard the extern “C”.

/* SPDX-License-Identifier: BSD-2-Clause

/**
 * @file
 *
 * @ingroup TheGroupForThisFile
 *
 * @brief Short "Table of Contents" Description of File Contents
 *
 * A short description of the purpose of this file.
 */

/*
 * Copyright (C) <FIRST YEAR>, <LAST YEAR> <COPYRIGHT HOLDER>
 *
 * Redistribution and use in source and binary forms, with or without
 * modification, are permitted provided that the following conditions
 * are met:
 * 1. Redistributions of source code must retain the above copyright
 *    notice, this list of conditions and the following disclaimer.
 * 2. Redistributions in binary form must reproduce the above copyright
 *    notice, this list of conditions and the following disclaimer in the
 *    documentation and/or other materials provided with the distribution.
 *
 * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
 * AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
 * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
 * ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE
 * LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
 * CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
 * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
 * INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
 * CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
 * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
 * POSSIBILITY OF SUCH DAMAGE.
 */

#ifndef _FOO_BAR_BAZ_H
#define _FOO_BAR_BAZ_H

#include <foo/bar/xyz.h>

#ifdef __cplusplus
extern "C" {
#endif

/* Declarations, defines, macros, inline functions, etc. */

#ifdef __cplusplus
}
#endif

#endif /* _FOO_BAR_BAZ_H */
6.3.5.3. C/C++/Assembler Source File Template

Use the following template for C, C++, and assembler source files (here implementation of <foo/bar/baz.h>). For the <FIRST YEAR>, <LAST YEAR>, and <COPYRIGHT HOLDER> placeholders see Copyright and License Block.

/* SPDX-License-Identifier: BSD-2-Clause */

/**
 * @file
 *
 * @ingroup TheGroupForThisFile
 *
 * @brief Short "Table of Contents" Description of File Contents
 *
 * A short description of the purpose of this file.
 */

/*
 * Copyright (C) <FIRST YEAR>, <LAST YEAR> <COPYRIGHT HOLDER>
 *
 * Redistribution and use in source and binary forms, with or without
 * modification, are permitted provided that the following conditions
 * are met:
 * 1. Redistributions of source code must retain the above copyright
 *    notice, this list of conditions and the following disclaimer.
 * 2. Redistributions in binary form must reproduce the above copyright
 *    notice, this list of conditions and the following disclaimer in the
 *    documentation and/or other materials provided with the distribution.
 *
 * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
 * AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
 * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
 * ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE
 * LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
 * CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
 * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
 * INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
 * CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
 * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
 * POSSIBILITY OF SUCH DAMAGE.
 */

#ifdef HAVE_CONFIG_H
#include "config.h"
#endif

#include <foo/bar/baz.h>

/* Definitions, etc. */
6.3.5.4. Python File Template

Use the following template for Python source files and other files which accept a #-style comment block. For the <FIRST YEAR>, <LAST YEAR>, and <COPYRIGHT HOLDER> placeholders see Copyright and License Block.

#!/usr/bin/env python
# SPDX-License-Identifier: BSD-2-Clause

# File documentation block

# Copyright (C) <FIRST YEAR>, <LAST YEAR> <COPYRIGHT HOLDER>
#
# Redistribution and use in source and binary forms, with or without
# modification, are permitted provided that the following conditions
# are met:
# 1. Redistributions of source code must retain the above copyright
#    notice, this list of conditions and the following disclaimer.
# 2. Redistributions in binary form must reproduce the above copyright
#    notice, this list of conditions and the following disclaimer in the
#    documentation and/or other materials provided with the distribution.
#
# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
# AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
# IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
# ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE
# LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
# CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
# SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
# INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
# CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
# ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
# POSSIBILITY OF SUCH DAMAGE.
6.3.5.5. reStructuredText File Template

Use the following template for reStructuredText (reST) source files. For the <FIRST YEAR>, <LAST YEAR>, and <COPYRIGHT HOLDER> placeholders see Copyright and License Block.

.. SPDX-License-Identifier: CC-BY-SA-4.0

.. Copyright (C) <FIRST YEAR>, <LAST YEAR> <COPYRIGHT HOLDER>

6.3.6. Generating a Tools Patch

The RTEMS patches to the development tools are generated using a command like this

where the options are:

  • -N and -P take care of adding and removing files (be careful not to

include junk files like file.mybackup)

  • -r tells diff to recurse through subdirectories

  • -c is a context diff (easy to read for humans)

  • -u is a unified diff (easy for patch to apply)

Please look at the generated PATCHFILE and make sure it does not contain anything you did not intend to send to the maintainers. It is easy to accidentally leave a backup file in the modified source tree or have a spurious change that should not be in the PATCHFILE.

If you end up with the entire contents of a file in the patch and can’t figure out why, you may have different CR/LF scheme in the two source files. The GNU open-source packages usually have UNIX style CR/LF. If you edit on a Windows platform, the line terminators may have been transformed by the editor into Windows style.

6.3.7. Naming Rules

6.3.7.1. General Rules
  • Avoid abbreviations.

    • Exception: when the abbreviation is more common than the full word.

    • Exception: For well-known acronyms.

  • Use descriptive language.

  • File names should be lower-case alphabet letters only, plus the extension. Avoid symbols in file names.

  • Prefer to use underscores to separate words, rather than CamelCase.or !TitleCase.

  • Local-scope variable names are all lower case with underscores between words.

  • CPP macros are all capital letters with underscores between words.

  • Enumerated (enum) values are all capital letters with underscores between words, but the type name follows the regular rules of other type names.

  • Constant (const) variables follow the same rules as other variables. An exception is that a const that replaces a CPP macro might be all capital letters for backward compatibility.

  • Type names, function names, and global scope names have different rules depending on whether they are part of the public API or are internal to RTEMS, see below.

User-Facing APIs

The public API routines follow a standard API like POSIX or BSD or start with rtems_. If a name starts with rtems_, then it should be assumed to be available for use by the application and be documented in the User’s Guide.

If the method is intended to be private, then make it static to a file or start the name with a leading _.

Classic API

  • Public facing APIs start with rtems_ followed by a word or phrase to indicate the Manager or functional category the method or data type belongs to.

  • Non-public APIs should be static or begin with a leading _. The required form is the use of a leading underscore, functional area with leading capital letter, an underscore, and the method with a leading capital letter.

POSIX API

  • Follow the rules of POSIX.

RTEMS Internal Interfaces

Super Core

The Super Core. is organized in an Object-Oriented fashion. Each score Handler is a Package, or Module, and each Module contains type definitions, functions, etc. The following summarizes our conventions for using names within SuperCore. Modules.

  • Use “Module_name_Particular_type_name” for type names.

  • Use “_Module_name_Particular_function_name” for functions names.

  • Use “_Module_name_Global_or_file_scope_variable_name” for global or file scope variable names.

Within a structure:

  • Use “Name” for struct aggregate members.

  • Use “name” for reference members.

  • Use “name” for primitive type members.

As shown in the following example:

typedef struct {
    Other_module_Struct_type    Aggregate_member_name;
    Other_module_Struct_type   *reference_member_name;
    Other_module_Primitive_type primitive_member_name;
  } The_module_Type_name;

BSP

  • TODO.

6.4. Documentation Guidelines

6.4.1. Application Configuration Options

Add at least an index entry and a label for the configuration option. Use a pattern of CONFIGURE_[A-Z0-9_]+ for the option name. Use the following template for application configuration feature options:

.. index:: CONFIGURE_FEATURE

.. _CONFIGURE_FEATURE:

CONFIGURE_FEATURE
-----------------

CONSTANT:
    ``CONFIGURE_FEATURE``

OPTION TYPE:
    This configuration option is a boolean feature define.

DEFAULT CONFIGURATION:
    If this configuration option is undefined, then the described feature is not
    enabled.

DESCRIPTION:
    In case this configuration option is defined, then feature happens.

NOTES:
    Keep the description short.  Add all special cases, usage notes, error
    conditions, configuration dependencies, references, etc. here to the notes.

Use the following template for application configuration integer and initializer options:

.. index:: CONFIGURE_VALUE

.. _CONFIGURE_VALUE:

CONFIGURE_VALUE
-----------------

CONSTANT:
    ``CONFIGURE_VALUE``

OPTION TYPE:
    This configuration option is an integer (or initializer) define.

DEFAULT VALUE:
    The default value is X.

VALUE CONSTRAINTS:
    The value of this configuration option shall satisfy all of the following
    constraints:

    * It shall be greater than or equal to A.

    * It shall be less than or equal to B.

    * It shall meet C.

DESCRIPTION:
    The value of this configuration option defines the Y of Z in W.

NOTES:
    Keep the description short.  Add all special cases, usage notes, error
    conditions, configuration dependencies, references, etc. here to the notes.

6.5. Python Development Guidelines

Python is the preferred programming language for the RTEMS Tools. The RTEMS Tools run on the host computer of an RTEMS user or maintainer. These guidelines cover the Python language version, the source code formatting, use of static analysis tools, type annotations, testing, code coverage, and documentation. There are exceptions for existing code and third-party code. It is recommended to read the PEP 8 - Style Guide for Python Code and the Google Python Style Guide.

6.5.1. Python Language Versions

Although the official end-of-life of Python 2.7 was on January 1, 2020, the RTEMS Project still cares about Python 2.7 compatibility for some tools. Every tool provided by the RTEMS Project which an RTEMS user may use to develop applications with RTEMS should be Python 2.7 compatible. Examples are the build system, the RTEMS Source Builder, and the RTEMS Tester. The rationale is that there are still some maintained Linux distributions in the wild which ship only Python 2.7 by default. An example is CentOS 7 which gets maintenance updates until June 2024. Everything an RTEMS maintainer uses should be written in Python 3.6.

6.5.2. Python Code Formatting

Good looking code is important. Unfortunately, what looks good is a bit subjective and varies from developer to developer. Arguing about the code format is not productive. Code reviews should focus on more important topics, for example functionality, testability, and performance. Fortunately, for Python there are some good automatic code formatters available. All new code specifically developed for the RTEMS Tools should be piped through the yapf Python code formatter before it is committed or sent for review. Use the default settings of the tool (PEP 8 coding style).

You can disable the automatic formatting by the tool in a region starting with the #yapf: disable comment until the next # yapf: enable comment, for example

# yapf: disable
FOO = {
    # ... some very large, complex data literal.
}

BAR = [
    # ... another large data literal.
]
# yapf: enable

For a single literal, you can disable the formatting like this:

BAZ = {
    (1, 2, 3, 4),
    (5, 6, 7, 8),
    (9, 10, 11, 12),
}  # yapf: disable

6.5.3. Static Analysis Tools

Use the flake8 and pylint static analysis tools for Python. Do not commit your code or send it for review if the tools find some rule violations. Run the tools with the default configuration. If you have problems to silence the tools, then please ask for help on the Developers Mailing List. Consult the tool documentation to silence false positives.

6.5.4. Type Annotations

For Python 3.6 or later code use type annotations. All public functions of your modules should have PEP 484 type annotations. Check for type issues with the mypy static type checker.

6.5.5. Testing

Write tests for your code with the pytest framework. Use the monkeypatch mocking module. Do not use the standard Python unittest and unittest.mock modules. Use coverage run -m pytest to run the tests with code coverage support. If you modify existing code or contribute new code to a subproject which uses tests and the code coverage metric, then do not make the code coverage worse.

6.5.5.1. Test Organization

Do not use test classes to group tests. Use separate files instead. Avoid deep test directory hierarchies. For example, place tests for mymodule.py in tests/test_mymodule.py. For class-specific tests use:

  • mymodule.py:class First \(\rightarrow\) tests/test_mymodule_first.py

  • mymodule.py:class Second \(\rightarrow\) tests/test_mymodule_second.py

  • mymodule.py:class Third \(\rightarrow\) tests/test_mymodule_third.py

You can also group tests in other ways, for example:

  • mymodule.py \(\rightarrow\) tests/test_mymodule_input.py

  • mymodule.py \(\rightarrow\) tests/test_mymodule_output.py

6.5.6. Documentation

Document your code using the PEP 257 - Docstring Conventions. Contrary to PEP 257, use the descriptive-style ("""Fetches rows from a Bigtable.""") instead of imperative-style ("""Fetch rows from a Bigtable.""") as recommended by Comments and Docstrings - Functions and Methods. Use the Sphinx format. The sphinx-autodoc-typehints helps to reuse the type annotations for the documentation. Test code does not need docstrings in general.

6.5.7. Existing Code

Existing code in the RTEMS Tools may not follow the preceding guidelines. The RTEMS Project welcomes contributions which bring existing code in line with the guidelines. Firstly, run the yapf code formatter through the existing code of interest. Add # yapf: disable comments to avoid reformatting in some areas if it makes sense. If the existing code has no unit tests, then add unit tests before you modify existing code by hand. With the new unit tests aim at a good code coverage especially in the areas you intend to modify. While you review the code add docstrings. Run the static analysers and fix the rule violations. Please keep in mind that also trivial modifications can break working code. Make sure you have some unit tests. Add type annotations unless the code should be Python 2.7 compatible. Concentrate on the public interfaces.

6.5.8. Third-Party Code

Try to not modify imported third-party code. In case there are issues with third-party code, then at least write a bug report or otherwise contact the upstream project. Reimport the third-party code after the issue is fixed in the upstream project. Only temporarily modify imported third-party code until a solution integrated in the upstream is available.

6.6. Change Management

Major decisions about RTEMS are made by the core developers in concert with the user community, guided by the Mission Statement. We provide access to our development sources via a Git Repository (see these Instructions for details).

TBD - ??? what in the Wiki could go here

6.7. Issue Tracking

The RTEMS Project uses Trac to manage all change requests and problem reports and refers to either as a ticket.

The bug reporting procedure is documented in the RTEMS User Manual.

TBD Review process, workflows, etc.

7. Software Test Plan Assurance and Procedures

7.1. Testing and Coverage

Testing to verify that requirements are implemented is a critical part of the high integrity processes. Similarly, measuring and reporting source and decision path coverage of source code is critical.

Needed improvements to the RTEMS testing infrastructure should be done as part of the open project. Similarly, improvements in RTEMS coverage reporting should be done as part of the open project. Both of these capabilities are part of the RTEMS Tester toolset.

Assuming that a requirements focused test suite is added to the open RTEMS, tools will be needed to assist in verifying that requirements are “fully tested.” A fully tested requirement is one which is implemented and tested with associated logical tracing. Tools automating this analysis and generating reporting and alerts will be a critical part of ensuring the master technical data does not bit rot.

Must use tools from:

TBD - Change URL to git.rtems.org and list support tools RTEMS Tools Project: https://github.com/RTEMS/rtems-tools

Scope, Procedures, Methodologies, Tools TBD - Write content

7.1.1. Test Suites

All RTEMS source distributions include the complete RTEMS test suites. These tests must be compiled and linked for a specific BSP. Some BSPs are for freely available simulators and thus anyone may test RTEMS on a simulator. Most of the BSPs which can execute on a simulator include scripts to help automate running them.

The RTEMS Project welcomes additions to the various test suites and sample application collections. This helps improve coverage of functionality as well as ensure user use cases are regularly tested.

The following functional test suites are included with RTEMS.

  • Classic API Single Processor Test Suite

  • POSIX API Test Suite

  • File System Test Suite

  • Support Library Test Suite (libtests)

  • Symmetric Multiprocessing Test Suite

  • Distributed Multiprocessing Test Suite

  • Classic API Ada95 Binding Test Suite

The following timing test suites are included with RTEMS.

  • Classic API Timing Test Suite

  • POSIX API Timing Test Suite

  • Rhealstone Collection

  • Benchmarks Collecction

The RTEMS source distribution includes two collections of sample applications.

  • Sample Applications (built as RTEMS tests)

  • Example Applications (built as RTEMS user applications)

The RTEMS libbsd package includes its own test suite.

7.1.1.1. Legacy Test Suites

The following are available for the legacy IPV4 Network Stack:

  • Network Demonstration Applications

Post RTEMS 4.10, ITRON API support was removed. The following test suites are only available if the ITRON API support is present in RTEMS.

  • ITRON API Test Suite

  • ITRON API Timing Test Suite

7.1.2. RTEMS Tester

TBD - Convert the following to Rest and insert into this file TBD https://devel.rtems.org/wiki/Testing/Tester TBD - Above file is horribly out of date. Find new docs and refer to them

8. Software Test Framework

8.1. The RTEMS Test Framework

The RTEMS Test Framework helps you to write test suites. It has the following features:

  • Implemented in standard C11

  • Runs on at least FreeBSD, MSYS2, Linux and RTEMS

  • Test runner and test case code can be in separate translation units

  • Test cases are automatically registered at link-time

  • Test cases may have a test fixture

  • Test checks for various standard types

  • Supports test case planning

  • Test case scoped dynamic memory

  • Test case destructors

  • Test case resource accounting to show that no resources are leaked during the test case execution

  • Supports early test case exit, e.g. in case a malloc() fails

  • Individual test case and overall test suite duration is reported

  • Procedures for code runtime measurements in RTEMS

  • Easy to parse test report to generate for example human readable test reports

  • Low overhead time measurement of short time sequences (using cycle counter hardware if a available)

  • Configurable time service provider for a monotonic clock

  • Low global memory overhead for test cases and test checks

  • Supports multi-threaded execution and interrupts in test cases

  • A simple (polled) put character function is sufficient to produce the test report

  • Only text, global data and a stack pointer must be set up to run a test suite

  • No dynamic memory is used by the framework itself

  • No memory is aggregated throughout the test case execution

8.1.1. Nomenclature

A test suite is a collection of test cases. A test case consists of individual test actions and checks. A test check determines if the outcome of a test action meets its expectation. A test action is a program sequence with an observable outcome, for example a function invocation with a return status. If the test action outcome is all right, then the test check passes, otherwise the test check fails. The test check failures of a test case are summed up. A test case passes, if the failure count of this test case is zero, otherwise the test case fails. The test suite passes if all test cases pass, otherwise it fails.

8.1.2. Test Cases

You can write a test case with the T_TEST_CASE() macro followed by a function body:

T_TEST_CASE(name)
{
   /* Your test case code */
}

The test case name must be a valid C designator. The test case names must be unique within the test suite. Just link modules with test cases to the test runner to form a test suite. The test cases are automatically registered via static constructors.

Test Case Example
#include <t.h>

static int add(int a, int b)
{
    return a + b;
}

T_TEST_CASE(a_test_case)
{
    int actual_value;

    actual_value = add(1, 1);
    T_eq_int(actual_value, 2);
    T_true(false, "a test failure message");
}
Test Case Report
B:a_test_case
P:0:8:UI1:test-simple.c:13
F:1:8:UI1:test-simple.c:14:a test failure message
E:a_test_case:N:2:F:1:D:0.001657

The B line indicates the begin of test case a_test_case. The P line shows that the test check in file test-simple.c at line 13 executed by task UI1 on processor 0 as the test step 0 passed. The invocation of add() in line 12 is the test action of test step 0. The F lines shows that the test check in file test-simple.c at line 14 executed by task UI1 on processor 0 as the test step 1 failed with a message of “a test failure message”. The E line indicates the end of test case a_test_case resulting in a total of two test steps (N) and one test failure (F). The test case execution duration (D) was 0.001657 seconds. For test report details see: Test Reporting.

8.1.3. Test Fixture

You can write a test case with a test fixture with the T_TEST_CASE_FIXTURE() macro followed by a function body:

T_TEST_CASE_FIXTURE(name, fixture)
{
   /* Your test case code */
}

The test case name must be a valid C designator. The test case names must be unique within the test suite. The fixture must point to a statically initialized read-only object of type T_fixture. The test fixture provides methods to setup, stop and tear down a test case. A context is passed to the methods. The initial context is defined by the read-only fixture object. The context can be obtained by the T_fixture_context() function. It can be set within the scope of one test case by the T_set_fixture_context() function. This can be used for example to dynamically allocate a test environment in the setup method.

Test Fixture Example
#include <t.h>

static int initial_value = 3;

static int counter;

static void
setup(void *ctx)
{
    int *c;

    T_log(T_QUIET, "setup begin");
    T_eq_ptr(ctx, &initial_value);
    T_eq_ptr(ctx, T_fixture_context());
    c = ctx;
    counter = *c;
    T_set_fixture_context(&counter);
    T_eq_ptr(&counter, T_fixture_context());
    T_log(T_QUIET, "setup end");
}

static void
stop(void *ctx)
{
    int *c;

    T_log(T_QUIET, "stop begin");
    T_eq_ptr(ctx, &counter);
    c = ctx;
    ++(*c);
    T_log(T_QUIET, "stop end");
}

static void
teardown(void *ctx)
{
    int *c;

    T_log(T_QUIET, "teardown begin");
    T_eq_ptr(ctx, &counter);
    c = ctx;
    T_eq_int(*c, 4);
    T_log(T_QUIET, "teardown end");
}

static const T_fixture fixture = {
    .setup = setup,
    .stop = stop,
    .teardown = teardown,
    .initial_context = &initial_value
};

T_TEST_CASE_FIXTURE(fixture, &fixture)
{
    T_assert_true(true, "all right");
    T_assert_true(false, "test fails and we stop the test case");
    T_log(T_QUIET, "not reached");
}
Test Fixture Report
B:fixture
L:setup begin
P:0:0:UI1:test-fixture.c:13
P:1:0:UI1:test-fixture.c:14
P:2:0:UI1:test-fixture.c:18
L:setup end
P:3:0:UI1:test-fixture.c:55
F:4:0:UI1:test-fixture.c:56:test fails and we stop the test case
L:stop begin
P:5:0:UI1:test-fixture.c:28
L:stop end
L:teardown begin
P:6:0:UI1:test-fixture.c:40
P:7:0:UI1:test-fixture.c:42
L:teardown end
E:fixture:N:8:F:1

8.1.4. Test Case Planning

Each non-quiet test check fetches and increments the test step counter atomically. For each test case execution the planned steps can be specified with the T_plan() function.

void T_plan(unsigned int planned_steps);

This function must be invoked at most once in each test case execution. If the planned test steps are set with this function, then the final test steps after the test case execution must be equal to the planned steps, otherwise the test case fails.

Use the T_step_*(step, …) test check variants to ensure that the test case execution follows exactly the planned steps.

Test Planning Example
#include <t.h>

T_TEST_CASE(wrong_step)
{
    T_plan(2);
    T_step_true(0, true, "all right");
    T_step_true(2, true, "wrong step");
}

T_TEST_CASE(plan_ok)
{
    T_plan(1);
    T_step_true(0, true, "all right");
}

T_TEST_CASE(plan_failed)
{
    T_plan(2);
    T_step_true(0, true, "not enough steps");
    T_quiet_true(true, "quiet test do not count");
}

T_TEST_CASE(double_plan)
{
    T_plan(99);
    T_plan(2);
}

T_TEST_CASE(steps)
{
    T_step(0, "a");
    T_plan(3);
    T_step(1, "b");
    T_step(2, "c");
}
Test Planning Report
B:wrong_step
P:0:0:UI1:test-plan.c:6
F:1:0:UI1:test-plan.c:7:planned step (2)
E:wrong_step:N:2:F:1
B:plan_ok
P:0:0:UI1:test-plan.c:13
E:plan_ok:N:1:F:0
B:plan_failed
P:0:0:UI1:test-plan.c:19
F:*:0:UI1:*:*:actual steps (1), planned steps (2)
E:plan_failed:N:1:F:1
B:double_plan
F:*:0:UI1:*:*:planned steps (99) already set
E:double_plan:N:0:F:1
B:steps
P:0:0:UI1:test-plan.c:31
P:1:0:UI1:test-plan.c:33
P:2:0:UI1:test-plan.c:34
E:steps:N:3:F:0

8.1.5. Test Case Resource Accounting

The framework can check if various resources are leaked during a test case execution. The resource checkers are specified by the test run configuration. On RTEMS, checks for the following resources are available

  • workspace and heap memory,

  • file descriptors,

  • POSIX keys and key value pairs,

  • RTEMS barriers,

  • RTEMS user extensions,

  • RTEMS message queues,

  • RTEMS partitions,

  • RTEMS periods,

  • RTEMS regions,

  • RTEMS semaphores,

  • RTEMS tasks, and

  • RTEMS timers.

Resource Accounting Example
#include <t.h>

#include <stdlib.h>

#include <rtems.h>

T_TEST_CASE(missing_sema_delete)
{
    rtems_status_code sc;
    rtems_id id;

    sc = rtems_semaphore_create(rtems_build_name('S', 'E', 'M', 'A'), 0,
        RTEMS_COUNTING_SEMAPHORE, 0, &id);
    T_rsc_success(sc);
}

T_TEST_CASE(missing_free)
{
    void *p;

    p = malloc(1);
    T_not_null(p);
}
Resource Accounting Report
B:missing_sema_delete
P:0:0:UI1:test-leak.c:14
F:*:0:UI1:*:*:RTEMS semaphore leak (1)
E:missing_sema_delete:N:1:F:1:D:0.004013
B:missing_free
P:0:0:UI1:test-leak.c:22
F:*:0:UI1:*:*:memory leak in workspace or heap
E:missing_free:N:1:F:1:D:0.003944

8.1.6. Test Case Scoped Dynamic Memory

You can allocate dynamic memory which is automatically freed after the current test case execution. You can provide an optional destroy function to T_zalloc() which is called right before the memory is freed. The T_zalloc() function initializes the memory to zero.

void *T_malloc(size_t size);

void *T_calloc(size_t nelem, size_t elsize);

void *T_zalloc(size_t size, void (*destroy)(void *));

void T_free(void *ptr);
Test Case Scoped Dynamic Memory Example
#include <t.h>

T_TEST_CASE(malloc_free)
{
    void *p;

    p = T_malloc(1);
    T_assert_not_null(p);
    T_free(p);
}

T_TEST_CASE(malloc_auto)
{
    void *p;

    p = T_malloc(1);
    T_assert_not_null(p);
}

static void
destroy(void *p)
{
    int *i;

    i = p;
    T_step_eq_int(2, *i, 1);
}

T_TEST_CASE(zalloc_auto)
{
    int *i;

    T_plan(3);
    i = T_zalloc(sizeof(*i), destroy);
    T_step_assert_not_null(0, i);
    T_step_eq_int(1, *i, 0);
    *i = 1;
}
Test Case Scoped Dynamic Memory Report
B:malloc_free
P:0:0:UI1:test-malloc.c:8
E:malloc_free:N:1:F:0:D:0.005200
B:malloc_auto
P:0:0:UI1:test-malloc.c:17
E:malloc_auto:N:1:F:0:D:0.004790
B:zalloc_auto
P:0:0:UI1:test-malloc.c:35
P:1:0:UI1:test-malloc.c:36
P:2:0:UI1:test-malloc.c:26
E:zalloc_auto:N:3:F:0:D:0.006583

8.1.7. Test Case Destructors

You can add test case destructors with T_add_destructor(). They are called automatically at the test case end before the resource accounting takes place. Optionally, a registered destructor can be removed before the test case end with T_remove_destructor(). The T_destructor structure of a destructor must exist after the return from the test case body. Do not use stack memory or dynamic memory obtained via T_malloc(), T_calloc() or T_zalloc() for the T_destructor structure.

void T_add_destructor(T_destructor *destructor,
   void (*destroy)(T_destructor *));

void T_remove_destructor(T_destructor *destructor);
Test Case Destructor Example
#include <t.h>

static void
destroy(T_destructor *dtor)
{
    (void)dtor;
    T_step(0, "destroy");
}

T_TEST_CASE(destructor)
{
    static T_destructor dtor;

    T_plan(1);
    T_add_destructor(&dtor, destroy);
}
Test Case Destructor Report
B:destructor
P:0:0:UI1:test-destructor.c:7
E:destructor:N:1:F:0:D:0.003714

8.1.8. Test Checks

A test check determines if the actual value presented to the test check meets its expectation. The actual value should represent the outcome of a test action. If the actual value is all right, then the test check passes, otherwise the test check fails. A failed test check does not stop the test case execution immediately unless the T_assert_*() test variant is used. Each test check increments the test step counter unless the T_quiet_*() test variant is used. The test step counter is initialized to zero before the test case begins to execute. The T_step_*(step, …) test check variants verify that the test step counter is equal to the planned test step value, otherwise the test check fails.

8.1.8.1. Test Check Parameter Conventions

The following names for test check parameters are used throughout the test checks:

step

The planned test step for this test check.

a

The actual value to check against an expected value. It is usually the first parameter in all test checks, except in the T_step_*(step, …) test check variants, here it is the second parameter.

e

The expected value of a test check. This parameter is optional. Some test checks have an implicit expected value. If present, then this parameter is directly after the actual value parameter of the test check.

fmt

A printf()-like format string. Floating-point and exotic formats may be not supported.

8.1.8.2. Test Check Condition Conventions

The following names for test check conditions are used:

eq

The actual value must equal the expected value.

ne

The actual value must not equal the value of the second parameter.

ge

The actual value must be greater than or equal to the expected value.

gt

The actual value must be greater than the expected value.

le

The actual value must be less than or equal to the expected value.

lt

The actual value must be less than the expected value.

If the actual value satisfies the test check condition, then the test check passes, otherwise it fails.

8.1.8.3. Test Check Variant Conventions

The T_quiet_*() test check variants do not increment the test step counter and only print a message if the test check fails. This is helpful in case a test check appears in a tight loop.

The T_step_*(step, …) test check variants check in addition that the test step counter is equal to the specified test step value, otherwise the test check fails.

The T_assert_*() and T_step_assert_*(step, …) test check variants stop the current test case execution if the test check fails.

The following names for test check type variants are used:

ptr

The test value must be a pointer (void *).

mem

The test value must be a memory area with a specified length.

str

The test value must be a null byte terminated string.

nstr

The length of the test value string is limited to a specified maximum.

char

The test value must be a character (char).

schar

The test value must be a signed character (signed char).

uchar

The test value must be an unsigned character (unsigned char).

short

The test value must be a short integer (short).

ushort

The test value must be an unsigned short integer (unsigned short).

int

The test value must be an integer (int).

uint

The test value must be an unsigned integer (unsigned int).

long

The test value must be a long integer (long).

ulong

The test value must be an unsigned long integer (unsigned long).

ll

The test value must be a long long integer (long long).

ull

The test value must be an unsigned long long integer (unsigned long long).

i8

The test value must be a signed 8-bit integer (int8_t).

u8

The test value must be an unsigned 8-bit integer (uint8_t).

i16

The test value must be a signed 16-bit integer (int16_t).

u16

The test value must be an unsigned 16-bit integer (uint16_t).

i32

The test value must be a signed 32-bit integer (int32_t).

u32

The test value must be an unsigned 32-bit integer (uint32_t).

i64

The test value must be a signed 64-bit integer (int64_t).

u64

The test value must be an unsigned 64-bit integer (uint64_t).

iptr

The test value must be of type intptr_t.

uptr

The test value must be of type uintptr_t.

ssz

The test value must be of type ssize_t.

sz

The test value must be of type size_t.

8.1.8.4. Boolean Expressions

The following test checks for boolean expressions are available:

void T_true(bool a, const char *fmt, ...);
void T_assert_true(bool a, const char *fmt, ...);
void T_quiet_true(bool a, const char *fmt, ...);
void T_step_true(unsigned int step, bool a, const char *fmt, ...);
void T_step_assert_true(unsigned int step, bool a, const char *fmt, ...);

void T_false(bool a, const char *fmt, ...);
void T_assert_false(bool a, const char *fmt, ...);
void T_quiet_true(bool a, const char *fmt, ...);
void T_step_true(unsigned int step, bool a, const char *fmt, ...);
void T_step_assert_true(unsigned int step, bool a, const char *fmt, ...);

The message is only printed in case the test check fails. The format parameter is mandatory.

Boolean Test Checks Example
#include <t.h>

T_TEST_CASE(example)
{
    T_true(true, "test passes, no message output");
    T_true(false, "test fails");
    T_quiet_true(true, "quiet test passes, no output at all");
    T_quiet_true(false, "quiet test fails");
    T_step_true(2, true, "step test passes, no message output");
    T_step_true(3, false, "step test fails");
    T_assert_false(true, "this is a format %s", "string");
}
Boolean Test Checks Report
B:example
P:0:0:UI1:test-example.c:5
F:1:0:UI1:test-example.c:6:test fails
F:*:0:UI1:test-example.c:8:quiet test fails
P:2:0:UI1:test-example.c:9
F:3:0:UI1:test-example.c:10:step test fails
F:4:0:UI1:test-example.c:11:this is a format string
E:example:N:5:F:4
8.1.8.5. Generic Types

The following test checks for data types with an equality (==) or inequality (!=) operator are available:

void T_eq(T a, T e, const char *fmt, ...);
void T_assert_eq(T a, T e, const char *fmt, ...);
void T_quiet_eq(T a, T e, const char *fmt, ...);
void T_step_eq(unsigned int step, T a, T e, const char *fmt, ...);
void T_step_assert_eq(unsigned int step, T a, T e, const char *fmt, ...);

void T_ne(T a, T e, const char *fmt, ...);
void T_assert_ne(T a, T e, const char *fmt, ...);
void T_quiet_ne(T a, T e, const char *fmt, ...);
void T_step_ne(unsigned int step, T a, T e, const char *fmt, ...);
void T_step_assert_ne(unsigned int step, T a, T e, const char *fmt, ...);

The type name T specifies an arbitrary type which must support the corresponding operator. The message is only printed in case the test check fails. The format parameter is mandatory.

8.1.8.6. Pointers

The following test checks for pointers are available:

void T_eq_ptr(const void *a, const void *e);
void T_assert_eq_ptr(const void *a, const void *e);
void T_quiet_eq_ptr(const void *a, const void *e);
void T_step_eq_ptr(unsigned int step, const void *a, const void *e);
void T_step_assert_eq_ptr(unsigned int step, const void *a, const void *e);

void T_ne_ptr(const void *a, const void *e);
void T_assert_ne_ptr(const void *a, const void *e);
void T_quiet_ne_ptr(const void *a, const void *e);
void T_step_ne_ptr(unsigned int step, const void *a, const void *e);
void T_step_assert_ne_ptr(unsigned int step, const void *a, const void *e);

void T_null(const void *a);
void T_assert_null(const void *a);
void T_quiet_null(const void *a);
void T_step_null(unsigned int step, const void *a);
void T_step_assert_null(unsigned int step, const void *a);

void T_not_null(const void *a);
void T_assert_not_null(const void *a);
void T_quiet_not_null(const void *a);
void T_step_not_null(unsigned int step, const void *a);
void T_step_assert_not_null(unsigned int step, const void *a);

An automatically generated message is printed in case the test check fails.

8.1.8.7. Memory Areas

The following test checks for memory areas are available:

void T_eq_mem(const void *a, const void *e, size_t n);
void T_assert_eq_mem(const void *a, const void *e, size_t n);
void T_quiet_eq_mem(const void *a, const void *e, size_t n);
void T_step_eq_mem(unsigned int step, const void *a, const void *e, size_t n);
void T_step_assert_eq_mem(unsigned int step, const void *a, const void *e, size_t n);

void T_ne_mem(const void *a, const void *e, size_t n);
void T_assert_ne_mem(const void *a, const void *e, size_t n);
void T_quiet_ne_mem(const void *a, const void *e, size_t n);
void T_step_ne_mem(unsigned int step, const void *a, const void *e, size_t n);
void T_step_assert_ne_mem(unsigned int step, const void *a, const void *e, size_t n);

The memcmp() function is used to compare the memory areas. An automatically generated message is printed in case the test check fails.

8.1.8.8. Strings

The following test checks for strings are available:

void T_eq_str(const char *a, const char *e);
void T_assert_eq_str(const char *a, const char *e);
void T_quiet_eq_str(const char *a, const char *e);
void T_step_eq_str(unsigned int step, const char *a, const char *e);
void T_step_assert_eq_str(unsigned int step, const char *a, const char *e);

void T_ne_str(const char *a, const char *e);
void T_assert_ne_str(const char *a, const char *e);
void T_quiet_ne_str(const char *a, const char *e);
void T_step_ne_str(unsigned int step, const char *a, const char *e);
void T_step_assert_ne_str(unsigned int step, const char *a, const char *e);

void T_eq_nstr(const char *a, const char *e, size_t n);
void T_assert_eq_nstr(const char *a, const char *e, size_t n);
void T_quiet_eq_nstr(const char *a, const char *e, size_t n);
void T_step_eq_nstr(unsigned int step, const char *a, const char *e, size_t n);
void T_step_assert_eq_nstr(unsigned int step, const char *a, const char *e, size_t n);

void T_ne_nstr(const char *a, const char *e, size_t n);
void T_assert_ne_nstr(const char *a, const char *e, size_t n);
void T_quiet_ne_nstr(const char *a, const char *e, size_t n);
void T_step_ne_nstr(unsigned int step, const char *a, const char *e, size_t n);
void T_step_assert_ne_nstr(unsigned int step, const char *a, const char *e, size_t n);

The strcmp() and strncmp() functions are used to compare the strings. An automatically generated message is printed in case the test check fails.

8.1.8.9. Characters

The following test checks for characters (char) are available:

void T_eq_char(char a, char e);
void T_assert_eq_char(char a, char e);
void T_quiet_eq_char(char a, char e);
void T_step_eq_char(unsigned int step, char a, char e);
void T_step_assert_eq_char(unsigned int step, char a, char e);

void T_ne_char(char a, char e);
void T_assert_ne_char(char a, char e);
void T_quiet_ne_char(char a, char e);
void T_step_ne_char(unsigned int step, char a, char e);
void T_step_assert_ne_char(unsigned int step, char a, char e);

An automatically generated message is printed in case the test check fails.

8.1.8.10. Integers

The following test checks for integers are available:

void T_eq_xyz(I a, I e);
void T_assert_eq_xyz(I a, I e);
void T_quiet_eq_xyz(I a, I e);
void T_step_eq_xyz(unsigned int step, I a, I e);
void T_step_assert_eq_xyz(unsigned int step, I a, I e);

void T_ne_xyz(I a, I e);
void T_assert_ne_xyz(I a, I e);
void T_quiet_ne_xyz(I a, I e);
void T_step_ne_xyz(unsigned int step, I a, I e);
void T_step_assert_ne_xyz(unsigned int step, I a, I e);

void T_ge_xyz(I a, I e);
void T_assert_ge_xyz(I a, I e);
void T_quiet_ge_xyz(I a, I e);
void T_step_ge_xyz(unsigned int step, I a, I e);
void T_step_assert_ge_xyz(unsigned int step, I a, I e);

void T_gt_xyz(I a, I e);
void T_assert_gt_xyz(I a, I e);
void T_quiet_gt_xyz(I a, I e);
void T_step_gt_xyz(unsigned int step, I a, I e);
void T_step_assert_gt_xyz(unsigned int step, I a, I e);

void T_le_xyz(I a, I e);
void T_assert_le_xyz(I a, I e);
void T_quiet_le_xyz(I a, I e);
void T_step_le_xyz(unsigned int step, I a, I e);
void T_step_assert_le_xyz(unsigned int step, I a, I e);

void T_lt_xyz(I a, I e);
void T_assert_lt_xyz(I a, I e);
void T_quiet_lt_xyz(I a, I e);
void T_step_lt_xyz(unsigned int step, I a, I e);
void T_step_assert_lt_xyz(unsigned int step, I a, I e);

The type variant xyz must be schar, uchar, short, ushort, int, uint, long, ulong, ll, ull, i8, u8, i16, u16, i32, u32, i64, u64, iptr, uptr, ssz, or sz.

The type name I must be compatible to the type variant.

An automatically generated message is printed in case the test check fails.

8.1.8.11. RTEMS Status Codes

The following test checks for RTEMS status codes are available:

void T_rsc(rtems_status_code a, rtems_status_code e);
void T_assert_rsc(rtems_status_code a, rtems_status_code e);
void T_quiet_rsc(rtems_status_code a, rtems_status_code e);
void T_step_rsc(unsigned int step, rtems_status_code a, rtems_status_code e);
void T_step_assert_rsc(unsigned int step, rtems_status_code a, rtems_status_code e);

void T_rsc_success(rtems_status_code a);
void T_assert_rsc_success(rtems_status_code a);
void T_quiet_rsc_success(rtems_status_code a);
void T_step_rsc_success(unsigned int step, rtems_status_code a);
void T_step_assert_rsc_success(unsigned int step, rtems_status_code a);

An automatically generated message is printed in case the test check fails.

8.1.8.12. POSIX Error Numbers

The following test checks for POSIX error numbers are available:

void T_eno(int a, int e);
void T_assert_eno(int a, int e);
void T_quiet_eno(int a, int e);
void T_step_eno(unsigned int step, int a, int e);
void T_step_assert_eno(unsigned int step, int a, int e);

void T_eno_success(int a);
void T_assert_eno_success(int a);
void T_quiet_eno_success(int a);
void T_step_eno_success(unsigned int step, int a);
void T_step_assert_eno_success(unsigned int step, int a);

The actual and expected value must be a POSIX error number, e.g. EINVAL, ENOMEM, etc. An automatically generated message is printed in case the test check fails.

8.1.8.13. POSIX Status Codes

The following test checks for POSIX status codes are available:

void T_psx_error(int a, int eno);
void T_assert_psx_error(int a, int eno);
void T_quiet_psx_error(int a, int eno);
void T_step_psx_error(unsigned int step, int a, int eno);
void T_step_assert_psx_error(unsigned int step, int a, int eno);

void T_psx_success(int a);
void T_assert_psx_success(int a);
void T_quiet_psx_success(int a);
void T_step_psx_success(unsigned int step, int a);
void T_step_assert_psx_success(unsigned int step, int a);

The eno value must be a POSIX error number, e.g. EINVAL, ENOMEM, etc. An actual value of zero indicates success. An actual value of minus one indicates an error. An automatically generated message is printed in case the test check fails.

POSIX Status Code Example
#include <t.h>

#include <sys/stat.h>
#include <errno.h>

T_TEST_CASE(stat)
{
    struct stat st;
    int status;

    errno = 0;
    status = stat("foobar", &st);
    T_psx_error(status, ENOENT);
}
POSIX Status Code Report
B:stat
P:0:0:UI1:test-psx.c:13
E:stat:N:1:F:0

8.1.9. Log Messages and Formatted Output

You can print log messages with the T_log() function:

void T_log(T_verbosity verbosity, char const *fmt, ...);

A newline is automatically added to terminate the log message line.

Log Message Example
#include <t.h>

T_TEST_CASE(log)
{
    T_log(T_NORMAL, "a log message %i, %i, %i", 1, 2, 3);
    T_set_verbosity(T_QUIET);
    T_log(T_NORMAL, "not verbose enough");
}
Log Message Report
B:log
L:a log message 1, 2, 3
E:log:N:0:F:0

You can use the following functions to print formatted output:

int T_printf(char const *, ...);

int T_vprintf(char const *, va_list);

int T_snprintf(char *, size_t, const char *, ...);

In contrast to the corresponding standard C library functions, floating-point and exotic formats may not be supported. On some architectures supported by RTEMS, floating-point operations are only supported in special tasks and may be forbidden in interrupt context. The formatted output functions provided by the test framework work in every context.

8.1.10. Time Services

The test framework provides two unsigned integer types for time values. The T_ticks unsigned integer type is used by the T_tick() function which measures time using the highest frequency counter available on the platform. It should only be used to measure small time intervals. The T_time unsigned integer type is used by the T_now() function which returns the current monotonic clock value of the platform, e.g. CLOCK_MONOTONIC.

T_ticks T_tick(void);

T_time T_now(void);

The reference time point for these two clocks is unspecified. You can obtain the test case begin time with the T_case_begin_time() function.

T_time T_case_begin_time(void);

You can convert time into ticks with the T_time_to_ticks() function and vice versa with the T_ticks_to_time() function.

T_time T_ticks_to_time(T_ticks ticks);

T_ticks T_time_to_ticks(T_time time);

You can convert seconds and nanoseconds values into a combined time value with the T_seconds_and_nanoseconds_to_time() function. You can convert a time value into separate seconds and nanoseconds values with the T_time_to_seconds_and_nanoseconds() function.

T_time T_seconds_and_nanoseconds_to_time(uint32_t s, uint32_t ns);

void T_time_to_seconds_and_nanoseconds(T_time time, uint32_t *s, uint32_t *ns);

You can convert a time value into a string represention. The time unit of the string representation is seconds. The precision of the string represention may be nanoseconds, microseconds, milliseconds, or seconds. You have to provide a buffer for the string (T_time_string).

const char *T_time_to_string_ns(T_time time, T_time_string buffer);

const char *T_time_to_string_us(T_time time, T_time_string buffer);

const char *T_time_to_string_ms(T_time time, T_time_string buffer);

const char *T_time_to_string_s(T_time time, T_time_string buffer);
Time String Example
#include <t.h>

T_TEST_CASE(time_to_string)
{
    T_time_string ts;
    T_time t;
    uint32_t s;
    uint32_t ns;

    t = T_seconds_and_nanoseconds_to_time(0, 123456789);
    T_eq_str(T_time_to_string_ns(t, ts), "0.123456789");
    T_eq_str(T_time_to_string_us(t, ts), "0.123456");
    T_eq_str(T_time_to_string_ms(t, ts), "0.123");
    T_eq_str(T_time_to_string_s(t, ts), "0");

    T_time_to_seconds_and_nanoseconds(t, &s, &ns);
    T_eq_u32(s, 0);
    T_eq_u32(ns, 123456789);
}
Time String Report
B:time_to_string
P:0:0:UI1:test-time.c:11
P:1:0:UI1:test-time.c:12
P:2:0:UI1:test-time.c:13
P:3:0:UI1:test-time.c:14
P:4:0:UI1:test-time.c:17
P:5:0:UI1:test-time.c:18
E:time_to_string:N:6:F:0:D:0.005250

You can convert a tick value into a string represention. The time unit of the string representation is seconds. The precision of the string represention may be nanoseconds, microseconds, milliseconds, or seconds. You have to provide a buffer for the string (T_time_string).

const char *T_ticks_to_string_ns(T_ticks ticks, T_time_string buffer);

const char *T_ticks_to_string_us(T_ticks ticks, T_time_string buffer);

const char *T_ticks_to_string_ms(T_ticks ticks, T_time_string buffer);

const char *T_ticks_to_string_s(T_ticks ticks, T_time_string buffer);

8.1.11. Code Runtime Measurements

You can measure the runtime of code fragments in several execution environment variants with the T_measure_runtime() function. This function needs a context which must be created with the T_measure_runtime_create() function. The context is automatically destroyed after the test case execution.

typedef struct {
    size_t sample_count;
} T_measure_runtime_config;

typedef struct {
    const char *name;
    int flags;
    void (*setup)(void *arg);
    void (*body)(void *arg);
    bool (*teardown)(void *arg, T_ticks *delta, uint32_t tic, uint32_t toc,
        unsigned int retry);
    void *arg;
} T_measure_runtime_request;

T_measure_runtime_context *T_measure_runtime_create(
    const T_measure_runtime_config *config);

void T_measure_runtime(T_measure_runtime_context *ctx,
    const T_measure_runtime_request *request);

The runtime measurement is performed for the body request handler of the measurement request (T_measure_runtime_request). The optional setup request handler is called before each invocation of the body request handler. The optional teardown request handler is called after each invocation of the body request handler. It has several parameters and a return status. If it returns true, then this measurement sample value is recorded, otherwise the measurement is retried. The delta parameter is the current measurement sample value. It can be altered by the teardown request handler. The tic and toc parameters are the system tick values before and after the request body invocation. The retry parameter is the current retry counter. The runtime of the operational setup and teardown request handlers is not measured.

You can control some aspects of the measurement through the request flags (use zero for the default):

T_MEASURE_RUNTIME_ALLOW_CLOCK_ISR

Allow clock interrupts during the measurement. By default, measurements during which a clock interrupt happened are discarded unless it happens two times in a row.

T_MEASURE_RUNTIME_REPORT_SAMPLES

Report all measurement samples.

T_MEASURE_RUNTIME_DISABLE_VALID_CACHE

Disable the ValidCache execution environment variant.

T_MEASURE_RUNTIME_DISABLE_HOT_CACHE

Disable the HotCache execution environment variant.

T_MEASURE_RUNTIME_DISABLE_DIRTY_CACHE

Disable the DirtyCache execution environment variant.

T_MEASURE_RUNTIME_DISABLE_MINOR_LOAD

Disable the Load execution environment variants with a load worker count less than the processor count.

T_MEASURE_RUNTIME_DISABLE_MAX_LOAD

Disable the Load execution environment variant with a load worker count equal to the processor count.

The execution environment variants (M:V) are:

ValidCache

Before the body request handler is invoked a memory area with twice the size of the outer-most data cache is completely read. This fills the data cache with valid cache lines which are unrelated to the body request handler.

You can disable this variant with the T_MEASURE_RUNTIME_DISABLE_VALID_CACHE request flag.

HotCache

Before the body request handler is invoked the body request handler is called without measuring the runtime. The aim is to load all data used by the body request handler to the cache.

You can disable this variant with the T_MEASURE_RUNTIME_DISABLE_HOT_CACHE request flag.

DirtyCache

Before the body request handler is invoked a memory area with twice the size of the outer-most data cache is completely written with new data. This should produce a data cache with dirty cache lines which are unrelated to the body request handler. In addition, the entire instruction cache is invalidated.

You can disable this variant with the T_MEASURE_RUNTIME_DISABLE_DIRTY_CACHE request flag.

Load

This variant tries to get close to worst-case conditions. The cache is set up according to the DirtyCache variant. In addition, other processors try to fully load the memory system. The load is produced through writes to a memory area with twice the size of the outer-most data cache. The load variant is performed multiple times with a different set of active load worker threads (M:L). The active workers range from one up to the processor count.

You can disable these variants with the T_MEASURE_RUNTIME_DISABLE_MINOR_LOAD and T_MEASURE_RUNTIME_DISABLE_MAX_LOAD request flags.

On SPARC, the body request handler is called with a register window setting so that window overflow traps will occur in the next level function call.

Each execution in an environment variant produces a sample set of body request handler runtime measurements. The minimum (M:MI), first quartile (M:Q1), median (M:Q2), third quartile (M:Q3), maximum (M:MX), median absolute deviation (M:MAD), and the sum of the sample values (M:D) is reported.

Code Runtime Measurement Example
#include <t.h>

static void
empty(void *arg)
{
    (void)arg;
}

T_TEST_CASE(measure_empty)
{
    static const T_measure_runtime_config config = {
        .sample_count = 1024
    };
    T_measure_runtime_context *ctx;
    T_measure_runtime_request req;

    ctx = T_measure_runtime_create(&config);
    T_assert_not_null(ctx);

    memset(&req, 0, sizeof(req));
    req.name = "Empty";
    req.body = empty;
    T_measure_runtime(ctx, &req);
}
Code Runtime Measurement Report
B:measure_empty
P:0:0:UI1:test-rtems-measure.c:18
M:B:Empty
M:V:ValidCache
M:N:1024
M:MI:0.000000000
M:Q1:0.000000000
M:Q2:0.000000000
M:Q3:0.000000000
M:MX:0.000000009
M:MAD:0.000000000
M:D:0.000000485
M:E:Empty:D:0.208984183
M:B:Empty
M:V:HotCache
M:N:1024
M:MI:0.000000003
M:Q1:0.000000003
M:Q2:0.000000003
M:Q3:0.000000003
M:MX:0.000000006
M:MAD:0.000000000
M:D:0.000002626
M:E:Empty:D:0.000017046
M:B:Empty
M:V:DirtyCache
M:N:1024
M:MI:0.000000007
M:Q1:0.000000007
M:Q2:0.000000007
M:Q3:0.000000008
M:MX:0.000000559
M:MAD:0.000000000
M:D:0.000033244
M:E:Empty:D:1.887834875
M:B:Empty
M:V:Load
M:L:1
M:N:1024
M:MI:0.000000000
M:Q1:0.000000002
M:Q2:0.000000002
M:Q3:0.000000003
M:MX:0.000000288
M:MAD:0.000000000
M:D:0.000002421
M:E:Empty:D:0.001798809
[... 22 more load variants ...]
M:E:Empty:D:0.021252583
M:B:Empty
M:V:Load
M:L:24
M:N:1024
M:MI:0.000000001
M:Q1:0.000000002
M:Q2:0.000000002
M:Q3:0.000000003
M:MX:0.000001183
M:MAD:0.000000000
M:D:0.000003406
M:E:Empty:D:0.015188063
E:measure_empty:N:1:F:0:D:14.284869

8.1.12. Test Runner

You can call the T_main() function to run all registered test cases.

int T_main(const T_config *config);

The T_main() function returns 0 if all test cases passed, otherwise it returns 1. Concurrent execution of the T_main() function is undefined behaviour.

You can ask if you execute within the context of the test runner with the T_is_runner() function:

bool T_is_runner(void);

It returns true if you execute within the context of the test runner (the context which executes for example T_main()). Otherwise it returns false, for example if you execute in another task, in interrupt context, nobody executes T_main(), or during system initialization on another processor.

On RTEMS, you have to register the test cases with the T_register() function before you call T_main(). This makes it possible to run low level tests, for example without the operating system directly in boot_card() or during device driver initialization. On other platforms, the T_register() is a no operation.

void T_register(void);

You can run test cases also individually. Use T_run_initialize() to initialize the test runner. Call T_run_all() to run all or T_run_by_name() to run specific registered test cases. Call T_case_begin() to begin a freestanding test case and call T_case_end() to finish it. Finally, call T_run_finalize().

void T_run_initialize(const T_config *config);

void T_run_all(void);

void T_run_by_name(const char *name);

void T_case_begin(const char *name, const T_fixture *fixture);

void T_case_end(void);

bool T_run_finalize(void);

The T_run_finalize() function returns true if all test cases passed, otherwise it returns false. Concurrent execution of the runner functions (including T_main()) is undefined behaviour. The test suite configuration must be persistent throughout the test run.

typedef enum {
    T_EVENT_RUN_INITIALIZE,
    T_EVENT_CASE_EARLY,
    T_EVENT_CASE_BEGIN,
    T_EVENT_CASE_END,
    T_EVENT_CASE_LATE,
    T_EVENT_RUN_FINALIZE
} T_event;

typedef void (*T_action)(T_event, const char *);

typedef void (*T_putchar)(int, void *);

typedef struct {
    const char *name;
    char *buf;
    size_t buf_size;
    T_putchar putchar;
    void *putchar_arg;
    T_verbosity verbosity;
    T_time (*now)(void);
    size_t action_count;
    const T_action *actions;
} T_config;

With the test suite configuration you can specifiy the test suite name, the put character handler used the output the test report, the initial verbosity, the monotonic time provider and an optional set of test suite actions. Only the test runner calls the put character handler, other tasks or interrupt handlers write to a buffer which is emptied by the test runner on demand. You have to specifiy this buffer in the test configuration. The test suite actions are called with the test suite name for test suite run events (T_EVENT_RUN_INITIALIZE and T_EVENT_RUN_FINALIZE) and the test case name for the test case events (T_EVENT_CASE_EARLY, T_EVENT_CASE_BEGIN, T_EVENT_CASE_END and T_EVENT_CASE_LATE).

8.1.13. Test Verbosity

Three test verbosity levels are defined:

T_QUIET

Only the test suite begin, system, test case end, and test suite end lines are printed.

T_NORMAL

Prints everything except passed test lines.

T_VERBOSE

Prints everything.

The test verbosity level can be set within the scope of one test case with the T_set_verbosity() function:

T_verbosity T_set_verbosity(T_verbosity new_verbosity);

The function returns the previous verbosity. After the test case, the configured verbosity is automatically restored.

An example with T_QUIET verbosity:

A:xyz
S:Platform:RTEMS
[...]
E:a:N:2:F:1
E:b:N:0:F:1
E:c:N:1:F:1
E:d:N:6:F:0
Z:xyz:C:4:N:9:F:3

The same example with T_NORMAL verbosity:

A:xyz
S:Platform:RTEMS
[...]
B:a
F:1:0:UI1:test-verbosity.c:6:test fails
E:a:N:2:F:1
B:b
F:*:0:UI1:test-verbosity.c:12:quiet test fails
E:b:N:0:F:1
B:c
F:0:0:UI1:test-verbosity.c:17:this is a format string
E:c:N:1:F:1
B:d
E:d:N:6:F:0
Z:xyz:C:4:N:9:F:3

The same example with T_VERBOSE verbosity:

A:xyz
S:Platform:RTEMS
[...]
B:a
P:0:0:UI1:test-verbosity.c:5
F:1:0:UI1:test-verbosity.c:6:test fails
E:a:N:2:F:1
B:b
F:*:0:UI1:test-verbosity.c:12:quiet test fails
E:b:N:0:F:1
B:c
F:0:0:UI1:test-verbosity.c:17:this is a format string
E:c:N:1:F:1
B:d
P:0:0:UI1:test-verbosity.c:22
P:1:0:UI1:test-verbosity.c:23
P:2:0:UI1:test-verbosity.c:24
P:3:0:UI1:test-verbosity.c:25
P:4:0:UI1:test-verbosity.c:26
P:5:0:UI1:test-verbosity.c:27
E:d:N:6:F:0
Z:xyz:C:4:N:9:F:3

8.1.14. Test Reporting

The test reporting is line based which should be easy to parse with a simple state machine. Each line consists of a set of fields separated by colon characters (:). The first character of the line determines the line format:

A

A test suite begin line. It has the format:

A:<TestSuite>

A description of the field follows:

<TestSuite>

The test suite name. Must not contain colon characters (:).

S

A test suite system line. It has the format:

S:<Key>:<Value>

A description of the fields follows:

<Key>

A key string. Must not contain colon characters (:).

<Value>

An arbitrary key value string. May contain colon characters (:).

B

A test case begin line. It has the format:

B:<TestCase>

A description of the field follows:

<TestCase>

A test case name. Must not contain colon characters (:).

P

A test pass line. It has the format:

P:<Step>:<Processor>:<Task>:<File>:<Line>

A description of the fields follows:

<Step>

Each non-quiet test has a unique test step counter value in each test case execution. The test step counter is set to zero before the test case executes. For quiet test checks, there is no associated test step and the character * instead of an integer is used to indicate this.

<Processor>

The processor index of the processor which executed at least one instruction of the corresponding test.

<Task>

The name of the task which executed the corresponding test if the test executed in task context. The name ISR indicates that the test executed in interrupt context. The name ? indicates that the test executed in an arbitrary context with no valid executing task.

<File>

The name of the source file which contains the corresponding test. A source file of * indicates that no test source file is associated with the test, e.g. it was produced by the test framework itself.

<Line>

The line of the test statement in the source file which contains the corresponding test. A line number of * indicates that no test source file is associated with the test, e.g. it was produced by the test framework itself.

F

A test failure line. It has the format:

F:<Step>:<Processor>:<Task>:<File>:<Line>:<Message>

A description of the fields follows:

<Step> <Processor> <Task> <File> <Line>

See above P line.

<Message>

An arbitrary message string. May contain colon characters (:).

L

A log message line. It has the format:

L:<Message>

A description of the field follows:

<Message>

An arbitrary message string. May contain colon characters (:).

E

A test case end line. It has the format:

E:<TestCase>:N:<Steps>:F:<Failures>:D:<Duration>

A description of the fields follows:

<TestCase>

A test case name. Must not contain colon characters (:).

<Steps>

The final test step counter of a test case. Quiet test checks produce no test steps.

<Failures>

The count of failed test checks of a test case.

<Duration>

The test case duration in seconds.

Z

A test suite end line. It has the format:

Z:<TestSuite>:C:<TestCases>:N:<OverallSteps>:F:<OverallFailures>:D:<Duration>

A description of the fields follows:

<TestSuite>

The test suite name. Must not contain colon characters (:).

<TestCases>

The count of test cases in the test suite.

<OverallSteps>

The overall count of test steps in the test suite.

<OverallFailures>

The overall count of failed test cases in the test suite.

<Duration>

The test suite duration in seconds.

Y

Auxiliary information line. Issued after the test suite end. It has the format:

Y:ReportHash:SHA256:<Hash>

A description of the fields follows:

<Hash>

The SHA256 hash value of the test suite report from the begin to the end of the test suite.

M

A code runtime measurement line. It has the formats:

M:B:<Name>

M:V:<Variant>

M:L:<Load>

M:N:<SampleCount>

M:S:<Count>:<Value>

M:MI:<Minimum>

M:Q1:<FirstQuartile>

M:Q2:<Median>

M:Q3:<ThirdQuartile>

M:MX:<Maximum>

M:MAD:<MedianAbsoluteDeviation>

M:D:<SumOfSampleValues>

M:E:<Name>:D:<Duration>

A description of the fields follows:

<Name>

A code runtime measurement name. Must not contain colon characters (:).

<Variant>

The execution variant which is one of ValidCache, HotCache, DirtyCache, or Load.

<Load>

The active load workers count which ranges from one to the processor count.

<SampleCount>

The sample count as defined by the runtime measurement configuration.

<Count>

The count of samples with the same value.

<Value>

A sample value in seconds.

<Minimum>

The minimum of the sample set in seconds.

<FirstQuartile>

The first quartile of the sample set in seconds.

<Median>

The median of the sample set in seconds.

<ThirdQuartile>

The third quartile of the sample set in seconds.

<Maximum>

The maximum of the sample set in seconds.

<MedianAbsoluteDeviation>

The median absolute deviation of the sample set in seconds.

<SumOfSampleValues>

The sum of all sample values of the sample set in seconds.

<Duration>

The runtime measurement duration in seconds. It includes time to set up the execution environment variant.

Example Test Report
A:xyz
S:Platform:RTEMS
S:Compiler:7.4.0 20181206 (RTEMS 5, RSB e0aec65182449a4e22b820e773087636edaf5b32, Newlib 1d35a003f)
S:Version:5.0.0.820977c5af17c1ca2f79800d64bd87ce70a24c68
S:BSP:erc32
S:RTEMS_DEBUG:1
S:RTEMS_MULTIPROCESSING:0
S:RTEMS_POSIX_API:1
S:RTEMS_PROFILING:0
S:RTEMS_SMP:1
B:timer
P:0:0:UI1:test-rtems.c:26
P:1:0:UI1:test-rtems.c:29
P:2:0:UI1:test-rtems.c:33
P:3:0:ISR:test-rtems.c:14
P:4:0:ISR:test-rtems.c:15
P:5:0:UI1:test-rtems.c:38
P:6:0:UI1:test-rtems.c:39
P:7:0:UI1:test-rtems.c:42
E:timer:N:8:F:0:D:0.019373
B:rsc_success
P:0:0:UI1:test-rtems.c:59
F:1:0:UI1:test-rtems.c:60:RTEMS_INVALID_NUMBER == RTEMS_SUCCESSFUL
F:*:0:UI1:test-rtems.c:62:RTEMS_INVALID_NUMBER == RTEMS_SUCCESSFUL
P:2:0:UI1:test-rtems.c:63
F:3:0:UI1:test-rtems.c:64:RTEMS_INVALID_NUMBER == RTEMS_SUCCESSFUL
E:rsc_success:N:4:F:3:D:0.011128
B:rsc
P:0:0:UI1:test-rtems.c:48
F:1:0:UI1:test-rtems.c:49:RTEMS_INVALID_NUMBER == RTEMS_INVALID_ID
F:*:0:UI1:test-rtems.c:51:RTEMS_INVALID_NUMBER == RTEMS_INVALID_ID
P:2:0:UI1:test-rtems.c:52
F:3:0:UI1:test-rtems.c:53:RTEMS_INVALID_NUMBER == RTEMS_INVALID_ID
E:rsc:N:4:F:3:D:0.011083
Z:xyz:C:3:N:16:F:6:D:0.047201
Y:ReportHash:SHA256:e5857c520dd9c9b7c15d4a76d78c21ccc46619c30a869ecd11bbcd1885155e0b

8.1.15. Test Report Validation

You can add the T_report_hash_sha256() test suite action to the test suite configuration to generate and report the SHA256 hash value of the test suite report. The hash value covers everything reported by the test suite run from the begin to the end. This can be used to check that the report generated on the target is identical to the report received on the report consumer side. The hash value is reported after the end of test suite line (Z) as auxiliary information in a Y line. Consumers may have to reverse a \n to \r\n conversion before the hash is calculated. Such a conversion could be performed by a particular put character handler provided by the test suite configuration.

8.1.16. Supported Platforms

The framework runs on FreeBSD, MSYS2, Linux and RTEMS.

8.2. Test Framework Requirements for RTEMS

The requirements on a test framework suitable for RTEMS are:

8.2.1. License Requirements

TF.License.Permissive

The test framework shall have a permissive open source license such as BSD-2-Clause.

8.2.2. Portability Requirements

TF.Portability

The test framework shall be portable.

TF.Portability.RTEMS

The test framework shall run on RTEMS.

TF.Portability.POSIX

The test framework shall be portable to POSIX compatible operating systems. This allows to run test cases of standard C/POSIX/etc. APIs on multiple platforms.

TF.Portability.POSIX.Linux

The test framework shall run on Linux.

TF.Portability.POSIX.FreeBSD

The test framework shall run on FreeBSD.

TF.Portability.C11

The test framework shall be written in C11.

TF.Portability.Static

Test framework shall not use dynamic memory for basic services.

TF.Portability.Small

The test framework shall be small enough to support low-end platforms (e.g. 64KiB of RAM/ROM should be sufficient to test the architecture port, e.g. no complex stuff such as file systems, etc.).

TF.Portability.Small.LinkTimeConfiguration

The test framework shall be configured at link-time.

TF.Portability.Small.Modular

The test framework shall be modular so that only necessary parts end up in the final executable.

TF.Portability.Small.Memory

The test framework shall not aggregate data during test case executions.

8.2.3. Reporting Requirements

TF.Reporting

Test results shall be reported.

TF.Reporting.Verbosity

The test report verbosity shall be configurable. This allows different test run scenarios, e.g. regression test runs, full test runs with test report verification against the planned test output.

TF.Reporting.Verification

It shall be possible to use regular expressions to verify test reports line by line.

TF.Reporting.Compact

Test output shall be compact to avoid long test runs on platforms with a slow output device, e.g. 9600 Baud UART.

TF.Reporting.PutChar

A simple output one character function provided by the platform shall be sufficient to report the test results.

TF.Reporting.NonBlocking

The ouptut functions shall be non-blocking.

TF.Reporting.Printf

The test framework shall provide printf()-like output functions.

TF.Reporting.Printf.WithFP

There shall be a printf()-like output function with floating point support.

TF.Reporting.Printf.WithoutFP

There shall be a printf()-like output function without floating point support on RTEMS.

TF.Reporting.Platform

The test platform shall be reported.

TF.Reporting.Platform.RTEMS.Git

The RTEMS source Git commit shall be reported.

TF.Reporting.Platform.RTEMS.Arch

The RTEMS architecture name shall be reported.

TF.Reporting.Platform.RTEMS.BSP

The RTEMS BSP name shall be reported.

TF.Reporting.Platform.RTEMS.Tools

The RTEMS tool chain version shall be reported.

TF.Reporting.Platform.RTEMS.Config.Debug

The shall be reported if RTEMS_DEBUG is defined.

TF.Reporting.Platform.RTEMS.Config.Multiprocessing

The shall be reported if RTEMS_MULTIPROCESSING is defined.

TF.Reporting.Platform.RTEMS.Config.POSIX

The shall be reported if RTEMS_POSIX_API is defined.

TF.Reporting.Platform.RTEMS.Config.Profiling

The shall be reported if RTEMS_PROFILING is defined.

TF.Reporting.Platform.RTEMS.Config.SMP

The shall be reported if RTEMS_SMP is defined.

TF.Reporting.TestCase

The test cases shall be reported.

TF.Reporting.TestCase.Begin

The test case begin shall be reported.

TF.Reporting.TestCase.End

The test case end shall be reported.

TF.Reporting.TestCase.Tests

The count of test checks of the test case shall be reported.

TF.Reporting.TestCase.Failures

The count of failed test checks of the test case shall be reported.

TF.Reporting.TestCase.Timing

Test case timing shall be reported.

TF.Reporting.TestCase.Tracing

Automatic tracing and reporting of thread context switches and interrupt service routines shall be optionally performed.

8.2.4. Environment Requirements

TF.Environment

The test framework shall support all environment conditions of the platform.

TF.Environment.SystemStart

The test framework shall run during early stages of the system start, e.g. valid stack pointer, initialized data and cleared BSS, nothing more.

TF.Environment.BeforeDeviceDrivers

The test framework shall run before device drivers are initialized.

TF.Environment.InterruptContext

The test framework shall support test case code in interrupt context.

8.2.5. Usability Requirements

TF.Usability

The test framework shall be easy to use.

TF.Usability.TestCase

It shall be possible to write test cases.

TF.Usability.TestCase.Independence

It shall be possible to write test cases in modules independent of the test runner.

TF.Usability.TestCase.AutomaticRegistration

Test cases shall be registered automatically, e.g. via constructors or linker sets.

TF.Usability.TestCase.Order

It shall be possible to sort the registered test cases (e.g. random, by name) before they are executed.

TF.Usability.TestCase.Resources

It shall be possible to use resources with a life time restricted to the test case.

TF.Usability.TestCase.Resources.Memory

It shall be possible to dynamically allocate memory which is automatically freed once the test case completed.

TF.Usability.TestCase.Resources.File

It shall be possible to create a file which is automatically unlinked once the test case completed.

TF.Usability.TestCase.Resources.Directory

It shall be possible to create a directory which is automatically removed once the test case completed.

TF.Usability.TestCase.Resources.FileDescriptor

It shall be possible to open a file descriptor which is automatically closed once the test case completed.

TF.Usability.TestCase.Fixture

It shall be possible to use a text fixture for test cases.

TF.Usability.TestCase.Fixture.SetUp

It shall be possible to provide a set up handler for each test case.

TF.Usability.TestCase.Fixture.TearDown

It shall be possible to provide a tear down handler for each test case.

TF.Usability.TestCase.Context

The test case context shall be verified a certain points.

TF.Usability.TestCase.Context.VerifyAtEnd

After a test case exection it shall be verified that the context is equal to the context at the test case begin. This helps to ensure that test cases are independent of each other.

TF.Usability.TestCase.Context.VerifyThread

The test framework shall provide a function to ensure that the test case code executes in normal thread context. This helps to ensure that operating system service calls return to a sane context.

TF.Usability.TestCase.Context.Configurable

The context verified in test case shall be configurable at link-time.

TF.Usability.TestCase.Context.ThreadDispatchDisableLevel

It shall be possible to verify the thread dispatch disable level.

TF.Usability.TestCase.Context.ISRNestLevel

It shall be possible to verify the ISR nest level.

TF.Usability.TestCase.Context.InterruptLevel

It shall be possible to verify the interrupt level (interrupts enabled/disabled).

TF.Usability.TestCase.Context.Workspace

It shall be possible to verify the workspace.

TF.Usability.TestCase.Context.Heap

It shall be possible to verify the heap.

TF.Usability.TestCase.Context.OpenFileDescriptors

It shall be possible to verify the open file descriptors.

TF.Usability.TestCase.Context.Classic

It shall be possible to verify Classic API objects.

TF.Usability.TestCase.Context.Classic.Barrier

It shall be possible to verify Classic API Barrier objects.

TF.Usability.TestCase.Context.Classic.Extensions

It shall be possible to verify Classic API User Extensions objects.

TF.Usability.TestCase.Context.Classic.MessageQueues

It shall be possible to verify Classic API Message Queue objects.

TF.Usability.TestCase.Context.Classic.Partitions

It shall be possible to verify Classic API Partition objects.

TF.Usability.TestCase.Context.Classic.Periods

It shall be possible to verify Classic API Rate Monotonic Period objects.

TF.Usability.TestCase.Context.Classic.Regions

It shall be possible to verify Classic API Region objects.

TF.Usability.TestCase.Context.Classic.Semaphores

It shall be possible to verify Classic API Semaphore objects.

TF.Usability.TestCase.Context.Classic.Tasks

It shall be possible to verify Classic API Task objects.

TF.Usability.TestCase.Context.Classic.Timers

It shall be possible to verify Classic API Timer objects.

TF.Usability.TestCase.Context.POSIX

It shall be possible to verify POSIX API objects.

TF.Usability.TestCase.Context.POSIX.Keys

It shall be possible to verify POSIX API Key objects.

TF.Usability.TestCase.Context.POSIX.KeyValuePairs

It shall be possible to verify POSIX API Key Value Pair objects.

TF.Usability.TestCase.Context.POSIX.MessageQueues

It shall be possible to verify POSIX API Message Queue objects.

TF.Usability.TestCase.Context.POSIX.Semaphores

It shall be possible to verify POSIX API Named Semaphores objects.

TF.Usability.TestCase.Context.POSIX.Shms

It shall be possible to verify POSIX API Shared Memory objects.

TF.Usability.TestCase.Context.POSIX.Threads

It shall be possible to verify POSIX API Thread objects.

TF.Usability.TestCase.Context.POSIX.Timers

It shall be possible to verify POSIX API Timer objects.

TF.Usability.Assert

There shall be functions to assert test objectives.

TF.Usability.Assert.Safe

Test assert functions shall be safe to use, e.g. assert(a == b) vs. assert(a = b) vs. assert_eq(a, b).

TF.Usability.Assert.Continue

There shall be assert functions which allow the test case to continue in case of an assertion failure.

TF.Usability.Assert.Abort

There shall be assert functions which abourt the test case in case of an assertion failure.

TF.Usability.EasyToWrite

It shall be easy to write test code, e.g. avoid long namespace prefix rtems_test_*.

TF.Usability.Threads

The test framework shall support multi-threading.

TF.Usability.Pattern

The test framework shall support test patterns.

TF.Usability.Pattern.Interrupts

The test framework shall support test cases which use interrupts, e.g. spintrcritical*.

TF.Usability.Pattern.Parallel

The test framework shall support test cases which want to run code in parallel on SMP machines.

TF.Usability.Pattern.Timing

The test framework shall support test cases which want to measure the timing of code sections under various platform conditions, e.g. dirty cache, empty cache, hot cache, with load from other processors, etc..

TF.Usability.Configuration

The test framework shall be configurable.

TF.Usability.Configuration.Time

The timestamp function shall be configurable, e.g. to allow test runs without a clock driver.

8.2.6. Performance Requirements

TF.Performance.RTEMS.No64BitDivision

The test framework shall not use 64-bit divisions on RTEMS.

8.3. Off-the-shelf Test Frameworks

There are several off-the-shelf test frameworks for C/C++. The first obstacle for test frameworks is the license requirement (TF.License.Permissive).

8.3.1. bdd-for-c

In the bdd-for-c framework the complete test suite must be contained in one file and the main function is generated. This violates TF.Usability.TestCase.Independence.

8.3.2. CBDD

The CBDD framework uses the C blocks extension from clang. This violates TF.Portability.C11.

8.3.3. Google Test

Google Test 1.8.1 is supported by RTEMS. Unfortunately, it is written in C++ and is to heavy weight for low-end platforms. Otherwise it is a nice framework.

8.3.4. Unity

The Unity Test API does not meet our requirements. There was a discussion on the mailing list in 2013.

8.4. Standard Test Report Formats

8.4.1. JUnit XML

A common test report format is JUnit XML.

<?xml version="1.0" encoding="UTF-8" ?>
<testsuites id="xyz" name="abc" tests="225" failures="1262" time="0.001">
  <testsuite id="def" name="ghi" tests="45" failures="17" time="0.001">
    <testcase id="jkl" name="mno" time="0.001">
      <failure message="pqr" type="stu"></failure>
      <system-out>stdout</system-out>
      <system-err>stderr</system-err>
    </testcase>
  </testsuite>
</testsuites>

The major problem with this format is that you have to output the failure count of all test suites and the individual test suite before the test case output. You know the failure count only after a complete test run. This runs contrary to requirement TF.Portability.Small.Memory. It is also a bit verbose (TF.Reporting.Compact).

It is easy to convert a full test report generated by The RTEMS Test Framework to the JUnit XML format.

8.4.2. Test Anything Protocol

The Test Anything Protocol (TAP) is easy to consume and produce.

1..4
ok 1 - Input file opened
not ok 2 - First line of the input valid
ok 3 - Read the rest of the file
not ok 4 - Summarized correctly # TODO Not written yet

You have to know in advance how many test statements you want to execute in a test case. The problem with this format is that there is no standard way to provide auxiliary data such as test timing or a tracing report.

It is easy to convert a full test report generated by The RTEMS Test Framework to the TAP format.

9. Software Release Management

9.1. Release Process

The release process creates an RTEMS release. The process has a number of stages that happen before a release can be made, during the creation of the release procedure and after the release has been made.

9.1.1. Releases

RTEMS is released as a collection of ready to use source code and built documentation. Releases are publicly available on the RTEMS servers under https://ftp.rtems.org/pub/rtems/releases.

Releases are group under the major version number as a collection of directories consisting of the version number. This is termed a release series. A release may also contain release snapshots.

All releases must have a three digit version number and this can be optionally followed by a dash character (-) and an identifier, e.g. 5.1.0-acme-1.

The RTEMS Project reserves releases with only the three digit version number, e.g. 5.1.0. This identifies an RTEMS Project release.

9.1.1.1. Release Layout
  • All released source archives are XZ compressed tar files.

  • Top level contains:

README.txt:

A set of brief release instructions.

contrib:

Contributed sources. For example the release scripts used to create the release.

docs:

Compressed documentation build in HTML, Single page HTML and PDF formats. Provide compressed files for each document and a single archive of all the documentation. Provide an SHA512 check sum file.

rtems-<VERSION>-release-notes.pdf:

RTEMS Release notes document the changes in a release. This is a capture of the Trac report for the release’s milestone in PDF format.

sha512sum.txt:

SHA512 checksum of all files in this directory.

sources:

All source code referenced by the release.

9.1.1.2. Release Version Numbering

The release numbering scheme changed with RTEMS 5.

The master branch has the version N.0.0 with N being the next major release number. The release branch in a repository will be just the major number.

The first release of this series will have the version number N.1.0. The first bugfix release (minor release) of this series will have the version number N.2.0.

The release branch will have the version number N.M.1 with M being the last minor release of this series. Tools will use N as the version number and must be compatible with all releases and the release branch of the N series.

Examples:

  • 5.0.0 is the version number of the development master for the 5 series

  • 5.1.0 is the first release of the 5 series

  • 5.1.1 is the version number of the 5 series release branch right after the 5.1.0 release until 5.2.0 is released

  • 5.2.0 is the first bugfix release of the 5 series

  • 5.2.1 is the version number of the 5 series release branch right after the 5.2.0 release until 5.3.0 is released

  • 6.0.0 is the version number of the development master for the 6 series

9.1.1.3. Release Scripts
  • The release scripts are held in the top level repository https://git.rtems.org/rtems-release.git.

  • The scripts are written for FreeBSD and can run on FreeBSD 10 through FreeBSD 12. No other host operating system is supported for the release scripts. Updates are welcome if the changes do not affect the operation on FreeBSD.

  • A Python virutalenv environment is required for a working Sphinx documentation building environment. Follow the procedure in the rtems-docs.git top level README file.

  • Building a standard release requires you provide the release major number and the release’s remaining version string including any additional identifiers:

    ./rtems-release 5 1.0
    

    To create a release snapshot:

    ./rtems-release 5 0.0-m2003
    
  • A 3rd option of a release URL can be provided to create a test or deployable release. The URL is a base path the RSB uses to download the release source files from:

    ./rtems-release \
        -u https://ftp.rtems.org/pub/rtems/people/chrisj/releases \
        5 0.0-m2003-2
    
  • Building the release notes requires the Web Toolkit HTML to PDF converter be installed. The FreeBSD package is wkhtmltopdf.

9.1.1.4. Release Snapshots
  • Release snapshots are only created for the current development version of RTEMS. For example RTEMS 5 snapshot path is 5/5.0.0/5.0.0-m2003.

  • Release snapshots are based on the development sources and may be unstable or not suitable for use in production.

  • A release snapshot is created each month and is named as <major>/<version>/<version>-<YYMM> where YY is the last two digits of the current year and MM is the month as a two digit number.

  • In the lead up to a release more than one snapshot can be created by appending -<count> to the snapshot version string where <count> is incremented starting from 1. The first snapshot without a count is considered number 0.

  • Release snapshots maybe removed from the RTEMS servers at the discretion of the RTEMS project

9.1.2. Release Repositories

The following are the repositories that a release effects. Any repository action is to be performed in the following repositories:

  1. rtems.git

  2. rtems-docs.git

  3. rtems-examples.git

  4. rtems-libbsd.git

  5. rtems-source-builder.git

  6. rtems-tools

  7. rtems_waf

9.1.3. Pre-Release Procedure

  • All tickets must be resolved, closed or moved to a later milestone.

  • The following BSP must build using the RSB:

    • arm/beagleboneblack

  • Branch labels are the major number as branch releases increment the minor number. A branch is only created when the first major release is made.

    The commands to set a remote branch for a release in a repository are:

    git checkout -b <VERSION> origin/master
    git push origin <VERSION>
    

    Example:

    git clone ssh://chrisj@dispatch.rtems.org/data/git/rtems.git rtems.git
    cd rtems.git
    git checkout -b 5 origin/master
    git push origin 5
    

9.1.4. Release Procedure

The release procedure can be performed on any FreeBSD machine and uploaded to the RTEMS FTP server. You will need ssh access to the RTEMS server dispatch.rtems.org and suitable permissions to write into the FTP release path on the RTEMS server.

  1. To create the RTEMS release run the release script:

    ./rtems-release <VERSION> <REVISION>
    

    Example:

    cd
    mkdir -p development/rtems/releases
    cd development/rtems/releases
    git clone git://git.rtems.org/rtems-release.git rtems-release.git
    cd rtems-release.git
    ./rtems-release 5 1.0
    
  2. Copy the release to the RTEMS FTP server:

    ssh <user>@dispatch.rtems.org mkdir -p /data/ftp/pub/rtems/releases/<VERSION>
    scp -r <VERSION>.<REVISION> <user>@dispatch.rtems.org:/data/ftp/pub/rtems/releases/<VERSION>/.
    

    Example:

    ssh chrisj@dispatch.rtems.org mkdir -p /data/ftp/pub/rtems/releases/5
    scp -r 5.1.0 chrisj@dispatch.rtems.org:/data/ftp/pub/rtems/releases/5/.
    
  3. Verify the release has been uploaded by checking the link:

    https://ftp.rtems.org/pub/rtems/releases/<VERSION>/<VERSION>.<REVISION>

  4. Tag the release repositories with the following command:

    git checkout -b origin/<VERSION>
    git tag <TAG>
    git push origin <TAG>
    

    Example:

    git clone ssh://chrisj@dispatch.rtems.org/data/git/rtems.git rtems.git
    cd rtems.git
    git checkout -b origin/5
    git tag 5.1.0
    git push origin 5.1.0
    

9.1.5. Post-Release Procedure

The following procedures are performed after a release has been created.

  1. TBD

9.2. Software Change Report Generation

TBD - What goes here?

9.3. Version Description Document (VDD) Generation

TBD - discuss how generated. Preferably Dannie’s project

This URL may be of use but it probably Trac auto-generated and can only be referenced: https://devel.rtems.org/wiki/TracChangeLog

10. User’s Manuals

TBD - write and link to useful documentation, potential URLs:

Reference the RTEMS Classic API Guide

Reference any other existing user documentation

10.1. Documentation Style Guidelines

TBD - write me

11. Licensing Requirements

All artifacts shall adhere to RTEMS Project licensing requirements. Currently, the preferred licenses are:

  • “Two Clause BSD” (BSD-2-Clause) for source code, and

  • CC-BY-SA-4.0 license for documentation

Historically, RTEMS has been licensed under the GPL v2 with linking exception (https://www.rtems.org/license). It is preferred that new submissions be under one of the two preferred licenses. If you have previously submitted code to RTEMS under a historical license, please grant the project permission to relicense. See https://devel.rtems.org/ticket/3053 for details.

For example templates for what to include in source code and documentation, see Copyright and License Block.

11.1. Rationale

RTEMS is intended for use in real-time embedded systems in which the application is statically linked with the operating system and all support libraries. Given this use case, the RTEMS development team evaluated a variety of licenses with with the goal of promoting use while protecting both users and the developers.

Using the GNU General Public License Version 2 (GPLv2) unmodified was considered but discarded because the GPL can only be linked statically with other GPL code. Put simply, linking your application code statically with GPL code would cause your code to become GPL code. This would force both licensing and redistribution requirements onto RTEMS users. This was completely unacceptable.

The GNU Lesser General Public License Version 2 (LGPLv2) was also considered and deemed to not be a suitable license for RTEMS. This is because it either requires use of a shared library that can be re-linked, or release of the linked (application) code. This would require an RTEMS-based embedded system to provide a “relinking kit.” Again, this license would force an unacceptable requirement on RTEMS users and deemed unacceptable.

Newer versions of the GPL (i.e. version 3) are completely unsuitable for embedded systems due to the additions which add further restrictions on end user applications.

The historical RTEMS License is a modified version of the GPL version 2 that includes an exception to permit including headers and linking against RTEMS object files statically. This was based on the license used by GCC language runtime libraries at that time. This license allows the static linking of RTEMS with applications without forcing obligations and restrictions on users.

A problem for RTEMS is there are no copyleft licenses that are compatible with the deployment model of RTEMS. Thus, RTEMS Project has to reject any code that uses the GPL or LGPL, even though RTEMS has historically appeared to use the GPL itself – but with the exception for static linking, and also because an upstream GPL version 2 project could at any time switch to GPL version 3 and become totally unusable. In practice, RTEMS can only accept original code contributed under the RTEMS License and code that has a permissive license.

As stated above, the RTEMS Project has defined its preferred licenses. These allow generation of documentation and software from specification as well as allow end users to statically link with RTEMS and not incur obligations.

In some cases, RTEMS includes software from third-party projects. In those cases, the license is carefully evaluated to meet the project licensing goals. The RTEMS Project can only include software under licenses which follow these guidelines:

  • 2- and 3-clause BSD, MIT, and other OSI-approved non-copyleft licenses that permit statically linking with the code of different licenses are acceptable.

  • The historical RTEMS License is acceptable for software already in the tree. This software is being relicensed to BSD-2-Clause, rewritten, or removed.

  • GPL licensed code is NOT acceptable, neither is LGPL.

  • Software which is dual-licensed in a manner which prevents free use in commercial applications is not acceptable.

  • Advertising obligations are not acceptable.

  • Some license restrictions may be permissible. These will be considered on a case-by-case basis.

In practice, these guidelines are not hard to follow. Critically, they protect the freedom of the RTEMS source code and that of end users to select the license and distribution terms they prefer for their RTEMS-based application.

12. Appendix: Core Qualification Artifacts/Documents

An effort at NASA has been performed to suggest a core set of artifacts (as defined by BOTH NASA NPR 7150.2B and DO-178B) that can be utilized by a mission as a baselined starting point for “pre-qualification” for (open-source) software that is intended to be utilized for flight purposes. This effort analyzed the overlap between NPR 7150.2B and DO-178B and highlighted a core set of artifacts to serve as a starting point for any open-source project. These artifacts were also cross-referenced with similar activities for other NASA flight software qualification efforts, such as the open-source Core Flight System (cFS). Along with the specific artifact, the intent of the artifact was also captured; in some cases open-source projects, such as RTEMS, are already meeting the intent of the artifacts with information simply needing organized and formalized. The table below lists the general category, artifact name, and its intent. Please note that this table does NOT represent all the required artifacts for qualification per the standards; instead, this table represents a subset of the most basic/core artifacts that form a strong foundation for a software engineering qualification effort.

Table 1. Core Qualification Artifacts

Category

Artifact

Intent

Requirements

Software Requirements Specification (SRS)

Requirements Management

The project shall document the software requirements.

The project shall collect and manage changes to the software requirements.

The project shall identify, initiate corrective actions, and track until closure inconsistencies among requirements, project plans, and software products.

Requirements Test and Traceability Matrix

The project shall perform, document, and maintain bidirectional traceability between the software requirement and the higher-level requirement.

Validation

The project shall perform validation to ensure that the software will perform as intended in the customer environment.

Design and Implementation

Software Development or Management Plan

A plan for how you will develop the software that you are intent upon developing and delivering.

The Software Development Plan includes the objectives, standards and life cycle(s) to be used in the software development process. This plan should include: Standards: Identification of the Software Requirements Standards, Software Design Standards, and Software Code Standards for the project.

Software Configuration Management Plan

To identify and control major software changes, ensure that change is being properly implemented, and report changes to any other personnel or clients who may have an interest.

Implementation

The project shall implement the software design into software code.

Executable Code to applicable tested software.

Coding Standards Report

The project shall ensure that software coding methods, standards, and/or criteria are adhered to and verified.

Version Description Document (VDD)

The project shall provide a Software Version Description document for each software release.

Testing and Software Assurance Activities

Software Test Plan

Document describing the testing scope and activities.

Software Assurance/Testing Procedures

To define the techniques, procedures, and methodologies that will be used.

Software Change Report / Problem Report

The project shall regularly hold reviews of software activities, status, and results with the project stakeholders and track issues to resolution.

Software Schedule

Milestones have schedule and schedule is updated accordingly.

Software Test Report / Verification Results

The project shall record, address, and track to closure the results of software verification activities.

Usability

Software User’s Manual

The Software User Manual defines user instructions for the software.

In an effort to remain lightweight and sustainable for open-source projects, Table 1 above was condensed into a single artifact outline that encompasses the artifacts’ intents. The idea is that this living qualification document will reside under RTEMS source control and be updated with additional detail accordingly. The artifact outline is as follows:

13. Glossary

ABI

An acronym for Application Binary Interface.

API

An acronym for Application Programming Interface.

assembler language

The assembler language is a programming language which can be translated very easily into machine code and data. For this project assembler languages are restricted to languages accepted by the GNU assembler program for the target architectures.

C language

The C language for this project is defined in terms of C11.

C11

The standard ISO/IEC 9899:2011.

CCB

An acronym for Change Control Board.

Doorstop

Doorstop is a requirements management tool.

EARS

An acronym for Easy Approach to Requirements Syntax.

ELF

An acronym for Executable and Linkable Format.

GCC

An acronym for GNU Compiler Collection.

GNAT

GNAT is the GNU compiler for Ada, integrated into the GCC.

GNU

An acronym for GNU’s Not Unix.

interrupt service

An interrupt service consists of an Interrupt Service Routine which is called with a user provided argument upon reception of an interrupt service request. The routine is invoked in interrupt context. Interrupt service requests may have a priority and an affinity to a set of processors. An interrupt service is a software component.

Interrupt Service Routine

An ISR is invoked by the CPU to process a pending interrupt.

ISVV

An acronym for Independent Software Verification and Validation.

ReqIF

An acronym for Requirements Interchange Format.

RTEMS

An acronym for Real-Time Executive for Multiprocessor Systems.

software component

This term is defined by ECSS-E-ST-40C 3.2.28 as a “part of a software system”. For this project a software component shall be any of the following items and nothing else:

Please note that explicitly defined ELF symbols and assembler language data are considered a software component only if they are defined in a source code file. For example, this rules out symbols and data generated as side-effects by the toolchain (compiler, assembler, linker) such as jump tables, linker trampolines, exception frame information, etc.

software item

This term has the same meaning as software product.

software product

The software product is the RTEMS real-time operating system.

software unit

This term is defined by ECSS-E-ST-40C 3.2.24 as a “separately compilable piece of source code”. For this project a software unit shall be any of the following items and nothing else:

A software unit is a software component.

source code

This project uses the source code definition of the Linux Information Project: “Source code (also referred to as source or code) is the version of software as it is originally written (i.e., typed into a computer) by a human in plain text (i.e., human readable alphanumeric characters).”

task

This project uses the thread definition of Wikipedia: “a thread of execution is the smallest sequence of programmed instructions that can be managed independently by a scheduler, which is typically a part of the operating system.”

It consists normally of a set of registers and a stack. The scheduler assigns processors to a subset of the ready tasks. The terms task and thread are synonym in RTEMS. The term task is used throughout the Classic API, however, internally in the operating system implementation and the POSIX API the term thread is used.

A task is a software component.

thread

This term has the same meaning as task.

YAML

An acronym for YAML Ain’t Markup Language.

14. References

Bra97

Scott Bradner. Key words for use in RFCs to Indicate Requirement Levels. BCP 14, RFC Editor, March 1997. http://www.rfc-editor.org/rfc/rfc2119.txt. URL: http://www.rfc-editor.org/rfc/rfc2119.txt.

BA14

Jace Browning and Robert Adams. Doorstop: Text-Based Requirements Management Using Version Control. Journal of Software Engineering and Applications, 7:187–194, 2014. URL: http://www.scirp.org/pdf/JSEA_2014032713545074.pdf.

ECS09

ECSS. ECSS-E-ST-10-06C - Technical requirements specification. European Cooperation for Space Standardization, 2009. URL: https://ecss.nl/standard/ecss-e-st-10-06c-technical-requirements-specification/.

MW10

Alistair Mavin and Philip Wilkinson. Big Ears (The Return of Easy Approach to Requirements Engineering). In 18th Requirements Engineering Conference, 277–282. 11 2010. URL: https://www.researchgate.net/profile/Alistair_Mavin/publication/224195362_Big_Ears_The_Return_of_Easy_Approach_to_Requirements_Engineering/links/568ce39808ae197e426a075e/Big-Ears-The-Return-of-Easy-Approach-to-Requirements-Engineering.pdf, doi:10.1109/RE.2010.39.

MWGU16

Alistair Mavin, Philip Wilkinson, Sarah Gregory, and Eero Uusitalo. Listens Learned (8 Lessons Learned Applying EARS). In 24th International Requirements Engineering Conference. September 2016. URL: https://www.researchgate.net/profile/Alistair_Mavin/publication/308970788_Listens_Learned_8_Lessons_Learned_Applying_EARS/links/5ab0d42caca2721710fe5017/Listens-Learned-8-Lessons-Learned-Applying-EARS.pdf, doi:10.1109/RE.2016.38.

MWHN09

Alistair Mavin, Philip Wilkinson, Adrian Harwood, and Mark Novak. Easy approach to requirements syntax (EARS). In 17th Requirements Engineering Conference, 317–322. 10 2009. URL: https://www.researchgate.net/profile/Alistair_Mavin/publication/224079416_Easy_approach_to_requirements_syntax_EARS/links/568ce3bf08aeb488ea311990/Easy-approach-to-requirements-syntax-EARS.pdf, doi:10.1109/RE.2009.9.

Mot88

Motorola. Real Time Executive Interface Definition. Motorola Inc., Microcomputer Division and Software Components Group, Inc., January 1988. DRAFT 2.1. URL: https://ftp.rtems.org/pub/rtems/publications/RTEID-ORKID/RTEID-2.1/RTEID-2_1.pdf.

Uus12

Eero Uusitalo. EARS quick reference sheet. January 2012. URL: https://aaltodoc.aalto.fi/bitstream/handle/123456789/12861/D5_uusitalo_eero_2012.pdf.

VIT90

VITA. Open Real-Time Kernel Interface Definition. VITA, the VMEbus International Trade Association, August 1990. Draft 2.1. URL: https://ftp.rtems.org/pub/rtems/publications/RTEID-ORKID/ORKID-2.1/ORKID-2_1.pdf.

WB13

Karl Wiegers and Joy Beatty. Software Requirements. Microsoft Press, 3 edition, 2013. ISBN 0735679665, 9780735679665.