Framework validation: performance

General rules for measuring and validating the performance of the Framework Module.

Introduction

The OSGi Framework supports different types of measurements that are used to validate application behavior on different target platforms.

Types of measurements

The following types of measurements are supported:

• Full (default) – Give detailed information about framework behavior and installed bundles during the startup.
• Intermediate – Give information related mainly to the time for installing, resolving and starting of bundles and of the framework modules.
• Simple (default) – Give information only about the framework startup.
• Memory – If is used together with 1, 2 or 3, additionally gives information about heap memory usage.

The server_common.bat (Windows) and server_common.sh (Linux) scripts must be edited in order to setup a measurement. The relevant line in the OS-specific script is as follows:

• Windows: if defined FWMEASUREMENTS set FEATURES=%FEATURES% -Dmbs.measurements.full=true
• Linux: if [ -n "$FWMEASUREMENTS" ]; then FEATURES="$FEATURES -Dmbs.measurements.full=true";

In both cases, the part "-Dmbs.measurements.full=true" must be replaced with the required measurements. Use the following properties (corresponding to the types of measurement explained above):

• -Dmbs.measurements.simple=true
• -Dmbs.measurements.intermediate=true
• -Dmbs.measurements.full=true
• -Dmbs.measurements.memory=true

In most cases, the combination "-Dmbs.measurements.intermediate=true -Dmbs.measurements.memory=true" gives enough information on the framework behavior. Note that the measurements reflect the framework behavior and we recommend to use a lower level if you do not need detailed information.

You might also use the measurements option of the server script - it will launch full OSGi Runtime measurements. It is recommended to set the output to a file, e.g. type the following command in the command prompt in the bin/vms/<vm_name> directory:

server measurements >log.txt

Validating services

When a framework validation is performed all bundles and the services thay are registering are validated. To specify a set of services to be validated or disable services validation altogether the match.service.count bundle tag tribute is used. If set to true, the validator will check if the number of services registered by the bundle is the same as the number of services described in the validation XML. If set to true, and there are no services described for the bundle, the validation will pass only if the bundle does not register any services. The default value is "false".

Setup of an Image

If you perform measurements on the target device, it is recommended that you copy only needed components. In most cases, only the following files are needed:

• bin
  • vms - domain.crp, server_common (for the corresponding OS)
    • <jvm_name> - Only scripts for the corresponding OS
• bundles - Only bundles from the boot.ini file
• lib - Only serverXxx.jar for the corresponding JVM

Modification of the configuration files might be needed.

You can use the Image Builder in Eclipse to build easily an image with the components suitable for the target platform included at the right place.

How to prepare and analyze measurements

1. Never use measurement details from the first startup of the framework as it performs additional bootstrapping operations. Execute the server script and then type exit in the local console (or kill the framework Java process if there is no access to the console).
2. Execute in a command prompt in the bin/vms/<vm_name> directory the line

   server measurements >log.txt

   It will generate a log file with the measurements.

Framework options related to performance

Upon request, it is possible to generate a server.jar without including functionality, which is not needed for the corresponding project. For example, features like security, transactions, fault manager, debugs, tracer can be removed and it will lead to reducing startup time and size of the JAR files.

If you are using the Image Builder to build images, you can easily configure the performance features from the Framework Options section of the image description.

Time for execution of the test cases

HTML logs from the TEE contain detailed time information that can be used for performance evaluation on the target device. Generated result can be compared to other devices or PCs.

Measurement of temporary memory

Notice that the number of temporary objects (objects that are used temporarily only and are quickly collected by the garbage collector) is relevant to the startup time. The temporary object count determines the workload of the garbage collector which usually consumes CPU time that could otherwise be used for application startup. The number of temporary objects and their influence on the performance can be measured by using the mBProfiler tool during the system startup. In this case, it is recommended to use at least the second start of the framework.