Introduction

You can use QNX Advanced Graphics to create and deploy application user interfaces. At the core of the toolkit is a technology called the QNX Graphics Framework (GF). GF provides the essential services that allow you to build interfaces that range from the smallest 2D environment to full-featured 3D environments combining 3D and 2D along with web browsers, email clients and so on. All graphic programs built with QNX Advanced Graphics use services of the QNX Graphics Framework.

This chapter introduces you to the basic terms and concepts you'll use when developing a GF application.

• Overview of the QNX Graphics Framework architecture
• Which to use? A comparison of the QNX Graphics Framework to Photon

Overview of the QNX Graphics Framework (GF) architecture

The QNX Graphics Framework (GF) is a framework for creating application user interfaces without the overhead of a full windowing system. Its architecture allows it to render directly to hardware, making it faster and more responsive in an embedded environment where resources are limited. The GF achieves this by providing direct access to graphics drivers (there's no message-passing or context-switching while rendering), and by using hardware acceleration where possible. This makes it perfect for embedded environments, where it can be used as the basic graphics layer for anything from a simple fullscreen UI to a complex windowing system for multiple application GUIs. It can also act as a porting layer for existing UIs.

For embedded 3D requirements, the GF supports the OpenGL ES 1.0 specification, a well-defined subset of OpenGL designed for embedded applications. The Neutrino implementation supports the Common profile and EGL platform interface layers.

The GF is just one of several components that provide services for graphical applications. For a typical GF application, keyboard and mouse input could be managed by some sort of input manager (for example, io-hid), font rendering could be handled by a font rendering system (such as Bitstream's Font Fusion library), and image loading could be handled by an image library (such as the Neutrino image library, which is included with QNX Advanced Graphics).

An example of GF application architecture.

While the QNX Graphics Framework is intended for fullscreen application user interfaces, it can handle multiple threads, and multiple applications, and you can even run GF/OpenGL ES applications alongside Photon if required.

The QNX Graphics Framework uses a client/server model, with the server (io-display) managing device resources (such as memory and layers) and hardware access. The client (the GF application) renders directly to the hardware. Clients obtain a lock by calling gf_draw_begin() to ensure that only one thread or application has access to the hardware at a time. The io-display process provides increased robustness by catching abnormal client termination, cleaning up resources after termination, and reviving the shared mutex if it was held by an abnormally terminated client.

All hardware access by GF applications is brokered by the monitor process, io-display. The monitor also allocates all video memory, and handles application termination (both normal and abnormal) by releasing allocated memory, and releasing locked hardware (if necessary). The monitor is “invisible” to GF clients, as all interactions are handled internally by the library. This diagram illustrates the relationship of the monitor process to GF clients:

The monitor process.

Once io-display has started, applications can use the QNX Graphics Framework library (libgf) and OpenGL ES libraries to acquire a graphics context and render to the device.

For more information about configuring the monitor process, and what components you need to distribute on target systems, see the Embedding GF Applications chapter. Also see the io-display in the Neutrino Utilities Reference.

GF components

Let's look at the conceptual components of the QNX Graphics Framework: devices, displays, layers, surfaces, and contexts.

Devices and displays are the logical entities representing hardware components managed by GF. A device is a piece of graphics hardware, which is attached to and managed by io-display via the appropriate devg-* driver. Each device has an entry in /dev/io-display.

Each device has at least one display, which represents the video display viewed by the end-user. Some devices are multi-headed, meaning they support multiple displays.

Layers represent an area of memory that's shown on a display. Some modern graphics hardware supports multiple layers, though from the GF perspective all devices have at least one layer (layer 0). Layers provide a great deal of flexibility in laying out content on a display. For example, you can put a scrolling map on a background layer, and GUI controls for scrolling the map on a front layer. The application can scroll the map smoothly without redrawing the GUI controls, eliminating flicker and reducing CPU load. In any situation where you want to display text or a GUI over top of visual information, layering can be very useful.

The relationship between a device, its displays, and their layers is illustrated in this figure:

Device, display, and layer hierarchy.

A surface is piece of memory that the GF library can render to. If the surface is connected to a layer (and the layer is visible), anything rendered on the surface appears on the layer's display. The surface must have a pixel format that is supported by the layer so that objects rendered onto the surface appear correctly. Generally, a layer targets (or is associated with) a single surface. An exception is planar data, such as planar YUV, which requires three surfaces (one for each video component).

---

Planar YUV is a video display format. The GF library doesn't support rendering to planar YUV; however, you can display planar YUV data that's already been rendered by another application.

---

In general, the GF allocates a surface in video memory, though when you create the surface, you can request that it be optimized for CPU access, which generally means it will be allocated in system RAM.

A context is a structure that maintains information about a rendering target between calls to rendering functions, such as the pen width, foreground and background colors, and clipping rectangle. A context targets a surface, and is an argument to all GF drawing functions.

This figure illustrates how contexts, surfaces, and layers are related:

Contexts, surfaces, and layers.

GF compared to Photon

Should you use Photon or GF as your UI framework? Let's compare the strengths and weaknesses of two systems: