We'd all like to split heavy work into a separate thread, but how can we do it with existing code? We take an existing OCR library and run it in one thread of a GUI OCR application.
There’s a lot of talk these days about multicore technology and multithreaded development. The trend toward multiple compute cores is growing and having a profound impact on developers across various markets. Clearly, threading is no longer the exclusive domain of server-application developers. For instance, desktop PCs ship with four cores and even phones have dual-core CPUs these days.
As a result, multithreading is particularly beneficial for applications that have a lot of complicated interoperating control tasks that switch back and forth on a regular basis. Multithreaded programming is also a useful paradigm for performing time-consuming operations without freezing the user interface of an application.
The benefits of multicore technology and multithreading to maximize advancements in hardware is clear. However, getting there is no simple or easy task, unless of course you are using Trolltech’s Qt®, the standard framework for high-performance, cross-platform application development. Moving to a multicore world means that application developers will have to write programs differently and that’s what Qt is about.
Making multithreading easy
Qt provides thread support in the form of platform-independent threading classes, a thread-safe way of posting events, and signal-slot connections across threads. This makes it easy to develop portable multithreaded Qt applications and take advantage of multiprocessor machines.
Earlier versions of Qt offered an option to build the library without thread support. Since Qt 4 (latest generation of Qt released in June 2005), threads are always enabled.
Qt includes the following thread classes:
1) QThread provides the means to start a new thread.
2) QThreadStorage provides per-thread data storage.
3) QMutex provides a mutual exclusion lock, or mutex.
4) QMutexLocker is a convenience class that automatically locks and unlocks a QMutex.
5) QReadWriteLock provides a lock that allows simultaneous read access.
6) QReadLocker and QWriteLocker are convenience classes that automatically lock and unlock a QReadWriteLock.
7) QSemaphore provides an integer semaphore.
8) QWaitCondition provides a way for threads to go to sleep until woken up by another thread.
To create a thread, subclass QThread and reimplement its run() function. For example:
class MyThread : public QThread
{
Q_OBJECT
protected:
void run();
};
void MyThread::run()
{
…
}
Then, create an instance of the thread object and call QThread::start(). The code that appears in the run() reimplementation will then be executed in a separate thread. Creating threads is explained in more detail in the QThread documentation.
Note that QCoreApplication::exec() must always be called from the main thread (the thread that executes main()), not from a QThread. In GUI applications, the main thread is also called the GUI thread because it’s the only thread that is allowed to interact with the graphics server.
In addition, you must create the QApplication (or QCoreApplication) object before you can create a QThread.
Synchronizing threads
The QMutex, QReadWriteLock, QSemaphore and QWaitCondition classes provide means to synchronize threads. While the main idea with threads is that they should be as concurrent as possible, there are points where threads must stop and wait for other threads. For example, if two threads try to write to the same global variable simultaneously, the result is usually an application crash.
QMutex provides a mutually exclusive lock, or mutex. At most one thread can hold the mutex at any time. If a thread tries to acquire the mutex while the mutex already locked, the thread will be put to sleep until the thread that current holds the mutex unlocks it. Mutexes are often used to protect accesses to shared data (i.e., data that can be accessed from multiple threads simultaneously). In the reentrancy and thread-safety section below, we will use it to make a class thread-safe.
QReadWriteLock is similar to QMutex, except that it distinguishes between “read” and “write” access to shared data and allows multiple readers to access the data simultaneously. Using QReadWriteLock instead of QMutex when it is possible can make multithreaded programs more concurrent.
QSemaphore is a generalization of QMutex that protects a certain number of identical resources. In contrast, a mutex protects exactly one resource. The Semaphores example shows a typical application of semaphores: synchronizing access to a circular buffer between a producer and a consumer.
QWaitCondition allows a thread to wake up other threads when some condition has been met. One or many threads can block waiting for a QWaitCondition to set a condition with wakeOne() or wakeAll(). Use wakeOne() to wake one randomly selected event or wakeAll() to wake them all. The Wait Conditions example shows how to solve the producer-consumer problem using QWaitCondition instead of QSemaphore.
Reentrancy and thread safety
Throughout the Qt documentation, the terms reentrant and thread-safe are used to specify how a function can be used in multithreaded applications:
A reentrant function can be called simultaneously by multiple threads provided that each invocation of the function references unique data.
A thread-safe function can be called simultaneously by multiple threads when each invocation references shared data. All access to the shared data is serialized.
By extension, a class is said to be reentrant if any of its functions can be called simultaneously by multiple threads on different instances of the class, and thread-safe if it even works if the different threads operate on the same instance.
Note that the terminology in this domain isn’t entirely standardized. POSIX uses a somewhat different definition of reentrancy and thread-safety for its C APIs. When dealing with an object-oriented C++ class library such as Qt, the definitions must be adapted.
Most C++ classes are inherently reentrant, since they typically only reference member data. Any thread can call such a member function on an instance of the class, as long as no other thread is calling a member function on the same instance. For example, the Counter class below is reentrant:
class Counter
{
public:
Counter() { n = 0; }
void increment() { ++n; }
void decrement() { –n; }
int value() const { return n; }
private:
int n;
};
The class isn’t thread-safe, because if multiple threads try to modify the data member n, the result is undefined. This is because C++’s ++ and – operators aren’t necessarily atomic. Indeed, they usually expand to three machine instructions:
Load the variable’s value in a register.
Increment or decrement the register’s value.
Store the register’s value back into main memory.
If thread A and thread B load the variable’s old value simultaneously, increment their register, and store it back, they end up overwriting each other, and the variable is incremented only once!
Clearly, the access must be serialized: Thread A must perform steps 1, 2, 3 without interruption (atomically) before thread B can perform the same steps; or vice versa. An easy way to make the class thread-safe is to protect all access to the data members with a QMutex:
class Counter
{
public:
Counter() { n = 0; }
void increment() { QMutexLocker locker(&mutex); ++n; }
void decrement() { QMutexLocker locker(&mutex); –n; }
int value() const { QMutexLocker locker(&mutex); return n; }
private:
mutable QMutex mutex;
int n;
};
The QMutexLocker class automatically locks the mutex in its constructor and unlocks it when the destructor is invoked, at the end of the function. Locking the mutex ensures that access from different threads will be serialized. The mutex data member is declared with the mutable qualifier because we need to lock and unlock the mutex in value(), which is a const function.
Most Qt classes are reentrant and not thread-safe, to avoid the overhead of repeatedly locking and unlocking a QMutex. For example, QString is reentrant, meaning that you can use it in different threads, but you can’t access the same QString object from different threads simultaneously (unless you protect it with a mutex yourself). A few classes and functions are thread-safe; these are mainly thread-related classes such as QMutex, or fundamental functions such as QCoreApplication::postEvent().
Threads and QObjects
QThread inherits QObject. It emits signals to indicate that the thread started or finished executing, and provides a few slots as well.
More interesting is that QObjects can be used in multiple threads, can emit signals that invoke slots in other threads and can post events to objects that “live” in other threads. This is possible because each thread is allowed to have its own event loop.
QObject is reentrant. Most of its subclasses — such as QTimer, QTcpSocket, QUdpSocket, QHttp, QFtp and QProcess — are also reentrant, making it possible to use these classes from multiple threads simultaneously. Note that these classes are designed to be created and used from within a single thread; creating an object in one thread and calling its functions from another thread is not guaranteed to work. There are three constraints to be aware of:
1) The child of a QObject must always be created in the thread where the parent was created. This implies, among other things, that you should never pass the QThread object (this) as the parent of an object created in the thread (since the QThread object itself was created in another thread).
2) Event driven objects may only be used in a single thread. Specifically, this applies to the timer mechanism and the network module. For example, you cannot start a timer or connect a socket in a thread that is not the object’s thread.
3) You must ensure that all objects created in a thread are deleted before you delete the QThread. This can be done easily by creating the objects on the stack in your run() implementation.
Although QObject is reentrant, the widget classes, notably QWidget and all its subclasses, are not reentrant. They can be used only from the main thread. As noted earlier, QCoreApplication::exec() must also be called from that thread.
In practice, the impossibility of using GUI classes in other threads than the main thread can easily be worked around by putting time-consuming operations in a separate worker thread and displaying the results on screen in the main thread when the worker thread is finished. This is the approach used for implementing the Mandelbrot and the Blocking Fortune Client example.
Per-thread event loop
Each thread can have its own event loop. The initial thread starts its event loops using QCoreApplication::exec(); other threads can start an event loop using QThread::exec(). Like QCoreApplication, QThread provides an exit(int) function and a quit() slot.
An event loop in a thread makes it possible for the thread to use certain Qt classes that require the presence of an event loop (such as QTimer, QTcpSocket, and QProcess). It also makes it possible to connect signals from any threads to slots of a specific thread. This is explained in more detail in the “Signals and Slots Across Threads” section below.
A QObject instance is said to live in the thread in which it is created. Events to that object are dispatched by that thread’s event loop. The thread in which a QObject lives is available using QObject::thread().
Note that for QObjects that are created before QApplication, QObject::thread() returns zero. This means that the main thread will only handle posted events for these objects; other event processing is not done at all for objects with no thread. Use the QObject::moveToThread() function to change the thread affinity for an object and its children (the object cannot be moved if it has a parent).
Calling delete on a QObject from a thread other than the one where it is created (or accessing the object in other ways) is unsafe, unless you can guarantee that the object isn’t processing events at the same moment. Use QObject::deleteLater() instead; it will post a DeferredDelete event, which the event loop of the object’s thread will eventually pick up.
If no event loop is running, events won’t be delivered to the object. For example, if you create a QTimer object in a thread but never call exec(), the QTimer will never emit its timeout() signal. Calling deleteLater() won’t work either. (These restrictions apply to the main thread as well.)
You can manually post events to any object in any thread at any time using the thread-safe function QCoreApplication::postEvent(). The events will automatically be dispatched by the event loop of the thread where the object was created.
Event filters are supported in all threads, with the restriction that the monitoring object must live in the same thread as the monitored object. Similarly, QCoreApplication::sendEvent() (unlike postEvent()) can only be used to dispatch events to objects living in the thread from which the function is called.
Signals and slots across threads
The straightforward mechanisms for multithreaded programming provided in the Qt framework, include the high-level abstraction for inter-object communication called signals-and-slots.
Qt supports four types of signal-slot connections:
1) With direct connections, the slot gets called immediately when the signal is emitted. The slot is executed in the thread that emitted the signal (which is not necessarily the thread where the receiver object lives).
2) With queued connections, the slot is invoked when control returns to the event loop of the thread to which the object belongs. The slot is executed in the thread where the receiver object lives.
3) With blocking queued connections, the slot is invoked just like with normal queued connections, but the current thread blocks until the slot was delivered.
4) With auto connections (the default), the behavior is the same as with direct connections if the signal is emitted in the thread where the receiver lives; otherwise, the behavior is that of a queued connection.
The connection type can be specified by passing an additional argument to connect(). Be aware that using direct connections when the sender and receiver live in different threads is unsafe, for the same reason that calling any function on an object living in another thread is unsafe.
QObject::connect() itself is thread-safe.
Threads and implicit sharing
Qt uses an optimization called implicit sharing for many of its value class, notably QImage and QString. In many people’s minds, implicit sharing and multithreading are incompatible concepts, because of the way the reference counting is typically done. One solution is to protect the internal reference counter with a mutex, but this is prohibitively slow. Earlier versions of Qt didn’t provide a satisfactory solution to this problem.
Beginning with Qt 4, implicit shared classes can safely be copied across threads, like any other value classes. They are fully reentrant. The implicit sharing is really implicit. This is implemented using atomic reference counting operations, which are implemented in assembly language for the different platforms supported by Qt. Atomic reference counting is very fast, much faster than using a mutex (see also Implementing atomic operations).
This having been said, if you access the same object in multiple threads simultaneously (as opposed to copies of the same object), you still need a mutex to serialize the accesses, just like with any reentrant class.
To sum up, implicitly shared classes in Qt 4 are really implicitly shared. Even in multithreaded applications, you can safely use them as if they were plain, non-shared, reentrant classes.
Example: thread-safe OCR
As an example for threaded GUI code, we’re creating a multithreaded OCR application using the Tesseract OCR engine. OCR can be time consuming and we want to prevent that the UI is not responsive during the calculations. Using Qt’s signal and slots, it is easy to create an application without worrying about mutexes or locking. Qt does it all for us.
Our application has two classes. OcrViewer is the main GUI window that displays the image, OcrThread is the class that invokes the Tesseract OCR engine. OcrThread has one signal, “ocrFinished”, which is emitted when the OCR operation is done.
OcrViewer creates the GUI and has one instance of OcrThread as member. When the user starts the OCR operation, the current image is passed to the thread class and the thread is started. The thread creates a copy of the image, but since QImage is implicitely shared, no actual image data
is copied and the operation is very fast. Note that the “startJob()” function is still run within the main (GUI) thread. Once the thread’s “start()” function is invoked, a new thread is started and the “run()” function is called. “run()” runs within the new thread, while the main (GUI) thread continues in parallel. The user can move or resize the main window and the application will still be responsive. While Tesseract is computing, a progress bar will move and the user can select the next image for OCRing.
At the end of the “run()” function, we use QMetaObject::invokeMethod() to signal the end of the OCR operation. We use a queued signal to make sure the signal emission is thread safe. We pass the result of the OCR operation as parameter to the signal. Again, Qt’s signal and slot mechanism makes sure that the result is not deep-copied, but safely delivered to the GUI thread.
What’s next for Qt?
A research project that is a strong candidate for becoming part of Qt is Qt Concurrent, a C++ template library for writing multithreaded applications.
Qt Concurrent provides high-level APIs that makes it possible to write multithreaded programs without using low-level threading primitives such as critical sections, mutexes or wait conditions.
Programs written with Qt Concurrent automatically adjust the number of threads used according to the number of processor cores available. This means that applications written today will continue to scale when deployed on multicore systems in the future.
The library includes functional programming style APIs for parallel list processing, a MapReduce implementation for shared-memory (non-distributed) systems, and classes for managing asynchronous computations in GUI applications.
Harald Fernengel is a professional services engineer for Trolltech GmbH, the developers of Qt.
