In this article from the book Game Programming using Qt by authors Witold Wysota and Lorenz Haas, you will be taught how to communicate with the Internet servers and with sockets in general. First, we will have a look at QNetworkAccessManager, which makes sending network requests and receiving replies really easy. Building on this basic knowledge, we will then use Google’s Distance API to get information about the distance between two locations and the time it would take to get from one location to the other.
(For more resources related to this topic, see here.)
QNetworkAccessManager
The easiest way to access files on the Internet is to use Qt’s Network Access API. This API is centered on QNetworkAccessManager, which handles the complete communication between your game and the Internet.
When we develop and test a network-enabled application, it is recommended that you use a private, local network if feasible. This way, it is possible to debug both ends of the connection and the errors will not expose sensitive data. If you are not familiar with setting up a web server locally on your machine, there are luckily a number of all-in-one installers that are freely available. These will automatically configure Apache2, MySQL, PHP, and much more on your system. On Windows, for example, you could use XAMPP (http://www.apachefriends.org/en) or the Uniform Server (http://www.uniformserver.com); on Apple computers there is MAMP (http://www.mamp.info/en); and on Linux, you normally don’t have to do anything since there is already localhost. If not, open your preferred package manager, search for a package called apache2 or similar, and install it. Alternatively, have a look at your distribution’s documentation.
Before you go and install Apache on your machine, think about using a virtual machine like VirtualBox (http://www.virtualbox.org) for this task. This way, you keep your machine clean and you can easily try different settings of your test server. With multiple virtual machines, you can even test the interaction between different instances of your game. If you are on UNIX, Docker (http://www.docker.com) might be worth to have a look at too.
Downloading files over HTTP
For downloading files over HTTP, first set up a local server and create a file called version.txt in the root directory of the installed server. The file should contain a small text like “I am a file on localhost” or something similar. To test whether the server and the file are correctly set up, start a web browser and open http://localhost/version.txt. You then should see the file’s content. Of course, if you have access to a domain, you can also use that. Just alter the URL used in the example correspondingly. If you fail, it may be the case that your server does not allow to display text files. Instead of getting lost in the server’s configuration, just rename the file to version .html. This should do the trick!
Result of requesting http://localhost/version.txt on a browser
As you might have guessed, because of the filename, the real-life scenario could be to check whether there is an updated version of your game or application on the server. To get the content of a file, only five lines of code are needed.
Time for action – downloading a file
First, create an instance of QNetworkAccessManager:
QNetworkAccessManager *m_nam = new QNetworkAccessManager(this);Since QNetworkAccessManager inherits QObject, it takes a pointer to QObject, which is used as a parent. Thus, you do not have to take care of deleting the manager later on. Furthermore, one single instance of QNetworkAccessManager is enough for an entire application. So, either pass a pointer to the network access manager in your game around or, for ease of use, create a singleton pattern and access the manager through that.
A singleton pattern ensures that a class is instantiated exactly once. The pattern is useful for accessing application-wide configurations or—in our case—an instance of QNetworkAccessManager. On the wiki pages for qtcentre.org and qt-project.org, you will find examples for different singleton patterns. A simple template-based approach would look like this (as a header file):
template <class T> class Singleton { public: static T& Instance() { static T _instance; return _instance; } private: Singleton(); ~Singleton(); Singleton(const Singleton &); Singleton& operator=(const Singleton &); };In the source code, you would include this header file and acquire a singleton of a class called MyClass with:
MyClass *singleton = &Singleton<MyClass>::Instance();If you are using Qt Quick, you can directly use the view instance of QNetworkAccessManager:
QQuickView *view = new QQuickView; QNetworkAccessManager *m_nam = view->engine()->networkAccessManager();
Secondly, we connect the manager’s finished() signal to a slot of our choice. For example, in our class, we have a slot called downloadFinished():
connect(m_nam, SIGNAL(finished(QNetworkReply*)), this, SLOT(downloadFinished(QNetworkReply*)));Then, it actually request’s the version.txt file from localhost:
m_nam->get(QNetworkRequest(QUrl("http://localhost/version.txt")));With get(), a request to get the contents of the file, specified by the URL, is posted. The function expects QNetworkRequest, which defines all the information needed to send a request over the network. The main information of such a request is naturally the URL of the file. This is the reason why QNetworkRequest takes a QUrl as an argument in its constructor. You can also set the URL with setUrl() to a request. If you like to define some additional headers, you can either use setHeader() for the most common header or use setRawHeader() to be fully flexible. If you want to set, for example, a custom user agent to the request, the call would look like:
QNetworkRequest request;
request.setUrl(QUrl("http://localhost/version.txt"));
request.setHeader(QNetworkRequest::UserAgentHeader, "MyGame");
m_nam->get(request);The setHeader() function takes two arguments, the first is a value of the enumeration QNetworkRequest::KnownHeaders, which holds the most common—self-explanatory—headers such as LastModifiedHeader or ContentTypeHeader, and the second is the actual value. You could also have written the header by using of setRawHeader():
request.setRawHeader("User-Agent", "MyGame");When you use setRawHeader(), you have to write the header field names yourself. Beside that, it behaves like setHeader().
A list of all available headers for the HTTP protocol Version 1.1 can be found in section 14 at http://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html#sec14.
With the get() function we requested the version.txt file from localhost. All we have to do from now on is to wait for the server to reply. As soon as the server’s reply is finished, the slot downloadFinished() will be called. That was defined by the previous connection statement. As an argument the reply of type QNetworkReply is transferred to the slot and we can read the reply’s data and set it to m_edit, an instance of QPlainTextEdit, using the following code:
void FileDownload::downloadFinished(QNetworkReply *reply) {
const QByteArray content = reply->readAll();
m_edit->setPlainText(content);
reply->deleteLater();
}Since QNetworkReply inherits QIODevice, there are also other possibilities to read the contents of the reply including QDataStream or QTextStream to either read and interpret binary data or textual data. Here, as fourth command, QIODevice::readAll() is used to get the complete content of the requested file in a QByteArray. The responsibility for the transferred pointer to the corresponding QNetworkReply lies with us, so we need to delete it at the end of the slot. This would be the fifth line of code needed to download a file with Qt. However, be careful and do not call delete on the reply directly. Always use deleteLater() as the documentation suggests!
Have a go hero – extending the basic file downloader
If you haven’t set up a localhost, just alter the URL in the source code to download another file. Of course, having to alter the source code in order to download another file is far from an ideal approach. So try to extend the dialog, by adding a line edit where you can specify the URL you want to download. Also, you can offer a file dialog to choose the location to where the downloaded file should be saved.
Error handling
If you do not see the content of the file, something went wrong. Just as in real life, this can always happen so we better make sure, that there is good error handling in such cases to inform the user what is going on.
Time for action – displaying a proper error message
Fortunately QNetworkReply offers several possibilities to do this. In the slot called downloadFinished() we first want to check if an error occurred:
if (reply->error() != QNetworkReply::NoError)
{/* error occurred */}The function QNetworkReply::error() returns the error that occurred while handling the request. The error is encoded as a value of type QNetworkReply::NetworkError. The two most common errors are probably these:
|
Error code |
Meaning |
|
ContentNotFoundError |
This error indicates that the URL of the request could not be found. It is similar to the HTTP error code 404. |
|
ContentAccessDenied |
This error indicates that you do not have the permission to access the requested file. It is similar to the HTTP error 401. |
You can look up the other 23 error codes in the documentation. But normally you do not need to know exactly what went wrong. You only need to know if everything worked out—QNetworkReply::NoError would be the return value in this case—or if something went wrong.
Since QNetworkReply::NoError has the value 0, you can shorten the test phrase to check if an error occurred to:
if (reply->error()) { // an error occurred }
To provide the user with a meaningful error description you can use QIODevice::errorString(). The text is already set up with the corresponding error message and we only have to display it:
if (reply->error()) {
const QString error = reply->errorString();
m_edit->setPlainText(error);
return;
}In our example, assuming we had an error in the URL and wrote versions.txt by mistake, the application would look like this:
If the request was a HTTP request and the status code is of interest, it could be retrieved by QNetworkReply::attribute():
reply->attribute(QNetworkRequest::HttpStatusCodeAttribute)Since it returns QVariant, you can either use QVariant::toInt() to get the code as an integer or QVariant::toString() to get the number as a QString. Beside the HTTP status code you can query through attribute() a lot of other information. Have a look at the description of the enumeration QNetworkRequest::Attribute in the documentation. There you also will find QNetworkRequest::HttpReasonPhraseAttribute which holds a human readable reason phrase of the HTTP status code. For example “Not Found” if an HTTP error 404 occurred. The value of this attribute is used to set the error text for QIODevice::errorString(). So you can either use the default error description provided by errorString() or compose your own by interpreting the reply’s attributes.
If a download failed and you want to resume it or if you only want to download a specific part of a file, you can use the range header:
QNetworkRequest req(QUrl("...")); req.setRawHeader("Range", "bytes=300-500"); QNetworkReply *reply = m_nam->get(req);In this example only the bytes 300 to 500 would be downloaded. However, the server must support this.
Downloading files over FTP
As simple as it is to download files over HTTP, as simple it is to download a file over FTP. If it is an anonymous FTP server for which you do not need an authentication, just use the URL like we did earlier. Assuming there is again a file called version.txt on the FTP server on localhost, type:
m_nam->get(QNetworkRequest(QUrl("ftp://localhost/version.txt")));That is all, everything else stays the same. If the FTP server requires an authentication you’ll get an error, for example:
Setting the user name and the user password to access an FTP server is likewise easy. Either write it in the URL or use QUrl functions setUserName() and setPassword(). If the server does not use a standard port, you can set the port explicitly with QUrl::setPort().
To upload a file to a FTP server use QNetworkAccessManager::put() which takes as first argument a QNetworkRequest, calling a URL that defines the name of the new file on the server, and as second argument the actual data, that should be uploaded. For small uploads, you can pass the content as a QByteArray. For larger contents, better use a pointer to a QIODevice. Make sure the device is open and stays available until the upload is done.
Downloading files in parallel
A very important note on QNetworkAccessManager: it works asynchronously. This means you can post a network request without blocking the main event loop and this is what keeps the GUI responsive. If you post more than one request, they are put on the manager’s queue. Depending on the protocol used they get processed in parallel. If you are sending HTTP requests, normally up to six requests will be handled at a time. This will not block the application. Therefore, there is really no need to encapsulate QNetworkAccessManager in a thread, unfortunately, this unnecessary approach is frequently recommended all over the Internet. QNetworkAccessManager already threads internally. Really, don’t move QNetworkAccessManager to a thread—unless you know exactly what you are doing.
If you send multiple requests, the slot connected to the manager’s finished() signal is called in an arbitrary order depending on how quickly a request gets a reply from the server. This is why you need to know to which request a reply belongs. This is one reason why every QNetworkReply carries its related QNetworkRequest. It can be accessed through QNetworkReply::request().
Even if the determination of the replies and their purpose may work for a small application in a single slot, it will quickly get large and confusing if you send a lot of requests. This problem is aggravated by the fact that all replies are delivered to only one slot. Since most probably there are different types of replies that need different treatments, it would be better to bundle them to specific slots, specialized for a special task. Fortunately this can be achieved very easily. QNetworkAccessManager::get() returns a pointer to the QNetworkReply which will get all information about the request you post with get(). By using this pointer, you can then connect specific slots to the reply’s signals.
For example if you have several URLs and you want to save all linked images from these sites to the hard drive, then you would request all web pages via QNetworkAccessManager::get() and connect their replies to a slot specialized for parsing the received HTML. If links to images are found, this slot would request them again with get(). However, this time the replies to these requests would be connected to a second slot, which is designed for saving the images to the disk. Thus you can separate the two tasks, parsing HTML and saving data to a local drive.
The most important signals of QNetworkReply are.
The finished signal
The finished() signal is equivalent with the QNetworkAccessManager::finished() signal we used earlier. It is triggered as soon as a reply has been returned—successfully or not. After this signal has been emitted, neither the reply’s data nor its metadata will be altered anymore. With this signal you are now able to connect a reply to a specific slot. This way you can realize the scenario outlined previously.
However, one problem remains: if you post simultaneous requests, you do not know which one has finished and thus called the connected slot. Unlike QNetworkAccessManager::finished(), QNetworkReply::finished() does not pass a pointer to QNetworkReply; this would actually be a pointer to itself in this case. A quick solution to solve this problem is to use sender(). It returns a pointer to the QObject instance that has called the slot. Since we know that it was a QNetworkReply, we can write:
QNetworkReply *reply = qobject_cast<QNetworkReply*>(sender());
if (!reply)
return;This was done by casting sender() to a pointer of type QNetworkReply.
Whenever casting classes that inherit QObject, use qobject_cast. Unlike dynamic_cast it does not use RTTI and works across dynamic library boundaries.
Although we can be pretty confident the cast will work, do not forget to check if the pointer is valid. If it is a null pointer, exit the slot.
Time for action – writing OOP conform code by using QSignalMapper
A more elegant way that does not rely on sender(), would be to use QSignalMapper and a local hash, in which all replies that are connected to that slot are stored. So whenever you call QNetworkAccessManager::get() store the returned pointer in a member variable of type QHash<int, QNetworkReply*> and set up the mapper. Let’s assume that we have following member variables and that they are set up properly:
QNetworkAccessManager *m_nam;
QSignalMapper *m_mapper;
QHash<int, QNetworkReply*> m_replies;Then you would connect the finished() signal of a reply this way:
QNetworkReply *reply = m_nam->get(QNetworkRequest(QUrl(/*...*/)));
connect(reply, SIGNAL(finished()), m_mapper, SLOT(map()));
int id = /* unique id, not already used in m_replies*/;
m_replies.insert(id, reply);
m_mapper->setMapping(reply, id);What just happened?
First we post the request and fetch the pointer to the QNetworkReply with reply. Then we connect the reply’s finished signal to the mapper’s slot map(). Next we have to find a unique ID which must not already be in use in the m_replies variable. One could use random numbers generated with qrand() and fetch numbers as long as they are not unique. To determine if a key is already in use, call QHash::contains(). It takes the key as an argument against which it should be checked. Or even simpler: count up another private member variable. Once we have a unique ID we insert the pointer to QNetworkReply in the hash using the ID as a key. Last, with setMapping(), we set up the mapper’s mapping: the ID’s value corresponds to the actual reply.
At a prominent place, most likely the constructor of the class, we already have connected the mappers map() signal to a custom slot. For example:
connect(m_mapper, SIGNAL(mapped(int)), this,
SLOT(downloadFinished(int)));When the slot downloadFinished() is called, we can get the corresponding reply with:
void SomeClass::downloadFinished(int id) {
QNetworkReply *reply = m_replies.take(id);
// do some stuff with reply here
reply->deleteLater();
}QSignalMapper also allows to map with QString as an identifier instead of an integer as used above. So you could rewrite the example and use the URL to identify the corresponding QNetworkReply; at least as long as the URLs are unique.
The error signal
If you download files sequentially, you can swap the error handling out. Instead of dealing with errors in the slot connected to the finished() signal, you can use the reply’s signal error() which passes the error of type QNetworkReply::NetworkError to the slot. After the error() signal has been emitted, the finished() signal will most likely also be emitted shortly.
The readyRead signal
Until now, we used the slot connected to the finished() signal to get the reply’s content. That works perfectly if you deal with small files. However, this approach is unsuitable when dealing with large files since they would unnecessarily bind too many resources. For larger files it is better to read and save transferred data as soon as it is available. We get informed by QIODevice::readyRead() whenever new data is available to be read. So for large files you should type in the following:
connect(reply, SIGNAL(readyRead()), this, SLOT(readContent()));
file.open(QIODevice::WriteOnly);This will help you connect the reply’s signal readyRead() to a slot, set up QFile and open it. In the connected slot, type in the following snippet:
const QByteArray ba = reply->readAll();
file.write(ba);
file.flush();Now you can fetch the content, which was transferred so far, and save it to the (already opened) file. This way the needed resources are minimized. Don’t forget to close the file after the finished() signal was emitted.
In this context it would be helpful if one could know upfront the size of the file one wants to download. Therefore, we can use QNetworkAccessManager::head(). It behaves like the get() function, but does not transfer the content of the file. Only the headers are transferred. And if we are lucky, the server sends the “Content-Length” header, which holds the file size in bytes. To get that information we type:
reply->head(QNetworkRequest::ContentLengthHeader).toInt();With this information, we could also check upfront if there is enough space left on the disk.
The downloadProgress method
Especially when a big file is being downloaded, the user usually wants to know how much data has already been downloaded and how long it will approximately take for the download to finish.
Time for action – showing the download progress
In order to achieve this we can use the reply’s downloadProgress() signal. As a first argument it passes the information on how many bytes have already been received and as a second argument how many there are in total. This gives us the possibility to indicate the progress of the download with QProgressBar. As the passed arguments are of type qint64 we can’t use them directly with QProgressBar since it only accepts int. So in the connected slot we first calculate the percentage of the download progress:
void SomeClass::downloadProgress(qint64 bytesReceived,
qint64 bytesTotal) {
qreal progress = (bytesTotal < 1) ? 1.0
: bytesReceived * 100.0 / bytesTotal;
progressBar->setValue(progress * progressBar->maximum());
}What just happened?
With the percentage we set the new value for the progress bar where progressBar is the pointer to this bar. However, what value will progressBar->maximum() have and where do we set the range for the progress bar? What is nice is that you do not have to set it for every new download. It is only done once, for example in the constructor of the class containing the bar. As range values I would recommend:
progressBar->setRange(0, 2048);The reason is that if you take for example a range of 0 to 100 and the progress bar is 500 pixels wide, the bar would jump 5 pixels forward for every value change. This will look ugly. To get a smooth progression where the bar expands by 1 pixel at a time, a range of 0 to 99.999.999 would surely work but would be highly inefficient. This is because the current value of the bar would change a lot without any graphical depiction. So the best value for the range would be 0 to the actual bar’s width in pixel. Unfortunately, the width of the bar can change depending on the actual widget width and frequently querying the actual size of the bar every time the value change is also not a good solution. Why 2048, then? The idea behind this value is the resolution of the screen. Full HD monitors normally have a width of 1920 pixels, thus taking 2^11, aka 2048, ensure that a progress bar runs smoothly, even if it is fully expanded. So 2048 isn’t the perfect number but a fairly good compromise. If you are targeting smaller devices, choose a smaller, more appropriate number.
To be able to calculate the remaining time for the download to finish you have to start a timer. In this case use QElapsedTimer. After posting the request with QNetworkAccessManager::get() start the timer by calling QElapsedTimer::start(). Assuming the timer is called m_timer, the calculation would be:
qint64 total = m_timer.elapsed() / progress;
qint64 remaining = (total – m_timer.elapsed()) / 1000;QElapsedTimer::elapsed() returns the milliseconds counting from the moment when the timer was started. This value divided by the progress equals the estimated total download time. If you subtract the elapsed time and divide the result by 1000, you’ll get the remaining time in seconds.
Using a proxy
If you like to use a proxy you first have to set up a QNetworkProxy. You have to define the type of the proxy with setType(). As arguments you most likely want to pass QNetworkProxy::Socks5Proxy or QNetworkProxy::HttpProxy. Then set up the host name with setHostName(), the user name with setUserName() and the password with setPassword(). The last two properties are, of course, only needed if the proxy requires an authentication. Once the proxy is set up you can set it to the access manager via QNetworkAccessManager::setProxy(). Now, all new requests will use that proxy.
Summary
In this article you familiarized yourself with QNetworkAccessManager. This class is at the heart of your code whenever you want to download or upload files to the Internet. After having gone through the different signals that you can use to fetch errors, to get notified about new data or to show the progress, you should now know everything you need on that topic.
Resources for Article:
Further resources on this subject:
- GUI Components in Qt 5[article]
- Code interlude – signals and slots [article]
- Configuring Your Operating System [article]
