REST API Development with Qt 6
This post describes an experiment using Qt 6.7’s REST APIs to explore Stripe’s payment model, and what I learned building a small desktop developer tool.
Recent Qt releases have included several conveniences for developing clients of remote REST APIs. I recently tried it out with the Stripe payments REST API to get to grips with the Qt REST API in the real world. The overloading of the term API is unhelpful, I find, but hopefully not too confusing here.
As with almost everything I try out, I created Qt desktop tooling as a developer aid to exploring the Stripe API and its behavior. Naming things is hard, but given that I want to put a “Q” in the name, googling “cute stripes” gives lots of hits about fashion, and the other too-obvious-to-say pun, I’ve pushed it to GitHub as “Qashmere“:
setAlternatingRowColors(true);
Developers using REST APIs will generally be familiar with existing tooling such as Postman and Bruno, for synthesizing calls to collections of REST APIs. Indeed, Qashmere uses the Stripe Postman JSON definition to present the collection of APIs and parameters. Such tools have scripting interfaces and state to create workflows that a client of the REST API needs to support, like “create a payment, get the id of the payment back from the REST API and then cancel the payment with the id”, or “create a payment, get the id of the payment back from the REST API and then confirm it by id with a given credit card”.
So why create Qashmere? In addition to REST APIs, Stripe maintains objects which change state over time. The objects remain at REST until acted on by an external force, and when such an action happens a notification is sent to clients about those state changes, giving them a chance to react. I wanted to be able to collect the REST requests/responses and the notified events and present them as they relate to the Stripe objects. Postman doesn’t know about events or about Stripe objects in particular, except that it is possible to write a script in Postman to extract the object which is part of a JSON payload. Postman also doesn’t know that if a Payment Intent is created, there are a subset of next steps which could be in a workflow, such as cancel, capture or confirm payment etc.
Something that I discovered in the course of trying this out is that when I confirm a Payment Intent, a new Charge object is created and sent to me with the event notification system. Experimental experiences like that help build intuition.
Stripe operates with real money, but it also provides for sandboxes where synthetic payments, customers etc can be created and processed with synthetic payment methods and cards. As Qashmere is only useful as a developer tool or learning aid, it only works with Stripe sandboxes.
Events from Stripe are sent to pre-configured web servers owned by the client. The web servers need to have a public IP address, which is obviously not appropriate for a desktop application. A WebSocket API would be more suitable and indeed the stripe cli tool uses a WebSocket to receive events, but the WebSocket protocol is not documented or stable. Luckily the stripe cli tool can be used to relay events to another HTTP server, so Qashmere runs a QHttpServer for that purpose.
Implementation with Qt REST API
The QRestReply wraps a QNetworkReply pointer and provides convenience API for accessing the HTTP return code and for creating a QJsonDocument from the body of the response. It must be created manually if using QNetworkAccessManager directly. However the new QRestAccessManager wraps a QNetworkAccessManager pointer, again to provide convenience APIs and overloads for making requests that are needed in REST APIs (though some less common verbs like OPTIONS and TRACE are not built-in). The QRestAccessManager has conveniences like overloads that provide a way to supply callbacks which already take the QRestReply wrapper object as a parameter. If using a QJsonDocument request overload, the “application/json” Content-Type is automatically set in the header.
One of the inconveniences of QRestAccessManager is that in Qashmere I use an external definition of the REST API from the Postman definition which includes the HTTP method. Because the QRestAccessManager provides strongly typed API for making requests I need to do something like:
if (method == "POST") { rest.post(request, requestData, this, replyHandler);} else if (method == "GET") { rest.get(request, this, replyHandler);} else if (method == "DELETE") { rest.deleteResource(request, this, replyHandler);}There is a sendCustomRequest class API which can be used with a string, but it does not have an overload for QJsonDocument, so the convenience of having the Content-Type header set is lost. This may be an oversight in the QRestAccessManager API.
Another missing feature is URL parameter interpolation. Many REST APIs are described as something like /v1/object/:object_id/cancel, and it would be convenient to have a safe way to interpolate the parameters into the URL, such as:
QUrl result = QRestAccessManager::interpolatePathParameters( "/v1/accounts/:account_id/object/:object_id/cancel", { {"account_id", "acc_1234"}, {"object_id", "obj_5678"} });This is needed to avoid bugs such as a user-supplied parameter containing a slash for example.
Coding Con Currency
In recent years I’ve been writing and reading more Typescript/Angular code which consumes REST services, and less C++. I’ve enjoyed the way Promises work in that environment, allowing sequences of REST requests, for example, to be easy to write and read. A test of a pseudo API could await on requests to complete and invoke the next one with something like:
requestFactory.setBaseURL("http://some_service.com");async testWorkflow(username: string, password: string) { const loginRequest = requestFactory.makeRequest("/login"); const loginRequestData = new Map(); loginRequestData.setParam("username", username); loginRequestData.setParam("password", password); const loginResponse = await requestAPI.post( loginRequest, loginRequestData); const bearerToken = loginResponse.getData(); requestAPI.setBearerToken(bearerToken); const listingRequest = requestFactory.makeRequest("/list_items"); const listingResponse = await requestAPI.get(listingRequest); const listing = JSON.parse(listingResponse.getData()); const firstItemRequest = requestFactory.makeRequest( "/retrieve_item/:item_id", { item_id: listing[0].item_id } ); const firstItem = await requestAPI.get(firstItemRequest);}The availability of async functions and the Promise to await on make a test like this quite easy to write, and the in-application use of the API uses the same Promises, so there is little friction between application code and test code.
I wanted to see if I can recreate something like that based on the Qt networking APIs. I briefly tried using C++20 coroutines because they would allow a style closer to async/await, but the integration friction with existing Qt types was higher than I wanted for an experiment.
Using the methods in QtFuture however, we already have a way to create objects representing the response from a REST API. The result is similar to the Typescript example, but with different ergonomics, using .then instead of the async and await keywords.
struct RestRequest{ QString method; QString requestUrl; QHttpHeaders headers; QHash<QString, QString> urlParams; QUrlQuery queryParams; std::variant<QUrlQuery, QJsonDocument> requestData;};struct RestResponse{ QJsonDocument jsonDoc; QHttpHeaders headers; QNetworkReply::NetworkError error; QUrl url; int statusCode;};QFuture<RestResponse> makeRequest(RestRequest restRequest){ auto url = interpolatePathParameters( restRequest.requestUrl, restRequest.urlParams); auto request = requestFactory.createRequest(url); auto requestBodyDoc = extractRequestContent(restRequest.requestData); auto requestBody = requestBodyDoc.toJson(QJsonDocument::Compact); auto reply = qRestManager.sendCustomRequest(request, restRequest.method.toUtf8(), requestBody, &qnam, [](QRestReply &) {}); return QtFuture::connect(reply, &QNetworkReply::finished).then( [reply]() { QRestReply restReply(reply); auto responseDoc = restReply.readJson(); if (!responseDoc) { throw std::runtime_error("Failed to read response"); } RestResponse response; response.jsonDoc = *responseDoc; response.statusCode = restReply.httpStatus(); response.error = restReply.error(); response.headers = reply->headers(); response.url = reply->url(); return response; } );}The QRestAccessManager API requires the creation of a dummy response function when creating a custom request because it is not really designed to be used this way. The result is an API accepting a request and returning a QFuture with the QJsonDocument content. While it is possible for a REST endpoint to return something else, we can follow the Qt philosophy of making the most expected case as easy as possible, while leaving most of the rest possible another way. This utility makes writing unit tests relatively straightforward too:
RemoteAPI remoteApi;remoteApi.setBaseUrl(QUrl("https://dog.ceo"));auto responseFuture = remoteApi.makeRequest( {"GET", "api/breed/:breed/:sub_breed/images/random", {}, { {"breed", "wolfhound"}, {"sub_breed", "irish"} }});QFutureWatcher<RestResponse> watcher;QSignalSpy spy(&watcher, &QFutureWatcherBase::finished);watcher.setFuture(responseFuture);QVERIFY(spy.wait(10000));auto jsonObject = responseFuture.result().jsonDoc.object();QCOMPARE(jsonObject["status"], "success");QRegularExpression regex( R"(https://images\.dog\.ceo/breeds/wolfhound-irish/[^.]+.jpg)");QVERIFY(regex.match(jsonObject["message"].toString()).hasMatch());The result is quite similar to the Typescript above, but only because we can use spy.wait. In application code, we still need to use .then with a callback, but we can additionally use .onFailed and .onCanceled instead of making multiple signal/slot connections.
With the addition of QtFuture::whenAll, it is easy to make multiple REST requests at once and react when they are all finished, so perhaps something else has been gained too, compared to a signal/slot model:
RemoteAPI remoteApi;remoteApi.setBaseUrl(QUrl("https://dog.ceo"));auto responseFuture = remoteApi.requestMultiple({ { "GET", "api/breeds/list/all", }, {"GET", "api/breed/:breed/:sub_breed/images/random", {}, {{"breed", "german"}, {"sub_breed", "shepherd"}}}, {"GET", "api/breed/:breed/:sub_breed/images/random/:num_results", {}, {{"breed", "wolfhound"}, {"sub_breed", "irish"}, {"num_results", "3"}}}, {"GET", "api/breed/:breed/list", {}, {{"breed", "hound"}}},});QFutureWatcher<QList<RestResponse>> watcher;QSignalSpy spy(&watcher, &QFutureWatcherBase::finished);watcher.setFuture(responseFuture);QVERIFY(spy.wait(10000));auto four_responses = responseFuture.result();QCOMPARE(four_responses.size(), 4);QCOMPARE(four_responses[0].jsonDoc.object()["status"], "success");QVERIFY(four_responses[0].jsonDoc.object()["message"]. toObject()["greyhound"].isArray());QRegularExpression germanShepherdRegex( R"(https://images.dog.ceo/breeds/german-shepherd/[^.]+.jpg)");QCOMPARE(four_responses[1].jsonDoc.object()["status"], "success");QVERIFY(germanShepherdRegex.match( four_responses[1].jsonDoc.object()["message"].toString()).hasMatch());QRegularExpression irishWolfhoundRegex( R"(https://images.dog.ceo/breeds/wolfhound-irish/[^.]+.jpg)");QCOMPARE(four_responses[2].jsonDoc.object()["status"], "success");auto irishWolfhoundList = four_responses[2].jsonDoc.object()["message"].toArray();QCOMPARE(irishWolfhoundList.size(), 3);QVERIFY(irishWolfhoundRegex.match(irishWolfhoundList[0].toString()). hasMatch());QVERIFY(irishWolfhoundRegex.match(irishWolfhoundList[1].toString()). hasMatch());QVERIFY(irishWolfhoundRegex.match(irishWolfhoundList[2].toString()). hasMatch());QCOMPARE(four_responses[3].jsonDoc.object()["status"], "success");auto houndList = four_responses[3].jsonDoc.object()["message"].toArray();QCOMPARE_GE(houndList.size(), 7);QVERIFY(houndList.contains("afghan"));QVERIFY(houndList.contains("basset"));QVERIFY(houndList.contains("blood"));QVERIFY(houndList.contains("english"));QVERIFY(houndList.contains("ibizan"));QVERIFY(houndList.contains("plott"));QVERIFY(houndList.contains("walker"));setAutoDeleteReplies(false);
I attempted to use new API additions in recent Qt 6 versions to interact with a few real-world REST services. The additions are valuable, but it seems that there are a few places where improvements might be possible. My attempt to make the API feel closer to what developers in other environments might be accustomed to had some success, but I’m not sure QFuture is really intended to be used this way.
Do readers have any feedback? Would using QCoro improve the coroutine experience? Is it very unusual to create an application with QWidgets instead of QML these days? Should I have used PyQt and the python networking APIs?