Cutelyst just got a new release!

Thanks to the release of Virtlyst – a web manager for virtual machines, several bugs were found and fixed in Cutelyst, the most important one in this release is the WebSockets implementation that broke due the addition of HTTP/2 to cutelyst-wsgi2.

Fixing this was quite interesting as when I fixed the issue the first time, it started to make deleteLater() calls on null objects which didn’t crash but since Qt warn’s you about that it was hurting performance. Then I fixed the deleteLater() calls and WebSockets broke again a little of gdb foo and I found out that when I asked for the websocket to close Qt was emitting disconnected as a result of that call, and that signal was deleting the object we rely when we called close, which crashed the app.

As a new feature Context class has two new methods to help with async apps, a wait() method that creates a counter based local event loop and a next() method to decrease the event loop counter till quitting it. This way you can for example connect two QNetworkReply::finished() signals to Context::next() and call Context::wait(2); so once the two requests are finished wait() returns and the client receives the data.

Codacy was also added to increase the number of checks in our code base, and it has already shown some real issues.

UPDATE: Right after release I found a cppcheck code fix broke listening on TCP sockets, yes, writing more unit tests especially for the WSGI module is top on my TODO list. So 2.2.1 is out now

Get it here https://github.com/cutelyst/cutelyst/releases/tag/v2.2.1

Over the weekend, while some KDE people were in Toulouse improving Akonadi, and other KDE people were in Berlin improving Plasma, I was in Goteborg at FOSS-North showing off some KDE things.

Anyone who saw our FOSDEM booth knows the setup. We still had the same blue table (thanks, Sune) and selection of low-power ARM blinkenlights, the Pine64 and a Pinebook. I still think that “hey, Plasma runs fine on an overpowered x86 laptop” is not particularly interesting, but that “the past six months have seen serious work on reducing Plasma’s resource usage aimed specifically at this kind of device” is. Different from FOSDEM is that I could now run one of the just-released Netrunner images for the Pinebook.

There are pictures circulating of a staged fist-fight between me and Bastian, who ran the GNOME booth. Except for those 30 seconds, we were good neighbours and I’d like to thank Bastian for keeping an eye on things when the KDE booth wasn’t otherwise occupied.

Bigger thanks to Helio, who came from not-Bavaria to help at the stand as well and give a software archaeology talk. And biggest thanks to Johan, for setting up and running the whole conference.

I gave a talk on governance — call it “Open Source Project Governance 101, with a brief mention of how KDE does it specifically”. A half hour is long enough for an overview, not long enough to really go over all the considerations you might have along each of the different axes of decision. I’m more than happy to be talking with people about specific issues in governance. I didn’t see many talks themselves: just one on deep-dive C++ and OpenGL stuff (I didn’t understand all of what Patricia does) and reproducible-builds tools like diffoscope (Chris’s talk gave a nice overview).

As you might have seen before, there were some issues with CryFS for which I had to switch Vault to use EncFS by default for Plasma 5.12 since it is a long-term support release.

Now that 5.12 is behind us, I’ve had the time to revamp the CryFS backend to use the latest features that Sebastian implemented in cryfs 0.9.9.

Plasma Vault

Plasma Vault now communicates the error messages reported by cryfs clearly and is able to upgrade the cryfs filesystem when you upgrade to a new cryfs version.

The new version of Plasma Vault is ready for testing (it is in master, Plasma 5.13 beta will be released in a few weeks).

---

You can support my work on , or you can get my book Functional Programming in C++ at if you're into that sort of thing.

This blog post is long overdue, but now that I’m back home from the KDE PIM Sprint in Toulouse, which took place last weekend, there’s some more news to report.

Akonadi Improvements

On the sprint, I finally finished and merged a new improvement in Akonadi called Notification Payloads. I will not go into the technical details here, the most important thing is that this new improvement will notably reduce the CPU and disk load in Akonadi, especially during intensive operations like email sync. It should also help with the long-standing issue regarding errors and email duplication when using POP3 and local mail filters. Finally, this new feature opens doors to further improvements and optimizations like server-side change recording (technicalities here) and ultimately being able to
shut down Akonadi Resources when they are not needed and start them on-demand, thus saving some more resources.

As I was touching the internal notification system in Akonadi I also improved the relevant debugging tools in Akonadi Console, our developer and debugging tool for Akonadi. Based on input from Sandro I also added Logs view. Thanks to that it’s now possible to see debug output from all running Akonadi applications straight in the Akonadi Console without the need to restart Akonadi or the application from the terminal to see the debug output. This will make it easier for users to provide us with relevant information to help us debug and solve their Akonadi issues.

Kontact Improvements

This was just a minor change, but it finally solved my long-standing issue with Kontact and Breeze: the side-pane icons to choose between different Kontact modules were colorful – the only non-monochromatic part of Kontact which was so obviously not fitting into the rest of the UI. With a tiny change, the icons are now also monochromatic, making the Kontact window look more uniform.

Native Gmail authentication for IMAP and SMTP

For a while now the IMAP resource supports logging into Gmail accounts using the so-called OAuth method, where you provide your credentials into the Gmail login window which also supports two-factor authentication. The IMAP Resources was forcing the OAuth method with Gmail for everyone, but this requirement has now been relaxed. Although the IMAP resource will choose this method by default it’s possible now to also choose the traditional authentication methods like with any other email provider.

Secondly, the OAuth support has finally landed also into our SMTP module which is used for sending emails, so if you select this method in your Outgoing account configuration with Gmail, you no longer need to use “App-specific passwords” from Gmail.

Syndication Cleanup

The Syndication library is used to retrieve and parse RSS and ATOM feeds and is used among others by Akregator. We have now cleaned up the library and removed some redundant dependencies so that we will eventually be able to move it into KDE Frameworks so that even more applications can benefit from it.

Going to Windows

Thanks to a huge effort from Hannah we are now able to build Akonadi and other parts of the KDE PIM stack on Windows. While we are still a long way away from having Kontact properly running on Windows, we managed to get Akonadi to work on Windows with some other programs. Windows is a huge platform and Kontact with all its features and functionality could be a good competition to established PIM solutions there and a huge potential to grow our user and developer base. While we still focus primarily on Linux, we are slowly looking forward to extending our reach to Windows.

Bugfixes

A lot of them. Big thanks to David Faure who spent a big part of the weekend debugging his IMAP resource to figure out why it keeps getting stuck on occasions. He fixed several issues in the IMAP resource so that it properly reconnects after server connection is lost or times out (due to poor internet connectivity usually) and also found and fixed some issues in Akonadi syncing code.

Future

What’s next then? We will continue to work towards a stable release for Windows,
and hopefully soon finish the rewrite of the indexing and search infrastructure
in KDE PIM to make it faster, reliable and more useful again. There’s also a lot
of smaller tasks and improvements to look into during the year.

It's been 2 days since the GSoC accepted student list was announced and I'm still getting goosebumps thinking about the moment I saw my name on the website. I started contributing to open source after attending a GSoC session in our college by one of our senior and a previous GSoC student with KDE: Aroonav …

Continue reading "Google Summer of Code 2018 with KDE"

At the end of my previous post we concluded with yet another question. Indeed, on the 2017 KDEPIM contributor network we found out that Christian Mollekopf while being a very consistent committer didn't appear as centrality as we would expect. Yet from the topology he seemed to act as a bridge between the core contributors and contributors with a very low centrality. This time we'll try to look into this and figure out what might be going on.

My first attempt at this was to try to look into the contributor network on a different time period and see how it goes. If we take two snapshots of the network for the two semesters of 2017, how would it look? Well, easy to do with my current scripts so let's see!

Alright, it still looks similar to the picture we got for the full 2017... Christian is still on the outter rings of our network and bridging toward low centrality nodes. Only difference is that he has a slightly higher centrality value than during the whole year. Needless to say just that semester doesn't learn us much. Time to look at the second semester then.

Ah-Ah! Now we see something new, Christian is now mostly disconnected from the network! He is part of a clique containing him and Michael Bohlender. Looking further at their activity they are indeed focusing almost exclusively on Kube. Michael was in fact one of those low centrality nodes Christian was bridging to previously.

So what are we looking at? It seems to be the birth of an insular sub-team in the KDEPIM community. It's technically not a fork since they're working on a specific software but this clique configuration indicates they moved their focus there, they didn't attract the rest of the KDEPIM community to contribute (yet?) and they stopped contributing completely to the wider KDEPIM effort (at least for the time frame we've been looking at). The community got split there.

Now we could leave it at that and consider it like a detail... or... if you're like me and want not only to produce those graphs and metrics but wonder if some of those things could be turned into useful tools for community stewardship in general and the Community Working Group in particular, you won't stop there.

From the two networks above and the one I produced the last time it's clear that we need to deal with time... From a single network we freeze the time and get a configuration for a given period. If we ever want to see that something like the clique we saw appearing here can be detected we need a less static view.

For the time being, we will look at individual centrality of a contributor over time. For that we will get their monthly centrality value in the network over a three months sliding window (previous month, current month and next month). Since it's also interesting to have an idea of the activity of the contributor over the time period, we'll also plot the normalized monthly activity of the contributor. Finally, since centrality is dependent on the team size, we'll plot the normalized team size on the period.

Regarding that last plot, a few more words because it's a fairly important one that Volker Krause helped me realize during the KDEPIM sprint because of his own plots and discussing them with him, unfortunately it's also what makes the centrality tricky to read. The centrality value of a node is a value between 0 and 1, if a node is not connected at all it gets a 0 if a node is connected to all other nodes it gets a 1. So obviously, if the team is large you need way more connections to get a high centrality than in a small team.

Corollary of the point above is that centrality values variation are meaningful only during a stable team size. If we're a period of decreasing or increasing team size variations on a centrality can occur for a node even though it would have maintained the exact same connections! And that's why we have the third plot on the team size in the graphs below to get an idea on how much trust we can put in the variation of the centrality plot.

Alright, with that out of the way (although it'll keep haunting us while reading those plots), now it's time to explore those plots. We won't look only at one, I think it's a good idea to look at more than one contribution pattern before coming back to Christian. To get there and keep those plots somewhat comparable I'll drastically expand the time period we'll look at, instead of looking at 2017 only, we'll go all the way back to 2007! This way we can see more of KDEPIM's history and get patterns also from old timers. Let's start!

So first thing first, we see the evolution of the team size in KDEPIM on the last ten years. Interestingly we see the decrease that Paul Adams was pointing out in his last Akademy talk... but it's not reaching the ground and it looks like it stabilized at least since 2014. Is it the case for the whole of KDE? Does the commit activity look the same globally? Clearly questions I'll have to investigate as well, it never ends! :-)

In any case this variation on team size seems to indicate that we can look at the centrality variations from 2007 to end of 2009 or from 2014 to the end of 2017 somewhat safely. Of course the team size keeps varying but it's more noise than a real trend so it should be fine overall.

With that in mind, what we can see from Till is a former core contributor who slowly stopped to contribute. This is crystal clear just from his activity plot and the centrality plot as expected follows the same pattern. It's indeed less correlated with his activity in the 2010 to 2012 period but that's to be expected with the downward trend in team size.

This second graph is now about Volker Krause. We can see the top activity he had during 2009 and because the team size was large at the time it required such a high activity for him to have his centrality spike as well. The mystery spike of September 2016 is what prompted the display of the team size plot. He had only a very tiny activity that month which generated a surge in centrality... well it turns out that even though he did only a hand full of commits some of them were on build system files which tend to be touched by others and because of the smaller team size than in 2009 the variation get amplified.

So now that we're done with our two "example" core contributors... let's get back in the territory of the very active contributors of the past year...

Let's look at Laurent in this third graph, clearly he has been contributing to KDEPIM for a long time but overall not on a very high volume. It really started to increase around 2012 so I guess that's when he slowly took over maintainership of KMail. As expected that's when we see his centrality raising as well as he was getting involved with more and more components of KDEPIM. Of course it's slightly amplified by the decrease of team size over the 2012 - 2014 period but he kept getting more central even after that.

And finally, this fourth graph gets us back to Christian. Clearly he joined KDEPIM at the end of 2010, from that point on he looks like any other future contributor with increased activity correlating with increased centrality (watch out for the decrease of team size until 2014 which amplifies a bit the effect on that period). Then during 2014 we have a somewhat stable centrality and activity. Some noise but nothing out of the ordinary over a year. It gets interesting after that though. During 2015 we see his activity increasing again but at the same time his centrality starts dropping a first time. It then stays somewhat stable while his activity spikes. And toward the end of 2017 centrality completely drops. This is a very different pattern from all the other contributors we looked at.

In my opinion, the interesting observation is that by looking at the contributor network, we see the clique only appearing at the second semester of 2017, but, on the centrality graph we see this pattern of increasing activity with decreasing centrality starting in 2015! Two years before the community split is visible.

Now the question I have, and I think it'll be a tough one so I might leave it unanswered for a little while. Could we detect this kind of pattern early? Could we detect without too much false positive (even though there always will be some of them)?

I think it's important to think about that because in that particular case, assuming we'd have such a tool, the Community Working Group would have been warned of a team split to come and maybe step in to see if they could save the situation. Currently our Community Working Group is mostly working in reactive mode since they talk to people when a conflict emerges, with such a tool they could also try to be proactive and check on a team if the "increasing activity with decreasing centrality" pattern emerges. I think it would be nice if they could do this and talk to people before too many feelings were hurt.

It'll take time to get there, if at all. But I think it's worth looking into.

new gold editing pattern

• fix for dodge maximized visibility mode in multi-screen environment
• when copying default layouts make sure they are writable in the destination
• new protocol to communicate between applets and Latte in order to
  inform them when they are in a Latte panel/dock and when they dont want any change in their main icon behavior. (#983)

The Elisa team is happy to announce the first bug fix release for the 0.1 version.

The following changes have been integrated:

• c5bc988 search for more cover names in directory by Alexander Stippich (Fix bug 392843);
• 5113d00 remove the huge playlist icon by Alexander Stippich (Fix bug 392782).

One important note, Elisa is now available from flathub. Only the stable releases will be distributed through flathub.

Flathub Elisa page

This release also has a windows setup at the time of release. Thanks a lot to the help from the Craft and binary-factory teams. Without them, I would not be able to do it.

Elisa source code tarball is available here. The Windows installer is also available from this place.

Let’s have some more Usability & Productivity!

I’ve initiated a big project: overhauling KDE Open & Save dialogs for greater usability and productivity. If you would like to follow along, here is the meta-task tracking the work: https://phabricator.kde.org/T8552

So far, we’ve:

• Fixed column widths (Scott Harvey)
• Added view mode buttons to the toolbar (Me, Nate Graham)
• Made the dialog always save the view settings even when canceling or closing (Me, Nate Graham)

All of this work–and hopefully the rest of the “open/save panel improvement” work–will ship with KDE Frameworks 5.46.

(Also, what do people think of the above style for these bullet points instead of the typical one I’ve used below?)

As usual, there’s more! Here’s the other Usability & Productivity-related work from the past week:

New Features

• Kate and other programs using the KTextEditor framework gained syntax highlighting for GDB files (KDE Phabricator revision D11902, implemented in KDE Frameworks 5.46, authored by Milian Wolff)
• Konsole now supports more XTerm-style cursor shapes (KDE Bug 347323, implemented in KDE Applications 18.08.0, authored by Ahmad Samir)

Bugfixes

• Bold, italic, and underline effects now work again for Kate’s syntax highlighting modes (KDE Phabricator revision D12221, fixed in KDE Frameworks 5.46, authored by Christoph Cullman)
• Dolphin’s icon view grid spacing no longer changes unpredictably when previews are turned on or off, and has tighter spacing when previews are on and icons are large (KDE bug 393306, fixed in KDE Applications 18.04.1, authored by me, Nate Graham):
  
• Gwenview no longer crashes when zooming after reloading an SVG image (KDE bug 359736, fixed in KDE Applications 18.04.0, authored by Peter Mühlenpfordt)
• Gwenview’s “Failed to save” warning dialog no longer displays raw HTML in it (KDE bug 393170, fixed in KDE Applications 18.04.04, authored by Peter Mühlenpfordt)
• Okular no longer loses highlighted search results when a page is rotated (KDE bug 387282, fixed in KDE Applications 18.04, authored by Ahmad Osama)
• The Media Frame widget’s mode chooser now displays the modes corrently even for languages where the words are very long (KDE bug 393232, fixed in KDE Plasma 5.12, authored by Friedrich Kossebau)

UI Polish & Improvement

• Changes to a Konsole profile’s keybingings are now immediately applied to that profile (KDE Phabricator revision D12255, improved in KDE Applications 18.08, authored by Ahmad Samir)
• Plasma and Plasma widgets now all use the same color picker widget (KDE Phabricator revisions D12330 and D12318, fixed in KDE Plasma 5.13, authored by Friedrich Kossebau:
  
• Gwenview now positions the viewport more intelligently after crop and resize operations (KDE Phabricator revisions D12167 and D12170, authored by Peter Mühlempfordt)
• Gwenview no longer shows duplicate confirmation dialogs when overwriting an existing image (KDE Phabricator revision D12346, fixed in KDE Applications 18.04.1, improved by Peter Mühlenpfordt)
• The KDirOperator widget (commonly used for inline file browsers in apps like Kate, Kile, and RKWard, as well as open and save dialogs) now has a reload button in the context menu (KDE bug 199994, improved in KDE Frameworks 5.46, authored by me, Nate Graham)

Might seem a little light this week, but trust me, there’s a ton of stuff in progress right now. It may not quite be ready yet, but it will be awesome once it is!

If my efforts to perform, guide, and document this work seem useful and you’d like to see more of them, then consider becoming a patron on Patreon, LiberaPay, or PayPal.

In February after Plasma 5.12 was released we held a meeting on how we want to improve Wayland support in Plasma 5.13. Since its beta is now less than one month away it is time for a status report on what has been achieved and what we still plan to work on.

Also today started a week-long Plasma Sprint in Berlin, what will hopefully accelerate the Wayland work for 5.13. So in order to kick-start the sprint this is a good opportunity to sum up where we stand now.

QT_QPA_PLATFORM

Let us start with a small change, but with huge implications: the decision to not set the environment variable QT_QPA_PLATFORM to wayland anymore in Plasma’s startup script.

Qt based applications use this environment variable to determine the platform plugin they should load. The environment variable was set to wayland in Plasma’s Wayland session in order to tell Qt based applications that they should act like Wayland native clients. Otherwise they load the default plugin, which is xcb and means that they try to be X clients in a Wayland session.

This also works, thanks to Xwayland, but of course in a Wayland session we want as many applications to be Wayland native clients as possible. That was probably the rationale behind setting the environment variable in the first place. The problem is though, that this is not always possible. While KDE applications are compiled with the Qt Wayland platform plugin, some third-party Qt applications were not. A prominent example is the Telegram desktop client, which would just give up on launch in a Wayland session because of that.

With the change this is no longer a problem. Not being forced in its QT_QPA_PLATFORM environment variable to some unavailable plugin the Telegram binary will just execute using the xcb plugin and therefore run as Xwayland client in our Wayland session.

One drawback is that this now applies to all Qt based applications. While the Plasma processes were adjusted to now be able to select the Wayland plugin themselves based on session information other applications might not although the wayland plugin might be availale and then still run as Xwayland clients. But this problem might go away with Qt 5.11, which is supposed to either change the behavior of QT_QPA_PLATFORM itself or feature a new environment variable such that an application can express preferences for plugins and fallback if to the first supported one by the session.

Martin Flöser, who wrote most of the patches for this change, talked about it and the consequences in his blog as well.

Screencasts

A huge topic on Desktop Wayland was screen recording and sharing. In the past application developers had a single point of entry to write for in order to receive screencasts: the XServer. In Wayland the compositor as Wayland server has replaced the XServer and so an application would need to talk to the compositor if it wants access to screen content.

This rightfully raised the fear that now developers of screencast apps would need to write for every other Wayland compositor a different backend to receive video data. As a spoiler: luckily this won’t be necessary.

So how did we achieve this? First of all support for screencasts had to be added to KWin and KWayland. This was done by Oleg Chernovskiy. While this is still a KWayland specific interface the trick was now to proxy via xdg-desktop-portal and PipeWire. Jan Grulich jumped in and implemented the necessary backend code on the xdg-desktop-portal side.

A screencast app therefore in the future only needs to talk with xdg-desktop-portal and receive video data through PipeWire on Plasma Wayland. Other compositors then will have to add a similar backend to xdg-desktop-portal as it was done by Jan, but the screencast app stays the same.

Configure your mouse

I wrote a system settings module (KCM) for touchpad configuration on Wayland last year. The touchpad KCM had higher priority than the Mouse KCM back then because there was no way to configure anything about a touchpad on Wayland, while there was a small hack in KWin to at least control the mouse speed.

Still this was no long term solution in regards to the Mouse KCM, and so I wrote a libinput based Wayland Mouse KCM similar to the one I wrote for touchpads.

I went one step further and made the Mouse KCM interact with Libinput on X as well. There was some work on this in the Mouse KCM done in the past, but now it features a fitting Ui like on Wayland and uses the same backend abstraction.

Dmabuf-based Wayland buffers

Fredrik Höglund uploaded patches for review to add support for dmabuf-based Wayland buffer sharing. This is a somewhat technical topic and will not directly influence the user experience in 5.13. But it is to see in the context of bigger changes upstream in Wayland, X and Mesa. The keyword here is buffer modifiers. You can read more about them in this article by Daniel Stone.

Per output color correction

Adjusting the colors and overall gamma of displays individually is a feature, which is quite important to some people and is provided in a Plasma X session via KGamma in a somewhat simplistic fashion.

Since I wrote Night Color as a replacement for Redshift in our Wayland session not long ago I was already somewhat involved in the color correction game.

But this game is becoming increasingly more complex: my current solution for per output color correction includes changes to KWayland, KWin, libkscreen, libcolorcorrect and adds a KCM replacing KGamma on Wayland to let the user control it.

Additionally there are different opinions on how this should work in general and some explanations by upstream more confused me than guided me to the one best solution. I will most likely ignore these opinions for the moment and concentrate on the one solution I have at the moment, which might already be sufficient for most people. I believe it will be actually quite nice to use, for example I plan to provide a color curve widget borrowed from Krita to set the color curves via some control points and curve interpolation.

More on 5.13 and beyond

In the context of per output color correction another topic, which I am working on right now, is abstracting our output classes in KWin’s Drm and Virtual backends to the compositing level. This will first enable my color correction code to be nicely integrated and I anticipate in the long term will be even necessary for two other far more important topics: layered rendering and compositing per output, what will improve performance and allow different refresh rates on multi-monitor setups. But these two tasks will need much more time.

Scaling on Wayland can be done per output and while I am no expert on this topic from what I heard because of that and for other reasons scaling should work much better on Wayland than on X. But there is currently one huge drawback in our Wayland session: we can only scale integer factors. To change this David Edmundson has posted patches for review adding support for xdg-output to KWayland and to KWin. This is one step in allowing fractional scaling on Wayland. There is more to do according to Davd and since he takes part in the sprint I hope we can talk about scaling on Wayland extensively in order for me to better understand the current mechanism and what all needs to be changed in order to provide fractional scaling.

At last there is cursor locking, which is in theory supported by KWin, but in practice does not work well in the games I tried it with. I hope to start work on this topic before 5.13, but I will most likely not finish it for 5.13.

So overall there is lots of progress, but still quite some work to do. In this regard I am certain the Plasma Sprint this week will be fruitful. We can discuss problems, exchange knowledge and simply code in unity (no pun intended). If you have questions or feedback that you want us to address at this sprint, feel free to comment this article.

Last week I was at work and start to listen my boss said: "We need to show this to our director".  So I went to my coworker table to see what was happening. So they were using Gource to make a video about the git history of the project. Gource is a software version control... Continue Reading →

Virtlyst is a web tool that allows you to manage virtual machines.

In essence it’s a clone of webvirtmgr, but using Cutelyst as the backend, the reasoning behind this was that my father in law needs a server for his ASP app on a Win2k server, the server has only 4 GiB of RAM and after a week running webvirtmgr it was eating 300 MiB close to 10% of all available RAM. To get a VNC or SPICE tunnel it spawns websockify which on each new instance around 20 MiB of RAM get’s used.

I found this unacceptable, a tool that is only going to be used once in a while, like if the win2k freezes or goes BSOD, CPU usage while higher didn’t play a role on this.

Choosing a web interface for KVM/libvirt is no easy task, I have used Archipel and while it’s a nice app it is a pain to install, OpenStack is also overly complicated, and had a button labeled as “Terminate” button that deleted the instance entirely. A simple tool that’s specially simple to install was what I needed, and in fact is what a bunch of people need, here at work, OpenStack was also discarded due the hard installation process and a mix of virsh/virt-manager is used.

Since webvirtmgr is a DJango app, using it’s html templates was a breeze, Grantlee didn’t work with them out of the box, it has a few missing features but one can work around those, libvirt API is also quite well documented so development was quite fast, it took me 3 weeks but could easily be reduced to less than 2 if I had a bit more time.

So Virtlyst v1.0.0 is out, it uses less than 10MiB of RAM, and implements the VNC/SPICE web socket proxy in the same process thus each connections increases a almost nothing of memory usage. The software is already in production, and has all features of webvirtmgr except the migration part, that I plan to add in a future release.

Following the steps of some developer fellows I also created a Patreon account, so if you like what I do and would like to support my work please consider becoming a Patron.

Enough said, go get it!

https://github.com/cutelyst/Virtlyst/archive/v1.0.0.tar.gz

Startup is one of the rougher aspects of the Plasma experience and therefore something we’ve put some time into fixing

Why is startup slow?

The primary reason for Plasma startup being slow is simply that it does a lot of things, including:

• Session restore
• Config migration
• Setting keyboard layouts
• Most esoterically, making sure your colour scheme gets synced to the xrdb database, so that loading xfig or dia or any other pre-Qt/GTK1 application still has the correct colours in your Plasma session

Too many to list. This means it’s easily a whole second or more before we even start loading plasmashell, one of the more time consuming parts of startup.
This slow load gets exaggerated by us not removing the splash screen till everything is actually properly loaded.

However, it’s still an area we can improve.

About startup

Startup consists of mulitple components invoked at 3 different levels.

• Kcminit
  Config modules (kcms) typically update a system when the user applies the settings, however as we need to restore settings there is an optional hook called on startup for various modules.
• KDE Daemons (kded)
  Evolving from the more static nature above may services require running in the background waiting for external events. Some modules are loaded on demand, others are loaded explicitly during startup.
• Autostarted applications
  Meaning launcing core applications such as plasmashell/krunner as well as other applications a user might have selected; such as nextcloud or kgpg
• User shell scripts
  For anything else

These effectively all have their loading invoked at 3 different levels; as defined in the relevant kcminit/kded/autostart .desktop files. Note these may be shipped as core plasma, but can also include completely 3rd party items.

• Before plasmashell and other core autostart services
• Before user applications are restored / autoloaded
• After everything is restored and

A slightly more detailed version can be seen here.

Profiling

The most important part of any speed work is correctly analysing it.
systemd-bootchart is nearly perfect for this job, but it’s filled with a lot of system noise.

I’ve made a fork of systemd-bootchart that only tracks processes owned by that user on github.
To use put the following line into:
/etc/X11/Xsession.d/00-bootchart

/usr/lib/systemd/systemd-bootchart -o /tmp -r &

To profile wayland sessions, mod the SDDM config wayland SessionCommand option.

After 20 seconds an SVG will be placed in /tmp.

An example

Below shows the boot process of my personal machine. The important part to track is when ksplashqml gets destroyed, that signifies that boot is complete. In my case this happens after 4.5 seconds.

Just some of the many fixes

There have been and continue to be lots of tiny changes around the code. Mostly using the awesome tool hotspot here are just a few of the more significant and interesting ones.

A mysterious gap of nothingness

This stupid bug was caused by ksmserver blocking whilst it completed launching phase 1 of kded daemons before it continued onto loading the next section.
Problem is plasmashell and other GUI apps loaded in phase0 first task is to register with the X session manager, ksmserver. End result is we lockup in X code till
ksmserver is done. Solved by a logic rewrite.

Multiple starts of the same daemon

This is the result of the following code in a bunch of processes all started at once.

if (! serviceExistsOnDBus) {
startExecutable(…)
}

Whilst the daemon did handle this case correctly and the false starts aborted, it’s still a lot of time spend linking a process we don’t want.
Solution in this case was to port to DBus activation which is designed for this exact case.

Shortcuts writing on load

When profiling kglobalaccel5 we found it spent a lot of time writing entries on boot, which is a clear sign of something being wrong. The cause was quite complex. Powerdevil would report that the user visible name of the executable (kded) was “Powerdevil”. KScreen would load and report that the username of the executable (also kded) was “kded” this would cause the shortcut daemon to think locales had changed and forcibly reload and save everything.

Summary

Startup is an area that is being improved and we have tonnes of ideas left of things to do, but because it’s dealing with unpredictable 3rd parties it’s an area which requires 90% reading and 10% coding.

If you do have an excessively long boot time, please do try creating a boot chart as above, it will really help start to narrow down where we have problem. I hope to see some detailed reports on bugzilla soon.

Netrunner Pinebook

This week, there was the launch of Netrunner for Pinebook (1) (2). A lot of effort has gone into building the software, improving performance, getting automation and CI in place. That’s effort that benefits the wider KDE community, the wider Free Software-on-ARM community, and more. This is the laptop I’d take with me on a bicycle camping trip — clean, cheap and affordable, although heavier than a phone.

But there is an under-appreciated bit regarding images for an ARM laptop — or pre-installed Linux distro’s in general. And that’s the first-run experience. The Netrunner Pinebook image is delivered so that it boots to the Plasma 5 desktop, no passwords asked, etc. The user is called “live”, the password is “live”, and nothing is personalized. It’s possible, though not particularly secure, to use the laptop this way in a truly disposable fashion. A first-run application helps finalize the configuration of the device by creating a named user, among other things.

One of the under-documented features of Calamares is that it can operate as a first-run application as well as a system installer. This is called “OEM Mode“, because it’s of greatest interest to OEMs .. but also to distro’s that ship an image for users to flash onto (micro)SD card for use in a device.

Main Desktop on Pinebook

This screenshot is from Netrunner on the Pinebook, and you can see the purple-ish Calamares logo labeled “First Time Run This Installer” on the desktop — it’s partly hidden by the system information KCM. That runs Calamares in OEM mode, hardly distinguishable from the live-ISO-installer-mode that it usually has.

A Calamares OEM configuration generally:

• Resizes the image file to fill the entire card,
• Creates a user (the real one, personalised) with a password,
• Performs some initial configuration of the real user,
• Adds packages and system configuration based on that configuration, and
• Does some cleanup (e.g. removing Calamares, since you only need it once).

Configuring Calamares as an OEM installer is no different from any other Calamares installation: make sure the config files end up in /etc/calamares, and then set dontChroot to true (that’s the real difference between OEM mode and regular).

The way Calamares development works, downstreams — that is the distro’s, and in this case Netrunner — just file feature requests for the stuff they need in Calamares, and after some back-and-forthing to make sure it’s sufficiently general, I write it. So the run up to Netrunner on the Pinebook saw some Calamares development aimed specifically at the OEM mode, and some extra bug reports. Nothing, though, that isn’t equally applicable to any other distro that needs a first-run installer.

There is a lot more that could be done in a first-run installer. KaOS has a really nice one, basically hand-holding through initial KDE Plasma Desktop setup. Pardus and Pisi Linux have something similar. A downside is that the more you do, the more specialized it becomes — it would be nice to have a good GNOME counterpart to the Plasma Look-and-Feel selection module in Calamares, but that quickly leads to a multitude of modules and dependencies (not to mention that I can’t write it all myself).

Anyway, that’s my little square inch claim to being useful to the Netrunner Pinebook project (which deletes itself).

Calamares is a distribution-independent (Linux) system installer. Outside of the “big five” distro’s, many smaller “boutique” distro’s need an installer, and Calamares is a highly configurable one that they can use. There’s a few dozen distro’s that I know of that use it (although I’ve only actually installed maybe six of them).

One optional feature — optional at the discretion of the distro which is deploying Calamares, that is — is the use of GeoIP location to find the time zone of the device being installed. This is used to automatically set the clock, for instance. If it’s not switched on, or there’s no network, Calamares defaults to a location defined by the distro — generally New York, although there’s an Austria-based distro that I think defaults to UTC.

For years, the default and suggested GeoIP provider has been freegeoip.net. That service is shutting down — well, being upgraded to a nicer API, at a different location, and with different access limitations. Older ISO images that have Calamares configured to use GeoIP with that service will cease to get usable GeoIP data on july 1st, 2018.

I don’t actually know which distro’s have GeoIP enabled, nor what provider they use.

However, the fact that this provider is shutting down prompted me to go over the existing code and make it more flexible and more general (prodding from Gabriel and Phil and Harald helped here as well). So Calamares is now ready to use more, and different, providers of GeoIP data. The providers can’t agree on using “time_zone” or “timezone” or “TimeZone” as the attribute (JSON) or element (XML) name where the data lives, so there’s a little bit more code and configuration needed. But you (as a distro) can now shop around and pick a GeoIP provider that respects your privacy.

I use to joke that the last week before foss-north is the worst – everything is done, all that is left is the stress.

This year, we have the broadest program yet. 25 speakers talking about everything from community policies, GPU isolation, blockchain, historical KDE software, retro computers, IoT, Android, SailfishOS, bug triaging, crowd funding, software updates, yocto, home automation, design to sub-atomic particles.

You can still get a ticket (and make sure to bring a friend) at foss-north . Welcome!

The latest release of CMake has landed in FreeBSD. Prior to release we had good contact with KitWare via the bug tracker so there were few surprises left in the actual release. There were still a few last-minute fixes left, in KDE applications no less. Here is a brief summary of changes we made:

• FindQt doesn’t match the way FreeBSD ports are built and installed, so we defer to QMake rather than looking for directories,
• FindOpenMP gets a tweak because it won’t find the system pthreads library when gcc (for Fortran) is used,
• FindBLAS gets a larger tweak because BLAS may need to link to libgcc_s, and which gcc_s that is needs to be figured out via ldd(1).

Some older patches have gone away because upstream has picked them up. Tweaks downstream, in package-building-terms, of cmake that were necessary:

• check_include_files respects the required libraries, which can be a surprise when the required libraries have been found but not fully plugged into the build (e.g. missing -L flags).
• the order of includes in automoc sources has changed, which reveals places where a C++ header file doesn’t actually include all of the headers it needs to fully define the types it uses; previously the include order might implicitly include them and the issue is papered over.

CPack now fully supports producing FreeBSD packages from a build / install tree by default, so for non-ports software which uses CMake, cpack -G FREEBSD does the right thing. Previously, this was a non-default tweak to CPack as built in FreeBSD ports.
Edit: as of CMake 3.11.0_1, CPack no longer supports producing FreeBSD packages. There were some unexplored corners of the build process that cause build failures when the FreeBSD pkg(8) support is enabled. So it’s off again until we shine some more light into those corners.

.. and a PS., CMake 3.11.1 has just been released, which reverts the change to check_include_files which I’d been working around.

“I put Qt on my ARM” : The only fun thing about ARM ever

The team over at Netrunner have just announced the launch of Netrunner 18.03 Idolon for the Pinebook. This is the direct result of a year of collaboration between the Netrunner, Pine and KDE Communities in a effort to drive down memory consumption, fix glitches in the graphics stack and enabling accelerated video decode, all of which has resulted in a product that showcases the coming together of the amazing software from KDE and some brilliant hardware engineering from the folks over at Pine.

It’s been quite a journey for my colleagues and I at Blue Systems in putting together this product. We have had to delve into areas where we originally did not have the expertise to fix bugs and constantly push the boundaries of our abilities. This was especially challenging in the ARM world since there are parts of the stack that were proprietary, meaning we cannot debug those parts, leading to many frustrating evenings having been spent on trying to reverse engineer buggy behaviour.

Personally, the launch of Netrunner on the Pinebook marks a milestone towards a goal that I’ve wanted to see realised ever since I discussed it with my peers at UDS N, nearly 10 years ago: Delivering Plasma on cheap and affordable computing devices. Being able to run a fully accelerated open source desktop environment on a device that costs 100 USD to me is a amazing accomplishment. It confidently signals to the world that Plasma is a versatile and mature product that spans across form factors and offers a compelling  alternative to other main stream products on these low end devices.

Shameless Plug: If you enjoy using KDE’s software products, please consider donating

This blog is about KDE Connect, a project to communicate across all your devices. For example, with KDE Connect you can receive your phone notifications on your desktop computer, control music playing on your desktop from your phone, or use your phone as a remote control for your desktop.

In my previous blog, I told you about some of my improvements to the remote media control of KDE Connect. I ended with a note that local album art was not supported yet. Also, no album art was displayed in the media notification or on the lock screen. In the meanwhile, both features have been implemented! If you have KDE Connect 1.8.1 on Android, and KDE Connect 1.3 on your desktop, you'll have both of these features. So all your album art from VLC or Elisa is now visible on your phone!

Secondly, I've been working a bit on KDE Connect's bluetooth support. The code was mostly working already, but the remaining stuff is (of course) the hardest part! Nevertheless, more and more parts start working, so I assume it'll come your way in a couple of months. I'll post an update when it's ready for testing.

For, quite literally a year or more, KMail and Akonadi on FreeBSD have been only marginally useful, at best. KDE4 era KMail was pretty darn good, but everything after that has had a number of FreeBSD users tearing out their hair. Sure, you can go to Trojitá, which has its own special problems and is generally “meh”, or bail out entirely to webmail, but .. KMail is a really great mail client when it works. Which, on Linux desktops, is nearly always, and on FreeBSD, iswas nearly never.

I looked at it with Dan and Volker last summer, briefly, and we got not much further than “hmm”. There’s a message about “The world is going to end!” which hardly makes sense, it means that a message has been truncated or corrupted while traversing a UNIX domain socket.

Now Alexandre Martins — praise be! — has wandered in with a likely solution. KDE Bug 381850 contains a suggestion, which deserves to be publicised (and tested):

sysctl net.local.stream.recvspace=65536
sysctl net.local.stream.sendspace=65536

The default FreeBSD UNIX local socket buffer space is 8kiB. Bumping the size up to 64kiB — which matches the size that Linux has by default — suddenly makes KMail and Akonadi shine again. No other changes, no recompiling, just .. bump the sysctls (perhaps also in /etc/sysctl.conf) and KMail from Area51 hums along all day without ending the world.

Since changing this value may have other effects, and Akonadi shouldn’t be dependent on a specific buffer size anyway, I’m looking into the Akonadi code (encouraged by Dan) to either automatically size the socket buffers, or to figure out where in the underlying code the assumption about buffer size lives. So for now, sysctl can make KMail users on FreeBSD happy, and later we hope to have things fully automatic (and if that doesn’t pan out, well, pkg-message exists).

PS. Modern KDE PIM applications — Akonadi, KMail — which live in the deskutils/ category of the official FreeBSD ports were added to the official tree April 10th, so you can get your fix now from the official tree.