Plasma Mobile

Back around 2006, when the Plasma project was started by Aaron Seigo and a group of brave hackers (among which, yours truly) we wanted to create a user interface that is future-proof. We didn’t want to create something that would only run on desktop devices (or laptops), but a code-base that grows with us into whatever the future would bring. Mobile devices were already getting more powerful, but would usually run entirely different software than desktop devices. We wondered why. The Linux kernel served as a wonderful example. Linux runs on a wide range of devices, from super computers to embedded systems, you would set it up for the target system and it would run largely without code changes. Linux architecture is in fact convergent. Could we do something similar at the user interface level?

Plasma Netbook

In 2007, Asus introduced the Eee PC, a small, inexpensive laptop. Netbooks proved to be all the rage at that point, so around 2009, we created Plasma Netbook, proving for the first time that we could actually serve different device user interfaces from the same code-base. There was a decent amount of code-sharing, but Plasma Netbook also helped us identifying areas in which we wanted to do better.

Plasma Mobile (I)

Come 2010, we got our hands on an N900 by Nokia, running Maemo, a mobile version of Linux. Within a week, during a sprint, we worked on a proof-of-concept mobile interface of Plasma:

Well, Nokia-as-we-knew-it is dead now, and Plasma never materialized on Nokia devices.

Plasma Active

Plasma Active was built as a successor to the early prototypes, and our first attempt at creating something for end-users. Conceived in 2011, the idea was not just to produce a simple Plasma user interface for a tablet device, but also deliver on a range of novel ideas for interaction with the device, closely related to the semantic desktop. Interlinked documents, contacts, sharing built right into the core, not just a “dumb” platform to run apps on, but a holistic system that allows users to manage their digital life on the fly. While Plasma Active had great promise and a lot of innovative potential, it never materialized for end-users in part due to lack of interest from both, the KDE community itself, but also from people on the outside. This doesn’t mean that the work put into it was lost, but thanks to a convergent code-base, many improvements made primarily with Plasma Active in mind have improved Plasma for all its users and continue to do so today. In many ways, Active proved valuable as a playground, as a clean slate where we want to take the technology, and how we can improve our developemnt process. It’s not a surprise that Plasma 5 today is developed in a process very similar to how we approached Plasma Active back then.

Plasma Mobile (II)

Learning from the Plasma Active project, in 2015 we regrouped and started to build a rather simple smartphone user interface, along with a reference software stack that would allow us not only to develop Plasma Mobile further, but to allow us to run on a growing number of devices. Plasma Mobile (II)’s goal wasn’t to get the most innovative of interfaces out, but to create a bread-and-butter platform, a base to develop applications on. From a technology point of view, Plasma is actually very small. It shares approximately 95% of the code with its desktop companion, widgets, and increasingly applications are interchangeable between the two.

Plasma Mobile (in any shape or form) has never been this close to actually making it into the hands and pockets of end users. A collaboration project with Purism, a company bringing privacy and software freedom to end-users, we may create the first Plasma phone for end users and have it on the market as soon as januari 2019. If you want to support this project, the crowdfunding campaign has just passed the 40% mark, and you can be part of it — either by joining the development crew, or by pre-ordering a device and thereby funding the development.

So, back in Randa I was splitting my energies and attentions in many pieces. Some attention went to making pancakes and running the kitchen in the morning — which is stuff I take credit for, but it is really Grace, and Scarlett, and Thomas who did the heavy lifting, and Christian and Mario who make sure the whole thing can happen. And the attendees of the Randa meeting who pitch in for the dishes after lunch and dinner. The Randa meetings are more like a campground than a 5-star hotel, and we work together to make the experience enjoyable. So thanks to everyone who pitched in.

Part of a good sprint is keeping the attendees healthy and attentive — otherwise those 16-hour hacking days really get to you, in spite of the fresh Swiss air.

Frederik encouraged us to touch the floor and the ceiling with his acro-yoga sessions; it is good to get out of the hacking room and get into shape. These sessions also teach us all trust and cooperation. And between sessions, he fixed a Qt bug that probably affects Calamares accessibility.

Calamares had some more bugs squashed, its accessibility improved — although I need to test that again and again on different setups now that I’m home, since it needs time to re-build Qt and all. Even with this fix, the goal to reduce Calamares’ root-needs remains.

You can read more of what the attendees in Randa achieved on planet KDE (e.g. kdenlive, snappy, kmymoney, marble, kube, Plasma mobile, kdepim, and kwin). I’d like to give a special shout out to Manuel, who taught me one gesture in Italian Sign Langauage — which is different from American or Dutch Sign Language, reminding me that there’s localization everywhere.

For me, being back home means sitting back at my big FreeBSD box, hacking on Calamares and ARPA and CMake and stuff again, with a renewed sense of purpose and team thanks to having worked together in Randa. If you like what KDE achieves in Randa, consider still supporting the fundraiser so that we can return to the mountains next year.

Another successful Randa meeting! I spent most of my days working on snappy packaging for KDE core applications, and I have most of them done!

Snappy Builds on KDE Neon

We need testers! Please see Using snappy to get started.

In the evenings I worked on getting all my appimage work moved into the KDE infrastructure so that the community can take over.

I learned a great deal about accessibility and have been formulating ways to improve KDE neon in this area.

Randa meetings are crucial to the KDE community for developer interaction, brainstorming, and bringing great new things to KDE.
I encourage all of you to please consider a donation at https://www.kde.org/fundraisers/randameetings2017/

This week has been focused on finishing the development of persistent notifications at the top of the music views. They are intended to provide information about what happen with actions the user can take to improve things.

The following items have been pushed:

• Clean up of dependencies. Now all frameworks come with a description. Some have been downgraded to optional or recommended. Some have been upgraded to recommended ;
• A new version of persistent notifications.

The new persistent notifications

The player can be in four states:

• No notifications are active

  No notifications

   

• One notification is activeOne notification

   

• More than one notifications are active. Only one is shown such that vertical space is preserved

  Multiple notifications with the first visible

• The notifications area is expanded and several notifications are visible

  Show multiple notifications

   

If the user choose to act, the buttons are temporarily disabled to provide instant feedback and the notification disappear when the root cause is fixed (like in the example, if music tracks are discovered or Baloo configuration is modified or Baloo music indexer is disabled in Elisa.).

I have tried to provide smooth transitions between each of those states. Some may still be missing. Please do not hesitate to provide feedback on this feature.

I plan to add more notifications of this kind when the software wants to provide feedback to the user and asks him to choose what he prefers.

Next week, I should continue to improve integration with Baloo. I would also like to improve (in fact allow) the keyboard interaction.

Please read the following guest post from Łukasz who joined me last week in Randa to work on KMyMoney.   Good things have happened in Randa and are going to happen to KMyMoney afterwards. During that time I’ve started to work on the appearance of the application in regard to colors. I think it’s a …

Continue reading "KMyMoney’s Łukasz Wojniłowicz in Randa"

Just came back home yesterday from Randa Meetings 2017. This year, even though my major motive for the sprint was to use Qt 5.8’s Qt Speech module instead of custom Java for text-to-speech during navigation, that could not be achieved because of a bug which made the routes not appear in the app in the first place. And this bug is reproducible both by using latest code, and old-enough code, and is even there in the prod app in the Google Play Store itself. So, although most of my time had gone in deep-diving on the issue, unfortunately I was not able to find the root-cause to it eventually. I will need to pick up on that in the coming weeks again when I get time, to get it fixed. Apart from that, I managed to fix a few more bugs, and work on adding a splash screen to the Android app:

• Made the bookmarks dialog responsive to touch everywhere. When we tapped on the Search field in the app, if nothing was written in that field then the bookmarks dialog showed up there. If the number of bookmarks was <= 6, then the far-lower part of the dialog did nothing on being tapped. Made that portion choose the last item from the bookmarks list in that case.

• There was an issue where pressing the back-button always led to closing the app instead of handling that event depending on the various scenarios. On looking, it was found that the root-cause for the issue was lying in the code of Kirigami itself. Reported the same to Marco Martin, and fixed the issue in the Marble app via a workaround by using Kirigami.AbstractApplicationWindow instead of Kirigami.ApplicationWindow in the Qml. Also reported the issue that svg icons were not showing up in the app on Plasma Mobile because of svg icons not being enabled there.

• Worked on adding a splash screen to the Marble Maps app. The highest resolution of the Marble logo in png that we had at the moment was 192 x 192, which looked a bit blurry, as if it had been scaled up, so I just displayed it as 100×100 for the splash screen to make it look sharp. Later, Torsten created a 256×256 version of the png from an svg, which got rid of the blur in the previous png. So I added that in the splash screen later, and the icon there looks much bigger, sharper, and non-blurry now.
  

Apart from work, there was snow this year, did some interesting acro-yoga with Frederik, did a 3-hour hike to walk across the longest pedestrian suspension bridge in the world, went on a tour to Zermatt to catch a glimpse of the Matterhorn, and ended it all with having a delicious barbecue with Emmanuel and Tomaz somewhere up in the mountains on the final evening. Thanks to Mario for organizing the meetings and the cheese and chocolates, without which no work could have been done.

I’ve spent the last few days with fellow KDE hackers in beautiful Randa in the Swiss Mountains.
It’s an annual event that focuses on a specific topic every year, and this time accessibility was up, so Michael and me made our way up here to improve Kube in that direction (and to enjoy the scenic surroundings of course).

I have to admit we didn’t spend all our time on accessibilty, since we’ve recently done a big push in that direction already with keyboard navigation and translation preparations. Anyhow, here’s an overview on what we’ve worked on:

Query performance

While it might now be a showstopper, there is an annoyance in Kube that if you initially sync a lot of mail (my INBOX alone contains 40k) this not only uses a lot of CPU in the synchronizer process (which is expected), but also in the Kube client process, which seems odd at first.

The reason for this are the live queries, which update themselves whenever the resource signals that it has a new revision available (A new revision could be a new mail or a modification for instance). Usually, when we process an update we do so incrementally. If you just received a new mail or two, or something has been marked as read etc., that is also the sensible thing to do. It’s a comparatively tiny amount of work, and we really want incremental changes in the result anyways, otherwise we’d mess up the client’s UI state (selection, scroll position, …).

However, when we fetch 40k mails that also means we get 40k updates that we have to process and it’s surprisingly hard to avoid that:

• We don’t know in advance that there will be 40k updates, so we have to process the updates initially.
• We can’t really skip any updates, because any update might affect what we have in the view. We just don’t know until we’ve already processed it.
• We can’t just fall back to redoing the full query because we need incremental changes, otherwise you’d see a blinking UI until the sync is done.
• We don’t want to just stop updating until the sync is done, because in most cases you already have useful data to work with, so this is really a background task, and we don’t want to block all updates.

If you end up processing the larger part of the data-set as incremental update it becomes naturally really expensive, because you can’t leverage the indexes which made queries efficient in the first place. So this is not something that can just be optimized away.

This means some middleground has to be struck which allows for updates, but doesn’t render the application maxed out because it’s updating so fast.

A possible approach to fix this would be the following:

• Set some threshold when it becomes cheaper to redo the query instead of processing the updates.
• If the threshold is reached, redo the query from scratch.
• Diff the current results with the redone results so we can still update the UI incrementally.
• Apply some event compression to the revision updates, so we land in the optimized path quicker.

Given that this really isn’t a huge problem right now, since the application does remain responsive (querying is done in a separate thread), and this will only ever happen if you sync a large folder for the first time, I won’t spend a lot of time on that for now. The most important part was figuring out why this problem appeared in the first place, and understanding it’s reasons and implications. This will allow us to attack it properly once we get to it.

HTML composer

Kube’s composer is currently a simple text field where you can type plain text. This is obviously not the end goal.
To explore our options in providing a more advanced composer, we tried prototyping a simple HTML editor.
Unfortunately our experiments showed that QML isn’t really up to that yet. The official example falls flat on it’s face when trying it (it simply fails except for a few specific cases), and the “API” to do any sort of formatting is more a hack than anything else.

Granted, the API docs of the TextArea also state that you can’t modify the internal document, or in other words, no HTML composer for us.

The options we have left are:

• Some custom text editing component (sounds like a lot of work, probably is).
• Try to hack around TextAreas deficiencies until we find some magic combination that works.
• Use a WebView and implement everything in javascript (now that’s just sad).

For now the item is postponed.

GPG Support

We’ve scoped out what we want to reach for initial GPG support in Kube. While we already have read-only GPG support (we can decrypt mails and verify signatures), there is some more work to be done until we can also sign and encrypt messages we’re sending. The focus will be on keeping things simple to get the job done, and not build the most advanced GPG interface ever.

Things that will happen are:

• Key selection for the account that will then be used for encryption.
• Key setup for new accounts that don’t have a key yet (either by importing an existing key or creating a new one)
• Encryption option in the composer that depends on the availability of keys for all recipients.
• Key selection for users in the addressbook.

All of it can be found on Phabricator

Cross platform builds

Since we had some cross platform building experts on site, I explored some of our options:

• Cross compiling for Windows with MXE.
• Building on Windows and OSX using craft.
• Building with some custom scripts on Windows.
• Building with macports on OSX.
• Building with brew on OSX.

Although I had mixed feelings about craft back from the days when I used emerge (the craft predecessor) to build Kontact on Windows,
it at least looks like the most sensible option still.

Crosscompiling using MXE seems ok-ish for Windows; I had some success for parts of the stack and the rest will just be the grunt work of making tarballs and writing build definition files. At least that way you don’t have to work on a Windows system. However I heard that the compiled end-result will just not be as good as using native compilers, so perhaps it’s not worth it to trade some building convenience for a sub-par end result.
Also, writing the definition files is pretty exactly the same as for craft, and MXE would be for Windows only.

So, looks like I’ll be doing another round with craft sometime soonish.

Plasma phone

It kinda works.

Bushan Sha quickly wipped up a package for the Plasma Phone and installed it, which allowed us to see Kube running for the first time on a mobile device, which felt surprisingly cool

There clearly are some rough edges, such as the UI being completely unsuitable for that form factor (who would have thought…), and a bug in one of the QML components which breaks folder selection using a touch screen. Still, it was nice to see =)

Input validation

To improve the UX with forms in Kube, or more specifically the configuration dialog, we wanted to provide some feedback to the user on what fields need to be filled out. Michael implemented a neat little visualization that that marks invalid input fields and gently fades away once the input becomes valid.

So far we only validate that fields are not empty, but that can be changed by assigning new validators that e.g. check that an email address field indeed contains a valid email address.

Recap

It was once again a very productive week, with lot’s of good food and a couple of fun hikes in between to clear out the head.
Besides the usual work going on it’s invaluable to have an exchange with other people that just might have solved the problem already that you’re going to spend the next few weeks on. So in many ways this is not only about a concentrated work effort, but also about professional training and exchange, and that is only possible in when you’re sitting together with those people.

I’d like to thank the whole team that organized the sprint, took care of us with great food and lot’s of chocolate and generally just made our stay in Randa great once more.

If you made it all the way down here and skipped the fundraiser on top, please consider donating something, it’s the only way we can keep doing meetings like this and they are invaluable for us as a community.

Last week I had a chance to attend the Randa meetings 2017, my plan was to work on the Plasma Mobile during the sprint, improve the state of current images. A few things changed over the course of the week:

New shiny wallpaper and packages

Starting from 8 Sept, we started using the packages from KDE Neon buildsystem instead of building them on the Plasma Mobile CI, this provides few benefits,

• No need to separately build the common things like Qt5, KDE Frameworks, Plasma
• Always up-to-date packages, no need to manually re-trigger builds for common things

We also upgraded to newer version of Qt, Qt 5.9 from the older version of Qt 5.7.1, this aligns our stack with whatever KDE Neon dev unstable edition is using.

Rootfs for non-nexus Qualcomm devices

Non-nexus Qualcomm devices use a modified version of the hwcomposer.h which is not ABI compatible with the generic version of the android API headers, so for them the libhybris and kwin need to be separately built. Previously our CI infrastructure didn’t support this workflow, however now we are able to create the rootfs for such devices. Below is an image of Plasma Mobile running on Xiomi Mi device.

Marble maps

At Randa, we had a chance to test the newer version of the Marble maps, which has much more improved performance when zooming out-in. We were able to happily search locate Randa in the Marble maps.

QML based mobile friendly “konsole”

Previously we were using Konsole from the KDE Applications as a terminal emulator, however Konsole user interface was not mobile-friendly, and it was not possible to e.g, tab complete the command, go through history, etc. At Randa Emmanuel Lepage Vallee pointed us to qmltermwidget, which powers the terminal application in Ubuntu Touch and Cool-retro-term application. Marco Martin was able to write a simple QML application around qmltermwidget which can replace Konsole in Plasma Mobile.

Kirigami based koko

During GSoC 2017 lot of work has been put to port the normal QtQuick based image viewer application Koko to kirigami2, you can follow the work of Student developer Atul Sharma on his blog. At the Randa Sprint, we were able to test this work on Plasma Mobile and it shows great improvements compared to the older version.

Improved window switching experience

Due to a bug in the Plasma Mobile shell, shell windows such as task switcher and shell itself were also appearing in the task switcher, as a result of that user was able to close the shell window which ultimately ends up crashing the plasmashell. We were able to fix this bug in the plasma-workspace and plasma-phone-components so that Plasma Mobile shell windows would not appear in the task switcher.

Kube(!!!)

Yes, that is Kube in the first picture, we were able to run the Kube on the Plasma Mobile, it works in theory on Plasma Mobile, however the user interface is not mobile-friendly and needs various fixes to be usable on mobile. We will work with the Kube team to fix the issues and then test them on Plasma Mobile in the future.

Conclusion

That is all for now, I would like to thank the all the donors who donated for the Randa Meetings 2017, and made it possible for me to travel to Randa, Switzerland.

If you liked the work done on Plasma Mobile during the Randa Sprint, please support us on Randa 2017 fundraiser to make events like Randa possible in the future.

“Kube is a modern communication and collaboration client built with QtQuick on top of a high performance, low resource usage core. It provides online and offline access to all your mail, contacts, calendars, notes, todo’s and more. With a strong focus on usability, the team works with designers and UX experts from the ground up, to build a product that is not only visually appealing but also a joy to use.”

For more info, head over to: kube.kde.org

• We now synchronize both contacts and drafts when opening the composer. The former are required for autocompletion, the latter to show your drafts.
• A maillist filter bar to replace the currently defunct search.
• We now synchronize the currently selected folder every 5min.
• Bumped Qt requirement to 5.9 so we can finally use the focus stealing prevention of the webengineview.
• Automatically launch into the account configuration if no accounts are set-up.
• A single account mode for deployments that support only a single account. This can currently only be activated with a switch in the qml code.
• Prototyped a simple html editor, unfortunately the QML TextArea API is just not there yet and the end result ended up buggy. Postponed for now.
• Worked out a plan for gpg based encryption that is now available as phabricator tickets for the next milestone.
• Improved input form validation and feedback.
• Did some work towards improving a performance bottleneck in live queries when a lot (thousands) of updates are coming in (such as during the initial sync). Not quite there yet though.
• Did some cross compilation experiments with MXE for Windows and without MXE for Android. WIP
• Witnessed Kube on the Plasma Phone. This will require an adapted UI but generally seems to actually work.
• Fixed some layouting issues where some TextAreas wouldn’t resize with the rest of the UI.

Kube Commits, Sink Commits

Previous updates

We have a new round of Kate installers for Windows. This round comes with some installer-bug fixes (all project plugin dlls) and support for editorconfig files http://editorconfig.org/.

In the above screen-shot you can see the project plugin analyzing Kate code. This works if you have cppcheck installed and added to the path.

Grab the installers now at download.kde.org:  Kate-setup-17.08.1-KF5.38-32bit or Kate-setup-17.08.1-KF5.38-64bit

Official Raspbian images based on Debian Stretch by default has the Pixel desktop environment and will login new users to it. But if you have had a Raspbian installation with another DE (such as LXDE), here are the steps to install and login to the Pixel desktop.

1. apt-get install raspberrypi-ui-mods
2. sed -i 's/^autologin-user=pi/#autologin-user=pi/' /etc/lightdm/lightdm.conf
3. update-alternatives --set x-session-manager /usr/bin/startlxde-pi
4. sed -i 's/^Session=.*/Session=lightdm-xsession/' ${USER}/.dmrc

Make sure the user’s ‘.dmrc’ file is updated with the new startlxde-pi session as that is where lightdm login manager looks to decide which desktop should be launched.

Let me start by annoying you with some pictures:

<3 Randa

The misty mountains below which the coders dwell.

We totally had snow in September. Well, not in Randa, but in the valley next to it, but still. SNOW!

And now for the serious part: in my last blog post, I talked about achieving our main goal for this year’s Randa meetings – we successfully ported the entire Kontact away from the obsoleted KDateTime class. Since we finished this on Thursday, there was still enough time left to start working on something new and exciting.

Volker and Frederik went on to work on a KWin plugin to simulate various kinds of color blindness which will help developers to see how visually impaired users see their software, I did a bit more code clean-up after the porting and a bit of code-review.

On Friday morning Volker and I discussed search in KDE PIM. Broken and unreliable search was one of the most often mentioned issues in the KMail User Survey, which we ran for the past month and a half, so I decided to tackle the problem and make our indexing and searching infrastructure fast and reliable.

The final plan consists of several phases – starting with reorganizing our current code to put related pieces of code (like email indexing and querying code) into a single place to make the code easier to maintain. This phase is already progressing and I hope to finish it within the next week. The second phase will involve moving the code that is responsible for indexing data into the backend Resources – whenever a backend retrieves an email from an IMAP server or an event from Google Calendar it will also index it and will send the index data alongside the raw data to Akonadi. Akonadi will then take care for just writing the data into the Xapian database. This will speed up indexing, reduce the IO load and will ensure that all data are reliably indexed and stored before they are presented in Kontact. The third phase will involve changing Kontact to query the index database directly, instead of asking Akonadi to do it for us. This will speed up the search and provide results faster. The final phase will focus on which data we are actually indexing. As they say – less is sometimes more – so having fewer, but better-defined data will allow us to provide better and more exact search results to the user.

Once this is settled, we can make applications to depend on the search index – for example KOrganizer will be able to query the index to only get events from e.g. December 2017 instead of fetching all events from the calendar and then figuring out if they should be displayed or not, making calendars of even the busiest people to load instantaneously.

All in all, it was an extremely productive hackfest for the PIM team and I’d again like to thank Mario, Christian and the rest of the Randa team for organizing the hackfest. You guys rock!

And remember, you can still donate to the Randa fundraiser to make future productive sprints possible!

Last time we heard the publictransport applet was being re-written was almost a year back now. Since then, it has indeed gone through some sorts of rewrite, but at the Randa meetings, 2017, this has taken a whole new course. Since the time I began porting the publictransport dataengine, to be used with a new … Continue reading Plasma publictransport rewrite – Part II

On the morning of Day 5 we chased and fixed a problem that was introduced a long time ago but never caused any trouble. The code goes back into the KDE3 version of KMyMoney and was caused by some changes inside Qt5. The fix prevents a crash when saving a transaction which opens an additional …

Continue reading "Randa 2017 – Databases are back to KMyMoney"

Marco Martin recently posted about some of the improvements in krunner, today I want to show some of the effort into navigating the Plasma panels.

This video shows a user navigating the plasma panel using voice and keyboard. A shortcut focusses the panel, and then one can use tab and cursor keys as normal. In future we will improve our key-focus visual indicators, and allow for richer interaction.

What makes Plasma different to existing apps

Plasma in general has been a sore point with regards to accessibility as it doesn’t follow some of the exact same concepts as a traditional toolkit. Some of these convention breakages are by design, in krunner you wouldn’t want to have to tab to a list of results in order to navigate results with the cursor keys. Unfortunately these changes, if done non-optimally, really conflict with core concepts of focus and screen reader parsing. On top of that, we have the issues of an emerging toolkit, which needs that extra push to get right.

Why is this work useful?

Blind people aside, the work here has multiple other advantages.

We need better keyboard navigation, regardless. You often hear people say they prefer console apps; it’s not because they’re inherently better, but because being “handicapped” forces them to have good keyboard handling. We should be matching or beating that.

Also, approaching a large code base from a completely different angle has helped to tidy up some complex code that has built up over the years. Kickoff key handling is now not only better but the code is 2/3 the size.

—

On Day 3, we started out at 7:02 as usual with the team responsible for breakfast meeting in the kitchen. KMyMoney wise, we worked some more on keyboard navigation and porting to KF5. The dialog to open a database and the logic around it have been rewritten/fixed, so that it is now possible to collect …

Continue reading "Randa 2017 – Days 3 and 4"