Note: This is the English translation of the article first published in German

Intro

The ESE Congress is one of the lead events for Embedded Software Engineering in Germany.

This year it was held digitally for the first time, so that participation was also possible via video. Over five days, there were 3 keynotes and 96 technical presentations from all areas of embedded software development.

Anton Kreuzkamp from KDAB talked about custom code refactoring with clang tooling. Keep reading, for our presentation of his contribution to the ESE conference proceedings.

Good static analysis can save a lot of effort and time. With customized static code analysis, the project code can be checked not only for general programming errors but also for project-specific conventions and best practices. The Clang Compiler Framework provides the ideal basis for this.

The programming language C++ manages the balancing act between maximum performance, which is essential in the embedded sector, on the one hand, and maximum code correctness through a high level of abstraction on the other. The balancing act is achieved by focusing on compile-time checks and opportunities for optimization. Calculations that could be carried out more efficiently by low-level code should not, where possible, be rewritten by the developer, but by the compiler. Additionally, errors should already be excluded during compilation, instead of taking up valuable computing time for checks at runtime.

Clang has become very popular in recent years and has long since established itself as one of the most important C and C++ compilers. This success is due not least to the architecture of Clang itself. Clang is not just another compiler, but a compiler framework. The essential parts of the compiler are a carefully designed library, thus enabling the diverse landscape of analysis and refactoring tools that has already emerged around the framework based on the LLVM project.

The command-line tool, clang-tidy, offers static code analysis and checks compliance with coding conventions, among other things, but can also refactor code independently. The clang-format tool can automatically standardize the coding style. The Clazy tool, which was developed by the author’s company, supplements the compiler with a variety of warnings around the Qt software framework and warns of frequent anti-patterns in the use of the same. Many other useful tools exist in the Clang universe, as well. Even integrated development environments, such as Qt Creator or CLion, rely on the Clang Compiler Framework for syntax highlighting, code navigation, auto-completion, and refactoring.

Anyone who knows the tools of the Clang world in their entirety is well positioned as a C or C++ developer. But if you want to get everything out of the technology, that is not the end of the story. The LibTooling library, on which most Clang tools are based, also allows you to create your own customized code analysis and refactoring tools, with little effort.

I’ll give you an example. A small but recurring piece of the puzzle of embedded software is the exponentiation of real numbers, mostly with static, natural exponents. Of course, the std::pow function would be used for this, had it not been determined in extensive profiling that on-the-target architecture std::pow(x, 4) is many times slower than x*x*x and forms a bottleneck in particularly performance-critical code. The senior developer of the project has therefore created a template function, usable as utils::pow<4>(x). And thanks to compiler optimizations, it’s just as nimble as the manual variant¹. Nevertheless, since then the usual std::pow variant has crept in again at various places in the code, and even several hundred thousand lines of code have not been ported consistently.

The first attempt to automate the refactoring is, of course, the search and replace with a regular expression. std::pow\((.*), (\d+)\) already finds the simplest cases. But what about the cases where the “std::” is omitted or the second parameter is more complicated than an integer literal?

¹ Note: On many common platforms the same optimization can be achieved by using the compiler flag -ffast-math. The compiler will then independently replace the std::pow call with appropriate CPU instructions.

Installing LLVM and Clang

Those who cannot install Clang or LLVM via your trusted package manager can get the framework via Github. Prerequisites for a successful installation are Git, CMake, Ninja and an existing C++ compiler.

git clone https://github.com/llvm/llvm-project.git
cd llvm-project
mkdir build; cd build
cmake -G Ninja ../llvm -DLLVM_ENABLE_PROJECTS="clang;clang-tools-extra" -DLLVM_BUILD_TESTS=ON  # Enable tests; default is off.
ninja
ninja check       # Test LLVM only.
ninja clang-test  # Test Clang only.
ninja install

The first steps

As a basis for our own Clang tool, we use a code example from the Clang documentation. Here, it’s reduced to the essentials. [1]

#include "clang/Frontend/FrontendActions.h"
#include "clang/Tooling/CommonOptionsParser.h"
#include "clang/Tooling/Tooling.h"
#include "llvm/Support/CommandLine.h"

using namespace clang::tooling;
using namespace llvm;

int main(int argc, const char **argv) {
  CommonOptionsParser optionsParser(argc, argv,
    llvm::cl::GeneralCategory
  );
  ClangTool tool(optionsParser.getCompilations(),
                 optionsParser.getSourcePathList());

  return tool.run(
    newFrontendActionFactory<clang::SyntaxOnlyAction>().get()
  );
}

With this, we already have the first executable program. CMake makes it easy to create the necessary build scripts. All we have to do is find the Clang package and link our program to the imported targets clang-cpp, LLVMCoreund LLVMSupport:

cmake_minimum_required(VERSION 3.11)
project(kdab-supercede-stdpow-checker)

find_package(Clang REQUIRED)

add_executable(kdab-supercede-stdpow-checker
    kdab-supercede-stdpow-checker.cpp)
target_link_libraries(kdab-supercede-stdpow-checker
    clang-cpp LLVMCore LLVMSupport)

install(TARGETS kdab-supercede-stdpow-checker
    RUNTIME DESTINATION bin)

Using our development environment or the command line, we can now compile our program and run it against our code.

Before we test the newly created tool, it is recommended to install it into the same directory where the clang compiler is located (e.g. /usr/bin). This is because clang-based tools need some built-in headers, which they look for relative to their installation path, depending on the version, e.g. in ../lib/clang/10.0.1/include. In the analyzed code, for example, the header stddef.h would be missing. Those who get errors when starting the program have, in all probability, fallen into this trap.

$ cd /path/to/kdab-supercede-stdpow-checker
$ mkdir build; cd build
$ cmake -G Ninja -DCMAKE_INSTALL_PREFIX=/usr ..
$ ninja
$ sudo ninja install
$ cd /path/to/our/embedded/project
$ kdab-supercede-stdpow-checker -p build/dir fileToCheck.cpp

So far, our tool checks the syntax of the C++ file and throws errors, if, for example, non-existent functions are called. Next, we want to find the code passages that cause our problem.

Find relevant code points with AST matchers

The AST, the Abstract Syntax Tree, is a data structure consisting of a multitude of classes with links to each other, which represents the structure of the code that will be analyzed. For example, an IfStmt links to an Expr object that represents the condition of an if statement and a Stmt object that represents the “then” or the “else” branch. An AST matcher can be thought of as a regular expression on the AST; it’s a data structure that represents and finds a particular pattern in the AST. AST matchers are programmed for Clangs LibTooling in a special syntax. For each type of language construct or node in the AST, there is a function that returns a matcher of the corresponding type. These functions, in turn, take as parameters other matchers that impose additional conditions on the code. Multiple parameters are treated as an AND operation. For instance, the following code snippet creates a matcher that matches function declarations that are called “draw” and have void as the return type.

auto drawMatcher
    = functionDecl(hasName("draw"), returns(voidType()));

This fits, for example, the following two declarations:

void draw();

void MyWidget::draw(WidgetPainter *p) {
    p->drawRectangle(…);
}

In order to be able to access the individual parts of the interesting code segment later, the sub matchers can be assigned names with a bind statement, which can then be used to reference the AST node that matches the matcher. For example, if we want to find function calls whose second argument is an integer literal and want to access this later, we can prepare this with the following matcher:

auto m =
  callExpr(hasArgument(1, integerLiteral().bind("secondArg")));

A complete list of all available matchers can be found at [2].

To speed up the creation of AST matchers, Clang comes with the command line tool clang-query, which can be used to interactively test matchers and inspect the found AST section. The enable output detailed-ast command enables the output of the AST section found by the AST matcher, and the match command creates and starts an AST matcher. The syntax used in clang-query is similar to the C++ syntax.

$ clang-query -p build main.cpp
clang-query> enable output detailed-ast
clang-query> match  callExpr(callee(functionDecl(hasName("pow"), isInStdNamespace())), hasArgument(1, expr()))

Match #1:

/path/to/ClangToolingTestApp/main.cpp:27:14: note: "root" binds here
    auto x = std::pow(pi, getExp(pi));
                             ^~~~~~~~~~~~~~~~~~~~~~~~ 
Binding for "root":                                                                                                                                                                          
CallExpr
|-ImplicitCastExpr <FunctionToPointerDecay> 
| `-DeclRefExpr Function 'pow'  (FunctionTemplate 'pow')                                                                                                      
|-ImplicitCastExpr 'double' <LValueToRValue>
| `-DeclRefExpr 'const double' lvalue Var 'pi'
`-CallExpr 'int' 
 |-ImplicitCastExpr <FunctionToPointerDecay> 
 | `-DeclRefExpr 'int (double)' lvalue Function 'getExp'
 `-ImplicitCastExpr 'double' <LValueToRValue> 
   `-DeclRefExpr 'const double' lvalue Var 'pi'


1 match.
clang-query> quit

The matcher can thus be refined interactively, piece by piece. For our goal of finding calls to std::pow which can be replaced by a call to the templated function utils::pow, the following matcher is goal-directed:

callExpr(
    callee(
        functionDecl(hasName("pow"), isInStdNamespace())
          .bind("callee")
    ),
    hasArgument(0, expr().bind("base")),
    hasArgument(1, expr().bind("exponent"))
).bind("funcCall");

This matcher finds function calls to std:: pow, if it has a second argument (index 1) that is an arbitrary expression. The name of the called function is “pow” and the function is defined in the namespace, std. We title the arbitrary expression “exponent,” the called function the “callee,” and the function call itself is “funcCall.”

Analysis, diagnosis and automatic code correction

. In order to be able to do something with the found code ranges, a MatchCallback must still be registered to the matcher. The callback is a class we will implement, which is derived from MatchFinder::MatchCallback and implements the method run(const MatchFinder::MatchResult &Result). This is where our analysis of the found code snippets takes place. In addition, we define a SupercedeStdPowAction class, which (in order to be able to apply our code fixes later) derives from the FixitAction class and contains both our MatchCallback and a MatchFinder, through which we can initiate the search of the AST. Finally, we replace the clang::SyntaxOnlyAction in the main function with our SupercedeStdPowAction.

class StdPowChecker : public MatchFinder::MatchCallback {
public :
    StdPowChecker() = default;

    void run(const MatchFinder::MatchResult &result) override {}
};

class SupercedeStdPowAction : public FixItAction {
public:
    SupercedeStdPowAction() {
        m_finder.addMatcher(stdPowMatcher, &m_stdPowChecker);
    }

    std::unique_ptr<ASTConsumer>
    CreateASTConsumer(CompilerInstance &, StringRef) override {
        return m_finder.newASTConsumer();
    }

public:
    MatchFinder m_finder;
    StdPowChecker m_stdPowChecker;
};

int main(int argc, const char **argv) {
  // [...]

  return tool.run(
      newFrontendActionFactory<SupercedeStdPowAction>().get()
  );
}

We now fill the function StdPowChecker::run with our actual check code. First, we can get the AST nodes as pointers using the names assigned to the sub matchers:

const CallExpr *callExpr
    = result.Nodes.getNodeAs<CallExpr>("funcCall");
const FunctionDecl *callee
    = result.Nodes.getNodeAs<FunctionDecl>("callee");
const Expr *base = result.Nodes.getNodeAs<Expr>("base");
const Expr *exponent = result.Nodes.getNodeAs<Expr>("exponent");

The objects obtained by this means provide extensive information about the entities they represent, for example, the number, names and types of the function parameters; the type and value-category (LValue-/RValue) of the expression; and the value of an integer literal. In addition to the value of a literal, the value of any expression can also be queried if it is known at compile time. In our case, we are interested in whether the second argument could also be in a template parameter. For this, the expression must be constexpr. exponent->isCXX11ConstantExpr(*result.Context) gives us the answer. If the answer is true, we know that utils::pow is applicable and the more performant alternative.

In order to issue a warning, as we know it from compiler warnings, we use the so-called DiagnosticsEngine, which we can access via the AST context:

auto &diagEngine = result->Context->getDiagnostics();
unsigned ID = diagEngine.getDiagnosticIDs()->getCustomDiagID(
    DiagnosticIDs::Warning,
    "std::pow is called with integer constant expression. "
    "Use utils::pow instead.");
diagEngine.Report(exponent->getBeginLoc(), ID);

If we want to not only to warn, but directly improve the code, we can add a so-called FixitHint to the report. In our case, we need to reorder the arguments of the function call. To do this, we need the code of the arguments as a string. This can be achieved with the following code:

auto &sm = result->Context->getSourceManager();
auto &lo = result->Context->getLangOpts();
auto baseRng
    = Lexer::getAsCharRange(base->getSourceRange(), sm, lo);
auto expRng
    = Lexer::getAsCharRange(exponent->getSourceRange(), sm, lo);
auto callRng
    = Lexer::getAsCharRange(callExpr->getSourceRange(), sm, lo);

auto baseStr = Lexer::getSourceText(baseRng, sm, lo);
auto expStr = Lexer::getSourceText(callRng, sm, lo);

From this, we can build a FixitHint by taking the character range of the function call as input and using the argument code to assemble the new code. We can pass the FixitHint created in this way via the stream operator to the diagnostic object that the DiagEngine.Report() call created earlier. llvm::Twine helps to assemble strings efficiently.

diagEngine.Report(exponent->getBeginLoc(), ID)
  << FixItHint::CreateReplacement(callRng,
     (llvm::Twine("utils::pow<") + expStr + ">(" + baseStr + ")"
     ).str());

The Practical Test

After putting all parts together and compiling the code, we would also like to test our result on actual code. To avoid making it too easy for Clang, we pass a macro and a call to a function to std::pow, each of which can be deduced to an integer constant. In addition, we alias the standard namespace and call std::pow via it.

#include <iostream>
#include <cmath>
#include "utils.h"

#define DIM 2
constexpr int getExp(double x) {
    return static_cast<int>(x);
}

namespace StdLib = std;
using namespace std;

int main() {
  constexpr double pi = 3.141596;

  std::cout << "(2Pi)^2 = " << std::pow(2*pi, DIM) << endl;
  std::cout << "Pi^3 = " << StdLib::pow(pi, getExp(pi)) << endl;
}

If the software we’re analyzing also uses CMake as a build system, then we can get it to create a so-called compilation database with the parameter, DCMAKE_EXPORT_COMPILE_COMMANDS=ON, which our Clang tool can use to get the necessary include paths and compiler flags. We pass this database to our tool by passing the build directory where we previously ran CMake as a parameter. If this is not available, we can manually pass the compiler parameters to the tool by appending double hyphens, followed by the compiler parameters, after the source files that will be analyzed.

$ cd /path/to/ClangToolingTestApp/build
$ cmake -DCMAKE_EXPORT_COMPILE_COMMANDS=ON ..
$ cd ..
$ kdab-supercede-stdpow-checker -p build main.cpp -- -std=c++17

main.cpp:16:49: warning: std::pow is called with integer constant expression. Use utils::pow instead. 
   std::cout << "(2Pi)^2 = " << std::pow(2*pi, DIM) << endl;
                                ~~~~~~~~~~~~~~~^~~~ 
                                utils::pow<DIM>(2*pi) 
main.cpp:16:49: note: FIX-IT applied suggested code changes

main.cpp:17:47: warning: std::pow is called with integer constant expression. Use utils::pow instead. 
   std::cout << "Pi^3 =" << StdLib::pow(pi, getExp(pi)) << endl;
                            ~~~~~~~~~~~~~~~~^~~~~~~~~~~ 
                            utils::pow<getExp(pi)>(pi) 
main.cpp:17:47: note: FIX-IT applied suggested code changes 
2 warnings generated.

Conclusion

Putting all the pieces together, we have created a refactoring tool that is tailored to our project-specific needs, with just under 100 lines of code. Unlike a purely text-based refactoring tool, our implementation is capable of interpreting macros, aliases, and constexpr expressions. With Clang’s LibTooling as the foundation, the whole world of static code analysis and full code understanding is at our disposal. Via use of ASTContext, we have access to symbol tables. And with a single call to the CFG::buildCFG function, we can generate a control flow graph from the AST. The Preprocessor class allows us to inspect macro expansions and includes. In the other direction, clang::EmitLLVMOnlyAction gives us access to the LLVM Intermediate Representation, a language and machine independent abstraction of the generated machine code.

To get an overview of the capabilities of Clang’s internal libraries, the “Internals Manual” of the Clang documentation [3] is recommended. The complete code of the refactoring tool created in this article can be found at [4].

Bibliography

1. https://clang.llvm.org/docs/LibTooling.html
2. https://clang.llvm.org/docs/LibASTMatchersReference.html
3. http://clang.llvm.org/docs/InternalsManual.html
4. https://github.com/akreuzkamp/kdab-supercede-stdpow-checker

Author

Anton Kreuzkamp is a software developer at KDAB, where he develops, among other things, tooling for the analysis of C++ and Qt-based software and works as a trainer and technical consultant. KDAB is one of the leading software consulting companies for architecture, development and design of Qt, C++ and OpenGL applications on desktop, embedded and mobile platforms. KDAB is also one of the largest independent contributors to Qt. KDAB’s tools and extensive experience in building, debugging, profiling and porting complex applications help developers worldwide to realise successful projects.

Do you need similar support?

If you want to solve a similar problem in your software project, don’t hesitate to Contact us.

The post Can’t the Compiler Do That? appeared first on KDAB.

Breeze Redesign and Blue Ocean

Do you know a person, project or organisation doing great work in open tech in the UK? We want to hear about it. We are looking for nominations for people and projects working on open source software, hardware and data. We are looking for companies or orgnisations working in fintech with open, helping achieve the objectives of any of the United Nations Sustainable Development Goals. Nominations are open for projects, organisations and individuals that demonstrate outstanding contribution and impact for the Diversity and Inclusion ecosystem. This includes solving unique challenges, emphasis transparency of opportunities, mentorship, coaching and nurturing the creation of diverse, inclusive and neurodiverse communities. And individuals who you admire either under 25 or of any age.

Self nominations are welcome and encouraged. You can also nominate in more than one category.

Nominations may be submitted until 11.59pm on 13 June 2021.

Awards Event 11 November 2021.

Those categories again:

Hardware – sponsored by The Stack
Software – sponsored by GitLab
Data
Financial Services – sponsored by FINOS
Sustainability – sponsored by Centre for Net Zero
Belonging Network – sponsored by Osmii
Young Person (under 25) – sponsored by JetStack
Individual – sponsored by Open Source Connections

Read more and find the nomination form on the OpenUK website.

Winners of Awards 2020, First edition

Young Person • Josh Lowe
Individual • Liz Rice
Financial Services and Fintech in Open Source • Parity
Open Data • National Library of Wales
Open Hardware • LowRISK
Open Source Software • HospitalRun

We are pleased to announce a new stable release of MauiKit and Maui Apps!

Stable release

This version of Maui brings new features and bug fixes to Maui’s applications and the frameworks they rely on. The changes introduced in this release will make your experience with apps like the Index file manager, Vvave music player, and Nota text editor much more enjoyable and fun.

Read on to find out how Maui’s software has improved and what new features to look out for:

Maui Weekly Report 9

Maui Weekly Report 10

Maui Weekly 11

MauiKit 1.2.2

MauiKit is the framework used to build Maui applications. It comes with common widgets crafted to work well on small phone screens and bigger desktop computer monitors. But that’s not all: MauiKit also comes with a set of utilities that help developers build applications more quickly with more complex and ready-made widgets. These include a text editor, an image viewer, or a file explorer, among others.

In this updated version, the graphical components have been polished. New widgets have been added to the MauiKit Core library, making navigation easier on phones, desktop computers, and other devices. On Android, all the Maui Apps now make use of a dark style.

The following video goes over a few of the newly added changes to the MauiKit framework that directly improve all of Maui’s apps.

Maui Apps

The Maui Project provides a set of applications to cover your phone, tablet, or desktop needs. The Maui line of apps currently includes a file manager (Index), a music player (Vvave), an image viewer (Pix), a text editor (Nota), an app for note-taking (Buho), a terminal emulator (Station), an app to manage your contacts (Communicator), a document viewer (Shelf), a video player (Clip), a camera app (Booth), and a web browser (Sol).

The Index file manager comes with a quick typing search feature that allows you to find files more easily as you type on a desktop computer with a keyboard. This feature is also found in Pix and other image gallery apps and in any other apps in general that allow you to browse your files. Index also includes an Overview section that shows you the most recently downloaded files, or your most recently accessed images and audio files, as well as your favorite files.

The Vvave, the music player, lets you edit music files’ metadata information within the app and get information about the current playing artist’s and album’s song. Pix, the image viewer and gallery manager, has a similar feature and can display EXIF metadata in a dialog with information about the camera’s model, ISO, and other technical details. Pix also allows you to edit images by doing transformations such as cropping and rotating.

The Nota text editor adds a Find & Replace feature and gives you an inline notification when the document you have opened has been modified or removed by another application. The improvements made to the text editor are part of one of the MauiKit sub-frameworks, so these features are also included in the new version of Buho, the note-taking app.

The MPV library powers the Clip video player. This allows you to play remote videos, such as videos from popular websites like YouTube. In fact, Clip comes with a dedicated view from where you can search and open videos from YouTube.

The terminal emulator is Station, and it’s a super touch-friendly console. This new version comes with an easy way to add command shortcuts to save typing on handheld touch devices like phones or tablets.

All of the apps that make use of tab views now use the MauiKit convergent TabView widget. TabView switches from a mobile-friendly tab system to the common tab style used on a desktop computer, depending on where you are using the app.

You can now try out all these changes on Android and Linux phones, tablets, and desktop computers.

You can find a more detailed list of the changes mentioned below:

Index 1.2.2

Download Packages | Open a Report

Highlights:

•  Comes with the new convergent tabs view.
•  Adds quick typing to jump search on desktop.
•  Allows mounting removable devices.
•  Introduces multiselection gesture on touch-based devices — hover over items to select them.
•  The embedded terminal is now available if the QMLTermWidget is installed.
•  The navigation history issues have been fixed.
•  The previewer works now on the Overview items.

Index 1.2.2 on Desktop and Phone.

Vvave 1.2.2

Download Packages | Open a Report

Highlights:

• Comes with a new metadata editor.
• Adds quick typing search on albums and artists’ views.
• Allows you to retrieve information about artists and albums.
• Adds a repeat playback option.

Vvave 1.2.2 on Desktop and Phone.

Pix 1.2.2

Download Packages | Open a Report

Highlights:

• Introduces an image editor with rotation, cropping, and other transformations.
• Can display EXIF metadata information in a dialog.
• Adds quick typing search.

Pix 1.2.2 on Desktop and Phone.

Nota 1.2.2

Download Packages | Open a Report

Highlights:

• Introduces a new Find & Replace feature.
• Comes with the convergent tabs view.
• The alerts header that informs of file changes has now been fixed — included from MauiKit::TextEditor.
• The embedded terminal is now available if the QMLTermWidget package is installed.

Nota 1.2.2 on Desktop and Phone.

Buho 1.2.2

Download Packages | Open a Report

Highlights:

• Adds a confirmation dialog for removing notes.
• Adds a more coherent SelectionBar list item delegates.

Buho 1.2.2 on Desktop and Phone.

Station 1.2.2

Download Packages | Open a Report

Highlights:

• Adds controls to trigger and save frequently used commands quickly.
• Now uses the new convergent tabs design from MauiKit.

Station 1.2.2 on Desktop and Phone.

Communicator 1.2.2

Download Packages | Open a Report

Highlights:

• Updated to the latest MauiKit controls
• We’ve moved code from MauiKit to fetch contact info from Android API to Communicator’s source codebase.

Clip 1.1.1

Download Packages | Open a Report

Highlights:

•  It uses the libmpv as its backend

Clip 1.2.2 on Desktop and Phone.

Shelf 1.1.1

Download Packages | Open a Report

Highlights:

• Updated to latest MauiKit controls.

For this release, We will distribute the packages directly from the MauiKit official webpage.

The packages for Linux AMD and ARM AppImages, and Windows EXE, will be released soon. We will keep you informed when this happens.

Some apps might be missing features or present some bugs; if it is the case that you want to report a bug or a feature request, you can open a ticket at the corresponding project repo. You can access repositories from the Maui Invent Group.

Release cycle

Are you a developer and want to start developing cross-platform and convergent apps, targeting, among other things, the upcoming Linux mobile devices? Then join us on Telegram: https://t.me/mauiproject.

If you are interested in testing this project and helping out with translations or documentation, you are also more than welcome.

The Maui Project is a free and open-source software from the KDE Community developed by Nitrux Latinoamericana S.C.We are present on Twitter and Mastodon:

• https://twitter.com/maui_project.
• https://mastodon.technology/@mauiproject.

The post Maui 1.2.2 Release appeared first on Nitrux — #YourNextOS.

Nitrux is a Linux distribution that offers a relatively up-to-date system, including, of course, the latest stable version of Plasma and the KDE Frameworks, which comes by default in the distribution. However, it’s also possible to further test the beta releases and even the Git builds, all thanks to the hard work of the KDE Neon developers. This is fantastic since we can enjoy all the new features that the KDE community is working on before they’re tagged as stable, for example, the floating dock feature in Latte Dock.

In this tutorial, we’ll update our Plasma and KF5 Frameworks all the way to the Git builds. It goes without saying that you should only do this at your own risk.

With that said, we’ll start by updating to dev/stable and then to dev/unstable.

Nitrux 1.4.0 w/ floating Latte Dock.

Difficulty: ★☆☆☆☆

First, we’ll create a copy of the APT sources file that we’ll be using. To do this, do the following.

• sudo cp /etc/apt/sources.list.d/neon-user-repo.list /etc/apt/sources.list.d/neon-dev-stable-repo.list

Then, in a Terminal, we’ll use sed to edit the source file and change the repository. To do this, do the following.

• sudo sed '+s+/user+/dev/stable+g' /etc/apt/sources.list.d/neon-dev-stable-repo.list

Now, we’ll disable the stable Neon repository (user). To do this, do the following.

• sudo sed '+s+deb+#deb+g' /etc/apt/sources.list.d/neon-user-repo.list

Then update the package index and proceed to apply the upgrades. To do this, do the following.

• sudo apt update && sudo apt dist-upgrade --no-install-recommends

You can now log out and log in to load the new libraries and software.

Finally, we’ll do the same steps and create a new sources file and disable the dev/stable repository.

Create a copy of the APT sources file that we’ll be using. To do this, do the following.

• sudo cp /etc/apt/sources.list.d/neon-user-repo.list /etc/apt/sources.list.d/neon-dev-unstable-repo.list

Then, in a Terminal, we’ll use sed to edit the source file and change the repository. To do this, do the following.

• sudo sed '+s+/user+/dev/unstable+g' /etc/apt/sources.list.d/neon-dev-unstable-repo.list

Now, we’ll disable the dev/stable Neon repository. To do this, do the following.

• sudo sed '+s+deb+#deb+g' /etc/apt/sources.list.d/neon-dev-stable-repo.list

Then update the package index and proceed to apply the upgrades. To do this, do the following.

• sudo apt update && sudo apt dist-upgrade --no-install-recommends

You can now log out and log in to load the new libraries and software. Besides updating Plasma and the KDE Frameworks, you will update the default KDE Gear applications like Ark.

Note: The KDE Neon repository uses the same ‘Suite’ and ‘Codename’ names as Ubuntu does (https://wiki.debian.org/DebianRepository/Format#Suite and https://wiki.debian.org/DebianRepository/Format#Codename) in this case ‘focal.’

The KDE Neon developers also use a custom version number for the packages they build, i.e., “5.81.0-0xneon+20.04+focal+release+build20,” packages from Debian would say “5.81.0,” both Debian packages being the same version of the same software.

The KDE Neon repository is completely independent of any Ubuntu infrastructure. The Ubuntu maintainers do not manage it; as such, it’s not an Ubuntu repository.

Please be aware that the default source “neon-user-repo” will be overwritten when the package nitrux-repository-config is updated. You will have to manually disable it again to avoid mixing packages from the ‘user’ and ‘dev/unstable’ repositories.

That’s it; this concludes today’s tutorial.

The post Testing the latest Plasma and KF5 in Nitrux appeared first on Nitrux — #YourNextOS.

Here’s our first update on how the work for Krita 5 is going!We’re bringing you these updates to show what’s going on in Krita and why it’s a good idea to join Krita development fund! But first a short video by Ramon Miranda showing one the new features in Krita 5: the painting recorder. The painting recorder was created by Shi Yan and improved by Dmitry Utkin. Eoin O’Neill is sponsored to improve the interaction with ffmpeg. Also, looks at the nice new icons created by Timothee Giet!

This is the last month when we can still add new features. From June 1st, it’ll be bug fixing only. Not all features start with sponsored developers: many new features are coded by volunteers. Our sponsored developers then work with the volunteers to get the features ready. So here’s what got in last week:

• Working together with Dmitry and Emmet, Know Zero completed a new feature: importing animations as frames, like animated gif files or mp4 clips. He also made it possible to export animated gifs with transparent backgrounds.
• Then there’s Nabil Magfur Usman’s 2 point perspective grid assistant
• Deif Lou added new texture blending modes for brush presets like hard mix, color dodge, color burn, overlay. Here is an in-depth discussion. Test image done by Hulmanen:
• Agata Cacko was sponsored to work on the MyPaint brush engine. The MyPaint brush engine was created by Ashwin Dhakaita for the 2020 Google Summer of Code program. There were still a few things missing, like support for mirrored painting, and that’s what Agata added. Agata also worked on a new resource import dialog, as part of the resource rewrite.
• Sharaf Zaman was sponsored to work on Krita for Android and ChromeOS. He implemented window management for dialog windows in Krita, so the dialogs can be resized and moved around. And he fixed a lot of Android and ChromeOS specific issues as well!
• Ivan Yossi was sponsored to work on Krita on macOS. He fixed the quicklook and finder preview plugins. We’ll have to make a Krita 4.4.4 release for this, though. And now he’s investigating whether we can finally speed up the GPU canvas on macOS.
• Felime Lema moved loading the thumbnails for the recent files in the welcome screen to a background thread, which solves the slow startup of Krita if you have ginormous tiff files recently used.
• Dmitry Kazakov was sponsored to work on brush engines. Specifically, he worked on a new smudge brush engine. He also restored the cumulative undo feature that got broken in 2015.
  
• Reinold Rojas has been working on improving the grid brush engine: he added a shortcut for changing the offset of the current grid brush. It’s still work in progress!
• Emmet and Eoin have been sponsored to work on changes in the way onion skinning works, so people can use reference frames. That might only land in Krita 5.1, though…
• Wolthera has been sponsored to work on the manual: we want the manual up to date when we launch Krita 5, and there are so many changes, and so many new things to describe.
• Halla has been sponsored to work on making sure that we have a good default brush preset selected when Krita starts for the first time, and on making it possible to create new versions of brush tips.

So, if you like what we’re doing, join the development fund!

The post Krita Dev Fund Campaign — First Update! appeared first on Krita.

(or: “Accidental Autobiography”)

Can you believe how time flies? Today, ten years ago my first ever KDE patch was merged. A little while later I was granted KDE developer rights with write access to all of KDE’s git repositories. This power was somewhat frightening, after having submitted not even a hand full of patches at that time, and it actually took many years for the thrill of hitting Return on a “git push” to abate. Let me take this decennial as an opportunity to tell you stories from back in the days and how I ended up where I did, where I surely would not be without KDE!

I actually started writing on this blog post last December, to have plenty of time for collecting trivia and ideas, never before seen prototype screenshots, and more. I surely wouldn’t have thought this to turn into half an autobiography. Mind that I’ll try my best to verify the statements that follow but they can still be inaccurate or skewed from being just memories. Now grab a cup of your favorite beverage, sit back, and join me on this trip down memory lane.

My earliest recollection of using KDE was in 1999 on SuSE Linux 6.1. Back when the “KDE Desktop Environment” was the actual product name, which is why you still hear people say “I use KDE 5” today. I still fondly remember how it had an isometric K for its start button rather than the gears logo we know and love today. I was also quite fascinated by the green and blue, depending on your edition, crystal formations on SuSE packaging – the physical cardboard boxes, that is. You could even find those 2 kg boxes that featured a tome of a manual and several CD-ROMs at your local electronics store.

I still used Windows as my main operating system, though, originally Windows 95 and Windows NT 4.0, later Windows 98 because a video editing software I wanted required it, and finally Windows XP, which would be the last Windows release I used on a daily basis. I occasionally toyed around with later SuSE releases, I went to 6.4 and then 7.1, which was the first one to come on a DVD. We didn’t have a DVD drive back then, so I had to put up with the 7 or so CDs it came with. For some reason I don’t remember we had to reinstall the system several times and at some point we created a backup copy of the first CD, and as if we had known, it actually shattered in the drive on the next installation attempt.

At some point during the KDE 3 days I managed to wipe my hard disk while playing around with “dd” and was so upset that it didn’t warn me that I was about to override my “/dev/hda” with zeroes (hello, sudo!), that I turned my back on Linux for a while. I still had it running on my second PC but my main one was Windows XP only – after restoring a backup. Still, some school project I was tasked to do over the summer holidays was irrecoverably lost.

Going Linux full time

It wasn’t until July 2009 that I made the switch to Linux full-time. My computer was giving me random blue screens quite frequently and at some point I was so done with it that I just wiped the hard disk and installed Linux on it: Kubuntu 9.04 with Plasma 4.2.2 it was. I actually missed that whole 4.0 debacle, and from what I heard, I might not even be around now if I had :-) I just remember reading an article about 4.0 on heise, thinking “gosh, that black panel is horrendous.”

When Kubuntu wasn’t any more stable either, I thought to run a memtest, et voila: one of the RAM banks had gone bad, resulting in recurring page faults. At that point, however, Windows was already gone and I tried to just deal with the new operating environment. I believe that might actually have worked… Immediately I started tinkering with it, enabling backports repositories, and trying out beta releases. Reading Planet KDE became part of my morning routine, checking out what all those talented people came up every day. I’m sure many others out there are also being inspired by those blog posts, and I feel guilty for not blogging a whole lot more! One post I distinctly remember was about a rewritten Widget Explorer, albeit apparently from a later date than I thought it was until I just looked it up.

At this point I already had a 24″ monitor (16:10! Before that 16:9 plague took over consumer monitors) and that ATI All-in-Wonder Radeon 9000 with 64 MB of RAM just wasn’t able to run Compiz, Emerald, and whatever else was hot at the time, smoothly, and I never managed to get its analog TV tuner working on Linux either. Eventually, I got myself a shiny new computer with a Core i5-750 and I started playing with kdesrc-build, remotely monitoring its progress from school on my first Android smartphone, looking at a terminal window on my desktop via VPN because, hell, I had no idea SSH was a thing.

I also wanted to contribute to KDE but all I had programmed with before was Visual Basic, QBasic, BlitzBasic, and PHP – I surely didn’t know anything about C++, so I started by filing bug reports. One of my earliest bug reports I found was actually about notifications, go figure. Later I joined the German translators to whose mailing list I am still subscribed today and I occasionally send them bugs, but honestly I just can’t be bothered to use SVN anymore.

My first patch!

I’ve always been a visual person, wanting to provide a graphical user interface for the things I did, but since I didn’t know any of the programming or scripting languages needed to bring up a proper dialog window, I wrote horrible Bash scripts, with kdialog to provide some kind of graphical user interface, and I was a huge fan of the “–passivepopup” option resulting in a Plasma notification popup. I start to see a pattern here! The “–icon” command line argument affected the application’s windows, such as message boxes, but the notification popup was hard-coded to a generic “dialog-information” icon.

I can’t actually recollect how I wrote the patch or built it, where I looked for how to provide a patch – KDE was using Review Board back then, so the process has changed a lot in the meantime, but documentation has improved, too. Today we have a wonderful Get Involved page on our Wiki, and you don’t necessarily have to write code, there’s lots of other fun ways to contribute! It took me a while to find the original commit as I totally forgot it was actually committed by Christoph Feck. It wasn’t until my second patch (I think) that Aaron Seigo nudged me to apply for a developer account.

Browsing through the unmaintained projects history on GitLab shows that I was a lot more active that summer than I remembered. For instance, fascinated by logging into Windows 7 on a friend’s new computer, I added an option to KWin’s login effect to let KSplash fade to black and then to the desktop rather than slowly dissolving (the transition also used to be a lot slower, as in 2 seconds or so, than it is today). The option ended up not turned on by default as it was deemed too big of a break in visual continuity from login screen to desktop or something but the option still exists today, and I even kept it when I ported the effect to JavaScript two years later.

In July I attended my first KDE event, the KDE 4.7 release party in Karlsruhe. It’s when I met some of the people behind KDE for the first time: Lydia Pintscher, Martin Flöser, Sput, Torch (RIP), and a few others. Needless to say, we all sat there at a long table in a corner at Vogelbräu with our laptops out, and Torch showed how he could change virtual desktops by tilting his laptop, using the HDD’s fall damage protection accelerometer to trigger that. The next month I did a family roadtrip on the North American East coast, and I still recall sitting on the bed in a motel somewhere on the outskirts of Montreal watching an interview by Martin (on my newly acquired and later banned Samsung Galaxy Tab 10.1) about KWin on the Road to Wayland during the 2011 Desktop Summit in Berlin.

Drawing bars in Inkscape

Later that year I began studying computer sciences and if you decide on which university to actually go to on the very last minute, it’s surprisingly hard to find an apartment, duh. This meant I spent my first semester in a shabby Autobahn motel, 45 minutes away from uni by car, or like three hours by public transport.

The smartphone revolution still felt pretty much in its infancy, the Nokia N9 was still a thing, I was still using my first ever smartphone, and Android just learned to do hardware-accelerated 2D painting on phones with Android 4.0 released in October. Free Wifi wasn’t as ubiquitous as it is today and my apartment did not feature any Internet connection whatsoever. Mobile reception on the Autobahn in the middle of nowhere can still be a challenge in Neuland today but I managed to find a carrier which got me a decent 3G connection using a USB modem stick. For maximum coverage, I plugged it into a USB extension cord and stuck it to a window.

To be honest, I did not expect that it would just work under Linux. However, once I plugged it in, Plasma prompted me for the PIN and after selecting the correct carrier from a list of providers, I was well under way. This is when I noticed the Network Manager icon in the panel would just show a generic cell phone icon when connected via mobile broadband but I wanted signal strength bars like on my phone. Patching the network applet itself was straightforward but now I had to fire up Inkscape and draw a few dozens of signal strength bars for GPRS, EDGE, UMTS, from 0%, 20%, 40%, etc. Of course I had no idea you still had to pixel-align lines in an SVG to make 1 unit lines render crisply but my patch was accepted nonetheless and there was no VDG as we know it today to point out the blurry edges, so I quite proudly sat there in my room moving my antenna about, rejoicing at the signal bars fading in and out.

Obsessed with pixels

I failed to recollect what I actually did in 2012 and git history confirms that I wasn’t very active in KDE at that time. My highlight of that year was the KDE 4.8 Release Party in Karlsruhe in January, though. This time I didn’t have to drive home for the night but slept on Sput’s couch and after returning from the bar we spent the rest of the night watching StarCraft 2 replays. A while later I “Joined The Game” and set up a recurring, tax-deductible by the way, donation, to the KDE e.V. to help support my favorite open source community. In Summer I bought one of those newfangled Ultrabooks, an Asus ZenBook, which came with a FullHD screen – if you ever wondered where my obsession with pixels came from, this was it. Qt didn’t gain proper high dpi scaling support until Qt 5.4 so I started an effort to address bugs with large fonts caused by hard-coded layout and icon sizes throughout KDE libraries and applications.

As part of that I set out to fix the icon sizes in the icon dialog to follow your preferences. Partway through I realized I might as well clean up the code base and revamp its UI entirely, as the dialog still looked exactly from when I was a kid toying around assigning all sorts of icons to random folders on the system, which disappointingly didn’t show up when I booted back into Windows. Unfortunately, there’s quite a few different modes and flags the dialog supports; add to that complexity of custom Qt item view delegate painting and, I think, a looming feature freeze in preparation for KDE Frameworks 5, and I just never finished it. I still quite like the UI I came up with, so feel free to take it as an inspiration and rewrite the damn thing already!

Batteries and more!

2013 was the year I really got sucked into this KDE phenomenon. I did an apprenticeship at a web design agency but didn’t want to use their iMacs so I did BYOD before it was cool. I consider myself quite lucky to have been able to use Plasma on all the jobs I’ve had so far, apart from temporary project assignments, which were actually a welcome change to check out what nifty features I could steal get inspired from for Plasma.

Obviously, I needed an external mouse to work, so I paired one of those dreadful Bluetooth Apple Magic Mice they had lying around in the office with my laptop. That’s when I noticed that it was actually reporting its battery status to the system. However, Plasma could not cope with the fact that there were two batteries present, only one of which actually powered the system: it would take the average of laptop and mouse battery and display nonsensical charge percentage in the panel. I naturally couldn’t accept the fact that OS X could handle that but “we” couldn’t.

I learned that UPower added an “is power supply” property that indicated whether a battery was actually powering the device or merely reporting status of a connected peripheral, which I added to KDE’s Solid hardware abstraction framework – of course accidentally breaking ABI and builds for any other platform than Linux in the process. After cleaning up the mess I made I then patched Battery Monitor to handle those devices correctly as well as have PowerDevil’s (Plasma’s power management service) “low battery” calculation take this property into account. Furthermore, I also added a dedicated notification to warn when your mouse or keyboard are about to run out of juice.

Soon I realized the overall Battery Monitor layout could use some serious overhaul, and I began rewriting it basically from scratch, including the keyboard brightness support someone added to PowerDevil around the same time. I was a huge fan of Daniel Nicoletti’s work at the time and took inspiration from his then-new Print-manager applet: the battery list with its expandable details section was pretty much copy-pasted from it. KDE SC 4.11 to be released later that year was the final Plasma 4.x feature release, so I made sure my new Battery Monitor was in good shape for inclusion, or else it would have had to wait another year to see the light of day.

While I believe the layout fit Plasma 4 very well, it feels quite out of place on Plasma 5, having remained virtually unchanged from its initial release, compared to all the other applets that over time got polished as part of the KDE Consistency Goal. I would love to give it another complete overhaul at some point but so far I found none of the design suggestions people brought to me compelling. Finally, when KDE SC 4.11 came out in August 2013 I invited to a release party dinner in Stuttgart, trying to replicate the fun events I attended earlier. Alas, only two other people showed up. It was still a nice evening with good food and – of course – we had our laptops out and were playing around in VirtualBox with then-new Windows 8, giggling at its hideous non-intuitive Metro user interface.

Attending my first ever sprint

In December 2013 I stayed at Aaron Seigo’s place over the weekend and we visited Zürich Christkindlimarkt. They put up a massive Christmas tree decorated with Swarovski crystals in the platform hall of Zürich central station, alongside a bunch of christmas-markety boths. I believe I paid 8 CHF for a cup of Glühwi there. It was also the time KDE Connect was very new, so when Aaron left to visit his parents-in-law that Sunday, and called me to let him back in, I was perplexed to see a notification on my laptop that my phone which sat on the kitchen table was receiving a call. I just had both phone and laptop on the same WiFi network and the rest was magic – it truly felt like the future!

The next year, Alex Fiestas invited me to my first in-person KDE sprint in Barcelona’s Blue System offices in April 2014. It was the KDE Frameworks sprint during which important decisions, such as the rapid monthly release cycle, were decided upon. Having been my first real solo trip abroad, I was quite nervous. I will never forget that unease feeling of “how come my backpack is so light?” while waiting at the bus stop. It quickly dawned on me that I forgot to pack my laptop charger! USB-C power delivery is becoming ubiquitous nowadays but surely, not bringing that giant Asus charger brick (which is the most annoying thing on a sprint as it basically occupies three outlets on an extension cord) would have been a disaster. I lived close by that bus stop, luckily I was there early, and still had time to run back and fetch the charger. This was the day I started using checklists for everything.

When I arrived in the afternoon I met for the first time Alex Fiestas, David Faure, Kevin Ottens, David Edmundson, and others I can’t remember :) I believe I shared an apartment with Mark Gaiser and Alex Merry for the week. During the sprint I worked mostly on Solid again, trying to build a proper QML import and model for querying and enumerating hardware. However, I wasn’t very experienced with Qt yet and completely lost when the objects in my model, managed by a shared pointer in C++, would randomly disappear on the QML side when the ListView created and destroyed items as you scrolled through the list. That model was never finished in all these years but I assume the remnants of it can still be found in some abandoned branch in Solid’s repository.

An amusing coincidence was that a group of students from my former school was in town that week. I’ve been to Barcelona twice before from student exchange programmes, so I joined them for that obligatory visit to the Museu Nacional d’Art de Catalunya. Sunday morning I had to catch my flight back around noon but left the apartment a little late and then had huge trouble finding the right bus stop for the Aerobus at Plaza España, because of course it’d be opposite of where I arrived, or so I thought. Having missed two buses and the ride taking more than half an hour, time was running out. Naturally, when I finally got to the airport, there was a huge queue at the check-in desk. Fortunately, there had been a printer at the office, so I was checked in online already and got a sheet of paper with my boarding pass on it (I did not trust my phone with that sort of thing then) and arrived at the gate just as boarding was commencing.

My first Akademy

In Summer 2014 Jos Poortvliet persuaded me to sign up for my very first Akademy, KDE’s annual developer conference, in Brno, Czechia in September. Getting there was quite a chore, as I had to fly into Vienna and then proceed to Brno by coach. That bus departed every other hour but I didn’t want to risk missing it so I had to spend three hours at Vienna airport. I still vividly remember a thread on the Akademy mailing list about “bring your own toilet paper” and indeed the hotel we stayed at featured the roughest, grayest recycling tissue I have ever seen. The week was great fun with delicious food and made me wonder why I didn’t go to an Akademy sooner – make sure, whenever we can meet again in person that you, dear reader, go, too!

We have now arrived in the year 2015 where I will wrap up this post. This blog instance was set up in that year and from now on you’ll be able to follow my development on your own. While there are surely many more fun stories and anecdotes to tell, there has to be some material left for a sequel ;) I hope you enjoyed this history lesson and I can only encourage you to do the same, reflect on what you’ve achieved and tell the world, inspire others! Bhushan Shah once told me how some of my posts on Google+ years ago got him into KDE and can you imagine KDE and Plasma Mobile without him? I owe my entire professional career, pretty much all my debugging, and programming skills to all those talented people in the KDE Community.

Plasma Mobile team is happy to announce PlaMo Gear release service.

This releases are aimed towards providing monthly release for smaller applications which make important part of Plasma Mobile experience.

You can find details about sources in information page

The Annual General Meeting (AGM) of KDE e.V., the association that supports the KDE community financially and organizationally, has been announced. As usual, it takes place at Akademy – online again this year, and the AGM is on monday june 21st, the longest day in the northern hemisphere – and will hopefully consist of quick and boring financial stuff, followed by a spirited discussion on the goals and activities and working groups of the e.V.

KDE Gear 21.04 including Kate out in the wild!

KDE Gear 21.04 was released some weeks ago.

If you use some distribution like e.g. the rolling release Arch Linux (or Arch derivates like Manjaro), you might already have updated to Kate from that release.

As show in the 21.04 pre-release post, Kate 21.04 really has some nifty new features and improvements on all fronts over 20.12.

Window Store Update => Done!

Unfortunately, unlike for the Linux (or BSD and alike) distributions, where the hard working packagers do the work for us, for e.g. the Windows Store updates, we must do this on our own.

Naturally, thanks to the nice infrastructure like Craft and the Binary Factory, this is no purely manual work.

It more or less just is picking some nightly build and ensuring it properly works.

Thanks to the people in the Kate team, this time, not just me tried the version out (or just me and Hannah .)

And yeah, we found some crashs that need fixing before we could upload the version. (interesting enough in the KTextEditor framework, that behaves just fine on Unices…)

All these fixes will be in the next KDE Frameworks release, until then, Craft blueprints will contain the needed patches.

Kate 21.04 should now be in the Windows Store for all regions. If you don’t know the box art picture there, head over to our mascot page.

Git Integration on Windows => There we are!

My most loved new feature (beside the faster search in files & polished quick open) is the nice Git integration we now have.

You want to create commits in Kate? Check.

You want to push stuff in Kate? Check.

You want to create a feature branch in Kate? Check ;=)

You want to do this on Windows => Yeah, that works perfectly as long as the Git client is installed and in your PATH.

If you use Windows (and you use the store version), I hope this update will help you to use Git with Kate on Windows.

If you want to installed 21.04 in a different way, head to our Get Kate page, using the Windows Store isn’t mandatory at all!

If you use e.g. macOS, yes, the latest stable build will provide naturally all features there, too!

Windows Store statistics

To conclude this post, let’s take a look at the overall store statistics for the KDE e.V. published applications.

Below the overall acquisitions since the applications are in the store, sorted by overall number:

• Okular - More than a reader - 95,101 acquisitions

• Kate - Advanced Text Editor - 95,086 acquisitions

• Filelight - Disk Usage Visualizer - 22,430 acquisitions

• Kile - A user-friendly TeX/LaTeX editor - 11,921 acquisitions

• KStars - Astronomy Software - 5,473 acquisitions

• Elisa - Modern Music Player - 5,221 acquisitions

• LabPlot - Scientific plotting and data analysis - 659 acquisitions

As you see, both Okular & Kate now nearly have 100,000 acquisitions, I think that is rather nice!

Compared to my post end of 2020, Okular has the first time overtaken Kate in the total acquisitions.

In any case, these numbers show that not just a few people really use our stuff on Windows, nice to see this work is appreciated.

Comments?

A matching thread for this can be found here on r/KDE.