July 08, 2019

OpenExpo is an event aimed at businesses and the public sector. Top topics usually revolve around cloud computing, big and open data, IoT, and as of late, blockchain technologies. 2019 was its sixth edition, held on the 20th of June in “La Nave” on the outskirts of Madrid.

Organisers tell us that 2800 visitors attended this year’s event. There were about 120 speakers and 70 exhibitors with booths. From what we could garner, most visitors were representatives of public institutions, consulting companies, and software development companies, especially from the field of cloud computing.

The Booth

KDE’s booth was right next to the entrance; on the right as you went in, in an area called the “Innovation & Community Village”. We were one of five exhibitors in the area. On our right was the FSFE. I happened to know one of the people staffing, which was nice.

KDE’s booth at OpenExpo 2019.

Behind us was a father-and-son outfit showing 3D printers. Apart from owning a shop, they apparently run courses in their neighbourhood, and that is what earned them a spot in the “Community Village”.

Then there were some people with a DIY go-kart/scooter/tricycle thingy (?). They opened a big, colourful box full of interesting-looking pieces, didn’t do anything with them, and then left.

Finally, on the other side of our table was a company/community that virtualised desktops in the browser. Interesting stuff.

There were six tables and it was first come, first served. I was first, so I picked a front-facing table. Each table was 180 by 80 cm, which is big compared to what we often get in other events, and gave us plenty of space to set up our things. There was a space for our banner in a corner, as you can see in the photograph. We added a screen on a stand behind us that ran videos showcasing Plasma, Plasma Mobile (PlaMo), Kirigami and Applications on a loop. You can see the screen in the background of the photo.

On the table, we laid out the following items:

From left to right, a Nexus5X phone running Plasma Mobile, a Raspberry Pi with a touch screen also running Plasma Mobile, the Pinebook 14” $99 netbook, and a KDE Slimbook II.

We also had 100 stickers: 50 stickers of Katie using a phone and with the Plasma Mobile URL, and 50 Konqi stickers with KDE.org URL. The Konqi ones ran out first.

Katie and Konqi stickers.

The aim of our table spread was three-fold. First, we wanted to show people “shopping” for software that Plasma and other KDE applications are “end-user ready”. Secondly, we intended to show how Plasma is light and can work on a wide variety of devices, including devices usually used in setups where embedded electronics are required (the Raspberry PI); low-powered, ARM-based netbooks (the Pinebook); and as a potential mobile environment (the Nexus 5X). Finally, we wanted to demonstrate how applications, thanks to Kirigami, can adapt to different hardware and screen configurations.

The overarching aim was to see if we could convince administrators of large deployments (for example, schools) that Plasma and KDE Applications would be a good choice for their users. We were also seeking contributors and sponsors for KDE, and looking to convince companies that KDE has good solutions for developing graphical applications.

What I did

To attract and engage visitors, I used several tactics I had used in the past, and that seem to work well. I stood outside the booth and approached visitors that showed interest in our spread.

I found out where the visitors were coming from and adapted my spiel to that. I demoed Plasma on laptops for administrators of large deployments, showing off features and pointing out how it was fast and snappy even on low-spec hardware.

I showed the proof-of-concepts of Plasma Mobile on Yocto (Raspberry Pi) and on postmarketOS (Nexus 5) to managers of companies that developed for several platforms. They could check for themselves how Kirigami could let them create cross-platform applications, including for Android (I had my own phone on hand for this), and how it would allow them to create applications that would adapt to different sizes of screens.

At the end of each demonstration, I encouraged visitors to scan the QRs so they could leave with more information they could research for themselves.

The thing that most attracted the visitors’ attention was the Pinebook – when they read it cost 99 USD. That sparked interest in the underlying hardware, and in what software would run on an underpowered device. A lot of people also picked up the SBC for some reason. The Pine64 I had brought along was only there to show what kind of hardware was in the Pinebook, but it seems that… er… naked electronics are inherently fascinating to visitors at these kinds of events.

After the Pinebook, the most popular devices where the phone and the Raspberry Pi with its touchscreen. A lot of visitors asked if the phone was already for sale, thinking that a pure GNU + Linux phone was already a thing and they had somehow missed it. Even though I had to burst their bubble, they were satisfied that at least some progress was going on, both in the realms of mobile phones and vehicle infotainment systems.

Visitors

The scanning application provided by the organisers of the event was very useful, and I scanned 54 people in total, but, of course, I talked to more than that. By my calculations, about 50% more one-to-one, which puts the number of people I interacted one-on-one with between 75 and 80. Four or five times while I was delivering my spiel, a small crowd of 5 to 10 people congregated around me, so a conservative total number of people I talked to would be around 100.

Many of them were system administrators specialised in cloud computing, one of the main topics of the event. Others managed large networks of end-user machines for schools, libraries and other public institutions. There were also plenty of CEOs, CTOs and other C*Os, both attending for the talks and “shopping” for new open source development software. They are the people who found things like Kirigami interesting.

There were Linux desktop end-users in the mix, too. Many of them did not use Plasma (a few did), and they were under the impression that Plasma was heavy. The Pinebook disproved that, but this (that KDE software is bloated) is something we have seen before, and we clearly must continue to work towards dispelling this notion.

I tried to make sure that visitors to the booth walked away with something to remember us by. Stickers with KDE.org URLs on them until they run out; my card, in case they needed more information; or at the very least, the links to more information in the browsers on their phones, as I encouraged people to scan the QRs associated with each item on the table.

Mission(s) Accomplished?

One of the things I set out to do was to generate some publicity for KDE in the mainstream media, since it was announced that journalists from some big Spanish newspapers, radios and TVs would be there. Unfortunately, I did not see them.

However, I was not disappointed with the day, since we achieved other things on the list. We made contacts within several Madrilian institutions, like the leaders of the MAX Linux distribution, deployed in many Madrilian schools. They are currently using MATE for their desktop, but after reviewing our spread, the said they would give Plasma a try. I will be following up with them.

Continuing with public institutions, we also talked to the people who manage the libraries in Alcorcón, sysadmins from the Congreso de los Diputados and the Ministerio de Economía and Hacienda, and developers from Correos, the Spanish post office. There were representatives from several universities, both students and professors. All visitors were impressed by Plasma’s feature set, performance and flexibility, and were excited about trying it out at work and at home.

The students from the LibreLabUCM of the Universidad Complutense de Madrid later wrote to me and asked how they could contribute. They were especially interested in contributing to Plasma Mobile.

We had a mixed bag when it came to visitors from private enterprises. There were both coders and managers among the people who came to the booth, as well as freelancing consultants. Many of the managers, including CEOs, CTOs and product managers, and all the consultants seemed to be “shopping” for FLOSS to boost productivity (the former) or to add to their portfolio (the latter). Although they were mainly after infrastructure-like software, like cloud management systems, they would often become interested when I demoed Kirigami-based software and showed them it was possible to create good-looking, graphical applications for most platforms that would adapt to different screen sizes and shapes.

From the bigger, more recognisable companies, we had visitors from IBM, Oracle, BT, Atos, Allfunds Bank and Wacom. From smaller, Spanish joints we met people from VASS, Zylk, Zendata and Certelia.

Lessons Learnt

The first lesson I learnt was not to try and do this alone again. Over twelve hours of standing and greeting visitors is not good for an unfit, overweight 53-year-old. Being alone also meant I had to rely on the kindness of the people in the FSFE booth when I had to go foraging for water and food, or for when I needed a bathroom break — thanks Pablo and Erik!

But, seriously, next time we should show off some “naked” electronics. This fascinates attendees for some reason. We should maybe acquire the RISC-V board we showed in FOSDEM. These kinds of things attract visitors like a magnet.

I noticed many visitors looking over the booth from afar, trying to figure out who we were before approaching. As the roll-up banner was to one side, it was not always obvious that it was associated with us. A solution would be to always make sure we have a tablecloth or a prominent flag with our logo, name, and URL handy. We had both at the booth at FOSDEM, and I’m pretty sure that helped.

The stickers ran out rather quickly. By two o’clock there were none left. It wasn’t a big issue, because the event wasn’t the type that attracted merch scavengers, and most people were more interested in what we had on display than in stockpiling goodies. But it would still have been nice to have had more. Also, vinyl die-cut stickers are expensive: 60 euros for 100 stickers.

Speaking of printed merch, maybe we should make attractive flyers with coloured pictures, snappy explanatory bites, shortened URLs and no marketing speak, relevant to what is on show at the booth. Not everybody has QR scanning software on their phones, and a printed guide explaining what we were showing at the booth would’ve helped and served as a reminder if attendees could’ve taken it with them.

Was it worth it?

Yes. We made a lot of contacts with companies and institutions that would have been difficult to get in touch with any other way. We also heard about problems they have, and we can use that to see what solutions we can offer. Both things will ultimately help grow the number of companies that use KDE technologies (like Kirigami) in their products, as well as help us convince institutions to deploy our software (like Plasma and Applications) for their users.

Update

It has been a long time since I posted a blog (1 month+). In fact it might even seem the Krita ran on Android and now GSoC is done. Well, not quite. There’s still a lot to be done.

Let’s see what we worked on :)

Build

First and foremost was managing the build. If you have a look at README.android, you will see that’s a bunch of environment variables and few steps. If you look at underlying code, it was even uglier with lots of boiler plate. So, we had to refactor it and we did and made the build system a bit pleasant. (4467ad274)

OpenGL Canvas

Next, was a bug in the OpenGL canvas, when we enabled hardware acceleration, the canvas would turn black. QPainter being painfully slow, so it certainly isn’t the option, so we really rely on OpenGL ES.
So we had to fix this, it took me quite some time to fix and the solution was simple in the end.
Results? We can draw on OpenGL canvas now!

Autosave

If you have played any heavy game on Android, then there is one thing which you might know. If you leave the app and come back after a few minutes all the progress is lost or the connection to the server is terminated (it happens less often with new devices having a lot of memory).
This could happens with Krita as well, so now as soon as we get Activity#onPause(), we call the JNI wrapper Java_org_krita_android_JNIWrappers_saveState to save the state synchronously.
I did try to save it asynchronously (which uses slots and signal), but I found out, as soon as the main Qt thread was paused, the entire queued connection mechanism was brought to halt.

Touch events

For past week, I have been working on adding touch support to the canvas. What I mean by touch support is, to handle “finger paint” events and we’ve been successful in doing so. We can now draw on canvas now, using our fingers, not just a pen!

To do this we simply consider the touch events with one touch point as mouse click/update/release events (the same way we do this for tablet events).

But… there is a difference!

Krita did support touch drawing on touch screen windows/linux laptops. So, why didn’t it work for android devices? In windows, if QTouchEvent wasn’t handled, Qt would automatically generate QMouseEvent for it. But the same wasn’t true for android.
There is a way, however, to simply ignore touch events and use mouse events, even for android (by manipulating Qt::WA_AcceptTouchEvents flag). This wouldn’t work for Krita because we still use touch events to rotate/zoom/move canvas. So, we now explicitly handle touch events.

Rotation

We can now rotate the canvas using gestures! A small clip: rotation-canvas

It was pretty simple as well. First the way KisTouchShortcut::match worked, wouldn’t allow both zooming and rotation to co-exist, because it distinguishes the different KisTouchShortcuts based on number of minimum and maximum touch points. For both zooming and rotating, it was two.

So, for them to work in harmony, I created another class, KisZoomAndRotateAction, which delegates the call to KisZoomAction and KisRotateAction.

Secondly, in KisRotateAction, we just find out the angle between the lines. Line being, “the line” passing through the two touchpoints on the canvas.
So, the two lines are:

  1. Initial position of fingers
  2. Final position of fingers

(this is a bit hard to explain, please look at code).

Anything else?
Yes, this.

That's all! Thank you for reading, I’ll try to be more regular with my blog now :)

It’s finally here!

After more than one and a half years there finally is a new release. Kaidan 0.4.0 is the biggest update until now and apart from some bug-fixes and many minor and major features increasing the usability, Kaidan now has multiplatform-support for all common operating systems like Linux, Windows, Android and macOS.

But have a look at the changelog yourself:

Changelog

Build system:

  • Support for Android (ilyabizyaev)
  • Support for Ubuntu Touch (jbb)
  • Support for macOS (ilyabizyaev)
  • Support for Windows (ilyabizyaev)
  • Support for iOS (ilyabizyaev)
  • Add KDE Flatpak (jbb)
  • Switch Android builds to CMake with ECM (ilyabizyaev)
  • Improve Linux AppImage build script (ilyabizyaev)
  • Add additional image formats in AppImage (jbb)

New features:

  • Show proper notifications using KNotifications (lnj)
  • Add settings page for changing passwords (jbb, lnj)
  • Add XEP-0352: Client State Indication (gloox/QXmpp) (lnj)
  • Add media/file (including GIFs) sharing (lnj, jbb)
  • Full back-end rewrite to QXmpp (lnj)
  • Implement XEP-0363: HTTP File Upload and UploadManager for QXmpp (lnj)
  • Use XEP-0280: Message Carbons from QXmpp (lnj)
  • Use XEP-0352: Client State Indication from QXmpp (lnj)
  • Check incoming messages for media links (lnj)
  • Implement XEP-0308: Last Message Correction (lnj, jbb)
  • Make attachments downloadable (lnj)
  • Implement XEP-0382: Spoiler messages (xavi)
  • Kaidan is now offline usable (lnj)
  • Kaidan is able to open xmpp: URIs (lnj)
  • New logo (ilyabizyaev)
  • Show presence information of contacts (lnj, melvo)
  • Add EmojiPicker from Spectral with search and favorites functionality (jbb, fazevedo)
  • Highlight links in chat and make links clickable (lnj)
  • New about dialog instead of the about page (ilyabizyaev)
  • Add image preview in chat and before sending (lnj)
  • Send messages on Enter, new line on Ctrl-Enter (ilyabizyaev)
  • ‘Add contact’ is now the main action on the contacts page (lnj)
  • Elide contact names and messages in roster (lnj)
  • Chat page redesign (ilyabizyaev)
  • Display passive notifications when trying to use online actions while offline (lnj)
  • Automatically reconnect on connection loss (lnj)
  • Contacts page: Display whether online in title (lnj)
  • Add different connection error messages (jbb)
  • Use QApplication when building with QWidgets (notmart)
  • Ask user to approve subscription requests (lnj)
  • Remove contact action: Make JIDs bold (lnj)
  • Add contact sheet: Ask for optional message to contact (lnj)
  • Add empty chat page with help notice to be displayed on start up (jbb)
  • Redesign log in page (sohnybohny)
  • Add Copy Invitaion URL action (jbb)
  • Add ‘press and hold’ functionality for messages context menu (jbb)
  • Add copy to clipboard function for messages (jbb)
  • Add mobile file chooser (jbb)
  • Highlight the currently opened chat on contacts page (lnj)
  • Remove predefined window sizes (lnj)
  • Use new Kirigami application header (nicofee)
  • Make images open externally when clicked (jbb)
  • Use QtQuickCompiler (jbb)
  • Display upload progress bar (lnj)
  • Add text+color avatars as fallback (lnj, jbb)
  • Remove diaspora log in option (lnj)

Misc:

  • Forget passwords on log out (lnj)
  • Append four random chars to resource (lnj)
  • Save passwords in base64 instead of clear text (lnj)
  • Generate the LICENSE file automatically with all git authors (lnj)
  • Store ubuntu touch builds as job artifacts (lnj)
  • Add GitLab CI integration (jbb)

Fixes:

  • Fix blocking of GUI thread while database interaction (lnj)
  • Fix TLS connection bug (lnj)
  • Don’t send notifications when receiving own messages via. carbons (lnj)
  • Fix timezone bug of message timestamps (lnj)
  • Fix several message editing bugs (lnj)
  • Fix black icons (jbb)
  • Fix rich text labels in Plasma Mobile (lnj)
  • Small Plasma Mobile fixes (jbb)

Bug reports go to our issue tracker as always and translations are managed on Weblate.

Download:

PS: We’re searching for someone with an iPhone who can build & test Kaidan for iOS: contact us!

Could you tell us something about yourself?

My name is Enrique Gan. I live in California and I’m currently a computer science student.

Do you paint professionally, as a hobby artist, or both?

I’m definitely just a hobbyist artist but I’d like to start making money eventually. But, it’s been a while since I studied art for an extended period of time but I’d like to get back into it.

What genre(s) do you work in?

I like concept art and anime a lot, but I like to try out different genres and see what I can learn from each one.

Whose work inspires you most — who are your role models as an artist?

When I first committed a lot of my time to art, it was in 2014 and it was coincidentally also the time when I found a youtube channel of an artist named Sinix. I always thought art was for geniuses only, but after learning about Sinix and his art, I was convinced that anyone can be an artist. His work didn’t conform completely to mainstream appeal but I was profoundly captivated by how he draws. Other artists I like included Sachin Teng, Andrew Hem, Kim Jung Gi, Shirow Miwa, Richard Schmidt, and countless others.

How and when did you get to try digital painting for the first time?

I tried digital painting later in 2014 when I heard that GIMP was a free program and that some artists like CT Chrysler used it. So I tried it out with a mouse but I couldn’t do much with it because I was still pretty new to digital painting.

What makes you choose digital over traditional painting?

It lets me store a lot of paintings and export with ease. Also, it’s a lot cheaper since I already have a computer and an entry level graphics tablet isn’t too expensive compared to buying a lot of paint.

How did you find out about Krita?

Sycra Yasin posted a video showcasing Krita back in 2013 and I ended up trying it out some time in the summer of 2015. I think I got my first and current drawing tablet a month after and started churning out digital art with Krita since.

What was your first impression?

It was a lot more art orientated than GIMP was and it looked very professional like Photoshop.

What do you love about Krita?

I really like the brush engine and I’m really impressed that software of this quality is completely free and open source. I’ve always had a soft spot for open source.

What do you think needs improvement in Krita? Is there anything that really annoys you?

I think there are some UI things that confuse just me because I never really read the manual aside from the brush making portion. I don’t think I do anything wild either so I haven’t had the opportunity to find many bugs.

What sets Krita apart from the other tools that you use?

It’s free and is catering towards artists. Other free art programs are very simple or restrained but Krita is the whole package.

If you had to pick one favourite of all your work done in Krita so far, what would it be, and why?

I’ve really liked some of my more recent work like the red girl portrait I did.

What techniques and brushes did you use in it?

I use the most basic brushes imaginable. This includes a horizontal flat brush that doesn’t rotate, and another one that rotates. Sometimes I paint with a circle brush that has opacity on pressure. Recently I started using a simple color blending brush called the palette knife. It comes with Krita by default..

Where can people see more of your work?

I have an instagram, twitter, and artstation all under the name pitganart, as well as a website called pitganart.com. I also have a twitch account where I stream often called PitEG.

https://instagram.com/pitganart
https://twitter.com/pitganart
https://artstation.com/pitganart
https://www.twitch.tv/piteg

Anything else you’d like to share?

I’m really happy to have been interviewed ��

July 07, 2019

Since my last post about the LSP client progress in May I didn’t work on that project at all I think.

But the good news is, somebody else did scratch that itch on his own ;=)

We have now a prototype plugin in kate.git master, see lspclient in the addons directory.

It is not compiled per default, you can turn it on via:

cmake -DCMAKE_INSTALL_PREFIX=“your prefix” -DENABLE_LSPCLIENT=ON “kate src dir”

It shares no code with my initial prototype. The author started this without knowing of my work. This might actually be not that bad, as this plugin is composed of a much smaller code base. This allows to get familiar with the code easier as with the code I copied over from Qt Creator for my initial try.

But, even nicer, it does actually work a lot better than my variant, already now!

What does work (tested with clangd and kate.git/work projects) at the moment:

  • Auto completion: you get the proper LSP server provided completion items

  • Outline view: Get an extra tool view with the symbols of your current view

  • Document highlight: highlight all occurrences of a variable/… inside the current view

  • Code navigation: jump to the definition/declaration

There is still a lot of stuff missing and this is all prototype quality. For example the document highlight implementation I added has no way to clear the highlighting at the moment beside document reload.

But given I just needed one hour to add the document highlight support, I would say the code base is easy to adjust.

=> If you have time and want a good LSP client, now you can join the fun and have direct results.

As the author was kind enough to move his work on the plugin to the KDE infrastructure, feel welcome to show up on kwrite-devel@kde.org and help out! All development discussions regarding this plugin happen there. We are happy to accept patches, too, if you are a new contributor!

It’s time for week 78 in KDE’s Usability & Productivity initiative! This week I think people have been taking a breather following a super intense sprint, and some are even on vacation–myself included. So this week’s report is going to ba a bit light, but it’s still got a few cool goodies!

New Features

Bugfixes & Performance Improvements

User Interface Improvements

Next week, your name could be in this list! Not sure how? Just ask! I’ve helped mentor a number of new contributors recently and I’d love to help you, too! You can also check out https://community.kde.org/Get_Involved, and find out how you can help be a part of something that really matters. You don’t have to already be a programmer. I wasn’t when I got started. Try it, you’ll like it! We don’t bite!

If you find KDE software useful, consider making a tax-deductible donation to the KDE e.V. foundation.

This year I will be going to my second Akademy to meet my KDE friends again, discuss about future plans for the community during BoF sessions, participate in workshops, code and learn more about free software, KDE projects and Qt! One more interesting thing is that this time I am going to present a talk … Continue reading I am going to Akademy 2019!

July 05, 2019

From 19th to 25th of June, all the Plasma team gathered in Valencia, graciously hosted by the Slimbook people in their office. This was a special sprint, as it was co-located with the Usability sprint together with some VDG members. While some of the time each team was occupied in their own discussions, there were a big margin of overlap, allowing us to have a lot of discussions about the design and usability of our beloved Plasma desktop shell.

We now have plans in the coming months for several improvements across the board, including further improvements on the new shiny notification framework by Kai Uwe.

Also, we talked (and worked on) plans for further improving our Wayland support, including middle mouse button clipboard, and screen rotation for phone, tablets and 2 in 1 laptops).

On my end, a big part of this sprint was dedicated to an encompassing plan to refactor and redesign how desktop plasmoids work and are managed. I had there both UI discussions with the VDG and loong coding sessions on it.

The Desktop/FolderView containments use a big infrastructure written in Javascript which has some problems and isn’t touched much since a lot of time.

Some time ago I set myself the task of making the management of desktop plasmoids more touchscreen-friendly, so i started modifying that code, until.. I started to design a complete reimplementation written in C++ ��

This new implementation is much more robust, is faster and a bit leaner on the memory. Most important, is now a separate QML plugin, so is not anymore an implementation internal in the standard desktop, but if somebody wants to write his/her own containment for personalized plasma shells (for instance for a particular embedded device which is not a traditional desktop/laptop: we want plasma more and more usable as a set of construction blocks for the main UI of any kind of device).

In fact, it’s planned for Plasma Mobile to use the same layout manager component, to make the user experience “similar but different” and have less code duplication, while maintaining the UI very distinct between the two very different device types.

UI-wise in the desktop it doesn’t change much for now. the most notable difference is visible resize handles that make managing the layout and moving/resizing the plasmsoids much easier and more intuitive. Especially with touchscreen: now while manipulating plasmoids via touch, the resize handles become way bigger, and is possible to move and resize via a pinch gesture as well.

Lastly (for now!) the behavior during screen resolution switch improved a lot: if you connect a projector with a smaller resolution that relayouts your desktop, or play a fullscreen game at a tiny resolution, when the resolution is restored, everything gets back to normal, no more applets all over the place after changing resolution ��

All of this should make it into Plasma 5.17.

I've been working all week, instead of trying to deliver a feature I tried to write and organize the whole class, and then slowly write all the small functions Read More...

July 04, 2019

Well, it seems I got my blog back online just in time to write about security vulnerabilities. Well, let’s call it a security weakness in Calamares and the initramfs tools in a lot of distributions.

Weakness

Two CVE’s were files against Calamares this week, but I’ll only write about lax file permissions on initramfs images here. See the CVE database for more details.

The issue comes down to this: when creating an initramfs (which is done as root), a sensitive file is read. The initramfs file (a cpio archive) is created with lax permissions, and so any user who can read the initramfs file can then extract the contents of the sensitive file.

From the point of view of Calamares, the solution is to make sure that the initramfs is created with less lax file permissions. Simple, hey?

In principle, the umask is responsible for masking out file permissions bits, so a umask of 077 (octal!) would prevent group and other users (i.e. all the non-privileged users) from reading the initramfs. So all Calamares needs to do is set up a good umask before calling the tools, right?

If only it were that simple.

Distro’s

There’s a lot of distro’s out there. Calamares serves them without discriminating on toolset, language, or target audience. I’m really happy to have downstreams that write careful bug reports, who join on IRC to describe problems, who coach users about reporting problems. But there’s a lot of them, and there’s lots of different tools used to create initramfs out there.

Here are the tools I know about:

  • dracut (Fedora derivatives) doesn’t seem to be vulnerable, it has had a safer umask since 2012 (with a CVE and fix for a special case in 2016). So dracut does the right thing, always: sets umask 077 and the initramfs is not world-readable.
  • update-initramfs (Debian derivatives) has a configuration item for the umask in initramfs.conf, but this isn’t set in the Calamares-using Debian derivatives I looked at. Setting umask in the environment has no effect, because the tool actively resets the umask, to a lax value unless configured otherwise.
  • mkinitcpio (Arch derivatives) doesn’t seem to have anything at hand. It obeys umask, but only when the initramfs file doesn’t exist yet; it does not reset file permissions on an overwritten file.

Personally I feel the tools need to be fixed. Debian derivatives should install an initramfs.conf that sets UMASK=077. Users of mkinitcpio should make sure the umask is set right when generating the initramfs for the first time.

But what I think, personally, doesn’t directly affect distro’s, so Calamares is getting another (security-oriented) release today, with the following changes which are very unusual for an installer that doesn’t have many opinions:

  • Distro’s that use the initramfs module in Calamares will get a configuration snippet that sets UMASK=077.
  • Distro’s that use the initcpio module in Calamares will get all their initramfs’es set to safe permissions during installation.

After installation there’s not much I can do; those that have a safer configuration snippet will keep using it, while mkinitcpio users are dependent on having a safe umask when a kernel update happens.

Development

Three distro’s I used for testing things for this work – and I needed a variety, to see all the initramfs variations – were Chakra and ArcoLinux and Lubuntu.

These distro’s have a very very different approach to a Live CD and what ends up installed; Chakra gives you a full and wonderful KDE Plasma experience with all the bells and whistles (and latte dock too). ArcoLinux gives you a minimal XFCE-based Live experience, even as it installs a KDE Plasma environment to your hard disk. Lubuntu is fast and light-weight but I notice it’s not-Plasma all the time because I have so many KDE Plasma shortcuts in my muscle memory. So it’s a nice place to visit, but I wouldn’t want to live there.

I can heartily recommend some distro-shopping every now and then.

.. and after that I can get back to KDE-on-FreeBSD.

Look for a new Calamares release friday the 5th, with fixes, but it will take a while for that to get through to distro’s.

From 19.6. to 25.6. I attended this year’s combined Plasma and Usability & Productivity sprint in the beautiful city of Valencia.

It was a great opportunity to meet old and new friends, drink beer and sangria on the rooftop and of course do some hacking.

First we discussed about the future development of Plasma, especially the Wayland experience. I was particularly interested in how we can solve the two missing pieces in KDE Connect on Wayland, Keyboard input and clipboard synchronization.

We did not only discuss things but got our hands dirty as well tough. I took the opportunity to work on several components, most of which I haven’t worked on much before:

  • I’ve reworked the system tray settings to be more concise and usable. Furthermore I worked on hiding certain system tray elements when they are not in a useful state, making the system tray less cluttered. I also fixed the appearance of the disk quota icon in the system tray settings and the system tray itself when using a dark theme.
  • After I cleaned the code and polished the export menu I worked on integrating an image annotator in Spectacle. Kudos to Damir Porobic for doing the hard part of it by providing kImageAnnotator.
  • I implemented a more human friendly sort order description in Dolphin.
  • I improved the user experience of Purpose, the framework used for sharing stuff. Cancelling a share doesn’t show an error any more. The share dialogs for KDE Connect and bluetooth got a huge makeover. The share menu now shows ellipsis where semantically appropriate.
  • Next to polishing the menus I implemented sharing via Purpose in Gwenview.

The sprint was generously hosted by Slimbook. Thank you very much for that!

Sprints like this are only possible due to the donations KDE e.V. is getting. Please consider donating if you want us to be able to further improve our software.

The KDE Plasma and Usability teams recently converged on the beautiful Spanish city of Valencia for a combined development sprint. The teams admired Valencia's medieval architecture and stayed up until midnight eating sumptuous Mediterranean food. But of course, the real purpose was work!

We camped out in the offices of the Slimbook company, which were generously made available for the sprint. The aim was not only to hack on Plasma and the Usability & Productivity initiative, but also to benefit from the cross-pollination opportunities provided by hosting both sprints at the same time and place.

The result was a huge amount of work done on Plasma, KWin, Dolphin, Spectacle, and many other bits of KDE software.

Present for the Plasma sprint were Kai Uwe Broulik, David Edmundson, Nicolas Fella, Eike Hein, Roman Gilg, Aleix Pol Gonzalez, Marco Martin, and Bhushan Shah. They had quite a busy agenda:

  • Plasma 5.16's new notification system received a great deal of polish
  • Fixed a ton of bugs in the Plasma Browser Integration
  • Rewrote the widget positioning code for the desktop, making it much more robust, future-proof, and usable on touch:



  • Started work on making the Task Manager understand window stacking order, which will allow it to implement new interaction modes for grouped windows (e.g. bring forward the last-used window when clicked)
  • Worked on architecture improvements for the Task Manager to unify its different presentation modes and improve code robustness
  • Worked on a variety of architecture improvements for KWin to make it more future-proof, which, among other things, will improve multi-screen handling
  • Improved the user interface for the System Tray's settings window
  • Added calculator and unit conversion functionality to Kickoff and Application Dashboard

Kickoff now integrates a calculator and a unit conversion utility.

In addition to making technical progress, the Plasma and Usability teams got together to discuss a number of long-standing Plasma issues, and figure out how to resolve them:

We wanted to make it easier to test a custom-compiled version of Plasma. To do so, we implemented changes that allow you to integrate your custom-compiled Plasma into SDDM by running a single command, after which you can log into it normally. For more information, see this article.

We thought it would be a good idea to make more it obvious and discoverable that Plasma is made up of widgets, and show how they are configured. To do this, we decided to create a new "global edit mode" that's triggerable from within System Settings, as this is where new users generally expect everything to be configured. In this global edit mode, all widgets become visibly configurable, editable, removable, etc. We also want to make it easy to change the wallpaper in this mode. With all that done, we'll be able to remove the Desktop Toolbox as it currently exists.

There was a need to unify the disparate scaling methods, so we decided to visually connect the scale factor chooser with the "Force Fonts DPI" setting, since the former actually affects the latter, but not the other way around. This should make it clear that the scaling slider is the primary way to scale the screen, and the "force fonts DPI" control is nothing more than a way to tweak things further.

We needed Plasma to respect the system-wide scale factor on X11, so we came up with a path forward and a plan for getting it done!

We planned out how to add power actions to the lock screen. We concluded that not only does this make sense, but it will be necessary for Plasma Mobile anyway. In a multi-user environment, the user will have to enter an admin password to shut down or restart the machine when other users are also logged in.


Even during down time, KDE carries on coding!

Over in the Usability & Productivity room we had Méven Car, Albert Astals Cid, Noah Davis, Filip Fila, Nate Graham, and David Redondo. The agenda was similarly jam-packed, and included the following:

  • We ported Spectacle to use KGlobalAccel and away from KHotKeys, made the quit-after-copy feature finally work, and added support for drawing annotations on newly-taken screenshots
  • We implemented user-configurable sort ordering for wallpaper slideshows
  • Dolphin received human-readable sort order text and an auto-play-on-hover feature for media files
  • We added inline name feedback when creating new files or folders
  • Users can optionally close windows in the Present Windows effect with a middle-click
  • Many user interface improvements have been made to the Purpose framework, which implements sharing support in many apps (Dolphin, Spectacle, Okular, Gwenview as of recently, and so on)
  • We started working on improving the default selection of pictures available for user account avatars
  • Initial work has been done on a new "Recently used" feature for Dolphin and the file dialogs that will pull its data from a single consistent location and actually notice everything

We also came to some significant conclusions related to higher-level goals. For example, we plan to pay for professional user research to generate new "personas" and target user groups that represent the people using our software. We will use these personas as the basis for professional usability testing for Plasma, Dolphin, Gwenview, Okular, and other components of a basic desktop.

Additionally, we discussed how we can add release notes data to our apps' AppStream data, so that it shows up in software center apps like Discover. The big blocker was getting the required translations added to the tarball. We've started a dialogue with AppStream maintainer Matthias Klumpp regarding a new feature to pull translations from a remote location, which would support our workflow. The conversation is proceeding nicely so far.

Finally, VDG member Noah Davis dug deep into Breeze to work on visual consistency improvements related to selection highlights. Given his growing familiarity with the code, he's well on his way to becoming the next Breeze maintainer!

All in all, it was a very productive week. KDE Plasma and apps are in a great place right now, and the team's effort to further improve things will reach you in upcoming versions, so stay tuned!

Dot Categories:

A due premise: this is not a guide! While at the end I managed to build an image for my phone, it doesn't even boot. So, this post should be taken just as a collection of notes for myself, written to be able to reproduce the build and keep track of those steps I might want to revisit later.

The reason why I'm publishing these notes in my blog is that I believe they can still be useful for other people who should stumble into the same errors, and also to give you an idea of how much fun (or lack thereof) porting is.

It's a very long post, but unless you are starting to port Halium or Ubports to a new device, you can safely stop reading now.

I hope to write some better porting news in some future post. :-)

Before we start: getting the info

The first step is finding out a few informations about our device, and whether a LineageOS port for our device exists (hadn't there been a LineageOS port, I would have been out of luck: I certainly wouldn't have the time to do one myself). From the Settings application we can find the device name, the model number, the Android version and the kernel version:

  • Device name: "Samsung Galaxy J3 (2016)"
  • Model: SM-J320F
  • Android version: 5.1.1
  • Kernel version: 3.10.65-[…]

With this information we can look for a LineageOS port, either via a search engine or directly in the XDA forums; in my case the thread is this one. The post announcing the port should contain links to the source code, besides some other information. We do especially care about:

  • the device codename: j3xlte
  • the CM/LineageOS version: 14.1
  • links to Device Tree, Kernel, Local Manifest (keep the local manifest handy, as you are going to need it soon)

To be on the safe side, get a copy of everything. In case these are GitHub repositories, forking them is enough.

Getting root access on the device

Another precondition for a successful port is the ability to be able to flash a custom kernel and rootfs image. The existance of a LineageOS port is not a guarantee that we can flash it: some devices are sold with a locked bootloader in certain markets or when sold by certain mobile operators. So, before spending time on the port, we should make sure that we can hack on our device freely.

In order to get adb working, I had to enable "Developer mode", which is done by hitting the "Build numer" item under the "Software info" page for 10 times in a row. Then from the newly appeared "Developer options" menu I enabled "USB debugging".

Being a total novice to Android phones, I had to go through several attempts and internet searches, before being able to get root access on the device (which I needed in order to read the /etc/fstab file, for example): by default, adb only let me as normal user, and su didn't work.

In order to flash a rootkit on this phone, we need to use Heimdall, because on Samsung devices this is what is used instead of fastboot. I just followed the instructions in the Linux/README file and could easily build and install it.

We are going to flash TWRP and SuperSU; a link to the former can be found in the XDA post about the LineageOS port for our device (under the "Download" button) and the latter from its own homepage (in my case I had to download the "Recovery flashable" zip file, and for some weird reason I got the 2.76 version, but I bet that any newer version works as well). Once I got these files, I followed these steps:

  • Extract TWRP in my host machine: cd /tmp && tar xvf ~/Downloads/sm-j320fgm_twrp_3.1.1_f2fs.tar.md5 (that leaves a recovery.img file in my /tmp)
  • Upload SuperSU to the phone:
    adb push ~/Downloads/UPDATE-SuperSU-v2.76-20160630161323.zip /sdcard/supersu.zip
  • Power off the phone
  • Power it on by keeping the VOLUME DOWN and HOME keys pressed while pressing the power button. The phone will be brought up into Odin mode.
  • Run
    sudo heimdall flash  --RECOVERY /tmp/recovery.img
  • Before the phone reboots, start keeping the VOLUME UP (note: it's not the same as before!) and HOME keys, to make it reboot into recovery and not into the ordinary image. If the phone boots into the ordinary image it will overwrite the TWRP we just flashed, so, if that happens, you need to start from scratch.

  • Find supersu (the zip file) and install it.

If you reboot the phone and use adb shell, you'll be able to run su and get root rights. At this point, I charged myself with hopes and got to the real work.

Let the Odyssey start

I just followed the "First steps" instructions step by step, with no major surprises.

Once done with that, I moved to the "Get sources" page. Here it was not just a matter of following the instructions: assembling the manifest file took me several iterations, so I had to come back to this step a few times. Anyway, I followed the instructions for the halium-7.1 version, given that my device has a LineageOS 14.1 port. Then:

  • I created a git branch in the halium/devices/ tree, to keep track of my changes
  • I added the manifest halium/devices/manifests/samsung_j3xlte.xml (you can see the final version in my repository, but initially I was missing some important lines)
  • Run JOBS=2 ./halium/devices/setup j3xlte
  • The makefile needed to be changed: the device/samsung/j3xlte/setup-makefiles.sh script (from the device tree) was missing a mkdir -p ../../../$OUTDIR near the beginning.

At this point I continued to the "Build sources" page. Here starts a long trial and error phase, and the longer part of this post. For the sake of documentation, I will show you all the commands I ran, including failed attempts (there are many of them), just to give you an idea of what problems you might run into with your own port, and hopefully help in finding a solution.

I first ran the breakfast command for my device:

$ breakfast j3xlte
including vendor/cm/vendorsetup.sh
build/core/product_config.mk:254: *** _nic.PRODUCTS.[[device/samsung/j3xlte/lineage.mk]]: "device/samsung/sharkls-common/sharkls.mk" does not exist.  Stop.
build/core/product_config.mk:254: *** _nic.PRODUCTS.[[device/samsung/j3xlte/lineage.mk]]: "device/samsung/sharkls-common/sharkls.mk" does not exist.  Stop.
build/core/product_config.mk:254: *** _nic.PRODUCTS.[[device/samsung/j3xlte/lineage.mk]]: "device/samsung/sharkls-common/sharkls.mk" does not exist.  Stop.
Device j3xlte not found. Attempting to retrieve device repository from LineageOS Github (http://github.com/LineageOS).
Repository for j3xlte not found in the LineageOS Github repository list. If this is in error, you may need to manually add it to your local_manifests/roomservice.xml.
build/core/product_config.mk:254: *** _nic.PRODUCTS.[[device/samsung/j3xlte/lineage.mk]]: "device/samsung/sharkls-common/sharkls.mk" does not exist.  Stop.
build/core/product_config.mk:254: *** _nic.PRODUCTS.[[device/samsung/j3xlte/lineage.mk]]: "device/samsung/sharkls-common/sharkls.mk" does not exist.  Stop.

** Don't have a product spec for: 'lineage_j3xlte'
** Do you have the right repo manifest?

It looks like the device/samsung/sharkls-common was not checked out properly. After re-doing with force-sync, it failed with other errors:

$ breakfast j3xlte
including vendor/cm/vendorsetup.sh
build/core/product_config.mk:254: *** _nic.PRODUCTS.[[device/samsung/j3xlte/lineage.mk]]: "device/common/gps/gps_eu_supl.mk" does not exist.  Stop.
build/core/product_config.mk:254: *** _nic.PRODUCTS.[[device/samsung/j3xlte/lineage.mk]]: "device/common/gps/gps_eu_supl.mk" does not exist.  Stop.
build/core/product_config.mk:254: *** _nic.PRODUCTS.[[device/samsung/j3xlte/lineage.mk]]: "device/common/gps/gps_eu_supl.mk" does not exist.  Stop.
Device j3xlte not found. Attempting to retrieve device repository from LineageOS Github (http://github.com/LineageOS).
Repository for j3xlte not found in the LineageOS Github repository list. If this is in error, you may need to manually add it to your local_manifests/roomservice.xml.
build/core/product_config.mk:254: *** _nic.PRODUCTS.[[device/samsung/j3xlte/lineage.mk]]: "device/common/gps/gps_eu_supl.mk" does not exist.  Stop.
build/core/product_config.mk:254: *** _nic.PRODUCTS.[[device/samsung/j3xlte/lineage.mk]]: "device/common/gps/gps_eu_supl.mk" does not exist.  Stop.

** Don't have a product spec for: 'lineage_j3xlte'
** Do you have the right repo manifest?

Indeed, I had nothing under device/common. So, I looked under halium/devices/manifests to see if any other manifest was populating this directory, and found that the bq_krillin.xml file had such a line. So, I copied that line (the one adding the lineageos/android_device_common project) into my manifest. After doing that, the breakfast command succeeded:

$ breakfast j3xlte
including vendor/cm/vendorsetup.sh
Trying dependencies-only mode on a non-existing device tree?

============================================
PLATFORM_VERSION_CODENAME=REL
PLATFORM_VERSION=7.1.1
LINEAGE_VERSION=14.1-20190619-UNOFFICIAL-j3xlte
TARGET_PRODUCT=lineage_j3xlte
TARGET_BUILD_VARIANT=userdebug
TARGET_BUILD_TYPE=release
TARGET_BUILD_APPS=
TARGET_ARCH=arm
TARGET_ARCH_VARIANT=armv7-a-neon
TARGET_CPU_VARIANT=cortex-a7
TARGET_2ND_ARCH=
TARGET_2ND_ARCH_VARIANT=
TARGET_2ND_CPU_VARIANT=
HOST_ARCH=x86_64
HOST_2ND_ARCH=x86
HOST_OS=linux
HOST_OS_EXTRA=Linux-4.15.0-51-generic-x86_64-with-Ubuntu-16.04-xenial
HOST_CROSS_OS=windows
HOST_CROSS_ARCH=x86
HOST_CROSS_2ND_ARCH=x86_64
HOST_BUILD_TYPE=release
BUILD_ID=NOF27B
OUT_DIR=/mnt/Lavoro/halium/out
RECOVERY_VARIANT=twrp
WITH_SU=true
============================================

Building the kernel

According to the livestream, the proper script to check for the kernel config is not the Mer one, but halium/halium-boot/check-kernel-config (this is also documented in the Ubports - Building Halium boot documentation page).

At this point I realized that I didn't have a kernel: an entry starting with kernel/ should always be in the manifest file. So I added the kernel line taken from the LineageOS local manifest file.

I then looked for the config file, which I eventually found in arch/arm/configs/j3xlte_defconfig. So I ran

./halium/halium-boot/check-kernel-config kernel/samsung/sharkls/arch/arm/configs/j3xlte_defconfig -w

which reported no errors. However, in the command output I noticed this:

 CONFIG_TMPFS_POSIX_ACL is already set
 CONFIG_DEFAULT_SECURITY is set, but to y "selinux" y not "apparmor".
 Setting CONFIG_DEFAULT_SECURITY="apparmor" correctly
sed: -e expression #1, char 60: unterminated `s' command
 Setting CONFIG_SECURITY_APPARMOR_BOOTPARAM_VALUE=1

When I checked the config file, I noticed that AppArmor was not really enabled. So I changed it manually.

Next, the fstab. I had the Ubports and Halium guides both opened in my browser, and I went for the "Fix mounts" step from the Ubports guide. I used find to locate the fstab, and found two:

The recovery file seemed already fine to me, while the rootdir one had most entry duplicated; ones with f2fs and xattrs, the other ones with ext4. I commented out the f2fs ones and ran mka halium-boot:

[...]
/mnt/Lavoro/halium/out/build-aosp_arm.ninja is missing, regenerating...
============================================
PLATFORM_VERSION_CODENAME=REL
PLATFORM_VERSION=7.1.1
LINEAGE_VERSION=
TARGET_PRODUCT=aosp_arm
TARGET_BUILD_VARIANT=eng
TARGET_BUILD_TYPE=release
TARGET_BUILD_APPS=
TARGET_ARCH=arm
TARGET_ARCH_VARIANT=armv7-a
TARGET_CPU_VARIANT=generic
TARGET_2ND_ARCH=
TARGET_2ND_ARCH_VARIANT=
TARGET_2ND_CPU_VARIANT=
HOST_ARCH=x86_64
HOST_2ND_ARCH=x86
HOST_OS=linux
HOST_OS_EXTRA=Linux-4.15.0-51-generic-x86_64-with-Ubuntu-16.04-xenial
HOST_CROSS_OS=windows
HOST_CROSS_ARCH=x86
HOST_CROSS_2ND_ARCH=x86_64
HOST_BUILD_TYPE=release
BUILD_ID=NOF27B
OUT_DIR=/mnt/Lavoro/halium/out
============================================
Checking build tools versions...
find: ‘device/*/generic’: No such file or directory
find: ‘device/unknown’: No such file or directory
find: ‘device/android’: No such file or directory
find: ‘device/*/generic’: No such file or directory
find: ‘device/unknown’: No such file or directory
find: ‘device/android’: No such file or directory
halium/hybris-boot/Android.mk:69: ********************* /boot appears to live on ERROR: *fstab* not found
halium/hybris-boot/Android.mk:70: ********************* /data appears to live on ERROR: *fstab* not found
halium/hybris-boot/Android.mk:73: *** There should be a one and only one device entry for HYBRIS_BOOT_PART and HYBRIS_DATA_PART.
build/core/ninja.mk:166: recipe for target '/mnt/Lavoro/halium/out/build-aosp_arm.ninja' failed
make: *** [/mnt/Lavoro/halium/out/build-aosp_arm.ninja] Error 1
make: Leaving directory '/mnt/Lavoro/halium'

#### make failed to build some targets (11 seconds) ####

this seems to be documented in the "Common system build errors" page: I had to rerun source build/envsetup.sh and breakfast j3xlte. After that, the mka halium-boot command failed with

device/samsung/sharkls-common/ims/sec_samsung/Android.mk:25: build/core/java_library.mk: No such file or directory
build/core/ninja.mk:166: recipe for target '/mnt/Lavoro/halium/out/build-lineage_j3xlte.ninja' failed
make: *** [/mnt/Lavoro/halium/out/build-lineage_j3xlte.ninja] Error 1
make: Leaving directory '/mnt/Lavoro/halium'

which looks like a progress. The issue about the missing java_library.mk has been raised in the #halium channel; possible solutions include:

  • symlink from static_java_library.mk
  • edit the Android.mk file

I went for the former; this probably needs to be fixed in Halium.

Reran mka halium-boot, got some improvement: fstab is found:

halium/hybris-boot/Android.mk:70: ********************* /boot appears to live on 
halium/hybris-boot/Android.mk:71: ********************* /data appears to live on /dev/block/platform/sdio_emmc/by-name/userdata
halium/hybris-boot/Android.mk:74: *** There should be a one and only one device entry for HYBRIS_BOOT_PART and HYBRIS_DATA_PART.
build/core/ninja.mk:166: recipe for target '/mnt/Lavoro/halium/out/build-lineage_j3xlte.ninja' failed
make: *** [/mnt/Lavoro/halium/out/build-lineage_j3xlte.ninja] Error 1
make: Leaving directory '/mnt/Lavoro/halium'

It looks like my fstab was not complete, but missing the /boot partition. given that none of the fstab I located in my source tree had a line for the boot partition, I checked out the Android's fstab from the phone, via adb shell. There is no fstab in /etc, but two fstab files right in the root directory of the filesystem: fstab.sc8830 with exactly the same contents as the recovery fstab I found before in device/samsung/sharkls-common/recovery/root/fstab.sc8830, and a fstab.goldfish:

127|root@j3xlte:/ # cat /fstab.goldfish                                        
# Android fstab file.
#<src>                                                  <mnt_point>         <type>    <mnt_flags and options>                              <fs_mgr_flags>
# The filesystem that contains the filesystem checker binary (typically /system) cannot
# specify MF_CHECK, and must come before any filesystems that do specify MF_CHECK
/dev/block/mtdblock0                                    /system             ext4      ro,noatime,barrier=1                                         wait
/dev/block/mtdblock1                                    /data               ext4      noatime,nosuid,nodev,barrier=1,nomblk_io_submit      wait,check
/dev/block/mtdblock2                                    /cache              ext4      noatime,nosuid,nodev  wait,check
/devices/platform/goldfish_mmc.0                        auto                vfat      defaults                                             voldmanaged=sdcard:auto

While mount says:

~ # mount
rootfs on / type rootfs (rw,seclabel)
tmpfs on /dev type tmpfs (rw,seclabel,nosuid,relatime,mode=755)
devpts on /dev/pts type devpts (rw,seclabel,relatime,mode=600)
proc on /proc type proc (rw,relatime,gid=3009,hidepid=2)
sysfs on /sys type sysfs (rw,seclabel,relatime)
selinuxfs on /sys/fs/selinux type selinuxfs (rw,relatime)
tmpfs on /tmp type tmpfs (rw,seclabel,relatime)
pstore on /sys/fs/pstore type pstore (rw,seclabel,relatime)
adb on /dev/usb-ffs/adb type functionfs (rw,relatime)
adb on /dev/usb-ffs/adb type functionfs (rw,relatime)
/dev/block/mmcblk0p27 on /data type ext4 (rw,seclabel,relatime,data=ordered)
/dev/block/mmcblk0p27 on /sdcard type ext4 (rw,seclabel,relatime,data=ordered)
/dev/block/mmcblk0p24 on /cache type ext4 (rw,seclabel,relatime,data=ordered)

None of the above helps in locating the boot partition. I then tried to look under /dev/block:

root@j3xlte:/dev/block/platform/sdio_emmc/by-name # ls -l
lrwxrwxrwx root     root              2019-06-26 19:13 CACHE -> /dev/block/mmcblk0p24
lrwxrwxrwx root     root              2019-06-26 19:13 FOTA_SIG -> /dev/block/mmcblk0p11
lrwxrwxrwx root     root              2019-06-26 19:13 HIDDEN -> /dev/block/mmcblk0p26
lrwxrwxrwx root     root              2019-06-26 19:13 KERNEL -> /dev/block/mmcblk0p20
lrwxrwxrwx root     root              2019-06-26 19:13 PARAM -> /dev/block/mmcblk0p16
lrwxrwxrwx root     root              2019-06-26 19:13 PERSDATA -> /dev/block/mmcblk0p23
lrwxrwxrwx root     root              2019-06-26 19:13 PERSISTENT -> /dev/block/mmcblk0p22
lrwxrwxrwx root     root              2019-06-26 19:13 RECOVERY -> /dev/block/mmcblk0p21
lrwxrwxrwx root     root              2019-06-26 19:13 RESERVED2 -> /dev/block/mmcblk0p19
lrwxrwxrwx root     root              2019-06-26 19:13 SBOOT -> /dev/block/mmcblk0p1
lrwxrwxrwx root     root              2019-06-26 19:13 SBOOT2 -> /dev/block/mmcblk0p2
lrwxrwxrwx root     root              2019-06-26 19:13 SYSTEM -> /dev/block/mmcblk0p25
lrwxrwxrwx root     root              2019-06-26 19:13 efs -> /dev/block/mmcblk0p17
lrwxrwxrwx root     root              2019-06-26 19:13 l_fixnv1 -> /dev/block/mmcblk0p3
lrwxrwxrwx root     root              2019-06-26 19:13 l_fixnv2 -> /dev/block/mmcblk0p4
lrwxrwxrwx root     root              2019-06-26 19:13 l_gdsp -> /dev/block/mmcblk0p9
lrwxrwxrwx root     root              2019-06-26 19:13 l_ldsp -> /dev/block/mmcblk0p7
lrwxrwxrwx root     root              2019-06-26 19:13 l_modem -> /dev/block/mmcblk0p8
lrwxrwxrwx root     root              2019-06-26 19:13 l_runtimenv1 -> /dev/block/mmcblk0p12
lrwxrwxrwx root     root              2019-06-26 19:13 l_runtimenv2 -> /dev/block/mmcblk0p13
lrwxrwxrwx root     root              2019-06-26 19:13 l_warm -> /dev/block/mmcblk0p10
lrwxrwxrwx root     root              2019-06-26 19:13 pm_sys -> /dev/block/mmcblk0p5
lrwxrwxrwx root     root              2019-06-26 19:13 prodnv -> /dev/block/mmcblk0p18
lrwxrwxrwx root     root              2019-06-26 19:13 rsvdfixnv1 -> /dev/block/mmcblk0p6
lrwxrwxrwx root     root              2019-06-26 19:13 td_runtimenv1 -> /dev/block/mmcblk0p14
lrwxrwxrwx root     root              2019-06-26 19:13 td_runtimenv2 -> /dev/block/mmcblk0p15
lrwxrwxrwx root     root              2019-06-26 19:13 userdata -> /dev/block/mmcblk0p27

This, combined from the PIT file which we can extract with Heimdall from the "Odin mode", gives us a better understanding of the device partitions:

$ sudo heimdall print-pit --no-reboot
Heimdall v1.4.2

Copyright (c) 2010-2017 Benjamin Dobell, Glass Echidna
http://www.glassechidna.com.au/

This software is provided free of charge. Copying and redistribution is
encouraged.

If you appreciate this software and you would like to support future
development please consider donating:
http://www.glassechidna.com.au/donate/

Initialising connection...
Detecting device...
Claiming interface...
Setting up interface...

Initialising protocol...
Protocol initialisation successful.

Beginning session...

Some devices may take up to 2 minutes to respond.
Please be patient!

Session begun.

Downloading device's PIT file...
PIT file download successful.

Entry Count: 31
Unknown 1: 1598902083
Unknown 2: 844251476
Unknown 3: 20563
Unknown 4: 17490
Unknown 5: 14136
Unknown 6: 13619
Unknown 7: 0
Unknown 8: 0


--- Entry #0 ---
Binary Type: 0 (AP)
Device Type: 2 (MMC)
Identifier: 80
Attributes: 2 (STL Read-Only)
Update Attributes: 1 (FOTA)
Partition Block Size/Offset: 0
Partition Block Count: 1024
File Offset (Obsolete): 0
File Size (Obsolete): 0
Partition Name: BOOT
Flash Filename: spl.img
FOTA Filename: 


--- Entry #1 ---
Binary Type: 0 (AP)
Device Type: 2 (MMC)
Identifier: 90
Attributes: 2 (STL Read-Only)
Update Attributes: 1 (FOTA)
Partition Block Size/Offset: 0
Partition Block Count: 2048
File Offset (Obsolete): 0
File Size (Obsolete): 0
Partition Name: BOOT2
Flash Filename: spl2.img
FOTA Filename: 


--- Entry #2 ---
Binary Type: 0 (AP)
Device Type: 2 (MMC)
Identifier: 70
Attributes: 5 (Read/Write)
Update Attributes: 1 (FOTA)
Partition Block Size/Offset: 1024
Partition Block Count: 1024
File Offset (Obsolete): 0
File Size (Obsolete): 0
Partition Name: PIT
Flash Filename: J3XLTE.pit
FOTA Filename: 


--- Entry #3 ---
Binary Type: 0 (AP)
Device Type: 2 (MMC)
Identifier: 71
Attributes: 5 (Read/Write)
Update Attributes: 1 (FOTA)
Partition Block Size/Offset: 2048
Partition Block Count: 6144
File Offset (Obsolete): 0
File Size (Obsolete): 0
Partition Name: MD5HDR
Flash Filename: md5.img
FOTA Filename: 


--- Entry #4 ---
Binary Type: 0 (AP)
Device Type: 2 (MMC)
Identifier: 1
Attributes: 5 (Read/Write)
Update Attributes: 1 (FOTA)
Partition Block Size/Offset: 8192
Partition Block Count: 4096
File Offset (Obsolete): 0
File Size (Obsolete): 0
Partition Name: SBOOT
Flash Filename: sboot.bin
FOTA Filename: 


--- Entry #5 ---
Binary Type: 0 (AP)
Device Type: 2 (MMC)
Identifier: 2
Attributes: 5 (Read/Write)
Update Attributes: 1 (FOTA)
Partition Block Size/Offset: 12288
Partition Block Count: 4096
File Offset (Obsolete): 0
File Size (Obsolete): 0
Partition Name: SBOOT2
Flash Filename: sboot2.bin
FOTA Filename: 


--- Entry #6 ---
Binary Type: 0 (AP)
Device Type: 2 (MMC)
Identifier: 3
Attributes: 5 (Read/Write)
Update Attributes: 1 (FOTA)
Partition Block Size/Offset: 16384
Partition Block Count: 2048
File Offset (Obsolete): 0
File Size (Obsolete): 0
Partition Name: l_fixnv1
Flash Filename: nvitem1.bin
FOTA Filename: 


--- Entry #7 ---
Binary Type: 0 (AP)
Device Type: 2 (MMC)
Identifier: 4
Attributes: 5 (Read/Write)
Update Attributes: 1 (FOTA)
Partition Block Size/Offset: 18432
Partition Block Count: 2048
File Offset (Obsolete): 0
File Size (Obsolete): 0
Partition Name: l_fixnv2
Flash Filename: nvitem.bin
FOTA Filename: 


--- Entry #8 ---
Binary Type: 0 (AP)
Device Type: 2 (MMC)
Identifier: 5
Attributes: 5 (Read/Write)
Update Attributes: 1 (FOTA)
Partition Block Size/Offset: 20480
Partition Block Count: 2048
File Offset (Obsolete): 0
File Size (Obsolete): 0
Partition Name: pm_sys
Flash Filename: PM_sharkl_arm7.bin
FOTA Filename: 


--- Entry #9 ---
Binary Type: 0 (AP)
Device Type: 2 (MMC)
Identifier: 6
Attributes: 5 (Read/Write)
Update Attributes: 1 (FOTA)
Partition Block Size/Offset: 22528
Partition Block Count: 2048
File Offset (Obsolete): 0
File Size (Obsolete): 0
Partition Name: rsvdfixnv1
Flash Filename: 
FOTA Filename: 


--- Entry #10 ---
Binary Type: 0 (AP)
Device Type: 2 (MMC)
Identifier: 7
Attributes: 5 (Read/Write)
Update Attributes: 1 (FOTA)
Partition Block Size/Offset: 24576
Partition Block Count: 8192
File Offset (Obsolete): 0
File Size (Obsolete): 0
Partition Name: l_ldsp
Flash Filename: SPRDLTEDSP.img
FOTA Filename: 


--- Entry #11 ---
Binary Type: 0 (AP)
Device Type: 2 (MMC)
Identifier: 8
Attributes: 5 (Read/Write)
Update Attributes: 1 (FOTA)
Partition Block Size/Offset: 32768
Partition Block Count: 32768
File Offset (Obsolete): 0
File Size (Obsolete): 0
Partition Name: l_modem
Flash Filename: SPRDCP.img
FOTA Filename: 


--- Entry #12 ---
Binary Type: 0 (AP)
Device Type: 2 (MMC)
Identifier: 9
Attributes: 5 (Read/Write)
Update Attributes: 1 (FOTA)
Partition Block Size/Offset: 65536
Partition Block Count: 8192
File Offset (Obsolete): 0
File Size (Obsolete): 0
Partition Name: l_gdsp
Flash Filename: SPRDGDSP.img
FOTA Filename: 


--- Entry #13 ---
Binary Type: 0 (AP)
Device Type: 2 (MMC)
Identifier: 10
Attributes: 5 (Read/Write)
Update Attributes: 1 (FOTA)
Partition Block Size/Offset: 73728
Partition Block Count: 8192
File Offset (Obsolete): 0
File Size (Obsolete): 0
Partition Name: l_warm
Flash Filename: SPRDWDSP.img
FOTA Filename: 


--- Entry #14 ---
Binary Type: 0 (AP)
Device Type: 2 (MMC)
Identifier: 11
Attributes: 5 (Read/Write)
Update Attributes: 1 (FOTA)
Partition Block Size/Offset: 81920
Partition Block Count: 2048
File Offset (Obsolete): 0
File Size (Obsolete): 0
Partition Name: FOTA_SIG
Flash Filename: 
FOTA Filename: 


--- Entry #15 ---
Binary Type: 0 (AP)
Device Type: 2 (MMC)
Identifier: 12
Attributes: 5 (Read/Write)
Update Attributes: 1 (FOTA)
Partition Block Size/Offset: 83968
Partition Block Count: 2048
File Offset (Obsolete): 0
File Size (Obsolete): 0
Partition Name: l_runtimenv1
Flash Filename: 
FOTA Filename: 


--- Entry #16 ---
Binary Type: 0 (AP)
Device Type: 2 (MMC)
Identifier: 13
Attributes: 5 (Read/Write)
Update Attributes: 1 (FOTA)
Partition Block Size/Offset: 86016
Partition Block Count: 2048
File Offset (Obsolete): 0
File Size (Obsolete): 0
Partition Name: l_runtimenv2
Flash Filename: 
FOTA Filename: 


--- Entry #17 ---
Binary Type: 0 (AP)
Device Type: 2 (MMC)
Identifier: 14
Attributes: 5 (Read/Write)
Update Attributes: 1 (FOTA)
Partition Block Size/Offset: 88064
Partition Block Count: 2048
File Offset (Obsolete): 0
File Size (Obsolete): 0
Partition Name: td_runtimenv1
Flash Filename: 
FOTA Filename: 


--- Entry #18 ---
Binary Type: 0 (AP)
Device Type: 2 (MMC)
Identifier: 15
Attributes: 5 (Read/Write)
Update Attributes: 1 (FOTA)
Partition Block Size/Offset: 90112
Partition Block Count: 2048
File Offset (Obsolete): 0
File Size (Obsolete): 0
Partition Name: td_runtimenv2
Flash Filename: 
FOTA Filename: 


--- Entry #19 ---
Binary Type: 0 (AP)
Device Type: 2 (MMC)
Identifier: 16
Attributes: 5 (Read/Write)
Update Attributes: 1 (FOTA)
Partition Block Size/Offset: 92160
Partition Block Count: 4096
File Offset (Obsolete): 0
File Size (Obsolete): 0
Partition Name: PARAM
Flash Filename: param.lfs
FOTA Filename: 


--- Entry #20 ---
Binary Type: 0 (AP)
Device Type: 2 (MMC)
Identifier: 17
Attributes: 5 (Read/Write)
Update Attributes: 5 (FOTA)
Partition Block Size/Offset: 96256
Partition Block Count: 40960
File Offset (Obsolete): 0
File Size (Obsolete): 0
Partition Name: efs
Flash Filename: efs.img
FOTA Filename: 


--- Entry #21 ---
Binary Type: 0 (AP)
Device Type: 2 (MMC)
Identifier: 18
Attributes: 5 (Read/Write)
Update Attributes: 5 (FOTA)
Partition Block Size/Offset: 137216
Partition Block Count: 10240
File Offset (Obsolete): 0
File Size (Obsolete): 0
Partition Name: prodnv
Flash Filename: prodnv.img
FOTA Filename: 


--- Entry #22 ---
Binary Type: 0 (AP)
Device Type: 2 (MMC)
Identifier: 19
Attributes: 5 (Read/Write)
Update Attributes: 1 (FOTA)
Partition Block Size/Offset: 147456
Partition Block Count: 12288
File Offset (Obsolete): 0
File Size (Obsolete): 0
Partition Name: RESERVED2
Flash Filename: 
FOTA Filename: 


--- Entry #23 ---
Binary Type: 0 (AP)
Device Type: 2 (MMC)
Identifier: 20
Attributes: 5 (Read/Write)
Update Attributes: 1 (FOTA)
Partition Block Size/Offset: 159744
Partition Block Count: 40960
File Offset (Obsolete): 0
File Size (Obsolete): 0
Partition Name: KERNEL
Flash Filename: boot.img
FOTA Filename: 


--- Entry #24 ---
Binary Type: 0 (AP)
Device Type: 2 (MMC)
Identifier: 21
Attributes: 5 (Read/Write)
Update Attributes: 1 (FOTA)
Partition Block Size/Offset: 200704
Partition Block Count: 40960
File Offset (Obsolete): 0
File Size (Obsolete): 0
Partition Name: RECOVERY
Flash Filename: recovery.img
FOTA Filename: 


--- Entry #25 ---
Binary Type: 0 (AP)
Device Type: 2 (MMC)
Identifier: 22
Attributes: 5 (Read/Write)
Update Attributes: 1 (FOTA)
Partition Block Size/Offset: 241664
Partition Block Count: 2048
File Offset (Obsolete): 0
File Size (Obsolete): 0
Partition Name: PERSISTENT
Flash Filename: 
FOTA Filename: 


--- Entry #26 ---
Binary Type: 0 (AP)
Device Type: 2 (MMC)
Identifier: 23
Attributes: 5 (Read/Write)
Update Attributes: 5 (FOTA)
Partition Block Size/Offset: 243712
Partition Block Count: 18432
File Offset (Obsolete): 0
File Size (Obsolete): 0
Partition Name: PERSDATA
Flash Filename: persdata.img
FOTA Filename: 


--- Entry #27 ---
Binary Type: 0 (AP)
Device Type: 2 (MMC)
Identifier: 24
Attributes: 5 (Read/Write)
Update Attributes: 5 (FOTA)
Partition Block Size/Offset: 262144
Partition Block Count: 409600
File Offset (Obsolete): 0
File Size (Obsolete): 0
Partition Name: CACHE
Flash Filename: cache.img
FOTA Filename: 


--- Entry #28 ---
Binary Type: 0 (AP)
Device Type: 2 (MMC)
Identifier: 25
Attributes: 5 (Read/Write)
Update Attributes: 5 (FOTA)
Partition Block Size/Offset: 671744
Partition Block Count: 4194304
File Offset (Obsolete): 0
File Size (Obsolete): 0
Partition Name: SYSTEM
Flash Filename: system.img
FOTA Filename: 


--- Entry #29 ---
Binary Type: 0 (AP)
Device Type: 2 (MMC)
Identifier: 26
Attributes: 5 (Read/Write)
Update Attributes: 5 (FOTA)
Partition Block Size/Offset: 4866048
Partition Block Count: 81920
File Offset (Obsolete): 0
File Size (Obsolete): 0
Partition Name: HIDDEN
Flash Filename: hidden.img
FOTA Filename: 


--- Entry #30 ---
Binary Type: 0 (AP)
Device Type: 2 (MMC)
Identifier: 27
Attributes: 5 (Read/Write)
Update Attributes: 5 (FOTA)
Partition Block Size/Offset: 4947968
Partition Block Count: 0
File Offset (Obsolete): 0
File Size (Obsolete): 0
Partition Name: userdata
Flash Filename: userdata.img
FOTA Filename: remained

Ending session...
Releasing device interface...

Looking at the name, size and flash filename of the partitions, we can guess that the BOOT and BOOT2 are about splash screens, and what we need to use as our boot partition is in fact the KERNEL partition: /dev/block/mmcblk0p20 (note how the Identifier field from the PIT file matches what we got from block devices in the /dev filesystem). So, we edit our fstab file and add a line

/dev/block/platform/sdio_emmc/by-name/KERNEL   /boot    ext4 ro     wait

From the IRC #halium channel, I understood that the fixup mountpoints step is needed. The block I added to the halium/hybris-boot/fixup-mountpoints script is this:

    "j3xlte")
        sed -i \
            -e 's block/platform/sdio_emmc/by-name/SYSTEM mmcblk0p25 ' \
            -e 's block/platform/sdio_emmc/by-name/userdata mmcblk0p27 ' \
            -e 's block/platform/sdio_emmc/by-name/CACHE mmcblk0p24 ' \
            -e 's block/platform/sdio_emmc/by-name/efs mmcblk0p17 ' \
            -e 's block/platform/sdio_emmc/by-name/prodnv mmcblk0p18 ' \
            -e 's block/platform/sdio_emmc/by-name/KERNEL mmcblk0p20 ' \
            "$@"
        ;;

I then ran mka halium-boot again, and it finally started to build the kernel. But it soon stopped:

/mnt/Lavoro/halium/kernel/samsung/sharkls/kernel/cgroup.c: In function 'subsys_cgroup_allow_attach':
/mnt/Lavoro/halium/kernel/samsung/sharkls/kernel/cgroup.c:2138:37: error: invalid operands to binary != (have 'kuid_t' and 'kuid_t')
   if (current != task && cred->euid != tcred->uid &&
                                     ^
/mnt/Lavoro/halium/kernel/samsung/sharkls/kernel/cgroup.c:2139:18: error: invalid operands to binary != (have 'kuid_t' and 'kuid_t')
       cred->euid != tcred->suid)
                  ^

Luckily, this is one of the known build errors, so I didn't waste a lot of time on it: setting CONFIG_USER_NS=n in kernel/samsung/sharkls/arch/arm/configs/j3xlte_defconfig fixed it. Then the build failed with

  LD      init/built-in.o
mm/built-in.o: In function `set_AKSM_level':
/mnt/Lavoro/halium/kernel/samsung/sharkls/mm/ksm.c:1822: undefined reference to `get_minfree_high_value'
/mnt/Lavoro/halium/kernel/samsung/sharkls/Makefile:786: recipe for target 'vmlinux' failed
make[1]: *** [vmlinux] Error 1
Makefile:130: recipe for target 'sub-make' failed
make: *** [sub-make] Error 2

Looking at the code, I understood I could probably switch off this code path with CONFIG_ADAPTIVE_KSM=n. After doing that, got another error:

  LD      init/built-in.o
arch/arm/lib/lib.a(memcmpksm.o):(.ARM.exidx+0x0): undefined reference to `__aeabi_unwind_cpp_pr0'
/mnt/Lavoro/halium/kernel/samsung/sharkls/Makefile:786: recipe for target 'vmlinux' failed
make[1]: *** [vmlinux] Error 1
Makefile:130: recipe for target 'sub-make' failed
make: *** [sub-make] Error 2

Disabling also CONFIG_KSM_ASSEMBLY_MEMCMP did the trick:

  CC      /mnt/Lavoro/halium/vendor/sprd/wcn/wifi/sc2331/6.0/sprdwl.mod.o
  LD [M]  /mnt/Lavoro/halium/vendor/sprd/wcn/wifi/sc2331/6.0/sprdwl.ko
make[1]: Leaving directory '/mnt/Lavoro/halium/out/target/product/j3xlte/obj/KERNEL_OBJ'
make: Leaving directory '/mnt/Lavoro/halium/vendor/sprd/wcn/wifi/sc2331/6.0'
[ 77% 7/9] Target dt image: /mnt/Lavoro/halium/out/target/product/j3xlte/dt.img
DTB combiner:
  Input directory: '/mnt/Lavoro/halium/out/target/product/j3xlte/obj/KERNEL_OBJ/arch/arm/boot/dts/'
  Output file: '/mnt/Lavoro/halium/out/target/product/j3xlte/dt.img'
Found file: sprd-scx35l_sharkls_j3xlte_rev03.dtb ... chipset: 9830, platform: 3, rev: 131072
Found file: sprd-scx35l_sharkls_j3xlte_rev02.dtb ... chipset: 9830, platform: 2, rev: 131072
Found file: sprd-scx35l_sharkls_j3xlte_rev06.dtb ... chipset: 9830, platform: 6, rev: 131072
Found file: sprd-scx35l_sharkls_j3xlte_rev07.dtb ... chipset: 9830, platform: 7, rev: 131072
Found file: sprd-scx35l_sharkls_j3xlte_rev04.dtb ... chipset: 9830, platform: 4, rev: 131072
Found file: sprd-scx35l_sharkls_j3xlte_rev01.dtb ... chipset: 9830, platform: 1, rev: 131072
Found file: sprd-scx35l_sharkls_j3xlte_rev05.dtb ... chipset: 9830, platform: 5, rev: 131072
Found file: sprd-scx35l_sharkls_j3xlte_rev00.dtb ... chipset: 9830, platform: 0, rev: 131072
=> Found 8 unique DTB(s)

Generating master DTB... completed
Made DT image: /mnt/Lavoro/halium/out/target/product/j3xlte/dt.img
[100% 9/9] Install: /mnt/Lavoro/halium/out/target/product/j3xlte/halium-boot.img
make: Leaving directory '/mnt/Lavoro/halium'

#### make completed successfully (01:28 (mm:ss)) ####

Building the Halium system image

Continuing following the Halium documentation, the next step was building the system image with the mka systemimage command. Of course, it failed:

make: Entering directory '/mnt/Lavoro/halium'
============================================
PLATFORM_VERSION_CODENAME=REL
PLATFORM_VERSION=7.1.1
LINEAGE_VERSION=14.1-20190702-UNOFFICIAL-j3xlte
TARGET_PRODUCT=lineage_j3xlte
TARGET_BUILD_VARIANT=userdebug
TARGET_BUILD_TYPE=release
TARGET_BUILD_APPS=
TARGET_ARCH=arm
TARGET_ARCH_VARIANT=armv7-a-neon
TARGET_CPU_VARIANT=cortex-a7
TARGET_2ND_ARCH=
TARGET_2ND_ARCH_VARIANT=
TARGET_2ND_CPU_VARIANT=
HOST_ARCH=x86_64
HOST_2ND_ARCH=x86
HOST_OS=linux
HOST_OS_EXTRA=Linux-4.15.0-52-generic-x86_64-with-Ubuntu-16.04-xenial
HOST_CROSS_OS=windows
HOST_CROSS_ARCH=x86
HOST_CROSS_2ND_ARCH=x86_64
HOST_BUILD_TYPE=release
BUILD_ID=NOF27B
OUT_DIR=/mnt/Lavoro/halium/out
RECOVERY_VARIANT=twrp
WITH_SU=true
============================================
Running kati to generate build-lineage_j3xlte.ninja...
No need to regenerate ninja file
Starting build with ninja
ninja: Entering directory `.'
ninja: error: '/mnt/Lavoro/halium/out/target/product/j3xlte/obj/SHARED_LIBRARIES/libcurl_intermediates/export_includes', needed by '/mnt/Lavoro/halium/out/target/product/j3xlte/obj/SHARED_LIBRARIES/libstrongswan_intermediates/import_includes', missing and no known rule to make it
build/core/ninja.mk:151: recipe for target 'ninja_wrapper' failed
make: *** [ninja_wrapper] Error 1
make: Leaving directory '/mnt/Lavoro/halium'

#### make failed to build some targets (2 seconds) ####

I found similar (though not exactly the same) errors in #halium: the solution is to locate the .mk file causing the build of the package requiring the missing file, and remove it. I ran a repo grep libstrongswan, and found that it's mentioned in device/samsung/sharkls-common/sharkls.mk. I removed it from there, but I got the same build error. Removing the whole block of PRODUCT_PACKAGES led to some progress:

build/core/binary.mk:1253: vendor/sprd/modules/libcamera/Android.mk: libcamisp2.0: Unused source files: isp2.0/calibration/backup isp2.0/third_lib/alc_ip
build/core/binary.mk:1253: vendor/sprd/open-source/libs/vpu/mmf/openmax/libomxil-bellagio-0.9.3/Android.mk: omxregister-bellagio: Unused source files: src/omxregister.h
No private recovery resources for TARGET_DEVICE j3xlte
build/core/Makefile:34: warning: overriding commands for target `/mnt/Lavoro/halium/out/target/product/j3xlte/system/etc/wifi/wpa_supplicant.conf'
build/core/base_rules.mk:316: warning: ignoring old commands for target `/mnt/Lavoro/halium/out/target/product/j3xlte/system/etc/wifi/wpa_supplicant.conf'
Starting build with ninja
ninja: Entering directory `.'
ninja: error: '/mnt/Lavoro/halium/out/target/product/j3xlte/obj/STATIC_LIBRARIES/libbt-utils_intermediates/export_includes', needed by '/mnt/Lavoro/halium/out/target/product/j3xlte/obj/EXECUTABLES/engpc_intermediates/import_includes', missing and no known rule to make it
build/core/ninja.mk:151: recipe for target 'ninja_wrapper' failed
make: *** [ninja_wrapper] Error 1

I then removed engpc from the same sharkls.mk file:

Starting build with ninja
ninja: Entering directory `.'
ninja: error: '/mnt/Lavoro/halium/out/target/product/j3xlte/obj/STATIC_LIBRARIES/libbootloader_message_intermediates/export_includes', needed by '/mnt/Lavoro/halium/out/target/product/j3xlte/obj/EXECUTABLES/vold_intermediates/import_includes', missing and no known rule to make it
build/core/ninja.mk:151: recipe for target 'ninja_wrapper' failed
make: *** [ninja_wrapper] Error 1

The last comment from here suggests that vold might not be needed (though some people in #halium claimed that it could be needed, and indeed in the Meizu Pro 5 by Canonical I do have that program under /system/bin/). Anyway, I edited build/target/product/base.mk and removed vold from the PRODUCT_PACKAGES variable. I also read that it's recommended to run the make command as LANG=C mka systemimage, since the outcome can be affected by the user locale. Next:

Starting build with ninja
ninja: Entering directory `.'
ninja: error: '/mnt/Lavoro/halium/out/target/product/j3xlte/obj/SHARED_LIBRARIES/libfmjni_intermediates/export_includes', needed by '/mnt/Lavoro/halium/out/target/product/j3xlte/obj/SHARED_LIBRARIES/libengfm_intermediates/import_includes', missing and no known rule to make it
build/core/ninja.mk:151: recipe for target 'ninja_wrapper' failed
make: *** [ninja_wrapper] Error 1

This is mentioned in vendor/sprd/wcn/platform/wcnd/Android.mk, so let's remove it from there (LOCAL_SHARED_LIBRARIES variable):

Starting build with ninja
ninja: Entering directory `.'
ninja: error: 'vendor/samsung/j3xlte/proprietary/bin/at_distributor', needed by '/mnt/Lavoro/halium/out/target/product/j3xlte/system/bin/at_distributor', missing and no known rule to make it
build/core/ninja.mk:151: recipe for target 'ninja_wrapper' failed
make: *** [ninja_wrapper] Error 1

This is brought in by device/samsung/j3xlte/proprietary-files.txt, so I commented it out from there. This didn't seem to make any difference, and a mka clean didn't help either. Turns out it's also mentioned in vendor/samsung/j3xlte/j3xlte-vendor-blobs.mk, so I removed it from there too (PRODUCT_COPY_FILES variable). Reading the comment block on top of this makefile we understand that it's generated by the setup-makefiles.sh script based on the contents of proprietary-files.txt. So, one option is to re-run the setup-makefiles.sh script, but I preferred to take a no risk approach and edit both files by hand. The next failure was about mfgloader, which is also mentioned in the same file:

Starting build with ninja
ninja: Entering directory `.'
ninja: error: 'vendor/samsung/j3xlte/proprietary/bin/mfgloader', needed by '/mnt/Lavoro/halium/out/target/product/j3xlte/system/bin/mfgloader', missing and no known rule to make it
build/core/ninja.mk:151: recipe for target 'ninja_wrapper' failed
make: *** [ninja_wrapper] Error 1

Let's try to remove it as well, from the same two files:

Starting build with ninja
ninja: Entering directory `.'
ninja: error: 'vendor/samsung/j3xlte/proprietary/bin/sprdSleepLog', needed by '/mnt/Lavoro/halium/out/target/product/j3xlte/system/bin/sprdSleepLog', missing and no known rule to make it
build/core/ninja.mk:151: recipe for target 'ninja_wrapper' failed
make: *** [ninja_wrapper] Error 1

Next is modemd (exactly same logs). At this point I started suspecting that this was not the right way to proceed, that that instead I was just missing all the proprietary blobs. The device tree repository has an extract-files.sh which seems to get these binaries from a device. After looking at its source, it looks like it must be run from the device/samsung/j3xlte directory with the device connected and reachable via adb. Before running it, I reverted my changes to the proprietary-files.txt file, in order to retrieve all the needed blobs. Running the extraction tool failed once:

./extract-files.sh 
846 KB/s (92260 bytes in 0.106s)
108 KB/s (9664 bytes in 0.086s)
106 KB/s (9444 bytes in 0.086s)
279 KB/s (34092 bytes in 0.119s)
44 KB/s (1992 bytes in 0.043s)
115 KB/s (5296 bytes in 0.044s)
2 KB/s (265 bytes in 0.087s)
40 KB/s (3770 bytes in 0.090s)
25 KB/s (2247 bytes in 0.087s)
4 KB/s (427 bytes in 0.087s)
0 KB/s (70 bytes in 0.087s)
remote object '/system/bin/IPSecService' does not exist

but after removing the missing file from proprietary-files.txt, the extraction completed successfully. However, one file was still not retrieved, as this line from the output showed:

[...]
4779 KB/s (1386252 bytes in 0.283s)
405 KB/s (37928 bytes in 0.091s)
failed to copy '/system/vendor/firmware/vbc_eq' to '../../../vendor/samsung/j3xlte/proprietary/vendor/firmware/vbc_eq': Permission denied
3015 KB/s (415875 bytes in 0.134s)
[...]

I checked the situation by connecting to the device with adb shell, and the file was there but readable by root only. After copying the file to /data/user and calling chmod 0666 vbc_eq, I could manually retrieve it with

adb pull /data/user/vbc_eq ../../../vendor/samsung/j3xlte/proprietary/vendor/firmware/vbc_eq

I preferred not to touch the permissions directly on the original file, not to risk messing up with the system; and after retrieving the file, I removed my copy from the device. The I ran

JOBS=2 ./halium/devices/setup j3xlte

once again. That completed fine, and the next mka systemimage failed with another error:

Starting build with ninja
ninja: Entering directory `.'
ninja: error: '/mnt/Lavoro/halium/out/target/product/j3xlte/obj/STATIC_LIBRARIES/libbootloader_message_intermediates/export_includes', needed by '/mnt/Lavoro/halium/out/target/product/j3xlte/obj/EXECUTABLES/init_intermediates/import_includes', missing and no known rule to make it
build/core/ninja.mk:151: recipe for target 'ninja_wrapper' failed
make: *** [ninja_wrapper] Error 1

I played with the android_system_core repository, which is checked out in system/core/, to remove references to bootloader_message from init/builtins.cpp, and I also removed it from the LOCAL_STATIC_LIBRARIES variable in the system/core/init/Android.mk file. Next:

Starting build with ninja
ninja: Entering directory `.'
ninja: error: '/mnt/Lavoro/halium/out/target/product/j3xlte/obj/STATIC_LIBRARIES/libminui_intermediates/export_includes', needed by '/mnt/Lavoro/halium/out/target/product/j3xlte/obj/EXECUTABLES/healthd_intermediates/import_includes', missing and no known rule to make it
build/core/ninja.mk:151: recipe for target 'ninja_wrapper' failed
make: *** [ninja_wrapper] Error 1

This is listed in build/target/product/core_base.mk and build/target/product/embedded.mk and needs to be removed from both (it was disabled in Halium's init, so it's definitely not needed). It then failed with:

Starting build with ninja
ninja: Entering directory `.'
ninja: error: 'bootable/recovery-twrp/etc/init.rc', needed by '/mnt/Lavoro/halium/out/target/product/j3xlte/ramdisk-recovery.cpio', missing and no known rule to make it
build/core/ninja.mk:151: recipe for target 'ninja_wrapper' failed
make: *** [ninja_wrapper] Error 1

The recovery-twrp project is listed in device/samsung/sharkls-common/lineage.dependencies, so let's try adding a line

<project path="bootable/recovery-twrp" name="omnirom/android_bootable_recovery" remote="github" revision="android-7.1" />

to our manifest file and run JOBS=2 ./halium/devices/setup j3xlte once more. After that, mka systemimage gives another error:

Starting build with ninja
ninja: Entering directory `.'
ninja: error: '/mnt/Lavoro/halium/out/host/linux-x86/bin/imgdiff', needed by '/mnt/Lavoro/halium/out/target/product/j3xlte/obj/PACKAGING/recovery_patch_intermediates/recovery_from_boot.p', missing and no known rule to make it
build/core/ninja.mk:151: recipe for target 'ninja_wrapper' failed
make: *** [ninja_wrapper] Error 1

Adding bootable/recovery-twrp to the subdirs variable in build/core/main.mk seems to get us further. But before showing you the next error, I need to stop for a second, because at this point I started suspecting that the previous error about the bootloader_message library could go away now that I started using the same recovery-twrp tree used by the LineageOS porter. So I reverted my changes about the bootloader_message library and indeed it looked like the previous error was gone forever. And it might be that some of the other stuff I disabled above might be restored back; but I decided to leave it for later (it's documented in this blog post, after all, so I can go back to it at any time) and proceed with the build:

Starting build with ninja
ninja: Entering directory `.'
[  0% 125/16364] host C++: libadb <= system/core/adb/usb_linux.cpp
FAILED: /bin/bash -c "(PWD=/proc/self/cwd  prebuilts/clang/host/linux-x86/clang-2690385/bin/clang++ -I device/samsung/sharkls-common/include -I system/core/adb -I /mnt/Lavoro/halium/out/host/linux-x86/obj/STATIC_LIBRARIES/libadb_intermediates -I /mnt/Lavoro/halium/out/host/linux-x86/gen/STATIC_LIBRARIES/libadb_intermediates -I libnativehelper/include/nativehelper \$(cat /mnt/Lavoro/halium/out/host/linux-x86/obj/STATIC_LIBRARIES/libadb_intermediates/import_includes) -isystem system/core/include -isystem system/media/audio/include -isystem hardware/libhardware/include -isystem hardware/libhardware_legacy/include -isystem libnativehelper/include -isystem frameworks/native/include -isystem frameworks/native/opengl/include -isystem frameworks/av/include -isystem frameworks/base/include -isystem tools/include -isystem /mnt/Lavoro/halium/out/host/linux-x86/obj/include -c    -fno-exceptions -Wno-multichar -m64 -Wa,--noexecstack -fPIC -no-canonical-prefixes -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=2 -fstack-protector -D__STDC_FORMAT_MACROS -D__STDC_CONSTANT_MACROS -DANDROID -fmessage-length=0 -W -Wall -Wno-unused -Winit-self -Wpointer-arith -O2 -g -fno-strict-aliasing -DNDEBUG -UDEBUG  -D__compiler_offsetof=__builtin_offsetof -Werror=int-conversion -Wno-reserved-id-macro -Wno-format-pedantic -Wno-unused-command-line-argument -fcolor-diagnostics   --gcc-toolchain=prebuilts/gcc/linux-x86/host/x86_64-linux-glibc2.15-4.8 -fstack-protector-strong    --gcc-toolchain=prebuilts/gcc/linux-x86/host/x86_64-linux-glibc2.15-4.8 --sysroot prebuilts/gcc/linux-x86/host/x86_64-linux-glibc2.15-4.8/sysroot -Bprebuilts/gcc/linux-x86/host/x86_64-linux-glibc2.15-4.8/x86_64-linux/bin -target x86_64-linux-gnu   -Wsign-promo  -Wno-inconsistent-missing-override   --gcc-toolchain=prebuilts/gcc/linux-x86/host/x86_64-linux-glibc2.15-4.8 --sysroot prebuilts/gcc/linux-x86/host/x86_64-linux-glibc2.15-4.8/sysroot -isystem prebuilts/gcc/linux-x86/host/x86_64-linux-glibc2.15-4.8/x86_64-linux/include/c++/4.8 -isystem prebuilts/gcc/linux-x86/host/x86_64-linux-glibc2.15-4.8/x86_64-linux/include/c++/4.8/x86_64-linux -isystem prebuilts/gcc/linux-x86/host/x86_64-linux-glibc2.15-4.8/x86_64-linux/include/c++/4.8/backward -target x86_64-linux-gnu  -Wall -Wextra -Werror -Wno-unused-parameter -Wno-missing-field-initializers -Wvla -DADB_REVISION='\"80811505f1ff-android\"' -fvisibility=hidden -DADB_HOST=1 -fPIC -std=c++14 -Wexit-time-destructors -D_USING_LIBCXX -std=gnu++14 -nostdinc++  -Werror=int-to-pointer-cast -Werror=pointer-to-int-cast  -Werror=address-of-temporary -Werror=null-dereference -Werror=return-type    -MD -MF /mnt/Lavoro/halium/out/host/linux-x86/obj/STATIC_LIBRARIES/libadb_intermediates/usb_linux.d -o /mnt/Lavoro/halium/out/host/linux-x86/obj/STATIC_LIBRARIES/libadb_intermediates/usb_linux.o system/core/adb/usb_linux.cpp ) && (cp /mnt/Lavoro/halium/out/host/linux-x86/obj/STATIC_LIBRARIES/libadb_intermediates/usb_linux.d /mnt/Lavoro/halium/out/host/linux-x86/obj/STATIC_LIBRARIES/libadb_intermediates/usb_linux.P; sed -e 's/#.*//' -e 's/^[^:]*: *//' -e 's/ *\\\\\$//' -e '/^\$/ d' -e 's/\$/ :/' < /mnt/Lavoro/halium/out/host/linux-x86/obj/STATIC_LIBRARIES/libadb_intermediates/usb_linux.d >> /mnt/Lavoro/halium/out/host/linux-x86/obj/STATIC_LIBRARIES/libadb_intermediates/usb_linux.P; rm -f /mnt/Lavoro/halium/out/host/linux-x86/obj/STATIC_LIBRARIES/libadb_intermediates/usb_linux.d )"
In file included from system/core/adb/usb_linux.cpp:25:
In file included from device/samsung/sharkls-common/include/linux/usb/ch9.h:35:
In file included from device/samsung/sharkls-common/include/uapi/linux/usb/ch9.h:36:
In file included from device/samsung/sharkls-common/include/linux/types.h:5:
In file included from device/samsung/sharkls-common/include/uapi/linux/types.h:13:
In file included from prebuilts/gcc/linux-x86/host/x86_64-linux-glibc2.15-4.8/sysroot/usr/include/linux/posix_types.h:4:
device/samsung/sharkls-common/include/linux/stddef.h:11:2: error: expected identifier
        false   = 0,
        ^
prebuilts/clang/host/linux-x86/clang-2690385/bin/../lib64/clang/3.8.256229/include/stdbool.h:38:15: note: expanded from macro 'false'
#define false false
              ^

(see here for the full log). After some resultless internet search and asking around, I tried to look into the issue myself, and noticed that in that failed command there's a mix of headers coming from the userspace and from the kernel (which usually are not used together). Since we are building userspace programs now, it seems a bit strange that there are also kernel headers being used (those under device/samsung/sharkls-common). Looking at system/core/adb/usb_linux.cpp, I saw that it contains a line #include <linux/usb/ch9.h> which causes the inclusion of device/samsung/sharkls-common/include/linux/usb/ch9.h; but maybe there are other ch9.h files in the build tree, provided by userspace libraries and not by the kernel? Indeed, a find showed that I had also a ./prebuilts/gcc/linux-x86/host/x86_64-linux-glibc2.15-4.8/sysroot/usr/include/linux/usb/ch9.h file, which should probably be used instead of the kernel's one. So I copied the failing command line (starting from what comes after FAILED:) into a terminal and ran it, and could reproduce the issue; but if after removing the first include directive (-I device/samsung/sharkls-common/include) the command succeeded.

The line is added in build/core/binary.mk:

my_c_includes := $(TOPDIR)$(TARGET_SPECIFIC_HEADER_PATH) $(my_c_includes)

(I found it out by modifying the file, and adding random text strings around the right-hand parts of these assignments, then running the build again and see which random strings were surrounding the device/samsung/sharkls-common/include text in the failing command line). So, we need to find why TARGET_SPECIFIC_HEADER_PATH is set. I ran a repo grep TARGET_SPECIFIC_HEADER_PATH and found that it's being set in device/samsung/sharkls-common/BoardConfigCommon.mk. Browsing the various directory trees, and double-checking the manifest file, I realized that the device/samsung/sharkls-common should not contain the kernel source tree, yet in my local disk it did (and all the files were untracked by git). It was probably the result of one of my previous attempts of compiling the manifest file, where some projects were checked out locally but I didn't clean them up after updating the manifest. I therefore ran a git clean -dfx in that directory, and the kernel source were gone. The next build continued for a while, then failed:

[ 76% 12454/16364] target thumb C: libengbt <= vendor/sprd/open-source/apps/engmode/bt/bt_cmd_executer.c
FAILED: /bin/bash -c "(PWD=/proc/self/cwd  prebuilts/clang/host/linux-x86/clang-2690385/bin/clang -I device/samsung/sharkls-common/include -I system/bt/stack/include -I system/bt/include -I vendor/sprd/open-source/apps/engmode/bt -I /mnt/Lavoro/halium/out/target/product/j3xlte/obj/SHARED_LIBRARIES/libengbt_intermediates -I /mnt/Lavoro/halium/out/target/product/j3xlte/gen/SHARED_LIBRARIES/libengbt_intermediates -I libnativehelper/include/nativehelper \$(cat /mnt/Lavoro/halium/out/target/product/j3xlte/obj/SHARED_LIBRARIES/libengbt_intermediates/import_includes) -isystem system/core/include -isystem system/media/audio/include -isystem hardware/libhardware/include -isystem hardware/libhardware_legacy/include -isystem libnativehelper/include -isystem frameworks/native/include -isystem frameworks/native/opengl/include -isystem frameworks/av/include -isystem frameworks/base/include -isystem hardware/ril/include -isystem /mnt/Lavoro/halium/out/target/product/j3xlte/obj/include -isystem bionic/libc/arch-arm/include -isystem bionic/libc/include -isystem bionic/libc/kernel/uapi -isystem bionic/libc/kernel/common -isystem bionic/libc/kernel/uapi/asm-arm -isystem bionic/libm/include -isystem bionic/libm/include/arm -c    -fno-exceptions -Wno-multichar -msoft-float -ffunction-sections -fdata-sections -funwind-tables -fstack-protector-strong -Wa,--noexecstack -Werror=format-security -D_FORTIFY_SOURCE=2 -fno-short-enums -no-canonical-prefixes -mcpu=cortex-a7 -mfpu=neon-vfpv4 -D__ARM_FEATURE_LPAE=1 -mfloat-abi=softfp -DANDROID -fmessage-length=0 -W -Wall -Wno-unused -Winit-self -Wpointer-arith -Werror=return-type -Werror=non-virtual-dtor -Werror=address -Werror=sequence-point -Werror=date-time -DNDEBUG -g -Wstrict-aliasing=2 -DNDEBUG -UDEBUG  -D__compiler_offsetof=__builtin_offsetof -Werror=int-conversion -Wno-reserved-id-macro -Wno-format-pedantic -Wno-unused-command-line-argument -fcolor-diagnostics -nostdlibinc  -target arm-linux-androideabi    -target arm-linux-androideabi -Bprebuilts/gcc/linux-x86/arm/arm-linux-androideabi-4.9/arm-linux-androideabi/bin    -std=gnu99     -mthumb -Os -fomit-frame-pointer -fno-strict-aliasing   -DHAS_NO_BDROID_BUILDCFG -DSPRD_WCNBT_MARLIN -fPIC -D_USING_LIBCXX   -Werror=int-to-pointer-cast -Werror=pointer-to-int-cast  -Werror=address-of-temporary -Werror=null-dereference -Werror=return-type  -MD -MF /mnt/Lavoro/halium/out/target/product/j3xlte/obj/SHARED_LIBRARIES/libengbt_intermediates/bt_cmd_executer.d -o /mnt/Lavoro/halium/out/target/product/j3xlte/obj/SHARED_LIBRARIES/libengbt_intermediates/bt_cmd_executer.o vendor/sprd/open-source/apps/engmode/bt/bt_cmd_executer.c ) && (cp /mnt/Lavoro/halium/out/target/product/j3xlte/obj/SHARED_LIBRARIES/libengbt_intermediates/bt_cmd_executer.d /mnt/Lavoro/halium/out/target/product/j3xlte/obj/SHARED_LIBRARIES/libengbt_intermediates/bt_cmd_executer.P; sed -e 's/#.*//' -e 's/^[^:]*: *//' -e 's/ *\\\\\$//' -e '/^\$/ d' -e 's/\$/ :/' < /mnt/Lavoro/halium/out/target/product/j3xlte/obj/SHARED_LIBRARIES/libengbt_intermediates/bt_cmd_executer.d >> /mnt/Lavoro/halium/out/target/product/j3xlte/obj/SHARED_LIBRARIES/libengbt_intermediates/bt_cmd_executer.P; rm -f /mnt/Lavoro/halium/out/target/product/j3xlte/obj/SHARED_LIBRARIES/libengbt_intermediates/bt_cmd_executer.d )"
vendor/sprd/open-source/apps/engmode/bt/bt_cmd_executer.c:30:10: fatal error: 'bt_types.h' file not found
#include "bt_types.h"
         ^
1 error generated.

Indeed the bt_types.h was nowhere to be found in my source tree. A search in GitHub reveals that this file is provided by libncf-nci, so it's probably related to NFC. The local manifest from LineageOS indeed had a few lines with NFC-related projects (at least, judging from their name):

<project path="vendor/nxp-nfc/opensource/Nfc" name="LineageOS/android_vendor_nxp-nfc_opensource_Nfc" revision="cm-14.1" />
<project path="vendor/nxp-nfc/opensource/frameworks" name="LineageOS/android_vendor_nxp-nfc_opensource_frameworks" revision="cm-14.1" />
<project path="vendor/nxp-nfc/opensource/libnfc-nci" name="LineageOS/android_vendor_nxp-nfc_opensource_libnfc-nci" revision="cm-14.1" />

I initially excluded them from my manifest, given that my device doesn't have NFC, but at this point I thought I'd better include them, just for the sake of building the binaries (the alternative being removing the binaries required these libraries, or hacking on them to remove the dependency; this is a decision I might want to revisit later). So I added them to my manifest file (adding the proper remote attribute!), re-ran the setup script which fetched the newly-added repositories, but the build still failed with the same error, as the include lines were not updated. However, adding vendor/nxp-nfc/opensource/libnfc-nci/src/include to the TARGET_SPECIFIC_HEADER_PATH variable in device/samsung/sharkls-common/BoardConfigCommon.mk caused the build to fail even earlier. I then tried to add it to the local project being built (engmode), by changing the LOCAL_C_INCLUDES variable in vendor/sprd/open-source/apps/engmode/bt/Android.mk and that brought me just a little further:

[ 97% 1507/1552] target thumb C: libengbt <= vendor/sprd/open-source/apps/engmode/bt/bt_cmd_executer.c
FAILED: /bin/bash -c "(PWD=/proc/self/cwd  prebuilts/clang/host/linux-x86/clang-2690385/bin/clang -I device/samsung/sharkls-common/include -I system/bt/stack/include -I vendor/nxp-nfc/opensource/libnfc-nci/src/include -I system/bt/include -I vendor/sprd/open-source/apps/engmode/bt -I /mnt/Lavoro/halium/out/target/product/j3xlte/obj/SHARED_LIBRARIES/libengbt_intermediates -I /mnt/Lavoro/halium/out/target/product/j3xlte/gen/SHARED_LIBRARIES/libengbt_intermediates -I libnativehelper/include/nativehelper \$(cat /mnt/Lavoro/halium/out/target/product/j3xlte/obj/SHARED_LIBRARIES/libengbt_intermediates/import_includes) -isystem system/core/include -isystem system/media/audio/include -isystem hardware/libhardware/include -isystem hardware/libhardware_legacy/include -isystem libnativehelper/include -isystem frameworks/native/include -isystem frameworks/native/opengl/include -isystem frameworks/av/include -isystem frameworks/base/include -isystem hardware/ril/include -isystem /mnt/Lavoro/halium/out/target/product/j3xlte/obj/include -isystem bionic/libc/arch-arm/include -isystem bionic/libc/include -isystem bionic/libc/kernel/uapi -isystem bionic/libc/kernel/common -isystem bionic/libc/kernel/uapi/asm-arm -isystem bionic/libm/include -isystem bionic/libm/include/arm -c    -fno-exceptions -Wno-multichar -msoft-float -ffunction-sections -fdata-sections -funwind-tables -fstack-protector-strong -Wa,--noexecstack -Werror=format-security -D_FORTIFY_SOURCE=2 -fno-short-enums -no-canonical-prefixes -mcpu=cortex-a7 -mfpu=neon-vfpv4 -D__ARM_FEATURE_LPAE=1 -mfloat-abi=softfp -DANDROID -fmessage-length=0 -W -Wall -Wno-unused -Winit-self -Wpointer-arith -Werror=return-type -Werror=non-virtual-dtor -Werror=address -Werror=sequence-point -Werror=date-time -DNDEBUG -g -Wstrict-aliasing=2 -DNDEBUG -UDEBUG  -D__compiler_offsetof=__builtin_offsetof -Werror=int-conversion -Wno-reserved-id-macro -Wno-format-pedantic -Wno-unused-command-line-argument -fcolor-diagnostics -nostdlibinc  -target arm-linux-androideabi    -target arm-linux-androideabi -Bprebuilts/gcc/linux-x86/arm/arm-linux-androideabi-4.9/arm-linux-androideabi/bin    -std=gnu99     -mthumb -Os -fomit-frame-pointer -fno-strict-aliasing   -DHAS_NO_BDROID_BUILDCFG -DSPRD_WCNBT_MARLIN -fPIC -D_USING_LIBCXX   -Werror=int-to-pointer-cast -Werror=pointer-to-int-cast  -Werror=address-of-temporary -Werror=null-dereference -Werror=return-type  -MD -MF /mnt/Lavoro/halium/out/target/product/j3xlte/obj/SHARED_LIBRARIES/libengbt_intermediates/bt_cmd_executer.d -o /mnt/Lavoro/halium/out/target/product/j3xlte/obj/SHARED_LIBRARIES/libengbt_intermediates/bt_cmd_executer.o vendor/sprd/open-source/apps/engmode/bt/bt_cmd_executer.c ) && (cp /mnt/Lavoro/halium/out/target/product/j3xlte/obj/SHARED_LIBRARIES/libengbt_intermediates/bt_cmd_executer.d /mnt/Lavoro/halium/out/target/product/j3xlte/obj/SHARED_LIBRARIES/libengbt_intermediates/bt_cmd_executer.P; sed -e 's/#.*//' -e 's/^[^:]*: *//' -e 's/ *\\\\\$//' -e '/^\$/ d' -e 's/\$/ :/' < /mnt/Lavoro/halium/out/target/product/j3xlte/obj/SHARED_LIBRARIES/libengbt_intermediates/bt_cmd_executer.d >> /mnt/Lavoro/halium/out/target/product/j3xlte/obj/SHARED_LIBRARIES/libengbt_intermediates/bt_cmd_executer.P; rm -f /mnt/Lavoro/halium/out/target/product/j3xlte/obj/SHARED_LIBRARIES/libengbt_intermediates/bt_cmd_executer.d )"
In file included from vendor/sprd/open-source/apps/engmode/bt/bt_cmd_executer.c:30:
vendor/nxp-nfc/opensource/libnfc-nci/src/include/bt_types.h:22:10: fatal error: 'data_types.h' file not found
#include "data_types.h"
         ^
1 error generated.

(note that in order to speed up the development I was no longer running mka systemimage, but mka libengbt, which is the library affected by the failure). I had three candidates:

vendor/nxp-nfc/opensource/libnfc-nci/p61-jcop-kit/inc/data_types.h
vendor/nxp-nfc/opensource/libnfc-nci/src/gki/ulinux/data_types.h
vendor/nxp-nfc/opensource/libnfc-nci/halimpl/bcm2079x/gki/ulinux/data_types.h

they didn't seem to be very different, so I picked the first one. I got then another failure:

[ 35% 12/34] target thumb C: libengbt <= vendor/sprd/open-source/apps/engmode/bt/bt_cmd_executer.c
FAILED: /bin/bash -c "(PWD=/proc/self/cwd  prebuilts/clang/host/linux-x86/clang-2690385/bin/clang -I device/samsung/sharkls-common/include -I system/bt/stack/include -I vendor/nxp-nfc/opensource/libnfc-nci/src/include -I vendor/nxp-nfc/opensource/libnfc-nci/p61-jcop-kit/inc/ -I system/bt/include -I vendor/sprd/open-source/apps/engmode/bt -I /mnt/Lavoro/halium/out/target/product/j3xlte/obj/SHARED_LIBRARIES/libengbt_intermediates -I /mnt/Lavoro/halium/out/target/product/j3xlte/gen/SHARED_LIBRARIES/libengbt_intermediates -I libnativehelper/include/nativehelper \$(cat /mnt/Lavoro/halium/out/target/product/j3xlte/obj/SHARED_LIBRARIES/libengbt_intermediates/import_includes) -isystem system/core/include -isystem system/media/audio/include -isystem hardware/libhardware/include -isystem hardware/libhardware_legacy/include -isystem libnativehelper/include -isystem frameworks/native/include -isystem frameworks/native/opengl/include -isystem frameworks/av/include -isystem frameworks/base/include -isystem hardware/ril/include -isystem /mnt/Lavoro/halium/out/target/product/j3xlte/obj/include -isystem bionic/libc/arch-arm/include -isystem bionic/libc/include -isystem bionic/libc/kernel/uapi -isystem bionic/libc/kernel/common -isystem bionic/libc/kernel/uapi/asm-arm -isystem bionic/libm/include -isystem bionic/libm/include/arm -c    -fno-exceptions -Wno-multichar -msoft-float -ffunction-sections -fdata-sections -funwind-tables -fstack-protector-strong -Wa,--noexecstack -Werror=format-security -D_FORTIFY_SOURCE=2 -fno-short-enums -no-canonical-prefixes -mcpu=cortex-a7 -mfpu=neon-vfpv4 -D__ARM_FEATURE_LPAE=1 -mfloat-abi=softfp -DANDROID -fmessage-length=0 -W -Wall -Wno-unused -Winit-self -Wpointer-arith -Werror=return-type -Werror=non-virtual-dtor -Werror=address -Werror=sequence-point -Werror=date-time -DNDEBUG -g -Wstrict-aliasing=2 -DNDEBUG -UDEBUG  -D__compiler_offsetof=__builtin_offsetof -Werror=int-conversion -Wno-reserved-id-macro -Wno-format-pedantic -Wno-unused-command-line-argument -fcolor-diagnostics -nostdlibinc  -target arm-linux-androideabi    -target arm-linux-androideabi -Bprebuilts/gcc/linux-x86/arm/arm-linux-androideabi-4.9/arm-linux-androideabi/bin    -std=gnu99     -mthumb -Os -fomit-frame-pointer -fno-strict-aliasing   -DHAS_NO_BDROID_BUILDCFG -DSPRD_WCNBT_MARLIN -fPIC -D_USING_LIBCXX   -Werror=int-to-pointer-cast -Werror=pointer-to-int-cast  -Werror=address-of-temporary -Werror=null-dereference -Werror=return-type  -MD -MF /mnt/Lavoro/halium/out/target/product/j3xlte/obj/SHARED_LIBRARIES/libengbt_intermediates/bt_cmd_executer.d -o /mnt/Lavoro/halium/out/target/product/j3xlte/obj/SHARED_LIBRARIES/libengbt_intermediates/bt_cmd_executer.o vendor/sprd/open-source/apps/engmode/bt/bt_cmd_executer.c ) && (cp /mnt/Lavoro/halium/out/target/product/j3xlte/obj/SHARED_LIBRARIES/libengbt_intermediates/bt_cmd_executer.d /mnt/Lavoro/halium/out/target/product/j3xlte/obj/SHARED_LIBRARIES/libengbt_intermediates/bt_cmd_executer.P; sed -e 's/#.*//' -e 's/^[^:]*: *//' -e 's/ *\\\\\$//' -e '/^\$/ d' -e 's/\$/ :/' < /mnt/Lavoro/halium/out/target/product/j3xlte/obj/SHARED_LIBRARIES/libengbt_intermediates/bt_cmd_executer.d >> /mnt/Lavoro/halium/out/target/product/j3xlte/obj/SHARED_LIBRARIES/libengbt_intermediates/bt_cmd_executer.P; rm -f /mnt/Lavoro/halium/out/target/product/j3xlte/obj/SHARED_LIBRARIES/libengbt_intermediates/bt_cmd_executer.d )"
vendor/sprd/open-source/apps/engmode/bt/bt_cmd_executer.c:31:10: fatal error: 'btm_api.h' file not found
#include "btm_api.h"
         ^
1 error generated.

I search in the internet revealed that this is part of Bluedroid, which I assume is the Android bluetooth stack; so I looked in the Halium manifest directory for a repo line:

$ git grep blue
motorola_potter.xml:    <project path="external/bluetooth/bluedroid" name="android_external_bluetooth_bluedroid" remote="los" revision="cm-12.1" />

This one seemed promising, so I added it to my manifest. I ran the setup script again, and had a look at the newly downloaded external/bluetooth/bluedroid/stack/include/ directory: that not only included the btm_api.h file, but also the bt_types.h which we were needing before. So I went back and tried removing the two include lines related to the nxp-nfc projects and replaced them with this one from bluedroid. I got the same issue about the missing data_types.h file, so I added back the vendor/nxp-nfc/opensource/libnfc-nci/p61-jcop-kit/inc line. I then got an error about a missing tb_target.h file, which is found in external/bluetooth/bluedroid/include/, so I added this line as well. This let me build libengbt successfully. The systemimage target instead failed later:

[ 12% 502/3912] target thumb C++: audio.primary.sc8830 <= vendor/sprd/modules/audio/normal/audiotest/auto_audio_v2.cpp
FAILED: /bin/bash -c "(PWD=/proc/self/cwd  prebuilts/clang/host/linux-x86/clang-2690385/bin/clang++ -I device/samsung/sharkls-common/include -I external/tinyalsa/include -I external/expat/lib -I system/media/audio_utils/include -I system/media/audio_effects/include -I vendor/sprd/open-source/apps/engmode -I vendor/sprd/modules/audio/normal/vb_pga -I vendor/sprd/modules/audio/normal/record_process -I vendor/sprd/modules/audio/normal/include -I vendor/sprd/modules/audio/normal/DumpData -I vendor/sprd/modules/audio/normal/audiotest -I vendor/sprd/modules/audio/normal/record_nr -I vendor/sprd/modules/audio/normal/skd -I vendor/sprd/modules/audio/normal/resample_api -I vendor/sprd/modules/audio/normal/custom_mmi -I vendor/sprd/modules/resampler -I vendor/sprd/modules/audio/normal/libaudioril -I vendor/sprd/open-source/libs/libatci -I vendor/sprd/modules/audio/normal/vb_effect/v2 -I vendor/sprd/modules/audio/normal -I /mnt/Lavoro/halium/out/target/product/j3xlte/obj/SHARED_LIBRARIES/audio.primary.sc8830_intermediates -I /mnt/Lavoro/halium/out/target/product/j3xlte/gen/SHARED_LIBRARIES/audio.primary.sc8830_intermediates -I libnativehelper/include/nativehelper \$(cat /mnt/Lavoro/halium/out/target/product/j3xlte/obj/SHARED_LIBRARIES/audio.primary.sc8830_intermediates/import_includes) -isystem system/core/include -isystem system/media/audio/include -isystem hardware/libhardware/include -isystem hardware/libhardware_legacy/include -isystem libnativehelper/include -isystem frameworks/native/include -isystem frameworks/native/opengl/include -isystem frameworks/av/include -isystem frameworks/base/include -isystem hardware/ril/include -isystem /mnt/Lavoro/halium/out/target/product/j3xlte/obj/include -isystem bionic/libc/arch-arm/include -isystem bionic/libc/include -isystem bionic/libc/kernel/uapi -isystem bionic/libc/kernel/common -isystem bionic/libc/kernel/uapi/asm-arm -isystem bionic/libm/include -isystem bionic/libm/include/arm -c    -fno-exceptions -Wno-multichar -msoft-float -ffunction-sections -fdata-sections -funwind-tables -fstack-protector-strong -Wa,--noexecstack -Werror=format-security -D_FORTIFY_SOURCE=2 -fno-short-enums -no-canonical-prefixes -mcpu=cortex-a7 -mfpu=neon-vfpv4 -D__ARM_FEATURE_LPAE=1 -mfloat-abi=softfp -DANDROID -fmessage-length=0 -W -Wall -Wno-unused -Winit-self -Wpointer-arith -Werror=return-type -Werror=non-virtual-dtor -Werror=address -Werror=sequence-point -Werror=date-time -DNDEBUG -g -Wstrict-aliasing=2 -DNDEBUG -UDEBUG  -D__compiler_offsetof=__builtin_offsetof -Werror=int-conversion -Wno-reserved-id-macro -Wno-format-pedantic -Wno-unused-command-line-argument -fcolor-diagnostics -nostdlibinc  -target arm-linux-androideabi    -target arm-linux-androideabi -Bprebuilts/gcc/linux-x86/arm/arm-linux-androideabi-4.9/arm-linux-androideabi/bin    -fvisibility-inlines-hidden -Wsign-promo  -Wno-inconsistent-missing-override -nostdlibinc  -target arm-linux-androideabi   -mthumb -Os -fomit-frame-pointer -fno-strict-aliasing  -fno-rtti -D_POSIX_SOURCE -Wno-multichar -g -DAUDIO_SPIPE_TD -D_LPA_IRAM -DVOIP_DSP_PROCESS -DAUDIO_HAL_ANDROID_N_API -DFM_VERSION_IS_GOOGLE -fPIC -D_USING_LIBCXX -std=gnu++14  -Werror=int-to-pointer-cast -Werror=pointer-to-int-cast  -Werror=address-of-temporary -Werror=null-dereference -Werror=return-type    -MD -MF /mnt/Lavoro/halium/out/target/product/j3xlte/obj/SHARED_LIBRARIES/audio.primary.sc8830_intermediates/audiotest/auto_audio_v2.d -o /mnt/Lavoro/halium/out/target/product/j3xlte/obj/SHARED_LIBRARIES/audio.primary.sc8830_intermediates/audiotest/auto_audio_v2.o vendor/sprd/modules/audio/normal/audiotest/auto_audio_v2.cpp ) && (cp /mnt/Lavoro/halium/out/target/product/j3xlte/obj/SHARED_LIBRARIES/audio.primary.sc8830_intermediates/audiotest/auto_audio_v2.d /mnt/Lavoro/halium/out/target/product/j3xlte/obj/SHARED_LIBRARIES/audio.primary.sc8830_intermediates/audiotest/auto_audio_v2.P; sed -e 's/#.*//' -e 's/^[^:]*: *//' -e 's/ *\\\\\$//' -e '/^\$/ d' -e 's/\$/ :/' < /mnt/Lavoro/halium/out/target/product/j3xlte/obj/SHARED_LIBRARIES/audio.primary.sc8830_intermediates/audiotest/auto_audio_v2.d >> /mnt/Lavoro/halium/out/target/product/j3xlte/obj/SHARED_LIBRARIES/audio.primary.sc8830_intermediates/audiotest/auto_audio_v2.P; rm -f /mnt/Lavoro/halium/out/target/product/j3xlte/obj/SHARED_LIBRARIES/audio.primary.sc8830_intermediates/audiotest/auto_audio_v2.d )"
vendor/sprd/modules/audio/normal/audiotest/auto_audio_v2.cpp:21:9: warning: 'LOG_TAG' macro redefined [-Wmacro-redefined]
#define LOG_TAG "SPRD_AUDIOTRACK"
        ^
system/core/include/log/log.h:65:9: note: previous definition is here
#define LOG_TAG NULL
        ^
vendor/sprd/modules/audio/normal/audiotest/auto_audio_v2.cpp:186:39: warning: unused parameter 'user' [-Wunused-parameter]
void playerCallback( int event, void* user, void *info )
                                      ^
vendor/sprd/modules/audio/normal/audiotest/auto_audio_v2.cpp:245:9: error: use of undeclared identifier 'AUDIO_DEVICE_OUT_FM_HEADSET'; did you mean 'AUDIO_DEVICE_OUT_WIRED_HEADSET'?
        AUDIO_DEVICE_OUT_FM_HEADSET,
        ^~~~~~~~~~~~~~~~~~~~~~~~~~~
        AUDIO_DEVICE_OUT_WIRED_HEADSET
system/media/audio/include/system/audio.h:711:5: note: 'AUDIO_DEVICE_OUT_WIRED_HEADSET' declared here
    AUDIO_DEVICE_OUT_WIRED_HEADSET             = 0x4,
    ^

This is an issue reported here, The link mentioned there is for CyanogenMod 13, but I found one for LineageOS 14.1 here. So I ran

cd device/samsung/sharkls-common/patches/
./apply_sprd-diff.sh

This failed on a couple of patches, for which I reported an issue here, though it's likely not important for us (as it mainly regards Java code). The build proceeded:

[ 10% 436/4011] target thumb C++: libstagefright <= frameworks/av/media/libstagefright/CameraSource.cpp
FAILED: /bin/bash -c "(PWD=/proc/self/cwd  prebuilts/clang/host/linux-x86/clang-2690385/bin/clang++ -I device/samsung/sharkls-common/include -I ./frameworks/av/include/media/ -I ./frameworks/av/media/libavextensions -I ./frameworks/av/media/libstagefright/mpeg2ts -I ./frameworks/av/include/media/stagefright/timedtext -I ./frameworks/native/include/media/hardware -I ./frameworks/native/include/media/openmax -I ./external/flac/include -I ./external/tremolo -I ./external/libvpx/libwebm -I ./system/netd/include -I system/media/audio_utils/include -I hardware/qcom/media/sc8830/mm-core/inc -I ./external/stagefright-plugins/include -I frameworks/av/media/libstagefright -I /mnt/Lavoro/halium/out/target/product/j3xlte/obj/SHARED_LIBRARIES/libstagefright_intermediates -I /mnt/Lavoro/halium/out/target/product/j3xlte/gen/SHARED_LIBRARIES/libstagefright_intermediates -I libnativehelper/include/nativehelper \$(cat /mnt/Lavoro/halium/out/target/product/j3xlte/obj/SHARED_LIBRARIES/libstagefright_intermediates/import_includes) -isystem system/core/include -isystem system/media/audio/include -isystem hardware/libhardware/include -isystem hardware/libhardware_legacy/include -isystem libnativehelper/include -isystem frameworks/native/include -isystem frameworks/native/opengl/include -isystem frameworks/av/include -isystem frameworks/base/include -isystem hardware/ril/include -isystem /mnt/Lavoro/halium/out/target/product/j3xlte/obj/include -isystem bionic/libc/arch-arm/include -isystem bionic/libc/include -isystem bionic/libc/kernel/uapi -isystem bionic/libc/kernel/common -isystem bionic/libc/kernel/uapi/asm-arm -isystem bionic/libm/include -isystem bionic/libm/include/arm -c    -fno-exceptions -Wno-multichar -msoft-float -ffunction-sections -fdata-sections -funwind-tables -fstack-protector-strong -Wa,--noexecstack -Werror=format-security -D_FORTIFY_SOURCE=2 -fno-short-enums -no-canonical-prefixes -mcpu=cortex-a7 -mfpu=neon-vfpv4 -D__ARM_FEATURE_LPAE=1 -mfloat-abi=softfp -DANDROID -fmessage-length=0 -W -Wall -Wno-unused -Winit-self -Wpointer-arith -Werror=return-type -Werror=non-virtual-dtor -Werror=address -Werror=sequence-point -Werror=date-time -DNDEBUG -g -Wstrict-aliasing=2 -DNDEBUG -UDEBUG  -D__compiler_offsetof=__builtin_offsetof -Werror=int-conversion -Wno-reserved-id-macro -Wno-format-pedantic -Wno-unused-command-line-argument -fcolor-diagnostics -nostdlibinc  -target arm-linux-androideabi    -target arm-linux-androideabi -Bprebuilts/gcc/linux-x86/arm/arm-linux-androideabi-4.9/arm-linux-androideabi/bin    -fvisibility-inlines-hidden -Wsign-promo  -Wno-inconsistent-missing-override -nostdlibinc  -target arm-linux-androideabi   -mthumb -Os -fomit-frame-pointer -fno-strict-aliasing  -fno-rtti -Wno-multichar -Werror -Wno-error=deprecated-declarations -Wall -DENABLE_STAGEFRIGHT_EXPERIMENTS -DUSE_SPRD_COLORFORMAT -fPIC -D_USING_LIBCXX -fsanitize=unsigned-integer-overflow,signed-integer-overflow -fsanitize-trap=all -ftrap-function=abort -std=gnu++14  -Werror=int-to-pointer-cast -Werror=pointer-to-int-cast  -Werror=address-of-temporary -Werror=null-dereference -Werror=return-type    -MD -MF /mnt/Lavoro/halium/out/target/product/j3xlte/obj/SHARED_LIBRARIES/libstagefright_intermediates/CameraSource.d -o /mnt/Lavoro/halium/out/target/product/j3xlte/obj/SHARED_LIBRARIES/libstagefright_intermediates/CameraSource.o frameworks/av/media/libstagefright/CameraSource.cpp ) && (cp /mnt/Lavoro/halium/out/target/product/j3xlte/obj/SHARED_LIBRARIES/libstagefright_intermediates/CameraSource.d /mnt/Lavoro/halium/out/target/product/j3xlte/obj/SHARED_LIBRARIES/libstagefright_intermediates/CameraSource.P; sed -e 's/#.*//' -e 's/^[^:]*: *//' -e 's/ *\\\\\$//' -e '/^\$/ d' -e 's/\$/ :/' < /mnt/Lavoro/halium/out/target/product/j3xlte/obj/SHARED_LIBRARIES/libstagefright_intermediates/CameraSource.d >> /mnt/Lavoro/halium/out/target/product/j3xlte/obj/SHARED_LIBRARIES/libstagefright_intermediates/CameraSource.P; rm -f /mnt/Lavoro/halium/out/target/product/j3xlte/obj/SHARED_LIBRARIES/libstagefright_intermediates/CameraSource.d )"
frameworks/av/media/libstagefright/CameraSource.cpp:134:16: error: use of undeclared identifier 'OMX_SPRD_COLOR_FormatYVU420SemiPlanar'; did you mean 'OMX_QCOM_COLOR_FormatYVU420SemiPlanar'?
        return OMX_SPRD_COLOR_FormatYVU420SemiPlanar;
               ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
               OMX_QCOM_COLOR_FormatYVU420SemiPlanar
./frameworks/native/include/media/openmax/OMX_IVCommon.h:169:5: note: 'OMX_QCOM_COLOR_FormatYVU420SemiPlanar' declared here
    OMX_QCOM_COLOR_FormatYVU420SemiPlanar = 0x7FA30C00,
    ^
1 error generated.

I realized that there were more patches to be applied, and that the script stopped as soon as it stumbled into a failing patch. So, I took all the failing patches out of the way:

mv sprd-diff/frameworks_base.diff{,.skip}
mv sprd-diff/frameworks_opt.diff{,.skip}
mv sprd-diff/packages_apps.diff{,.skip}
mv sprd-diff/packages_services.diff{,.skip}
mv sprd-diff/system_bt.diff{,.skip} # This looks potentially worth a revisit: should we have a system/bt dir?

The next build failed here:

[ 81% 3506/4283] target thumb C: libbt-vendor <= vendor/sprd/wcn/bt/libbt/src/bt_vendor_sprd.c
FAILED: /bin/bash -c "(PWD=/proc/self/cwd  prebuilts/clang/host/linux-x86/clang-2690385/bin/clang -I device/samsung/sharkls-common/include -I vendor/sprd/wcn/bt/libbt/include -I system/bt/hci/include -I vendor/sprd/wcn/bt/libbt/conf/sprd/marlin/include -I vendor/sprd/wcn/bt/libbt -I /mnt/Lavoro/halium/out/target/product/j3xlte/obj/SHARED_LIBRARIES/libbt-vendor_intermediates -I /mnt/Lavoro/halium/out/target/product/j3xlte/gen/SHARED_LIBRARIES/libbt-vendor_intermediates -I libnativehelper/include/nativehelper \$(cat /mnt/Lavoro/halium/out/target/product/j3xlte/obj/SHARED_LIBRARIES/libbt-vendor_intermediates/import_includes) -isystem system/core/include -isystem system/media/audio/include -isystem hardware/libhardware/include -isystem hardware/libhardware_legacy/include -isystem libnativehelper/include -isystem frameworks/native/include -isystem frameworks/native/opengl/include -isystem frameworks/av/include -isystem frameworks/base/include -isystem hardware/ril/include -isystem /mnt/Lavoro/halium/out/target/product/j3xlte/obj/include -isystem bionic/libc/arch-arm/include -isystem bionic/libc/include -isystem bionic/libc/kernel/uapi -isystem bionic/libc/kernel/common -isystem bionic/libc/kernel/uapi/asm-arm -isystem bionic/libm/include -isystem bionic/libm/include/arm -c    -fno-exceptions -Wno-multichar -msoft-float -ffunction-sections -fdata-sections -funwind-tables -fstack-protector-strong -Wa,--noexecstack -Werror=format-security -D_FORTIFY_SOURCE=2 -fno-short-enums -no-canonical-prefixes -mcpu=cortex-a7 -mfpu=neon-vfpv4 -D__ARM_FEATURE_LPAE=1 -mfloat-abi=softfp -DANDROID -fmessage-length=0 -W -Wall -Wno-unused -Winit-self -Wpointer-arith -Werror=return-type -Werror=non-virtual-dtor -Werror=address -Werror=sequence-point -Werror=date-time -DNDEBUG -g -Wstrict-aliasing=2 -DNDEBUG -UDEBUG  -D__compiler_offsetof=__builtin_offsetof -Werror=int-conversion -Wno-reserved-id-macro -Wno-format-pedantic -Wno-unused-command-line-argument -fcolor-diagnostics -nostdlibinc  -target arm-linux-androideabi    -target arm-linux-androideabi -Bprebuilts/gcc/linux-x86/arm/arm-linux-androideabi-4.9/arm-linux-androideabi/bin    -std=gnu99     -mthumb -Os -fomit-frame-pointer -fno-strict-aliasing   -fPIC -D_USING_LIBCXX   -Werror=int-to-pointer-cast -Werror=pointer-to-int-cast  -Werror=address-of-temporary -Werror=null-dereference -Werror=return-type  -MD -MF /mnt/Lavoro/halium/out/target/product/j3xlte/obj/SHARED_LIBRARIES/libbt-vendor_intermediates/src/bt_vendor_sprd.d -o /mnt/Lavoro/halium/out/target/product/j3xlte/obj/SHARED_LIBRARIES/libbt-vendor_intermediates/src/bt_vendor_sprd.o vendor/sprd/wcn/bt/libbt/src/bt_vendor_sprd.c ) && (cp /mnt/Lavoro/halium/out/target/product/j3xlte/obj/SHARED_LIBRARIES/libbt-vendor_intermediates/src/bt_vendor_sprd.d /mnt/Lavoro/halium/out/target/product/j3xlte/obj/SHARED_LIBRARIES/libbt-vendor_intermediates/src/bt_vendor_sprd.P; sed -e 's/#.*//' -e 's/^[^:]*: *//' -e 's/ *\\\\\$//' -e '/^\$/ d' -e 's/\$/ :/' < /mnt/Lavoro/halium/out/target/product/j3xlte/obj/SHARED_LIBRARIES/libbt-vendor_intermediates/src/bt_vendor_sprd.d >> /mnt/Lavoro/halium/out/target/product/j3xlte/obj/SHARED_LIBRARIES/libbt-vendor_intermediates/src/bt_vendor_sprd.P; rm -f /mnt/Lavoro/halium/out/target/product/j3xlte/obj/SHARED_LIBRARIES/libbt-vendor_intermediates/src/bt_vendor_sprd.d )"
In file included from vendor/sprd/wcn/bt/libbt/src/bt_vendor_sprd.c:31:
vendor/sprd/wcn/bt/libbt/include/bt_vendor_sprd.h:33:10: fatal error: 'bt_vendor_lib.h' file not found
#include "bt_vendor_lib.h"
         ^
1 error generated.

I had this file in external/bluetooth/bluedroid/hci/include/bt_vendor_lib.h, and I was about to add this directory to the vendor/sprd/wcn/bt/libbt/Android.mk file when I realized that this file was already including $(BDROID_DIR)/hci/include and defined BDROID_DIR := $(TOP_DIR)system/bt. So I understood that I should have changed my manifest file to check out the bluedroid repo under system/bt (which also meant that I was wrong in excluding the sprd-diff/system_bt.diff patch before). I also had a quick look at other manifest files, and I saw that they were typically getting system/bt from the LineageOS android_system_bt repo. Therefore I went to modify my manifest file, changing the bluedroid line to

<project path="system/bt" name="android_system_bt" remote="los" revision="cm-14.1" />

and removed the three vendor/nxp-nfc repositories I added a while before. Then I manually applied the patch in device/samsung/sharkls-common/patches/sprd-diff/system_bt.diff and I revisited the vendor/sprd/open-source/apps/engmode/bt/Android.mk to remove all the include lines I added before. The next build succeeded:

[...]
[ 99% 2250/2255] build /mnt/Lavoro/halium/out/target/product/j3xlte/obj/NOTICE.html
Combining NOTICE files into HTML
Combining NOTICE files into text
[ 99% 2254/2255] Target system fs image: /mnt/Lavoro/halium/out/target/product/j3xlte/obj/PACKAGING/systemimage_intermediates/system.img
BuildImage: in_dir = /mnt/Lavoro/halium/out/target/product/j3xlte/system, out_file = /mnt/Lavoro/halium/out/target/product/j3xlte/obj/PACKAGING/systemimage_intermediates/system.img
fs type is not ext4
Running:  mkuserimg.sh -s /mnt/Lavoro/halium/out/target/product/j3xlte/system /mnt/Lavoro/halium/out/target/product/j3xlte/obj/PACKAGING/systemimage_intermediates/system.img ext4 system 2147483648 -D /mnt/Lavoro/halium/out/target/product/j3xlte/system -L system /mnt/Lavoro/halium/out/target/product/j3xlte/root/file_contexts.bin
make_ext4fs -s -T -1 -S /mnt/Lavoro/halium/out/target/product/j3xlte/root/file_contexts.bin -L system -l 2147483648 -a system /mnt/Lavoro/halium/out/target/product/j3xlte/obj/PACKAGING/systemimage_intermediates/system.img /mnt/Lavoro/halium/out/target/product/j3xlte/system /mnt/Lavoro/halium/out/target/product/j3xlte/system
Creating filesystem with parameters:
    Size: 2147483648
    Block size: 4096
    Blocks per group: 32768
    Inodes per group: 8192
    Inode size: 256
    Journal blocks: 8192
    Label: system
    Blocks: 524288
    Block groups: 16
    Reserved block group size: 127
Created filesystem with 1381/131072 inodes and 55467/524288 blocks
Running ['mkuserimg.sh', '-s', '/mnt/Lavoro/halium/out/target/product/j3xlte/system', '/mnt/Lavoro/halium/out/target/product/j3xlte/obj/PACKAGING/systemimage_intermediates/system.img', 'ext4', 'system', '2147483648', '-D', '/mnt/Lavoro/halium/out/target/product/j3xlte/system', '-L', 'system', '/mnt/Lavoro/halium/out/target/product/j3xlte/root/file_contexts.bin'] command, exit code = 0
[ 99% 2254/2255] Construct recovery from boot
failed to reconstruct target deflate chunk 1 [(null)]; treating as normal
chunk 0: type 0 start 0 len 5582858
chunk 1: type 2 start 5582858 len 8611584
chunk 2: type 0 start 9205682 len 673886
Construct patches for 3 chunks...
patch   0 is 212 bytes (of 5582858)
patch   1 is 4311951 bytes (of 3622824)
patch   2 is 207 bytes (of 673886)
chunk   0: normal   (         0,    5582858)         212
chunk   1: deflate  (   5582858,    8574717)     4311951  (null)
chunk   2: normal   (  14157575,     674057)         207
[100% 2255/2255] Install system fs image: /mnt/Lavoro/halium/out/target/product/j3xlte/system.img
/mnt/Lavoro/halium/out/target/product/j3xlte/system.img+/mnt/Lavoro/halium/out/target/product/j3xlte/obj/PACKAGING/recovery_patch_intermediates/recovery_from_boot.p maxsize=2192424960 blocksize=135168 total=197470210 reserve=22167552
make: Leaving directory '/mnt/Lavoro/halium'

#### make completed successfully (01:22 (mm:ss)) ####

Just to be on the safe side (to make sure I didn't have stale files from previous build attempts), I ran

mka clean
mka halium-boot
LANG=C mka systemimage

again, without encountering any errors. I also ran mka hybris-boot as per the Halium documentation, as it might be a good idea to test the Halium reference image first. hybris-boot.img was built successfully at the first try.

Flashing and testing the Halium reference image

Following the Halium documentation, I downloaded the Halium rootfs image:

mkdir rootfs
cd rootfs
wget 'http://bshah.in/halium/halium-rootfs-20170630-151006.tar.gz'

A small section of the documentation is dedicated on how to install the boot image for Samsung devices; it closely matches the steps I took before to install TWRP, which is comforting.

adb reboot download    # much easier than the weird key combinations!
sudo heimdall flash --BOOT out/target/product/j3xlte/hybris-boot.img

The last command ended with this output:

Uploading BOOT
100%
BOOT upload successful

Ending session...
ERROR: Failed to receive session end confirmation!
Releasing device interface...

and the device was stuck on the same screen. I then thought of using TWRP to flash this device. I rebooted in download mode and installed TWRP as explained near the beginning of this blog post, then I uploaded my boot image to the SD card:

adb push out/target/product/j3xlte/hybris-boot.img /sdcard/

In TWRP, I did a wipe first: I wiped off the "System", "Data" and "cache" partitions (you can choose which partitions to wipe under the "Advanced wipe" option). Then I went back and pressed the "Install" button, then "Install Image" and picked my hybris-boot.img file (and chose "Boot" as target, of course). Make sure you don't reboot at this stage, or you'll lose adb access and the rest of the commands will fail.

I then installed the halium-install script:

git clone https://gitlab.com/JBBgameich/halium-install.git

I then downloaded the original Samsung ROM (see below for the link) and unzipped it in the rootfs directory I created before. That produced a file named J320FXXU0AQK2_J320FOXE0AQK2_J320FXXU0AQI1_HOME.tar.md5, from which I extracted the system.img and uploaded it to the device along with the halium rootfs, as explained in the "Halium guide":

tar xvf J320FXXU0AQK2_J320FOXE0AQK2_J320FXXU0AQI1_HOME.tar.md5 system.img
../halium-install/halium-install -p halium -v halium-rootfs-20170630-151006.tar.gz system.img

The last command complained about simg2img not being in $PATH, despite the fact that it was. A small change to the script fixed this. Once that was fixed it successfully pushed the files to the device, after asking me which password I wanted to set for the root user (on the phone): I chose a familiar "0000".

After rebooting, however, the device didn't go beyond the boot screen (with a red line warning about SELinux not being enabled). It just got stuck there, and was not even detected via USB.

Therefore, it's time to end this post with a "To be continued" message, and with a bonus section on how to restore the stock image below.

Restoring the Samsung stock ROM

The stock ROM can be downloaded from sammobile.com; you can restore the original image with this commands:

unzip J320*.zip
tar xvf J320*.tar.md5
sudo heimdall flash --KERNEL boot.img --CACHE cache.img \
    --HIDDEN hidden.img --l_fixnv2 nvitem.bin --PARAM param.lfs \
    --pm_sys PM_sharkl_arm7.bin --RECOVERY recovery.img \
    --SBOOT sboot.bin --SBOOT2 sboot2.bin --BOOT spl.img \
    --l_modem SPRDCP.img --l_gdsp SPRDGDSP.img --l_ldsp SPRDLTEDSP.img \
    --l_warm SPRDWDSP.img --SYSTEM system.img

If you were so crazy as to follow my steps, now running the command above is probably the wisest thing you could do. :-)

July 03, 2019

The coding period has now extended over a month and quite a few improvements have been merged into KIOFuse. In my last post I mentioned the development of a KIO error to FUSE error mapping and 32 bit support.

However, interestingly enough it took quite a long time for the 32-bit support branch to be merged. This was because of a test that didn’t fail nor pass – it froze. The test suite would never finish and the process would only respond to SIGKILL. After days of debugging it was determined that fuse_notify_inval_* functions don’t play well when writeback caching is enabled and hence there is now a patch to disable it. Of course this will incur a performance hit as writes will go straight to KIOFuse, and hence straight to disk (although the kernel may cache our write requests to our own cache). Whilst this is unfortunate, seeming as most KIO slaves are network based, switching from a writeback caching policy to a writethrough one is unlikely to hamper performance too much.

In other news, KIOFuse can now handle SIGTERM, SIGINT and SIGHUP signals. Signal handlers can only call async-signal-safe functions. However in Qt there is a bit of hack one can perform, as inspired by this tutorial. Hence, in response to these signals, KIOFuse will flush all dirty nodes to disk, meaning no sudden data loss!

Mounts can now have their password changed.

The lookup function has now been optimised. Previously a lookup would call KIO::listDir, which was totally unnecessary – a KIO::stat would suffice and this is what the patch has switched to. It also increased the data buffer from 1MB to 14MB.

Unmount support has currently been postponed. It is proving problematic to implement reliably and unmounting only really provides a marginal benefit, so it is yet to be seen if we will implement it at all. The current WIP patch can be found here.

It has been decided that slaves that do not implement KIO::stat will not be supported. It’s a bit of a hassle to implement with extremely marginal benefit. There are only a few slaves that don’t implement KIO::stat, such as fonts and thumbnails.

An issue with KDE Connect not working properly has been fixed upstream. I haven’t 100% confirmed which patch has fixed this for us, but I’m placing my bets on this one.

Currently, the Google Drive API reports a size of zero for files that are not supported by GDrive, such as odt files , and their proprietary formats – i.e. Google Docs. Whilst we can update the size quite easily by downloading the file, the file turns out corrupted, and is only openable if the program has a repair option (such as LibreOffice). Unfortunately, I’ve not been able to find out why exactly this is happening, and have not come up with a fix. Currently this is being shelved and I hope to revisit it at a later date with a fresher mind.

We’d welcome anyone to use and test KIOFuse. Feel free to notify us on any bugs or performance issues by opening an issue and you can even contribute a patch if you wish!

When I started with Ngrx the Entity module didn't exist. My state consisted of arrays of objects. The reducers and selectors were array manipulations. It worked well but if the state had a large number of objects, the filter and maps were expensive and required lots of code.
The Entity library made it much simpler. My reducers are much less code and dramatically simple in comparison. The selectors are about half the length. It all works quickly, is easy to set up.

Essentially the data is stored as an entity object. { [id: string|number]: dataobj} There is a list of id's, which can be sorted. You access a specific object using the id as a property. entity[id]. If you have a list of id's, idlist.map(i => entity[i] will give you a list of objects. The Entity can sort the ids, extract whatever key you want from your data. But what if you have a relational data structure?

For example, you have a contact list.
interface Contact {
  id: string;
  name: string;
  note: string;
}

You also have an email address list. Each contact may have one or many email addresses.
interface Email {
  id: string;
  email: string;
  note: string;
  contactid: string;
}

For each contact, the email addresses would have the contact id to relate them.

A selector on the email Entity would have to do something like this to get the related emails for the selected contact.

export const getSelectedEmails = createSelector(
   getEmailEntities,
   getEmailIds,
   getSelectedContact,
   ( entities, ids, contact) => {
      return ids.filter(i => entities[i].contactid === contact.id? true: false)
})

Right back to filtering long arrays to extract a few matching items.

That is a pretty simple situation. In real life it gets complicated quickly. Maybe you have an attribute property on the email, where the email address is tagged various ways; receive only, business, personal, project specific, etc. You might want to filter on those attributes as well. You might want to do a many to many relation where the same email address belongs to many people.

This is what I needed. So I came up with another property of the Entity State.
interface Keys {
   [key: string]: { [id: string| number]: ids[] }
}

This is how the keys are extracted. It looks similar to the stock Entity

export const adapter: EntityRelationAdapter<InventoryData> = createEntityRelationAdapter<InventoryData>({
  selectId: Inventorydata => Inventorydata.tid,  selectKey: {
    item: InventoryData => InventoryData.itemtid,    vendoritem: InventoryData => InventoryData.vendorinvoice,    resourcetype: InventoryData => InventoryData.resourcetype,    }
});
 selectKey has functions that extract keys, which creates a list of keys. So in your selector you would do something like keys['item'][itemid] to get a list of ids that are related to the item data.

As with the Entity the array of ids is created as each item is added to the entity state. I haven't implemented a sort function, mostly because I don't need it, and typically the related lists are short and easy to sort in a selector.

Take a look. It is useful to me.

https://github.com/derekkite/relation-entity

The Fedora package smc-fonts has a set of Malayalam fonts (AnjaliOldLipi, Kalyani, Meera, Rachana, RaghuMalayalamSans and Suruma) maintained by SMC. We used to package all these fonts as a single zip file hosted at https://savannah.nongnu.org/projects/smc. These fonts were last updated in 2014 for Fedora, leaving them at version 6.1.

Since then, a lot of improvements were made to these fonts — glyph additions/corrections, opentype layout changes, fontTools based build system and separate source repository for each font etc.. There were lengthy discussions on the release management of the fonts, and it was partially the reason fonts were not updated in Fedora. Once it was agreed to follow different version number for each font, and a continuous build+release system was put in place at Gitlab, we could ensure that fonts downloaded from SMC website were always the latest version.

To reflect the updates in Fedora, we had to decide how to handle the monolithic source package at version 6.1 versus the new individual releases (e.g. Rachana is at version 7.0.1 as of this writing). In a discussion with Pravin Satpute, we agreed to obsolete the existing fonts package and give each font its own package.

Vishal Vijayaraghavan kindly stepped up and did the heavy lifting of creating the new packages, and we now even build the ttf font file from the source. See RHBZ#1648825 for details.

With all that in place, in Fedora 30, all these fonts are in latest version — for instance, see Rachana package. The old package smc-fonts no longer exists, instead each individual package such as smc-rachana-fonts or smc-meera-fonts can be installed. Our users will now be able to enjoy the improvements made over the years — including updated Unicode coverage, new glyphs, improved existing glyphs, much better opentype shaping etc.

July 02, 2019

Finally, I can write myself as a professional developer, cause for the first time I got paid, that also by cash, for writing code. So after the boost battle of last month, it was time to test the algorithm out in the wild and not limit it to the confined boundary of the tests.

July 01, 2019

I [partially, only 2 days out of the 7] attended the Usability & Productivity Sprint 2019 in Valencia two weekends ago.

I was very happy to meet quite some new developer blood, which is something we had been struggling a bit to get lately, so we're starting to get on the right track again :) And I can only imagine it'll get better and better due to the "Onboarding" goal :)

During the sprint we had an interesting discussion about how to get more people to know about usability, and the outcome is that probably we'll try to get some training to members of KDE to increase the knowledge of usability amongst us. Sounds like a good idea to me :)

On the more "what did *you* actually do" side:
* worked on fixing a crash i had on the touchpad kded, (already available on the latest released Plasma!)
* finished part of the implementation for Optional Content Group Links support in Okular (i started that 3 years ago and i was almost sure i had done all the work, but i clearly had not)
* Did some code reviews on existing okular phabricator merge requests (so sad i'm still behind though, we need more people reviewing there other than me)
* Together with Nicolas Fella worked on allowing extra fields from json files to be translated, we even documented it!
* Changed lots of applications released on KDE Applications to follow the KDE Applications versioning scheme, the "winner" was kmag, that had been stuck in version 1.0 for 15 years (and had more than 440 commits since then)
* Fixed a small issue with i18n in kanagram

I would like to thank SLIMBOOK for hosting us in their offices (and providing a shuttle from them to the hotel) and the KDE e.V. for sponsoring my attendance to the sprint, please donate to KDE if you think the work done at sprints is important.

Well, it’s been a long six-to-eight months. But [bobulate] is back, and some of the pent-up blogging needs are ready to be unleashed on Planet KDE and wherever else.

Late last year there were some hiccups with my hosting provider, which led to SSL issues. Those were solved, and I carried on with the existing hosting. Then in february or so the MySQL server at the hosting provider went down, and I filed some tickets, grumbled a bit, and figured it would resolve itself. After all, this blog isn’t a staggeringly important piece of internet infrastructure, and I could let off notifications through the Calamares announcements for my major work, and through Twitter for KDE packaging on FreeBSD.

And february dragged on into april, may, with no resolution of the issues in sight, and then a letter arrived from the Dutch internet authority saying that my hoster was no longer an official registry and that my domains were now floating around.

That’s when some form of panic struck – although in the end I only lost one of them to a domain-hijacker in Hong Kong. I’ve switched hosting to another Free-Software-friendly place, switched out Wordpress for the much easier-to-manage Jekyll, and will be re-building the archives as I go along.

Free Software Stuff

To justify having this on the Planet, let’s talk a little about Free Software.

  • Calamares, a Linux system installer used by a bunch of boutique (specialized) distro’s, continues chugging along. I decided this spring to try to stick to a two-week release cycle, in order to get things out the door – any things – rather than grind though a set milestone. That’s worked pretty well, getting small and incremental improvements out the door much faster – although the big stuff still takes a long time to work out. I’ll be sticking to this schedule after summer vacations.
  • KDE-FreeBSD continues to hum along, Tobias is doing most of the work while I’m busy, but together we manage to keep the packaing up-to-date with all the latest releases.
  • I wrote a little meeting-management bot for Matrix, based on what I remember from the IRC channel #koffie from EFnet long, long ago.

Summer vacation’s been mentioned. After those vacations, it’s september, and so

I'm going to Akademy 2019

Elisa is a music player developed by the KDE community that strives to be simple and nice to use. We also recognize that we need a flexible product to account for the different workflows and use-cases of our users.

We focus on a very good integration with the Plasma desktop of the KDE community without compromising the support for other platforms (other Linux desktop environments, Windows and Android).

We are creating a reliable product that is a joy to use and respects our users privacy. As such, we will prefer to support online services where users are in control of their data.

I am happy to announce the release of 0.4.2 version of the Elisa music player.

The following fixes have been added to this release:

  • Fix restore of tracks with missing metadata in playlist (this was the case for tracks without album metadata) by Matthieu Gallien ;
  • Fix view selector not following the color theme (BUG 408435) by Matthieu Gallien.
Elisa with Breeze DarkFixed Elisa with Breeze Dark

Getting Involved

I would like to thank everyone who contributed to the development of Elisa, including code contributions, code reviews, testing, and bug reporting and triaging. Without all of you, I would have stopped working on this project.

New features and fixes are already being worked on. If you enjoy using Elisa, please consider becoming a contributor yourself. We are happy to get any kind of contributions!

We have some tasks that would be perfect junior jobs. They are a perfect way to start contributing to Elisa. There are more not yet reported here but reported in bugs.kde.org.

The flathub Elisa package allows an easy way to test this new release.

Elisa source code tarball is available here. There is no Windows setup. There is currently a blocking problem with it (no icons) that is being investigated. I hope to be able to provide installers for later bugfix versions.

The phone/tablet port project could easily use some help to build an optimized interface on top of Kirigami. It remains to be seen how to handle this related to the current desktop UI. This is something very important if we want to also support free software on mobile platforms.

Gcompris

Multiple dataset Migration of an Activity

This post is a step by step tutorial for adding multiple datasets to an activity in Gcompris.
The procedure of adding multiple datasets to an activity is fairly simple in Gcompris. The steps for it are given below.
Note: In these steps we'll refer the activity in consideration as current_activity. Also we assume that we plan to add 3 datasets to current_activity.

PROCEDURE

  1. Add the following line to current_activity/ActivityInfo.qml file

    levels: "1,2,3"

    The above line indicates that that the activity will contain 3 datasets and will automatically create the dataset selection menu for the activity with 3 options.
    example

    import GCompris 1.0

    ActivityInfo {
    name: "money/Money.qml"
    difficulty: 2
    icon: "money/money.svg"
    author: "Bruno Coudoin &lt;bruno.coudoin@gcompris.net&gt;"
    demo: false
    //: Activity title
    title: qsTr("Money")
    //: Help title
    description: qsTr("Practice money usage")
    // intro: "Click or tap on the money to pay."
    //: Help goal
    goal: qsTr("You must buy the different items and give the exact price. At higher levels, several items are displayed, and you must first calculate the total price.")
    //: Help prerequisite
    prerequisite: qsTr("Can count")
    //: Help manual
    manual: qsTr("Click or tap on the coins or paper money at the bottom of the screen to pay. If you want to remove a coin or note, click or tap on it on the upper screen area.")
    credit: ""
    section: "math money measures"
    createdInVersion: 0
    levels: "1,2,3"
    }
  2. Create a resource directory inside the current_activity folder and inside it create separate folders for separate datasets with the name of the folder representing dataset number. The resultant directory structure would be as follows.
    +-- current_activity
    | ++-- resource
    | +++-- 1
    | ++++-- Data.qml
    | +++-- 2
    | ++++-- Data.qml
    | +++-- 3
    | ++++-- Data.qml

  3. Create a Data.qml file inside each dataset folder in the following format

    • objective - It will contain the text corresponding to this dataset that would be shown in the dataset selection menu.
    • difficulty - contains the difficulty of the dataset.
    • data - contains the actual data of the dataset The following example demonstrates the layout.
    import QtQuick 2.6
    import GCompris 1.0
    import "../../../../core"

    Dataset {
    objective: qsTr("Set and display time on analog clock for full half and quarters of an hour.")
    difficulty: 2
    data: [
    {
    "numberOfSubLevels": 5,
    "fixedMinutes": 0,
    "displayMinutesHand": false,
    "fixedSeconds": 0,
    "displaySecondsHand": false
    },
    {
    "numberOfSubLevels": 5,
    "fixedMinutes": 15,
    "displayMinutesHand": true,
    "fixedSeconds": 0,
    "displaySecondsHand": false
    },
    {
    "numberOfSubLevels": 5,
    "fixedMinutes": 30,
    "displayMinutesHand": true,
    "fixedSeconds": 0,
    "displaySecondsHand": false
    },
    {
    "numberOfSubLevels": 5,
    "fixedMinutes": 45,
    "displayMinutesHand": true,
    "fixedSeconds": 0,
    "displaySecondsHand": false
    }
    ]
    }
  4. In the current_activity/CurrentActivity.qml file add the following line to get the currenlty selected dataset.

              property var levels: activity.datasetLoader.item.data

    example

    QtObject {
    id: items
    property Item main: activity.main
    property alias background: background
    property GCSfx audioEffects: activity.audioEffects
    property alias answerModel: answerArea.pocketModel
    property alias pocketModel: pocketArea.pocketModel
    property alias store: store
    property alias instructions: instructions
    property alias tux: tux
    property var levels: activity.datasetLoader.item.data
    property alias tuxMoney: tuxMoney
    property alias bar: bar
    property alias bonus: bonus
    property int itemIndex
    property int pocketRows
    property var selectedArea
    property alias pocket: pocketArea.answer
    property alias answer: answerArea.answer
    }

    This way the variable levels will contain the data section of the selected dataset.

  5. The dataset can be extracted from the levels variable inside the js file as follows.

      dataset = items.levels
    var data = dataset[currentLevel]

You know that I'm not an early adopter. That's why it was only a couple of weeks ago when I decided to give Qbs a try, by using the good old Mappero (and its spin-off, Mappero Geotagger) as a test bench. Yes, I know that the Qt company is not going to maintain Qbs anymore in the future, but the little I knew about Qbs was enough to convince me that it's a project worth supporting. So, better late than never -- and hopefully the community (me included) will do a good job in keeping Qbs thriving.

Having Mappero build with Qbs was the simplest thing ever. The only issue I met was in building the unit tests, because I'm used to set the rpath on test executables in order to make it easy to run them uninstalled, and with qmake I achieved that with this:

QMAKE_RPATHDIR = $${QMAKE_LIBDIR}

In turns out that with Qbs you can do it in almost the same way, but for some reason I couldn't figure it out and I even reported a bug to which I got some nice suggestions, before eventually settling on this:

import qbs 1.0

Test {
    name: "path-test"

    files: [
        "path-test.cpp",
        "path-test.h",
        "paths.qrc",
    ]

    Depends { name: "Mappero" }
    cpp.rpaths: cpp.libraryPaths    // <-- this does the trick!
}

It's surprisingly similar to how it's done in qmake, so it's not clear even to me why I didn't guess that immediately. Anyway, that was literally my only problem, and you can see the whole set of Qbs files I wrote by having a look at this commit.

Given how easy the migration was, I thought I should also try to add a code coverage report; that's not something I had in my qmake build either, but it's something I really want to have in all my newer projects.

Teaching Qbs to make a code coverage report

Unfortunately, my search for examples on how to have Qbs prepare a coverage report was mostly insuccessful, but thanks to some amazing help from Christian in the #qbs IRC channel, this was not hard to achieve. So, I hope to be of some help myself too, by sharing how this works.

First of all, it must be said that Qbs doesn't know anything about code coverage, at all. However, it's possible (and often easy) to extend Qbs by adding your own Product with its own set of build rules, so here's the CoverageReport item for Mappero (though, it should be general enough to be reusable in your own project):

import qbs

Product {
    name: "coverage"

    property string outputDirectory: "coverage-html"
    property stringList extractPatterns: []

    builtByDefault: false
    files: ["**"]
    type: ["coverage.html"]

    Depends { productTypes: ["autotest-result"] }

    Rule {
        multiplex: true
        explicitlyDependsOnFromDependencies: ["autotest-result"]
        outputFileTags: "coverage.html"
        requiresInputs: false
        prepare: {
            var commands = []
            var captureCmd = new Command("lcov", [
                "--directory", project.sourceDirectory,
                "--capture",
                "--output-file", "coverage.info",
                "--no-checksum",
                "--compat-libtool",
            ]);
            captureCmd.description = "Collecting coverage data";
            captureCmd.highlight = "coverage";
            captureCmd.silent = false;
            commands.push(captureCmd);

            var extractArgs = []
            for (var i = 0; i < product.extractPatterns.length; i++) {
                extractArgs.push("--extract");
                extractArgs.push("coverage.info");
                extractArgs.push(product.extractPatterns[i]);
            }
            if (product.extractPatterns.length > 0) {
                extractArgs.push("-o");
                extractArgs.push("coverage.info");
                var extractCmd = new Command("lcov", extractArgs);
                extractCmd.description = "Extracting coverage data";
                extractCmd.highlight = "coverage";
                extractCmd.silent = false;
                commands.push(extractCmd);
            }

            var filterCmd = new Command("lcov", [
                "--remove", "coverage.info", 'moc_*.cpp',
                "--remove", "coverage.info", 'qrc_*.cpp',
                "--remove", "coverage.info", '*/tests/*',
                "-o", "coverage.info",
            ]);
            filterCmd.description = "Filtering coverage data";
            filterCmd.highlight = "coverage";
            filterCmd.silent = false;
            commands.push(filterCmd);

            var genhtmlCmd = new Command("genhtml", [
                "--prefix", project.sourceDirectory,
                "--output-directory", product.outputDirectory,
                "--title", "Code coverage",
                "--legend",
                "--show-details",
                "coverage.info",
            ]);
            genhtmlCmd.description = "Generate HTML coverage report";
            genhtmlCmd.highlight = "coverage";
            genhtmlCmd.silent = false;
            commands.push(genhtmlCmd);

            return commands;
        }
    }
}

The most important thing here are the references to the autotest-result tag: this is the tag used by the AutotestRunner Qbs item, which is responsible for running the unit tests. Referencing its product's tag in the Depends item and in the explicitlyDependsOnFromDependencies properties ensures that "building" our product will cause the unit tests to run. Other needed bits are the requiresInputs: false property, which means that our rule doesn't have any required inputs, and the builtByDefault: false property, which says that our coverage report should not be generated when just typing qbs. Instead, to run the tests and get the code coverage report one will have to request it explicitly, by typing

qbs -p coverage

The prepare property of the Rule is where the commands to generate the code coverage report are defined. Here we can use the Command item to invoke external programs, and we return a list of such items, so that the commands will be executed in sequence. Note that here I'm using lcov and expecting to find the coverage data produced by gcov, so this is probably not portable outside of Linux/gcc.

Using the CoverageReport item is quite easy: you just need to declare it, and specify which paths contain the coverage data that you are interested in (otherwise, lcov will collect data from all object files that it find under the build directory, which might not be what you desire):

    CoverageReport {
        condition: project.enableCoverage
        extractPatterns: [ '*/src/*.cpp', '*/lib/*.cpp' ]
    }

There's little more than that to be done. Of course, you need to find a way to pass the --coverage option to gcc when building your products, and for this I created a small buildconfig module in qbs/modules/buildconfig/BuildConfig.qbs which I depend on in all products which I wish to build with coverage enabled:

import qbs

Module {
    cpp.cxxFlags: project.enableCoverage ? ["--coverage"] : undefined
    cpp.dynamicLibraries: project.enableCoverage ? ["gcov"] : undefined

    Depends { name: "cpp" }
}

If all this looks scary, you should probably have a look at the diff which shows how I added code coverage reporting to qbs: hopefully you'll find that it's not that complex, after all.

I hope that Qbs users will find this interesting, and possibly improving my setup. Ideally we should try to get something like this part of Qbs itself, but portability outside of Linux / gcc is going to be an issue.

One of the main goals of this GSoC project is to have a fully working build of KDE ISO Image Writer on Windows to allow people that want to install KDE Neon to easily write the ISO image onto a USB flash drive.

In order to compile the code on Windows, I used Craft which is a cross-platform build system and package manager. With Craft, I could easily get the dependencies of KDE ISO Image Writer.

I started by writing a Craft blueprint which is a Python file that describes an application (or library) and list its dependencies which allows Craft to fetch the necessary packages before compiling the application.

My journey to get KDE ISO Image Writer running on Windows was not without its hurdles. I first had to figure out how to use Craft to compile an application from a Git repository but that was quickly solved after using Craft’s --help command. Then, I run into an issue with Qgpgme, which is used by KDE ISO Image Writer to verify the digital signature of an ISO image. I tried to compile using MSVC which failed systematically because of CMake complaining about not being able to finding Qgpgme. I learned from the KDE Windows team that Qgpgme can, at the moment, only be compiled using MinGW. I was finally able to compile KDE ISO Image Writer on Windows using Craft and MinGW.

In parallel to working on a Windows build of KDE ISO Image Writer, I continued my work on the user interface by implementing the designs made by the KDE Community. You can see in the following screenshots the new user interface running on Windows:

Writing an ISO image to a USB flash drive

For the last week I have been reading and understanding the code of kis_imagepipe_brush.cpp, and it has been an eye opening experience. Read More...

June 30, 2019

For those that don't know me, I'm relatively new to KDE and spend most of my time doing VDG (Visual Design Group) stuff . The Plasma/Usability & Productivity sprint in Valencia which took place from June 19th to June 26th, was my first ever KDE sprint. Although we were all working together, I was formally...... Continue Reading →

Actually, a smart pointer is quite simple: It is an object that manages another object by a certain strategy and cleans up memory, when the managed object is not needed anymore. The most important types of smart pointers are:

  • A unique pointer that models access to an object that is exclusively maintained by someone. The object is destroyed and its memory is freed then the managing instance destroys the unique pointer. Typical examples are std::unique_ptr or QScopedPointer.
  • A shared pointer is a reference counting pointer that models the shared ownership of an object, which is managed by several managing instances. If all managing instances release their partly ownership, the managed object is automatically destroyed. Typical examples are std::shared_ptr or QSharedPointer.
  • A weak pointer is a pointer to an object that is managed by someone else. The important use case here is to be able to ask if the object is still alive and can be accessed. One example is std::weak_ptr that can point to a std::shared_ptr managed object and can be used to check if the object managed by the shared pointer still exists and it can be used to obtain a shared pointer to access the managed object. Another example is QPointer, which is a different kind of weak pointer and can be used to check if a QObject still exists before accessing it.

For all these pointers one should always keep one rule in mind: NEVER EVER destroy the managed objects by hand, because the managed object must only be managed by the smart pointer object. Otherwise, how could the smart pointer still know if an object can still be accessed?! E.g. the following code would directly lead to a crash because of a double delete:

{
    auto foo = std::make_unique<Foo>();
    delete foo.get();
} // crash because of double delete when foo gets out of scope

This problem is obvious, now let’s look at the less obvious problems one might encounter when using smart pointers with Qt.

QObject Hierarchies

QObject objects and instances of QObject derived classes can have a parent object set, which ensures that child objects get destroyed whenever the parent is destroyed. E.g., think about a QWidget based dialog where all elements of the dialog have the QDialog as parent and get destroyed when the dialog is destroyed. However, when looking at smart pointers there are two problems that we must consider:

1. Smart Pointer Managed objects must not have a QObject parent

It’s as simple as the paragraph’s headline: When setting a QObject parent to an object that is managed by a smart pointer, Qt’s cleanup mechanism destroys your precious object whenever the parent is destroyed. You might be lucky and destroy your smart pointer always before the QObject parent is destroyed (and nothing bad will happen), but future developers or user of your API might not do it.

2. Smart Pointers per default call delete and not deleteLater

Calling delete on a QObject that actively participates in the event loop is dangerous and might lead to a crash. So, do not do it! – However, all smart pointers that I am aware of call “delete” to destroy the managed object. So, you actively have to take care of this problem by specifying a custom cleanup handler/deleter function. For QScopedPointer there already exists “QScopedPointerDeleteLater” as a predefined cleanup handler that you can specify. But you can do the same for std::unique_ptr, std::shared_ptr and QSharedPointer by just defining a custom deleter function and specifying it when creating the smart pointer.

Wrestling for Object Ownership with the QQmlEngine

Besides the QObject ownerships there is another, more subtle problem that one should be aware of when injecting objects into the QQmlEngine. When using QtQuick in an application, often there is the need to inject objects into the engine (I will not go into detail here, but for further reading see https://doc.qt.io/qt-5/qtqml-cppintegration-topic.html). The important important fact one should be aware of is that at this point there is a heuristic that decides whether the QML engine and its garbage collector assumes ownership of the injected objects or if the ownership is assumed to be on C++ side (thus managed by you and your smart pointers).

The general rule for the heuristic is named in the QObjectOwnership enum. Here, make sure that you note the difference between QObjects returned via a Q_PROPERTY property and via a call of a Q_INVOKABLE methods. Moreover, note that the description there misses the special case of when an Object has a QObject parent, then also the CppOwnership is assumed. For a detailed discussion of the issues there (which might show you a surprisingly hard to understand stack trace coming from the depths of the QML engine), I suggest reading this blog post.

Summing up the QML part: When you are using a smart pointer, you will hopefully not set any QObject parent (which automatically would have told the QML engine not to take ownership…). Thus, when making the object available in the QML engine, you must be very much aware about the way you are using to put the object into the engine and if needed, you must call the QQmlEngine::setObjectOwnership() static method to mark your objects specifically that they are handled by you (otherwise, bad things will happen).

Conclusion

Despite of the issues above, I very much favor the use of smart pointers. Actually, I am constantly switching to smart pointers for all projects I am managing or contributing. However, one must be a little bit careful and conscious about the side effects when using them in Qt-based projects. Even if they bring you a much simpler memory management, they do not relieve you from the need to understand how memory is managed in your application.

PS: I plan to continue with a post about how one could avoid those issues with the QML integration on an architectural level, soon; but so much for now, the post is already too long ��

Following Dan it’s my turn this time to provide you with an overview on what has happened around Kontact in the past two months. With more than 850 commits by 22 people in the KDE PIM repositories this can barely scratch the surface though.

KMail

Around email a particular focus area has been security and privacy:

  • Sandro worked on further hardening KMail against so-called “decryption oracle” attacks (bug 404698). That’s an attack where intercepted encrypted message parts are carefully embedded into another email to trick the intended receiver in accidentally decrypting them while replying to the attacker’s email.
  • Jonathan Marten added more fine-grained control over proxy settings for IMAP connections.
  • André improved the key selection and key approval workflow for OpenGPG and S/MIME.

Laurent also continued the work on a number of new productivity features in the email composer:

  • Unicode color emoji support in the email composer.
Color emoji selector in the KMail composer. Color emoji selector.
Grammalecte reporting a grammar error in the KMail composer. Grammalecte reporting a grammar error.
  • Markdown support in the email composer, allowing to edit HTML email content in markdown syntax.
Markdown editing with preview in the KMail composer. Markdown editing with preview.

This isn’t all of course, there’s plenty of fixes and improvements all around KMail:

  • Albert fixed fixed an infinite loop when the message list threading cache is corrupted.
  • David fixed a Kontact crash on logout (bug 404881).
  • Laurent fixed access to more than the first message when previewing MBox files (bug 406167).
  • The itinerary extraction plugin benefited from a number of improvements in the extractor engine, see this post for details.

And fixing papercuts and general polishing wasn’t forgotten either (most changes by Laurent):

  • Fix cursor jumping into the Bcc field in new email (bug 407967).
  • Fix opening the New Mail Notifier agent configuration.
  • Fix settings window being too small (bug 407143).
  • Fix account wizard not reacting to Alt-F4 (bug 388815).
  • Fix popup position in message view with a zoom level other than 100%.
  • Fix importing attached vCard files (bug 390900).
  • Add keyboard shortcut for locking/unlocking the search mode.
  • David fixed interaction issues with the status bar progress overlay.

KOrganizer

Around calendaring, most work has been related to the effort of making KCalCore part of KDE Frameworks 5, something that particularly benefits developers using KCalCore outside of KDE PIM. The changes to KCalCore also aimed at making it easier to use from QML, by turning more data types into implicitly shared value types with Q_GADGET annotations. This work should come to a conclusion soon, so we can continue the KF5 review process.

Of course this isn’t all that happened around calendaring, there were a few noteworthy fixes for users too:

  • Fixed an infinite loop in the task model in case of duplicate UIDs.
  • Improved visibilities of timeline/Gantt views in KOrganizer with dark color schemes.
  • Damien Caliste fixed encoding of 0 delay durations in the iCal format.

KAddressBook

Like calendaring, contact handling also saw a number of changes related to making KContacts part of KDE Frameworks 5. Reviewing the code using KContacts lead to a number of repeated patterns being upstreamed, and to streamlining the contact handling code to make it easier to maintain. As a side-effect, a number of issues around the KContacts/Grantlee integration were fixed, solving for example limitations regarding the localization of contact display and contact printing.

There is one more step required to complete the KContacts preparation for KDE Frameworks 5, the move from legacy custom vCard entries to the IMPP element for messaging addresses.

Akregator

Akregator also received a few fixes:

  • Heiko Becker fixed associating notifications with the application.
  • Wolfgang Bauer fixed a crash with Qt 5.12 (bug 371511).
  • Laurent fixed comment font size issues on high DPI screens (bug 398516), and display of feed comments in the feed properties dialog (bug 408126).

Common Infrastructure

The probably most important change in the past two months happened in Akonadi: Dan implemented an automatic recovery path for the dreaded “Multiple Merge Candidate” error (bug 338658). This is an error condition the Akonadi database state can end up in for still unknown reasons, and that so far blocked successful IMAP synchronization. Akonadi is now able to automatically recover from this state and with the next synchronization with the IMAP server put itself back into a consistent state.

This isn’t all though:

  • Another important fix was raising the size limit for remote identifiers to 1024 characters (bug 394839), also by Dan.
  • David fixed a number of memory leaks.
  • Filipe Azevedo fixed a few macOS specific deployment issues.
  • Dan implemented improvements for using PostreSQL as a backend for Akonadi.

The backend connectors also saw some work:

  • For the Kolab resource a memory management issues got fixed, and it was ported away from KDELibs4Support legacy code.
  • David Jarvie fixed a few configuration related issues in the KAlarm resource.
  • Laurent fixed configuration issues in the MBox resources.

The pimdataexporter utility that allows to import/export the entire set of KDE PIM settings and associated data has received a large number of changes too, with Laurent fixing various import/export issues and improving the consistency and wording in the UI.

Help us make Kontact even better!

Take a look at some of the junior jobs that we have! They are simple, mostly programming tasks that don’t require any deep knowledge or understanding of Kontact, so anyone can work on them. Feel free to pick any task from the list and reach out to us! We’ll be happy to guide you and answer all your questions. Read more here…

We’re up to week 77 in KDE’s Usability & Productivity initiative! This week’s report encompasses the latter half of the Usability & Productivity sprint. Quite a lot of great work got done, and two features I’m particularly excited about are in progress with patches submitted and under review: image annotation support in Spectacle, and customizable sort ordering for wallpaper slideshows. These are not done yet, but should be soon! Meanwhile, check out what’s already landed:

New Features

Bugfixes & Performance Improvements

User Interface Improvements

Pretty freakin’ sweet, huh?! It was a great development sprint and I’m really happy with how it went. I’ll be writing another more in-depth article about it, so stay tuned.

Next week, your name could be in this list! Not sure how? Just ask! I’ve helped mentor a number of new contributors recently and I’d love to help you, too! You can also check out https://community.kde.org/Get_Involved, and find out how you can help be a part of something that really matters. You don’t have to already be a programmer. I wasn’t when I got started. Try it, you’ll like it! We don’t bite!

If you find KDE software useful, consider making a tax-deductible donation to the KDE e.V. foundation.

June 29, 2019

Previously: 1st GSoC post 2nd GSoC post With the first phase of Google Summer of Code over it's high time some substantial progress on achieving the main goal of the project was presented. Since the last post, there's two things that have been done. First, Plasma is now going to be following upstream advice on...... Continue Reading →

June 28, 2019


Welcome Latte Dock v0.8.9  the LAST stable release for v0.8 branch!


Go get  v0.8.9   from, download.kde.orgor  store.kde.org*
-----
* archive has been signed with gpg key: 325E 97C3 2E60 1F5D 4EAD CF3A 5599 9050 A2D9 110E

Fixes:
  • fix: show notifications applet when present in Latte (for Plasma >= 5.16)

Latte v0.9:

For those following Latte news, in July first beta release of v0.9 branch will land and if everything goes on schedule v0.9 will replace v0.8 during first days of August as the officially supported stable version. Requirements for v0.9 are the same with v0.8:

Minimum requirements:
  • Qt >= 5.9
  • Plasma >=5.12
Proposed requirements:
  • Qt >= 5.12
  • Plasma >=5.15

Community Help:

Latte v0.9 introduces two new APIs that can be used from developers in order to leverage the full potential of the new version.

First API can be used from plasma applets to exchange information with Latte from qml code. This way we can have applets that work just fine with plasma panels and at the same time leveraging the maximum of capabilities from Latte docks/panels. My Window Applets are already using it in their latest versions.

Second API is related to new standalone Latte indicators and how they can be implemented from developers.

In order for these APIs to reach developers I would like these information to be present at kde techbase. I have already created a word style document for them but you would sorry me that I do not have the time to upload them in markup language in the referenced page. If you think you can help or take up this task please contact me through the relevant bug report in kde bug tracker . If no community interest appears, I will just upload the 15 pages pdf  somewhere on the net and forward all the developers interested at that link.


Donations:

You can find Latte at Liberapay if you want to support,     Donate using Liberapay


or you can split your donation between my active projects in kde store.

Older blog entries

 

 

A Twitter List by KdeList

Planet KDE is made from the blogs of KDE's contributors. The opinions it contains are those of the contributor. This site is powered by Rawdog and Rawdog RSS. Feed readers can read Planet KDE with RSS, FOAF or OPML.