In many cases, importing data into LabPlot for further analysis and visualization is the first step in the application:
LabPlot supports many different formats (CSV, Origin, SAS, Stata, SPSS, MATLAB, SQL, JSON, binary, OpenDocument Spreadsheets (ods), Excel (xlsx), HDF5, MQTT, Binary Logging Format (BLF), FITS, NetCDF, ROOT (CERN), LTspice, Ngspice) and we plan to add support for even more formats in the future. All of these formats have their reasons for existence as well as advantages and disadvantages. However, the performance of reading the data varies greatly between the different formats and also between the different CPU generations. In this post, we’ll show how long it takes to import a given amount of data in four different formats – ASCII/CSV, Binary, HDF5, and netCDF.
This post is not about promoting any of the formats, nor is it about doing very sophisticated measurements with different amounts and types of data and extensive CPU benchmarking. Rather, it’s about what you can (roughly) expect in terms of performance on the new and not so new hardware with the current implementation in LabPlot.
For this exercise, we import the data set with 1 integer column and 5 columns of float values (Brownian motion for 5 “particles”, one integer column for the index) with 50 Millions of rows which results into 300 Millions of numerical values:
We take 6 measurements for each format, ignore the first measurement, which is almost always an outlier due to the disk cache in the kernel and results in faster file reads on subsequent accesses, and calculate the averages:
As expected, the file formats that deal with binary representation internally (Binary, HDF5, NetCDF) provide significantly better performance compared to ASCII, and this difference becomes larger the slower the CPU is. The performance of HDF5 and NetCDF is almost the same because the newer version of NetCDF is based on HDF5.
The implementation in the data import code is straightforward. Ignoring for a moment the complexity with the different options affecting the behavior of the parser, different data types and other subleties, once everything is set up it’s just a matter of iterating over the data, parsing it and converting it into the internal structures. The logic inside the loop is fixed, and a linear behavior with respect to the total number of values to read is expected. This expectation is confirmed using the same CPU (we took the fastest CPU from the table above) and varying the total number of rows with the fixed number of columns:
The performance of the import is even more critical when dealing with external data that is frequently modified. In order to provide a smooth visualization in LabPlot for such “live data”, it’s important to optimize all steps involved here, like the import of the new data itself, as well as the recalculation in the algorithms (smoothing, etc.) and in the visualization part. For the next release(s), we’re now working to further optimize the implementation to handle more performance-critical scenarios in the near future. The results of these activities, funded by the NLnet grant, will be the subject of a dedicated post soon.
Tuesday, 5 November 2024. Today KDE releases a bugfix update to KDE Plasma 6, versioned 6.2.3.
Plasma 6.2 was released in October 2024 with many feature refinements and new modules to complete the desktop experience.
This release adds two weeks' worth of new translations and fixes from KDE's contributors. The bugfixes are typically small but important and include:
- Bluedevil: Correct PIN entry behavior. Commit.
- KWin: Backends/drm: don't set backlight brightness to 1 in HDR mode. Commit. Fixes bug #495242
- KDE GTK Config: Gracefully handle decoration plugin failing to load. Commit.
Update: we have fixed the issue and the update is rolling out in the beta channel. The production channel still uses 5.2.3, but that will be updated later.
On releasing the latest version of Krita in our Android/ChromeOS beta program, we discovered, too, late that there was a problem that could prevent Krita from starting.
Since the Google Play Store Console does not allow revering a release to an earlier version, we are now urgently working on a fix which we will release as soon as possible.
Our apologies for the inconvience.
The currentl nightly builds for Android work again, with some limitations:
- take care removing the store version of Krita does not remove the application data: your artwork could be lost.
- in the Nightly builds you need to install any brush presets separately
You can get the night builds here: Krita Next Nightly Builds. You will need to select the package that is right for the architecture of your device.
Installing the nightly builds requires enabling developer mode on your device and needs considerable technical insight.
If you do not feel comfortable with this, please wait until the new official release lands in the play store in a about two days.
Sunday, 3 November 2024
Dolphin's accessibility, Itinerary's travelling aids, and a bunch of new upcoming KDE apps
Welcome to a new issue of "This Week in KDE Apps"! Every week we cover as much as possible of what's happening in the world of KDE apps.
In this issue we discover what developers have been doing to make Dolphin, KDE's most popular (but not only!) file explorer, more accessible. We also take a look at all the new services now integrated into Itinerary that will help you on your travels, the new features for Kate that programmers will enjoy, improvements to Kleopatra to help you manage your certificates and the encryption of your messages, and the flurry of new applications that will soon be available in KDE's software catalog.
This week, we also kicked off our 2024 end-of-year fundraiser just in time for Halloween! Any monetary contribution, however small, will help us cover operational costs, salaries, travel expenses for contributors and in general just keep KDE bringing Free Software to the world. So consider doing a donation today!
Let's dig in!
Dolphin Manage your files
We improved the keyboard navigation of Dolphin, as pressing Ctrl+L multiple times will switch back and forth between focusing and selecting the location bar path and focusing the view. Pressing Escape in the location bar, will now move the focus to the active view (Felix Ernst, 24.12.0. Link).
Speaking of accessibility, the accessibility of the main view of Dolphin was completely overhauled to make it work with screen readers. This work was funded by NGI0 Entrust Fund, a fund established by NLnet with financial support from the European Commission's Next Generation Internet programme (Felix Ernst, 24.12.0. Link).
Another change is that Dolphin will now store its view properties inside the extended file attributes instead of creating hidden .directory
files when possible (Méven Car, 24.12.0. Link).
digiKam Photo Management Program
digiKam is KDE's powerful photo management software for both professional and aficionado photographers.
Michael Miller fixed an issue where faces from the facial recognition feature were not deleted when a user untagged or deleted a face (Michael Miller. Link).
Elisa Play local music and listen to online radio
Jack Hill fixed a few issues related to the lyrics feature. Clicking on the Lyrics button now takes you to the correct lyric and not the previous one, and the last line of the lyrics is not displayed completely (Jack Hill, 24.12.0. Link).
Jack also reworked the metadata dialogs to be more intuitive and correct some bugs (Jack Hill, 24.12.0. Link).
GCompris Educational game for children
GCompris, the educational software suite, got a new activity: Sketch! This fun tool lets children express their creativity and draw beautiful artworks (Timothée Giet. Link).
KDE Itinerary Digital travel assistant
Itinerary now supports extracting reservations from planway.com, flight tickets from VietJet Air and train tickets from the Thai state railway. Additionally, dates from day-specific train tickets from NS (Nederlandse Spoorwegen) are now correctly parsed (Volker Krause, 24.08.3. Link 1, link 2, link 3), and, while on an NS intercity train, you can also access the live journey information provided by the onboard WiFi (Carl Schwan, 24.12.0. Link).
Kate Advanced Text Editor
Kate continues to become more and more developer friendly with the changes made by Christoph Cullmann where the order of the tabs is correctly restored when restoring a previous session (Christoph Cullmann, 24.08.3. Link), and the options of the LSP Symbols are more easily discoverable as they are not only available via a context menu, but also within a menu button at the top (Waqar Ahmed, 24.12.0. Link).
Benjamin Port fixed the Appium UI tests and reenabled them on the CI (Benjamin Port, 28.03.0. Link).
Kdenlive Video editor
Kdenlive is KDE's full-featured video editor, which now lets you resize multiple items on the timeline at the same time. (Jean-Baptiste Mardelle, 24.12.0 Link).
Kleopatra Certificate manager and cryptography app
We redesigned Kleopatra's notepad and sign encrypt dialog. In the notepad, the text editor and the recipients view are also now side by side (Carl Schwan, 24.12.0. Link).
Additionally, Tobias worked on the notepad's result messages and error dialogs to make them clearer. (Tobias Fella, 24.12.0 Link 1, link 2).
Tobias also fixed a crash on non-kwin Wayland compositors (Tobias Fella, 24.08.3. Link), and Kleopatra has a new website (Carl Schwan. Link).
Kongress Conference companion
Kongress is an app which helps you navigate conferences and events.
The newest version will display more information about events in the event list. This includes whether the event is in your bookmarked events and the locations within the event (e.g. the rooms) (cah fof pai, 24.12.0. Link).
Speaking of conferences, multiple KDE people will be at the Chaos Communication Congress (38c3) in Hamburg this December! Come by and say hello!
KStars Desktop Planetarium
KStars is KDE's stargazing app that also helps you control your telescope for astrophotography.
We removed the "Simulate Eyepiece View" feature and stripped down EyepieceField
. The reason is the offerings of the eyepiece view feature have already been superseded by two more powerful and easier-to-use features in KStars: the HiPS Overlay and the "Views" feature (Akarsh Simha, 3.7.4. Link).
Kwave Sound editor
Mark Penner wrote a blog post about his work on KWave.
LabPlot Interactive Data Visualization and Analysis
LabPlot is KDE's complete suite of data analysis and visualisation tools.
The LabPlot developers added the RAND_MAX
programming constants for GSL (GNU Scientific Library) support. (Martin Marmsoler Link), and rewrote the AsciiFilter to increase the parsing speed, like when parsing livedata from an mqtt feed (Martin Marmsoler. Link).
Kuntal Bar also fixed various issues with HiDPI screens (Kuntal Bar, Link).
Marknote Write down your thoughts
Marknote lets you create rich text notes and easily organise them into notebooks.
The icons in the app are now correctly displayed when running Marknote on other platforms, like Windows (Gary Wang, Link).
Ruqola Rocket Chat Client
Ruqola, KDE's Rocket Chat client, received various fixes for its login, logout and network disconnection features (David Faure & Andras Mantia Link 1, link 2, link 3 and link 4), and the unread message bar now uses buttons instead of more subtle links (Joshua Goins. Link).
Spectacle Screenshot Capture Utility
Spectacle is the utility for taking screenshots and screencasts of your desktop and apps. We fixed an issue where Spectacle would take a screenshot of itself (Noah Davis, 24.08.3. Link).
Other Stuff
The FormCard components used by most Kirigami application are now more compact on the desktop (Carl Schwan, Kirigami Addons 1.6.0 Link).
The About Page provided by the FormCard component now displays more information about the components used by the application (e.g. Qt, KDE Frameworks) (Carl Schwan, Kirigami Addons 1.6.0. Link).
Playground
This section contains news about non released applications.
Arkade
Arkade, a collection of games written in QML, was updated to Qt6 (Carl Schwan. Link).
Whale
Claudio Cambra ported Whale, a QML based file manager and explorer, to Qt6 (Claudio Cambra. Link). Claudio also added a miller columns view to Whale (Claudio Cambra. Link), and implemented navigation history (Claudio Cambra. Link).
And all this too...
Justin Zobel fixed various appstream files to use the new way of declaring the developer's name (Justin Zobel, KRuler, Gwenview, KEuroCalc, ...).
We ported various projects to use declarative QML declaration for better maintainance and performance (Carl Schwan, Koko, Francis, Kalk).
... And Everything Else
This blog only covers the tip of the iceberg! If you’re hungry for more, check out Nate's blog about Plasma and be sure not to miss his This Week in Plasma series, where every Saturday he covers all the work being put into KDE's Plasma desktop environment.
For a complete overview of what's going on, visit KDE's Planet, where you can find all KDE news unfiltered directly from our contributors.
Get Involved
The KDE organization has become important in the world, and your time and contributions have helped us get there. As we grow, we're going to need your support for KDE to become sustainable.
You can help KDE by becoming an active community member and getting involved. Each contributor makes a huge difference in KDE — you are not a number or a cog in a machine! You don’t have to be a programmer either. There are many things you can do: you can help hunt and confirm bugs, even maybe solve them; contribute designs for wallpapers, web pages, icons and app interfaces; translate messages and menu items into your own language; promote KDE in your local community; and a ton more things.
You can also help us by donating. Any monetary contribution, however small, will help us cover operational costs, salaries, travel expenses for contributors and in general just keep KDE bringing Free Software to the world.
To get your application mentioned here, please ping us in invent or in Matrix.
Saturday, 2 November 2024
Surprise! This blog post series has now been moved to blogs.kde.org so it’s now open for others to participate and contribute! This week’s post can be found at https://blogs.kde.org/2024/11/02/this-week-in-plasma-spoooooky-ooooooooom-notifications
That’s probably where it should have been all along, as this work is much bigger than me. I’ll remain the editor-in-chief for now, but do welcome contributions to help lighten the load.
Unfortunately, due to GDPR restrictions, I’m unable to migrate existing email subscribers to the new email digest over there. So if you’d like to re-subscribe to “This week in Plasma.” head to https://newsletter.kde.org/subscription/form and re-subscribe.
I’ll still be blogging here about KDE topics of interest to me and hopefully you as well, just not the weekly Plasma news. So I do hope you’ll stick around.
Welcome to the new home of "This Week in Plasma"! No longer is it a private personal thing on my (Nate Graham's) blog, but now it's a weekly series hosted here on KDE's infrastructure, open to anyone's participation and contribution! I'll remain the editor-in-chief for now, and welcome contributions via direct push to the relevant merge request on invent.kde.org. And after a post is published, if you find a typo or broken link, feel free to just fix it.
Anyway, this week we added a useful service to detect out-of-memory (OOM) conditions, did some UI polishing, and also a lot of bug-fixing! Check it out:
Notable New Features
When the system has run out of memory (OOM) and the kernel terminated an app, there's now a little service to detect this and show you a system notification explaining what happened, plus suggestions for what you can do about it in the future. (Harald Sitter, 6.3.0. Link)
Notable UI Improvements
The Emoji Selector app now does sub-string matching from the middle of words too, so you can find emojis more easily. (Eren Karakas, 6.2.3. Link)
The grouping indicator in the Task Manager widget (which looks like a little plus sign) is no longer always green; now it follows the current accent color! (Tem PQD, 6.3.0. Link)
Appropriate symbolic icons from will now be automatically substituted for apps' system tray icons — when they exist in the icon theme. If you'd like to extend this to more apps, figure out what icon name the app asks for, then create a symbolic version of it, append -symbolic
to the name, and submit it to the Breeze Icons repo! (Marco Martin, 6.3.0. Link)
Notable Bug Fixes
Fixed a way that misbehaving XWayland-using apps could make KWin freeze. (Vlad Zahorodnii, 6.2.3. Link)
Fixed a bug causing notifications about modifier key changes (if you've turned them on) to not actually appear. (Nicolas Fella, 6.2.3. Link)
Fixed a bug in the Bluetooth pairing wizard that caused it to be annoying to enter digits. (Daniil-Viktor Ratkin, 6.2.3. Link)
Reviews on Discover's app pages now load properly when the app is accessed from the Home page, as opposed to after browsing or searching. (Aleix Pol Gonzalez, 6.2.3. Link)
Fixed the Task Manager widget's setting to reverse the direction that new tasks appear in so that it works as originally designed, and not only halfway there. (Michael Rivnak, 6.2.3. Link)
Fixed a bug that caused customized user avatars for other currently-logged-in users to not be displayed in the User Switcher widget. (Blazer Silving, 6.2.3. Link)
Fixed a bug causing the "Show Logout Screen" item in the desktop context menu to not show enough items if you had customized the "default logout option" in the past, back when we offered that as a user-facing setting. (Nate Graham, 6.2.3 Link)
With global animations set to "instant", window thumbnails in KWin's Desktop Grid effect are no longer too small. (Niccolò Venerandi, 6.2.3. Link)
When you've got the Application Dashboard widget set up with an icon naming style that causes the text to be long and get elided, hovering over the icon now shows a tooltip with the correct full text in it. (Tomislav Pap, 6.2.3. Link)
Fixed a case where KWin could crash when moving certain drawing tablet pens to the surface of the pad. (Vlad Zahorodnii, 6.3.0. Link)
Fixed an issue that caused XWayland-using apps to resize in a janky and jumpy manner. (Vlad Zahorodnii, 6.3.0. Link, see detailed blog post)
How You Can Help
KDE has become important in the world, and your time and contributions have helped us get there. As we grow, we need your support to keep KDE sustainable.
You can help KDE by becoming an active community member and getting involved somehow. Each contributor makes a huge difference in KDE — you are not a number or a cog in a machine!
You don’t have to be a programmer, either. Many other opportunities exist:
- Filter and confirm bug reports, maybe even identify their root cause
- Contribute designs for wallpapers, icons, and app interfaces
- Design and maintain websites
- Translate user interface text items into your own language
- Promote KDE in your local community
- …And a ton more things!
You can also help us by donating to our yearly fundraiser! Any monetary contribution — however small — will help us cover operational costs, salaries, travel expenses for contributors, and in general just keep KDE bringing Free Software to the world.
To get a new Plasma feature or a bugfix mentioned here, feel free to push a commit to the relevant merge request on invent.kde.org.
Friday, 1 November 2024
In Plasma 6.2, KWin switched from doing linear blending with HDR to blending in a gamma 2.2 space. Let’s take a look at what that means, and why it was done.
What is blending?
When KWin composites, it paints window by window, going by the order of how the windows are stacked - the bottom-most window first, the topmost window last. When a window is opaque, you just overwrite the pixels in the framebuffer with the ones from the window. When a window is semi-transparent though, we need to additionally do blending.
To do blending, the GPU calculates the value that the framebuffer should have with some equation that gets the pixel from the window and the existing value in the framebuffer, and outputs some appropriate value. Usually that equation is1
framebuffer = framebuffer.rgb * (1 - window.alpha) + window.rgb * window.alpha
where window.alpha
is a per-pixel value that describes how opaque the pixel is.
Blending with color management
In Plasma 5, compositing happened in the display color space. As displays could be assumed to be roughly the same, with brightness levels encoded in sRGB, that resulted in blending looking very similar everywhere.
With HDR in Plasma 6 however, that assumption was no longer true, so we had to do something else. As it was considered the most “correct” thing and allows us to ensure the exact same blending result even with displays that have wildly different colorspaces, linear blending was chosen for Plasma 6.0. The result of linear blending would look different from sRGB, but it would at least be consistent everywhere.
With linear blending, instead of just taking the rgb values as is, you first convert them into light-linear values, for example with sRGB you’d just do
rgb_linear = pow(rgb_sRGB, 2.2)
then apply the blending equation, and at the end convert it back to whatever encoding the screen needs.
Because of limitations in KWin’s renderer and the ancient OpenGL versions KWin supports, the only way to do this was to render first into a so-called shadow buffer2 with linear values, and after compositing a second shader pass would run, taking that shadow buffer as the input and outputting a buffer with non-linear values that’s suitable for sending to the screen.
The caveats of linear blending
It’ll be pretty obvious to most that doing a fullscreen copy each frame is not ideal for performance or battery life, but there was even a second problem: If you use only 8 or 10 bits per color (bpc) to store linear brightness values, you get visible banding in the dark areas of the image, as human vision is very non-linear. So on top of doing fullscreen copies, KWin also had to use a floating point buffer with 16 bpc, which uses twice the memory bandwidth vs. 8 bpc and makes performance and power usage even worse.
With that performance hit, we couldn’t enable this on all hardware, but had to restrict linear blending to those displays where HDR or an ICC profile is enabled. This threw the whole consistency benefit out of the window, because now a semi-transparent surface would look a lot more transparent just because you enabled HDR… causing the very problem we wanted to avoid!
SDR | HDR |
---|---|
We had to do something when blending in HDR though, so a different approach had to be taken.
Custom transfer functions
When storing colors in buffers, usually we use non-linear encodings with transfer functions like sRGB or PQ. All the transfer functions have some sort of luminance levels attached to them, for example an sRGB value of 1.0 is defined to result in a luminance of 80 nits in its reference viewing environment3.
There’s no reason to be restricted to those definitions though - we can make transfer functions that mean whatever we want or need. In KWin’s case, we switched the shadow buffer from using a linear encoding to a gamma 2.2 encoding, in which 1.0 means the maximum luminance of your monitor. This means we do blending in a very similar way as on SDR screens4, and translucent surfaces on the screen look pretty much the same again, no matter what display settings you have.
As the gamma 2.2 encoding allocates more numbers to darker parts of the image, we can also avoid banding with fewer bits per color. Whenever HDR is enabled and the driver supports buffers with 10 bpc, KWin now prefers to use that instead of 16 bpc, alleviating some of the performance issues. There was still more that could be done though…
KMS offloading
On most graphics cards, the scanout hardware that takes care of sending the image to the display has some fixed function blocks to change the colors in various ways - the most common being a simple look up table (LUT) per color channel.
Through the DRM/KMS API, the compositor can set this LUT, so we can use it to change the encoding from whatever we did blending in to the one the display needs. With HDR screens, this means we
- convert from our shadow buffer encoding with gamma 2.2 and the maximum luminance of the screen to linear
- possibly apply rgb factors for night light
- convert from linear to the PQ encoding the screen needs
With that LUT in place, we don’t have to run a shader pass to convert from the shadow buffer encoding to the screen anymore, so the whole fullscreen copy falls away.
Except for a few additional instructions in the shaders used for compositing, enabling HDR in Plasma 6.2 thus has no performance impact anymore on the vast majority of hardware!
in practice,
window.rgb
is already “pre-multiplied” withwindow.alpha
, but that doesn’t really matter here ↩it’s called that because it’s never actually shown on the screen ↩
the viewing environment basically defines a standardized room, in which the content can be viewed as intended without doing any brightness adjustments ↩
it’s not exactly the same though. If you need some exactly specific blending behavior in your application, it’s best if you do it yourself ↩
Let’s go for my web review for the week 2024-44.
What You Can Learn from Just Seven Pages by Hannah Arendt
Tags: tech, philosophy, history, politics
A very precious philosopher from the 20th century. Her texts are still very precious and resonate today. In this piece it’s focusing about tech relevant excerpts, she had plenty to say about today’s politics as well.
https://www.honest-broker.com/p/what-you-can-learn-from-just-seven
The Open Source AI Definition
Tags: tech, foss, ai, machine-learning
Nice initiative from the OSI. It is timely, such a definition was surely needed. The data information part seems fairly weak though… for sure you could make a system which doesn’t respect the four freedoms that way.
https://opensource.org/ai/open-source-ai-definition
Does Open Source AI really exist?
Tags: tech, foss, ai, machine-learning
Like me, you find the Open Source AI Definition weak on the training data information side? You’d be right and there’s a reason for it… it’s probably hiding quite some open washing for the larger models. This is a good explanation of the motives and consequences.
https://tante.cc/2024/10/16/does-open-source-ai-really-exist/
Platform Strategy and Its Discontents
Tags: tech, web, mobile, react, framework, criticism
This is definitely true. As long as web frontends are dominated by large frameworks, the web will always have subpar experience on mobile. And the solution isn’t going to come from the mobile providers too happy to gatekeep their app store.
https://infrequently.org/2024/10/platforms-are-competitions/#fn-failure-on-repeat-2
Matrix 2.0 Is Here!
Tags: tech, matrix, protocols
This is definitely getting there in terms of performance and usability. The mobile clients seem mature enough, just need the desktop clients to catch up before this becomes really something I’d feel confident enough to recommend and push for.
https://matrix.org/blog/2024/10/29/matrix-2.0-is-here/
What’s New in POSIX 2024
Tags: tech, unix, posix
It’s nice to see the standard still moves. Some of the additions are definitely welcome.
https://blog.toast.cafe/posix2024-xcu
Australia/Lord_Howe is the weirdest timezone
Tags: tech, time, culture, internationalization
Time management and timezones are definitely complicated. In a way it’s culture colliding with computers and localisation… it can’t be simple.
https://ssoready.com/blog/engineering/truths-programmers-timezones/
Improving SSH’s security with SSHFP DNS records
Tags: tech, tools, ssh, dns, security
Nice technique for automating the verification of SSH host keys. It’d be nice to see wider adoption.
https://blog.apnic.net/2022/12/02/improving-sshs-security-with-sshfp-dns-records/
Introducing zizmor: now you can have beautiful clean workflows
Tags: tech, github, ci, security, tools
Definitely an interesting tool. GitHub Actions workflow aren’t easy to setup while ensuring they’re secure, having a tool analyzing them for issues can only help.
https://blog.yossarian.net/2024/10/27/Now-you-can-have-beautiful-clean-workflows
Toward safe transmutation in Rust
Tags: tech, rust, type-systems, memory
Interesting progress on safe type casting in Rust. This should bring nice zero copy parsing of binary data in some cases.
https://lwn.net/SubscriberLink/994334/5e1f97f08916b494/
Lessons learned from a successful Rust rewrite
Tags: tech, c++, rust, legacy
This is a good view of what you’re getting into with the “rewrite it in Rust” knee-jerk reaction.
https://gaultier.github.io/blog/lessons_learned_from_a_successful_rust_rewrite.html
Vector Databases Are the Wrong Abstraction
Tags: tech, postgresql, databases, ai, machine-learning, language
I definitely like the approach of having vectorisation in the RDBMS directly. This is one less moving part, less complexity at the application level to synchronize everything together. In this case it’s a Postgres extension.
https://www.timescale.com/blog/vector-databases-are-the-wrong-abstraction/
Database Remote-Copy Tool For SQLite
Tags: tech, sqlite, databases, tools
Interesting, there’s now an official tool to replicate sqlite databases. It’s still early days, we’ll see which features it’ll get.
How to profile a performance issue using Spring Boot profiling tools
Tags: tech, java, spring, profiling
A quick tour of the available tools to profile Spring Boot applications.
https://foojay.io/today/how-to-profile-a-performance-issue-using-spring-boot-profiling-tools/
Working with stacked branches in Git is easier with –update-refs
Tags: tech, git, tools
Yet another Git option I missed. This is definitely useful, I’ll try it out.
https://andrewlock.net/working-with-stacked-branches-in-git-is-easier-with-update-refs/
pygfx
Tags: tech, python, 3d, webgpu, data-visualization
Looks like a very interesting Python library to build interactive 3d visualizations.
https://docs.pygfx.org/stable/index.html
Rudimentary 3D on the 2D HTML Canvas
Tags: tech, 2d, 3d, graphics, mathematics
Ever wondered how to simulate 3D from 2D based primitives? Here is a nice experiment explaining how to approach it.
https://www.charlespetzold.com/blog/2024/09/Rudimentary-3D-on-the-2D-HTML-Canvas.html
Classic 3D videogame shadow techniques
Tags: tests, 3d, graphics
A nice list of the techniques used to render shadows in games.
https://30fps.net/pages/videogame-shadows/
On Crafting Painterly Shaders
Tags: tech, 3d, graphics, shader
Very nice deep dive into a post-processing shader to create a painted scene effect.
https://blog.maximeheckel.com/posts/on-crafting-painterly-shaders/?ck_subscriber_id=2669647738
The Basics
Tags: tech, craftsmanship, learning, career
Indeed, those are fundamental traits to make sure you learn and make progress on your journey.
https://registerspill.thorstenball.com/p/the-basics
Background Work - by Kent Beck
Tags: tech, learning, career, craftsmanship
Another excellent piece from Kent Beck, he’s right that the real differentiator in our profession is about digging deep on topics, seeing them through even if that’s on the side. Curiosity is a key trait.
https://tidyfirst.substack.com/p/background-work
the death of the architect
Tags: tech, agile, architecture, history
Good explanation on how the agile movement scaled down about design over time in its literature. It’s probably its biggest failure. The good thing is that the pendulum is starting to swing in the other direction a bit (that’s probably why Beck is now working on a book series on software design).
https://explaining.software/archive/the-death-of-the-architect/
How to make Product give a shit about your architecture proposal
Tags: tech, quality, product-management, project-management
This is accurate in my opinion. Engineering and product teams need to properly negotiate, otherwise quality will suffer.
How to not be a prioritization machine
Tags: tech, project-management, product-management, decision-making
Interesting guidelines idea to help teams manage the priorities themselves. It’s written in the context of a product manager but I think it is lightweight and generic enough to apply in other contexts.
https://productmanagers.substack.com/p/how-to-not-be-a-prioritization-machine
We Fell For The Oldest Lie On The Internet
Tags: science, complexity
It’s sometimes extremely difficult to get to the original source of a scientific claim. Our corpus of science is so large and complex now that finding where a claim comes from can be a daunting task.
https://www.youtube.com/watch?v=bgo7rm5Maqg
Changes in primary visual and auditory cortex of blind and sighted adults following 10 weeks of click-based echolocation training | Cerebral Cortex | Oxford Academic
Tags: science, neuroscience
This is an amazing example of the brain plasticity. It’s also great to have a patch for increased quality of life with a training of only a few weeks.
https://academic.oup.com/cercor/article/34/6/bhae239/7696241?login=false
Bye for now!
The Qt Quick 3D project was first announced and released in 2019. Since then, it has seen numerous enhancements. We have concentrated on boosting performance, introducing new features, and expanding the module's capabilities.