These are my data storage notes, targeting primarily personal data backups: regular files (documents, photo and music collections, not databases), moderate volume, added or edited rarely, backups are managed manually.
The "3-2-1 rule" for backups suggests to keep at least 3 copies of data, on at least 2 different storage devices, with at least one copy off-site.
The exact requirements and methods to achieve those may depend on one's threat model: in addition to device failures, bit rot, and unauthorized access by scrapers, one may have to consider fire or flooding, burglaries and robberies, book burning campaigns and censorship with isolation, hardware seizures and imprisonment without ability to maintain the remaining backups for years, inability--or a limited ability--to acquire replacement storage devices, and even uncommon and hypothetical scenarios, such as a global high energy EMP.
Considering the information security "CIA" triad (confidentiality, integrity, availability), we need encryption, so that lost or decommissioned drives will not leak personal data (i.e., crypto-shredding can be employed); integrity checking, so that we will either read back the data that was written or detect data corruption (and preferably even repair it); varied and common technologies (hardware interfaces, drivers, filesystems, file formats), so that there will be a good chance that at least some of the backups can be accessed with reasonable effort in different situations in the future.
Most of the technologies covered here are usable for both backups and working storage. I prefer to use more general tools, since they tend to be better maintained, and learning them usually is a more useful time investment than learning specialized backup systems (but for those, see Bacula, Borg, restic, DAR), some of which are quite similar to actual file systems (e.g., Borg is), while apparently often lacking error correction codes and redundancy within a single repository, but those may still be suitable for the task. Fortunately in this case the variety is preferable, and one can combine those. See also: Debian Reference Manual - 10. Backup and recovery, BackupAndRecovery - Debian Wiki, Synchronization and backup programs - ArchWiki.
As for portability, judging by experimentation in 2024, Android
(as on Google Pixel phones) and Windows only support single
(Ex)FAT partitions on USB drives, and probably only with MBR or
without a partition table; no LUKS or filesystems such as Btrfs
and ext4. So having to give up on compatibility with those for
my regular backups, though when used for data transfer or
unavoidable otherwise, one can use VeraCrypt (open-source, but
not always considered FLOSS, for Windows, also supported for
opening by cryptsetup, but creation would require additional
tools: e.g., VeraCrypt itself or zuluCrypt) and
exFAT. The /\:*?"<>| characters must be avoided
in file names to stay compatible with exFAT.
Reliable computer hardware is desirable to minimize errors and hardware failures: an UPS, ECC memory, and quality hardware (including storage) in general.
External HDDs (or combinations of internal ones and external boxes) are inexpensive and handy for local backups, allowing to keep them safely disconnected most of the time, and to easily plug into virtually any computer when needed.
USB flash drives seem more suitable for off-site backups, being more robust for physical transfer. Flash memory is not suited for a long-term storage without power though, so it is suggested to have them powered up at least for a few hours per year, letting the controllers to do maintenance, or even do data scrubbing (via a filesystem, if it supports that, or simply by forcing reading of all the files, possibly by verifying checksums) to nudge the rewrites. Writing onto cheap Kingston USB thumb drives (e.g., 256 GB DT Exodia) can be very slow, especially once about 2/3 of space is used and with ext4 on top of LUKS: writing at about 200 KB/s (less than 1 GB per hour). Even if you are not in a hurry, it makes one to wonder whether the device malfunctions, so perhaps it is better to not neglect the write speed completely, even for backup storage devices. I saw Apacer USB flash drives of the same capacity, which are even cheaper, having sustained write speeds of about 10 MB/s, at least with exFAT.
Having an erratic USB port, bus, or wires (built into a Thermaltake chassis) that occasionally disconnects devices during active writing, I had a Transcend JetFlash (64 GB) thumb drive apparently dying (hanging on any writing attempt, "Device not responding to setup address.") after such a disconnect, while Kingston ones survived a few of those. As a side note, this seems more hazardous than non-ECC memory.
Optical drives (CD, DVD, Blu-ray) are commonly suggested for archieval, though they seem less convenient for updates and for usage in general, and it is not quite clear whether the recordable ("burned" with a laser and a dye, as opposed to being stamped at a factory) CDs and DVDs are that long-lasting, but apparently they are still quite durable (see 2024 Optical Media Durability Update). And some aim archival storage explicitly (e.g., M-DISC, mostly with BD).
Paper backups may be useful as well, and quite reliable, particularly for texts and images. Acid-free paper should be used for those, and one may play with bookbinding then. Some use QR codes and other two-dimensional barcodes to store arbitrary digital data on paper. Out of hardware, one would need a printer and a scanner for those, though I should investigate that better. To combine human-readability with relative machine-readability, special fonts like OCR-A and OCR-B can be useful, possibly combined with error correction codes.
One may also consider keeping backup storage devices and related items in a specialized storage shelf, a Faraday cage, or a fire-resistant and/or waterproof safe.
To go further than that, including storage of physical items, one may also look into general archieval- and collection-related materials, such as the Preservation Self-Assessment Program.
I find it useful (for the peace of mind, at least) to set a
bootable operating system on at least one of the backup drives,
with all the necessary software to read the backups. So there
usually is EFI system partition (ESP), an unencrypted partition
for /boot (GRUB2 can handle encrypted ones, but it
would not make much difference), an encrypted partition for the
rest of the system (to prevent possible data leaks via cache,
for instance, after backups are accessed from it), and a
separate encrypted partition for the backup itself.
When installing a system using an installer, on a machine with more than
one disk and some existing systems present, the installer would often use
a seemingly random ESP on one of the internal disks, instead of the one on
the backup drive. Fixing it may involve booting via the GRUB shell after
GRUB fails to find or access its config from the
/boot partition, remounting (and fixing in
/etc/fstab) /boot/efi/, to point to the correct
drive's ESP, and then running grub-install to install it
there. Also removing undesirable directories from ESP manually, and
adjusting things with efibootmgr. Or one can opt for a more
involved/manual installation, setting it properly at once: see, for
instance, "Installing Debian GNU/Linux from a Unix/Linux System" and
"Full
disk encryption, including /boot: Unlocking LUKS devices from GRUB".
Alternatively, or additionally, one may set a personalized live system image, as described in the Debian Live Manual and similar documents for other systems.
I do partitioning with fdisk, mostly because other
common tools (or at least their fancy user interfaces) tend to
be buggy, and/or to hide technical information, neither of which
is desirable when partitioning storage
devices. fdisk is nice, commonly available, and
works well. With the setups described below, it works to set
LUKS or an encrypted filesystems directly on a block device,
without any partitioning, but it may also be desirable to store
some public data backups on a separate partition of the same
storage device, unencrypted.
RAID 1 (or possibly 5, 6) is nice to set if there are spare disks, but usually not as critical for redundant personal backups as it is, for instance, for a production server.
As of 2021 and for Linux-based systems, some of the common software options are:
sha256sum (integrity)
Those can be combined, even the ones serving the same purpose: for instance, storing file checksums would not harm even if the underlying filesystem supports those already. Likewise, it should not harm to encrypt the more important files (cryptographic keys, passwords), even while storing those on encrypted disks.
Below are notes and command cheatsheets for the setups I use.
This is probably the most basic and widely supported setup for Linux-based systems. Only authenticated integrity checks are supported by cryptsetup (and those are experimental), so no CRC and no recovery from minor errors without RAID. Perhaps dm-integrity can be set separately to use CRC32C, but that would complicate the setup. Or it can be skipped altogether, since integrity checking is experimental, and wiping can slow down the process considerably (while skipping the wiping easily leads to errors).
Initial setup:
# Optionally, add: --type luks2 --integrity hmac-sha256 cryptsetup luksFormat /dev/sdXY cryptsetup open /dev/sdXY backup2 mkfs.ext4 /dev/mapper/backup2 cryptsetup close backup2 mkdir /var/lib/backup2
A typical session (CLI-based, though this is also handled by graphical file managers, such as Thunar):
cryptsetup open /dev/sdXY backup2 mount -t ext4 /dev/mapper/backup2 /var/lib/backup2/ # synchronize backups umount /var/lib/backup2/ cryptsetup close backup2
When done, in order to safely eject a device, run eject
/dev/sdX, or possibly udisksctl power-off -b
/dev/sdX.
To change a passphrase, cryptsetup luksChangeKey
/dev/sdXY.
For RAID with mdadm, see "dm-crypt + dm-integrity + dm-raid = awesome!".
ZFS is not modular like LUKS and friends, there are license compatibility issues, and it is rather unusual overall, but apparently a good filesystem containing all the features needed here.
Initial setup:
# Ensure that linux headers are installed, needed for zfs-dkms apt install linux-headers-amd64 # Install zfsutils-linux (from "contrib" repositories) apt install zfsutils-linux # Find a partition ID ls -l /dev/disk/by-id/ | grep sda4 # Use that ID to create a single-device pool. The "mirror" keyword # should be added to set RAID 1. zpool create tank usb-WD_Elements_...-part4 # Create an encrypted file system. mkdir /var/lib/backup/ # For redundancy within a dataset, add to the command below: -o copies=2 zfs create -o encryption=on -o keyformat=passphrase -o mountpoint=/var/lib/backup tank/backup
ZFS comes with its own mounting and unmounting commands, and if it is to be used from different systems, the pools should be exported and imported (or just force-imported). A typical session, assuming that it is used from different systems:
# List pools available for import zpool import # Import the pool zpool import tank # Mount an encrypted file system zfs mount -l tank/backup # (Synchronize backups here) # Unmount the file system (or it will happen on export) zfs unmount tank/backup # Unmount the pool (also unnecessary to do manually though) zfs unmount tank # Export the pool zpool export tank # And eject or udisksctl power-off -b, as mentioned above
To change a passphrase, zfs change-key tank/backup.
This one is set with the DUP profile for both metadata and data, adding redundancy, and with sha256 checksums (instead of the default crc32c), to reduce chances of collisions.
Initial setup:
# LUKS, as with ext4 cryptsetup luksFormat /dev/sdXY cryptsetup open /dev/sdXY backup # The file system mkfs.btrfs --csum sha256 -m dup -d dup -L backup /dev/mapper/backup cryptsetup close backup mkdir /mnt/backup
A session:
cryptsetup open /dev/sdXY backup mount -t btrfs /dev/mapper/backup /mnt/backup/ # synchronize backups here umount /mnt/backup/ cryptsetup close backup eject /dev/sdX udisksctl power-off -b /dev/sda
As mentioned above, it is important to be able to detect errors with some integrity checks, but one may also aim single-device redundancy for a recovery using that single device (and a better overall chance of successful data recovery), as well as calculate checksums on top of a filesystem (e.g., for ext4, which does not support those on its own).
For integrity checking with basic checksums, one can
use find and sha256sum or similar
tools:
# Store checksums
mkdir checksums
find . -type f ! -path './checksums*' -exec sha256sum {} \; \
> checksums/sha256
# Check them
sha256sum --quiet --check checksums/sha256
# Add new ones
find . -type f -newer checksums/sha256 ! -path './checksums*' \
-exec sha256sum {} \; >> checksums/sha256Alternatively:
# Store (new) checksums
mkdir -p checksums
find . -type f ! -path './checksums*' -exec sha256sum {} \; \
> checksums/$(date -I).sha256
# Compare them to old ones
diff -U 0 <(sort checksums/2026-01-12.sha256) \
<(sort checksums/2026-01-23.sha256) | less
For redundant error correction codes (forward error correction,
FEC), with ability to repair, one may employ par2,
dvdisaster (aiming optical discs),
zfec (a library with Python, C, Haskell APIs),
libfec (a C library), GNU Radio FEC API, though those may be
quite inefficient to use for collections of files that are
updated. There are projects like blockyarchive (blkar), but just
as specialized backup systems, they tend to require specialized
tools to access the files backed up with them at all. A software
RAID (1, 5, or 6) set on different partitions of the same device
is a more time-efficient way to achieve some redandancy within a
storage device, though less space-efficient, and protecting
against different bit rot patterns. ZFS's "copies" parameter and
Btrfs's DUP profile (for both data and metadata) do something
similar, storing multiple copies of blocks within a dataset.
S.M.A.R.T. monitoring and testing can be done with smartmontools, and usually supported even by external and older USB drives.
I normally use just rsync --archive for the initial
backup, then rsync --exclude='lost+found' --archive
--verbose --checksum --dry-run --delete to compare
backups and for data scrubbing, and
without --dry-run afterwards, if everything looks
fine. Using -rt or -r instead
of -a may be preferable sometimes though, if file
permissions and ownership data are not to be preserved.
For data erasure, dd is handy for wiping both disks
and partitions (before decommissioning drives, or if there were
unencrypted partitions before), e.g.:
dd status=progress if=/dev/urandom of=/dev/sdX bs=1M dd status=progress if=/dev/urandom of=/dev/sdXY bs=1M
GnuPG is there for individual file encryption, as well as for signing. In some cases it may be useful together with tar and gzip.
For more compact music backups, one may wish to backup just the files referenced from a playlist, and not the whole archive. An example command for counting the total size of files involved in a playlist:
xmllint --xpath '//*[local-name()="location"]/text()' music.xspf |
sed -E 's/&/\&/g' |
tr '\n' '\0' |
du -s --files0-from=- |
awk '{ sum += $1 } END { print sum }'
While rsync has the --files-from option, to work
with a given list of files only:
xmllint --xpath '//*[local-name()="location"]/text()' music.xspf | sed -E 's/&/\&/g' | rsync --dry-run -avz --files-from=- . ~/mnt/
Public data may be useful to backup as well: its regular sources may be (and are, here) censored/blocked by a government, or simply become unavailable because of a technical issue (along with the rest of the Internet if the issue is near the user). In that case, the focus should be on high availability, probably along with integrity, while confidentiality matters less, unless there is a risk of those materials being outlawed.
As for the data to backup (and later read) this way, Kiwix (with its OpenZIM archives) is a nice project. Its primary viewer may seem awkward for use in normal circumstances, but apparently it aims to be useful to general public and in bad circumstances: it provides archives as packages, while the viewer—with versions for every common OS—can also serve those to others in a local network via a web browser. library.kiwix.org provides, among others, indexed archives of Project Gutenberg (about 75,000 public domain books by 2026), Wikipedia, Wikisource, Wikibooks, Wikiversity, Wiktionary, ready.gov, WikiHow, various StackExchange projects, Khan Academy, and many smaller bits like ArchWiki, RationalWiki, Explain XKCD (contains the comics). textfiles.com provides archives of files grouped by category, which are well-compressed, curious, and entertaining. RFC Editor bulk retrieval is both useful and particularly nice: available in different formats, as either a whole archive, partial archives, or via rsync. The POSIX (SUS) specification is useful to have at hand: POSIX.1-2024 is available as an archive (see "Downloads"). Along those lines, there are programming language specifications (reports), and other relevant specifications and references: ISO C, Haskell Language Report, Scheme Reports, Python documentation downloads, RISC-V specification, Intel 64 and IA-32 Architectures Software Developer's Manual, AMD64 Architecture Programmer's Manual, Linux Foundation Referenced Specifications, USB specifications, Bluetooth specifications, ACPI and UEFI specifications, PostgreSQL manual, XMPP Extension Protocols, etc. As of 2026, those would take just 400 to 500 GB, even with images and some non-English versions added. While much of programming documentation, particularly manuals, library references, and sources, is available from system repositories; more on operating system mirroring below.
Other large and legal archives to consider for backing
up: Wikimedia Downloads, Complete OSM Data, maybe software
archives (such as Linux distributions and their sources), arXiv
and other Open Access sources. If one gets into tape
storage, Common Crawl can be considered. For select website
downloads, I use wget --mirror --page-requisites
--convert-links --no-parent --continue --adjust-extension
https://example.com/~foo/, occasionally adding something
like --exclude-directories=photos,pictures or just
listing URLs manually (since it can be hard to separate heavy
bits of little interest from the others otherwise), and
sometimes having to add --compression=gzip if wget
gets confused otherwise, or --max-redirect=0 if
there are redirects to semi-blocked websites with freezing
connections (and while trying to download those directly, given
that wget does not support SOCKS proxies). But some websites
make archives available (as mine does,
see ../files/archive.tgz), or they are hosted at
GitHub/Codeberg/Tilde/etc "pages", making the archive available
for download (also as mine does,
see codeberg.org/defanor/pages). Some wiki-based websites also
provide data dumps, static HTML or database ones.
Then there are copyright-infringing but much larger libraries like Library Genesis (a trimmed down, txt-only version used to be available at offlineos.com, but apparently not anymore), the-eye.eu books, Anna's Archive, Z-library, as well as music and movies (particularly long TV series may be suitable for hoarding; out of nice sci-fi ones, there are Doctor Who, Star Trek, Red Dwarf, Farscape, Lexx, Firefly, Defiance, Battlestar Galactica, Babylon 5, The X-Files, First Wave; plenty more can be found in Wikipedia; for humorous ones, see Black Books, The IT Crowd, Taskmaster, plenty of sitcoms), music videos, audio books (including BBC radio collections). The Pirate Bay or similar torrent trackers may help to find book collections, including MIT mathematics and physics books, Cambridge Histories, Oxford "Very Short Introductions". As well as works grouped by an author (e.g., Gardner, Feynman). Other topics to consider acquisition of modern (text)books on: major philosophy works, electronics and radio, engineering, sociology, economics, computing, cooking, physical exercises, survival, fiction, medicine (e.g., the Merck manual), any topics of interest and other sciences. Other individual books on physics and mathematics, history. Consider the list of books complementary to Wikisource and PG. Then there are various literary awards and charts: Pulitzer, Nebula, Locus, Bentley, Booker, Nature's analysis of the 100 most cited papers, The Guardian's top 100 books of all time, The Guardian's 100 best novels written in English, The NYT's 100 Best Books of the 21st Century, and similar lists. Possibly UN and other organizations' reports.
One can reduce PDF size (compress the images) with GhostScript
or ImageMagick, among others, sometimes reducing the size by an
order of magnitude: see "Efficient PDF optimization with
Ghostscript CLI". For instance: gs -q -sDEVICE=pdfwrite
-dPDFSETTINGS=/screen -dCompatibilityLevel=1.4 -o out.pdf
in.pdf (possibly with -dCompressFonts=true
and other options). Its -dFirstPage=$START
-dLastPage=$END options are also handy sometimes, to
extract pages of interest (including cases when some
crackpottery is attached to books: that is one of the ways in
which the crackpots try to promote it). While EPUBs (basically
ZIP archives with HTML and images) can be compressed by
compressing individual images within those. Sometimes files can
be removed from an EPUB archive, and it can be trimmed down by
passing through pandoc (which would remove included fonts, for
instance).
As for operating system mirroring, one still needs some binaries to bootstrap when mirroring sources, while sources are particularly useful to backup for potential isolated usage, ensuring the ability to study and customize those. Some of the options to consider are (with size estimates from January of 2026):
debmirror, for amd64 trixie (13.3) with
sources. While the Mirror Size page lists numbers for
mirroring all suites. One may also consider usage of a caching
proxy server, apt-cacher-ng, and Modifying Debian
CD. Unlike most others, Debian repositories contain all the
source packages, which include upstream sources.Debian, in addition to being an all-around good system, seems to be a good option for such mirroring as well. The mirroring itself is done rather easily:
sudo apt install debmirror debian-keyring gpg --no-default-keyring --keyring trustedkeys.gpg --import /usr/share/keyrings/debian-archive-keyring.gpg gpg --list-keys --keyring trustedkeys.gpg debmirror -v -d trixie -a amd64 --source -h mirror.mephi.ru --method=rsync /mnt/backup/debian/mirror/
An up-to-date live Debian CD/USB image is useful to store along with it, and perhaps a Debian wiki dump. As well as necessary additional firmware for one's hardware, and possibly firmware for devices other than regular computers, such as OpenWRT images for routers, GrapheneOS or LineageOS images for phones and tablets (along with individual program distributions, APKs; some software I use is listed in the note on mobile computing), KOReader for e-readers. Consider F-Droid mirroring and OpenWRT source code saving, or backups of individual packages.
OpenStax provides good and freely available textbooks under the CC BY license, available for download in PDF. See OpenStax GitHub repositories for their CNXML sources and related tools, though in 2024 I found it tricky to build HTML out of those, and then it still was not good enough for printing. LibreTexts is supposed to be similar, though the licensing information is unclear in some cases, some links lead to HTTP 404 errors, and some of the books are quite messy (attempting to embed YouTube videos into PDFs, having every other page filled with listings of undeclared licenses, or with "welcome" messages). While its subdomains (math, phys, etc) geo-block direct requests from Russia, the books are available without proxying via commons.libretexts.org. One can also search for libre book sources on platforms like GitHub, possibly querying for TeX sources: there are occasional seemingly decent and not well-known textbooks, like Introductory Physics: Building Models to Describe Our World, An Infinitely Large Napkin.
Statistical ("ML", "AI") models for LLMs (llama.cpp) and speech recognition (whisper.cpp) may be useful to collect as well. LLMs in particular, while they do hallucinate, also contain plenty of information, and in a way that may make it easier to retrieve in some cases.
YouTube videos may be useful to hoard as well: there are many
nice ones, including educational channels, and platforms like
that seem to be getting blocked quickly when a government tries
to block information flows (see censorship of YouTube). At 480p
most videos would be watchable and not take much space (perhaps
2 to 5 MB per minute), and one can download them with youtube-dl
(yt-dlp now), e.g.: youtube-dl --download-archive
archive.txt -f
'bestvideo[height<=480]+bestaudio/best[height<=480]'
'https://www.youtube.com/c/3blue1brown/videos' (see
also: some tricks to avoid throttling). I have collected
some video links, including interesting YouTube channels. I
think it is best to go after relatively information-dense ones
(lectures, online lessons) first, possibly followed by
entertainment-education, pop-sci, and documentaries.
When backing up private data to a remote (and usually less
trusted) machine, it should be encrypted and verified
client-side (so options like plain rsync over SSH are not
suitable), but preferably still allowing for incremental backups
(so tar and gpg are not suitable in general, either). One can
still employ LUKS or ZFS though, by accessing remote block
devices via iSCSI (in particular, tgt
and open-iscsi seem to work smoothly on Debian),
NBD, or similar protocols, possibly on top of IPsec or WireGuard
(though as of 2024, those are blocked in Russia between local
and foreign machines), tunnels made with SSH port forwarding,
TLS (e.g., with stunnel), or anything else establishing a secure
channel, to add encryption and a more secure authentication.
A test iSCSI setup example:
# server (192.168.1.2) apt install tgt dd if=/dev/zero of=/tmp/iscsi.disk bs=1M count=128 tgtadm --lld iscsi --op new --mode target --tid 1 --targetname iqn:2024-07:com.example:tmp-iscsi.disk tgtadm --lld iscsi --op show --mode target tgtadm --lld iscsi --op new --mode logicalunit --tid 1 --lun 1 -b /tmp/iscsi.disk tgtadm --lld iscsi --op new --mode account --user foo --password bar tgtadm --lld iscsi --op show --mode account tgtadm --lld iscsi --op bind --mode target --tid 1 --initiator-address 192.168.1.3 --initiator-name foo tgtadm --lld iscsi --op unbind --mode target --tid 1 --initiator-address 192.168.1.3 --initiator-name foo tgtadm --lld iscsi --op bind --mode target --tid 1 --initiator-address 192.168.1.3 # client (192.168.1.3) apt install open-iscsi lsscsi iscsiadm --mode discovery --type sendtargets --portal 192.168.1.2 iscsiadm --mode node --targetname iqn:2024-07:com.example:tmp-iscsi.disk --portal 192.168.1.2 --login iscsiadm --mode session --print=1 lsscsi # a block device is available at this point iscsiadm --mode node --targetname iqn:2024-07:com.example:tmp-iscsi.disk --portal 192.168.1.2 --logout
Apart from own (or rented) remote machines, such a setup can be used with "backup buddies", exchanging some of your local storage space for someone else's. Sneakernet-based backup buddies (that is, occasionally exchanging storage devices) is a fine and easier option for remote backup storage.
A popular option for remote backups is online services (aka "the cloud" and a few other names), with many people relying on those even in place of local backups, or any local storage (as with music and video streaming, hosted photo albums, password managers, book collections, general document storage), delegating all those worries to somebody else. It seems convenient, but decreases direct control over the data, introduces dependencies on the service providers' continued existence and continued acceptable terms of service, on network connectivity to them, on ability to transfer payments. In my--possibly unrepresentative--experience, all those are unreliable, but it may still work as a redundant backup copy for some, particularly in predictable democratic countries, with a reputable service provider. Throw in the rule of law and sensible laws (or some kind of a hypothetical anarchist or communist utopia), and one may worry less about keeping some information private, as well as about aiming long-term isolated backups of public information.
For less private data (perhaps for almost everything but cryptographic keys and passwords -- that is, explicit secrets), a good way to preserve it is by sharing with others: for instance, pictures from an event or gathering are commonly shared among all the participants, while creative works (particularly books and music) can be shared among people with similar interests or tastes. Everything work-related can be backed up on work machines. While the data that is not private at all, like this very note, or other own creative works under permissive licenses, is generally useful to publish, sharing even more widely.
One may consider use of relatively adverse services for both storage and transfer, such as censored and monitored ones. Usually they are best to avoid, but they may still be useful for redundancy, or when there are no other working options.
For file storage or sharing services, storage on block devices,
as described above, can be handled with a file-backed loop
device. GnuPG and other file- or stream-oriented methods would
also work, but since encrypted data may attract unwanted
attention, it is better at least to not advertise the encryption
with headers. An easy option to do that (using a passphrase
without additional files, the widely available openssl CLI tool)
with a single file or stream is openssl enc -aes-256-ctr
-nosalt -pbkdf2, but PBKDF2 is relatively weak (argon2id
is recommended), and this skips a salt entirely, so the
passphrase must be high-entropy then. Alternatively, one may
consider using and storing the salt, and employing argon2
manually, e.g.:
sudo apt install openssl argon2 SALT=$(openssl rand -hex 16) # 128 bits # https://www.rfc-editor.org/rfc/rfc9106#name-parameter-choice # The first recommended option argon2 $SALT -id -t 1 -p 4 -m 21 -l 32 # The second recommended option argon2 $SALT -id -t 3 -p 4 -m 16 -l 32 # Use -e or -r options for scripting
Another option is cryptsetup (dm-crypt) without LUKS: in its
plain mode, on a loop device, with some basic non-journaling
file system (such as ext2, or ext4 with journal disabled) on
it. The passphrase must also be high-entropy, since it does not
use a KDF: ideally matching the 128-bit key size, which would
take 8 random words picked out of a dictionary of 65536
words. Or a separately derived (or generated, simply random) key
must be supplied, perhaps with the --key-file
option. An example:
dd if=/dev/urandom of=test.img bs=1M count=128 sudo losetup --find --show test.img sudo cryptsetup open --type plain /dev/loop0 test # No journal, no reservation mke2fs -t ext4 -O ^has_journal -m 0 /dev/mapper/test mkdir test sudo mount /dev/mapper/test ./test # Add the files here sudo umount ./test sudo cryptsetup close test sudo losetup -d /dev/loop0
To make use of audio channels or audio data storage services
(including the ones that re-encode audio files), a
straightforward way is to use modem software, such
as minimodem. That can be combined with forward
error correction for reliability, and mixed with another audio
stream to stay more covert. One may also try setting a system in
the style of numbers stations, using TTS (text-to-speech,
festival or espeak) and STT (speech-to-text, CMU Sphinx or
more advanced ones).
For data encoding as text, one can use plain base64, some words-based encoding, maybe Markov chains (which would require custom tools and data though).
For video, a similarly ad hoc and basic approach is to encode a sequence of QR codes (or other matrix barcodes). But as for other options, one may also consider more involved steganography.
HMAC can be useful for authenticated integrity checks with such
services. Common CLI tools for that include
hmac256(1) from the libgcrypt20-dev
Debian package
and openssl-dgst(1ssl): openssl dgst -sha256
-hmac <key> [file ...].
I would not recommend to rely on online services generally, but using them for added redundancy, particularly for public data, may be okay, and potentially fun to play with. If arbitrary and random-looking data storage is not explicitly allowed by a service, it may lead to account suspension, including other services on that account (though it occasionally happens with online services even without such a trigger).
Here is my log of spotted and reported network abuse incidents. It started as private notes aiming to keep track of those being fixed, and to block the hosts if they keep spamming. I decided to make it public, since there is no private information in it (though I'm omitting the bits I may discover that aren't public, such as server administrator email addresses), and it may be of interest for people trying to decide whether reporting is worthwhile.
Below are incidents with spam messages that got through the usual filters: dates, hosts, the abuse contact and other report information, other notes.
postscreen_access.cidr.sendcloud.io REJECT
spammers into the file referenced by
postfix's check_client_access. dnswl.org returned
127.0.15.0 for it, reported it to them as spam.postscreen_access.cidr.postscreen_access.cidr./e-bike/i REJECT E-bike spam into
postfix's body_checks./e-?bike/ REJECT E-bike
spam (the i flag actually turns
case-insensitivity off), blacklisted 179.61.221.0/24
in postscreen_access.cidr.postscreen_access.cidr.A lot of network abuse (spam, vulnerability scans, brute-force attacks) comes from China, plenty from Russia as well. As a side note, Chinese researchers similarly spam the world with fabricated research papers (though apparently they try to combat it, up to a death penalty for researchers who commit fraud if it harms people). Apparently wider agreements, policies, and cultures help to fight network abuse about as well as technological methods do. I think it is okay to rate-limit regional IP address blocks (as described in the private server setup and simpler server setup notes), though one should think twice before blocking them completely: if there are non-abusive users, it would be unfair to them. And then there are large mail providers, particularly Gmail, not caring much about outgoing spam, while blocking them is a bad option, given the number of legitimate users: the ham-to-spam ratio is less than 1, but more than 0.
There are information security guides for different audiences around, including EFF's Surveillance Self-Defense and Email Self-Defense, NIST's Cybersecurity. But I fail to find concise, relatively general, and sensible guidelines aiming personal information security and information literacy to refer people to, so I wrote down the suggestions I would normally share.
I am not a security expert, but a programmer and a small-scale system administrator paying attention to security. So it is a good idea to consider these suggestions critically, just as any others, but I think that they will improve the average state of such guides.
Question things, do not trust blindly, require evidence and verifiability of claims, check those, do not share personal information or give away control without a good reason to, assume that "anything that can go wrong will go wrong" (Murphy's law). That is, employ scientific and engineering approaches, and try to stay honest: do not nudge things to look better (e.g., more trustworthy or certain) than they are; better to err on the side of safety, assuming that they may be worse than they seem. A lack of understanding makes one vulnerable to deception, so study the relevant subjects: how computers, banks, online stores, governments and scammers work, how software and relevant systems are developed, how the research used by those is done. Computing context is a part of it. Try to avoid fallacies and cognitive biases, as they tend to be exploited by adversaries.
Do not shy away from learning. It is tempting (and commonly suggested) to stick to certain newbie-friendly tools, but that is a very fragile approach: without sufficient understanding, people easily lose the tools (e.g., when the government blocks their secure messengers), or manage to misuse them (e.g., by ascribing strange and unexpected properties to the tools: no user-friendly UI or API will protect from a user assuming that anything going through a system becomes "secure" in all senses and for all purposes, for instance).
Conversely, when providing a service, publishing software, asking for information, sharing information or software, it is nice to make it easy for others to follow that: provide references, evidence, source code, explain why the requested information is required (and ensure that it actually is required); generally, do not ask to believe or trust blindly, do not encourage and normalize dangerous practices.
And as with any other pursuit, give it a try, do not give up, do not view it as "all or nothing": learning a little, paying some attention to security, and avoiding some of the potential losses that way is already better than being successfully attacked all the time.
Information security includes a few areas, but personal security usually revolves around privacy and confidentiality. Some of the common threat actors targeting individuals are scammers, oppressive governments, and thrill seekers. All those seem to be commonly underestimated: scammers' victims think that they cannot be scammed, and are surprised afterwards; thrill seekers are often neglected because "why would anybody want to do that?"; governments are often ignored because of one's political views (loyalty to the regime, beliefs that it will not turn authoritarian, is not authoritarian even after it turned so, abandoning presidential term limits, introducing numerous censorship laws and persecution of dissent, and so on; belief that they will not reach you) or learned helplessness. "I have nothing to hide" is another common sentiment, often extended to the private information of one's friends and family that they possess, useful to threat actors. That usually implies a certainty that the government is on your side and will stay that way, in addition to one's immunity to the other risks. And then there are the likes of "the world is just, I am good, so nothing bad can happen to me"; a variety of denial strategies and excuses, religious beliefs.
Entities collecting information, even if they do not use it against you intentionally and immediately, may also be viewed as threats, since they tend to leak it via data breaches, or to abuse it themselves later. Those include commercial companies, government organizations, and individuals.
People may also engage in a crime of opportunity if the conditions for that are created: e.g., someone picking up or buying a discarded unencrypted storage device may access (recover) the private data stored on it. Same with information made available online: apparently even IT professionals manage to accidentally allow unauthenticated access to databases quite regularly, making it a common source of data breaches.
The principle of least privilege is generally useful: share the minimum required information to receive a service, or give minimal required and controlled access to your system. E.g., buying most items, using most public transport, or visiting most public places should not require identifying yourself: doing so imposes an unnecessary risk. Likewise with running custom software to access online services, especially if it is closed-source (and possibly proprietary), so you cannot (and possibly not allowed to by the license) check what it is doing with your system. Communicating over the Internet does not require to provide your full name, phone number, or to identify yourself at all. Identifying yourself by sending pictures of documents is one of the sillier and dangerous practices. Software should not run with superuser (root) privileges, and generally the usual security mechanisms must not be bypassed, unless there is a good reason to.
If someone asks you to take unnecessary risks like that, that
itself is a cause for suspicion, and to look for other
options. Often it involves accepting inconveniences (such as
visiting places and standing in queues instead of using
proprietary software, dealing with paper documents, possibly
with cash, missing some online conversations), resisting peer
pressure (e.g., "just set a sensible password like 1234",
"install our software with curl | sh and run its
custom updater to be up to date", "let's run everything as root
to avoid dealing with permissions").
If the private information is not requested by a service, or superuser privileges are not requested by software, it is safest to not volunteer to provide those: e.g., use screen names for online services and as a system user name (which is used as the default name for information sent online occasionally: the best way to ensure that the real name is not leaked accidentally is to never enter it), use dedicated system users or sandboxing facilities to run programs.
Cryptography provides useful tools, perhaps encryption being the most notable one, useful for personal data storage (including encrypted backups), as well as for communication (over email or instant messengers, such as XMPP), and for channel security (for network connections). Another common use of cryptography is for data integrity checks.
Following general advice given above, one should look for trustworthy (transparent, verifiable, openly developed) tools, ideally using free and open-source software exclusively, retrieving it from trusted sources (such as operating system's repositories, where the packages are signed), preferably checking the code, but at least preferring the tools used and inspected by many.
I personally use mostly LUKS for disk encryption and OpenPGP for file and mail encryption and signing, on a Debian system. And TLS, SSH, IPsec, Wireguard for channel security. Those are widely available, well-known tools.
The usage of LUKS with cryptsetup(1) is described
in the personal data storage notes linked above, while that of
OpenPGP is described in GnuPG's user guides; it is supporetd out
of the box in mail clients such as mu4e (an Emacs client), mutt
(a standalone TUI client), Thunderbird (a standalone GUI
client), and the GnuPG's gpg(1) command-line tool
is fairly easy to use. For email, one may want to ensure that
the messages are encrypted not just for recipients, but also for
the sender, so that the sender can read them later: mutt does it
by default (the pgp_self_encrypt option), for mu4e
one should enable it
in mml-secure-openpgp-encrypt-to-self.
There are endless alternatives, which tend to incorporate newest and shiniest algorithms (which is dangerous by itself: better to stick to heavily analyzed ones), to be written in this month's most trendy language (possibly to be abandoned soon), clean of the backwards--or standards--compatibility cruft accumulated by older tools, and supposedly easier to use, providing fun colors and supportive emojis. Some also like to write their own software, but there are many gotchas and cryptographic attacks that basic algorithm descriptions do not mention, which may easily compromise the system. Both scammers and governments like to advertise malware as security software, occasionally to disguise attacks as security measures. While more legitimate commercial companies tend to sell virtually useless security products, but not necessarily malware: perhaps more of placebo. Security theater is a shady practice along those lines.
OpenPGP is criticized quite persisently, and it is indeed imperfect, as even its name points out: merely "pretty good". But as with other things, the "best" kind, as judged for a particular situation, is often that which is actually used at all, while OpenPGP usually beats the proposed alternatives in its applicability and (continued) availability, and in many cases its issues are irrelevant. There is a room for improvement though. For an alternative OpenPGP implementation, see Sequoia-PGP. Out of standalone (but incompatible with OpenPGP) encryption and signing alternatives, age and Minisign are somewhat prominent. While the OpenSSL CLI tool is more widely available and versatile. And then there are OTR, OMEMO, and MLS for IMs specifically. But I think it can be quite a rabbit hole, while GnuPG is versatile and good enough for most tasks, so at least it is worthwhile to look into first.
There are minor tactics and useful habits, some of which can be described as simply common sense:
xkcdpass), do not reuse those across
services, maybe do not reuse logins and other identifying
information, either. That may include things like the IP
address, web browser fingerprints, and so on.
Ensuring secure practices can be interesting and fun, and one may be enthusiastic about it, which helps to follow them. Then it is tempting to share that with others, improve their security practices, which is what I am trying to do by writing this. But keep in mind that people may simply not care about it, as many do not care about their health enough to take care of it, of environment (ecology, as well as politics), of self-improvement, and of a variety of other topics that yet others do care about. Even among those who do care about information security, the threat models and views on ways to achieve it may differ considerably, also as with the other mentioned topics. And it can be difficult to idly observe people you care about doing what you think is bad for them. I think a fine balance between being unhelpful and annoying is to let people know that you are willing to help, to answer and explain things when asked to, but not to try to force those onto others. And maybe to work on useful tools, infrastructure, and documentation in order to satisfy the impulses to share and help, as well as to learn more in the process.
The same principles apply to information security in organizations, when setting company's servers or developing enterprise software. Just as with software and hardware generally. There may be more bureaucratic approaches (with occasional checklists for compliance checks), scales are different, NIST's frameworks are more useful there, but it is basically the same thing.
Mobile computing can be a pain, especially when done in uncomfortable positions, on downsized and/or underpowered hardware, possibly in a noisy environment and while being distracted. Unsuitable conditions can also make it much harder to focus on computing-related activities. Yet a mobile computer is often better than nothing, and a comfortable workplace is not always available exactly where you want or need it to be.
These are my notes on dealing with mobile computers over time: mostly the software for underpowered computers with poor input and output capabilities, focusing on Linux-based systems.
I've been stuck with an old netbook (Intel Atom, 1 GB of main memory) for a couple of weeks, so wrote down some of the things I've learned. That's on Debian stable (Stretch was just released; using it with "non-free" repositories to get GNU documentation), with i3 window manager, and using Emacs for most of the tasks.
Wi-Fi is one of the most important things to set. This time,
both
wpa_cli and wicd claimed that the password is
wrong, but nmtui (NetworkManager TUI) has connected just
fine – though maybe it has messed up some settings for others somehow.
Wicd was hogging resources even while not doing anything useful, as Python
programs tend to do, so I've disabled it – it rarely worked
anyway. wpa_supplicant writes log messages such as "result=4"
and doesn't document those codes in its man page, requiring source code to
see what's going on. And NetworkManager just repeats those.
Firefox just starts for 30-40 seconds, and then lags even without JS. I gave up on it, and switched to w3m (emacs-w3m); web services such as online banking don't work with it, but it is keyboard-friendly, generally works, and does not lag too much. To use DDG for search, one should customize w3m-search-default-engine.
As for maps, there is FoxtrotGPS – an OpenStreetMap client that can cache and pre-download maps. It's pretty lightweight and usable.
For video playback, VLC appears to be more reliable than mplayer, even though has its issues (including bloating, lack of documentation, and resource hogging even while idle). Unfortunately, many videos are not available via bittorrent, being only hosted on youtube.com or similar websites; youtube-dl works to extract those.
One of the painful tasks to perform without a mouse is to copy and paste things between a terminal emulator and other programs (such as GUI Emacs). Actually it's somewhat awkward with a mouse, but even worse without it. Well, Emacs-to-terminal-emulator is easy: there are M-w to copy from Emacs and shift + insert to paste into a TE. Copying from a TE can be done by selecting with a touchpad, and then M-: (mouse-yank-primary (point)) RET in Emacs, though it won't work to insert into a TE; but turns out that one can emulate the middle mouse button by pressing the two touchpad keys simultaneously. It's not great, but works; perhaps a nicer way is to use a terminal multiplexer functionality for that, though then one may have to use nested terminal multiplexers, if they are also using those remotely. Or one could use an Emacs TE instead of a separate one, but that could also get awkward.
Speaking of terminal multiplexers: even though normally I'm not
using tmux, it is more useful to run remotely with
an unstable connection: a remote persistent session partially
compensates for the lack of a persistent connection and/or local
session.
Doing Haskell programming would be a pain on a netbook because
the REPL and cabal would require too much of
resources, so I've planned to use a remote server for that: just
run both Emacs and a REPL process there. Didn't have to do that
in those two weeks though.
xpdf, mupdf, and zathura are relatively lightweight and portable PDF viewers. Xpdf has ugly GUI buttons and a mostly useless left pane that takes space, others use partially qwerty-oriented (vi-style) key bindings (while I'm using Colemak), and the scrolling is quite messy in both mupdf and zathura (in mupdf, there's no way to tell whether you're at the end of a page or not, but scrolling by a little amount would jump a page if you're at the end; zathura may skip a line when scrolling with spacebar). Both xpdf and mupdf allow to adjust colors, zathura doesn't. So I've used both mupdf and zathura, but then discovered Emacs pdf-tools; didn't try it on a netbook, but it works nicely on a desktop: the colors are adjustable, keyboard-friendly, no notable issues like those with scrolling in others.
Bittorrent clients are not so nice to set and use: both rTorrent and
Transmission (transmission-daemon with transmission-cli) have broken Emacs
interfaces, which I gave up on after brief attempts to debug, since using
a netbook doesn't make debugging more fun. Transmission is nicer in that
it uses a daemon, which is more suitable for a program like that. To
simplify authentication, one should either use netrc
(.authinfo.gpg), or disable authentication and only allow
local connections:
"rpc-authentication-required": false, "rpc-bind-address": "127.0.0.1",
Then it's not so bad to control
with transmission-remote: -a and
-w options to add a torrent and write files into a
specified path, -l to list tasks, etc. The
Transmission IRC channel (#transmission at Freenode/Libera.chat)
is quite helpful, and minor bugs get fixed quickly there.
The situation with music players is pretty similar. I've tried
mpd multiple times before, and it never worked, but worked this
time (well, after mpc update); mpc is usable to
control it, even if not that fancy (i.e., plain CLI). There are
some Emacs packages: emms supports mpd, but tries
to handle all kinds of players, so the support is not so
great; bongo seems to have nicer UI, but doesn't
support mpd at all; mingus appears to work, but it
refreshes its whole buffer all the time, resulting in annoying
blinking and rendering it unusable. And there
is ncmpc, which is fine;
though ncmpc-lyrics has a lot of dependencies,
including Ruby. Music playback seems to be one of the most CPU
intensive tasks in a system with relatively little bloat.
The rest of my regular software is keyboard-oriented and lightweight: mu4e with mbsync for mail, circe/erc/rcirc for IRC, bitlbee and circe (later rexmpp) for XMPP, org-mode for notes and things like that, and other Emacs-based and CLI/TUI tools.
Later, in 2023, I have installed Debian 12.2 with Xfce on it. It takes almost 600 MB of main memory, leaving 400 for work. But by 2025, even Debian 13 dropped support for 32-bit systems.
During the unfortunate events in Russia in the early 2022, I decided to finally get a tablet computer while they are still available here and while I can afford one. At first I've looked into ones supported by LineageOS, but those were rather old ones, so I went for a model that is newer, and possibly can be supported later -- Samsung Galaxy Tab A8. I don't have much to compare it to (only used one Android phone out of similar devices, and just as a phone, for calls), but it appears to work and to be a tablet.
Samsung groups the awkward software required to be installed by the local government into the "law" group, so it's easy to remove it all at once. Avoiding Google and Samsung account creation, and aiming its usage as both a general household appliance (maybe for use in the kitchen, to read in bed, etc) and a useful device in an isolated wasteland if/when desktop computers will break and have no replacement, I've set F-Droid by downloading its APK, and then installed most of the software from it (though occasionally with APKs from their official websites too): OsmAnd for maps (including offline ones, from OSM); KOReader (as I use on an e-ink reader), Librera, OpenDocument Reader, and Kiwix to read things; VLC as a music and video player; Fennec (a Firefox version available from F-Droid); Sketches for basic sketching; Notes for note taking; a couple of fancier calculators with graphing; Conversations as an XMPP client; the Wikipedia client out of curiosity, but it turned out to be handy. Also Synthesia to try it out with a MIDI keyboard, which mostly worked, but that's proprietary. Termux provides plenty of regular GNU/Linux system functionality, including Emacs in its repositories.
I hear that ThinkPad (IBM originally, Lenovo now) laptops are nice for Linux, but they are expensive; Dell and Lenovo ones are commonly suggested for Linux-based systems too. Lenovo IdeaPad seem to be Linux-compatible, but cheaper than ThinkPad, with less advanced I/O (targeting consumers, not businesses). Here is one of the articles on the topic, linking more: On modern laptop requirements.
Issues with Wi-Fi hardware support are common; see Existing Linux Wireless drivers, ensure that there are drivers for a given laptop's hardware. Linux Hardware Database is another potentially helpful database.
One can also look into fwupd's vendor list to estimate Linux driver support from vendors, or perhaps the Linux on Laptops website, and other erlevant websites linked from the linuxhardware subreddit.
I've picked a relatively inexpensive Dell Vostro 3515, which seems suitable for non-gaming tasks and inexpensive: a 15.6-inch display, plastic, no discrete graphics card, Ryzen 5 3450U and 8 GB of main memory (2 of those are used as video memory, leaving about 6 for the rest of the system), 512 GB SSD, and a 8P8C/Ethernet port (many laptops don't have those anymore), in addition to the common set of I/O ports.
To boot from an USB stick with a Debian 11 installer, I tried to
add it in the boot options in the UEFI menu, but that was rather
confusing: it asked to choose an exact .efi file,
and then failed with a "Something has gone seriously wrong:
shim_init() failed" message. Apparently that's common on
laptops, with different Linux distributions and laptop vendors,
but I haven't found descriptions of any working solutions,
except for installing an older version first. What worked for me
is just to choose a different .efi file, and then
hold F12 during the boot to enter a boot menu, selecting the USB
stick from it.
I'm always uncertain about the size of a boot partition (and
sometimes about that of the ESP partition too), and how exactly
to set encryption (e.g., apparently one can encrypt even the
boot partition while using grub, but it doesn't seem that
useful, and would lead to double password prompts). And about
the swap partition too: usually just disabling it, but perhaps
it's more useful on a laptop, and it's commonly suggested to
use. I've settled on about 500 MB for ESP
(/boot/efi), 500 MB for /boot,
encrypted swap and ext4 root partition (/), without
a separate /home. Then tried Debian's guided
partitoning, and it did exactly that (after selecting use of
encryption and of a single partition), so I just went with
it. Though as of 2024, some recommend 1 GB or 2 GB
for /boot, with Ubuntu apparently defaulting to
almost 2 GB, and it is likely to be a pain to change later in
such a setup, without reinstalling everything. After updating to
Debian 13, which suggests at least 768 MB for /boot, I recreated
those, reducing EFI to 200 MB, and increasing the boot partition
to 800 MB.
In this case it was a Debian Xfce Live version, with non-free software and documentation (just as for the Debian 11 workstation). It is nice and almost everything works well out of the box, though DPI tends to be wrong on laptops: it is 96 by default, while laptop screens have something closer to 144. That can be adjusted in the "Appearance" settings, the "Fonts" tab. I have also adjusted the touchpad behaviour.
In 2023, after hardly any use, the laptop ceased to charge the
battery (it is on the "pending-charge" status all the time, even
at 0% charge, with any UEFI charging settings), unclear why. I
have not found a way to fix it so far. Also attempts to update
the UEFI/BIOS firmware via "BIOS flash update" lead to an
"invalid file" error. Some suggest to run it from FreeDOS, but
it relies on BIOS, and the laptop appears to only support UEFI
boot. Another option is Windows (possibly the live and
lightweight version, Windows PE), though microsoft.com bans
Russian addresses from downloading it, and bans hoster addresses
where proxies are hosted as well, as of 2023 (while dell.com
also refuses to serve requests from Russian addresses, but
proxies work with it). Plenty of images on The Pirate Bay (which
is blocked in Russia, but at least does not refuse to serve
requests coming from non-residential addresses, so proxies work)
though. I managed to install Windows ADK on a Windows 10
machine, then to prepare a Windows PE USB stick from it. Had to
add firmware files into the "media" directory (actually added
into a few locations, initially failing to find any), then to
run diskpart.exe and its rescan
command to find the firmware (I think it was on disk C). The
firmware complained that "The AC adapter and battery must be
plugged in before the system bios can be flashed", had to run it
with /forceit option. Then it seemed to be working,
but got stuck on "update progress: completed". I ended up
resetting the laptop, then it complained that "battery pack is
removed or less than 10%". I turned it off, unplugged the cable,
plugged it back again, and the charging LED finally stayed
on. Waited for half an hour, turned it on, it ran the BIOS
(UEFI) and EC update process again, but then rebooted itself. It
forgot where the boot media is, I pointed it manually to a
Debian's .efi file again. Then it booted and was
charging. Better to look for laptops with a sane firmware update
process.
With this laptop, I have also experienced odd touchpad issues, which unfortunately seem quite common: in this case, it ceases to move the cursor after a seemingly random time after the boot, though clicking works, and it is fine again after a reboot. Sounds similar to the "Touchpad stops working after a while" issue, but there is no touchpad mode setting in this laptop's BIOS/UEFI settings. Later noticed that Bluetooth does not work well, either, at least with a Bluetooth speaker: there are occasional audible interrupts, and a stream of kernel module error messages in the logs.
I acquired a Google Pixel 6a (not exported here officially, so
without a warranty, and no spare parts available; but at least
not certified in Russia, so no mandatory malware installed on
it), which has a plain Android system, and is supported by most
of the alternative Android distributions. The software to set on
it is similar to that on a tablet: F-Droid (with Guardian
Project repositories), then Conversations, ConnectBot, OsmAnd+
(with pale road style, 150% text size), Compass
(com.bobek.compass), Wikipedia, VLC, Fennec (+ uBlock Origin,
noscript, HTTPS everywhere), Tor Browser (with a bridge set
manually), Notes, Librera, Yaaic, Termux (with Emacs on it, as
well as openssh and rsync, and allowing it access to storage, so
that pictures and other files can be transferred over SSH with
rsync: for instance, to synchronize the pictures -- rsync
-av -e 'ssh -p 8022' --exclude='.trashed-*'
user@host:storage/dcim/OpenCamera/
~/Pictures/OpenCamera/; but by 2025 it ceased to work,
since Android increasingly locks everything down). Later I added
strongSwan and WG Tunnel (to connect to a home network as a
"road warrior") and baresip (though still mostly using
Conversations for calls), Just Another Workout Timer, Open
Camera, WiFiAnalyzer, Kiwix, Aegis Authenticator (hOTP/TOTP),
Orgzly (an org-mode viewer/editor), Material Files (a file
manager with WebDAV, FTP, SFTP, SMB support), a couple of games
(Shattered Pixel Dungeon, Mindustry), K-9 Mail, Briar.
The camera on this phone appears to produce rather bleak (washed out, desaturated) pictures, which is particularly apparent after enabling raw (DNG) picture writing. There are multiple ways to saturate it in darktable, but "tone curve" with independent CIELAB channels in particular is handy and versatile; the "denoise" module then helps to get rid of the produced noise. Perhaps one may also change the input color profile: colors look almost fine with sRGB instead of the embedded one; apparently it is a common problem with Pixel phones, see "Colours washed out from Pixel 7 DNG".
Lenovo IdeaPad Slim 3 16AHP10 looks like a fine option: the I/O
is not as good as on ThinkPad or IdeaBook ones, but it is
inexpensive, has a power-efficient CPU, okay specifications, and
Debian runs well on it. I have set it to "battery saving mode"
in the UEFI settings, booted from a Debian live Xfce USB stick,
partitioned its 512 GB disk on installation (using Debian's
regular installer, not the GUI Calamares one) as follows: 1 GB
for ESP, 1 GB for /boot, then LUKS with LVM on top:
80 GiB for the root file system, the rest for home; used ext4
this time, with noatime,nodiratime mount options
(see SSDOptimization in Debian Wiki). Then have set a more
suitable DPI (142 for a 16-inch screen with 1920 by 1200
resolution; see also: Arch Wiki HIDPI, Debian Wiki
MonitorDPI, Arch Wiki LightDM HIDPI) and larger fonts (12) in
Xfce settings, as well as xft-dpi=142
in /etc/lightdm/lightdm-gtk-greeter.conf,
ran sudo dpkg-reconfigure console-setup to set a
larger tty font size (DejaVu, 16x30),
added /usr/bin/setxkbmap -option "ctrl:nocaps" into
Xfce startup commands (another option is to set it
via /etc/default/keyboard,
e.g.: XKBOPTIONS="ctrl:nocaps,grp:shifts_toggle",
XKBLAYOUT="us,us,ru", XKBVARIANT="colemak,,"),
have set locale to C.UTF-8
in /etc/locale.conf, disabled the loud PC speaker
with
sudo rmmod pcspkr && echo 'blacklist pcspkr' |
sudo tee /etc/modprobe.d/nobeep.conf, set additional DNS
servers (74.82.42.42, 208.67.222.222, 8.8.8.8) for
NetworkManager, installed a few Firefox extensions (uBlock
Origin, noscript, FoxyProxy), configured input methods and
touchpad behavior in Xfce settings, configured some of its
panels, generated an SSH key with ssh-keygen, added
it to the agent with ssh-add ~/.ssh/id_ed25519,
disabled menu access keys in Xfce terminal preferences (so that
it does not intercept shortcuts like M-f), removed some of the
unnecessary packages, installed useful ones, configured a little
more:
sudo apt install task-laptop task-english smartmontools dkms
sudo apt remove live-task-localisation live-task-localisation-desktop
sudo apt autoremove
sudo apt remove 'hunspell*'
sudo -e /etc/apt/sources.list # Add "contrib non-free non-free-firmware"
sudo apt update
sudo apt install systemd-timesyncd openssh-server rsync emacs mu4e isync git \
elpa-{magit,haskell-mode,nov} ghc cabal-install texinfo mtr-tiny \
mpv vlc telnet xsltproc clementine lynx mutt irssi whois nmap ncat dnsutils \
knot-dnsutils tmux fbreader inkscape gimp lmms musescore libxml2-utils \
xkcdpass wireguard tinc tor obfs4proxy shadowsocks-libev kiwix kiwix-tools \
autoconf autoconf-doc libtool pkgconf libexpat1-dev libgsasl-dev \
libssl-dev libcurl4-openssl-dev build-essential dino-im goldendict \
dict-freedict-{deu-eng,fra-eng,lat-eng,eng-rus,deu-rus,fra-rus,eng-deu} \
dict-gcide transmission pandoc audacity festival sox postgresql \
nginx libnginx-mod-http-dav-ext aptitude jmtpfs emacs-common-non-dfsg \
texlive texlive-plain-generic texlive-xetex texlive-lang-cyrillic \
blueman sqlite3 libsqlite3-dev gcc gcc-doc glibc-doc-reference \
python3-{sympy,scipy,numpy,matplotlib,psycopg,doc} \
info guile-3.0 guile-3.0-doc oathtool iotop
# ... darktable blender librecad freecad kicad evince
# prosody coturn uacme inspircd mumble-server mumble qemu-system
# libvirt-clients libvirt-daemon-system virtinst dnsmasq-base bridge-utils
# debian-reference-en debian-kernel-handbook linux-doc user-mode-linux-doc
sudo -e /etc/ssh/sshd_config # "PasswordAuthentication no"
# Disable some services: going to run them manually, as needed.
sudo systemctl disable --now tor tinc shadowsocks-libev nginx postgresql \
bluetooth
killall pulseaudio # restart to load pulseaudio-module-bluetooth
# (for PipeWire, use libspa-0.2-bluetooth instead)
# Optionally, upload hardware information to the linux hardware database
sudo hw-probe --all --upload
# Enable battery conservation mode (remembered between boots),
# so it does not charge above 80%.
# Also can be done with TLP, STOP_CHARGE_THRESH_BAT0=1.
echo 1 | sudo tee /sys/bus/platform/drivers/ideapad_acpi/VPC2004:00/conservation_modeThen it was left to copy personal files (dotfiles, documents, books, music, etc) onto it, and configure things further, but this is a basic initial setup.
One may try to select "Advanced install options", "Text installer", "Expert Install" in the installer, so that there will be an option to install a "normal" system instead of "live", but it still installs some of those "live-task" packages.
Though the hardware seems to work well generally, eventually I
noticed an I/O error during reading from its SSD, reporting a
timeout and a controller reset, similar to "Ubuntu 24.04 freezes
with "nvme nvme0: I/O" timeout error" or "NVME timeout woes";
have not tried adding pcie_aspm=off or
nvme_core.default_ps_max_latency_us=100
nvme_core.io_timeout=3000 into /etc/default/grub myself,
yet, but probably will try it, if it will keep
happening. Apparently those things happen on some laptops. While
Wi-Fi, Bluetooth, touchpad, and SD card reader work smoothly.
I have also set Debian 13 and Windows 11 on another 16AHP10 laptop (not for myself), and surprisingly, while Wi-Fi worked on Debian out of the box, it required to manually install drivers on Windows.
Kobo devices are supported by Koreader, among a few others. Apparently both Kobo and reMarkable are suitable for running Linux and custom software on them; see "E-ink is so Retropunk".
Some devices, particularly infrequently used and mostly stationary ones, such as radio receivers, may be awkward to power: batteries may be wasteful for stationary ones, and inconvenient to keep in a usable state for infrequently used devices, while inbuilt cheap AC-to-DC converters are unreliable and occasionally humming, and mimicking batteries with external AC-to-DC converters is tricky (they tend to provide higher voltages than 1.5 V of common batteries, and connecting those snugly would be tricky). A good option is to pick devices relying on external AC-to-DC converters, as laptops, phones, and e-readers do: it is usable for both direct usage and battery recharging, and battery chargers can be very simple, not adding complexity; many devices use USB for DC power input these days.
By 2026, there are laptops supporting USB-C for power input.
I quite like email: perhaps not so much because of its design or technical qualities, but because nice tools exist and there are plenty of users, so it can be used for communication easily. Though even the design is not bad: SMTP by itself is quite usable, OpenPGP is better than plain text messages (though could be much better, and there is criticizm), it is all open and federated. Some of the email criticizm goes as far as to propose to replace it with something, but without proposing any viable alternative, so it does not seem like the time to abolish it yet, and here are some email-related notes.
Dovecot can also be used for SASL (for both Dovecot and Postfix). See the private server setup and simpler server setup documentation for more precise instructions, and possibly the "user authentication" note for more options.
DNSBL records appear for no apparent (or discoverable) reason in spamhaus's CSS blacklist (part of ZEN), /64 IPv6 subnets at once; delisting procedure is automated but complicated by Google captcha and partially broken (it reports success without actually delisting, and sometimes reports a captcha error even after solving the captcha, which is quite hard when using Tor). See also: Blacklisted by Spamhaus SBLCSS.
One way to mitigate it is to stick to
IPv4: smtp_address_preference = ipv4
in /etc/postfix/main.cf. Another one is to get a
/64 IPv6 subnet, assuming that they don't just blacklist subnets
at random.
Gmail (and maybe other large email providers) would occasionally mark/hide messages coming from smaller servers (and/or just not from themselves) as spam, even with SPF, DKIM, whitelists, messages being sent/delivered from them to you first. Not much can be done about it: once a mail server accepts a message, it is its responsibility to deliver it. Large commercial companies just keep messing up interoperability, as they always do.
Not much spam gets through with just configured Postfix and
postscreen, but when it does, it should be possible to report
the abuse to its ISP. Though spam from those who accept such
reports and resolve the issues is unlikely, and as a last resort
there are client_checks (or a firewall) to reject
messages from spammy IP addresses or subnets. But one should be
careful with that, since it is rather frustrating (and all too
common) when you're a good actor being treated as a bad one.
Dealing with spam coming form large providers is about as tricky as sending messages to them: they deliver spam just as regular messages, don't get blacklisted by honeypots automatically, and you proboably don't want to blacklist them manually because of all the legitimate users. Yet Gmail's abuse report form seems to be broken (simply nothing happens when I hit "submit": no network requests or UI changes, even with JS enabled), and their support is infamousely unreachable even by their own users. Then there's IP address's abuse contact (ripe-contact@google.com), but since they are their own hoster, it's probably also broken (as with the web UI, there's no visible reaction, not even automated; though at least there's a possibility of it working).
For more on incoming spam, see my network abuse notes.
Residential ISPs tend to block incoming SMTP connections, which is supposed to stop spam somehow, but if it was not for that, an IP address without NAT (and preferably static) would be sufficient at least to receive email directly, without a remote server. To get around that, there are services for port redirection, though I have not tried any, and they seem to be odd and/or to cost about as much as a remote VM (similarly to paid email).
Both notmuch and mu4e use xapian, which provides fast search. It is also nice to compose and read messages in Emacs (unless you are a vi user, perhaps), so I target those.
Some prefer mutt, which has a simpler configuration, and less modular, more self-contained. But its default key bindings are based on those of Vim, QWERTY-oriented, which is awkward if you use a different keyboard layout. Thunderbird is quite bloated, but perhaps more suitable for casual users, including mail services that require OAuth. It also supports OpenPGP now, but Maildir is not quite supported. Evolution looks similar to Thunderbird. Claws Mail looked odd and half-baked all around to me each time I tried it over the years, but it is a relatively lightweight GUI client, supporting OpenPGP and Maildir, but not OAuth, being similar in that to most other lightweight clients. But I focus on simpler Emacs clients (such as mu4e) in the following sections.
mbsync can be used to retrieve messages via IMAP, and Postfix
can also be set locally to get more flexibility and better SASL
options than emacs smtpmail library provides (see
the user authentication note).
SSH-only setup allows to use just SSH keys, with no SMTP or IMAP between client and server. Messages can be sent with a remote sendmail, while a remote Maildir can be accessed via sshfs, or messages can be retrieved with, for instance, doveadm sync. An example with relevant mu4e context variables:
(message-send-mail-function . message-send-mail-with-sendmail) (sendmail-program . "/home/defanor/bin/example-sendmail.sh") (mu4e-get-mail-command . ,(concat "doveadm sync sh -c " "\"SSH_AUTH_SOCK=$SSH_AUTH_SOCK ssh mail.example.com doveadm dsync-server\""))
And example-sendmail.sh:
#!/bin/sh ssh mail.example.com /usr/sbin/sendmail "$@"
Though an issue with this method of synchronisation (as
described here, without additional customisations) is that
messages removed from mu4e would be reloaded
by doveadm sync, and one would have to
use doveadm search and doveadm expunge
instead, or switch to IMAP for cleanup. Or use the sshfs method.
Another caveat is that even setting the remote sendmail script
as sendmail in $PATH won't necessarily
make all the programs to use it: for instance, git would still
require to set it explicitly (in .gitconfig or as a
command-line argument), as "smtp-server":
[sendemail]
smtpServer = /home/defanor/bin/example-sendmail.shLater it turned out to be handy to set mail sending this way, while retrieving it via IMAP, when a public provider (Yandex) that I used for work email with my own domain, to avoid dependencies on a personal server, decided to charge for using a custom domain and disabled SMTP. That way, the work server does not have to accept SMTP connections still, and it was already configured to send mail notifications from local clients (for both a website hosted there and munin), while incoming mail is handled as it used to be.
GnuPG can be used with mu4e (and perhaps most of the other common Emacs MUAs) out of the box, does not require any special setup.
While git-send-email(1) bypasses mu4e, receiving
patches still requires to point git (or another DVCS) to a
message that is normally first seen in one's MUA. I find it
handy to define a custom mu4e message action that simply
does (kill-new (mu4e-message-field msg :path)), so
that the result can then be fed into git-am(1).
Sometimes people attach large files (particularly
high-resolution images of their pets) to messages, which quickly
inflate the total mail archive size, complicating their backups
and migrations. When the messages also contain texts, it is
undesirable to remove the correspondence, but some MUAs can
remove individual parts. Particularly mutt can: backup the
maildir, save the attachments separately if needed, mutt
-f maildir-path, then open a message, v to
view attachments, select an image, d to delete
it. Then one may have to rebuild indexes, synchronize messages.
While there are different views and advices on email etiquette, relatively common ones are to use plain text, to properly quote relevant parts of messages when needed, to avoid bloating messages with signatures, and of course to adhere to general writing practices. Or, in other words, to be considerate and make minimal assumptions about readers' MUAs. RFC 1855 (Netiquette Guidelines) is worth reading.
With seemingly decent email providers (e.g., fastmail.com (banned in Russia), migadu.com), accounts cost like a hosted VM (VPS, VDS, or whatever they are called this year) or more, so it may be desirable to get a remote VM at once. Although there are slightly cheaper (or even partially free) ones as well: mailbox.org (blocked in Russia), runbox.com, mailfence.com (also blocked in Russia), posteo.de, maybe mailo.com (blocked in Russia). As for free ones, there is a few seemingly fine options, though usually they don't seem that nice after an attempt to use them; the ones commonly advertised as secure and/or ethical tend to not even provide SMTP and/or IMAP, not to mention SSH. Domain registrars tend to provide email services, though the quality varies. And there are ones like sdf.org and other pubnixes, including tildeverse ones, financed primarily with donations. Also disroot.org, dismail.de (no new registrations since 2021-05-28 though), riseup.net (rather politicized, blocked in Russia).
In 2024, I registered at Microsoft's hotmail.com, but my account (which only received one confirmation message from OpenStreetMap) was locked in a couple of days, with Microsoft claiming that it violated an unidentified part of the agreement, and that they need my phone number in order to resolve it. Apparently people who provide their phone numbers are unexpectedly locked out of Microsoft accounts as well, and there are regular stories like that about Google's Gmail, too. Though it also looks like many people do manage to use those larger services. As mentioned above, I ran into an unpleasant change of service terms with Yandex as well. Also receiving Gmail spam, reporting it, but spam from the same addresses keeps coming afterwards. Interaction with those larger commercial IT companies is generally a bad experience.
My primary concern with using private email for everything has been that regarding reliability, which is actually broader than just email. And if it is set on a single machine that you also use for everything else, that is a single point of failure for many things.
There are potential issues with public services as well: the companies that maintain those can go out of business, usually can do whatever they want with user accounts and data (commonly selling the data, messing up authentication and blocking accounts for strange reasons, with no way to contact customer support, sometimes mangling messages, restricting access to accounts until you provide more of personal data after a policy change), with the services they provide (including turning unlimited plans into limited ones, free into paid, cheap into more expensive), etc. Even technical issues with larger services may be equally or more common: though they have dedicated staff, larger setups tend to be considerably more complex and unusual, hence less reliable.
But private ones require regular payments and maintenance. It is not much harder than maintaining your personal machine, and usually cheaper than paying for an internet connection, electricity, and so on, but it is an additional burden. Very small one, but collecting things like that is always unpleasant: there is no shortage of other ways to get into trouble simply by staying idle.
Using 2-3 servers instead of one and teaming up with others (for both payments and maintenance) may be helpful to mitigate those issues, but that requires some trust. That is a hard part, since not many people seem to care about service providers, control, etc. Maybe it is a good approach though: worrying about all the small things and possibilities may be too much, whether one uses a private or a public service.
It is particularly unfortunate when other online services depend on email, allowing email-based account recovery: that way, the loss of an email address compromises those accounts as well. Sometimes it is possible to set a two-factor authentication, with the second factor being something relatively sensible, like TOTP, effectively disabling email-based account recovery that way, since that usually only allows to reset the password.
While music itself is pleasant to listen to, the theory behind it, along with maths for processing or synthesizing it, as well as the process of performing it, can be quite fun.
Music theory for nerds is a great starting point. "What Makes Music Sound Good?" is another overview and introduction, though perhaps more opinionated.
Some of the related and interesting research areas are those of music origin and purpose, such as evolutionary musicology, and how it's perceived by humans: psychoacoustics, music psychology, music and emotion.
The Ask HN: Tools to learn music theory? discussion contains a few more relevant links.
Open Music Theory looks like a nice textbook.
As with computing and maths, it is useful to study history of the subject as well, so that more of it will make sense, and it will be easier to put into a perspective. Videos on history of music can be found on YouTube, as well as on PeerTube, where some of the PianoTV videos are available.
The PCM format is to audio basically what netpbm/PPM/PGM/PBM/PNM is to graphics: very simple and straightforward, can be played with ffplay and others, easy to generate programmatically and write into a file without any encoder libraries, as well as to read without a special decoder. Audio I/O libraries (e.g., PortAudio) and codec libraries (e.g., libopus) tend to work with it.
DCT/DFT are often involved in processing (and in compression, also similarly to graphics), Mel-frequency cepstrum can be useful and/or interesting to look into.
Audacity is handy for checking the spectrum and notes in it, for music transcription and other checks.
To practice playing piano using a MIDI keyboard, one needs at least a software synthesizer and some music scores.
The keyboard in this case is M-Audio Keystation 88 MK3, which worked easily with Linux (5.10, Debian), Windows 10, and an Android tablet (Samsung Galaxy Tab A8, connected with a USB-A-to-USB-C adapter). For a synthesizer, I've used Yoshimi on Linux, LMMS (mostly with its sf2/soundfont plugin) on Linux and Windows, and Synthesia (not in F-Droid repositories, and I don't have a Google account, but grabbed an apk from their website) on Android.
MuseScore allows to compose sheet music and export it into MIDI rather quickly and easily, and there are more editors and converters of that kind available from Debian repositories.
PianoBooster looks like a nice trainer, akin to GNU Typist, but I found it quite annoying that it counts it as a mistake if you press a key too soon, so switched back to just reading scores and playing from those.
I use my computer screen to read sheet music, with the keyboard stand placed behind my computer chair, so it has to be zoomed in (Xfce's zooming in is quite handy when software can't zoom in on its own), and scrolling is needed for larger compositions, but the regular computer keyboard and mouse are out of reach. The MIDI keyboard has directional keys, messages from which come from a separate MIDI port; I haven't found readily available software (possibly LMMS plugins) helping to scroll the notes from a MIDI keyboard, but it took just a small script to achieve:
import mido
from xdo import xdo
# apt install python3-mido python3-xdo libportmidi-dev python3-rtmidi
# perhaps can be done in bash, with something like amidi + xdotool
# https://gitlab.com/dkg/python-xdo/-/blob/main/xdo/__init__.py
# https://gitlab.com/cunidev/gestures/-/wikis/xdotool-list-of-key-codes
# print(mido.get_input_names())
mapping = {
96: 'Page_Up',
97: 'Page_Down',
98: 'Left',
99: 'Right',
100: 'space'
}
x = xdo()
with mido.open_input('Keystation 88 MK3:Keystation 88 MK3 MIDI 2 24:1') as port:
for msg in port:
# print(msg)
if msg.note in mapping and msg.velocity == 127:
x.send_keysequence_window(mapping[msg.note])Fingering Scales on the Piano is a handy outline.
IMSLP.org is a nice source of public domain or otherwise freely available scores (including solo piano arrangements). Additionally, there are MIDI music collections around, which are lightweight, but encode melodies, which can then be viewed as scores (e.g., with MuseScore 2). Musopen also provides sheet music, as well as recordings of classical music.
Music composition seems to be rather similar to poetry, and to arts in general: a creative process, but one can reuse a musical form, learn and use a variety of approaches and tricks (by analyzing existing works, in addition to just reading about techniques), experiment and try things out.
Music appreciation seems useful to study as well; "Inside the Score" is one of the YouTube channels focusing on that.
Ryan Leach on YouTube makes nice videos explaining the composition process. David Bennett Piano brings up plenty of interesting subjects and analyzes songs.
While I don't sing, a brief look into it suggests that as with most of other skills, it's primarily about learning and practicing, exercising.
Yet even without singing, it is interesting to learn about vocal registers and related topics.
Possibly it is a wrong way to learn and practice, but I found it fun to set a tuner program on a phone (e.g., Tuner from F-Droid repositories), and try hitting notes.
For Android, there is the Open Ear program, available from Android: seems to be a little buggy (making noises), but allows to practice recognition of scale degrees.
The musictheory.net website also provides exercises, including those for ear training. A similar one to train playing from memory is lend-me-your-ears.specr.net.
And I composed a shell script using SoX for practice of identification of scale degrees and intervals.
Sometimes I find myself questioning the usefulness of these amateur music studies, particularly of playing instruments (while the theory and composition may conceivably be applied somehow), but it helps to view it as a recreational activity, quite similar to a game: the process itself should be enjoyable.
Commonly in IT-related discussions it is proposed that the world would be better if computer users learned to use their computers a bit better, to which somebody replies that plumbers don't expect knowledge of plumbing from you, and likewise you shouldn't expect computing knowledge from users. While the analogy is arguable, I'd rather draw the opposite conclusion from it: it is quite useful to spend some time learning about plumbing and electrical wiring, even if you're not going to do those yourself: it would at least help to identify building code violations and dangerous setups before those lead to issues, and will give a better idea what to ask for, which appliances you can plug into outlets, what are the options and how things can be improved. Otherwise the expected quality seems to be close to that of an outsourced to a random contractor piece of software (see also: "Why is it so hard to buy things that work well?"). Additionally, it is satisfying to know in a bit more detail how the things you use daily are set and work.
One can draw a few more parallels: just as programmers, different plumbers, electricians, and carpenters do things differently, argue which ways or technologies work better (or at all), tend to ignore instructions and documentation (and then run into trouble because of it), neglect documenting their work or disappear with supposedly written documentation, sometimes set wiring or plumbing dangerously, neglect maintainability (cementing pipes or splices without boxes in, or adding some trendy electronics, possibly even "smart" stuff, sometimes inside walls or ceilings -- setting more important components to go out of order and be hard to fix in a few months or years), and generally there are people who just hack things together without learning much about the subject, creating issues for everyone involved in the future, but there are more careful and knowledgeable ones too. And their tools are of varying quality, with both tools and materials being underspecified in stores. Most people do low-impact work, pretty much in any profession. Even though it is often said that the size of an error in software can be disproportional to its consequences, small errors in critical places can be disastrous with plumbing or wiring as well (but in most cases they will not matter much, in any of those areas). And just as programmers, they can do some programming/control: with ladder logic and other electronics for electricians, and with logic valves and other hydraulics for plumbers.
As for differences, judging by some search and chatter, online electrician communities tend to be private (sometimes requiring a license number to join), and information is limited in order to discourage DIY enthusiasts from making unsafe things themselves. Though of course there is plenty of such information in public anyway, and there are dedicated DIY communities. Additionally, while in IT the paywalled ISO and IEC standards are not the most common kind, apparently in other industries they are.
Apparently the risks and costs are different in developed countries, where work is commonly done by qualified and licensed workers, with professional liability insurance, and even checked regularly by inspectors in some cases. In Moscow I was able to find some mentions of qualified contractors, legal contracts (and possibility of civil lawsuits), warranty, and separate insurance, though the experience suggests that it probably will not work smoothly. Speaking of contractors, there is an interesting discussion on house building with contractors. But as a general rule, many people do not hesitate to accept money for jobs they are not qualified to do properly, possibly do not even realize that they are not (while many more seem comfortable even with jobs involving actively and knowingly hurting others, let alone doing a poor job), and the expectations of contractor work should be lowered in the absence of sufficient control (such as enforced regulations, or perhaps just a qualified third party revising the work).
These notes include some highlights, important things to pay attention to, and hopefully can serve as an entry point for starting the dive into these subjects. Nothing advanced is written here, since I'm a newbie myself.
Personal protective equipment should be used; hurting and possibly crippling yourself is undesirable.
The most basic and commonly needed for housework items are spectacles (or a face shield) and gloves (different kinds), possibly followed by respirators (depending on a task).
As a side note, some of the other workwear appears to be practical and comfortable, unaffected by awkward fashion trends.
When poking electrical wiring (e.g., changing light fixtures), it is better to have a voltage tester. I like a no-contact voltage tester, in part because it also helps to check where hot wires run inside a wall, but apparently they may be unreliable. Some of those count as multimeters: sometimes including a pyrometer, often a flashlight, sometimes voltage and resistance meters, and/or other regular multimeter functionality.
It may be hard to justify acquisition of tools without planning to use them frequently, but then it is easy to end up without tools. Buying complex and expensive electrical tools or machine tools for a single task would most likely be an overkill, but basic hand tools may be a better fit for a DIY enthusiast: usually they don't take much space and even good ones aren't expensive, and likely they will still work in a decade or few.
Apparently rubber usually degrades faster than metal, wooden, or even plastic parts, so it is useful to avoid rubbery handles for the tools you plan to have for years.
Toolboxes are commonly used to store and carry the tools, and multiple ones may solve the issue with having to haul a single heavy one each time you need something.
Since reviews, benchmarks, or even specifications tend to be unavailable, one rough heuristic for tool quality is its manufacturer (company, brand, and/or country). As with pretty much any other devices, established tool manufacturers tend to be in Germany (and around, elsewhere in Europe), Japan, Taiwan, and the US. Mainland China produces a lot of cheap stuff, often labelling it with an importer's brand; apparently supermarket chains use those.
As an example of a shady brand, as of 2021, there are GROSS brand's products in Russia, with a few "distributor" websites saying it is made in Germany, but the trademark is registered by the Chinese "Matrize Handels-GmbH" (the dash is not a typo; there are trademark search websites where one can look those up) company, with relation to some Russian folks who patented a spirit level recently. The "МИР ИНСТРУМЕНТА" chain occasionally mentions that it is their trademark, and occasionally just says that they are the exclusive distributor. Reportedly "Мастернэт" does something similar, distancing itself from its brands. Apparently a very similar thing happens with cookware (Gipfel and others), possibly with electronics (ERA and Rexant: don't pretend to be German, but apparently ship most of the production from China while positioning it as local), light fixtures (apparently Arte Lamp is like that, along with InStyle and Divinare: advertised as Italian, made in China, seems to only sell in Russia, the trademark belongs to a Lithuanian company, used to belong to one from Panama, the Italian domain name's contact information points into Moscow, unclear whether the referenced Italian company and office exist at all: the company doesn't show up in searches, the address on Google Maps doesn't have the office marked, and that address doesn't match the approximate location mentioned in the company description, the description itself looks half-baked, and for some reason many retailers sell them for exactly the same price).
I've looked into local (Russian) tools specifically, wondering whether there is anything good (or at all), given Russia's industrial sector being similar to that of Germany in size, and all the fuss about import substitution even in IT (where it looked silly, with rebranded software and everything). Apparently at least 1/3 of the industrial sector is resource extraction (though judging by the list of largest Russian firms, it is mostly extraction and its support/servicing), and it is mostly defense, aerospace, and automotive industries otherwise (though those heavily depend on imported components and equipment as well, basically doing only the final assembly locally), while the import substitution mostly aimed agriculture, automotive industry, and IT. Looks like there are some small tool manufacturers, though rather often upon closer inspection it turns out that most of the production comes from China (PRC), even if they have some kind of local manufacturing.
Some of the consumable products that are nice to have at home, in the decreasing order by estimated frequency they tend to be needed with, roughly in the order I would acquire them: cleaning products, light bulbs, batteries, duct tape, WD-40, electrical tape, lubricants (silicone grease seems to be versatile), sealants, glues for different purposes, various connectors and cables, maybe solder and flux, screws and wall plugs, bolts and nuts, nails.
Everything involved has a limited lifespan, which is usually a fraction of human life expectancy, so one is likely to have to replace everything a few times in their lifetime, and it is nice to plan for that.
It is important to be able to turn everything off (water, electricity, gas), either for planned maintenance or in case of an emergency. So far I had most trouble with isolation valves, which were painted over, rusty, hidden, or absent. Circuit breakers tend to be in a better shape, but still not always quickly and easily accessible (while they should be), and not always labelled.
It is a good idea to learn about standard current ratings, and to ensure that the actual wiring matches those. Residual-current circuit breakers (RCCB, aka RCD, aka GFCI) may be desirable.
All the connections (valves, splices) should be accessible without breaking the walls, ceilings, or floors. It wouldn't harm to have all the pipes and wires accessible as well, which is unusual for homes, but can be found in industrial/commercial settings.
Some of the accessible electrical wiring setups involve cable trays, electrical conduits, raised floors, dropped ceilings, cable raceways (sometimes hidden in moulding). Though before jumping into those, one should also learn about building codes (in Russia it is правила устройства электроустановок, though it is often said that Russian ones are stricter than US and EU ones, apparently contributing to them not being followed, and to the corruption, leading to more fires and other incidents; in the US there is National Electrical Code), some of which require certain useful cable insulation properties (e.g., for it to not fall apart in a few years, to not catch and transfer fire), mechanical protection, appropriate circuit breakers, and possibly more. Actually if the building codes are followed, even regular concealed wiring should be accessible and somewhat adjustable.
Ideally ventilation systems should be accessible as well, to extract the birds falling into those and for inspection in general.
Lightweight furniture, relatively easy to move around, is useful for not blocking access to some parts of a room. And it is nice to keep surfaces accessible for cleaning, preferably even without moving anything.
Casual plumbing is mostly about screwing things together, and occasionally dealing with rust. It is nice to know about piping and plumbing fittings, along with sealant types, and perhaps about work with pipes for more advanced plumbing. Common tasks, like replacing a tap, usually would only require a sealant (a teflon tape is rather neat for tapered thread sealing, but a liquid sealant is still needed to seal wall-adjacent bits) and a plumber wrench or an adjustable spanner, preferably a spirit level. Plastic plumbing for low-pressure waste water usually doesn't require even that, but only relies on gaskets for sealing.
There are videos available online where plumbers demonstrate the process, and one can ask a hired plumber to help them for safer practice.
As for taps and other sanitary fittings themselves, companies from the same places where decent tools are made seem to produce the best ones (at the time of writing, Grohe is a nice and popular option).
A malfunctioning washing machine, a leaky pipe, and other things can lead to unexpected flooding, which is dangerous and deals water damage to the floors below (summoning unhappy neighbours in case of an apartment building). A floor drain can help to mitigate it, though it may be tricky to set even during renovations. Other options are drain pans, isolation valves with water sensors. To avoid even minor leaks reaching the neighbors, the floors around piping may be required to be waterproofed, and either lower than the adjacent ones, or separated with a threshold (but such requirements are commonly violated).
Lighting is usually divided into task lighting (direct and bright), ambient lighting (soft, indirect or diffused), and decorative (or accent) lighting.
Kitchen task lighting should be more than 500 lux (or at least 50 lightcandles), 1000 lux is recommended for electronic manufacturing. I went for a 1000 lumens/meter LED strip for mine. The linked Wikipedia article includes reference illuminance values for various other situations.
A high colour rendering index (CRI) is important for the lit items to look "right": as one would expect them to look, similar to being lit by the Sun.
Sunlight colour temperature at Earth's surface (after it gets through the atmosphere) varies, but seems to be around 6500 K, and that is our definition of white colour. I went for 5000 K, which looks fairly white as well, and was just the maximum available locally at the time with a high CRI. Philips light bulbs at 4000 K look yellowish, even though they are labeled "neutral white". 6500 K ones seem fine to me.
As with computers, it is useful to have good (possibly overkill) heat dissipation and PSU (in case if LED strips are used; AC-to-DC converters themselves recommend to pick their maximum load as expected load + 20%, and some recommend to rather make it 30--40%, though going further would probably be wasteful, and possibly they will not work as efficiently at lower loads). As with many other devices, one should pay attention to ingress protection (IP) codes, especially for kitchens and bathrooms.
One complete list of materials for setting kitchen cabinet lighting nicely (with a "gap" for going from over-cabinet to under-cabinet lighting, or just for skipping some areas: that is, with a cable soldered between two runs of the strip) is a LED strip, a PSU (and possibly a power cord or a rewirable plug to plug it into an outlet, with a switch), a LED channel, a two-wire cable (the electric current can be high at low voltages, so the wire diameter should be appropriate), electrical tape and/or heat-shrink tubing, double-sided tape (LED strips usually have them pre-applied, but they are also handy to glueing LED channels to cabinets) or some other mounting materials, solder and flux. The tools needed to assemble it then are a soldering iron (although one can go for solderless connectors instead), a hacksaw (unless chunks of aluminium profile fit well without it, or can be cut by others), preferably a file, scissors, likely screwdrivers for PSU's screw terminals, a wire stripper if available.
As for LED bulbs, cheap ones by lesser-known manufacturers usually end up costing more because of their short lifespans, while being dim and flickering. Philips ones are mostly fine and commonly available, though they buzz audibly. And there are benchmarks around.
And once again, the stuff that is non-trivial to replace should be reliable (which usually means simple), so that you have less of a headache when things break: simple light fixtures with replaceable light bulbs (and without built-in AC-to-DC converters and LED strips) are preferable, so that the complex and unreliable parts are easy to replace (and it is easy to find and choose a replacement quickly as well). Among those, the simplest (most straightforward and reliable) ones are perhaps those without lampshades or excessively fancy elements, just with light bulbs poking out, with enough space around them, and easiest to replace.
Major appliances tend to be relatively heavy and expensive, so it is a good idea to choose and install them carefully.
For instance, usually there are special requirements for washing machines (or launder rooms, and/or other rooms with water/sinks), involving a separate circuit breaker (usually a 20-ampere one, not the more usual 16). Though even small appliances like electric kettles may easily trip the circuit breakers if turned on together while connected to the same circuit.
Apart from the quality and appliance-specific specifications, easy repairability and maintenance are preferable: availability of genuine spare parts (long-term support by the manufacturer), ease of their replacement, availability and cost of regular servicing/maintenance. One can look up spare parts on manufacturers' websites, and repairs tend to be about part replacement, the videos of which are often available online.
Generally appliances aren't built to last these days, but rather to work for their expected lifespan and be utilized then. It seems that manufacturers are pretty good at optimizing them for those lifespans (and/or for planned obsolescence), so they start falling apart at that point.
There is an appeal in lightweight furniture, as mentioned above, but it may still have wooden parts, or one may prefer wood for longevity, aesthetics, or other reasons. Hardwood is more durable and more fire-resistant than softwood, but also heavier and more expensive. Hardwood comes from angiosperm trees (mostly deciduous, broad-leaved, flowering), and the wood has pores, while softwood -- from gymnosperm trees (coniferous), and the wood has resin canals or ducts.
For a reference, some of the hardwoods are: oak, cherry, apple, alder, balsa, beech, birch, ebony, maple. Some of the softwoods are: pine, fir, spruce.
See WikiHow's "How to Determine the Wood Type Used in Furniture" and similar guides to attempt to identify the wood types used in furniture.
It may also be a good idea to avoid wood veneer, preferring solid wood furniture, which exposes the wood it is made out of. Apart from avoiding the usual issues of likely-imperfect "faking" (or decoration), solid wood allows for easier identification and easier fixes (or restoration).
Relatively common mattress lengths are 200 and 190 cm; the latter is insufficient for a 180 cm person to lie on the back comfortably, even with a relatively compact pillow. Footboards make it even more annoying, I find those best to avoid in most cases, even with longer beds.
"How to Pest-Proof Your Home".
Proper documentation is hard (especially if it is others doing the construction), but it is a good idea to take pictures during all kinds of construction and renovations: questions tend to arise later, with the answers being forgotten and hidden behind the walls, floors, and ceilings. Storing recipes is a good idea too, since later one may need to acquire matching materials or just check what was used.
LibreCAD handles building information modeling with its Arch workbench (and the external BIM workbench), which includes pipes.
For any type of heating (or cooling) a good insulation is useful and important, but in some cases it is even required in order to make them practical or achievable.
Air intake is a useful feature: in addition to delivering fresh air, it can filter that air, and even do some heat exchange with the air already in the room, for increased efficiency. Otherwise some end up running an AC with an open window, leading to a low efficiency, while the dust keeps coming.
Apartment buildings tend to have communal heating, but for a separate house that is an additional headache. Even if a house is unused, letting it to freeze is dangerous not just for pipes and furniture, but also for all the surfaces that can be claimed by mold, or the materials intended to stay indoors. In some places freezing isn't a concern, but around Moscow temperatures can go down to -40, though average low in the coldest months is closer to -9, and -20 is rather rare even at night. A common estimate for a 200 m² house is 10 kW of heat per hour, which matches my observations.
That is a bit much for electrical heating (not just costly, but also loading the grid and the electrical wires quite a bit: at 220 volts that'd be around 45 amperes); gas heating costs almost as much by 2021 (gas prices went up), and seems to be rather clunky, requiring increased maintenance. Heat pumps seem promising though: electricity-powered, but with heat output a few times higher than the consumed electricity. Air source heat pumps may be more suitable for warmer temperatures, dropping in efficiency quite notably at lower ones, when they are needed the most. Ground source ones don't have that problem, but require either large areas or deep holes, and the holes may require licensing and fees (or to stay within limits: as of 2021 in Moscow, those are apparently to extract not more than 100 m³ of water, for own non-commercial use), not to mention relatively high upfront costs. I'll probably investigate that and update these notes later.
But an important thing to keep in mind is that maintaining a house can be pretty expensive because of this; it is an opportunity to unexpectedly find yourself responsible to pay regularly, which is stressful, which is not great for overall mood and well-being. Building a smaller house, with better insulation, planning the heating from the beginning, and asking yourself whether you really want or need a house in the first place would all be good ideas.
For some devices, particularly heating, remote monitoring and/or control may be needed. Boilers, for instance, use semi-closed protocols like OpenTherm, which are tricky to interact with. I plan to investigate this later, but here are some links related to libre and customizable systems for that (IoT/"smart home" sorts of things): openHAB, Home Assistant, Homebridge, ESPHome. Some do it from scratch, see "How I wrote my own Smart Home software". I try to simply avoid such functionality for devices that do not need remote control, thereby avoiding unnecessary exposure to vulnerabilities commonly found in those, often even picking mechanical devices over electronic ones (to avoid dependencies on power, in addition to those on Internet connection, as well as the added complexity), but for vacuum cleaners, one may consider Valetudo as a local-only option (though not for a multi-floor setting), if a robotic vacuum cleaner is wanted and it is challenging to find one autonomous by default.
Public utilities and other communal services tend to be low-quality and hard to get fixed around here, complete with scammy field service technicians: whether those are commercial organizations (with some competition) or the ones funded out of municipal or government budgets, with mandatory regular payments, issues are not getting solved unless you pay somebody unofficially to actually fix them. Sometimes such services are sold by technicians themselves. When there is a fixed price for a service, one of the tactics the technicians, veterinarians, and likely others employ is to claim that the price is for the work itself, but the consumables should be paid for separately (even while it certainly is not so, and some companies would call you afterwards, asking whether it happened -- though without warning in advance that it happens).
While going off the grid in an apartment completely is not viable, it seems sensible to reduce the number of such utilities, rather similarly to software, hardware, or online service dependencies with bad vendors.
A Pattern Language looks like a nice book on architecture and adjacent topics, with its copyleft version available.
This is an opinionated overview of the available software packaging and deployment options, including those for commercial software and binary releases.
Proper and usable software installation may include placement of executable files, data files, documentation such as man pages and info manuals, configuration files with appropriate permissions, init system configurations (e.g., init scripts, service files) for daemons, freedesktop.org configurations for desktop applications. As well as creation of system users, a way to uninstall the software, proper handling of configuration files (not just overriding user configuration). And preferably it should be manageable using standard system tools, since it is a pain to use multiple package managers to maintain a system.
There are nuances and incompatibilities among major GNU/Linux distributions (not to mention other UNIX-like systems, or different OS families), so it's not a straightforward task. For instance, it used to be a pain to write init files for different distributions because of the available software; now there is systemd on most of the major ones. Generally, automated and reliable integration of different software components is tricky.
In the least hacky out of common scenarios, upstream developers release the software following the standards and conventions, which is packaged for different systems by maintainers, and necessary adjustments are introduced.
But this does not work for commercial software, or for binary releases of smaller projects. Neither does it guarantee that installation and maintenance would not be a pain (cf. most of the issue trackers), though it does not have to be.
Sometimes this approach works poorly with FLOSS projects as
well: notably, Rust packages tend to have many dependencies,
with some of those usually aiming nightly compiler builds, and
it is generally assumed that it is installed circumventing a
system package manager. Many non-Rust projects also consider
themselves special, and suggest custom installation options:
AppImage installers, other container images with whole operating
systems included, manual building (sometimes with odd build
tools) and installation, just some odd scripts, possibly with
"curl ... | sh". This leads into the territory of
the "ad hoc mess" and "masked mess" sections below.
Build systems such as GNU autotools, or the language-specific ones such as Cabal for Haskell programs, can be used for packaging and installation on their own. Autotools even try to deal with system incompatibilities, but still don't cover all the tasks (such as user creation or portable service installation), dependency resolution and automatic installation are partial at best (for language-specific package managers), and of course the software installed that way isn't manageable with a package manager native to a given system.
It is even less suitable for binary software distributions: build systems are mostly for building, as the name suggests.
As examples, autotools-generated and similar Makefile-based
tarballs use make install, cabal
install can install Haskell programs globally.
For Python programs something like PIPX_HOME=/opt/pipx/
PIPX_BIN_DIR=/usr/local/bin/ pipx install --system-site-packages
. can be used: it is still a language-specific and
system-independent system, though installing partially into
non-standard paths and using virtual environments, having a
notably worse system integration, more similar to those from the
next section. Although cabal, while pulling dependencies from
Hackage, by default similarly mostly ignores the underlying
system with its package manager, and bundles those dependencies
together (just in a statically linked executable file, rather
than a venv).
Custom shell scripts or Makefile, curl |
sh installation, various other custom installers, manual
code copying, lengthy and awkward installation instructions,
DVCS-based deployment (with private keys and passwords
occasionally being in a repository and/or hardcoded), and
virtual machine images seem to be used rather often for in-house
or "enterprise" commercial software. It is a mess and a
nightmare to maintain, usually matching the software it is used
for, but perhaps worth mentioning as a bad example.
For quick and dirty packaging though, tar and a few shell
commands can work fine: tar czf the files, then
unpack with tar --same-owner -C / -xvf, while using
shell commands for that unpacking, adding users, installing
dependencies, enabling services.
There are projects that do roughly what is described in the previous section, but with dedicated websites full of marketing slogans and making those solutions not so custom by getting more people to use the same kind of a solution. For instance, Flatpak and AppImage (primarily for desktop applications), Docker. Their issues are not very different from those with the ad hoc approaches (i.e., poor system integration), though they introduce a possibility of at least non-standard package management, and may patch some of the issues that arise.
Containerization with system images in particular I find similar to people taking screenshots of texts, web pages, PDFs, or other documents instead of copying the relevant text or saving those, and perhaps then pasting them into a graphics-capable word processor to save on a disk. Or even taking a picture of a screen with a camera. That is, capture the needed state with familiar and generic tools, even if it's inefficient and/or somewhat lossy; later those are also readable with generic tools, and it does the job in most cases.
Compared to a nice setup, such containerization introduces unnecessary abstraction layers and bloat, but compared to a messy one, it keeps the mess contained. Which makes it particularly desirable in commercial software development, perhaps.
This is the one I like the most so far: write a program as an
upstream developer (following FHS and other standards and
conventions, using portable libraries when available), then
package it as a maintainer (following Debian New Maintainers'
Guide), then deploy and configure it as an administrator. I used
to cut some corners for packaging, using cabal copy
--destdir=deb, but as of 2022, with Debian 11 and Cabal
3, perhaps a more proper and straightforward approach is
something like the following (just for a regular binary package;
can be tweaked to provide source/profiling/doc packages as
well):
sudo apt install devscripts build-essential lintian haskell-devscripts cabal-debian -m 'name <name@example.com>' --disable-profiling --disable-haddock # comment out DEB_SETUP_BIN_NAME in debian/rules debuild -i -b
Or, a less neat way, to build with Hackage packages, and then package the executable:
mkdir -p deb/usr/bin/ cabal install foo --prefix=/usr --install-method=copy --installdir=deb/usr/bin/ --overwrite-policy=always dpkg-deb --build deb/ foo_1.2.3.deb
A generic script for that:
#!/bin/sh
# This script packages Haskell programs into deb packages.
set -e
PACKAGE_DIR="$1"
if [ ! -d "$PACKAGE_DIR" ]
then
echo "Usage: package-cabal.sh <directory>"
exit 1
fi
cd "$PACKAGE_DIR"
CABAL_FILE=$(ls ./*.cabal)
if [ ! -f "$CABAL_FILE" ]
then
echo "No cabal file found in $PACKAGE_DIR"
exit 2
fi
VERSION=$(sed -n 's/^version: *\(.*\)$/\1/p' "$CABAL_FILE")
EXE_NAME=$(sed -n 's/^executable *\(.*\)$/\1/p' "$CABAL_FILE")
DEBNAME="${EXE_NAME}_${VERSION}-0.deb"
mkdir -p deb/usr/bin/
cabal install --prefix=/usr --install-method=copy \
--installdir=deb/usr/bin/ --overwrite-policy=always
dpkg-deb --build deb/ "${DEBNAME}"There is a "getting started" guide on packaging Haskell projects for Debian as well.
For Python packaging on Debian, there is dh-python (with some
documentation in the dh_python3(1) man page): one
can take an existing package as an example (e.g., apt-get
source xkcdpass),
compose debian/{rules,control,changelog}, setup.py
(apparently no support for pyproject.toml yet), and
produce a package with dpkg-buildpackage -us -uc. I
think the packages I had to install before that
are dh-virtualenv dh-python debhelper
python3-docutils.
For other guides, see Packaging - Debian Wiki, Arch package guidelines.
An upside of such an approach is that software properly integrates into the system, so the regular practices can be applied. Sometimes it also makes you to adjust the software to make it easier to package and maintain.
A downside is that properly maintaining a custom repository (with timely key rollover) is a responsibility, and the documentation seems to mostly aim FLOSS project inclusion into the primary repositories. Hence all the third-party repositories that break updates, apparently. I use standalone packages (without repositories) on Debian instead, but it leaves open the problem of distribution and updates via a system package manager.
Here are some tips and tricks for writing software and packaging it in such a way that it would be relatively painless to deploy and maintain (apart from the regular "follow the standards and conventions").
For applications that use PostgreSQL, it is handy to default to an empty (but configurable, of course) connection string: it will just work with a local database and peer authentication, simplifying the necessary deployment steps. While providing a way to specify the connection string (and not, say, just credentials) keeps it very flexible.
To prepare a database (create tables, define roles and security policies, stored procedures and aggregations, views, insert initial data, etc), a handy approach is to add an "init" mode into the application, which would simply read SQL files from a data directory (which should be packaged and installed there) and execute those in a single transaction, potentially prompting a user for an initial application administrator password. It seems straightforward, yet rarely gets done any nicely. This can also be combined with schema updates by prefixing SQL file names with version numbers.
One of the uses of VMs and containers is sandboxing, but some (and more lightweight) service sandboxing is fairly easy to set with systemd and AppArmor or SELinux. A sample service file I use:
[Unit] Description=Sample Service After=syslog.target [Service] Type=simple User=sampleservice Environment="sampledb=port=5432 dbname=sampledb" ExecStart=/usr/bin/sampleservice # service sandboxing PrivateTmp=yes ReadOnlyPaths=/ NoNewPrivileges=yes PrivateUsers=yes ProtectControlGroups=yes ProtectHome=yes PrivateDevices=yes ProtectKernelModules=yes ProtectKernelTunables=yes ProtectSystem=strict SystemCallFilter=@system-service SystemCallFilter=~@privileged @resources CapabilityBoundingSet= ProtectClock=yes ProtectProc=invisible ProtectKernelLogs=yes ProtectHostname=yes RestrictSUIDSGID=yes SystemCallArchitectures=native RestrictAddressFamilies=~AF_PACKET RestrictRealtime=yes RestrictNamespaces=yes LockPersonality=yes RemoveIPC=yes AppArmorProfile=sampleservice [Install] WantedBy=multi-user.target
That references /etc/apparmor.d/sampleservice:
#include <tunables/global>
profile sampleservice {
#include <abstractions/base>
#include <abstractions/nameservice>
#include <abstractions/openssl>
}
And something like the following
in deb/DEBIAN/postinst for Debian packaging:
#!/bin/sh
if [ "$1" = configure ]; then
# set users
id sampleservice >/dev/null 2>&1 || adduser --system --quiet --no-create-home sampleservice
# set AppArmor profiles
apparmor_parser -r -W -T /etc/apparmor.d/sampleservice
fiThese are my notes on setting and maintaining a desktop/workstation system, a successor to the older CentOS 7 workstation, to be used--among other things--with the private server setup and simpler server setup.
My goals were a working setup, along with an old system, simple
and close to the standard one, and with
encrypted /home (see also: personal data
storage). To avoid possible confusion during installation or
when some repairs are needed, I keep a sheet of paper with
partitions listed on it.
I went for Unofficial non-free images including firmware
packages, since I need GNU documentation and the Nvidia
proprietary driver anyway (unnecessary as of Debian 12, since
proprietary firmware is included into official images, and that
Nvidia card is not supported anymore), and it is more suitable
for a rescue USB stick. Picked a live Xfce image, to be able to
poke it briefly (and ensure that it works fine with the
hardware) before installation, as well as for possible later use
as a rescue system. Though live images come with a drawback of
installing live-task-* packages, including
localization ones for all the supported languages, so you end up
with hundreds of additional and unused packages to upgrade
regularly; netinst produces a cleaner system, but
they can also be removed manually afterwards. Xfce is not as
bloated and broken as GNOME and KDE, but not as half-baked and
broken as most of the others. Apparently MATE and Cinnamon aim a
similar level of complexity, and I hear good things about those,
too. I downloaded the image via BitTorrent, and as
the Installation Guide suggests, did the equivalent of cp
debian.iso /dev/sdX && sync.
There is a graphical installer available from the live system itself, which is handy for looking up documentation on the web while installing, but its functionality differs from that of the regular installer: there is no option to make an EFI system partition (ESP) explicitly, so I rebooted and used the regular installer. Although while installing Debian on another machine a bit later, I noticed that it would handle fine a FAT32 partition mounted into /boot/efi, without requiring to mark it explicitly as ESP.
As usual, I wanted to keep the old system usable and independent, so I have set this one on a separate disk, with a separate ESP, which I had to add (about 500 MB in size); the installer presented a warning about possibly making other systems hard to boot into if EFI is forced, but I've installed it on a separate disk (and adjusted UEFI boot priorities accordingly), so it was fine.
I used btrfs for a while, but decided to go with ext4 this time, since I use btrfs's advanced features less and less, while a simpler filesystem may be more reliable. Decided to minimize dealing with partitioning in the installer, and just made a single 500 GB partition for everything (not counting ESP, and while having 1.5 TB unpartitioned on the disk). No swap partition either, since in my experience it's not helpful and only freezes the system when something goes wrong. Didn't choose a network mirror to download new packages either, so the installation went quickly and smoothly.
While the en_US.UTF_8 locale is very
common, C.UTF_8 may be better to set at once, since
it has 24-hour time format, sensible string sorting, and DBMSes
(particularly PostgreSQL) are more portable when set with it,
not running into collation version mismatches on replication
between databases hosted on different operating systems. This is
simply adjusted in /etc/default/locale.
As with CentOS about 7 years prior to this setup, apparently the nouveau driver was causing the system to freeze, so I installed the NVIDIA Proprietary Driver.
Then I've added my user into the sudo group, have
set the keyboard layout to colemak with sudo
dpkg-reconfigure keyboard-configuration (since the
installer doesn't provide that option), have set it in Xfce's
settings to use the system layout (actually in a couple of
places, not sure why there are so many). While at it, removed
the useless bottom panel (application launcher), have set a dark
theme, nicer icons, disabled icons on the desktop.
As with servers, and perhaps more importantly than with those,
decent and varied nameservers should be set. In this
case /etc/resolv.conf mentions that it's generated
by NetworkManager (which is rather awkward and unnecessary, and
an example of little bloat task-xfce-desktop
pulls), so one can adjust nameservers with nm-connection-editor.
Then I've set the previously mentioned
encrypted /home (this method is a bit verbose,
since I've checked that things work as intended):
sudo fdisk /dev/sda # created another 500 GB partition for /home, sda3 sudo apt install cryptsetup sudo cryptsetup luksFormat /dev/sda3 sudo cryptsetup luksOpen /dev/sda3 enchome sudo mkfs.ext4 -L home /dev/mapper/enchome sudo cryptsetup close enchome sudo blkid | grep sda3 sudo -e /etc/crypttab # added the following: # enchome UUID=PARTITION_UUID_HERE none luks sudo -e /etc/fstab # added the following: # /dev/mapper/enchome /mnt/home ext4 defaults 0 2
Then rebooted to ensure that /mnt/home mounts fine,
moved the files from /home there (with cp
-a), renamed /home, have
set fstab to mount it
into /home. Rebooted again, checked again that
everything is fine, and removed the old /home.
One may also mount /tmp into memory, reducing the
data leaking to the unencrypted root filesystem, slightly
speeding up some tasks, and reducing disk usage; it works for me
and I like it, but there is plenty of criticizm and possible
issues with that:
tmpfs /tmp tmpfs size=1g,nosuid 0 0
Moved/imported my SSH and GPG keys, ~/.authinfo,
some other files.
I had to remap the "menu" key (keycode 135) to left alt, which
is always awkward and different; in Xfce I had to enter the GUI
settings, then "session and startup", and add the xmodmap
-e "keycode 135 = Alt_L" command there. Also had to unmap
C-M-f to be able to use it in Emacs, in "settings" - "keyboard"
- "application shortcuts".
XFCE's default key bindings for basic tiling functionality aim a numpad, which I do not have, but those can be adjusted in "settings" - "window manager" - "keyboard".
To disable GnuPG's annoying requirment to use non-alpha
characters in a passphrase (which is contrary to NIST SP
800-63B, and complains about passwords in the style of XKCD
#936, such as those generated with xkcdpass), echo
'min-passphrase-nonalpha 0' >>
~/.gnupg/gpg-agent.conf.
More software: sudo apt install emacs
emacs-common-non-dfsg telnet vlc tor mu4e isync rsync xsltproc
clementine git elpa-magit elpa-haskell-mode cabal-install lynx
whois nmap ncat dnsutils knot-dnsutils tmux fbreader inkscape
blender godot3 gimp darktable lmms musescore texlive
texlive-plain-generic auctex texlive-latex-extra texlive-science
python3-sympy octave octave-symbolic libxml2-utils
jmtpfs xkcdpass,
and better-defaults, mu4e-alert,
and cdlatex via Emacs's package manager (since they
weren't in the system repositories). Generally it's a good idea
to stick to a single package manager, since then you shouldn't
run into version mismatches. update-alternatives --config
editor to set vim as the default editor (running a new
emacs instance may be a bit slow for quick sudo -e
editos, emacsclient won't always work, setting a small emacs
clone just for that seems excessive, and the default nano is
just awkward, so vim is an okay option; though perhaps one can
also set emacs -Q -nw). Over time a bunch of other
things were added, including mpd (running as a user service) and
mpc, strongSwan, likely more development tools.
Then I set xterm and Emacs themes (.Xresources,
Elisp), from my dotfiles repository.
By 2022, I had to start using Tor bridges (since Tor is being
blocked around here, and Internet connectivity is crippled in
general, with Tor helping to fix some of it):
install obfs4proxy, then append
to /etc/tor/torrc:
UseBridges 1 ClientTransportPlugin obfs4 exec /usr/bin/obfs4proxy managed
And bridge records received from bridges.torproject.org or by
other means, prefixed with "Bridge" (Bridge obfs4
...). Though by 2024, many of those are blocked.
Configured Firefox: Sans Serif font, disallowed pages to choose their own fonts, increasing monospace font size to be the same as others (16), setting a minimal font size equal to those, "wp" keyword for Wikipedia search and "wt" for Wiktionary search, installing uBlock Origin (with "annoyance" lists additionally enabled) to cut out junk, NoScript to cut out more junk, FoxyProxy to use Tor for websites blacklisted around here and the ones I don't want to track me, HTTPS everywhere to mitigate local data retention practices (superceded by the Firefox's built-in HTTPS-Only Mode, which should be enabled in settings), Stylus to set a global dark theme for comfortable browsing when it is dark around.
Configured isync and Emacs, later installed rexmpp's
xmpp.el. Attempted a minimal Emacs configuration this time
(though most likely it'll grow), so used the built-in rcirc
(with rcirc-track-minor-mode and just
setting rcirc-server-alist), not much of mu4e
configuration. Something like this:
(require 'package)
(add-to-list 'package-archives '("melpa" . "https://melpa.org/packages/") t)
(package-initialize)
(require 'better-defaults)
(global-set-key [mode-line mouse-4] 'previous-buffer)
(global-set-key [mode-line mouse-5] 'next-buffer)
;; https://github.com/defanor/cyrillic-colemak
(require 'cyrillic-colemak)
(add-to-list 'custom-theme-load-path "~/.emacs.d/elisp/")
(load-theme 'blueish t)
(setq org-preview-latex-default-process 'dvisvgm
org-babel-python-command "python3"
org-src-preserve-indentation t)
(with-eval-after-load 'org
(plist-put org-format-latex-options :scale 1.5)
(require 'ob-python))
(rcirc-track-minor-mode t)
(setq rcirc-buffer-maximum-lines 2000
rcirc-server-alist
'(("irc.libera.chat" :port 6697 :encryption tls
:user-name "defanor" :channels ("#emacs")))
rcirc-authinfo
'(("libera.chat" sasl "defanor" "password-here")))
(require 'haskell-interactive-mode)
(require 'haskell-process)
(add-hook 'haskell-mode-hook 'interactive-haskell-mode)
(add-hook 'haskell-mode-hook 'haskell-decl-scan-mode)
(require 'html-wysiwyg)
(add-hook 'html-mode-hook 'html-wysiwyg-mode)
(add-hook 'after-init-hook #'mu4e-alert-enable-mode-line-display)
(setq mail-user-agent 'mu4e-user-agent
read-mail-command 'mu4e)
(with-eval-after-load "mu4e"
(require 'smtpmail)
(setq mml-secure-openpgp-encrypt-to-self t)
(defun suppress-messages (old-fun &rest args)
(cl-flet ((silence (&rest args1) (ignore)))
(advice-add 'message :around #'silence)
(unwind-protect
(apply old-fun args)
(advice-remove 'message #'silence))))
(advice-add 'mu4e-update-mail-and-index :around #'suppress-messages)
(advice-add 'mu4e-index-message :around #'suppress-messages)
(advice-add 'progress-reporter-done :around #'suppress-messages)
(setq mu4e-change-filenames-when-moving t)
(add-to-list
'mu4e-contexts
(make-mu4e-context
:name "thunix"
:enter-func (lambda ()
(mu4e-message "Switch to the thunix IMAP context")
;; (mu4e~request-contacts)
)
:leave-func (lambda () (mu4e-clear-caches))
:match-func (lambda (msg)
(when msg
(mu4e-message-contact-field-matches
msg
:to "defanor@thunix.net")))
:vars '( (user-mail-address . "defanor@thunix.net")
(user-full-name . "defanor")
(smtpmail-default-smtp-server . "thunix.net")
(smtpmail-local-domain . "thunix.net")
(smtpmail-smtp-user . "defanor")
(smtpmail-smtp-server . "thunix.net")
(smtpmail-stream-type . starttls)
(smtpmail-smtp-service . 25)
(message-send-mail-function . message-smtpmail-send-it)
(mu4e-get-mail-command . "mbsync -q thunix")
(mu4e-update-interval . 300)
(mu4e-view-show-addresses . t)
(mu4e-maildir . "~/Maildir/thunix/")
(mu4e-mu-home . "~/.mu/thunix")
(mu4e-user-mail-address-list . ("defanor@thunix.net"))
)))
;; more contexts here
)
And .mbsyncrc records like this:
IMAPAccount thunix Host thunix.net Port 993 User defanor SSLType IMAPS Pass "password-here" AuthMechs * IMAPStore thunix-remote Account thunix MaildirStore thunix-local Path ~/Maildir/thunix/ Inbox ~/Maildir/thunix/inbox/ Channel thunix Far :thunix-remote: Near :thunix-local: Patterns * !drafts Create Both Remove Both Expunge Both SyncState *
Then mu stores can be initialized with commands like mu
init --muhome=~/.mu/thunix --maildir=~/Maildir/thunix
--my-address=defanor@thunix.net.
This was a sufficient setup to listen to a radio (vlc
'http://s3.radionetz.de/1a-rock.mp3'; as of 2025-10-27
and 2026-01-14, that is blocked here, along with many CDNs and
hosting companies, some of the alternatives are vlc
'http://113fm.cdnstream1.com/1740_128', vlc
'https://s8.yesstreaming.net:7099/RblLgn',
see dir.xiph.org for other online radios), local music
collection (which I keep on a separate partition, so just
mounted it via fstab into the same path as before,
and the playlist also stored on it contained correct paths),
communicate (IRC, XMPP, email), do Haskell programming, browse
WWW relatively comfortably, play Discworld MUD over telnet, and
publish these notes. At that point I've adjusted dwproxy to be
able to build it using only dependencies from the system
repositories (for related rants and musings, see the notes
on software packaging and deployment and everyday programming in
Haskell), and built a few work projects: since it's Cabal 3 now,
had to set cabal.project in order to use internal libraries, and
made some other minor adjustments to handle newer versions of
dependencies. C projects (rexmpp in particular) also required
minor adjustments to handle newer versions of the compiler and
libraries, but fairly straightforward.
Realtime Policy and Watchdog Daemon (rtkit) can be quite spammy
in the logs with its debug messages, but that can be fixed by
overriding its systemd service (sudo systemctl edit
rtkit-daemon.service, followed by sudo systemctl
daemon-reload and sudo systemctl restart
rtkit-daemon.service to apply it) with the following:
[Service] LogLevelMax=info
Following the instructions (Chapter 4. Upgrades from Debian 11
(bullseye)), I executed apt full-upgrade to
find out that my graphics card (GTX 660) is not supported by the
NVIDIA proprietary driver anymore. Chose to not install the new
nvidia-driver, but that interrupted the process, so had
to apt --fix-broken install, and then apt
full-upgrade again. Afterwards
removed nvidia-driver,
chose mesa-diverted in update-glx --config
glx in order to de-blacklist nouveau drivers, rebooted,
the system only worked for some minutes before freezing,
rendering it unusable. Fortunately I have integrated graphics
here (Xeon E3-1275 v2 on ASUS P8C WS), which I picked precisely
because this sort of thing keeps happening; took the graphics
card out, connected the display to the motherboard's DVI
output. Apparently I disconnected the system disk while taking
the graphics card out, so failed to boot; then reconnected it,
and saw it via UEFI, but failed to boot still, with different
priorities (possibly messed up the UEFI boot settings while
poking them without the disk connected properly). Managed to
boot into the system by booting grub from a live USB stick, then
pointing it to the system's grub.cfg using grub shell's
configfile command. Tried to fix it with
efibootmgr, that did not work, but it worked to just
do grub-install and update-grub,
leading to a working system into which I can boot directly,
albeit without a graphics card. See GrubEFIReinstall for more
options.
Additionally, some texlive packages failed to update, and some fcitx5 ones were kept back.
Afterwards I did apt autoremove, which removed
telnet, so had to apt install telnet again.
mu4e broke as well: had to update mu4e-alert via Emacs, since it
came from melpa, but then it kept failing with "Mu server
process ended with exit code 1". Dug the approximate command out
of the sources (/usr/bin/mu server --debug
--muhome=~/.mu/thunix), executed it manually, saw the
error message: "error: expected schema-version 465, but got 451;
cannot auto-upgrade; please use 'mu init'", "Please
(re)initialize mu with 'mu init' see mu-init(1) for
details". Did mv ~/.mu/ ~/.mu-old/, then mu
init --muhome=~/.mu/thunix --maildir=~/Maildir/thunix
--my-address=defanor@thunix.net (and similar ones, for
other mailboxes), and then it worked. As many other programs,
mbsync deprecated "master/slave" terminology, introducing its
unique alternative: "far/near".
Had to M-x customize-group RET ansi-colors RET,
since ansi-color-names-vector became obsolete.
I had an unused PostgreSQL 13 (used primarily for local
testing), and PostgreSQL 15 was installed by the system upgrade,
so I just cleaned up the old version: sudo pg_dropcluster
--stop 13 main, sudo apt remove
postgresql-13 postgresql-client-13.
Then I was left with a bunch of other "installed,local" packages
(apt list '?narrow(?installed,
?not(?origin(Debian)))'), so cleaned some of those up,
after checking that they do not seem to be necessary: sudo
apt remove haskell-platform gcc-10 gcc-9-base gcc-10-base
clang-11 python-numpy-doc openjdk-11-jre openjdk-11-jdk
openjdk-11-jre-headless openjdk-11-jdk-headless libx264-160
libx265-192 libwebp6 libvpx6 libswresample3 libssl1.1 libsepol1
firmware-intelwimax linux-image-5.10.0-8-amd64
linux-image-5.10.0-23-amd64 iukrainian libffi7 libbpf0
libprocps8.
Had to use a workaround for the FBReader's hyphenation-after-each-word bug.
It is handy to host servers locally, particularly for communication: they are always available from the primary system then, the latency is reduced, regular TLS allows for peer-to-peer connections. As a downside, issues with the primary system also lead to downtime of those.
Eventually I decided that having a properly configured XMPP
server locally is useful as a backup, for lower-latency calls,
and to decrease load on remote servers. Having just an A record
pointing to my static IP address (a free dyndns service in this
case, to avoid dependencies on domain names at once), and port
forwarding configured on the router for ports 80, 5222, 5269,
5281, 3478, 49152-49155, I have set nginx and uacme to obtain an
X.509 certificate for TLS, configured nftables to decrease spam
in the logs (only accepting connections on port 80 when renewing
a certificate), then configured Prosody and coturn. sudo
apt install nginx uacme nftables prosody
coturn. My /etc/nftables.conf, slightly
abridged to focus on relevant parts:
#!/usr/sbin/nft -f
flush ruleset
table inet filter {
set not-clients {
type ipv4_addr
flags interval
elements = { 1.0.0.0/8 }
}
set blocks {
type ipv4_addr
flags interval
elements = { 1.1.1.1 }
}
set open-ports-s2s {
type inet_service
flags interval
elements = { 5269 }
}
set open-ports-c2s {
type inet_service
flags interval
elements = { 5222, 5281, 3478, 49152-49155 }
}
chain input {
type filter hook input priority 0; policy drop;
# Mitigate TCP reset attacks performed by the ISP.
ip saddr @blocks tcp sport 443 tcp flags rst drop;
# Allow traffic from established and related packets.
ct state established,related accept
# Allow loopback traffic.
iifname lo accept
# Allow incoming TCP and UDP packets on @open-ports-s2s.
tcp dport @open-ports-s2s accept;
udp dport @open-ports-s2s accept;
# Drop connections from spammy addresses.
ip saddr @not-clients drop;
# Allow incoming TCP and UDP packets on @open-ports-c2s.
tcp dport @open-ports-c2s accept;
udp dport @open-ports-c2s accept;
}
chain forward {
type filter hook forward priority 0;
}
chain output {
type filter hook output priority 0;
}
}
Then set /usr/local/bin/uacme-hook.sh,
modifying /usr/share/uacme/uacme.sh:
--- /usr/share/uacme/uacme.sh 2023-02-15 23:31:43.000000000 +0300
+++ /usr/local/bin/uacme-hook.sh 2024-01-30 09:49:06.505761694 +0300
@@ -16,7 +16,7 @@
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
-CHALLENGE_PATH="${UACME_CHALLENGE_PATH:-/var/www/.well-known/acme-challenge}"
+CHALLENGE_PATH="${UACME_CHALLENGE_PATH:-/var/www/html/.well-known/acme-challenge}"
ARGS=5
E_BADARGS=85
@@ -37,6 +37,8 @@
case "$TYPE" in
http-01)
printf "%s" "${AUTH}" > "${CHALLENGE_PATH}/${TOKEN}"
+ # Temporarily allow connections to port 80
+ sudo nft add element inet filter open-ports-s2s {80}
exit $?
;;
*)
@@ -48,7 +50,10 @@
"done"|"failed")
case "$TYPE" in
http-01)
+ sudo nft delete element inet filter open-ports-s2s {80}
rm "${CHALLENGE_PATH}/${TOKEN}"
exit $?
;;
*)Then:
sudo mkdir -p /var/www/html/.well-known/acme-challenge
sudo mkdir /etc/prosody/certs/example.com/
sudo touch /etc/prosody/certs/example.com/{fullchain,privkey}.pem
sudo chmod 640 /etc/prosody/certs/example.com/{fullchain,privkey}.pem
sudo chown root:prosody /etc/prosody/certs/example.com/{fullchain,privkey}.pem
sudo uacme -v new
sudo uacme -h /usr/local/bin/uacme-hook.sh issue example.com
sudo -e /etc/cron.daily/uacme-cert-update
sudo chmod +x /etc/cron.daily/uacme-cert-updateWith the following in /etc/cron.daily/uacme-cert-update:
#!/bin/sh set -e /usr/bin/uacme -h /usr/local/bin/uacme-hook.sh issue example.com cp /etc/ssl/uacme/example.com/cert.pem /etc/prosody/certs/example.com/fullchain.pem cp /etc/ssl/uacme/private/example.com/key.pem /etc/prosody/certs/example.com/privkey.pem
In /etc/turnserver.conf I have only set external-ip, static-auth-secret, use-auth-secret, max-port=49154.
Relevant lines of /etc/prosody/prosody.cfg.lua:
interfaces = { "192.168.1.8", "127.0.0.1", "::1" }
modules_enabled = {
--- [...]
-- Other modules
"turn_external";
"http";
}
-- TURN
turn_external_host = "example.com"
turn_external_secret = "secret here"
-- HTTP
http_host = "example.com"
VirtualHost "example.com"
Component "upload.example.com" "http_file_share"
Then restart or reload the services, add users with sudo
prosodyctl adduser <jid>, and it works.
For voice conferences, apparently a particularly easy to set and
properly working option is Mumble. sudo apt install
mumble-server mumble, set a password
in /etc/mumble-server.init, open UDP and TCP ports,
and it is ready to use with desktop clients or Mumla or Android.
Similarly to XMPP and voice conferences, one may set an IRC server (or a small network) for private chatting. InspIRCd is available from Debian repositories and easy to configure, simply by setting the desired hosts, names, and passwords in its configuration file. And links (the spanningtree module) for use with multiple servers. Anope IRC services seem popular, and also available from Debian repositories, but perhaps unnecessary for a small private (and possibly local) network. To make it available over Internet, one may want to both enforce TLS and add restrictions for those connection classes; to do so, one may define a single connection class allowing no connections, then inherit one for plain connections, and one for TLS connections on a different port (corresponding to the Internet-facing endpoint), with additional restrictions (e.g., requiring a password).
If a machine is shared among multiple users, one may prefer to encrypt home directories, or at least subdirectories within those, individually in addition to the block device (LUKS/dm-crypt) encryption. That can be done with fscrypt, eCryptfs (an older option; also other stacked file systems). For instance, to create an encrypted directory:
# with fscrypt # Enable and check the "encrypt" feature for the target ext4 file system sudo tune2fs -O encrypt /dev/sda1 sudo dumpe2fs /dev/sda1 | grep features # Install fscrypt and its libpam module at once sudo apt install fscrypt libpam-fscrypt # Setup fscrypt for the root partition (globally) sudo fscrypt setup # Create and encrypt a directory mkdir private fscrypt encrypt private/ # with eCryptfs sudo apt install ecryptfs-utils # Load the module sudo modprobe ecryptfs # Load it on boot as well echo ecryptfs | sudo tee /etc/modules-load.d/ecryptfs # Setup a private directory, in ~/Private/ ecryptfs-setup-private # Mount it ecryptfs-mount-private
See ecryptfs-migrate-home(8) for encryption of the
whole home directory.
It is easy to be annoyed by the need to eat, which generally takes more effort than other human physiological needs. But it can also be a source of joy, and as with many other areas, trying to avoid learning about it leads to poor and/or overpriced results. I consume food regularly, so decided to write down some notes on it.
There are healthy diet guidelines issued by WHO and some governments, which seem sensible: they recommend to consume a lot of fruits and vegetables (at least 400 grams per day), some cereal, meat/fish/eggs (apparently mostly as protein sources, though there are other sources as well; usually it is recommended to consume at least 0.8 g of protein per kg of body weight daily, but for certain exercise types some recommend to increase it, up to around 2 or 3 grams per kg), and dairy. They also recommend to reduce sugar, salt, and butter consumption. If you mix together and heat up sugar, salt, and butter, you get toffee or similar confections (depending on temperature and other minor parameters). Saturated fat and alcohol are also recommended to avoid, and physical activity is recommended by some of those (despite not being directly related to food).
Humans evolved to survive on available food in bad conditions, so it is not a big deal if proportions are imperfect: eating edible foods you are not sick of should keep you alive. Trying to balance them and not completely ignoring some of the food groups should be pretty healthy.
Apparently it is still unclear how useful dietary supplements (multivitamins and others) are, assuming a relatively healthy and diverse diet. Speaking of things with little evidence and a lot of marketing or followers, there also are "organic food", GMO conspiracy theories, fad diets. So beware of dubious marketing, if you try to stick to evidence/science/studies (though there is plenty more of controversy around topics like that, even around the studies: the ones people are commonly interested in, but which are tricky to study precisely).
For calculations of food nutrition, there are Recipe Nutrition Calculator and Food Calculator; for estimation of the needed calories, there are Calorie Calculator and another Calorie Calculator (which includes estimates of macronutrient needs). And a similar Macro Calculator (with more calculators on that website). Plenty more information at ExRx.net - Diet and Nutrition.
Cooking (and even eating) may be seen as a chore and inconvenience, but to many even computing does look that way, and in both cases it is possible to enjoy the process, making it a hobby. There are hacks one may attempt to reduce bothering with food (meal replacements, eating out, relying on others to cook in general), but the results seem similar to people avoiding bothering with computers: reluctance to put a bit of time and effort into learning leads to poor results and continuous frustration, while after learning the basics it is rather fun to tinker. Also as with computers, maths, and likely most of the other activities, it can be intimidating at first, but experience builds confidence and makes it easier. For increased inspiration and motivation, there are plenty of cooking videos available online. It does take some time though.
As an additional benefit, the breaks taken to make a coffee or cook something may count for the breaks commonly suggested to heavy computer users, to avoid just sitting all day long (with potential adverse effects on one's health).
To speed up both cooking and dishwashing, one can cut some corners, especially after learning which ones are okay to cut: perhaps cut vegetables into less consistently-sized and larger chunks, roll out the dough less carefully, cook faster at a higher heat, do not wash dishes overly thoroughly. But one should still be careful around the dangerous stuff found in the kitchen: contamination (from meat, eggs, etc), flame, hot oil, knives.
A nice kitchen setup and handy utensils are important for happy cooking. One of the important aspects (as with most of other manual labour) is lighting: see my notes on lighting for that.
The situation with cookware is similar to that with hand tools: odd brands hiding the companies/manufacturers behind them, sometimes pretending to be German, or local to a place they are sold at, while being Chinese or Indian. With established brands there is at least a hope for quality control, even if they are manufactured in China too. There are occasional stores that pick nice cookware/brands before selling it. And it wouldn't harm to investigate which utensils you need, which kinds of those exist, the properties of materials they are made of.
Some of the seemingly fine brands available here are Zwilling, WMF, Arcos. Possibly also Le Creuset, Beka. Vitrinor, Brabantia.
Common advices in articles and videos on kitchen setup are straightforward: keep the items you need often easily accessible and easy to fetch: pots and pans hanging (or otherwise fixed in a rack) instead of being stacked and stashed somewhere, knives on a magnetic bar, oils and spices visible and at hand, not hidden in a closed cabinet. While the unused stuff shouldn't occupy valuable space.
Much of food preparation is about slicing and chopping vegetables, which can be time-consuming. To speed it up, it's a good idea to learn how to hold and guide a knife properly, as well as to keep it sharp. Here is a couple of videos demonstrating the usage basics: "Basic Knife Skills", "The Only Knife Skills Guide You Need". For basic sharpening and re-aligning, see "How To Sharpen Dull Knives".
European-style knives are made of softer steel, making them easier to sharpen. while Japanese ones are harder (and harder to sharpen too). It's generally suggested to avoid buying knife sets/blocks, since they have some knives you will not use, while taking space.
Other interesting bits of information and references on the topic are available from the Hacker News thread on "Forming an Edge".
There is a Wikipedia article on cookware and bakeware. The most common materials:
I am mostly using stainless steel ones (without non-stick surfaces), with cast iron only for searing. But searing is possible with stainless steel, too, and one can use a single stainless steel saute pan (or saucier; a relatively deep or tall pan) for pretty much any cooking needs.
As with most other items, picking established manufacturers seems to be a good strategy.
There are various ways to discover new dishes, including Wikipedia's articles on regional cuisines (e.g., Italian cuisine, Spanish cuisine, American cuisine), websites with recipes, cooking shows, and just mentions during conversations, in books, and in movies. I found it useful to not dismiss dishes and food items based on distaste for local, cheap, and/or poorly cooked versions of those, since they can be very different. That applies to store-bought dishes as well, including confections: apparently many of the odd and unpleasant ones are just unsuccessful attempts to reproduce good ones.
Along with dishes themselves, meal structures are interesting to learn and experiment with: see outline of meals and Italian meal structure, for instance. And there's usually plenty to learn about each ingredient individually (which helps to pick more suitable for a given dish or otherwise better ones); as with many other things, Wikipedia is a good starting point, and then one can proceed to reading past online discussions and/or trying and discussing them online.
Discovering that odd local foods and beverages you have never liked are not consumed anywhere else, and one can live without them, is another nice possibility.
Some websites and databases with recipes: Wikimedia Cookbook, Supercook (handy recipe search based on available ingredients), SimplyRecipes, Allrecipes, Foods Guy, Public Domain Recipes.
It's useful to look closely into every ingredient: learn about their types, find reputable brands, try and find out which ones one prefers, and possibly even order them from specialized stores.
Rather often authentic ingredients will not be available (particularly in Russia, due to the import substitution), but then it is still better to substitute them and improvise than just to give up without trying.
There are books on cooking, including Wikimedia Cookbook for recipes, and On Food and Cooking, with nice and useful explanations of the common cooking processes (as well as some history and other bits), but plenty of useful knowledge can be absorbed from just cooking videos with explanations (though in a less systematic way). The Sourdough Framework is a seemingly nice book focusing on sourdough bread baking, though not so much on yeast-based bread baking. Here are some YouTube channels with nice recipes and occasional explanations:
There also are shows not strictly related to learning how to cook, but food-related and interesting and/or entertaining: Townsends often covers 18th century cooking practices, Tasting History with Max Miller presents recipes from more places and periods, Weird History and other history-related channels cover topics such as historical diets and history of various food products, Food Factory and standalone documentaries demonstrate (though usually only cursorily, not getting into industrial and manufacturing engineering) food mass manufacturing processes.
There are occasional nice websites dedicated to cooking around, such as National Center for Home Food Preservation, Serious Eats.
The Sad Bastard Cookbook: Food you can make so you don't die looks amusing, though as the title implies, the recipes are not particularly exciting.
I find it useful to compose a menu of the dishes you like and know how to cook. Below is mine, with meals grouped by complexity and cooking time.
The following dishes don't require any heating, or even much of cutting, and are mostly about putting things together (or just eating them whole):
The following dishes are comparable in complexity to brewing tea/coffee/cocoa, requiring just a few ingredients (hence little planning), little skill or action, and not much of attention:
Many of these can be used as bases to build upon: leftovers and fruits/vegetables can be added into soups (including pre-mixed ones), on top of frozen pizzas and lasagnas, into scrambled eggs, and into other dishes. Relishes are easy to add to some of those. Stocks/broths (either store-bought or homemade) are useful for soups, needed for risotto, and one can make sauces with them.
The following dishes require some planning to get and use the ingredients, some timings and attention, but still nothing fancy and not too easy to mess up:
While these take longer to cook, many of them can be stored in a refrigerator and re-heated for a few days, so the cooking time per meal is not long.
In addition to reducing the number of grocery store trips, planning (or sufficiently good improvisation) may help to spend less time cooking by preparing multiple meals at once, as well as to get nicer meals (with stocks, possibly sauces, and other homemade ingredients one may prepare separately and/or for multiple dishes at once). Materials on the topic can be found using the "meal prep" keywords.
When planning goes a bit wrong and mold appears, generally soft, liquid/moist, or porous foods should be discarded, while hard/firm ones may be kept after cutting out the moldy bits. See "Molds on Food: Are They Dangerous?", "Moldy Foods: When to Toss, When to Keep".
Some of the dish groups suitable (and commonly used) for meal prep are soups, containers with rice, some protein, vegetables, and sauces (refrigerated or frozen), and things like burritos, possibly frozen.
For cut or chopped vegetables, USDA's "How should I store cut fruit and vegetables?" suggests to refrigerate cut fruits and vegetables in covered containers; "So Fresh and So Clean: How to Store Cut Vegetables" and "Meal Prep Guide: How to Store Prepped Vegetables" are more detailed guides to refrigerating and freezing various vegetables.
For whole vegetables, I think generally one can see how they are stored in a grocery store, and store them similarly.
FoodSafety.gov's Cold Food Storage Chart is a handy general table.
While a lot of fruits and vegetables should be consumed, and many of the fresh ones have a rather short shelf life, an additional difficulty for me is that I have rather unpleasant reaction to biting some of their skins (goosebumps and brief toothache), so peeling is needed. Some also peel them simply because they don't like the skins, and in some cases those are not quite edible.
Blanching (putting food into boiling water, and optionally into cold water afterwards) sometimes makes skins easy to peel; works with tomatoes and peaches.
A process for pepper peeling is somewhat similar to blanching: it's easier to peel after a few minutes in an oven (or a grill, or rotating on an open fire) and 5 to 60 minutes in an airtight container (which is supposed to produce moisture under its skin). Apparently some people (those cooking chiles en nogada), using certain varieties of peppers, manage to carefully stuff them after skinning, but at least with bell peppers I found it to be very difficult, tiring, and time-consuming to peel a pepper without it falling apart, whether with a vegetable peeler and knife, or open fire or an oven, steaming, and knife. A much better idea is to go slightly less fancy and just make a casserole, with pepper (if you want it) simply chopped into the mix: then you don't care if it falls apart.
A vegetable peeler is handy for carrots and cucumbers. A bit less handy (but quite suitable) for potatoes. One can use it for pepper too, but for bell pepper it works better to cut it before peeling, so that there's no concave bits inaccessible to the peeler.
Pretty much everything can be peeled (skinned) with a knife, possibly leaving a bit more waste and/or taking a bit longer than with the alternatives.
For standalone snacks, apples (with a tough skin) can be replaced with pears. Also some apples are much softer than others: ripe (yellow) Golden Delicious is among nice ones.
Overcooking meat, poultry, or fish makes it tough and dry, yet it's pretty common, while undercooking is unsafe. So it's a good idea to use a thermometer, possibly to employ techniques that make it easier to reach and sustain desirable temperatures (that is, cooking longer, but at lower temperatures: poaching, sous vide). See Safe Minimum Internal Temperature Chart, "Keep food safe with time and temperature control". It should also be cooled quickly, and sometimes time can be traded for temperature. Here's a copy of the USDA chart, since I'm checking it often, and it'll save a click:
Somewhat related are tips on thawing (e.g., How to Defrost Fish): it's suggested to defrost in either a refrigerator or in cold water (if it has to be done quickly), but not at a room temperature.
Generally 1 cup approximately equals to 284 ml (though it can be 236 or 240 ml for an US cup), 1 tablespoon to 18 ml (15 ml for US ones), 1 teaspoon to 6 (5 for US) ml. Baker percentage is both handy and important for precision (along with usage of weights for flour), but often it is not used, and then an ingredient weight chart may be useful: 1 cup of all-purpose flour may be about 120 grams, for instance. Additionally, Imperial units are often used in recipes.
While survivalism is not my hobby, the end of 2024 in Russia feels like uncertain times, and it may be useful to be at least a little prepared for various eventualities, including food shortages, as those happen in similar situations sometimes; likely it will be too late to start preparing once they will begin.
Some lists: Long Term Food Storage List: 26 Foods with a Long Shelf Life, 32 Long Shelf Life Foods to Keep In Your Pantry, Long Shelf Life Foods – What Lasts Best, Long Life Food: A Comprehensive List of Shelf-Stable Essentials, 22 Foods That Last Up To 25 Years To Stockpile. The commonly suggested items for very long storage (on the order of 10 years) are salt, sugar, honey, white rice, popcorn, freeze dried fruits and vegetables, vinegar, corn starch, soda, instant coffee, powdered milk and eggs, dried beans and legumes, pasta, flour, whole grains, low-fat crackers, maybe rolled oats, buckwheat, canned foods. Proper storage is important, and much of that is commonly suggested to seal in Mylar (polyethylene-laminated aluminized BoPET) bags with oxygen absorbers to achieve a longer shelf life (see Oxygen Absorbers Recommended Amounts). Though real Mylar bags seem tricky to find in some places, and one may have to look into other options: food storage buckets, glass jars.