Category: Tech

So, unwisely, I thought I’d run apt -t bookworm-backports upgrade on a system here at home to grab all the available backports (note to future me: be more selective about this). This had worked on several other machines but failed on this one because of a missing libsystemd-shared.252.so file needed by systemd-machine-id-setup. I’m not immediately sure why (maybe I had never landed on that version on this host?) but wanted to share the solution that worked for me: copying the missing so files from another, as yet-unupgraded, machine’s /usr/lib/x86_64-linux-gnu/systemd dir. That lets systemd-machine-id-setup work for the apt upgrade run.

In case it helps anyone else later searching for this, the error message was:
Setting up systemd (254.26-1~bpo12+1) ...
systemd-machine-id-setup: error while loading shared libraries: libsystemd-shared-252.so: cannot op
en shared object file: No such file or directory
dpkg: error processing package systemd (--configure):
installed systemd package post-installation script subprocess returned error exit status 127
Errors were encountered while processing:
systemd

Gitea is awesome for self-hosting git repos. One of the neat features it has is package repository hosting, including for debian (their docs on this in general). For debian stable/bookworm/12, this works seamlessly at least in my experience.

Things get a bit hairier once you start using testing (trixie/13 at present). The root problem is that the go pgp library used produces signatures that the new apt tooling in 13 does not like, specifically in regards to signature verification. See here for an example bug report / fix. “apt update” from a testing machine will fail with ugly mumbling along the lines of “OpenPGP signature verification failed, sub-process sqv returned an error code (god hates you, everything is ruined forever) , error message is demonic howling, signing key unknown, E_FUCKED, no binding signature on this shit, boss!”. (God help you if google search lands you here on those keywords.) Gitea 1.24 contains newer dependencies that should fix this. BUT! The debian package repository signing key material has to be regenerated. And that is where the fun starts, as there’s no way to do this via web ui or cli as far as I can tell.

This process seems to do the right things, but it is admittedly a big hassle:

1. upgrade to 1.24.latest if not already done
2. delete all package versions/files through web ui. there should be no debian repo listed in the packages page
3. open up your database and delete the debian.private.key and debian.public.key rows for your user in the user_settings table (whether sqlite, postgres, whatever)
4. (I also got paranoid here and went on a cleaning sweep for all things “debian” in the package* tables as there were remnants despite there being no visible listing for a debian package repo after the delete package step above. I have not verified if this step is required, but it doesn’t seem to hurt per se.)
5. this part is important: restart the gitea service
6. re-upload your package file(s). you should notice the upload take a hair longer, as it’s regenerating the repository.key et al.
7. to verify, curl https://your-gitea-host:3000/api/packages/yourusername/debian/repository.key -o test, gpg –show-keys test. the dates shown should be the current day if everything worked.
8. you’ll have to re-download the repository.key into /etc/apt/keyrings/gitea-yourusername.asc (or wherever you put it, and matching the signed-by bit in the list file establishing gitea as a repo debian can draw from
9. apt update / upgrade should now work

You might be fooled, as I was, that the gitea command line “admin regenerate keys” command might do something useful to solve this, but alas it does not.

I’m always a bit nervous kicking off remote server OS upgrades, but shoutout to the Debian team for making this another smooth one from 11/bullseye to 12/bookworm. The only quasi-glitch I encountered was that the apache php module wasn’t installed automatically during the upgrade, but that took all of 30 seconds to fix.

Since TP-Link’s documentation is *awesome* for this (sarcasm alert), I thought I’d share what I finally figured out to get the self-signed cert replaced with one from a local CA that doesn’t make Chrome complain. This was the result of several hours of fiddling around, waiting for oc200 reboots, and getting uber-helpful error messages from the device (sarcasm meter explodes).

Pre-reqs:
1. passing familiarity with openssl command-line usage, including how to set up a local certificate authority (CA; out of scope for this post/rant but this looks decent as an intro: https://gist.github.com/Soarez/9688998)
2. the patience not to see how far you can shot-put the oc200 device
3. some machine with openssl installed (I used a linux machine running ubuntu 20.04 fwiw)

First, you’ll want to create a config file to save typing later and enable Subject Alternate Names for your cert (so the same cert’ll be valid from the controller raw ip, or name, or short name). Call it san.conf or similar.

[req]
req_extensions = req_ext
distinguished_name = req_distinguished_name
prompt=no

[req_distinguished_name]
countryName                     = 
stateOrProvinceName             = 
localityName                    = 
organizationalUnitName          = 
commonName                      = 
emailAddress                    = 

[req_ext]
subjectAltName = @alt_names

[alt_names] 
IP.1=192.168.5.2
DNS.1=oc200.lan.example.com
DNS.2=oc200

Now, on to making the actual cert (filenames may of course be altered to your taste, just be consistent):

1. openssl req -new -keyout oc200.key -sha256 -config san.conf -out oc200.csr
a. note that you probably want a password here (used by key).
b. if you’re ok with a less secure key sans password, add a -nodes argument above

2. openssl x509 -req -days 365 -in oc200.csr -CA -CAkey -CAcreateserial -extensions req_ext -extfile san.conf -out oc200.crt
a. note that recent chrome builds are moving to deny validity of certs with longer than one year. (imho this is overkill for rfc1918 networks, but c’est la guerre.)

3. openssl pkcs12 -export -in oc200.crt -inkey oc200.key -out oc200.pfx -CApath /etc/ssl/certs/ -CAfile -caname root -name oc200 -chain
a. note that you probably want to use a password here as well
b. note also that the Omada web ui is picky about filename extensions. you’ll want to end your pkcs12-exported cert to end in “.pfx” to keep it happy on upload later

From here, with the pfx file and (optionally, but recommended) the key and pfx passwords, you can proceed to Settings > Controller > HTTP Certificate session, upload your pfx file, fill in any required passwords (be sure to pay attention to which password is which re: key vs cert file aka keystore), and save at the bottom of the screen. You will then want to go to Settings > Maintenance > Hardware Controller section and reboot (this will take several minutes to complete).

You may also need to do a full flush of your browser’s cache if there were earlier attempts with the same identity cert (e.g. accidentally making it valid for too long and learning a painful lesson). You will need to import the root CA cert into your browser/OS trusted roots collection as well if you haven’t already done so (out of scope for this, but googling something like “import root ca cert ” would help).

Hope this saves someone a few hours of irritation. :)

After five years at Oracle, I got thoroughly fed up with BigCo bullshit and have switched to an early-stage startup at a friend’s recommendation. This will be the fourth company I’ve worked with him at over the last ~20 years. It is *so nice* to get back into days that are filled with code instead of meetings, and with a tech stack I like to boot (java, linux, postgres, AWS) as well as some tasty new things to explore. (I typo’d that as explode at first and let’s be real, that’d work too for technology.)

oldstable to stable, not much to see here, move along :)

pc.o is now on debian buster after being on stretch for a while. no underlying hardware changes this time around.

Updating from oldstable to stable (what finally provoked me to get this done was oldstable reaching EOL status, heh). No major changes since last “pc.o update” post on the software side. I did end up dropping down to a $5/mo plan vs $20, just because honestly it’s all the resources this really needs (one core, 1gb, ~20gb/disk). As always, I know not many other than me use this server, but if you notice anything odd, feel free to reach out.

(Minor update: turns out I had a free upgrade to 25gb of disk from 20gb pending in linode; applied that. Whee!)

An ocean of ink has been spilled about interviewing for jobs in the technical industry (see, just for example cracking the coding interview or get that job at google). I’m probably not going to cover any new ground here, but these are some basic tips for doing better on technical interviews.

This post was inspired by my repeated recent frustration as a hiring manager watching candidates that had potential (or even proven prior performance!) do poorly at the in-depth interview loop phase of the process. If every candidate took fifteen minutes to read and internalize these points in addition to however many hours they spend self-flagellating with Cormen et al’s algorithms tome, I am guessing the pass rate might double. Interviewing is not like regular work (for better or for worse), and competency at the latter does not a priori translate well to the unnatural constraints of you + interrogator + whiteboard.

• When you’re practicing for an interview that will involve whiteboard time, write out your solutions long-hand on paper first, and make a note of what you have to google to remember. Not that an interview will be passed or failed based on remembering all the printf format specifiers or whatever, but it doesn’t hurt to get them right either.
• Ask questions before you dive in! (I would stipulate “good questions” but really, anything more than silence that’s on topic would be better than nothing.) Many many many times a problem can be dramatically simplified by asking clarifying questions or stipulating a reasonable constraint. No interviewer will hold this against you; on the contrary, it’s usually regarded as a small but very positive signal for the candidate’s competency.
• Think out loud as much as you can. Remember that we’re not just looking for banged out code or whatever, but trying to see how you think. Thinking is 90% of technical work, the rest is just typing and irritating planning meetings. If nothing else, outline the solution that comes to mind at a high level and ask if that sounds reasonable before diving into implementing it. That alone can save you many precious minutes from being wasted by barking up the wrong tree.
• Post-solution follow up work: after you’ve banged out an implementation, take a moment to talk — even at a footnote level — about efficiency / avenues for possible improvement / how to test your work. For example, in a code interview, take a moment to walk through a very simple test case. These are hugely positive signals that the candidate is driven to do good, thorough work and possesses initiative/responsibility/etc.
• Take a deep breath and relax. :) I know (I know) this is maybe the hardest element of this list, because you’ve got all this stuff riding on whether or not you pass the interview, and rightly you’re a bit stressed out with all of it. But your brain will work better if you aren’t in fight-or-flight mode. Try to, as much as is possible, relax and view interview sessions as a friendly chat to solve a problem between colleagues rather than an Inquisition. Good interviewers will make allowances for nerves, but still, it goes a lot easier if you chill out a bit about things.

Good luck!

Now that oldstable is oldoldstable, I bit the bullet and upgraded to jessie from wheezy. The upgrade process was surprisingly smooth. Systemd… well, many megabytes of flames have been written about that before. There isn’t much I can add to it other than a) the idea still disturbs me on some level b) pragmatically, it seems to work ok so far c) I have so many other things to worry about.

Since Debian just dropped a new stable version, I’ll probably upgrade to that in a while (~ a quarter or so, or whenever the first patch release drops) to give fresh bugs time to shake out.

No system capacity changes with this update. When pc.o first migrated to linode, I don’t remember for sure, but I think it was something like 512mb ram and (10? 20?) gb of disk. Now for the same price it’s 4gb of ram and 48gb of disk. The former felt a bit cramped at times, but now … I almost feel embarrassed to have so much capacity for what is fundamentally a personal fuck-around-with-shit server. I’d move to a smaller plan but it seems like a pretty big hassle potentially for a savings of maybe $10/mo.

I’m sure this is interesting to all like four or five of you who use pc.o for anything, but I felt obligated to keep the tradition of saying something about system updates alive. As always if you do use pc.o and notice something weird, let me know and I’ll try to fix it.