Why an old phone

I’d wanted to self-host a small project of mine — a bot I run for myself. The blocker was always the hosting. I didn’t want to rent a VPS and run it remotely, set it up, patch it, keep it alive; and I didn’t want to pay more for a managed platform to skip that. So it stayed an idea.

What I had was an old OnePlus 3T in a drawer: a 2016 flagship, Snapdragon 821, 6 GB of RAM, still working but not switched on in years. Throwing a working computer away felt wrong, and for one small always-on service it’s plenty of machine: a couple of watts at idle, its own battery, no monthly bill.

The catch is that it’s a phone. The bootloader is locked from the factory, it runs Android, the battery isn’t meant to live on a charger, and there’s no fan. The rest is how I dealt with each of those.

One prerequisite: this only works because the OnePlus 3T’s bootloader can be unlocked. Newer phones increasingly can’t be, so part of reusing an old phone is picking one that allows it.

Getting Linux on the phone

postmarketOS is the distribution I used: mainline Linux, systemd, Alpine packages — an ordinary Linux system, not Android.

If you only want a Linux userspace to experiment with, you don’t have to replace anything. Termux on Android, with proot-distro, gives you a Debian or Alpine environment on top of the stock system, no unlocking required. But it’s still Android, with all its downsides.

Unlocking, and a backup

The OnePlus 3T unlocks with fastboot flashing unlock — no code to request from the vendor, no waiting period. Unlocking wipes the device, so the first step was a full backup of the stock system: every partition, 56 of 56, 3.4 GB, with a manifest of names and sizes. I needed it — I restored from it later, when the first build wouldn’t boot.

lk2nd

On msm8996 the practical way to boot a mainline kernel is lk2nd, a small open secondary bootloader. The primary bootloader is Qualcomm-signed and stays in place; lk2nd installs alongside it. You flash lk2nd-msm8996.img to the boot partition; it loads at startup, gives a clean fastboot interface, and boots a mainline boot image from a 512 KiB offset (it occupies the first 512 KiB of the partition itself). pmbootstrap knows about it and writes the real boot image to the right offset automatically.

It hangs at boot

My first postmarketOS build hung on the lk2nd splash, every boot. No USB network device appeared on my Mac, the screen stayed on the splash, and there was no way to tell whether the kernel had panicked early or lk2nd had never handed off. Without a serial console you’re guessing, and the msm8996 is from 2016 — it predates the USB debug engine later Snapdragons have, so its only debug serial is a 1.8 V UART muxed onto the USB connector, reachable with a jig I didn’t have and didn’t want to build for this. I restored the backup, went back to Android, and looked for another way to see what was happening.

There are ways to get boot diagnostics without a UART: fastboot oem log dumps lk2nd’s handoff log — which DTB it found, what it jumped to — and the kernel can be told to keep its last log in pstore, so a hung boot can be read back afterward. I also tried the usual Qualcomm hang workarounds on the command line — clk_ignore_unused, arm-smmu.disable_bypass, console=ttyMSM0 — none of which changed anything.

The newest postmarketOS release had bumped a lot at once, including the kernel, from 6.3.1 — where it had been for years — to 6.19.5, and that release hangs the OnePlus 3T before initramfs. I didn’t isolate whether it was the kernel alone or something else new in the release; I dropped back to the previous channel, which still ships 6.3.1, rebuilt, and it booted. USB networking came up at 172.16.42.1, the framebuffer was alive.

It can’t mount the filesystem

Booting reached initramfs and stopped:

Trying to mount subpartitions ...
ERROR: failed to mount subpartitions

then dropped to a debug shell reachable over telnet at 172.16.42.1:23. The cause was the sector size of the flash. The phone’s userdata is a 4 KB-sector (4Kn) UFS device, but the image’s partition table had been written assuming 512-byte sectors, so the kernel couldn’t read the nested GPT and never created the subpartition devices. The fix is a single flag — pmbootstrap install --sector-size 4096 — that the OnePlus 3T’s device profile is missing even though its msm8996 siblings set it.

fastboot doesn’t write userdata

The last surprise was that fastboot flash userdata is a no-op on this device — it reports success and writes nothing, leaving the old filesystem in place. Flashing has to go through TWRP recovery instead, writing the image straight to the by-name block devices with dd (or simg2img for a sparse one). TWRP’s dd is a stripped-down build, so it wants byte sizes and choked on the conv=fsync I reached for.

After that: Linux op3t 6.3.1-msm8996 aarch64, postmarketOS, 6 GB of RAM, the root filesystem auto-resized to fill the partition, reachable over SSH.

Making the build reproducible

Doing all of that by hand once is fine. Doing it again after every change isn’t, and one of my rules was that every manual step should end up in a script. So I moved the whole flow into one script that runs pmbootstrap in a container and flashes from the Mac.

pmbootstrap doesn’t run on macOS — it needs a Linux chroot and loop devices — so it runs in a privileged Linux container. Docker Desktop on an Apple-Silicon Mac runs an arm64 Linux VM, so pmbootstrap builds the aarch64 image natively, with no QEMU in the path. The project directory is bind-mounted in, the built images land back on the Mac, and flashing happens from the Mac over USB.

Getting the script working meant fixing a handful of bugs.

The empty image

One build reported success, the flash verified byte-for-byte, and the phone dropped straight to the initramfs debug shell. The image was a valid GPT wrapped around 1.3 GB of zeros — about 271 non-zero bytes in the whole file.

Inside the container, pmbootstrap couldn’t create the loop-partition device nodes:

ERROR: Unable to find the first partition of /dev/loop0, expected it to be at /dev/loop0p1!

Docker mounts /dev as a tmpfs, so when losetup -P asks the kernel to create /dev/loop0p1, the node never appears in the container’s /dev. pmbootstrap writes the partition table but can’t create or populate the filesystems inside it. Mounting a real devtmpfs over /dev in the build container makes the nodes appear.

It had shipped as a success because of a second bug. The install ran as pmbootstrap … install | tail inside docker exec sh -lc '…'. The outer script had set -o pipefail, but that doesn’t apply inside the inner shell docker exec starts, so the pipeline returned tail‘s exit code — zero — and the real failure was invisible. The fix was pipefail in the inner shell plus a post-export check that greps the image for filesystem content and aborts if there’s none.

The backup GPT in the wrong place

Writing the content-sized raw image (about 1.3 GB) onto the 53 GB userdata partition leaves the image’s backup GPT header in the middle of the disk, where the image ends, instead of at the end of the partition:

GPT: Alternate GPT header not at the end of the disk.

The kernel rejects a GPT whose backup header isn’t where it belongs, so the subpartitions never map. The flash step now relocates the backup GPT to the end of the device after writing, then confirms the subpartitions appear.

The flash under load

The flash had a few more failure modes:

• The image is raw, not Android-sparse, so simg2img rejects it. The script detects the format by magic number and uses dd for the raw one.
• adbd drops during the multi-gigabyte write — failed to read copy response: EOF, and the device disappears from adb devices. So the writer runs detached on the phone (nohup), and the host polls a result file and reconnects adb if it drops.
• A push once passed a size check and still produced a non-booting system, so the script verifies by reading the written region back off the device and comparing SHA-256 against the local image, not just the length.

Server problems specific to a phone

The battery you cannot remove

A server stays plugged in, and a lithium cell held full and warm degrades faster for it. The obvious move is to take the battery out and run on the charger. You can’t, quite: the power-management chip expects a cell to be present, and without one the SoC’s brief current spikes sag the supply rail enough to reboot the device. (You can fake it with a dummy-battery board and a strong regulated supply, but that’s more hardware than I wanted.)

So the battery stays, as a buffer for current spikes and short brownouts — not for outage runtime; I don’t need the phone to ride one out.

For a buffer you want the charge held low, around 3.7–3.8 V, roughly half. The ability to absorb spikes is flat across the middle of the range, and a lower voltage means slower calendar aging and less swelling, so there’s no reason to keep it full. A small systemd service does this with a proportional loop: every 20 seconds it nudges the charger’s input-current limit so the net battery current stays near zero at the target percentage, and the cell just floats.

WiFi

The WiFi is a QCA6174 on PCIe, and on the mainline kernel the link wouldn’t train:

qcom-pcie 600000.pcie: Phy link never came up

The host bridge enumerated but the chip behind it didn’t, and Bluetooth — the same combo part — returned -110, a timeout. I went down the power-and-clocks path: the WiFi enable regulator was on at 1.8 V, PERST was released, the reference clock was running at 19.2 MHz. Everything in the bring-up sequence was correct, the link was still dead, and I concluded the chip was faulty.

Then I booted the stock Android kernel through TWRP, and it enumerated the chip immediately as [168c:003e]. The hardware was fine; the fault was in the mainline kernel’s PCIe bring-up. I tried a range of kernels to narrow it down: 6.3.1 and 6.12.10 fail the link the same way, 6.19.5 doesn’t boot at all. So it wasn’t a regression between two versions — it was a standing gap for this device.

The fix was one line on the kernel command line, on 6.12.10:

pcie_aspm=off pci=nomsi

ASPM is PCIe’s link power management; disabling it lets the link train. pci=nomsi forces legacy interrupts, working around flaky MSI on this SoC. With both:

ath10k_pci 0000:01:00.0: enabling device

wlan0 came up, scanned, and connected. Bluetooth started working again too — it’s UART-attached, not on PCIe, so the command-line flags didn’t fix it directly; it came back once the chip was being brought up properly. This pmbootstrap ignores the device profile’s kernel command line, so the working flags had to go into the Android boot-image header directly, at a fixed offset, rather than the documented config.

Network access

The bot occasionally needs to show me something over a web page — a one-time link it sends me in Telegram — so it needs an HTTPS endpoint I can open, and only I should ever reach it. I tried three approaches before keeping the simplest.

Tailscale is a mesh VPN. Its Funnel feature exposes a service to the public internet at a stable https://<name>.ts.net URL, with no domain and no static IP. (Cloudflare Tunnel does something similar but wants a domain, so I skipped it.)

Then I tried to shrink that public surface: bring the funnel up only while a freshly issued, short-lived link was outstanding, and drop it once the last one expired — the bot toggling the tunnel through Tailscale, with a small watcher tracking the live links. It works, but it’s a lot of machinery to gate something that never had to be public.

It never did. I use this alone, and the links are only for me, so tailscale serve is enough: it proxies a local port to https://op3t.<tailnet>.ts.net, reachable only by my own devices on the tailnet. Valid HTTPS, no domain, no port forwarding, nothing on the public internet, and no tunnel logic in the bot — serve is set once on the host and left on.

The trade-off is that the device I open a link from has to be on the tailnet too, so Tailscale runs on my phone as well, switched on when I need it. The earlier designs were attempts to avoid exactly that; it wasn’t worth the complexity.

Deploying the service

The service is a small Telegram bot — Node and TypeScript, with SQLite for storage. You’d normally build it into a container image and run that. On a phone, the build itself is the part to avoid.

The scarce resource here isn’t RAM — 6 GB is far more than the bot needs — it’s flash wear, CPU, and heat, none of which have a fan behind them. What spends all three is the build, not the running. Installing dependencies, compiling the native SQLite binding from source, and running the TypeScript compiler is sustained CPU on a passively-cooled phone, exactly the load it handles worst, and a “pull and rebuild on the device” deploy repeats it every time.

So none of it happens on the phone. GitHub Actions builds the image on a native arm64 runner — npm ci, the native compile, tsc, all on a real Linux machine in CI — and pushes it to a registry. The device only pulls the finished image and runs it. So containers are actually easier on the hardware here than running the bot bare: the one expensive step happens in CI, not on the phone.

For managing what runs, I used Portainer — a web UI over Docker, with logs, container state, and a registry browser. I’d assumed a web UI would be a constant tax, so I measured it: at idle it sits around 80 MB of RAM and roughly 0% CPU, which on a 6 GB device is noise. It’s convenient and costs little.

Deploying comes down to this: CI has built a fresh image, and Portainer pulls it and recreates the container from a small compose file.

Container networking

With the image building in CI and Portainer ready, deploying the bot should have been the easy part. It surfaced two more problems.

containerd will not start

Installing Docker on this systemd-based postmarketOS pulls in everything you’d expect — the engine, the CLI, containerd, the compose plugin, even the matching nftables rules. But the service wouldn’t come up. dockerd sat in activating (start) and never finished, and containerd was the reason:

systemctl status containerd
  ExecStart=/usr/local/bin/containerd (code=exited, status=203/EXEC)

203/EXEC means systemd couldn’t execute the file at that path — and it can’t, because there’s nothing there. The package installs the binary at /usr/bin/containerd; the service unit it ships points at /usr/local/bin/containerd, the upstream default. Because Docker only wants containerd rather than requiring it, dockerd didn’t fail outright — it waited for a socket that would never appear. A systemd drop-in correcting the ExecStart path is the whole fix.

Up but unhealthy

With Docker running, the bot’s container started and then sat there marked unhealthy. Its log printed one startup line and stopped, and wget http://127.0.0.1:8000/health returned Connection reset by peer. The port was published, but the app inside hadn’t bound it yet, so Docker’s proxy accepted the connection and immediately dropped it — the process was stuck before it started its HTTP server. Its first real action on startup is an outbound call, so I looked at the network.

From a throwaway container on the same compose network, the gateway was reachable and the internet wasn’t:

ping 172.18.0.1   → ok
ping 1.1.1.1      → 100% packet loss

I suspected DNS. nft showed otherwise:

nft list ruleset | grep masquerade
  ip saddr 172.18.0.0/16 ... masquerade ... counter packets 0 bytes 0

Zero packets through the masquerade rule means packets never reached it, so name resolution was beside the point — nothing was leaving the container at all. The host firewall was dropping it. Its forward chain defaults to drop and accepts forwarding for interfaces named docker*, but the bot runs on a compose network whose bridge is named br-<hash>, which nothing accepted. In nftables a drop in any base chain on the path is final, so the masquerade rule downstream never ran.

A second problem was behind it, and this one was DNS: the container’s /etc/resolv.conf had been handed 100.100.100.100, Tailscale’s MagicDNS, which is reachable from the host but not from inside a container. Even once forwarding worked, name resolution would have failed for that reason. Two fixes: an nftables rule accepting the compose bridges, and a public DNS server in Docker’s daemon config so containers don’t inherit the unreachable one. After that the bot reached Telegram, bound its port, and went healthy.

Performance and scaling

Once it was running, the question is whether a 2016 phone is up to the job. Portainer draws a CPU graph per container, and the bot’s is flat 0% at idle with brief spikes to 6–8%.

That doesn’t scale linearly. Most of those spikes are fixed work that doesn’t grow with users: a scheduled poll every few minutes, the Telegram long-poll cycle, the baseline cost of waking a Node process. The rest is per-request work, and it’s I/O-bound — the bot spends those milliseconds waiting on network calls, not computing — so a single event loop interleaves many of them without saturating a core. More users add a little of the variable part, not another copy of the fixed part.

The number that matters for a Node service is how busy its single thread is. Node runs the bot on one thread; the other three cores aren’t helping that process. That thread is barely busy, and the first limits it would hit are external — API rate limits, and SQLite’s single writer — well before the SoC’s CPU.

That’s also why the reflex fix for “single-threaded” — four copies, one per core — is wrong here, and would break this bot immediately. A Telegram bot on long polling allows one poller per token; a second one gets a 409 Conflict. And four processes would contend on the same SQLite file. Real horizontal scaling would be a different shape: an ingress that only receives and enqueues, a queue partitioned by user so one user’s actions stay ordered, a pool of stateless workers, Postgres instead of file-backed SQLite, and a single elected scheduler so each job fires once. None of that is warranted for one user. For the workload it has, the phone has plenty of headroom.

The whole system

Here’s how the pieces fit:

            GitHub Actions (arm64)
                   │ build + push
                   ▼
                 GHCR ──pull──►  ┌───────────────────────────┐
                                 │  OnePlus 3T               │
  me ──Tailscale (mesh)──────────┤  postmarketOS / systemd   │
   https://op3t.<tailnet>.ts.net │   └─ Docker               │
                                 │        ├─ Portainer (UI)  │
                                 │        └─ service (bot)   │
                                 │  battery-guard (systemd)  │
                                 └───────────────────────────┘

The phone runs postmarketOS — mainline Linux with systemd. Docker runs two containers: Portainer for management and the bot itself. The bot’s image is built in CI and pulled from a registry; the phone never builds anything. Everything is reachable only inside my Tailscale network, with nothing on the public internet. A systemd service holds the battery near half charge, and the container DNS, the firewall rule, and the rest are baked into the build.

Reproducing it

The whole thing is two repositories. The host side — the build-and-flash pipeline, the helper services, and the setup script — is at github.com/arttttt/oneplus3t-pmos-server.

Bringing up a fresh device is short, because the fixes are already in the scripts:

1. Flash the image — install.sh builds it in the container and flashes it, with the sector-size, GPT, kernel, and WiFi fixes built in.
2. tailscale up and log in.
3. Run setup-host.sh — it installs Docker, corrects the containerd unit, sets up container DNS and the firewall rule, brings up Portainer, and publishes Portainer and the bot over Tailscale.
4. Create the Portainer admin account.
5. Deploy the bot stack and fill in its secrets.

The unscripted parts are the Tailscale login, the admin password, and the service’s secrets — left manual on purpose.

In the end

An old phone is a capable small server — an ARM SoC, a few watts at the wall, its own battery — and most of what stands between it and that role is software that still assumes it’s a phone. Undoing that assumption was the work. It’s a fair amount for one device, but it only has to be done once, and it put a working computer back to use instead of leaving it in a drawer.

If you are wondering what actually runs on it: a Telegram bot I wrote that hand-implements a dollar-cost-averaging (DCA) strategy across a few assets — a project I am still reshaping, so I will keep the specifics for another time. The code is at github.com/arttttt/CMIDCABot. The phone was the interesting part.