Support systemd socket activation

[pinage404]

Is your feature request related to a problem? Please describe.

I would like to preserve my laptop's resources by not starting the service before i need it

I'm lazy, i don't want to manually start it

I manage the service using systemd with podman

systemd offers a mechanism of "socket activation" to start a service on first connection (AFAIU)

podman have a tutorial that explain how to do it
https://github.com/podman-container-tools/podman/blob/main/docs/tutorials/socket_activation.md#socket-activation-of-containers

It works with others containers
but with LocalAI, the first resquest never ends and is never anwser

Describe the solution you'd like

I would like that LocalAI support systemd "socket activation"

I found this blog post explaining how to do it using Go
https://kilabit.info/journal/2026/systemd_socket_with_go/
but i don't know Go

Describe alternatives you've considered

• Manually start the service
• Don't use the service
• Try to optimize others programs that take of the resources

[K4r0qtuYNE5G4qgZ]

I don't use Docker and run LocalAI bare metal, but I did the same thing for my ComfyUI which also doesn't support it natively and now quickly adapted it for LocalAI (default port) for you. Paths/Ports may need to be adjusted, but the core logic should work.

If everything works, you can access eg. http://mylocalaiserver:8080 as usual; The browser/tab will wait (rotating circle) until LocalAI (Internal local Port 8081) has started and the port has been passed on, and then the LocalAI webUI should appear. Additionally LocalAI will automatically stop if the tab/browser is closed and there is no connection for approximately 10-15 minutes (Idle Timeout Script).

1. Create the Socket (Port 8080)

sudo nano /etc/systemd/system/localai-proxy.socket

[Unit]
Description=On-Demand Public Socket for LocalAI

[Socket]
ListenStream=8080
KeepAlive=true

[Install]
WantedBy=sockets.target

2. Create the Proxy Service

sudo nano /etc/systemd/system/localai-proxy.service

[Unit]
Description=Socket Proxy Gateway for LocalAI
Requires=localai-proxy.socket
After=localai-proxy.socket
# CRUCIAL: Forces the proxy to stall its launch until LocalAI passes its loop
Requires=localai.service
After=localai.service

[Service]
Type=simple
# This catches the systemd socket traffic from 8080 and pipes it to 8081 seamlessly
ExecStart=/lib/systemd/systemd-socket-proxyd 127.0.0.1:8081

[Install]
WantedBy=multi-user.target

3. Customize the LocalAI Service

sudo nano /etc/systemd/system/localai.service

[Unit]
Description=LocalAI Core Application Backend
After=network.target
AssertPathExists=/data/LocalAI

[Service]
Type=exec
User=localai
Group=localai
Environment="LOCALAI_ADDRESS=:8081"
WorkingDirectory=/data/LocalAI
# I use the LocalAI Binary here because bare metal, but I guess you could also use a docker/script here
ExecStart=/data/LocalAI/local-ai-linux-amd64
# Secure native loop: Waits until port 8081 is actively listening, then gives Go/Python a split second to settle
ExecStartPost=/bin/bash -c 'until ss -tlnp | grep -q ":8081 "; do sleep 1; done; sleep 0.5'

4. Create the Idle Timeout Script

sudo nano /usr/local/bin/localai-autoclose.sh

#!/bin/bash
PORT=8081
SERVICE="localai.service"

if systemctl is-active --quiet "$SERVICE"; then
    CONNECTIONS=$(ss -tapn | grep ":$PORT " | grep -v "LISTEN" | wc -l)
    if [ "$CONNECTIONS" -eq 0 ]; then
        echo "No active user sessions found on 8081. Purging LocalAI"
        # Stopping the core backend forces systemd to downscale the proxy cleanly
        systemctl stop localai.service
    fi
fi

sudo chmod +x /usr/local/bin/localai-autoclose.sh

sudo crontab -e

*/10 * * * * /usr/local/bin/localai-autoclose.sh >/dev/null 2>&1

5. Apply and Start the Chain

# Reload systemd configs
sudo systemctl daemon-reload

# Disable the backend apps from standard boot triggers
sudo systemctl stop localai.service
sudo systemctl disable localai.service
sudo systemctl disable localai-proxy.service

# Enable and start ONLY the proxy network socket listener
sudo systemctl enable localai-proxy.socket
sudo systemctl start localai-proxy.socket

[pinage404]

Thanks !

It works great

I digged a bit, i think that we can remove the cron with --exit-idle-time and StopWhenUnneeded

sudo nano /etc/systemd/system/localai-proxy.service

[Unit]
Description=Socket Proxy Gateway for LocalAI
Requires=localai-proxy.socket
After=localai-proxy.socket
# CRUCIAL: Forces the proxy to stall its launch until LocalAI passes its loop
Requires=localai.service
After=localai.service

[Service]
Type=simple
# This catches the systemd socket traffic from 8080 and pipes it to 8081 seamlessly
ExecStart=/lib/systemd/systemd-socket-proxyd --exit-idle-time=10m 127.0.0.1:8081 # ⬅️ add --exit-idle-time

[Install]
WantedBy=multi-user.target

sudo nano /etc/systemd/system/localai.service

[Unit]
Description=LocalAI Core Application Backend
After=network.target
AssertPathExists=/data/LocalAI
StopWhenUnneeded=true # ⬅️ add StopWhenUnneeded

[Service]
Type=exec
User=localai
Group=localai
Environment="LOCALAI_ADDRESS=:8081"
WorkingDirectory=/data/LocalAI
# I use the LocalAI Binary here because bare metal, but I guess you could also use a docker/script here
ExecStart=/data/LocalAI/local-ai-linux-amd64
# Secure native loop: Waits until port 8081 is actively listening, then gives Go/Python a split second to settle
ExecStartPost=/bin/bash -c 'until ss -tlnp | grep -q ":8081 "; do sleep 1; done; sleep 0.5'

If nginx.service has StopWhenUnneeded= set, then passing --exit-idle-time= to systemd-socket-proxyd allows both services to stop during idle periods.
https://man.archlinux.org/man/core/systemd/systemd-socket-proxyd.8.en#Simple_Example

To automatically stop everything when things are idle, you set systemd-socket-proxyd's --exit-idle-time option and also set StopWhenUnneeded=true on your program's real service unit ('realthing.service' here). Then when systemd-socket-proxyd is idle for long enough, it will exit, systemd will notice that the 'fred.service' unit is no longer active, see that there's nothing that needs your real service unit any more, and shut that unit down too, causing your real program to exit.
https://utcc.utoronto.ca/~cks/space/blog/linux/SystemdSocketProxydNotes

[K4r0qtuYNE5G4qgZ]

Thanks as well, I wasn't aware of this little trick.

I tested it with my ComfyUI and it works perfectly, eliminating the need for a cron script.

By the way, I did that with ComfyUI because, even when idle and nothing is loaded, it consumes about 80MB of VRAM on all GPUs, and unnecessarily heats them up.

With LocalAI I personally didn't find it necessary because when it's idle, it only uses a little RAM but is otherwise fully in standby mode. But this assumes you're using LocalAI's own idle system (Enable Watchdog, Enable Idle Check, Idle Timeout) to shut down the backends/models when idle.

But it is of course still useful to have this here, if someone wants to have LocalAI or other services like ComfyUI, Unsloth Studio, etc. completely on-demand to avoid wasting resources when no users are using them.