TypePersonal infrastructure project, running on my own VPS

EnvironmentUbuntu on an OVH VPS

StatusRunning. Two extensions, tested with a real client

Sourcegithub.com/marrecluse/self-hosted-voip-vpn (secrets replaced with placeholders)

Overview

I wanted to learn VOIP the way a system administrator meets it: stand up a real PBX, make it reachable only by the people who should reach it, and prove that calls are encrypted. The result is an Asterisk server where SIP signalling uses TLS, call audio uses SRTP, and neither is reachable from the public internet.

Problem

Plaintext SIP leaks who is calling whom, and plaintext RTP leaves call audio readable to anyone watching the traffic. SIP ports are also scanned constantly by attackers looking for weak extensions to abuse. The objective was a PBX with encryption enforced rather than optional, and an attack surface small enough to reason about.

My role

This is a solo project. I did the design, the Asterisk and WireGuard configuration, the firewall and fail2ban setup, the testing, and the write-up. It is a personal learning and portfolio project. It is not a client engagement.

Architecture

Only SSH and the WireGuard handshake are reachable from the internet. A client has to be on the VPN before it can even attempt to register.

Technology stack

Ubuntu, Asterisk with PJSIP (chan_pjsip), TLS, SRTP, WireGuard (the wg-easy container), UFW, fail2ban, baresip as the test client. Asterisk is installed natively with apt and runs under systemd rather than in Docker, because RTP wants direct access to specific interfaces and ports.

Implementation

• TLS-only SIP. pjsip.conf defines a single TLS transport on 5061. There is no UDP or TCP SIP transport at all, so unencrypted signalling is not possible even by accident.
• Mandatory SRTP. Every endpoint sets media_encryption=sdes with optimistic encryption off, so a call fails closed instead of silently falling back to plain RTP. The negotiated suite is AES_CM_128_HMAC_SHA1_80.
• VPN-only reachability. The transport is bound to the WireGuard address 10.8.0.1, not 0.0.0.0. The WireGuard admin UI is published on loopback only.
• Scoped firewall. UFW is default-deny, and the SIP and RTP ports are opened only to the 10.8.0.0/24 VPN subnet.
• fail2ban. It watches Asterisk’s auth log and bans a source for an hour after five failures in ten minutes. This matters less behind a VPN, but it covers a careless or compromised VPN peer.
• One registration per extension. max_contacts=1, so a leaked credential cannot quietly register from a second place.

Challenges

The most useful problem was a silent timeout. I had bound the TLS transport to the VPN interface and a test client could still not register: no auth error, no TLS error, just silence. A raw openssl s_client connection to the port also hung, which ruled out Asterisk and pointed at the network path.

The cause was that binding a service to an interface controls where it listens, not whether the firewall lets traffic reach it. UFW’s default-deny policy was dropping the connection because I had never added an allow rule for port 5061. The fix was explicit rules scoped to the VPN subnet:

sudo ufw allow from 10.8.0.0/24 to any port 5061 proto tcp
sudo ufw allow from 10.8.0.0/24 to any port 10000:10050 proto udp

These are the rules for my own network layout. Do not paste them onto another server without checking the subnet and ports.

Testing

I tested with a real SIP client (baresip over TLS), not just by checking that the config parsed. The Asterisk and client logs showed:

• REGISTER answered with 401 and a digest challenge, then 200 OK after the client sent credentials.
• INVITE to a test extension wired to Asterisk’s Echo() application, answered with 200 OK.
• srtp: audio: SRTP is Enabled (cryptosuite=AES_CM_128_HMAC_SHA1_80), which confirms the media was actually encrypted and not only negotiated.
• A clean BYE and 200 OK on hangup.

Output from the running server: the TLS transport and endpoints in Asterisk, and the fail2ban jail watching its auth log.

Results

An encrypted call can be registered, authenticated, answered and torn down correctly, and the only ports reachable from the internet are SSH and the WireGuard handshake. I have not load-tested it, so I make no claims about capacity or call quality at scale.

Lessons learned

• A bind address and a firewall rule are separate controls, and you need to check both.
• Verify with a real client and real logs. A config that parses is not the same as a config that works.
• Layered controls (VPN-only, scoped firewall, fail2ban) mean one mistake does not expose the whole system.

Limits to be honest about: the certificate is self-signed, which is fine for a private lab but not for a public deployment (that would use a proper CA). Two extensions and an Echo() test is a secure foundation, not a full PBX. Voicemail and inter-extension dialling are the next steps.

Read the full write-up · See the iOS client built for this server