VPN Server Behind Firewall: How-To Guide | OpenVPN

One of the most effective ways to secure your network is to deploy a Virtual Private Network (VPN) — and the second is to set up a VPN behind a firewall. However, setting up a VPN behind an existing firewall requires careful configuration to avoid connectivity issues and maintain security. 

This guide provides a comprehensive step-by-step approach to setting up an OpenVPN server behind a firewall, ensuring seamless remote access while preserving network integrity. For the purposes of this blog, we will specifically be referring to setting up Access Server behind an existing firewall. 

What is a firewall?

A firewall is a network security system that monitors and controls incoming and outgoing network traffic based on predetermined security rules. It acts as a barrier between a trusted internal network and untrusted external networks, such as the internet. Firewalls help prevent unauthorized access, mitigate cyber threats, and ensure data security.

There are different types of firewalls, including:

• Packet filtering firewalls that examine individual data packets and allow or block them based on source and destination addresses, ports, or protocols.
• Stateful inspection firewalls that keep track of active connections and make decisions based on the state of network traffic.
• Proxy firewalls that intercept network traffic and enforce security policies before forwarding requests.
• Next-generation firewalls (NGFWs) that incorporate features such as deep packet inspection (DPI), intrusion prevention, and application-layer filtering.

How firewalls interact with VPN servers

Firewalls can impact VPN traffic in several ways. By default, they may block VPN-related traffic to prevent unauthorized access or data leaks. Properly configuring a firewall to allow VPN connections while maintaining security is crucial. Without the right settings, VPN users may experience connectivity issues, restricted access, or degraded performance.

Firewalls typically interact with VPN servers in the following ways:

• Blocking or restricting ports: Some firewalls block common VPN ports like UDP 1194, preventing connections.
• Filtering protocols: Firewalls may restrict certain VPN protocols, such as UDP, which is commonly used by OpenVPN.
• Interfering with encryption: Some firewalls perform deep packet inspection, which can interfere with encrypted VPN traffic.
• Limiting bandwidth: Network administrators may configure bandwidth throttling for VPN connections, impacting performance.

Choosing the right VPN software

Getting the right VPN in place is the first step. If you’re not sure where to start, check out our IT Admin’s Guide to Evaluating Network Security Solutions. 

If you’re here, you likely already know that OpenVPN offers strong encryption, cross-platform compatibility, and extensive configurability. OpenVPN also supports multiple authentication methods, making it ideal for businesses that require secure remote access.

As a reminder, the rest of this guide will refer specifically to setting up Access Server behind a firewall. 

System requirements

Before setting up OpenVPN, ensure your system meets the following requirements:

• Hardware: A dedicated server with a modern processor, a minimum of 1GB of RAM, and sufficient storage.
• Operating system: If you’re using Access Server, a self-hosted VPN server, you will need a Linux machine to complete setup. CloudConnexa, our Cloud VPN service, and the OpenVPN Connect application are compatible with Windows, Linux, macOS, Android, and iOS.
• Software: OpenVPN and a reliable internet connection.
• Static IP address: Recommended for stability and ease of configuration.

Setting up a VPN server behind an existing firewall

CPU requirements

Access Server performs intensive Central Processing Unit (CPU) operations for encryption and decryption tasks. Therefore, we don't recommend running Access Server on a shared-purpose platform, such as a cPanel or configServer hosting server. Access Server shouldn't coexist with other applications on the same server in that way. Further, other programs that manage firewall rules will likely interfere with Access Server's firewall functionalities. While technically possible, running multiple programs of this nature simultaneously isn't advised.

Red Hat Enterprise Linux and firewalld

On Red Hat Enterprise Linux 8 and 9, we advise you to remove the firewalld daemon:

sudo su

systemctl stop firewalld

systemctl disable firewalld

yum erase firewalld

The default firewalld on RHEL conflicts with Access Server, which implements its own firewall rules for VPN traffic.

Why does Access Server use TCP 443 and TCP 943 ports?

Access Server works best over UDP. However, TCP 443 is the preferred fallback option when firewalls block UDP. If you’re managing your own firewall, you can open the UDP port, 1194, and allow that traffic.

You may be wondering why we recommend using TCP 443. The short answer: firewall compatibility. Many public networks block traffic on all ports except a few (e.g., HTTP, HTTPS, FTP, email). By running OpenVPN on TCP 443 (the default port for HTTPS), VPN connections can bypass these restrictions.

Basic setup: one network interface on a private network behind a firewall

Use Access Server to set up secure access to a private network behind a firewall. This configuration includes the following:

• Access Server hosted on your internal corporate network.
• Users outside the network gaining access through VPN tunnels.
• Access Server has one network interface to the private network.

Note: You may have other interfaces present on the system not utilized by Access Server.

For this configuration, the internet gateway forwards TCP/UDP port traffic from the public-facing IP address to Access Server’s private IP address. At a minimum, one TCP port (typically port 443) is forwarded. That TCP port can carry both the VPN tunnel traffic and the web server/VPN client traffic. Optionally, you can separate VPN tunneling from web server traffic, in which case an additional TCP or UDP port (e.g., UDP port 1193) is forwarded for the VPN tunnel.

A variation on this network configuration has Access Server with one interface attached to a DMZ network provided by the firewall. As mentioned above, the same forwarding of client traffic is required. Additionally, you may need to configure the firewall to allow traffic between Access Server and the private network behind the firewall.

For more detailed information, check out our documentation. 

Common pitfalls to avoid

• Blocking required ports: Ensure the firewall does not inadvertently block OpenVPN ports.
• Improper NAT configuration: Without proper NAT rules, VPN traffic may not route correctly.
• DNS issues: Configure DNS settings properly to prevent connectivity problems for VPN clients.
• Overly restrictive rules: Test and fine-tune firewall rules to balance security and usability.
• Deep packet inspection (DPI): Some firewalls use DPI to block VPN traffic. Using TCP 443 can help bypass these restrictions.

Conclusion

Setting up a VPN behind an existing firewall requires careful planning and configuration. By choosing Access Server, meeting system requirements, and properly configuring firewall rules, IT managers can establish a secure and efficient remote access solution. 

Ready to find out more? Join our weekly office hours for a group demo and discussion.