1. PRINCIPLES
In this scenario we will associate OpenVPN with an open source router called
Quagga to create a redundant triangle. The Operating systems used are Ubuntu Linux. Note that, while OpenVPN can be used on either Windows or Linux, Quagga works only on Linux.
The case study principle is that each site has two Internet links with two different
providers and each Internet link supports an OpenVPN tunnel to one of the two other sites.
If a tunnel is shut down due for example to a
provider failure, all the traffic will be re-routed through the other OpenVPN tunnel with the help of the OSPF dynamic routing protocol.
 | Please note that since this is an advanced scenario, you must fully understand how to use OpenVPN with preshared keys in IP mode and Quagga.
Use the web links under OpenVPN and Quagga sections to seek help. |
Top of the page
2. PICTURE
Top of the page
3. OPENVPN CONFIGURATION
| Before proceeding with the OpenVPN configurations, you must understand the following concepts: |
- OpenVPN Bases.
- OpenVPN Static key creation.
- Preshared key & IP mode OpenVPN tunnel creation .
The OpenVPN advanced settings case study can also be consulted for information.
***************************
Each Linux router has two OpenVPN tunnels to the other sites. The security mode is preshared keys, the tunnel mode is IP or tun, and a different key is used for each of the three links.
The requirements for using several OpenVPN tunnels on the same system are the following:
- The tunnel ports must be different.
- A separate configuration file must be built for each tunnel.
Here is a summary about the tunnels with the client/server designation, the UDP port and the key file name:
- Site A - Site B Tunnel: A is the server, B the client, port 2003, keyAB.txt
- Site A - Site C Tunnel: A is the server, C the client, port 2001, keyAC.txt
- Site B - Site C Tunnel: B is the server, C the client, port 2002, keyBC.txt
Let's create the OpenVPN config files. In order to use the OpenVPN startup script, the configuration files should have the “.conf” extension and they are to be placed in the /etc/openvpn directory.
Linux Site A
# /etc/openvpn/siteAB.conf
# Site A (server) - Site B (client)
dev tun0
ifconfig 10.7.0.9 10.7.0.10
secret /etc/openvpn/keyAB.txt
verb 2
port 2003 |
# /etc/openvpn/siteAC.conf
# Site A (server) - Site C (client)
dev tun1
ifconfig 10.7.0.1 10.7.0.2
secret /etc/openvpn/keyAC.txt
verb 2
port 2001 |
Linux Site B
# /etc/openvpn/siteBA.conf
# Site B (client) - Site A (server)
dev tun0
remote 50.0.2.52
ifconfig 10.7.0.10 10.7.0.9
secret /etc/openvpn/keyAB.txt
verb 2
port 2001 |
# /etc/openvpn/siteBC.conf
# Site B (server) - Site C (client)
dev tun1
ifconfig 10.7.0.5 10.7.0.6
secret /etc/openvpn/keyBC.txt
verb 2
port 2002 |
Linux Site C
# /etc/openvpn/siteCA.conf
# Site C (client) - Site A (server)
dev tun0
remote 50.0.1.51
ifconfig 10.7.0.2 10.7.0.1
secret /etc/openvpn/keyAC.txt
verb 2
port 2002 |
# /etc/openvpn/siteCB.conf
# Site C (client) - Site B (server)
dev tun1
remote 60.0.1.61
ifconfig 10.7.0.6 10.7.0.5
secret /etc/openvpn/keyAC.txt
verb 2
port 2002 |
Top of the page
4. QUAGGA CONFIGURATION
| Before proceeding to the Quagga configuration, make sure you know all there is to know about the Quagga install and howto tutorials. |
***************************
Three things have to be configured under Quagga:
1. The interface IP addresses.
2. The OSPF advertisements.
3. The OpenVPN gateway routing.
Be careful not to configure the interface IP addresses at the Linux level but only under Quagga.
Site A
configure terminal
interface eth0
description Link to Site C
ip address 50.0.1.51/24
link-detect
interface eth1
description Link to Site B
ip address 50.0.2.52/24
link-detect
interface lo
Virtual Local Network
ip address 10.1.1.1/32
link-detect
!
router ospf
network 10.1.1.0/32 area 0.0.0.0
network 10.7.0.0/30 area 0.0.0.0
network 10.7.0.8/30 area 0.0.0.0
!
ip route 60.0.2.62/32 50.0.2.1
ip route 70.0.1.71/32 50.0.1.1
!
! | --
|
|
|
|
|
1. Interfaces IP addresses
|
|
|
|
|
|
--
|
|
2. OSPF advertisements
|
--
|
3. OpenVPN gateways routing
|
-- |
Site B
configure terminal
interface eth0
description Link to Site C
ip address 60.0.1.61/24
link-detect
interface eth1
description Link to Site A
ip address 60.0.2.62/24
link-detect
interface lo
Virtual Local Network
ip address 10.2.2.2/32
link-detect
!
router ospf
network 10.2.2.2/32 area 0.0.0.0
network 10.7.0.4/30 area 0.0.0.0
network 10.7.0.8/30 area 0.0.0.0
!
ip route 50.0.2.52/32 60.0.2.1
ip route 70.0.2.72/32 60.0.1.1
!
! | --
|
|
|
|
|
1. Interfaces IP addresses
|
|
|
|
|
|
--
|
|
2. OSPF advertisements
|
--
|
3. OpenVPN gateways routing
|
-- |
Site C
configure terminal
interface eth0
description Link to Site A
ip address 70.0.1.71/24
link-detect
interface eth1
description Link to Site B
ip address 70.0.2.72/24
link-detect
interface lo
Virtual Local Network
ip address 10.3.3.3/32
link-detect
!
router ospf
network 10.3.3.3/32 area 0.0.0.0
network 10.7.0.0/30 area 0.0.0.0
network 10.7.0.4/30 area 0.0.0.0
!
ip route 60.0.1.61/32 70.0.2.1
ip route 50.0.1.51/32 70.0.1.1
!
! | --
|
|
|
|
|
1. Interfaces IP addresses
|
|
|
|
|
|
--
|
|
2. OSPF advertisements
|
--
|
3. OpenVPN gateways routing
|
-- |
*****************************************************************
Activate the IP forwarding on the three Linux systems:
IP forwarding is required to transfer packets between the network interfaces of a Linux system.
See a picture of the Linux kernel routing.
Note that the IP forwarding activation must be done at the Linux level since it is not possible to do it directly from the Quagga router.
| #echo "1" > /proc/sys/net/ipv4/ip_forward |
The command above will add the "1" value inside the /proc/sys/net/ipv4/ip_forward file and thus activate the IP forwarding.
If you want to keep the IP forwarding after a Linux reboot, type:
| #echo "net.ipv4.ip_forward = 1" >> /etc/sysctl.conf |
It is possible to check the ip_forwarding status at the Quagga router level:
IP forwarding is on
In this case the IP forwarding is activated.
Top of the page
5. CHECKS
Let's check the routing status from the Linux system located in Site A.
First check the openvpn process. You should see two of them, one per tunnel.
| Linux_SiteA#ps -ef | grep openvpn |
| UID | PID | PPID | C | STIME | TTY | TIME | CMD |
root
| 4495
| 1
| 0
| 08:26
| ?
| 00:00:00
| /usr/sbin/openvpn
--writepid /var/run/openvpn.siteAB.pid
--daemon ovpn-siteAB
--status /var/run/openvpn.siteAB.status 10
--cd /etc/openvpn
--config /etc/openvpn/siteAB.conf |
root
| 4502
| 1
| 0
| 08:26
| ?
| 00:00:00
| /usr/sbin/openvpn
--writepid /var/run/openvpn.keyAC.pid --daemon ovpn-keyAC
--status /var/run/openvpn.keyAC.status 10
--cd /etc/openvpn
--config /etc/openvpn/keyAC.conf
|
Check the routes from the Quagga platform:
| Quagga_SiteA#show ip route |
| Codes: | K - kernel route, C - connected, S - static, R - RIP, O - OSPF, |
| | | I - ISIS, B - BGP, > - selected route, * - FIB route |
| | | |
| C>* | 10.1.1.1/32 is directly connected, lo |
| C>* | 50.0.1.0/24 is directly connected, eth0 |
| C>* | 50.0.2.0/24 is directly connected, eth1 |
| C>* | 10.7.0.2/32 is directly connected, tun1 |
| C>* | 10.7.0.10/32 is directly connected, tun0 |
| | | |
| S>* | 60.0.2.62/32 [1/0] via 50.0.2.1, eth1 |
| S>* | 70.0.1.71/32 [1/0] via 50.0.1.1, eth0 |
| | | |
| O | 10.7.0.2/32 [110/10] is directly connected, tun1, 00:19:09 |
| O | 10.7.0.10/32 [110/10] is directly connected, tun0, 00:19:09 |
| | | |
| O>* | 10.2.2.2/32 [110/20] via 10.7.0.10, tun0, 00:07:56 |
| O>* | 10.7.0.6/32 [110/20] via 10.7.0.10, tun0, 00:07:56 |
| O>* | 10.7.0.9/32 [110/20] via 10.7.0.10, tun0, 00:07:56 |
| O>* | 10.3.3.3/32 [110/20] via 10.7.0.2, tun1, 00:00:48 |
| O>* | 10.7.0.1/32 [110/20] via 10.7.0.2, tun1, 00:00:48 |
| O>* | 10.7.0.5/32 [110/20] via 10.7.0.2, tun1, 00:00:48 |
Check the OSPF neighbors
| Quagga_SiteA#show ip ospf neighbor |
| Neighbor ID | Pri | State | Dead Time | Address | Interface | RXmtL | RqstL | DBsmL |
| 10.3.3.3 | 1 | Full/DROther | 36.522s | 10.7.0.2 | tun1:10.7.0.1 | 0 | 0 | 0 |
| 10.2.2.2 | 1 | Full/DROther | 33.610s | 10.7.0.10 | tun0:10.7.0.9 | 0 | 0 | 0 |
Check the OSPF routes.
| Quagga_SiteA#show ip ospf route |
| ============ OSPF network routing table ============ |
| N | 10.2.2.2/32 | [20] area: 0.0.0.0 |
| | | via 10.7.0.10, tun0 |
| N | 10.3.3.3/32 | [20] area: 0.0.0.0 |
| | | via 10.7.0.2, tun1 |
| N | 10.7.0.1/32 | [20] area: 0.0.0.0 |
| | | via 10.7.0.2, tun1 |
| N | 10.7.0.2/32 | [10] area: 0.0.0.0 |
| | | directly attached to tun1 |
| N | 10.7.0.5/32 | [20] area: 0.0.0.0 |
| | | via 10.7.0.2, tun1 |
| N | 10.7.0.6/32 | [20] area: 0.0.0.0 |
| | | via 10.7.0.10, tun0 |
| N | 10.7.0.9/32 | [20] area: 0.0.0.0 |
| | | via 10.7.0.10, tun0 |
| N | 10.7.0.10/32 | [10] area: 0.0.0.0 |
| | | directly attached to tun0 |
| | | |
| ============ OSPF router routing table ============= |
| | | |
| ============ OSPF external routing table =========== |
Please note that Quagga shows only the best OSPF routes. For example, for Cisco or Vyatta routers, the OSPF database contains all the routes learned for a network subnet.
Check the routes at the Linux level.
| Kernel IP routing table |
| Destination | Gateway | Genmask | Flags | Metric | Ref | Use | Iface |
| 50.0.1.0 | 0.0.0.0 | 255.255.255.0 | U | 0 | 0 | 0 | eth0 |
| 70.0.1.71 | 50.0.1.1 | 255.255.255.255 | UGH | 0 | 0 | 0 | eth0 |
| | | | | | | | |
| 50.0.2.0 | 0.0.0.0 | 255.255.255.0 | U | 0 | 0 | 0 | eth1 |
| 60.0.2.62 | 50.0.2.1 | 255.255.255.255 | UGH | 0 | 0 | 0 | eth1 |
| | | | | | | | |
| 10.2.2.2 | 10.7.0.10 | 255.255.255.255 | UGH | 20 | 0 | 0 | tun0 |
| 10.7.0.6 | 10.7.0.10 | 255.255.255.255 | UGH | 20 | 0 | 0 | tun0 |
| 10.7.0.9 0 | 10.7.0.10 | 255.255.255.255 | UGH | 20 | 0 | 0 | tun0 |
| 10.7.0.10 | 0.0.0.0 | 255.255.255.255 | UH | 0 | 0 | 0 | tun0 |
| | | | | | | | |
| 10.3.3.3 | 10.7.0.2 | 255.255.255.255 | UGH | 20 | 0 | 0 | tun1 |
| 10.7.0.1 | 10.7.0.2 | 255.255.255.255 | UGH | 20 | 0 | 0 | tun1 |
| 10.7.0.2 | 0.0.0.0 | 255.255.255.255 | UH | 0 | 0 | 0 | tun1 |
| 10.7.0.5 | 10.7.0.2 | 255.255.255.255 | UGH | 20 | 0 | 0 | tun1 |
Check the opened UDP ports.
| Quagga_SiteA#netstat -uae |
| Proto | Recv-Q | Send-Q | Local Address | Foreign Address | State | User | Inode |
| Active Internet connections (servers and established) |
| udp | 0 | 0 | *:2001 | *:* | | root | 15387 |
| udp | 0 | 0 | *:2003 | *:* | | root | 15352 |
See detailed information about netstat.
Top of the page
6. REDUNDANCY SCENARIO
It's time to test whether or not the redundancy triangle is working.
In order to do so, we unplug the physical cable connected to the "eth0" interface of Quagga in Site A. This will simulate an Internet provider failure.
The Site A - Site B OpenVPN tunnel will fail and Quagga in Site A will no longer learn OSPF networks from Quagga in Site C.
Thus, Quagga A will use the route through Quagga in Site B to reach Site C.
Check the routes from the Quagga router.
| Quagga_SiteA#show ip route |
| Codes: | K - kernel route, C - connected, S - static, R - RIP, O - OSPF, |
| | | I - ISIS, B - BGP, > - selected route, * - FIB route |
| | | |
| C>* | 10.1.1.1/32 is directly connected, lo |
| C>* | 50.0.1.0/24 is directly connected, eth0 |
| C>* | 50.0.2.0/24 is directly connected, eth1 |
| C>* | 10.7.0.2/32 is directly connected, tun1 |
| C>* | 10.7.0.10/32 is directly connected, tun0 |
| | | |
| S>* | 60.0.2.62/32 [1/0] via 50.0.2.1, eth1 |
| S>* | 70.0.1.71/32 [1/0] via 50.0.1.1, eth0 |
| | | |
| O | 10.7.0.2/32 [110/10] is directly connected, tun1, 00:19:09 |
| O | 10.7.0.10/32 [110/10] is directly connected, tun0, 00:19:09 |
| | | |
| O>* | 10.2.2.2/32 [110/20] via 10.7.0.10, tun0, 00:11:53 |
| O>* | 10.3.3.3/32 [110/30] via 10.7.0.10, tun0, 00:02:18 |
| O>* | 10.7.0.1/32 [110/30] via 10.7.0.10, tun0, 00:02:18 |
| O>* | 10.7.0.5/32 [110/30] via 10.7.0.10, tun0, 00:02:18 |
| O>* | 10.7.0.6/32 [110/20] via 10.7.0.10, tun0, 00:11:53 |
| O>* | 10.7.0.9/32 [110/20] via 10.7.0.10, tun0, 00:11:53 |
When the Site A - Site C tunnel is up, the route for 10.3.3.3/32 is learned as follows:
O>* 10.3.3.3/32 [110/20] via 10.7.0.2, tun1, 00:02:18
Check the OSPF neighbors.
| Quagga_SiteA#show ip ospf neighbor |
| Neighbor ID | Pri | State | Dead Time | Address | Interface | RXmtL | RqstL | DBsmL |
| 10.2.2.2 | 1 | Full/DROther | 33.610s | 10.7.0.10 | tun0:10.7.0.9 | 0 | 0 | 0 |
Traceroute Quagga in Site C to see if the packets are rerouted through site B.
The traceroute is launched at the Linux level because we need to use the "s" option to specify the source IP address.
The Traceroute or Ping options are not available under the Quagga platform.
| Linux_SiteA#traceroute -s 10.1.1.1 10.3.3.3 |
| traceroute to 10.3.3.3 (10.3.3.3) from 10.1.1.1, 30 hops max, 40 byte packets | 1 10.7.0.10 (10.7.0.10) 0.588 ms 0.471 ms 0.347 ms | 2 10.3.3.3 (10.3.3.3) 0.715 ms 1.734 ms 0.512 ms |
Check the OSPF database.
| Quagga_SiteA#show ip ospf database |
| | OSPF Router with ID (10.1.1.1) |
| | | Router Link States (Area 0.0.0.0) |
| | | | | | | | |
| Link ID | | ADV Router | Age | Seq# | CkSum | Link count |
| 10.1.1.1 | | 10.1.1.1 | 240 | 0x8000000d | 0x91df | 4 |
| 10.2.2.2 | | 10.2.2.2 | 816 | 0x80000006 | 0xa110 | 5 |
| 10.3.3.3 | | 10.3.3.3 | 242 | 0x80000040 | 0xbc81 | 4 |
Check the OSPF routes.
| Quagga_SiteA#show ip ospf route |
| ============ OSPF network routing table ============ |
| N | 10.2.2.2/32 | [20] area: 0.0.0.0 |
| | | via 10.7.0.10, tun0 |
| N | 10.3.3.3/32 | [30] area: 0.0.0.0 |
| | | via 10.7.0.10, tun0 |
| N | 10.7.0.1/32 | [30] area: 0.0.0.0 |
| | | via 10.7.0.10, tun0 |
| N | 10.7.0.2/32 | [10] area: 0.0.0.0 |
| | | directly attached to tun1 |
| N | 10.7.0.5/32 | [30] area: 0.0.0.0 |
| | | via 10.7.0.10, tun0 |
| N | 10.7.0.6/32 | [20] area: 0.0.0.0 |
| | | via 10.7.0.10, tun0 |
| N | 10.7.0.9/32 | [20] area: 0.0.0.0 |
| | | via 10.7.0.10, tun0 |
| N | 10.7.0.10/32 | [10] area: 0.0.0.0 |
| | | directly attached to tun0 |
| | | |
| ============ OSPF router routing table ============= |
| | | |
| ============ OSPF external routing table =========== |
When the Site A - Site C tunnel is up, the route for 10.3.3.3/32 is learned as follows:
N
| 10.3.3.3/32
| [20] area: 0.0.0.0
via 10.7.0.2, tun0 |
Back to the normal situation
When the internet link at Site A comes up, the OpenVPN tunnel Site A - site C is automatically rebuilt and at the same time the OSPF advertisements are learned anew from the Quagga router in Site C.
From Site A, 10.3.3.3 will be learned directly from Site C with a metric of 20 and indirectly through site B with a metric of 30.
As the route with the lowest metric is elected as the best route, the path to Site C will switch back to the Site A - Site C OpenVPN tunnel instead of the Site A - Site B - Site C tunnel used during the provider outage.
Top of the page
7. HUB SITE SCENARIO
In this scenario, Site B is considered as a hub site. The two network links on this site are high bandwidth. The link Site A - Site C is a low bandwidth phone link used for backup purpose.
If we keep the default OSPF settings, we will be in the same scenario as the one presented at the top of the page where the three links are active.
If Site A wants to reach Site C via Site B, we must increase the OSPF cost on the Site A - Site C link to a value superior than the OSPF cost via the Site B which is 30.
For the OpenVPN and Quagga configurations, we can keep the exact same settings as the scenario presented at the top of the page. We just need to add the OSPF cost values.
configure terminal
interface tun1
ip ospf cost 100 |
configure terminal
interface tun0
ip ospf cost 100 |
Quagga in Site A will receive two advertisements for 10.3.3.3 which is the local Site C network.
OSPF Advertisement learned from Quagga in Site C:
N
| 10.3.3.3/32
| [30] area: 0.0.0.0
via 10.7.0.10, tun0 |
OSPF Advertisement learned on the Quagga router in Site B:
N
| 10.3.3.3/32
| [110] area: 0.0.0.0
via 10.7.0.2, tun1 |
Please note that Quagga shows only the best OSPF routes. For example, for Cisco or Vyatta routers, the OSPF database contains all the routes learned for a network subnet.
With Quagga, we only see the best advertisement with a cost equals to 30 ("show ip ospf route" command). If the site A - site B tunnel is down, the second OSPF advertisement will be seen on the screen.
