In this paper we discuss a simple use case. Two offices with a Check Point Firewall Gateway connected via MPLS and both also with internet connection. The MPLS is used for inter-site connectivity but they are wanting to use the internet connection as a backup for this should the it fail.
Please feel welcome to post comments, questions or corrections at the end of the paper and I'll answer them promptly.
Background
The scenario is simple. We've an office in the UK and an office in the US. Both offices are connected over an MPLS network which provides inter-office connectivity essentially allowing both offices to talk to one another, exchange files, & use their VoIP phones to call each other. Each office also has an internet connection allowing the web, and allows both offices to host their own local websites: www.acme.com and www.acme.co.uk.
However, the MPLS has been know to go down from time to time interrupting business critical network traffic between both sites and effectively halting certain business processes. Not good. Not desirable.
You can see the environment detailed in the high level diagram.
Solution
At this point take a look at the detailed diagram.
Both sites already have a Check Point Firewall Gateway. We will set up a VPN between the two sites and configure this VPN to use a 'Virtual Tunnel Interface' (this is Route Based VPN!). This will present the VPN tunnel to the OS as a network point to point network interface, allowing us to run routing protocols over it and so fail over to it if the MPLS becomes unavailable.
Once the VPN is up we'll configure the routing suite on our Check Point Gateways and on the MPLS router.
We'll ensure that from US Network to UK Network via the VPN has a cost of 50 in routing terms, and that from US Network to UK Network via the MPLS has a cost of 10. As such, the MPLS will always be the favored route for network traffic to take between the two office. Only if the MPLS fails will the VPN carry inter-office traffic.
We've placed test hosts on the internal networks 1.0.0.99 & 2.0.0.99. These are simply running Debian and will be used to generate test traffic using 'ping' and 'netcat'.
Lab Detail
In my lab I have the following configured. You will need to install the software, up your network interfaces, and add default routes to UKFW and USFW and of course, cable the lab together (if you are not using virtual machines).
Also, you should ensure that /etc/rc.local for both INET and MPLS have the following lines:
echo 1 > /proc/sys/net/ipv4/ip_forward
fw unloadlocal
| Name | OS | Configuration | Routes
before OSPF enabled |
| INET | SecurePlatform | Installed with gateway only. Two interfaces with IP forwarding enabled | Direct routes only |
| MPLS | SecurePlatform Pro | Installed with gateway only. Firewall filtering is disabled with 'fw unloadlocal'. IP forwarding is enabled. This is nothing more than a router capable of OSPF | Direct routes only |
| UKFW | SecurePlatform Pro | Installed with gateway & SmartCenter | Direct routes & default route via INET |
| USFW | SecurePlatform Pro | Installed with gateway only. Managed from UKFWs SmartCenter | Direct routes & default route via INET |
Also, don't forget to configure the two test hosts, up their interfaces and give them a default route to their local firewall.
At this point check that each machine can ping its directly connected peers to ensure there are no cabling issues.
Routing Configuration
Log into USFW, UKFW & MPLS and use the following commands to configure the routing suite:
expert (access expert mode)
drouter start
router
en
configure terminal
Enter the appropriate configuration
exit
write memory
quit
drouter stop; service network restart; drouter start; watch route
The last command will stop the routing suite, clear out all the routes injected into the kernel, restart the routing suite and then display the routing table so you can watch as routes are added as the OSPF neighbors are brought online.
Routing Configuration on UKFW
| UKFW#show
running-config Building configuration... access-list 1 seq 5 permit 1.0.0.0 0.0.0.255 access-list 1 seq 10 permit 200.0.0.0 0.0.0.255 route-map control-ospf-redist permit 5 match ip address access-list 1 exit router ospf 101 network 200.0.0.0 0.0.0.255 area 0.0.0.0 network 9.9.9.0 0.0.0.3 area 0.0.0.0 redistribute direct route-map control-ospf-redist exit interface VTI ip ospf 101 area 0.0.0.0 cost 50 exit exit interface eth2 ip ospf 101 area 0.0.0.0 cost 5 exit exit UKFW# |
When
you initially configure OSPF for this scenario, you will probably use
the 'redistribute direct' command in order that knowledge of
the internal network 1.0.0.0/24 is send to the MPLS
and the
peer Check Point gateway. However in doing so, we redistribute ALL
directly connected interfaces, including the VTI and the internet
facing interface eth1. I've used a route map control-ospf-redist to ensure that only the internal network (and actuallly MPLS transit networks) are redistributed. Without this route map in place, you will find that the gateways can learn that the internet facing interface of its VPN peer (eth1) is availble via the MPLS network, and try to establish a VPN, and thus the VTI, over the MPLS link and not the internet as intended. We can also see the costs we've assigned to the interfaces to ensure the MPLS is preferred over the VPN. |
Routing Configuration on USFW
| USFW#show
running-config Building configuration... access-list 1 seq 5 permit 2.0.0.0 0.0.0.255 access-list 1 seq 10 permit 202.0.0.0 0.0.0.255 route-map control-ospf-redist permit 5 match ip address access-list 1 exit router ospf 101 network 202.0.0.0 0.0.0.255 area 0.0.0.0 network 9.9.9.0 0.0.0.3 area 0.0.0.0 redistribute direct route-map control-ospf-redist exit interface VTI ip ospf 101 area 0.0.0.0 cost 50 exit exit interface eth2 ip ospf 101 area 0.0.0.0 cost 5 exit exit USFW# |
The routing
configuration of USFW is identical to that of UKFW only with the
appropriate subnets for its location (e.g. 2.0.0.0/24 instead of
1.0.0.0/24) |
Routing configuration of MPLS
| MPLS#show
running-config Building configuration... router ospf 101 network 200.0.0.0 0.0.0.255 area 0.0.0.0 network 202.0.0.0 0.0.0.255 area 0.0.0.0 redistribute direct exit interface eth0 ip ospf 101 area 0.0.0.0 cost 5 exit exit interface eth1 ip ospf 101 area 0.0.0.0 cost 5 exit exit MPLS# |
MPLS is nothing
more than an OSPF enabled router with costs assigned to each interface. |
Check Point Configuration
The configuration is very simple. Both USFW & UKFW are configured with empty VPN domains added to a VPN community with Permanent Tunnels enabled. Then the VTI is enabled at the command line interface.
Security rulebase contains only a single "Any Any Any Accept Log" rule
SmartDefense is set to default, with protections that ship with R65, no update
No HFAs applied
Anything that isn't detailed in a screenshot is set to default
UKFW
USFW
VPN Community
Setting the VTI:
UKFW
To add a VTI you run the below command and detail: the local IP address you wish to use for the VTI, the remote IP address for the VTI, the name of the peer VPN gateway, and an interface name for the VTI. In my lab I used 'VTI'. I highly recommend that you use something more memorable. A better choice for me would have been VTI-UK-US.
interface add numbered 9.9.9.9 9.9.9.10 Router2USFW VTI
USFW
We run the same vpn shell commands but flip over the IP addresses and use a UKFW as the peer name:
interface add numbered 9.9.9.10 9.9.9.9 Router1UKFW VTI
Note: Router1UKFW and Router2USFW are the object names in SmartDashboard for both UK & US
One done on both ends, check you can ping the VTI IP address of the peer gateway.
For extra reassurance, you will see the ICMP being encrypted and decrypted in the Check Point log.
The Results!
Since the VTI is up now, we should see the routing tables in state 2 (see below), with the internal networks being reachable via the MPLS.
You should start a ping from the UK test host (1.0.0.99) to the US test host (2.0.0.99) and run tcpdump -n -l -i eth0 on MPLS. You should see the echo requests & replies indicating that the traffic is being routed by the MPLS network. State 2 is the 'business as usual' state, with the MPLS operating as normal.
[Expert@MPLS]# tcpdump -n -l -i eth0
tcpdump: listening on eth0
11:21:10.011350 1.0.0.99 > 2.0.0.99: icmp: echo request (DF)
11:21:10.016402 2.0.0.99 > 1.0.0.99: icmp: echo reply
At this point disable the eth0 interface on MPLS with the ifconfig eth0 down command. This emulates the MPLS link in the UK going down. The routing tables will now move to state 3. Traffic between the two sites is now being routed across the VTI. You can see the by looking at the firewall log. 
| UKFW | USFW |
| [Expert@UKFW]#
drouter stop; service network restart; route Kernel IP routing table Destination Gateway Genmask Flags Metric Ref Use Iface 9.9.9.10 * 255.255.255.255 UH 0 0 0 VTI 1.0.0.0 * 255.255.255.0 U 0 0 0 eth0 200.0.0.0 * 255.255.255.0 U 0 0 0 eth2 100.0.0.0 * 255.255.255.0 U 0 0 0 eth1 127.0.0.0 * 255.0.0.0 U 0 0 0 lo default 100.0.0.254 0.0.0.0 UG 0 0 0 eth1 [Expert@UKFW]# |
[Expert@USFW]#
drouter stop; service network restart; route Kernel IP routing table Destination Gateway Genmask Flags Metric Ref Use Iface 9.9.9.9 * 255.255.255.255 UH 0 0 0 VTI 2.0.0.0 * 255.255.255.0 U 0 0 0 eth0 202.0.0.0 * 255.255.255.0 U 0 0 0 eth2 101.0.0.0 * 255.255.255.0 U 0 0 0 eth1 127.0.0.0 * 255.0.0.0 U 0 0 0 lo default 101.0.0.254 0.0.0.0 UG 0 0 0 eth1 [Expert@USFW]# |
| MPLS | INET |
| [Expert@MPLS]#
drouter stop; service network restart; route Kernel IP routing table Destination Gateway Genmask Flags Metric Ref Use Iface 202.0.0.0 * 255.255.255.0 U 0 0 0 eth1 200.0.0.0 * 255.255.255.0 U 0 0 0 eth0 127.0.0.0 * 255.0.0.0 U 0 0 0 lo [Expert@MPLS]# |
[Expert@INET]#
drouter stop; service network restart; route Kernel IP routing table Destination Gateway Genmask Flags Metric Ref Use Iface 101.0.0.0 * 255.255.255.0 U 0 0 0 eth1 100.0.0.0 * 255.255.255.0 U 0 0 0 eth0 127.0.0.0 * 255.0.0.0 U 0 0 0 lo [Expert@INET]# |
State 2: Routing Protocols Enabled
| UKFW | USFW |
| [Expert@UKFW]#
route Kernel IP routing table Destination Gateway Genmask Flags Metric Ref Use Iface 224.0.0.2 * 255.255.255.255 UHD 0 0 0 lo 224.0.0.6 * 255.255.255.255 UHD 0 0 0 lo 9.9.9.9 localhost.local 255.255.255.255 UGH 0 0 0 lo 224.0.0.5 * 255.255.255.255 UHD 0 0 0 lo 9.9.9.10 * 255.255.255.255 UH 0 0 0 VTI localhost.local * 255.255.255.255 UH 0 0 0 lo 1.0.0.0 * 255.255.255.0 U 0 0 0 eth0 2.0.0.0 200.0.0.254 255.255.255.0 UGD 0 0 0 eth2 202.0.0.0 200.0.0.254 255.255.255.0 UGD 0 0 0 eth2 200.0.0.0 * 255.255.255.0 U 0 0 0 eth2 100.0.0.0 * 255.255.255.0 U 0 0 0 eth1 127.0.0.0 * 255.0.0.0 U 0 0 0 lo default 100.0.0.254 0.0.0.0 UG 0 0 0 eth1 [Expert@UKFW]# |
[Expert@USFW]#
route Kernel IP routing table Destination Gateway Genmask Flags Metric Ref Use Iface 224.0.0.2 * 255.255.255.255 UHD 0 0 0 lo 224.0.0.6 * 255.255.255.255 UHD 0 0 0 lo 9.9.9.9 * 255.255.255.255 UH 0 0 0 VTI 224.0.0.5 * 255.255.255.255 UHD 0 0 0 lo 9.9.9.10 localhost.local 255.255.255.255 UGH 0 0 0 lo localhost.local * 255.255.255.255 UH 0 0 0 lo 1.0.0.0 202.0.0.254 255.255.255.0 UGD 0 0 0 eth2 2.0.0.0 * 255.255.255.0 U 0 0 0 eth0 202.0.0.0 * 255.255.255.0 U 0 0 0 eth2 101.0.0.0 * 255.255.255.0 U 0 0 0 eth1 200.0.0.0 202.0.0.254 255.255.255.0 UGD 0 0 0 eth2 127.0.0.0 * 255.0.0.0 U 0 0 0 lo default 101.0.0.254 0.0.0.0 UG 0 0 0 eth1 [Expert@USFW]# |
| MPLS | INET |
| [Expert@MPLS]#
route Kernel IP routing table Destination Gateway Genmask Flags Metric Ref Use Iface 224.0.0.2 * 255.255.255.255 UHD 0 0 0 lo 224.0.0.6 * 255.255.255.255 UHD 0 0 0 lo 9.9.9.9 202.0.0.1 255.255.255.255 UGHD 0 0 0 eth1 224.0.0.5 * 255.255.255.255 UHD 0 0 0 lo 9.9.9.10 200.0.0.1 255.255.255.255 UGHD 0 0 0 eth0 localhost.local * 255.255.255.255 UH 0 0 0 lo 1.0.0.0 200.0.0.1 255.255.255.0 UGD 0 0 0 eth0 2.0.0.0 202.0.0.1 255.255.255.0 UGD 0 0 0 eth1 202.0.0.0 * 255.255.255.0 U 0 0 0 eth1 200.0.0.0 202.0.0.1 255.255.255.0 UGD 0 0 0 eth1 200.0.0.0 * 255.255.255.0 U 0 0 0 eth0 127.0.0.0 * 255.0.0.0 U 0 0 0 lo [Expert@MPLS]# |
[Expert@INET]#
drouter stop; service network restart; route Kernel IP routing table Destination Gateway Genmask Flags Metric Ref Use Iface 101.0.0.0 * 255.255.255.0 U 0 0 0 eth1 100.0.0.0 * 255.255.255.0 U 0 0 0 eth0 127.0.0.0 * 255.0.0.0 U 0 0 0 lo [Expert@INET]# |
State 3: Routing Protocols Enabled & MPLS interface eth0 down
| UKFW | USFW |
| [Expert@UKFW]#
route Kernel IP routing table Destination Gateway Genmask Flags Metric Ref Use Iface 224.0.0.2 * 255.255.255.255 UHD 0 0 0 lo 224.0.0.6 * 255.255.255.255 UHD 0 0 0 lo 9.9.9.9 localhost.local 255.255.255.255 UGH 0 0 0 lo 224.0.0.5 * 255.255.255.255 UHD 0 0 0 lo 9.9.9.10 * 255.255.255.255 UH 0 0 0 VTI localhost.local * 255.255.255.255 UH 0 0 0 lo 1.0.0.0 * 255.255.255.0 U 0 0 0 eth0 2.0.0.0 9.9.9.10 255.255.255.0 UGD 0 0 0 VTI 202.0.0.0 9.9.9.10 255.255.255.0 UGD 0 0 0 VTI 200.0.0.0 * 255.255.255.0 U 0 0 0 eth2 100.0.0.0 * 255.255.255.0 U 0 0 0 eth1 127.0.0.0 * 255.0.0.0 U 0 0 0 lo default 100.0.0.254 0.0.0.0 UG 0 0 0 eth1 [Expert@UKFW]# |
[Expert@USFW]#
route Kernel IP routing table Destination Gateway Genmask Flags Metric Ref Use Iface 224.0.0.2 * 255.255.255.255 UHD 0 0 0 lo 224.0.0.6 * 255.255.255.255 UHD 0 0 0 lo 9.9.9.9 * 255.255.255.255 UH 0 0 0 VTI 224.0.0.5 * 255.255.255.255 UHD 0 0 0 lo 9.9.9.10 localhost.local 255.255.255.255 UGH 0 0 0 lo localhost.local * 255.255.255.255 UH 0 0 0 lo 1.0.0.0 9.9.9.9 255.255.255.0 UGD 0 0 0 VTI 2.0.0.0 * 255.255.255.0 U 0 0 0 eth0 202.0.0.0 * 255.255.255.0 U 0 0 0 eth2 101.0.0.0 * 255.255.255.0 U 0 0 0 eth1 200.0.0.0 9.9.9.9 255.255.255.0 UGD 0 0 0 VTI 127.0.0.0 * 255.0.0.0 U 0 0 0 lo default 101.0.0.254 0.0.0.0 UG 0 0 0 eth1 [Expert@USFW]# |
| MPLS | INET |
| [Expert@MPLS]#
route Kernel IP routing table Destination Gateway Genmask Flags Metric Ref Use Iface 224.0.0.2 * 255.255.255.255 UHD 0 0 0 lo 224.0.0.6 * 255.255.255.255 UHD 0 0 0 lo 9.9.9.9 202.0.0.1 255.255.255.255 UGHD 0 0 0 eth1 224.0.0.5 * 255.255.255.255 UHD 0 0 0 lo 9.9.9.10 202.0.0.1 255.255.255.255 UGHD 0 0 0 eth1 localhost.local * 255.255.255.255 UH 0 0 0 lo 1.0.0.0 202.0.0.1 255.255.255.0 UGD 0 0 0 eth1 2.0.0.0 202.0.0.1 255.255.255.0 UGD 0 0 0 eth1 202.0.0.0 * 255.255.255.0 U 0 0 0 eth1 200.0.0.0 202.0.0.1 255.255.255.0 UGD 0 0 0 eth1 127.0.0.0 * 255.0.0.0 U 0 0 0 lo [Expert@MPLS]# ~ |
[Expert@INET]#
drouter stop; service network restart; route Kernel IP routing table Destination Gateway Genmask Flags Metric Ref Use Iface 101.0.0.0 * 255.255.255.0 U 0 0 0 eth1 100.0.0.0 * 255.255.255.0 U 0 0 0 eth0 127.0.0.0 * 255.0.0.0 U 0 0 0 lo [Expert@INET]# |