Building remote access to your cellular devices with Soracom?

Wireless and IoT
, ,
1

Introduction

Hi I’m taketo from Soracom.

Soracom provides cellular connectivity services for IoT with powerful APIs and the web console known as Soracom User Console to manage your connectivity. Please find out our products and featured videos here at Digi-Key.

In this topic, I will introduce an architecture and configurations to build remote access to your cellular devices from your private network with redundancy.
For security reason we have carrier grade NAT for cellular services. So as default your sims have only private IP addresses which are not reachable from the internet or your servers on your private network.
However, if you need to build a network system to access from your private network to your cellular devices. The proposed architecture in this topic is the answer!
It enabled your servers in your private network to reach your cellular devices with private IP addresses.

Tunneling and Overlay with Redundant Gate Peer

By configuring a Gate Peer within your private network environment, you can establish a connection from your private network back to your Soracom Virtual Private Gateway (VPG) in order to enable remote device access.

Gate Peer configuration with redundancy involves the following steps:

  • Create two servers that will act as the Gate Peer.
  • Register your servers and their networking parameters on your VPG.
  • Configure your servers to establish VXLAN connections using your VPG parameters.
  • Configure your servers to have a virtual ip address by VRRP
  • Configure Junction Redirection on your VPG with your virtual ip address

If you do not need redundancy, please follow SORACOM GATE: Gate Peer Configuration documentation for instructions.

Terminology

The Gate Peer is a Network equipment that extends a Soracom private network into your own private network. It connects the VXLAN tunnel to VPG. When creating a VPG, two Gate Peers are automatically created inside the VPG (each residing in a different AWS Availability Zone). This document explains the process of setting up corresponding Gate Peers within your environment. Gate Peers will handle the task of sending traffic between your network environment and the VPG, in order to allow device access. A Gate Peer consists of a network box device that supports VXLAN or a Linux-based server. This document introduces the procedure to configure Gate Peer based on EC2.

Outer IP Address - The Outer IP address is the connection destination endpoint IP address that Gate Peers mutually specify in the VXLAN connection settings. Each Gate Peer will have a physical network interface within its respective network environment. The Outer IP Address refers to the address assigned to this interface. For Gate Peers in the VPG, addresses in the 100.64.0.0/16 range will automatically be assigned. For Gate Peers in your private network, this address should be the IP address within your private network, such as within a 10.0.0.0/8, 172.16.0.0/12, or 192.168.0.0/16 block.

Inner IP Address - Each Gate Peer will also have a VXLAN interface which is used to send network traffic between Gate Peers. The Inner IP Address refers to the address assigned to the VXLAN interface.

An example of Architectures

The diagram below shows an example of architectures that you will build in the following steps. All the IP addresses are examples and may be different in your environment.

An example of Architectures
An example of Architectures1151×567 76.9 KB

---

Requirements

In order to establish the VXLAN connection, you must have an underlying network connection between your Soracom VPG and your private network. Currently, Gate Peers can be configured for use with one of the following underlying connections:

Before continuing with Gate Peer configuration, ensure that you have completed the following:

With these requirements completed, continue to configure your Gate Peers.

This document will use an AWS EC2 instance as the Gate Peer and refer to AWS-specific configuration instructions. However, in general the same steps can be applied when configuring a Gate Peer within a non-AWS environment.

Create Gate Peers

Create two Linux virtual machine instances in your private network in different isolated data centers, giving each of them a statically configured IP address within your private network’s CIDR block range. It is recommended to follow best practices for securing the machine, such as utilizing SSH keys instead of passwords for accessing the machine. Make a note of its static IP address, this will be your Gate Peer’s Outer IP Address and is needed later when registering this instance as a Gate Peer.

  • In your AWS environment, you should launch an Amazon Linux AMI or similar instance, configuring its network details to use the VPC where Soracom Canal has been configured.

:information_source: A Linux virtual machine is required in order to establish a VXLAN connection with the Soracom VPG. Please note that while VXLAN implementations are available for Windows servers, they are currently unstable and we do not recommend them for production environments.

Then configure the virtual machine’s networking configuration to allow connections on the following ports/protocols:

Protocol Port Direction Source Description
TCP 22 Inbound 0.0.0.0/0 Allow connections over SSH to configure instance.
UDP 4789 Inbound 100.64.0.0/16 Allow VXLAN traffic from VPG.
UDP 4789 Inbound 172.16.0.0/16 Allow VXLAN traffic from your private network.
ICMP — Inbound 0.0.0.0/0 Enable responses to ping requests.
  • For AWS environments, you can configure your AMI instance’s access policy by creating or updating its Security Group.

:warning: Configuring your virtual machine instance to allow inbound TCP traffic on port 22 from 0.0.0.0/0 (any source) is intended only for remotely connecting to the instance in order to configure it. Leaving this configuration as-is will expose your instance to other external access.

:warning: Once you have verified that your instance has been configured, you should update this configuration to restrict external access, or remove this configuration altogether. You may also prefer to similarly restrict or remove ICMP traffic.

You can also add any application-specific ports/protocols, such as TCP 80 or 443 in order to allow HTTP access to devices. Where necessary, specify your private network’s CIDR block as the Source address in order to confine remote access to requests originating from your private network. Also, refer to the AWS guide for VPC network security design and settings.

When using an AWS EC2 instance as a Gate Peer, the instance will reject any traffic where the destination does not match the EC2 network interface details by default, using a Source/Destination check policy. You must disable this policy so that this instance can receive traffic to send to the VPG. Please refer to the VPC NAT Instances: Disabling Source/Destination Checks documentation for relevant instructions.

  • For AWS environments, you can configure create and IAM Role for your Gate Peer servers.

To route traffic from your private network to devices through an active Gate Peer server. And when you have two Gate Peer servers for redundancy and if the standby Gate Peer server takes over the master Gate Peer server, you will need to change the routing table to route the traffic to the standby Gate Peer server.

To change routing tables from your Gate Peer servers, you must create and attach an IAM role with ec2:CreateRoute and ec2:DeleteRoute on the route tables of your VPC to your Gate Peer servers.

Register your Gate Peers and Virtual Gate Peer

1. Register your Gate Peers

Registering your Gate Peer servers tells your Soracom VPG about its existence, in order to establish the VXLAN connections between your private network and the VPG.

Gate Peer registration simply requires your server’s static IP address within your private network.

During registration, an Inner IP Address will be automatically assigned to your Gate Peer, which will be used to establish the VXLAN connection between your server and the VPG. You can manually specify this IP address in situations where you may have an existing IP address conflict, however in most cases auto-assignment is recommended.

You can register your Gate Peers from the User Console. Please repeat following steps for your two Gate Peers:

  1. Login to the User Console. From the Menu, open the VPG screen.
  2. From the list of VPGs, click the name of the VPG you want to configure to open its settings page.
  3. Click the Device LAN settings tab.
  4. Click the Add Gate Peer button.
  5. Enter the Tunnel Endpoint IP address (Outer IP Address) using your server’s static IP address within your private network’s CIDR block.
    If required, you can also specify the Device Subnet IP address (Inner IP Address). In most situations, leaving this blank is recommended.
    Then click the Create button.

Your servers will now be registered as Gate Peers, and will appear in the Gate Peers in your network section. Make a note of the displayed EC2 configuration script. We will use it in the next steps.

Note: The contents of the script displayed for each Gate Peer are different. Please change options of commands depending on your environment.

Now that the VPG is aware of the Gate Peers in our private network, we can configure our server to create the VXLAN connection in order to complete configuration.

2. Register your Virtual Gate Peer

Register Virtual Gate Peer in the same way as Gate Peer registration. For Outer IP, enter an IP that is not used on your private network (172.16.1.150 in this example), and for Inner IP, specify a Virtual IP (10.0.1.150 in this example) shared between Gate Peers shown in the network configuration diagram.

Configure your Gate Peer

Run the EC2 configuration script obtained in the previous step on EC2.
To build redundancy with Keepalived, you must register a virtual ip address as your “Virtual” Gate Peer. The outerIpAdress of your Virtual Gate Peer 172.16.1.150, for example, does not need to exist.
Make a note of the various IP addresses listed in the Gate Peers in VPG section.
Together with the IP addresses of your Gate Peer, we should have the following IP addresses noted:

Gate Peer Tunnel Endpoint IP address
outerIpAddress		 Device Subnet IP address
innerIpAddress
Your Gate Peer 1 in your network 172.16.1.58 10.0.66.46
Your Gate Peer 2 in your network 172.16.16.129 10.0.137.212
Your Virtual Gate Peer in your network 172.16.1.150 10.0.1.150
Soracom Gate Peer 1 in VPG 100.67.1.68 10.0.1.68
Soracom Gate Peer 2 in VPG (2) 100.67.1.84 10.0.1.84

Now, let’s use these IP addresses to configure our server. The following instructions are written for an AWS EC2 Amazon Linux AMI instance. If using other virtual machines, you may need to alter the commands according to your Linux environment.
Please repeat the following steps For your Gate Peers.

1.SSH to your Gate Peer server.

2.Run the following commands. The command options may need to be changed for your environment that you set up in the step:

  • Download soracom setup script:
wget https://soracom-files.s3-ap-northeast-1.amazonaws.com/gate-peer-tools/gate_init_vxlan.sh
chmod +x gate_init_vxlan.sh
  • Establish the VXLAN connection:
#This is an example for your Gate Peer 1.
sudo ./gate_init_vxlan.sh \
    eth0 \
    172.16.1.58 \
    vxlan0 \
    10.0.66.46 \
    16 \
    10 \
    100.67.1.68 \
    100.67.1.84 \
    172.16.16.129
  • This command uses the following options:
    • eth0 - Network interface name of gate peer.
    • 172.16.1.58 - The outerIpAddress of your Gate Peer 1. If you execute this command in your Gate Peer 2, replace here with the outerIpAddress of your Gate Peer 2 172.16.16.129.
    • 16 - Set the subnet mask for the device subnet IP address range. For example, if the device subnet is 10.0.0.0/16, set it to 16.
    • 10- Set the VXLAN Network Identifier to 10 and specify the remote VXLAN tunnel endpoint port.
    • 100.67.1.68- The outerIpAddress of Soracom Gate Peer 1 in VPG.
    • 100.67.1.84- The outerIpAddress of Soracom Gate Peer 2 in VPG.
    • 172.16.1.129 - The outerIpAddress of your Gate Peer 2. If you execute this command in your Gate Peer 2, replace here with the outerIpAddress of your Gate Peer 1 172.16.16.58

:information_source: Alternatively you could set up VXLAN connections without gate_init_vxlan.sh , please refer to SORACOM GATE: Gate Peer Configuration documentation for instructions and examples.

  • Enable ip forwarding.
sudo sed -i -e '$a net.ipv4.ip_forward = 1' /etc/sysctl.conf
  • To use an virtual ip with Keepalived, enable the ability to bind to an IP address that is nonlocal, meaning that it is not assigned to a device on the local system.
sudo sed -i -e '$a net.ipv4.ip_nonlocal_bind = 1' /etc/sysctl.conf
  • Apply configurations in /etc/sysctl.conf.
sudo sysctl -p
  • Install Keepalived.
sudo yum install keepalived
  • Edit /etc/keepalived/keepalived.conf and configure Keepalived.
#Gate Peer1: /etc/keepalived/keepalived.conf
global_defs {
    vrrp_garp_master_refresh 60
}
vrrp_instance VI_1 {
    state MASTER #Work as a master server
    interface vxlan0
    virtual_router_id 51
    priority 160 #Higher value than your Gate Peer 2
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass 1234
    }
    unicast_peer {
        10.0.137.212
    }
    virtual_ipaddress {
        10.0.1.150
    }
    notify /usr/local/bin/switch-route-trigger.sh
}

#Gate Peer2: /etc/keepalived/keepalived.conf
global_defs {
    vrrp_garp_master_refresh 60
}
vrrp_instance VI_1 {
    state BACKUP #Work as a standby(backup) server
    interface vxlan0
    virtual_router_id 51
    priority 150  #Lower value than your Gate Peer 2
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass 1234
    }
    unicast_peer {
        10.0.66.46
    }
    virtual_ipaddress {
        10.0.1.150
    }
    notify /usr/local/bin/switch-route-trigger.sh
}
  • Create a script /usr/local/bin/switch-route-trigger.sh to switch routing.
#!/bin/bash
####/usr/local/bin/switch-route-trigger.sh
TYPE=$1
NAME=$2
STATE=$3

case $STATE in
        "MASTER") echo "MASTER"
                  /usr/local/bin/switch-route.sh
                  exit 0
                  ;;
        "BACKUP") echo "BACKUP"
                  exit 0
                  ;;
        "FAULT")  echo "FAULT"
                  exit 0
                  ;;
        *)        echo "unknown state"
                  exit 1
                  ;;
esac
  • Create a script /usr/local/bin/switch-route.sh to switch routing.
#!/bin/bash
####/usr/local/bin/switch-route.sh
ROUTE_TABLE_ID="rtb-0fc81b93f08c6a437" #replace with your route table id.
TARGET_CIDR="10.0.0.0/16" #replace here with your device subnet.
echo "switching route to ${TARGET_CIDR} of route table ${ROUTE_TABLE_ID}"
aws ec2 delete-route --region ap-northeast-1 --route-table-id ${ROUTE_TABLE_ID} --destination-cidr-block ${TARGET_CIDR}
aws ec2 create-route --region ap-northeast-1 --route-table-id ${ROUTE_TABLE_ID} --destination-cidr-block ${TARGET_CIDR} --instance-id `wget -q -O - http://169.254.169.254/latest/meta-data/instance-id`

This command uses the following options:
ROUTE_TABLE_ID - The route table id of your VPC.
TARGET_CIDR - Your device subnet.
Change the permission of the scripts and make them executable.
chmod +x /usr/local/bin/switch-route-trigger.sh
chmod +x /usr/local/bin/switch-route.sh
  • Start Keepalived.
sudo systemctl start keepalived

Now Keepalived has been configured to your Gate Peer servers. And the virtual ip address 10.0.1.150 should be assigned to your Gate Peer 1 as a master server. And if your Gate Peer 1 goes down, the virtual ip address 10.0.1.150 will be attached to your Gate Peer 2.

3.Configure your Junction Redirection on your VPG

  • Refer to the SORACOM Junction: Redirection documentation for instructions.
    • Gateway IP address must be 10.0.1.150, which is the outerIpAddress of your Virtual Gate Peer in your network.

Now our Gate Peer has been properly configured to accept packets that need to be sent to the VPG, and forward them to the corresponding Gate Peers using the VXLAN connection, which in turn will route them to the device for us. Each packet will also contain our Gate Peer’s IP address as the packet source, ensuring any responses will be routed back to us correctly.

:warning: The interface and routing settings configured here are not permanent and will be lost if the Gate Peer is restarted. After testing remote device access, configure your server to perform this configuration automatically in order to ensure Gate Peer availability.

Confirm Remote Device Access

Now that the Gate Peer has been configured, you should be able to remotely access a cellular device attached to your Soracom VPG.

To test, first SSH to your application server.

Then from within the SSH session, you can test a ping command, curl an HTTP resource on the device, or even open up another ssh session.

ping 10.0.184.255
PING 10.0.184.255 (10.0.184.255) 56(84) bytes of data.
64 bytes from 10.0.184.255: icmp_seq=1 ttl=64 time=816 ms
64 bytes from 10.0.184.255: icmp_seq=2 ttl=64 time=403 ms
64 bytes from 10.0.184.255: icmp_seq=3 ttl=64 time=423 ms
64 bytes from 10.0.184.255: icmp_seq=4 ttl=64 time=422 ms

:information_source: Although routing between network environments via the VXLAN connection is set up between your Gate Peer and the Soracom VPG, you will need to perform additional network configuration within your private network in order to route local traffic to your Gate Peer. As this process varies for each network environment, please test your routing configuration in order to confirm that only traffic intended for Soracom Air devices is routed accordingly.

Confirm Gate Peer Redundancy

Now that the Gate Peer redundancy has been configured by Keepalived, the outerIpAddress of your Virtual Gate Peer 10.0.1.150 should be assigned to a standby Gate Peer server.

  • 1.Keep sending ICMP packets from a cellular device attached to your Soracom VPG.
    Please keep sending packers, while your Gate Peer 2 takes over.
ping 172.16.37.24
PING 172.16.37.24 (172.16.37.24) 56(84) bytes of data.
64 bytes from 172.16.37.24: icmp_seq=1 ttl=64 time=816 ms
64 bytes from 172.16.37.24: icmp_seq=2 ttl=64 time=403 ms
64 bytes from 172.16.37.24: icmp_seq=3 ttl=64 time=423 ms
64 bytes from 172.16.37.24: icmp_seq=4 ttl=64 time=422 ms
  • 2.SSH to your Gate Peer 1 and Gate Peer 2, and check if Gate Peer 1 is working as a Master server.
    Confirm if the virtual ip address 10.0.1.150 is attached to your Gate Peer 1.
##Gate Peer 1
ip a | grep vxlan0
7: vxlan0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 8951 qdisc noqueue state UNKNOWN group default qlen 1000
    inet 10.0.66.46/16 brd 10.0.255.255 scope global vxlan0
    inet 10.0.1.150/32 scope global vxlan0

##Gate Peer 2
ip a | grep vxlan0
6: vxlan0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 8951 qdisc noqueue state UNKNOWN group default qlen 1000
    inet 10.0.137.212/16 brd 10.0.255.255 scope global vxlan0
  • 3.Stop it’s Keepalived in your Gate Peer 1.
##Gate Peer 1
sudo systemctl stop keepalived
  • 4.Check if Gate Peer 2 takes over the virtual ip address 10.0.1.150.
##Gate Peer 1
ip a | grep vxlan0
7: vxlan0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 8951 qdisc noqueue state UNKNOWN group default qlen 1000
    inet 10.0.66.46/16 brd 10.0.255.255 scope global vxlan0

##Gate Peer 2
ip a | grep vxlan0
6: vxlan0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 8951 qdisc noqueue state UNKNOWN group default qlen 1000
    inet 10.0.137.212/16 brd 10.0.255.255 scope global vxlan0
    inet 10.0.1.150/32 scope global vxlan0
  • 5.Check if sending ICMP packets from a cellular device attached to your Soracom VPG has been successful.
ping 172.16.37.24
PING 172.16.37.24 (172.16.37.24) 56(84) bytes of data.
64 bytes from 172.16.37.24: icmp_seq=1 ttl=64 time=816 ms
64 bytes from 172.16.37.24: icmp_seq=2 ttl=64 time=403 ms
64 bytes from 172.16.37.24: icmp_seq=3 ttl=64 time=423 ms
64 bytes from 172.16.37.24: icmp_seq=4 ttl=64 time=422 ms

:information_source: While your Gate Peer 2 takes over the traffic, it executes scripts to change the route table of your VPC. You will see a record to route traffic for the device subnet 10.0.0.0/16 to Gate Peer 2.

Enable Internet Access from your cellar devices

By enabling Junction Redirection, your VPG forwards all the traffic from your cellular devices to your Gate Peer servers in your private network regardless if your VPG has an internet gateway. Because of this, your cellular devices lose internet connection. If your cellular devices need the internet, you must configure your Gate Peer servers to forward traffic bound for the internet.

  1. SSH to your Gate Peer servers, and enable NAT feature.
sudo iptables -t nat -A POSTROUTING -j MASQUERADE ! -d 172.16.0.0/16
  • This command uses the following options:
    • 172.16.0.0/16 - The network subnet of your private network.

:information_source: To integrate with your Route 53 private hosted zone, please refer to the other topic How to resolve DNS records in your private network with Soracom? for the details.

2 Likes