| Table of Contents |
|---|
In ICN's SDWAN usages, SFC (Service Function Chain) is designed to support Corp networks to connect to external internet with security connection. The SFC includes Security VNF (e.g. firewall etc.), WAN Opt CNF and SDWAN VNF/CNF, and SDWAN module is worked as software defined router which can be used to defined the rules when connect to external internet. Below diagram shows where SDWAN module located in the whole system.
Basic
...
Technologies
OpenWRT
The OpenWRT Project (https://openwrt.org/) is an open source project based on Linux, and it is primarily used on embedded devices to route network traffic. There are more than 3500 software packages which can be installed on OpenWRT via opkg package management system. OpenWRT provides both docker image and VM image to support virtualization solution (https://openwrt.org/docs/guide-user/virtualization/start). In ICN, we run OpenWRT in container.
OpenWRT Mwan3 package (a replacement for multiwan package) provides the capabilities for multiple WAN management: WAN interfaces management, outbound traffic rules, traffic load balancing etc.
ovn4nfv-k8s-plugin
ovn4nfv-k8s-plugin is a CNI plugin based on ovn. It can work together with Multus CNI to add multiple interfaces for the pod. One of the interfaces is the Multus default interface, it could be flannel, calico, etc. The other interfaces are added by ovn4nfv-k8s-plugin according the the pod annotation. With ovn4nfv-k8s-plugin, we can create virtual network in run-time. Also we can connect the pod with the provider network, this is important for CNF.
Design
...
Implementation
SDWAN module is implemented as CNF instead of VNF for better performance and proving of CNF can also be effect way to deploy SDWAN
SDWAN will leverage OpenWRT for it functionality:
- Base Image: x86-generic rootfs image (http://downloads.openwrt.org/releases/18.06.4/targets/x86/generic/)
- Packages:
- mwan3: multiple wans configuration and management
- luci-app-mwan3: web configuration interface for mwan3
- uhttp: web server for luci services
Deployment
Helm will be used to deploy SDWAN CNF as pod on nodes with external network connection with below parameters:
...
WAN interface definition (e.g. interface, member etc.)
...
config interface 'wan' option enabled '1' list track_ip '8.8.4.4' list track_ip '8.8.8.8' list track_ip '208.67.222.222' list track_ip '208.67.220.220' option reliability '2' option count '1' option timeout '2' option interval '5' option down '3' option up '8'
config member 'wan_m1_w3' option interface 'wan' option metric '1' option weight '3'
Initial Traffic rules (e.g. policy, rule etc.)
...
config policy 'wan_only' list use_member 'wan_m1_w3'
config rule 'sticky_even' option src_ip '0.0.0.0/0.0.0.1' option dest_port '443' option proto 'tcp' option use_policy 'wan_only'
Runtime Configuration
SDWAN traffic rules and WAN interfaces are required to be configured at runtime through Restful API interface.
OpenWRT luci invoking mechanism:
(1) logon: POST /cgi-bin/luci with luci_username and luci_password to get sysauth Cookie
e.g. wget --post-data "luci_username=root&luci_password=" http://192.168.56.2/cgi-bin/luci/
response header will include sysauch cookie, like: "Set-Cookie: sysauth=e5b2e5c2ae0099c078bb3cb72052ed95;"
(2) Call luci http service with sysauth Cookie
e.g. wget --header="Cookie:sysauth=e5b2e5c2ae0099c078bb3cb72052ed95" http://192.168.56.2/cgi-bin/luci
Mwan3 supported services:
interface_status: GET /cgi-bin/luci/admin/status/mwan/interface_status
...
Sample response:
"interfaces":{"wan":{"running":true,"score":0,"track_ip":[{"status":"down","latency":0,"packetloss":0,"ip":"208.67.220.220"},{"status":"down","latency":0,"packetloss":0,"ip":"208.67.222.222"},{"status":"down","latency":0,"packetloss":0,"ip":"8.8.4.4"},{"status":"down","latency":0,"packetloss":0,"ip":"8.8.8.8"}],"lost":536,"status":"offline","age":5,"turn":134},"wanb6":{"running":false,"score":0,"track_ip":[],"lost":0,"status":"","age":0,"turn":0},"wanb":{"running":false,"score":0,"track_ip":[],"lost":0,"status":"","age":0,"turn":0},"wan6":{"running":false,"score":0,"track_ip":[],"lost":0,"status":"","age":0,"turn":0}},"connected":{"ipv6":["Members:","fe80::\/64"],"ipv4":["Members:","224.0.0.0\/3","192.168.56.2","127.0.0.0\/8","127.255.255.255","192.168.56.0\/24","10.0.3.0\/24","10.0.3.15","127.0.0.0","192.168.56.255","127.0.0.1","192.168.56.0","10.0.3.0","10.0.3.255"]}
...
Note: configuration for WAN's interface, member, policy and rule are supported in the web UI, but with no direct restful API available. It needs implement luci cgi plugin to provide restful API for WAN configuration. In ICN, one interface is implemented as POC to support running commands in CNF through Rest API call.
command:
POST /cgi-bin/luci/admin/config/command
Execute commands in OpenWRT CNF
Normal Response Code: 200
Error Response Code: 400
Request:
...
SDWAN CNF Controller Implementations
The Akraino-SDEWAN-OutOfBoxNetworking.pptx describes the whole SDEWAN solution design.
ICN R3 release will focus on implementing the SDEWAN CNF and EWAN config Agent then designing the End-to-End demo with other components described in the SDEWAN solution.
Below diagram describes the interaction between SDEWAN CNF and EWAN config Agent.
Timeline
...
Start/stop/restart/reload SDWAN service, includes: mwan3, firewall/NAT, IpSec.
Reference: SDEWAN CNF#SDEWANService
...
Support MWAN3 rule/policy configuration.
Reference: SDEWAN CNF#MWAN3
OpenWRT Reference: https://openwrt.org/docs/guide-user/network/wan/multiwan/mwan3
...
Design: Feb.26
Implementation: Mar.12
...
WW08: Initial design Done
WW09: Implementation - 50%
WW10: 80%
WW11: done
...
Support firewall configuration for zone (general rule for a group of interfaces), forwarding (iptables forward), rule, redirect (DNAT/SNAT).
Reference: SDEWAN CNF#Firewall
OpenWRT Reference: https://openwrt.org/docs/guide-user/firewall/firewall_configuration
...
Design: Feb.26
Implementation: Mar.18
WW08: Initial design Done
WW09: design done (to be reviewed)
WW10/11/12: 50%
...
Support IPSec configuration for remote site, proposal.
Reference: https://wiki.akraino.org/display/AK/IPSec+Design#IPSecDesign-IPSecRestAPI
OpenWRT Reference: https://openwrt.org/docs/guide-user/services/vpn/ipsec/strongswan/start
(Note: OpenWRT Wiki page is out-of-date compare to 18.06 implementation which we used and the current design is based on openwrt ipsec code directly)
...
Define a SDWAN CNF with mwan3, firewall and IPSec configuration
Reference: Sdewan config Agent
...
Design: Done
Implementation: Feb. 26
WW08: CRD design done, implementation: - Done
...
Define MWAN3 configuration (policy, rule)
Reference: Sdewan config Agent
...
Design: Feb.26
Implementation: Mar.12
...
WW09: CRD design - Done
WW10/11: Done
...
Design: Feb.26
Implementation: Mar.18
...
WW08: initial design done
WW09: design done (to be reviewed)
WW10/11/12: implementation 80%
...
Define IPSec CRD (remote site, proposal)
Reference: https://wiki.akraino.org/display/AK/IPSec+Design#IPSecDesign-IPSecCRD
Scenario design: SD-EWAN Scenarios
...
SDEWAN CNF
...
Overview
SDEWAN is a solution to enable SDWAN functionalities include multiple WAN link support, WAN traffic management, NAT, firewall, IPSec and Traffic shaping etc. with focus to address the challenges when applying on edge computing environment like resource limitation, edge overlays, traffic sanitization, automation and cost sensitive etc. The solution includes below components:
- SDEWAN CNF: implemented based on OpenWRT, it enhances OpenWRT Luci web interface with SDEWAN controllers to provide Restful API for network functions' configuration and control.
- SDEWAN CRD Controller: implemented as k8s CRD Controller, it manages CRDs (e.g. Firewall related CRDs, Mwan3 related CRDs and IPSec related CRDs etc.) and internally calls SDEWAN Restful API to do CNF configuration.
- Overlay Controller: provides central control of SDEWAN overlay networks by automatically configuring the SDEWAN CNFs through SDEWAN CRD controller located in edge location clusters and hub clusters.
SDEWAN CNF
Sdewan CRD Controller
Presentation
ICN Weekly meeting video recordings- Weekly Akraino ICN Engineering Meeting