1 OpFlex agent-ovs Install Guide
2 ==============================
7 You'll need to install the following packages and their dependencies:
12 * openvswitch-gbp-kmod
17 Packages are available for Red Hat Enterprise Linux 7 and Ubuntu 14.04
18 LTS. Some of the examples below are specific to RHEL7 but you can run
19 the equivalent commands for upstart instead of systemd.
21 Note that many of these steps may be performed automatically if you're
22 deploying this along with a larger orchestration system.
24 Host Networking Configuration
25 -----------------------------
27 You'll need to set up your VM host uplink interface. You should
28 ensure that the MTU of the underlying network is sufficient to handle
29 tunneled traffic. We will use an example of setting up *eth0* as your
30 uplink interface with a vlan of 4093 used for the networking control
31 infrastructure and tunnel data plane.
33 We just need to set the MTU and disable IPv4 and IPv6
34 autoconfiguration. The MTU needs to be large enough to allow both the
35 VXLAN header and VLAN tags to pass through without fragmenting for
36 best performance. We'll use 1600 bytes which should be sufficient
37 assuming you are using a default 1500 byte MTU on your virtual machine
38 traffic. If you already have any NetworkManager connections configured
39 for your uplink interface find the connection name and proceed to the
40 next step. Otherwise, create a connection with (be sure to update the
41 variable UPLINK_IFACE as needed)::
44 nmcli c add type ethernet ifname $UPLINK_IFACE
46 Now, configure your interface as follows::
48 CONNECTION_NAME="ethernet-$UPLINK_IFACE"
49 nmcli connection mod "$CONNECTION_NAME" connection.autoconnect yes \
50 ipv4.method link-local \
52 802-3-ethernet.mtu 9000 \
53 ipv4.routes '224.0.0.0/4 0.0.0.0 2000'
55 Then bring up the interface with::
57 nmcli connection up "$CONNECTION_NAME"
59 Next, create the infrastructure interface using the infrastructure
60 VLAN (4093 by default). We'll need to create a vlan subinterface of
61 your uplink interface, the configure DHCP on that interface. Run the
62 following commands. Be sure to replace the variable values if needed. If
63 you're not using NIC teaming, replace the variable team0 below::
67 nmcli connection add type vlan ifname $UPLINK_IFACE.$INFRA_VLAN dev $UPLINK_IFACE id $INFRA_VLAN
68 nmcli connection mod vlan-$UPLINK_IFACE.$INFRA_VLAN \
69 ethernet.mtu 1600 ipv4.routes '224.0.0.0/4 0.0.0.0 1000'
70 sed "s/CLIENT_ID/01:$(ip link show $UPLINK_IFACE | awk '/ether/ {print $2}')/" \
71 > /etc/dhcp/dhclient-$UPLINK_IFACE.$INFRA_VLAN.conf <<EOF
72 send dhcp-client-identifier CLIENT_ID;
73 request subnet-mask, domain-name, domain-name-servers, host-name;
76 Now bring up the new interface with::
78 nmcli connection up vlan-$UPLINK_IFACE.$INFRA_VLAN
80 If you were successful, you should be able to see an IP address when you run::
82 ip addr show dev $UPLINK_IFACE.$INFRA_VLAN
84 OVS Bridge Configuration
85 ------------------------
87 We'll need to configure an OVS bridge which will handle the traffic
88 for any virtual machines or containers that are hosted on the VM
89 host. First, enable the openvswitch service and start it::
91 # systemctl enable openvswitch
92 ln -s '/usr/lib/systemd/system/openvswitch.service' '/etc/systemd/system/multi-user.target.wants/openvswitch.service'
93 # systemctl start openvswitch
94 # systemctl status openvswitch
95 openvswitch.service - Open vSwitch
96 Loaded: loaded (/usr/lib/systemd/system/openvswitch.service; enabled)
97 Active: active (exited) since Fri 2014-12-12 17:20:13 PST; 3s ago
98 Process: 3053 ExecStart=/bin/true (code=exited, status=0/SUCCESS)
99 Main PID: 3053 (code=exited, status=0/SUCCESS)
100 Dec 12 17:20:13 ovs-server.cisco.com systemd[1]: Started Open vSwitch.
102 Next, we can create an OVS bridge (you may wish to use a different
105 # ovs-vsctl add-br br0
107 34aa83d7-b918-4e49-bcec-1b521acd1962
112 ovs_version: "2.3.90"
114 Next, we configure a tunnel interface on our new bridge as follows::
116 # ovs-vsctl add-port br0 br0_vxlan0 -- \
117 set Interface br0_vxlan0 type=vxlan \
118 options:remote_ip=flow options:key=flow options:dst_port=8472
120 34aa83d7-b918-4e49-bcec-1b521acd1962
123 Interface "br0_vxlan0"
125 options: {dst_port="8472", key=flow, remote_ip=flow}
129 ovs_version: "2.3.90"
131 Open vSwitch is now configured and ready.
136 Before enabling the agent, we'll need to edit its configuration file,
137 which is located at "/etc/opflex-agent-ovs/opflex-agent-ovs.conf".
139 First, we'll configure the Opflex protocol parameters. If you're using
140 an ACI fabric, you'll need the OpFlex domain from the ACI
141 configuration, which is the name of the VMM domain you mapped to the
142 interface for this hypervisor. Set the "domain" field to this
143 value. Next, set the "name" field to a hostname or other unique
144 identifier for the VM host. Finally, set the "peers" list to contain
145 the fixed static anycast peer address of 10.0.0.30 and port 8009. Here
146 is an example of a completed section (bold text shows areas you'll
150 // The globally unique policy domain for this agent.
151 "domain": "[CHANGE ME]",
153 // The unique name in the policy domain for this agent.
154 "name": "[CHANGE ME]",
156 // a list of peers to connect to, by hostname and port. One
157 // peer, or an anycast pseudo-peer, is sufficient to bootstrap
158 // the connection without needing an exhaustive list of all
161 {"hostname": "10.0.0.30", "port": 8009}
165 // SSL mode. Possible values:
166 // disabled: communicate without encryption
167 // encrypted: encrypt but do not verify peers
168 // secure: encrypt and verify peer certificates
171 // The path to a directory containing trusted certificate
172 // authority public certificates, or a file containing a
173 // specific CA certificate.
174 "ca-store": "/etc/ssl/certs/"
178 Next, configure the appropriate policy renderer for the ACI
179 fabric. You'll want to use a stitched-mode renderer. You'll need to
180 configure the bridge name and the uplink interface name. The remote
181 anycast IP address will need to be obtained from the ACI configuration
182 console, but unless the configuration is unusual, it will be
185 // Renderers enforce policy obtained via OpFlex.
187 // Stitched-mode renderer for interoperating with a
188 // hardware fabric such as ACI
190 "ovs-bridge-name": "br0",
192 // Set encapsulation type. Must set either vxlan or vlan.
194 // Encapsulate traffic with VXLAN.
196 // The name of the tunnel interface in OVS
197 "encap-iface": "br0_vxlan0",
199 // The name of the interface whose IP should be used
200 // as the source IP in encapsulated traffic.
201 "uplink-iface": "eth0.4093",
203 // The vlan tag, if any, used on the uplink interface.
204 // Set to zero or omit if the uplink is untagged.
207 // The IP address used for the destination IP in
208 // the encapsulated traffic. This should be an
209 // anycast IP address understood by the upstream
210 // stitched-mode fabric.
211 "remote-ip": "10.0.0.32"
214 // Configure forwarding policy
216 // Configure the virtual distributed router
218 // Enable virtual distributed router. Set to true
219 // to enable or false to disable. Default true.
222 // Override MAC address for virtual router.
223 // Default is "00:22:bd:f8:19:ff"
224 "mac": "00:22:bd:f8:19:ff",
226 // Configure IPv6-related settings for the virtual
229 // Send router advertisement messages in
230 // response to router solicitation requests as
231 // well as unsolicited advertisements.
232 "router-advertisement": true
236 // Configure virtual distributed DHCP server
238 // Enable virtual distributed DHCP server. Set to
239 // true to enable or false to disable. Default
243 // Override MAC address for virtual dhcp server.
244 // Default is "00:22:bd:f8:19:ff"
245 "mac": "00:22:bd:f8:19:ff"
249 // Location to store cached IDs for managing flow state
250 "flowid-cache-dir": "DEFAULT_FLOWID_CACHE_DIR"
254 Finally, enable the agent service::
256 # systemctl enable agent-ovs
257 ln -s '/usr/lib/systemd/system/agent-ovs.service' '/etc/systemd/system/multi-user.target.wants/agent-ovs.service'
258 # systemctl start agent-ovs
259 # systemctl status agent-ovs
260 agent-ovs.service - Opflex OVS Agent
261 Loaded: loaded (/usr/lib/systemd/system/agent-ovs.service; enabled)
262 Active: active (running) since Mon 2014-12-15 10:03:42 PST; 5min ago
263 Main PID: 6062 (agent_ovs)
264 CGroup: /system.slice/agent-ovs.service
265 └─6062 /usr/bin/agent_ovs
267 The agent is now running and ready to enforce policy. You can add
268 endpoints to the local VM hosts using the OpFlex Group-based policy
269 plugin from OpenStack, or manually.