1 .. contents:: Table of Contents
4 =======================================================
5 ACL Remote ACL - Indirection Table to Improve Scale
6 =======================================================
7 ACL Remote ACL Indirection patches:
8 https://git.opendaylight.org/gerrit/#/q/topic:remote_acl_indirection
10 This spec is to enhance the initial implementation of ACL remote ACLs filtering which was released
11 in Boron. The Boron release added full support for remote ACLs, however the current implementation
12 does not scale well in terms of flows. The Carbon release will update the implementation to
13 introduce a new indirection table for ACL rules with remote ACLs, to reduce the number of necessary
14 flows, in cases where the port is associated with a single ACL. Due to the complication of
15 supporting multiple ACLs on a single port, the current implementation will stay the same for these
21 Today, for each logical port, an ACL rule results in a flow in the ACL table (ACL2). When a remote
22 ACL is configured on this rule, this flow is multiplied for each VM in the remote ACL, resulting in
23 a very large number of flows.
25 For example, consider we have:
28 - 50 VMs on each compute (5000 VMs total),
29 - All VMs are in a SG (SG1)
30 - This SG has a security rule configured on it with remote SG=SG1
31 (it is common to set the remote SG as itself, to set rules within the SG).
33 This would result in 50*5000 = 250,000 flows on each compute, and 25M flows in ODL MDSAL (!).
38 Neutron configuration of security rules, configured with remote SGs. This optimization will be
39 relevant only when there is a single security group that is associated with the port. In case
40 more than one security group is associated with the port - we will fallback to the current
41 implementation which allows full functionality but with possible flow scaling issues.
43 Rules with a remote ACL are used to allow certain types of packets only between VMs in certain
44 security groups. For example, configuring rules with the parent security group also configured
45 as a remote security group, allows to configure rules applied only for traffic between VMs in
46 the same security group.
48 This will be done in the ACL implementation, so any ACL configured with a remote ACL via a different
49 northbound or REST would also be handled.
54 This blueprint proposes adding a new indirection table in the ACL service in each direction, which
55 will attempt to match the "remote" IP address associated with the packet ("dst_ip" in Ingress ACL,
56 "src_ip" in Egress ACL), and set the ACL ID as defined by the ietf-access-control-list in the
57 metadata. This match will also include the ELAN ID to handle ports with overlapping IPs.
59 These flows will be added to the ACL2 table. In addition, for each such ip->SG flow inserted in
60 ACL2, we will insert a single SG metadata match in ACL3 for each SG rule on the port configured with
63 If the IP is associated with multiple SGs - it is impossible to do a 1:1 matching of the SG, so we
64 will not set the metadata at this time and fallback to the current implementation of matching all
65 possible IPs in the ACL table - for this ACL2 will have a default flow passing the unmatched packets
66 to ACL3 with an empty metadata SG_ID write (e.g. 0x0), to prevent potential garbage in the metadata
69 This means that on transition from a single SG on the port to multiple SG (and back), we would need
70 to remove/add these flows from ACL2, and insert the correct rules into ACL3.
74 - This is the ACL that has default allow rules - it is left untouched, and usually goes to ACL2.
78 - For each port with a single SG - we will match on the IPs and the ELAN ID (for tenant awareness)
79 here, and set the SG ID in the metadata, before going to the ACL3 table.
80 - For any port with multiple SGs (or with no SG) - an empty value (0x0) will be set as the SG ID in
81 the metadata, to avoid potential garbage in the SG ID, and goto ACL3 table.
85 - For each security rule that *doesn't have* a remote SG, we keep the behavior the same: ACL3
86 matches on rule, and resubmits to dispatcher if there is a match (Allow). The SG ID in the metadata
87 will **not** be matched.
88 - For each security rule that *does have* a remote SG, we have two options:
90 - For ports belonging to the remote SG that are associated with a single SG - there will be a
91 single flow per rule, matching the SG ID from the metadata (in addition to the other rule matches)
92 and allowing the packet.
93 - For ports belonging to the remote SG that are associated with multiple SGs - the existing
94 implementation will stay the same, multiplying the rule with all possible IP matches from the
95 remote security groups.
97 Considering the example from the problem description above, the new implementation would result in a
98 much reduced number of flows:
100 5000+50 = 5050 flows on each compute, and 505,000 flows in ODL MDSAL.
102 As noted above, this would require using part of the metadata for writing/matching of an ACL ID. We
103 would likely require at least 12 bits for this, to support up to 4K SGs, where 16 bits to support up
104 to 65K would be ideal. If the metadata bits are not available, we can use a register for this
107 In addition, the dispatcher will set the ELAN ID in the metadata before entering the ACL services,
108 to allow tenant aware IP to SG detection, supporting multi-tenants with IP collisions.
112 ACL3 will be added, and the flows in ACL2/ACL3 will be modified as noted above in the proposed change:
114 ============== ================================================= ===============================================================
116 ============== ================================================= ===============================================================
117 Dispatcher metadata=service_id:ACL write_metadata:(elan_id=ELAN, service_id=NEXT), goto_table:ACL1
118 ACL1 (211/241) goto_table:ACL2
119 ACL2 (212/242) metadata=ELAN_ID, ip_src/dst=VM1_IP write_metadata:(remote_acl=id), goto_table:ACL3
120 ACL2 (212/242) metadata=ELAN_ID, ip_src/dst=VM2_IP write_metadata:(remote_acl=id), goto_table:ACL3
122 ACL2 (212/242) goto_table:ACL3
123 ACL3 (213/243) metadata=lport, <acl_rule> resubmit(,DISPATCHER) :superscript:`(X)`
124 ACL3 (213/243) metadata=lport+remote_acl, <acl_rule> resubmit(,DISPATCHER) :superscript:`(XX)`
125 ACL3 (213/243) metadata=lport,ip_src/dst=VM1_IP, <acl_rule> resubmit(,DISPATCHER) :superscript:`(XXX)`
126 ACL3 (213/243) metadata=lport,ip_src/dst=VM2_IP, <acl_rule> resubmit(,DISPATCHER) :superscript:`(XXX)`
128 ============== ================================================= ===============================================================
130 | (X) These are the regular rules, not configured with any remote SG.
131 | (XX) These are the proposed rules with the optimization - assuming the lport is using a single ACL.
132 | (XXX) These are the remote SG rules in the current implementation, which we will fall back to if the lport has multiple ACLs.
136 Currently the Ingress ACLs use tables *40,41,42* and the Egress ACLs use tables *251,252,253*.
138 Table 43 is already proposed to be taken by SNAT, and table 254 is considered invalid by OVS.
139 To overcome this and align Ingress/Egress with symmetric numbering, I propose the following change:
141 - Ingress ACLs: 211, 212, 213, 214
142 - Egress ACLs: 241, 242, 243, 244
144 ACL1: INGRESS/EGRESS_ACL_TABLE
145 ACL2: INGRESS/EGRESS_ACL_REMOTE_ACL_TABLE
146 ACL3: INGRESS/EGRESS_ACL_FILTER_TABLE
148 ACL4 is used only for Learn implementation for which an extra table is required.
155 ---------------------
158 Clustering considerations
159 -------------------------
162 Other Infra considerations
163 --------------------------
166 Security considerations
167 -----------------------
170 Scale and Performance Impact
171 ----------------------------
172 See example in description.
173 The scale of the flows will be drastically reduced when using remote ACLs.
181 For fully optimized support in all scenarios for remote SGs, meaning including support for ports
182 with multiple ACLs on them, we did consider implementing a similar optimization.
184 However, for this to happen due to OpenFlow limitations we would need to introduce an internal
185 dispatcher inside the ACL services, meaning we loop the ACL service multiple times, each time
186 setting a different metadata SG value for the port.
188 For another approach we could use a bitmask, but this would limit the number of possible SGs to be
189 the number of bits in the mask, which is much too low for any reasonable use case.
193 Any configuration of ACL rules with remote ACLs will receive this optimization if the port is using
196 Functionality should remain as before in any case.
200 Install the ODL Karaf feature for NetVirt (no change):
202 - odl-netvirt-openstack
210 Refer to the Neutron CLI Reference [#]_ for the Neutron CLI command syntax for managing Security
211 Rules with Remote Security Groups.
218 Who is implementing this feature? In case of multiple authors, designate a primary assigne and other
223 - Alon Kochba <alonko@hpe.com>
224 - Aswin Suryanarayanan <asuryana@redhat.com>
233 `Task list in Carbon Trello
234 <https://trello.com/c/6WBbSSkr/145-acl-remote-acls-indirection-table-to-improve-scale-remote-acl-indirection>`_
251 We should add tests verifying remote SG configuration functionality.
252 There should be at least:
254 * One security rule allowing ICMP traffic between VMs in the same SG.
255 * One positive test, checking ICMP connectivity works between two VMs using the same SG.
256 * One negative test, checking ICMP connectivity does not work between two VMs, one using the SG
257 configured with the rule above, and the other using a separate security group with all directions
267 .. [#] Neutron Security Groups http://docs.openstack.org/user-guide/cli-nova-configure-access-security-for-instances.html