idnits 2.17.00 (12 Aug 2021) /tmp/idnits6687/draft-ietf-bier-te-arch-10.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- == There are 2 instances of lines with non-RFC6890-compliant IPv4 addresses in the document. If these are example addresses, they should be changed. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year == The document seems to lack the recommended RFC 2119 boilerplate, even if it appears to use RFC 2119 keywords -- however, there's a paragraph with a matching beginning. Boilerplate error? (The document does seem to have the reference to RFC 2119 which the ID-Checklist requires). -- The document date (July 9, 2021) is 315 days in the past. Is this intentional? Checking references for intended status: Proposed Standard ---------------------------------------------------------------------------- (See RFCs 3967 and 4897 for information about using normative references to lower-maturity documents in RFCs) -- Looks like a reference, but probably isn't: '3' on line 964 -- Looks like a reference, but probably isn't: '2' on line 972 -- Looks like a reference, but probably isn't: '1' on line 979 == Missing Reference: 'SI' is mentioned on line 1017, but not defined == Missing Reference: 'I' is mentioned on line 1024, but not defined == Missing Reference: 'VRF' is mentioned on line 2422, but not defined == Outdated reference: A later version (-06) exists of draft-ietf-bier-multicast-http-response-05 == Outdated reference: A later version (-04) exists of draft-ietf-bier-non-mpls-bift-encoding-03 == Outdated reference: A later version (-05) exists of draft-ietf-bier-te-yang-02 == Outdated reference: A later version (-16) exists of draft-ietf-teas-rfc3272bis-11 Summary: 0 errors (**), 0 flaws (~~), 10 warnings (==), 4 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 Network Working Group T. Eckert, Ed. 3 Internet-Draft Futurewei 4 Intended status: Standards Track G. Cauchie 5 Expires: January 10, 2022 Bouygues Telecom 6 M. Menth 7 University of Tuebingen 8 July 9, 2021 10 Tree Engineering for Bit Index Explicit Replication (BIER-TE) 11 draft-ietf-bier-te-arch-10 13 Abstract 15 This memo describes per-packet stateless strict and loose path 16 steered replication and forwarding for Bit Index Explicit Replication 17 packets (RFC8279). It is called BIER Tree Engineering (BIER-TE) and 18 is intended to be used as the path steering mechanism for Traffic 19 Engineering with BIER. 21 BIER-TE introduces a new semantic for bit positions (BP) that 22 indicate adjacencies, as opposed to BIER in which BPs indicate Bit- 23 Forwarding Egress Routers (BFER). BIER-TE can leverage BIER 24 forwarding engines with little changes. Co-existence of BIER and 25 BIER-TE forwarding in the same domain is possible, for example by 26 using separate BIER sub-domains (SDs). Except for the optional 27 routed adjacencies, BIER-TE does not require a BIER routing underlay, 28 and can therefore operate without depending on an Interior Gateway 29 Routing protocol (IGP). 31 As it operates on the same per-packet stateless forwarding 32 principles, BIER-TE can also be a good fit to support multicast path 33 steering in Segment Routing (SR) networks. 35 Status of This Memo 37 This Internet-Draft is submitted in full conformance with the 38 provisions of BCP 78 and BCP 79. 40 Internet-Drafts are working documents of the Internet Engineering 41 Task Force (IETF). Note that other groups may also distribute 42 working documents as Internet-Drafts. The list of current Internet- 43 Drafts is at https://datatracker.ietf.org/drafts/current/. 45 Internet-Drafts are draft documents valid for a maximum of six months 46 and may be updated, replaced, or obsoleted by other documents at any 47 time. It is inappropriate to use Internet-Drafts as reference 48 material or to cite them other than as "work in progress." 49 This Internet-Draft will expire on January 10, 2022. 51 Copyright Notice 53 Copyright (c) 2021 IETF Trust and the persons identified as the 54 document authors. All rights reserved. 56 This document is subject to BCP 78 and the IETF Trust's Legal 57 Provisions Relating to IETF Documents 58 (https://trustee.ietf.org/license-info) in effect on the date of 59 publication of this document. Please review these documents 60 carefully, as they describe your rights and restrictions with respect 61 to this document. Code Components extracted from this document must 62 include Simplified BSD License text as described in Section 4.e of 63 the Trust Legal Provisions and are provided without warranty as 64 described in the Simplified BSD License. 66 Table of Contents 68 1. Overview . . . . . . . . . . . . . . . . . . . . . . . . . . 3 69 1.1. Requirements Language . . . . . . . . . . . . . . . . . . 5 70 2. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 5 71 2.1. Basic Examples . . . . . . . . . . . . . . . . . . . . . 5 72 2.2. BIER-TE Topology and adjacencies . . . . . . . . . . . . 8 73 2.3. Relationship to BIER . . . . . . . . . . . . . . . . . . 9 74 2.4. Accelerated/Hardware forwarding comparison . . . . . . . 11 75 3. Components . . . . . . . . . . . . . . . . . . . . . . . . . 11 76 3.1. The Multicast Flow Overlay . . . . . . . . . . . . . . . 12 77 3.2. The BIER-TE Control Plane . . . . . . . . . . . . . . . . 12 78 3.2.1. The BIER-TE Controller . . . . . . . . . . . . . . . 13 79 3.2.1.1. BIER-TE Topology discovery and creation . . . . . 14 80 3.2.1.2. Engineered Trees via BitStrings . . . . . . . . . 14 81 3.2.1.3. Changes in the network topology . . . . . . . . . 15 82 3.2.1.4. Link/Node Failures and Recovery . . . . . . . . . 15 83 3.3. The BIER-TE Forwarding Plane . . . . . . . . . . . . . . 15 84 3.4. The Routing Underlay . . . . . . . . . . . . . . . . . . 16 85 3.5. Traffic Engineering Considerations . . . . . . . . . . . 16 86 4. BIER-TE Forwarding . . . . . . . . . . . . . . . . . . . . . 17 87 4.1. The Bit Index Forwarding Table (BIFT) . . . . . . . . . . 17 88 4.2. Adjacency Types . . . . . . . . . . . . . . . . . . . . . 18 89 4.2.1. Forward Connected . . . . . . . . . . . . . . . . . . 18 90 4.2.2. Forward_routed . . . . . . . . . . . . . . . . . . . 19 91 4.2.3. ECMP . . . . . . . . . . . . . . . . . . . . . . . . 19 92 4.2.4. Local Decap(sulation) . . . . . . . . . . . . . . . . 19 93 4.3. Encapsulation / Co-existence with BIER . . . . . . . . . 20 94 4.4. BIER-TE Forwarding Pseudocode . . . . . . . . . . . . . . 21 95 4.5. Basic BIER-TE Forwarding Example . . . . . . . . . . . . 24 96 4.6. BFR Requirements for BIER-TE forwarding . . . . . . . . . 26 98 5. BIER-TE Controller Operational Considerations . . . . . . . . 26 99 5.1. Bit position Assignments . . . . . . . . . . . . . . . . 26 100 5.1.1. P2P Links . . . . . . . . . . . . . . . . . . . . . . 27 101 5.1.2. BFER . . . . . . . . . . . . . . . . . . . . . . . . 27 102 5.1.3. Leaf BFERs . . . . . . . . . . . . . . . . . . . . . 27 103 5.1.4. LANs . . . . . . . . . . . . . . . . . . . . . . . . 28 104 5.1.5. Hub and Spoke . . . . . . . . . . . . . . . . . . . . 28 105 5.1.6. Rings . . . . . . . . . . . . . . . . . . . . . . . . 29 106 5.1.7. Equal Cost MultiPath (ECMP) . . . . . . . . . . . . . 30 107 5.1.8. Forward_routed adjacencies . . . . . . . . . . . . . 32 108 5.1.8.1. Reducing bit positions . . . . . . . . . . . . . 32 109 5.1.8.2. Supporting nodes without BIER-TE . . . . . . . . 33 110 5.1.9. Reuse of bit positions (without DNC) . . . . . . . . 33 111 5.1.10. Summary of BP optimizations . . . . . . . . . . . . . 35 112 5.2. Avoiding duplicates and loops . . . . . . . . . . . . . . 36 113 5.2.1. Loops . . . . . . . . . . . . . . . . . . . . . . . . 36 114 5.2.2. Duplicates . . . . . . . . . . . . . . . . . . . . . 36 115 5.3. Managing SI, sub-domains and BFR-ids . . . . . . . . . . 37 116 5.3.1. Why SI and sub-domains . . . . . . . . . . . . . . . 37 117 5.3.2. Assigning bits for the BIER-TE topology . . . . . . . 38 118 5.3.3. Assigning BFR-id with BIER-TE . . . . . . . . . . . . 39 119 5.3.4. Mapping from BFR to BitStrings with BIER-TE . . . . . 40 120 5.3.5. Assigning BFR-ids for BIER-TE . . . . . . . . . . . . 41 121 5.3.6. Example bit allocations . . . . . . . . . . . . . . . 41 122 5.3.6.1. With BIER . . . . . . . . . . . . . . . . . . . . 41 123 5.3.6.2. With BIER-TE . . . . . . . . . . . . . . . . . . 42 124 5.3.7. Summary . . . . . . . . . . . . . . . . . . . . . . . 43 125 6. BIER-TE and Segment Routing . . . . . . . . . . . . . . . . . 44 126 7. Security Considerations . . . . . . . . . . . . . . . . . . . 45 127 8. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 46 128 9. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . 46 129 10. Change log [RFC Editor: Please remove] . . . . . . . . . . . 46 130 11. References . . . . . . . . . . . . . . . . . . . . . . . . . 56 131 11.1. Normative References . . . . . . . . . . . . . . . . . . 56 132 11.2. Informative References . . . . . . . . . . . . . . . . . 56 133 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 58 135 1. Overview 137 BIER-TE is based on architecture, terminology and packet formats with 138 BIER as described in [RFC8279] and [RFC8296]. This document 139 describes BIER-TE in the expectation that the reader is familiar with 140 these two documents. 142 BIER-TE introduces a new semantic for bit positions (BP) that 143 indicate adjacencies, as opposed to BIER in which BPs indicate Bit- 144 Forwarding Egress Routers (BFER). With BIER-TE, the BIFT of each BFR 145 is only populated with BP that are adjacent to the BFR in the BIER-TE 146 Topology. Other BPs are empty in the BIFT. The BFR replicate and 147 forwards BIER packets to adjacent BPs that are set in the packet. 148 BPs are normally also cleared upon forwarding to avoid duplicates and 149 loops. This is detailed further below. 151 BIER-TE can leverage BIER forwarding engines with little or no 152 changes. It can also co-exist with BIER forwarding in the same 153 domain, for example by using separate BIER sub-domains. Except for 154 the optional routed adjacencies, BIER-TE does not require a BIER 155 routing underlay, and can therefore operate without depending on an 156 Interior Gateway Routing protocol (IGP). 158 As it operates on the same per-packet stateless forwarding 159 principles, BIER-TE can also be a good fit to support multicast path 160 steering in Segment Routing (SR) networks. 162 This document is structured as follows: 164 o Section 2 introduces BIER-TE with two reference forwarding 165 examples, followed by an introduction of the new concepts of the 166 BIER-TE (overlay) topology and finally a summary of the 167 relationship between BIER and BIER-TE and a discussion of 168 accelerated hardware forwarding. 170 o Section 3 describes the components of the BIER-TE architecture, 171 Flow overlay, BIER-TE layer with the BIER-TE control plane 172 (including the BIER-TE controller) and BIER-TE forwarding plane, 173 and the routing underlay. 175 o Section 4 specifies the behavior of the BIER-TE forwarding plane 176 with the different type of adjacencies and possible variations of 177 BIER-TE forwarding pseudocode, and finally the mandatory and 178 optional requirements. 180 o Section 5 describes operational considerations for the BIER-TE 181 controller, foremost how the BIER-TE controller can optimize the 182 use of BP by using specific type of BIER-TE adjacencies for 183 different type of topological situations, but also how to assign 184 bits to avoid loops and duplicates (which in BIER-TE does not come 185 for free), and finally how SI, sub-domains and BFR-ids can be 186 managed by a BIER-TE controller, examples and summary. 188 o Section 6 concludes the technology specific sections of document 189 by further relating BIER-TE to Segment Routing (SR). 191 Note that related work, [I-D.ietf-roll-ccast] uses Bloom filters 192 [Bloom70] to represent leaves or edges of the intended delivery tree. 193 Bloom filters in general can support larger trees/topologies with 194 fewer addressing bits than explicit BitStrings, but they introduce 195 the heuristic risk of false positives and cannot clear bits in the 196 BitString during forwarding to avoid loops. For these reasons, BIER- 197 TE uses explicit BitStrings like BIER. The explicit BitStrings of 198 BIER-TE can also be seen as a special type of Bloom filter, and this 199 is how related work [ICC] describes it. 201 1.1. Requirements Language 203 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 204 "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and 205 "OPTIONAL" in this document are to be interpreted as described in BCP 206 14 [RFC2119], [RFC8174] when, and only when, they appear in all 207 capitals, as shown here. 209 2. Introduction 211 2.1. Basic Examples 213 BIER-TE forwarding is best introduced with simple examples. 215 BIER-TE Topology: 217 Diagram: 219 p5 p6 220 --- BFR3 --- 221 p3/ p13 \p7 p15 222 BFR1 ---- BFR2 BFR5 ----- BFR6 223 p1 p2 p4\ p14 /p10 p11 p12 224 --- BFR4 --- 225 p8 p9 227 (simplified) BIER-TE Bit Index Forwarding Tables (BIFT): 229 BFR1: p1 -> local_decap 230 p2 -> forward_connected() to BFR2 232 BFR2: p1 -> forward_connected() to BFR1 233 p5 -> forward_connected() to BFR3 234 p8 -> forward_connected() to BFR4 236 BFR3: p3 -> forward_connected() to BFR2 237 p7 -> forward_connected() to BFR5 238 p13 -> local_decap 240 BFR4: p4 -> forward_connected() to BFR2 241 p10 -> forward_connected() to BFR5 242 p14 -> local_decap 244 BFR5: p6 -> forward_connected() to BFR3 245 p9 -> forward_connected() to BFR4 246 p12 -> forward_connected() to BFR6 248 BFR6: p11 -> forward_connected() to BFR5 249 p15 -> local_decap 251 Figure 1: BIER-TE basic example 253 Consider the simple network in the above BIER-TE overview example 254 picture with 6 BFRs. p1...p14 are the bit positions (BP) used. All 255 BFRs can act as an ingress BFR (BFIR), BFR1, BFR3, BFR4 and BFR6 can 256 also be egress BFRs (BFERs). Forward_connected() is the name for 257 adjacencies that are representing subnet adjacencies of the network. 258 Local_decap() is the name of the adjacency to decapsulate BIER-TE 259 packets and pass their payload to higher layer processing. 261 Assume a packet from BFR1 should be sent via BFR4 to BFR6. This 262 requires a BitString (p2,p8,p10,p12,p15). When this packet is 263 examined by BIER-TE on BFR1, the only bit position from the BitString 264 that is also set in the BIFT is p2. This will cause BFR1 to send the 265 only copy of the packet to BFR2. Similarly, BFR2 will forward to 266 BFR4 because of p8, BFR4 to BFR5 because of p10 and BFR5 to BFR6 267 because of p12. p15 finally makes BFR6 receive and decapsulate the 268 packet. 270 To send in addition to BFR6 via BFR4 also a copy to BFR3, the 271 BitString needs to be (p2,p5,p8,p10,p12,p13). When this packet is 272 examined by BFR2, p5 causes one copy to be sent to BFR3 and p8 one 273 copy to BFR4. When BFR3 receives the packet, p13 will cause it to 274 receive and decapsulate the packet. 276 If instead the BitString was (p2,p6,p8,p10,p12,p13,p15), the packet 277 would be copied by BFR5 towards BFR3 because of p6 instead of being 278 copied by BFR2 to BFR3 because of p5 in the prior case. This is 279 showing the ability of the shown BIER-TE Topology to make the traffic 280 pass across any possible path and be replicated where desired. 282 BIER-TE has various options to minimize BP assignments, many of which 283 are based on assumptions about the required multicast traffic paths 284 and bandwidth consumption in the network. 286 The following picture shows a modified example, in which Rtr2 and 287 Rtr5 are assumed not to support BIER-TE, so traffic has to be unicast 288 encapsulated across them. To emphasize non-L2, but routed/tunneled 289 forwarding of BIER-TE packets, these adjacencies are called 290 "forward_routed". Otherwise there is no difference in their 291 processing over the aforementioned "forward_connected" adjacencies. 293 In addition, bits are saved in the following example by assuming that 294 BFR1 only needs to be BFIR but not BFER or transit BFR. 296 BIER-TE Topology: 298 Diagram: 300 p1 p3 p7 301 ....> BFR3 <.... p5 302 ........ ........> 303 BFR1 (Rtr2) (Rtr5) BFR6 304 ........ ........> 305 ....> BFR4 <.... p6 306 p2 p4 p8 308 (simplified) BIER-TE Bit Index Forwarding Tables (BIFT): 310 BFR1: p1 -> forward_routed() to BFR3 311 p2 -> forward_routed() to BFR4 313 BFR3: p3 -> local_decap 314 p5 -> forward_routed() to BFR6 316 BFR4: p4 -> local_decap 317 p6 -> forward_routed() to BFR6 319 BFR6: p5 -> local_decap 320 p6 -> local_decap 321 p7 -> forward_routed() to BFR3 322 p8 -> forward_routed() to BFR4 324 Figure 2: BIER-TE basic overlay example 326 To send a BIER-TE packet from BFR1 via BFR3 to BFR6, the BitString is 327 (p1,p5). From BFR1 via BFR4 to BFR6 it is (p2,p6). A packet from 328 BFR1 to BFR3,BFR4 and from BFR3 to BFR6 uses (p1,p2,p3,p4,p5). A 329 packet from BFR1 to BFR3,BFR4 and from BFR4 to BFR uses 330 (p1,p2,p3,p4,p6). A packet from BFR1 to BFR4, and from BFR4 to BFR6 331 and from BFR6 to BFR3 uses (p2,p3,p4,p6,p7). A packet from BFR1 to 332 BFR3, and from BFR3 to BFR6 and from BFR6 to BFR4 uses 333 (p1,p3,p4,p5,p8). 335 2.2. BIER-TE Topology and adjacencies 337 The key new component in BIER-TE compared to BIER is the BIER-TE 338 topology as introduced through the two examples in Section 2.1. It 339 is used to control where replication can or should happen and how to 340 minimize the required number of BP for adjacencies. 342 The BIER-TE Topology consists of the BIFTs of all the BFR and can 343 also be expressed as a directed graph where the edges are the 344 adjacencies between the BFR labelled with the BP used for the 345 adjacency. Adjacencies are naturally unidirectional. BP can be 346 reused across multiple adjacencies as long as this does not lead to 347 undesired duplicates or loops as explained further down in the text. 349 If the BIER-TE topology represents (a subset of) the underlying 350 (layer 2) topology of the network as shown in the first example, this 351 may be called a "native" BIER-TE topology. A topology consisting 352 only of "forward_routed" adjacencies as shown in the second example 353 may be called an "overlay" BIER-TE topology. A BIER-TE topology will 354 both "forward_connected" and "forward_routed" adjacencies may be 355 called a "hybrid" BIER-TE topology. 357 2.3. Relationship to BIER 359 BIER-TE is designed so that is forwarding plane is a simple extension 360 to the BIER forwarding plane, hence allowing for it to be added to 361 BIER deployments where it can be beneficial. 363 BIER-TE is also intended as an option to expand the BIER architecture 364 into deployments where BIER may not be the best fit, such as 365 statically provisioned networks with needs for path steering but 366 without desire for distributed routing protocols. 368 1. BIER-TE inherits the following aspects from BIER unchanged: 370 1. The fundamental purpose of per-packet signaled packet 371 replication and delivery via a BitString. 373 2. The overall architecture consisting of three layers, flow 374 overlay, BIER(-TE) layer and routing underlay. 376 3. The supportable encapsulations, [RFC8296] or other (future) 377 encapsulations. 379 4. The semantic of all [RFC8296] header elements used by the 380 BIER-TE forwarding plane other than the semantic of the BP in 381 the BitString. 383 5. The BIER forwarding plane, with the exception of how bits 384 have to be cleared during replication. 386 2. BIER-TE has the following key changes with respect to BIER: 388 1. In BIER, bits in the BitString of a BIER packet header 389 indicate a BFER and bits in the BIFT indicate the BIER 390 control plane calculated next-hop toward that BFER. In BIER- 391 TE, bits in the BitString of a BIER packet header indicate an 392 adjacency in the BIER-TE topology, and only the BFRs that are 393 upstream of this adjacency have this bit populated with the 394 adjacency in their BIFT. 396 2. In BIER, the implied reference option for the core part of 397 the BIER layer control plane is the BIER extension to 398 distributed routing protocol, such as standardized in ISIS/ 399 OSPF extensions for BIER, [RFC8401] and [RFC8444]. The 400 reference option for the core part of the BIER-TE control 401 plane is the BIER-TE controller. Nevertheless, both BIER and 402 BIER-TE BIFT forwarding plane state could equally be 403 populated by any mechanism. 405 3. Assuming the reference options for the control plane, BIER-TE 406 replaces in-network autonomous path calculation by explicit 407 paths calculated by the BIER-TE controller. 409 3. The following element/functions described in the BIER 410 architecture are not required by the BIER-TE architecture: 412 1. BIRTs on BFR for BIER-TE are not required when using a BIER- 413 TE controller because the controller can directly populate 414 the BIFTs. In BIER, BIRTs are populated by the distributed 415 routing protocol support for BIER, allowing BFR to populate 416 their BIFTs locally from their BIRTs. Other BIER-TE control 417 plane or management plane options may introduce requirements 418 for BIRTs for BIER-TE BFR. 420 2. The BIER-TE layer forwarding plane does not require BFR to 421 have a unique BP and therefore also no unique BFR-id. See 422 for example See Section 5.1.3. 424 3. Identification of BFR by the BIER-TE control plane is outside 425 the scope of this specification. Whereas the BIER control 426 plane uses BFR-ids in its BFR to BFR signaling, a BIER-TE 427 controller may choose any form of identification deemed 428 appropriate. 430 4. BIER-TE forwarding does not use the BFR-id field of the BIER 431 packet header. 433 4. Co-existence of BIER and BIER-TE in the same network requires the 434 following: 436 1. The BIER/BIER-TE packet header needs to allow addressing both 437 BIER and BIER-TE BIFT. Depending on the encapsulation 438 option, the same SD may or may not be reusable across BIER 439 and BIER-TE. See Section 4.3. In either case, a packet is 440 always only forwarded end-to-end via BIER or via BIER-TE 441 (ships in the nights forwarding). 443 2. BIER-TE deployments will have to assign BFR-ids to BFR and 444 insert them into the BFR-id field of BIER packet headers as 445 BIER does, whenever the deployment uses (unchanged) 446 components developed for BIER that use BFR-id, such as 447 multicast flow overlays or BIER layer control plane elements. 448 See also Section 5.3.3. 450 2.4. Accelerated/Hardware forwarding comparison 452 Forwarding of BIER-TE is designed to easily build/program common 453 forwarding hardware with BIER. The pseudocode in Section 4.4 shows 454 how existing BIER/BIFT forwarding can be modified to support the 455 REQUIRED BIER-TE forwarding functionality, by using BIER BIFT's 456 "Forwarding Bit Mask" (F-BM): Only the clearing of bits to avoid 457 duplicate packets to a BFR neighbor is skipped in BIER-TE forwarding 458 because it is not necessary and could not be done when using BIER 459 F-BM. 461 Whether to use BIER or BIER-TE forwarding is simply a choice of the 462 mode of the BIFT indicated by the packet (BIER or BIER-TE BIFT). 463 This is determined by the BFR configuration for the encapsulation, 464 see Section 4.3. 466 3. Components 468 BIER-TE can be thought of being constituted from the same three 469 layers as BIER: The "multicast flow overlay", the "BIER layer" and 470 the "routing underlay". The following picture also shows how the 471 "BIER layer" is constituted from the "BIER-TE forwarding plane" and 472 the "BIER-TE control plane" represent by the "BIER-TE Controller". 474 <------BGP/PIM-----> 475 |<-IGMP/PIM-> multicast flow <-PIM/IGMP->| 476 overlay 478 BIER-TE [BIER-TE Controller] <=> [BIER-TE Topology] 479 control ^ ^ ^ 480 plane / | \ BIER-TE control protocol 481 | | | e.g. YANG/Netconf/RestConf 482 | | | PCEP/... 483 v v v 484 Src -> Rtr1 -> BFIR-----BFR-----BFER -> Rtr2 -> Rcvr 486 |<----------------->| 487 BIER-TE forwarding plane 489 |<- BIER-TE domain->| 491 |<--------------------->| 492 Routing underlay 494 Figure 3: BIER-TE architecture 496 3.1. The Multicast Flow Overlay 498 The Multicast Flow Overlay has the same role as described for BIER in 499 [RFC8279], Section 4.3. See also Section 3.2.1.2. 501 3.2. The BIER-TE Control Plane 503 In the BIER architecture [RFC8279], the BIER control plane is not 504 explicitly separated from the BIER forwarding plane, but instead 505 their functions are summarized together in Section 4.2. Example 506 standardized options for the BIER control plane include ISIS/OSPF 507 extensions for BIER, [RFC8401] and [RFC8444]. 509 For BIER-TE, the control plane includes at minimum the following 510 functionality. 512 1. During initial provisioning of the network and/or during 513 modifications of its topology and/or services: protocols and/or 514 procedures to establish BIER-TE BIFTs: 516 1. Determine the desired BIER-TE topology for a BIER-TE sub- 517 domains: the native and/or overlay adjacencies that are 518 assigned to BPs. 520 2. Determine the per-BFR BIFT from the BIER-TE topology. 522 3. Optionally assign BFR-id to BFIR for later insertion into 523 BIER-TE headers on BFIR. Alternatively, bfir-id in BIER 524 packet headers may be managed solely by the flow overlay 525 layer and/or be unused. 527 4. Install/update the BIFTs into the BFRs and optionally BFR-id 528 into BFIR. 530 2. During operations of the network: Protocols and/or procedures to 531 support creation/change/removal of overlay flows on BFIR: 533 1. Process the BIER-TE requirements for the multicast overlay 534 flow: BFIR and BFERs of the flow as well as policies for the 535 path selection of the flow. 537 2. Determine the BitStrings and optionally Entropy. 539 3. Install state on the BFIR to imposition the desired BIER 540 packet header(s) for packets of the overlay flow. 542 4. Install the necessary state on the BFERs to decapsulate the 543 BIER packet header and properly dispatch its payload. 545 3.2.1. The BIER-TE Controller 547 Notwithstanding other options, this architecture describes the BIER 548 control plane as shown in Figure 3 to consists of: 550 o A single centralized BIER-TE controller. 552 o Data-models and protocols to communicate between controller and 553 BFR in step 1, such YANG/Netconf/RestConf. 555 o Protocols to communicate between controller and BFIR in step 2, 556 such as BIER-TE extensions for [RFC5440]. 558 The BIER control plane could equally be implemented without any 559 active dynamic components by an operator via CLI on the BFRs. In 560 that case, operator configured local policy on the BFIR would have to 561 determine how to set the appropriate BIER header fields. The BIER-TE 562 control plane could also be decentralized and/or distributed, but 563 this document does not consider any additional protocols and/or 564 procedures which would then be necessary to coordinate its entities 565 to achieve the above described functionality. 567 3.2.1.1. BIER-TE Topology discovery and creation 569 Step 1.1 includes network topology discovery and BIER-TE topology 570 creation. The latter describes the process by which a Controller 571 determines which routers are to be configured as BFR and the 572 adjacencies between them. 574 In statically managed networks, such as in industrial environments, 575 both discovery and creation can be a manual/offline process. 577 In other networks, topology discovery may rely on protocols including 578 extending a Link-State-Protocol (LSP) based IGP into the BIER-TE 579 controller itself, [RFC7752] (BGP-LS) or [RFC8345] (Yang topology) as 580 well as BIER-TE specific methods, for example via 581 [I-D.ietf-bier-te-yang]. These options are non-exhaustive. 583 Dynamic creation of the BIER-TE topology can be as easy as mapping 584 the network topology 1:1 to the BIER-TE topology by assigning a BP 585 for every network subnet adjacency. In larger networks, it likely 586 involves more complex policy and optimization decisions including how 587 to minimize the number of BP required and how to assign BP across 588 different BitStrings to minimize the number of duplicate packets 589 across links when delivering an overlay flow to BFER using different 590 SIs/BitStrings. These topics are discussed in Section 5. 592 When the topology is determined, the BIER-TE Controller then pushes 593 the BitPositions/adjacencies to the BIFT of the BFRs, populating only 594 those SI:BitPositions to the BIFT of each BFR to which that BFR 595 should be able to send packets to - adjacencies connecting to this 596 BFR. 598 Communications between the BIER-TE Controller and BFRs (beside 599 topology discovery) is ideally via standardized protocols and data- 600 models such as Netconf/RestConf/Yang/PCEP. Vendor-specific CLI on 601 the BFRs is also an option (as in many other SDN solutions lacking 602 definition of standardized data model). 604 3.2.1.2. Engineered Trees via BitStrings 606 In BIER, the same set of BFER in a single sub-domain is always 607 encoded as the same BitString. In BIER-TE, the BitString used to 608 reach the same set of BFER in the same sub-domain can be different 609 for different overlay flows because the BitString encodes the paths 610 towards the BFER, so the BitStrings from different BFIR to the same 611 set of BFER will often be different, and the BitString from the same 612 BFIR to the same set of BFER can different for different overlay 613 flows for policy reasons such as shortest path trees, Steiner trees 614 (minimum cost trees), diverse path trees for redundancy and so on. 616 See also [I-D.ietf-bier-multicast-http-response] for a solution 617 describing this interaction. 619 3.2.1.3. Changes in the network topology 621 If the network topology changes (not failure based) so that 622 adjacencies that are assigned to bit positions are no longer needed, 623 the BIER-TE Controller can re-use those bit positions for new 624 adjacencies. First, these bit positions need to be removed from any 625 BFIR flow state and BFR BIFT state, then they can be repopulated, 626 first into BIFT and then into the BFIR. 628 3.2.1.4. Link/Node Failures and Recovery 630 When link or nodes fail or recover in the topology, BIER-TE could 631 quickly respond with out-of-scope FRR procedures such as 632 [I-D.eckert-bier-te-frr]. It can also more slowly react by 633 recalculating the BitStrings of affected multicast flows. This 634 reaction is slower than the FRR procedure because the BIER-TE 635 Controller needs to receive link/node up/down indications, 636 recalculate the desired BitStrings and push them down into the BFIRs. 637 With FRR, this is all performed locally on a BFR receiving the 638 adjacency up/down notification. 640 3.3. The BIER-TE Forwarding Plane 642 The BIER-TE Forwarding Plane constitutes of the following components: 644 1. On BFIR imposition of BIER header for packets from overlay flows. 645 This is driven by a combination of state established by the BIER- 646 TE control plane and/or the multicast flow overlay as explained 647 in Section 3.1. 649 2. On BFR (including BFIR and BFER), forwarding/replication of BIER 650 packets according to their BitString as explained below and 651 optionally Entropy. Processing of other BIER header fields such 652 as DSCP is outside the scope of this document. 654 3. On BFER removal of BIER header and dispatching of the payload 655 according to state created by the BIER-TE control plane and/or 656 overlay layer. 658 When the BIER-TE Forwarding Plane receives a packet, it simply looks 659 up the bit positions that are set in the BitString of the packet in 660 the Bit Index Forwarding Table (BIFT) that was populated by the BIER- 661 TE Controller. For every BP that is set in the BitString, and that 662 has one or more adjacencies in the BIFT, a copy is made according to 663 the type of adjacencies for that BP in the BIFT. Before sending any 664 copy, the BFR clears all BPs in the BitString of the packet for which 665 the BFR has one or more adjacencies in the BIFT, except when the 666 adjacency indicates "DoNotClear" (DNC, see Section 4.2.1). This is 667 done to inhibit that packets can loop. 669 3.4. The Routing Underlay 671 For forward_connected() adjacencies, BIER-TE is sending BIER packets 672 to directly connected BIER-TE neighbors as L2 (unicasted) BIER 673 packets without requiring a routing underlay. For forward_routed() 674 adjacencies, BIER-TE forwarding encapsulates a copy of the BIER 675 packet so that it can be delivered by the forwarding plane of the 676 routing underlay to the routable destination address indicated in the 677 adjacency. See Section 4.2.2 for the adjacency definition. 679 BIER relies on the routing underlay to calculate paths towards BFERs 680 and derive next-hop BFR adjacencies for those paths. This commonly 681 relies on BIER specific extensions to the routing protocols of the 682 routing underlay but may also be established by a controller. In 683 BIER-TE, the next-hops of a packet are determined by the BitString 684 through the BIER-TE Controller established adjacencies on the BFR for 685 the BPs of the BitString. There is thus no need for BFER specific 686 routing underlay extensions to forward BIER packets with BIER-TE 687 semantics. 689 Encapsulation parameters can be provisioned by the BIER-TE controller 690 into the forward_connected() or forward_routed() adjacencies directly 691 without relying on a routing underlay. 693 If the BFR intends to support FRR for BIER-TE, then the BIER-TE 694 forwarding plane needs to receive fast adjacency up/down 695 notifications: Link up/down or neighbor up/down, e.g. from BFD. 696 Providing these notifications is considered to be part of the routing 697 underlay in this document. 699 3.5. Traffic Engineering Considerations 701 Traffic Engineering ([I-D.ietf-teas-rfc3272bis]) provides performance 702 optimization of operational IP networks while utilizing network 703 resources economically and reliably. The key elements needed to 704 effect TE are policy, path steering and resource management. These 705 elements require support at the control/controller level and within 706 the forwarding plane. 708 Policy decisions are made within the BIER-TE control plane, i.e., 709 within BIER-TE Controllers. Controllers use policy when composing 710 BitStrings and BFR BIFT state. The mapping of user/IP traffic to 711 specific BitStrings/BIER-TE flows is made based on policy. The 712 specific details of BIER-TE policies and how a controller uses them 713 are out of scope of this document. 715 Path steering is supported via the definition of a BitString. 716 BitStrings used in BIER-TE are composed based on policy and resource 717 management considerations. For example, when composing BIER-TE 718 BitStrings, a Controller must take into account the resources 719 available at each BFR and for each BP when it is providing congestion 720 loss free services such as Rate Controlled Service Disciplines 721 [RCSD94]. Resource availability could be provided for example via 722 routing protocol information, but may also be obtained via a BIER-TE 723 control protocol such as Netconf or any other protocol commonly used 724 by a PCE to understand the resources of the network it operates on. 725 The resource usage of the BIER-TE traffic admitted by the BIER-TE 726 controller can be solely tracked on the BIER-TE Controller based on 727 local accounting as long as no forward_routed() adjacencies are used 728 (see Section 4.2.1 for the definition of forward_routed() 729 adjacencies). When forward_routed() adjacencies are used, the paths 730 selected by the underlying routing protocol need to be tracked as 731 well. 733 Resource management has implications on the forwarding plane beyond 734 the BIER-TE defined steering of packets. This includes allocation of 735 buffers to guarantee the worst case requirements of admitted RCSD 736 traffic and potential policing and/or rate-shaping mechanisms, 737 typically done via various forms of queuing. This level of resource 738 control, while optional, is important in networks that wish to 739 support congestion management policies to control or regulate the 740 offered traffic to deliver different levels of service and alleviate 741 congestion problems, or those networks that wish to control latencies 742 experienced by specific traffic flows. 744 4. BIER-TE Forwarding 746 4.1. The Bit Index Forwarding Table (BIFT) 748 The Bit Index Forwarding Table (BIFT) exists in every BFR. For every 749 sub-domain in use, it is a table indexed by SI:bit position and is 750 populated by the BIER-TE control plane. Each index can be empty or 751 contain a list of one or more adjacencies. 753 Like BIER, BIER-TE can support multiple sub-domains, each with a 754 separate BIFT. 756 In [RFC8279], Figure 2, indices into the BIFT are both SI:BitString 757 and BFR-id, where BitString is indicating a BP: BFR-id = SI * 2^BSL + 758 BP. As shown in Figure 4, in BIER-TE, only SI:BP are used as indices 759 into a BIFT because they identify adjacencies and not BFR. 761 ------------------------------------------------------------------ 762 | Index: | Adjacencies: | 763 | SI:bit position | or one or more per entry | 764 ================================================================== 765 | 0:1 | forward_connected(interface,neighbor{,DNC}) | 766 ------------------------------------------------------------------ 767 | 0:2 | forward_connected(interface,neighbor{,DNC}) | 768 | | forward_connected(interface,neighbor{,DNC}) | 769 ------------------------------------------------------------------ 770 | 0:3 | local_decap({VRF}) | 771 ------------------------------------------------------------------ 772 | 0:4 | forward_routed({VRF,}l3-neighbor) | 773 ------------------------------------------------------------------ 774 | 0:5 | | 775 ------------------------------------------------------------------ 776 | 0:6 | ECMP({adjacency1,...adjacencyN}, seed) | 777 ------------------------------------------------------------------ 778 ... 779 | BitStringLength | ... | 780 ------------------------------------------------------------------ 781 Bit Index Forwarding Table 783 Figure 4: BIFT adjacencies 785 The BIFT is programmed into the data plane of BFRs by the BIER-TE 786 Controller and used to forward packets, according to the rules 787 specified in the BIER-TE Forwarding Procedures. 789 Note that a BIFT index (SI:BP) may be populated in the BIFT of more 790 than one BFR. See Section 5.1.6 for an example of how a BIER-TE 791 controller could assign BPs to (logical) adjacencies shared across 792 multiple BFRs, Section 5.1.3 for an example of assigning the same BP 793 to different adjacencies, and Section 5.1.9 for guidelines regarding 794 re-use of BPs across different adjacencies. 796 {VRF} indicates the Virtual Routing and Forwarding context into which 797 the BIER payload is to be delivered. This is optional and depends on 798 the multicast flow overlay. 800 4.2. Adjacency Types 802 4.2.1. Forward Connected 804 A "forward_connected" adjacency is towards a directly connected BFR 805 neighbor using an interface address of that BFR on the connecting 806 interface. A forward_connected() adjacency does not route packets 807 but only L2 forwards them to the neighbor. 809 Packets sent to an adjacency with "DoNotClear" (DNC) set in the BIFT 810 MUST NOT have the bit position for that adjacency cleared when the 811 BFR creates a copy for it. The bit position will still be cleared 812 for copies of the packet made towards other adjacencies. This can be 813 used for example in ring topologies as explained below. 815 4.2.2. Forward_routed 817 A "forward_routed" adjacency is an adjacency towards a BFR that uses 818 a (tunneling) encapsulation which will cause the packet to be 819 forwarded by the routing underlay toward the adjacent BFR. This can 820 leverage any feasible encapsulation, such as MPLS or tunneling over 821 IP/IPv6, as long as the BIER-TE packet can be identified as a 822 payload. This identification can either rely on the BIER/BIER-TE co- 823 existence mechanisms described in Section 4.3, or by explicit support 824 for a BIER-TE payload type in the tunneling encapsulation. 826 "forward_routed" adjacencies are necessary to pass BIER-TE traffic 827 across non BIER-TE capable routers or to minimize the number of 828 required BP by tunneling over (BIER-TE capable) routers on which 829 neither replication nor path-steering is desired, or simply to 830 leverage path redundancy and FRR of the routing underlay towards the 831 next BFR. They may also be useful to a multi-subnet adjacent BFR to 832 leverage the routing underlay ECMP independent of BIER-TE ECMP 833 (Section 4.2.3). 835 4.2.3. ECMP 837 BIER ECMP is tied to the BIER BIFT processing semantic and are 838 therefore not directly usable with BIER-TE. 840 A BIER-TE "Equal Cost Multipath" (ECMP) adjacency has a list of two 841 or more non-ECMP adjacencies and a seed parameter. When a BIER-TE 842 packet is copied onto such an ECMP adjacency, an implementation 843 specific so-called hash function will select one out of the lists 844 adjacencies to which the packet is forwarded. This ECMP hash 845 function MUST select the same adjacency from that list for all 846 packets with the same entropy parameter. The seed parameter allows 847 to design hash functions that are easy to implement at high speed 848 without running into polarization issues across multiple consecutive 849 ECMP hops. See Section 5.1.7 for more explanations. 851 4.2.4. Local Decap(sulation) 853 A "local_decap" adjacency passes a copy of the payload of the BIER-TE 854 packet to the protocol within the BFR (IPv4/IPv6, Ethernet,...) 855 responsible for that payload according to the packet header fields. 856 A local_decap() adjacency turns the BFR into a BFER for matching 857 packets. Local_decap() adjacencies require the BFER to support 858 routing or switching for NextProto to determine how to further 859 process the packet. 861 4.3. Encapsulation / Co-existence with BIER 863 Specifications for BIER-TE encapsulation are outside the scope of 864 this document. This section gives explanations and guidelines. 866 Because a BFR needs to interpret the BitString of a BIER-TE packet 867 differently from a BIER packet, it is necessary to distinguish BIER 868 from BIER-TE packets. In the BIER encapsulation [RFC8296], the BIFT- 869 id field of the packet indicates the BIFT of the packet. BIER and 870 BIER-TE can therefore be run simultaneously, when the BIFT-id address 871 space is shared across BIER BIFT and BIER-TE BIFT. Partitioning the 872 BIFT-id address space is subject to BIER-TE/BIER control plane 873 procedures. 875 When [RFC8296] is used for BIER with MPLS, BIFT-id address ranges can 876 be dynamically allocated from MPLS label space only for the set of 877 actually used SD:BSL BIFT. This allows to also allocate non- 878 overlapping label ranges for BIFT-ids that are to be used with BIER- 879 TE BIFTs. 881 With MPLS, it is also possible to reuse the same SD space for both 882 BIER-TE and BIER, so that the same SD has both a BIER BIFT and 883 according range of BIFT-ids and a disjoint BIER-TE BIFT and non- 884 overlapping range of BIFT-ids. 886 When a fixed mapping from BSL, SD, SI is used without specifically 887 distinguishing BIER and BIER-TE, such as proposed for non-MPLS 888 forwarding with [RFC8296] in [I-D.ietf-bier-non-mpls-bift-encoding] 889 revision 04, section 5., then it is necessary to allocate disjoint 890 SDs to BIER and BIER-TE BIFT so that both can be addressed by the 891 BIFT-ids. The encoding proposed in section 6. of the same document 892 does not statically encode BSL or SD into the BIFT-id, but allows for 893 a mapping, and hence could provide for the same freedom as when MPLS 894 is being used (same or different SD for BIER/BIER-TE). 896 "forward_routed" requires an encapsulation permitting to unicast 897 BIER-TE packets to a specific interface address on a target BFR. 898 With MPLS encapsulation, this can simply be done via a label stack 899 with that addresses label as the top label - followed by the label 900 assigned to the (BSL,SD,SI) BitString. With non-MPLS encapsulation, 901 some form of IP encapsulation would be required (for example IP/GRE). 903 The encapsulation used for "forward_routed" adjacencies can equally 904 support existing advanced adjacency information such as "loose source 905 routes" via e.g. MPLS label stacks or appropriate header extensions 906 (e.g. for IPv6). 908 4.4. BIER-TE Forwarding Pseudocode 910 The following pseudocode, Figure 5, for BIER-TE forwarding is based 911 on the BIER forwarding pseudocode of [RFC8279], section 6.5 with one 912 modification. 914 void ForwardBitMaskPacket_withTE (Packet) 915 { 916 SI=GetPacketSI(Packet); 917 Offset=SI*BitStringLength; 918 for (Index = GetFirstbit position(Packet->BitString); Index ; 919 Index = GetNextbit position(Packet->BitString, Index)) { 920 F-BM = BIFT[Index+Offset]->F-BM; 921 if (!F-BM) continue; [3] 922 BFR-NBR = BIFT[Index+Offset]->BFR-NBR; 923 PacketCopy = Copy(Packet); 924 PacketCopy->BitString &= F-BM; [2] 925 PacketSend(PacketCopy, BFR-NBR); 926 // The following must not be done for BIER-TE: 927 // Packet->BitString &= ~F-BM; [1] 928 } 929 } 931 Figure 5: BIER-TE Forwarding Pseudocode for required functions, based 932 on BIER Pseudocode 934 In step [2], the F-BM is used to clear bit(s) in PacketCopy. This 935 step exists in both BIER and BIER-TE, but the F-BMs need to be 936 populated differently for BIER-TE than for BIER for the desired 937 clearing. 939 In BIER, multiple bits of a BitString can have the same BFR-NBR. 940 When a received packets BitString has more than one of those bits 941 set, the BIER replication logic has to avoid that more than one 942 PacketCopy is sent to that BFR-NBR ([1]). Likewise, the PacketCopy 943 sent to a BFR-NBR must clear all bits in its BitString that are not 944 routed across BFR-NBR. This protects against BIER replication on any 945 possible further BFR to create duplicates ([2]). 947 To solve both [1] and [2] for BIER, the F-BM of each bit needs to 948 have all bits set that this BFR wants to route across BFR-NBR. [2] 949 clears all other bits in PacketCopy->BitString, and [1] clears those 950 bits from Packet->BitString after the first PacketCopy. 952 In BIER-TE, a BFR-NBR is an adjacency, forward_connected, 953 forward_routed or local_decap. There is no need for [2] to suppress 954 duplicates in the way BIER does because in general, different BP 955 would never have the same adjacency. If a BIER-TE controller 956 actually finds some optimization in which this would be desirable, 957 then the controller is also responsible to ensure that only one of 958 those bits is set in any Packet->BitString, unless the controller 959 explicitly wants for duplicates to be created. 961 For BIER-TE, F-BM is handled as follows: 963 1. The F-BM of all bits without an adjacency has all bits clear. 964 This will cause [3] to skip further processing of such a bit. 966 2. All bits with an adjacency (with DNC flag clear) have an F-BM 967 that has only those bits set for which this BFR does not have an 968 adjacency. This causes [2] to clear all bits from 969 PacketCopy->BitString for which this BFR does have an adjacency. 971 3. [1] is not performed for BIER-TE. All bit clearing required by 972 BIER-TE is performed by [2]. 974 This Forwarding Pseudocode can support the REQUIRED BIER-TE 975 forwarding functions (see Section 4.6), forward_connected, 976 forward_routed() and local decap, but not the RECOMMENDED functions 977 DNC flag and multiple adjacencies per bit nor the OPTIONAL function, 978 ECMP adjacencies. The DNC flag cannot be supported when using only 979 [1] to mask bits. 981 The modified and expanded Forwarding Pseudocode in Figure 6 specifies 982 how to support all BIER-TE forwarding functions (required, 983 recommended and optional): 985 o This pseudocode eliminates per-bit F-BM, therefore reducing the 986 size of BIFT state by BitStringLength^2*SI and eliminating the 987 need for per-packet-copy masking operation except for adjacencies 988 with the DNC flag set: 990 * AdjacentBits[SI] are bits with a non-empty list of adjacencies. 991 This can be computed whenever the BIER-TE Controller updates 992 the adjacencies. 994 * Only the AdjacentBits need to be examined in the loop for 995 packet copies. 997 * The packets BitString is masked with those AdjacentBits before 998 the loop to avoid doing this repeatedly for every PacketCopy. 1000 o The code loops over the adjacencies because there may be more than 1001 one adjacency for a bit. 1003 o When an adjacency has the DNC bit, the bit is set in the packet 1004 copy (to save bits in rings for example). 1006 o The ECMP adjacency is shown. Its parameters are a 1007 ListOfAdjacencies from which one is picked. 1009 o The forward_local, forward_routed, local_decap() adjacencies are 1010 shown with their parameters. 1012 void ForwardBitMaskPacket_withTE (Packet) 1013 { 1014 SI=GetPacketSI(Packet); 1015 Offset=SI*BitStringLength; 1016 AdjacentBits = Packet->BitString &= ~AdjacentBits[SI]; 1017 Packet->BitString &= AdjacentBits[SI]; 1018 for (Index = GetFirstbit position(AdjacentBits); Index ; 1019 Index = GetNextbit position(AdjacentBits, Index)) { 1020 foreach adjacency BIFT[Index+Offset] { 1021 if(adjacency == ECMP(ListOfAdjacencies, seed) ) { 1022 I = ECMP_hash(sizeof(ListOfAdjacencies), 1023 Packet->Entropy, seed); 1024 adjacency = ListOfAdjacencies[I]; 1025 } 1026 PacketCopy = Copy(Packet); 1027 switch(adjacency) { 1028 case forward_connected(interface,neighbor,DNC): 1029 if(DNC) 1030 PacketCopy->BitString |= 1<<(Index-1); 1031 SendToL2Unicast(PacketCopy,interface,neighbor); 1033 case forward_routed({VRF},neighbor): 1034 SendToL3(PacketCopy,{VRF,}l3-neighbor); 1036 case local_decap({VRF},neighbor): 1037 DecapBierHeader(PacketCopy); 1038 PassTo(PacketCopy,{VRF,}Packet->NextProto); 1039 } 1040 } 1041 } 1042 } 1044 Figure 6: Complete BIER-TE Forwarding Pseudocode for required, 1045 recommended and optional functions 1047 4.5. Basic BIER-TE Forwarding Example 1049 [RFC Editor: remove this section.] 1051 THIS SECTION TO BE REMOVED IN RFC BECAUSE IT WAS SUPERCEEDED BY 1052 SECTION 1.1 EXAMPLE - UNLESS REVIEWERS CHIME IN AND EXPRESS DESIRE TO 1053 KEEP THIS ADDITIONAL EXAMPLE SECTION. ALVARO RETANA DID NOT MIND 1054 ANOTHER EXAMPLE. 1056 Step by step example of basic BIER-TE forwarding. This example does 1057 not use ECMP or forward_routed() adjacencies nor does it try to 1058 minimize the number of required BitPositions for the topology. 1060 [BIER-TE Controller] 1061 / | \ 1062 v v v 1063 . . 1064 | p13 p1 | . 1065 +- BFIR2 --+ | . 1066 | . | p2 p6 | . LAN2 1067 | . +-- BFR3 --+ . | 1068 | . | | p7 p11 | 1069 Src -+ . +-- BFER1 --+ 1070 | . | p3 p8 | . | 1071 | . +-- BFR4 --+ . +-- Rcv1 1072 | . | | . | 1073 | . | . 1074 | p14 p4 | . 1075 +- BFIR1 --+ | . 1076 | . +-- BFR5 --+ p10 p12 | 1077 LAN1 . | p5 p9 +-- BFER2 --+ 1078 . | . +-- Rcv2 1079 . . | 1080 . . LAN3 1081 . . 1082 IP |..... BIER-TE network.....| IP 1084 Figure 7: BIER-TE Forwarding Example 1086 pXX indicate the BitPositions number assigned by the BIER-TE 1087 Controller to adjacencies in the BIER-TE topology. For example, p9 1088 is the adjacency towards BFR5 on the LAN connecting to BFER2. 1090 BIFT BFIR2: 1091 p13: local_decap 1092 p2: forward_connected(BFR3) 1094 BIFT BFR3: 1095 p1: forward_connected(BFIR2) 1096 p7: forward_connected(BFER1) 1097 p8: forward_connected(BFR4) 1099 BIFT BFER1: 1100 p11: local_decap 1101 p6: forward_connected(BFR3) 1102 p8: forward_connected(BFR4) 1104 Figure 8: BIER-TE Forwarding Example Adjacencies 1106 ...and so on. 1108 For example, we assume that some multicast traffic seen on LAN1 needs 1109 to be sent via BIER-TE by BFIR2 towards Rcv1 and Rcv2. The BIER-TE 1110 Controller determines it wants it to pass this traffic across the 1111 following paths: 1113 -> BFER1 ---------------> Rcv1 1114 BFIR2 -> BFR3 1115 -> BFR4 -> BFR5 -> BFER2 -> Rcv2 1117 Figure 9: BIER-TE Forwarding Example Paths 1119 These paths equal to the following BitString: p2, p5, p7, p8, p10, 1120 p11, p12. 1122 This BitString is assigned by BFIR2 to the example multicast traffic 1123 received from LAN1. 1125 Then BFIR2 forwards this multicast traffic with BIER-TE based on that 1126 BitString. The BIFT of BFIR2 has only p2 and p13 populated. Only p2 1127 is in the BitString and this is an adjacency towards BFR3. BFIR2 1128 therefore clears p2 in the BitString and sends a copy towards BFR2. 1130 BFR3 sees a BitString of p5,p7,p8,p10,p11,p12. For those BPs, it has 1131 only adjacencies for p7,p8. It creates a copy of the packet to BFER1 1132 (due to p7) and one to BFR4 (due to p8). It clears p7, p8 before 1133 sending. 1135 BFER1 sees a BitString of p5,p10,p11,p12. For those BPs, it only has 1136 an adjacency for p11. p11 is a "local_decap" adjacency installed by 1137 the BIER-TE Controller to receive a copy of the BIER packet - dispose 1138 of the BIER header and pass the payload to IP multicast. IP 1139 multicast will then forward the packet out to LAN2 because it did 1140 receive PIM or IGMP joins on LAN2 for the traffic. 1142 Further processing of the packet in BFR4, BFR5 and BFER2 accordingly. 1144 4.6. BFR Requirements for BIER-TE forwarding 1146 BFR MUST support to configure the BIFT of sub-domains so that they 1147 use BIER-TE forwarding rules instead of BIER forwarding rules. Every 1148 BP in the BIFT MUST support to have zero or one adjacency. 1149 Forwarding MUST support the adjacency types forward_connected() with 1150 clear DNC flag, forward_routed() and local_decap. As explained in 1151 Section 4.4, these REQUIRED BIER-TE forwarding functions can be 1152 implement via the same Forwarding Pseudocode as BIER forwarding 1153 except for one modification (skipping one masking with F-BM). 1155 BIER-TE forwarding SHOULD support forward_connected() adjacencies 1156 with a set DNC flag, as this is highly useful to save bits in rings 1157 (see Section 5.1.6). 1159 BIER-TE forwarding SHOULD support more than one adjacency on a bit. 1160 This allows to save bits in hub&spoke scenarios (see Section 5.1.5). 1162 BIER-TE forwarding MAY support ECMP adjacencies to save bits in ECMP 1163 scenarios, see Section 5.1.7 for an example. This is a MAY 1164 requirement, because the deployment importance of ECMP adjacencies 1165 for BIER-TE is unclear as one can also leverage ECMP of the routing 1166 underlay via forwarded_routed adjacencies and/or might prefer to have 1167 more explicit control of the path chosen via explicit BP/adjacencies 1168 for each ECMP path alternative. 1170 5. BIER-TE Controller Operational Considerations 1172 5.1. Bit position Assignments 1174 This section describes how the BIER-TE Controller can use the 1175 different BIER-TE adjacency types to define the bit positions of a 1176 BIER-TE domain. 1178 Because the size of the BitString limits the size of the BIER-TE 1179 domain, many of the options described exist to support larger 1180 topologies with fewer bit positions (4.1, 4.3, 4.4, 4.5, 4.6, 4.7, 1181 4.8). 1183 5.1.1. P2P Links 1185 On a P2P link that connects two BFR, the same bit position can be 1186 used on both BFR for the adjacency to the neighboring BFR. A P2P 1187 link requires therefore only one bit position. 1189 5.1.2. BFER 1191 Every non-Leaf BFER is given a unique bit position with a local_decap 1192 adjacency. 1194 5.1.3. Leaf BFERs 1196 BFR1(P) BFR2(P) BFR1(P) BFR2(P) 1197 | \ / | | | 1198 | X | | | 1199 | / \ | | | 1200 BFER1(PE) BFER2(PE) BFER1(PE)----BFER2(PE) 1202 ^ U-turn link 1204 Leaf BFER / Non-Leaf BFER / 1205 PE-router PE-router 1207 Figure 10: Leaf vs. non-Leaf BFER Example 1209 A leaf BFERs is one where incoming BIER-TE packets never need to be 1210 forwarded to another BFR but are only sent to the BFER to exit the 1211 BIER-TE domain. For example, in networks where Provider Edge (PE) 1212 router are spokes connected to Provider (P) routers, those PEs are 1213 Leaf BFERs unless there is a U-turn between two PEs. 1215 Consider how redundant disjoint traffic can reach BFER1/BFER2 in 1216 Figure 10: When BFER1/BFER2 are Non-Leaf BFER as shown on the right 1217 hand side, one traffic copy would be forwarded to BFER1 from BFR1, 1218 but the other one could only reach BFER1 via BFER2, which makes BFER2 1219 a non-Leaf BFER. Likewise BFER1 is a non-Leaf BFER when forwarding 1220 traffic to BFER2. Note that the BFERs in the left hand picture are 1221 only guaranteed to be leaf-BFER by fitting routing configuration that 1222 prohibits transit traffic to pass through the BFERs, which is 1223 commonly applied in these topologies. 1225 All leaf-BFERs in a BIER-TE domain can share a single bit position. 1226 This is possible because the bit position for the adjacency to reach 1227 the BFER can be used to distinguish whether or not packets should 1228 reach the BFER. 1230 This optimization will not work if an upstream interface of the BFER 1231 is using a bit position optimized as described in the following two 1232 sections (LAN, Hub and Spoke). 1234 5.1.4. LANs 1236 In a LAN, the adjacency to each neighboring BFR is given a unique bit 1237 position. The adjacency of this bit position is a 1238 forward_connected() adjacency towards the BFR and this bit position 1239 is populated into the BIFT of all the other BFRs on that LAN. 1241 BFR1 1242 |p1 1243 LAN1-+-+---+-----+ 1244 p3| p4| p2| 1245 BFR3 BFR4 BFR7 1247 Figure 11: LAN Example 1249 If Bandwidth on the LAN is not an issue and most BIER-TE traffic 1250 should be copied to all neighbors on a LAN, then bit positions can be 1251 saved by assigning just a single bit position to the LAN and 1252 populating the bit position of the BIFTs of each BFRs on the LAN with 1253 a list of forward_connected() adjacencies to all other neighbors on 1254 the LAN. 1256 This optimization does not work in the case of BFRs redundantly 1257 connected to more than one LAN with this optimization because these 1258 BFRs would receive duplicates and forward those duplicates into the 1259 opposite LANs. Adjacencies of such BFRs into their LAN still need a 1260 separate bit position. 1262 5.1.5. Hub and Spoke 1264 In a setup with a hub and multiple spokes connected via separate p2p 1265 links to the hub, all p2p links can share the same bit position. The 1266 bit position on the hub's BIFT is set up with a list of 1267 forward_connected() adjacencies, one for each Spoke. 1269 This option is similar to the bit position optimization in LANs: 1270 Redundantly connected spokes need their own bit positions, unless 1271 they are themselves Leaf-BFER. 1273 This type of optimized BP could be used for example when all traffic 1274 is "broadcast" traffic (very dense receiver set) such as live-TV or 1275 situation-awareness (SA). This BP optimization can then be used to 1276 explicitly steer different traffic flows across different ECMP paths 1277 in Data-Center or broadband-aggregation networks with minimal use of 1278 BPs. 1280 5.1.6. Rings 1282 In L3 rings, instead of assigning a single bit position for every p2p 1283 link in the ring, it is possible to save bit positions by setting the 1284 "DoNotClear" (DNC) flag on forward_connected() adjacencies. 1286 For the rings shown in Figure 12, a single bit position will suffice 1287 to forward traffic entering the ring at BFRa or BFRb all the way up 1288 to BFR1: 1290 On BFRa, BFRb, BFR30,... BFR3, the bit position is populated with a 1291 forward_connected() adjacency pointing to the clockwise neighbor on 1292 the ring and with DNC set. On BFR2, the adjacency also points to the 1293 clockwise neighbor BFR1, but without DNC set. 1295 Handling DNC this way ensures that copies forwarded from any BFR in 1296 the ring to a BFR outside the ring will not have the ring bit 1297 position set, therefore minimizing the chance to create loops. 1299 v v 1300 | | 1301 L1 | L2 | L3 1302 /-------- BFRa ---- BFRb --------------------\ 1303 | | 1304 \- BFR1 - BFR2 - BFR3 - ... - BFR29 - BFR30 -/ 1305 | | L4 | | 1306 p33| p15| 1307 BFRd BFRc 1309 Figure 12: Ring Example 1311 Note that this example only permits for packets intended to make it 1312 all the way around the ring to enter it at BFRa and BFRb, and that 1313 packets will always travel clockwise. If packets should be allowed 1314 to enter the ring at any ring BFR, then one would have to use two 1315 ring bit positions. One for each direction: clockwise and 1316 counterclockwise. 1318 Both would be set up to stop rotating on the same link, e.g. L1. 1319 When the ingress ring BFR creates the clockwise copy, it will clear 1320 the counterclockwise bit position because the DNC bit only applies to 1321 the bit for which the replication is done. Likewise for the 1322 clockwise bit position for the counterclockwise copy. As a result, 1323 the ring ingress BFR will send a copy in both directions, serving 1324 BFRs on either side of the ring up to L1. 1326 5.1.7. Equal Cost MultiPath (ECMP) 1328 The ECMP adjacency allows to use just one BP per link bundle between 1329 two BFRs instead of one BP for each p2p member link of that link 1330 bundle. In Figure 13, one BP is used across L1,L2,L3. 1332 --L1----- 1333 BFR1 --L2----- BFR2 1334 --L3----- 1336 BIFT entry in BFR1: 1337 ------------------------------------------------------------------ 1338 | Index | Adjacencies | 1339 ================================================================== 1340 | 0:6 | ECMP({forward_connected(L1, BFR2), | 1341 | | forward_connected(L2, BFR2), | 1342 | | forward_connected(L3, BFR2)}, seed) | 1343 ------------------------------------------------------------------ 1345 BIFT entry in BFR2: 1346 ------------------------------------------------------------------ 1347 | Index | Adjacencies | 1348 ================================================================== 1349 | 0:6 | ECMP({forward_connected(L1, BFR1), | 1350 | | forward_connected(L2, BFR1), | 1351 | | forward_connected(L3, BFR1)}, seed) | 1352 ------------------------------------------------------------------ 1354 Figure 13: ECMP Example 1356 This document does not standardize any ECMP algorithm because it is 1357 sufficient for implementations to document their freely chosen ECMP 1358 algorithm. This allows the BIER-TE Controller to calculate ECMP 1359 paths and seeds. Figure 14 shows an example ECMP algorithm: 1361 forward(packet, ECMP(adj(0), adj(1),... adj(N-1), seed)): 1362 i = (packet(bier-header-entropy) XOR seed) % N 1363 forward packet to adj(i) 1365 Figure 14: ECMP algorithm Example 1367 In the following example, all traffic from BFR1 towards BFR10 is 1368 intended to be ECMP load split equally across the topology. This 1369 example is not meant as a likely setup, but to illustrate that ECMP 1370 can be used to share BPs not only across link bundles, but also 1371 across alternative paths across different transit BFR, and it 1372 explains the use of the seed parameter. 1374 BFR1 (BFIR) 1375 /L11 \L12 1376 / \ 1377 BFR2 BFR3 1378 /L21 \L22 /L31 \L32 1379 / \ / \ 1380 BFR4 BFR5 BFR6 BFR7 1381 \ / \ / 1382 \ / \ / 1383 BFR8 BFR9 1384 \ / 1385 \ / 1386 BFR10 (BFER) 1388 BIFT entry in BFR1: 1389 ------------------------------------------------------------------ 1390 | 0:6 | ECMP({forward_connected(L11, BFR2), | 1391 | | forward_connected(L12, BFR3)}, seed1) | 1392 ------------------------------------------------------------------ 1394 BIFT entry in BFR2: 1395 ------------------------------------------------------------------ 1396 | 0:7 | ECMP({forward_connected(L21, BFR4), | 1397 | | forward_connected(L22, BFR5)}, seed1) | 1398 ------------------------------------------------------------------ 1400 BIFT entry in BFR3: 1401 ------------------------------------------------------------------ 1402 | 0:7 | ECMP({forward_connected(L31, BFR6), | 1403 | | forward_connected(L32, BFR7)}, seed1) | 1404 ------------------------------------------------------------------ 1406 BIFT entry in BFR4, BFR5: 1407 ------------------------------------------------------------------ 1408 | 0:8 | forward_connected(Lxx, BFR8) |xx differs on BFR4/BFR5| 1409 ------------------------------------------------------------------ 1411 BIFT entry in BFR6, BFR7: 1412 ------------------------------------------------------------------ 1413 | 0:8 | forward_connected(Lxx, BFR9) |xx differs on BFR6/BFR7| 1414 ------------------------------------------------------------------ 1416 BIFT entry in BFR8, BFR9: 1417 ------------------------------------------------------------------ 1418 | 0:9 | forward_connected(Lxx, BFR10) |xx differs on BFR8/BFR9| 1419 ------------------------------------------------------------------ 1421 Figure 15: Polarization Example 1423 Note that for the following discussion of ECMP, only the BIFT ECMP 1424 adjacencies on BFR1, BFR2, BFR3 are relevant. The re-use of BP 1425 across BFR in this example is further explained in Section 5.1.9 1426 below. 1428 With the setup of ECMP in the topology above, traffic would not be 1429 equally load-split. Instead, links L22 and L31 would see no traffic 1430 at all: BFR2 will only see traffic from BFR1 for which the ECMP hash 1431 in BFR1 selected the first adjacency in the list of 2 adjacencies 1432 given as parameters to the ECMP. It is link L11-to-BFR2. BFR2 1433 performs again ECMP with two adjacencies on that subset of traffic 1434 using the same seed1, and will therefore again select the first of 1435 its two adjacencies: L21-to-BFR4. And therefore L22 and BFR5 sees no 1436 traffic. Likewise for L31 and BFR6. 1438 This issue in BFR2/BFR3 is called polarization. It results from the 1439 re-use of the same hash function across multiple consecutive hops in 1440 topologies like these. To resolve this issue, the ECMP adjacency on 1441 BFR1 can be set up with a different seed2 than the ECMP adjacencies 1442 on BFR2/BFR3. BFR2/BFR3 can use the same hash because packets will 1443 not sequentially pass across both of them. Therefore, they can also 1444 use the same BP 0:7. 1446 Note that ECMP solutions outside of BIER often hide the seed by auto- 1447 selecting it from local entropy such as unique local or next-hop 1448 identifiers. Allowing the BIER-TE Controller to explicitly set the 1449 seed gives the ability for it to control same/different path 1450 selection across multiple consecutive ECMP hops. 1452 5.1.8. Forward_routed adjacencies 1454 5.1.8.1. Reducing bit positions 1456 Forward_routed() adjacencies can reduce the number of bit positions 1457 required when the path steering requirement is not hop-by-hop 1458 explicit path selection, but loose-hop selection. Forward_routed() 1459 adjacencies can also allow to operate BIER-TE across intermediate hop 1460 routers that do not support BIER-TE. 1462 ............... 1463 ...BFR1--... ...--L1-- BFR2... 1464 ... .Routers. ...--L2--/ 1465 ...BFR4--... ...------ BFR3... 1466 ............... | 1467 LO 1468 Network Area 1 1470 Figure 16: Forward_routed Adjacencies Example 1472 Assume the requirement in Figure 16 is to explicitly steer traffic 1473 flows that have arrived at BFR1 or BFR4 via a shortest path in the 1474 routing underlay "Network Area 1" to one of the following three next 1475 segments: (1) BFR2 via link L1, (2) BFR2 via link L2, or (3) via 1476 BFR3. 1478 To enable this, both BFR1 and BFR4 are set up with a forward_routed 1479 adjacency bit position towards an address of BFR2 on link L1, another 1480 forward_routed() bit position towards an address of BFR2 on link L2 1481 and a third forward_routed() bit position towards a node address LO 1482 of BFR3. 1484 5.1.8.2. Supporting nodes without BIER-TE 1486 Forward_routed() adjacencies also enable incremental deployment of 1487 BIER-TE. Only the nodes through which BIER-TE traffic needs to be 1488 steered - with or without replication - need to support BIER-TE. 1489 Where they are not directly connected to each other, forward_routed 1490 adjacencies are used to pass over non BIER-TE enabled nodes. 1492 5.1.9. Reuse of bit positions (without DNC) 1494 bit positions can be re-used across multiple BFR to minimize the 1495 number of BP needed. This happens when adjacencies on multiple BFR 1496 use the DNC flag as described above, but it can also be done for non- 1497 DNC adjacencies. This section only discusses this non-DNC case. 1499 Because BP are cleared when passing a BFR with an adjacency for that 1500 BP, reuse of BP across multiple BFR does not introduce any problems 1501 with duplicates or loops that do not also exist when every adjacency 1502 has a unique BP. Instead, the challenge when reusing BP is whether 1503 it allows to still achieve the desired Tree Engineering goals. 1505 BP cannot be reused across two BFR that would need to be passed 1506 sequentially for some path: The first BFR will clear the BP, so those 1507 paths cannot be built. BP can be set across BFR that would (A) only 1508 occur across different paths or (B) across different branches of the 1509 same tree. 1511 An example of (A) was given in Figure 15, where BP 0:7, BP 0:8 and BP 1512 0:9 are each reused across multiple BFRs because a single packet/path 1513 would never be able to reach more than one BFR sharing the same BP. 1515 Assume the example was changed: BFR1 has no ECMP adjacency for BP 1516 0:6, but instead BP 0:5 with forward_connected() to BFR2 and BP 0:6 1517 with forward_connected() to BFR3. Packets with both BP 0:5 and BP 1518 0:6 would now be able to reach both BFR2 and BFR3 and the still 1519 existing re-use of BP 0:7 between BFR2 and BFR3 is a case of (B) 1520 where reuse of BP is perfect because it does not limit the set of 1521 useful path choices: 1523 If instead of reusing BP 0:7, BFR3 used a separate BP 0:10 for its 1524 ECMP adjacency, no useful additional path steering options would be 1525 enabled. If duplicates at BFR10 where undesirable, this would be 1526 done by not setting BP 0:5 and BP 0:6 for the same packet. If the 1527 duplicates where desirable (e.g.: resilient transmission), the 1528 additional BP 0:10 would also not render additional value. 1530 area1 1531 BFR1a BFR1b 1532 / \ 1533 .................................... 1534 . Core . 1535 .................................... 1536 | / \ / \ | 1537 BFR2a BFR2b BFR3a BFR3b BFR6a BFR6b 1538 /-------\ /---------\ /--------\ 1539 | area2 | | area3 | ... | area6 | 1540 | ring | | ring | | ring | 1541 \-------/ \---------/ \--------/ 1542 more BFR more BFR more BFR 1544 Figure 17: Reuse of BP 1546 Reuse may also save BPs in larger topologies. Consider the topology 1547 shown in Figure 20. A BFIR/sender (e.g.: video headend) is attached 1548 to area 1, and area 2...6 contain receivers/BFER. Assume each area 1549 had a distribution ring, each with two BPs to indicate the direction 1550 (as explained before). These two BPs could be reused across the 5 1551 areas. Packets would be replicated through other BPs for the Core to 1552 the desired subset of areas, and once a packet copy reaches the ring 1553 of the area, the two ring BPs come into play. This reuse is a case 1554 of (B), but it limits the topology choices: Packets can only flow 1555 around the same direction in the rings of all areas. This may or may 1556 not be acceptable based on the desired path steering options: If 1557 resilient transmission is the path engineering goal, then it is 1558 likely a good optimization, if the bandwidth of each ring was to be 1559 optimized separately, it would not be a good limitation. 1561 5.1.10. Summary of BP optimizations 1563 This section reviewed a range of techniques by which a BIER-TE 1564 Controller can create a BIER-TE topology in a way that minimizes the 1565 number of necessary BPs. 1567 Without any optimization, a BIER-TE Controller would attempt to map 1568 the network subnet topology 1:1 into the BIER-TE topology and every 1569 subnet adjacent neighbor requires a forward_connected() BP and every 1570 BFER requires a local_decap() BP. 1572 The optimizations described are then as follows: 1574 o P2P links require only one BP (Section 5.1.1). 1576 o All leaf-BFER can share a single local_decap() BP (Section 5.1.3). 1578 o A LAN with N BFR needs at most N BP (one for each BFR). It only 1579 needs one BP for all those BFR that are not redundantly connected 1580 to multiple LANs (Section 5.1.4). 1582 o A hub with p2p connections to multiple non-leaf-BFER spokes can 1583 share one BP to all spokes if traffic can be flooded to all 1584 spokes, e.g.: because of no bandwidth concerns or dense receiver 1585 sets (Section 5.1.5). 1587 o Rings of BFR can be built with just two BP (one for each 1588 direction) except for BFR with multiple ring connections - similar 1589 to LANs (Section 5.1.6). 1591 o ECMP adjacencies to N neighbors can replace N BP with 1 BP. 1592 Multihop ECMP can avoid polarization through different seeds of 1593 the ECMP algorithm (Section 5.1.7). 1595 o Forward_routed() adjacencies allow to "tunnel" across non-BIER-TE 1596 capable routers and across BIER-TE capable routers where no 1597 traffic-steering or replications are required (Section 5.1.8). 1599 o BP can generally be reused across nodes that do not need to be 1600 consecutive in paths, but depending on scenario, this may limit 1601 the feasible path steering options (Section 5.1.9). 1603 Note that the described list of optimizations is not exhaustive. 1604 Especially when the set of required path steering choices is limited 1605 and the set of possible subsets of BFERs that should be able to 1606 receive traffic is limited, further optimizations of BP are possible. 1607 The hub & spoke optimization is a simple example of such traffic 1608 pattern dependent optimizations. 1610 5.2. Avoiding duplicates and loops 1612 5.2.1. Loops 1614 Whenever BIER-TE creates a copy of a packet, the BitString of that 1615 copy will have all bit positions cleared that are associated with 1616 adjacencies on the BFR. This inhibits looping of packets. The only 1617 exception are adjacencies with DNC set. 1619 v v 1620 | | 1621 L1 | L2 | L3 1622 /-------- BFRa ---- BFRb ---------------------\ 1623 | . | 1624 | ...... Wrong link wiring | 1625 | . | 1626 \- BFR1 - BFR2 BFR3 - ... - BFR29 - BFR30 -/ 1627 | | L4 | | 1628 p33| p15| 1629 BFRd BFRc 1631 Figure 18: Miswired Ring Example 1633 With DNC set, looping can happen. Consider in Figure 18 that link L4 1634 from BFR3 is (inadvertently) plugged into the L1 interface of BFRa 1635 (instead of BFR2). This creates a loop where the rings clockwise bit 1636 position is never cleared for copies of the packets traveling 1637 clockwise around the ring. 1639 To inhibit looping in the face of such physical misconfiguration, 1640 only forward_connected() adjacencies are permitted to have DNC set, 1641 and the link layer port unique unicast destination address of the 1642 adjacency (e.g. MAC address) protects against closing the loop. 1643 Link layers without port unique link layer addresses should not be 1644 used with the DNC flag set. 1646 5.2.2. Duplicates 1647 BFIR1 1648 / \ 1649 / p2 \ p3 1650 BFR2 BFR3 1651 \ p4 / p5 1652 \ / 1653 BFER4 1655 Figure 19: Duplicates Example 1657 Duplicates happen when the graph expressed by a BitString is not a 1658 tree but redundantly connecting BFRs with each other. In Figure 19, 1659 a BitString of p2,p3,p4,p5 would result in duplicate packets to 1660 arrive on BFER4. The BIER-TE Controller must therefore ensure to 1661 only create BitStrings that are trees. 1663 When links are incorrectly physically re-connected before the BIER-TE 1664 Controller updates BitStrings in BFIRs, duplicates can happen. Like 1665 loops, these can be inhibited by link layer addressing in 1666 forward_connected() adjacencies. 1668 If interface or loopback addresses used in forward_routed() 1669 adjacencies are moved from one BFR to another, duplicates can equally 1670 happen. Such re-addressing operations must be coordinated with the 1671 BIER-TE Controller. 1673 5.3. Managing SI, sub-domains and BFR-ids 1675 When the number of bits required to represent the necessary hops in 1676 the topology and BFER exceeds the supported BitStringLength (BSL), 1677 multiple SIs and/or sub-domains must be used. This section discusses 1678 how. 1680 BIER-TE forwarding does not require the concept of BFR-id, but 1681 routing underlay, flow overlay and BIER headers may. This section 1682 also discusses how BFR-ids can be assigned to BFIR/BFER for BIER-TE. 1684 5.3.1. Why SI and sub-domains 1686 For BIER and BIER-TE forwarding, the most important result of using 1687 multiple SI and/or sub-domains is the same: Packets that need to be 1688 sent to BFERs in different SIs or sub-domains require different BIER 1689 packets: each one with a BitString for a different (SI,sub-domain) 1690 combination. Each such BitString uses one BSL sized SI block in the 1691 BIFT of the sub-domain. We call this a BIFT:SI (block). 1693 For BIER and BIER-TE forwarding themselves there is also no 1694 difference whether different SIs and/or sub-domains are chosen, but 1695 SI and sub-domain have different purposes in the BIER architecture 1696 shared by BIER-TE. This impacts how operators are managing them and 1697 how especially flow overlays will likely use them. 1699 By default, every possible BFIR/BFER in a BIER network would likely 1700 be given a BFR-id in sub-domain 0 (unless there are > 64k BFIR/BFER). 1702 If there are different flow services (or service instances) requiring 1703 replication to different subsets of BFERs, then it will likely not be 1704 possible to achieve the best replication efficiency for all of these 1705 service instances via sub-domain 0. Ideal replication efficiency for 1706 N BFER exists in a sub-domain if they are split over not more than 1707 ceiling(N/BitStringLength) SI. 1709 If service instances justify additional BIER:SI state in the network, 1710 additional sub-domains will be used: BFIR/BFER are assigned BFR-id in 1711 those sub-domains and each service instance is configured to use the 1712 most appropriate sub-domain. This results in improved replication 1713 efficiency for different services. 1715 Even if creation of sub-domains and assignment of BFR-id to BFIR/BFER 1716 in those sub-domains is automated, it is not expected that individual 1717 service instances can deal with BFER in different sub-domains. A 1718 service instance may only support configuration of a single sub- 1719 domain it should rely on. 1721 To be able to easily reuse (and modify as little as possible) 1722 existing BIER procedures including flow-overlay and routing underlay, 1723 when BIER-TE forwarding is added, we therefore reuse SI and sub- 1724 domain logically in the same way as they are used in BIER: All 1725 necessary BFIR/BFER for a service use a single BIER-TE BIFT and are 1726 split across as many SIs as necessary (see Section 5.3.2). Different 1727 services may use different sub-domains that primarily exist to 1728 provide more efficient replication (and for BIER-TE desirable path 1729 steering) for different subsets of BFIR/BFER. 1731 5.3.2. Assigning bits for the BIER-TE topology 1733 In BIER, BitStrings only need to carry bits for BFERs, which leads to 1734 the model that BFR-ids map 1:1 to each bit in a BitString. 1736 In BIER-TE, BitStrings need to carry bits to indicate not only the 1737 receiving BFER but also the intermediate hops/links across which the 1738 packet must be sent. The maximum number of BFER that can be 1739 supported in a single BitString or BIFT:SI depends on the number of 1740 bits necessary to represent the desired topology between them. 1742 "Desired" topology because it depends on the physical topology, and 1743 on the desire of the operator to allow for explicit path steering 1744 across every single hop (which requires more bits), or reducing the 1745 number of required bits by exploiting optimizations such as unicast 1746 (forward_routed), ECMP or flood (DNC) over "uninteresting" sub-parts 1747 of the topology - e.g. parts where different trees do not need to 1748 take different paths due to path steering reasons. 1750 The total number of bits to describe the topology vs. the number of 1751 BFERs in a BIFT:SI can range widely based on the size of the topology 1752 and the amount of alternative paths in it. The higher the percentage 1753 of non-BFER bits, the higher the likelihood, that those topology bits 1754 are not just BIER-TE overhead without additional benefit, but instead 1755 that they will allow to express desirable path steering alternatives. 1757 5.3.3. Assigning BFR-id with BIER-TE 1759 BIER-TE forwarding does not use the BFR-id, not does it require for 1760 the BFR-id field of the BIER header to be set to a particular value. 1761 However, other parts of a BIER-TE deployment may need a BFR-id, 1762 specifically overlay signaling, and in that case BFR need to also 1763 have BFR-ids for BIER-TE SDs. 1765 For example, for BIER overlay signaling, BFIR need to have a BFR-id, 1766 because this BFIR BFR-id is carried in the BFR-id field of the BIER 1767 header to indicate to the overlay signaling on the receiving BFER 1768 which BFIR originated the packet. 1770 In BIER, BFR-id = BSL * SI + BP, such that the SI and BP of a BFER 1771 can be calculated from the BFR-id and vice versa. This also means 1772 that every BFR with a BFR-id has a reserved BP in an SI, even if that 1773 is not necessary for BIER forwarding, because the BFR may never be a 1774 BFER but only a BFIR. 1776 In BIER-TE, for a non-leaf BFER, there is usually a single BP for 1777 that BFER with a local_decap() adjacency on the BFER. The BFR-id for 1778 such a BFER can therefore equally be determined as in BIER: BFR-id = 1779 SI * BitStringLength + BP. 1781 As explained in Section 5.1.3, leaf BFERs do not need such a unique 1782 local_decap() adjacency, likewise, BFIR who are not also BFER may not 1783 have a unique local_decap() adjacency either. For all those BFIR and 1784 (leaf) BFER, the controller needs to determine unique BFR-ids that do 1785 not collide with the BFR-ids derived from the non-leaf BFER 1786 local_decap() BPs. 1788 While this document defines no requirements how to allocate such BFR- 1789 id, a simple option is to derive it from the (SI,BP) of an adjacency 1790 that is unique to the BFR in question. For a BFIR this can be he 1791 first adjacency only populated on this BFIR, for a leaf-BFER, this 1792 could be the first BP with an adjacency towards that BFER. 1794 5.3.4. Mapping from BFR to BitStrings with BIER-TE 1796 In BIER, applications of the flow overlay on a BFIR can calculate the 1797 (SI,BP) of a BFER from the BFR-id of the BFER and can therefore 1798 easily determine the BitStrings for a BIER packet to a set of BFER 1799 with known BFR-ids. 1801 In BIER-TE this mapping needs to be equally supported for flow 1802 overlays. This section outlines two core options, based on how 1803 "complex" the Tree Engineering is that the BIER-TE controller 1804 performs for a particular application. 1806 "Independent branches": For a given flow overlay instance, the 1807 branches from a BFIR to every BFER are calculated by the BIER-TE 1808 controller to be independent of the branches to any other BFER. 1809 Shortest path trees are the most common examples of trees with 1810 independent branches. 1812 "Interdependent branches": When a BFER is added or deleted from a 1813 particular distribution tree, the BIER-TE controller has to 1814 recalculate the branches to other BFER, because they may need to 1815 change. Steiner trees are examples of interdependent branch trees. 1817 If "independent branches" are used, the BIER-TE Controller can signal 1818 to the BFIR flow overlay for every BFER an SI:BitString that 1819 represents the branch to that BFER. The flow overlay on the BIFR can 1820 then independently of the controller calculate the SI:BitString for 1821 all desired BFER by OR'ing their BitStrings. This allows for flow 1822 overlay applications to operate independently from the controller 1823 whenever it needs to determine which subset of BFERs need to receive 1824 a particular packet. 1826 If "interdependent branches" are required, the application would need 1827 to inquire the SI:BitString for a given set of BFER whenever the set 1828 changes. 1830 Note that in either case (unlike in BIER), the bits may need to 1831 change upon link/node failure/recovery, network expansion and network 1832 resource consumption by other traffic as part of traffic engineering 1833 goals (e.g.: re-optimization of lower priority traffic flows). 1834 Interactions between such BFIR applications and the BIER-TE 1835 Controller do therefore need to support dynamic updates to the 1836 SI:BitStrings. 1838 Communications between BFIR flow overlay and BIER-TE controller 1839 requires some way to identify BFER. If BFR-ids are used in the 1840 deployment, as outlined in Section 5.3.3, then those are the natural 1841 BFR identifier. If BFR-ids are not used, then any other unique 1842 identifier, such as the BFR-prefix of the BFR as of [RFC8279] could 1843 be used. 1845 5.3.5. Assigning BFR-ids for BIER-TE 1847 It is not currently determined if a single sub-domain could or should 1848 be allowed to forward both BIER and BIER-TE packets. If this should 1849 be supported, there are two options: 1851 A. BIER and BIER-TE have different BFR-id in the same sub-domain. 1852 This allows higher replication efficiency for BIER because their BFR- 1853 id can be assigned sequentially, while the BitStrings for BIER-TE 1854 will have also the additional bits for the topology. There is no 1855 relationship between a BFR BIER BFR-id and BIER-TE BFR-id. 1857 B. BIER and BIER-TE share the same BFR-id. The BFR-ids are assigned 1858 as explained above for BIER-TE and simply reused for BIER. The 1859 replication efficiency for BIER will be as low as that for BIER-TE in 1860 this approach. Depending on topology, only the same 20%..80% of bits 1861 as possible for BIER-TE can be used for BIER. 1863 5.3.6. Example bit allocations 1865 5.3.6.1. With BIER 1867 Consider a network setup with a BSL of 256 for a network topology as 1868 shown in Figure 20. The network has 6 areas, each with 170 BFRs, 1869 connecting via a core with 4 (core) BFRs. To address all BFERs with 1870 BIER, 4 SIs are required. To send a BIER packet to all BFER in the 1871 network, 4 copies need to be sent by the BFIR. On the BFIR it does 1872 not make a difference how the BFR-ids are allocated to BFER in the 1873 network, but for efficiency further down in the network it does make 1874 a difference. 1876 area1 area2 area3 1877 BFR1a BFR1b BFR2a BFR2b BFR3a BFR3b 1878 | \ / \ / | 1879 ................................ 1880 . Core . 1881 ................................ 1882 | / \ / \ | 1883 BFR4a BFR4b BFR5a BFR5b BFR6a BFR6b 1884 area4 area5 area6 1886 Figure 20: Scaling BIER-TE bits by reuse 1888 With random allocation of BFR-id to BFER, each receiving area would 1889 (most likely) have to receive all 4 copies of the BIER packet because 1890 there would be BFR-id for each of the 4 SIs in each of the areas. 1891 Only further towards each BFER would this duplication subside - when 1892 each of the 4 trees runs out of branches. 1894 If BFR-ids are allocated intelligently, then all the BFER in an area 1895 would be given BFR-id with as few as possible different SIs. Each 1896 area would only have to forward one or two packets instead of 4. 1898 Given how networks can grow over time, replication efficiency in an 1899 area will also easily go down over time when BFR-ids are network wide 1900 allocated sequentially over time. An area that initially only has 1901 BFR-id in one SI might end up with many SIs over a longer period of 1902 growth. Allocating SIs to areas with initially sufficiently many 1903 spare bits for growths can help to alleviate this issue. Or renumber 1904 BFERs after network expansion. In this example one may consider to 1905 use 6 SIs and assign one to each area. 1907 This example shows that intelligent BFR-id allocation within at least 1908 sub-domain 0 can even be helpful or even necessary in BIER. 1910 5.3.6.2. With BIER-TE 1912 In BIER-TE one needs to determine a subset of the physical topology 1913 and attached BFERs so that the "desired" representation of this 1914 topology and the BFER fit into a single BitString. This process 1915 needs to be repeated until the whole topology is covered. 1917 Once bits/SIs are assigned to topology and BFERs, BFR-id is just a 1918 derived set of identifiers from the operator/BIER-TE Controller as 1919 explained above. 1921 Every time that different sub-topologies have overlap, bits need to 1922 be repeated across the BitStrings, increasing the overall amount of 1923 bits required across all BitString/SIs. In the worst case, random 1924 subsets of BFERs are assigned to different SIs. This is much worse 1925 than in BIER because it not only reduces replication efficiency with 1926 the same number of overall bits, but even further - because more bits 1927 are required due to duplication of bits for topology across multiple 1928 SIs. Intelligent BFER to SI assignment and selecting specific 1929 "desired" subtopologies can minimize this problem. 1931 To set up BIER-TE efficiently for the above topology, the following 1932 bit allocation method can be used. This method can easily be 1933 expanded to other, similarly structured larger topologies. 1935 Each area is allocated one or more SIs depending on the number of 1936 future expected BFERs and number of bits required for the topology in 1937 the area. In this example, 6 SIs, one per area. 1939 In addition, we use 4 bits in each SI: bia, bib, bea, beb: bit 1940 ingress a, bit ingress b, bit egress a, bit egress b. These bits 1941 will be used to pass BIER packets from any BFIR via any combination 1942 of ingress area a/b BFR and egress area a/b BFR into a specific 1943 target area. These bits are then set up with the right 1944 forward_routed() adjacencies on the BFIR and area edge BFR: 1946 On all BFIRs in an area j|j=2...6, bia in each BIFT:SI is populated 1947 with the same forward_routed(BFRja), and bib with 1948 forward_routed(BFRjb). On all area edge BFR, bea in 1949 BIFT:SI=k|k=2...6 is populated with forward_routed(BFRka) and beb in 1950 BIFT:SI=k with forward_routed(BFRkb). 1952 For BIER-TE forwarding of a packet to a subset of BFERs across all 1953 areas, a BFIR would create at most 6 copies, with SI=1...SI=6, In 1954 each packet, the bits indicate bits for topology and BFER in that 1955 topology plus the four bits to indicate whether to pass this packet 1956 via the ingress area a or b border BFR and the egress area a or b 1957 border BFR, therefore allowing path steering for those two "unicast" 1958 legs: 1) BFIR to ingress are edge and 2) core to egress area edge. 1959 Replication only happens inside the egress areas. For BFER in the 1960 same area as in the BFIR, these four bits are not used. 1962 5.3.7. Summary 1964 BIER-TE can, like BIER, support multiple SIs within a sub-domain to 1965 allow re-using the concept of BFR-id and therefore minimize BIER-TE 1966 specific functions in any possible BIER layer control plane used in 1967 conjunction with BIER-TE, flow overlay methods and BIER headers. 1969 The number of BFIR/BFER possible in a sub-domain is smaller than in 1970 BIER because BIER-TE uses additional bits for topology. 1972 Sub-domains (SDs) in BIER-TE can be used like in BIER to create more 1973 efficient replication to known subsets of BFERs. 1975 Assigning bits for BFERs intelligently into the right SI is more 1976 important in BIER-TE than in BIER because of replication efficiency 1977 and overall amount of bits required. 1979 6. BIER-TE and Segment Routing 1981 SR aims to enable lightweight path steering via loose source routing. 1982 Compared to its more heavy-weight predecessor RSVP-TE, SR does for 1983 example not require per-path signaling to each of these hops. 1985 BIER-TE supports the same design philosophy for multicast. Like in 1986 SR, it relies on source-routing - via the definition of a BitString. 1987 Like SR, it only requires to consider the "hops" on which either 1988 replication has to happen, or across which the traffic should be 1989 steered (even without replication). Any other hops can be skipped 1990 via the use of routed adjacencies. 1992 BIER-TE bit position (BP) can be understood as the BIER-TE equivalent 1993 of "forwarding segments" in SR, but they have a different scope than 1994 SR forwarding segments. Whereas forwarding segments in SR are global 1995 or local, BPs in BIER-TE have a scope that is the group of BFR(s) 1996 that have adjacencies for this BP in their BIFT. This can be called 1997 "adjacency" scoped forwarding segments. 1999 Adjacency scope could be global, but then every BFR would need an 2000 adjacency for this BP, for example a forward_routed() adjacency with 2001 encapsulation to the global SR SID of the destination. Such a BP 2002 would always result in ingress replication though. The first BFR 2003 encountering this BP would directly replicate to it. Only by using 2004 non-global adjacency scope for BPs can traffic be steered and 2005 replicated on non-ingress BFR. 2007 SR can naturally be combined with BIER-TE and help to optimize it. 2008 For example, instead of defining bit positions for non-replicating 2009 hops, it is equally possible to use segment routing encapsulations 2010 (e.g. SR-MPLS label stacks) for the encapsulation of 2011 "forward_routed" adjacencies. 2013 Note that BIER itself can also be seen to be similar to SR. BIER BPs 2014 act as global destination Node-SIDs and the BIER BitString is simply 2015 a highly optimized mechanism to indicate multiple such SIDs and let 2016 the network take care of effectively replicating the packet hop-by- 2017 hop to each destination Node-SID. What BIER does not allow is to 2018 indicate intermediate hops, or terms of SR the ability to indicate a 2019 sequence of SID to reach the destination. This is what BIER-TE and 2020 its adjacency scoped BP enables. 2022 Both BIER and BIER-TE allow BFIR to "opportunistically" copy packets 2023 to a set of desired BFER on a packet-by-packet basis. In BIER, this 2024 is done by OR'ing the BP for the desired BFER. In BIER-TE this can 2025 be done by OR'ing for each desired BFER a BitString using the 2026 "independent branches" approach described in Section 5.3.3 and 2027 therefore also indicating the engineered path towards each desired 2028 BFER. This is the approach that 2029 [I-D.ietf-bier-multicast-http-response] relies on. 2031 7. Security Considerations 2033 If [RFC8296] is used, BIER-TE shares its security considerations. 2035 BIER-TE shares the security considerations of BIER, [RFC8279], with 2036 the following overriding or additional considerations. 2038 In BIER, the standardized methods for the routing underlays as well 2039 as to distribute BFR-ids and BFR-prefixes are IGPs such as specified 2040 in [RFC8401] for IS-IS and in [RFC8444] for OSPF. Attacking the 2041 protocols for the BIER routing underlay or BIER layer control plane, 2042 or impairment of any BFR in a domain may lead to successful attacks 2043 against the results of the routing protocol, enabling DoS attacks 2044 against paths or addressing (BFR-id, BFR-prefixes) used by BIER. 2046 The reference model for the BIER-TE layer control plane is a BIER-TE 2047 controller. When such a controller is used, impairment of individual 2048 BFR in a domain causes no impairment of the BIER-TE control plane on 2049 other BFR. If a routing protocol is used to support forward_routed() 2050 adjacencies, then this is still an attack vector as in BIER, but only 2051 for BIER-TE forward_routed() adjacencies, and no other adjacencies. 2053 Whereas IGP routing protocols are most often not well secured through 2054 cryptographic authentication and confidentiality, communications 2055 between controllers and routers such as those to be considered for 2056 the BIER-TE controller/control-plane can and are much more commonly 2057 secured with those security properties, for example by using Secure 2058 SHell (SSH), [RFC4253] for NetConf ([RFC6241]), or via Transport 2059 Layer Security (TLS), such as [RFC8253] for PCEP, [RFC5440], or 2060 [RFC7589] for NetConf. 2062 For additional, BIER-TE independent security considerations for the 2063 use of a central BIER-TE controller, the security section of the 2064 protocols and security options in the previous paragraph apply. In 2065 addition, the security considerations of [RFC4655] apply. 2067 The most important attack vector in BIER-TE is misconfiguration, 2068 either on the BFR themselves or via the BIER-TE controller. 2069 Forwarding entries with DNC could be set up to create persistent 2070 loops, in which packets only expire because of TTL. To minimize the 2071 impact of such attacks (or more likely unintentional misconfiguration 2072 by operators and/or bad BIER-TE controller software), the BIER-TE 2073 forwarding rules are defined to be as strict in clearing bits as they 2074 are. The clearing of all bits with an adjacency on a BFR prohibits 2075 that a looping packet creates additional packet amplification through 2076 the misconfigured loop on the packets second or further times around 2077 the loop, because all relevant adjacency bits would have been cleared 2078 on the first round through the loop. In result, BIER-TE has the same 2079 degree of looping packets as possible with unintentional or malicious 2080 loops in the routing underlay with BIER or even with unicast traffic. 2082 Deployments especially where BIER-TE would likely be beneficial may 2083 include operational models where actual configuration changes from 2084 the controller are only required during non-productive phases of the 2085 networks life-cycle, such as in embedded networks or in manufacturing 2086 networks during e.g. plant reworking/repairs. In these type of 2087 deployments, configuration changes could be locked out when the 2088 network is in production state and could only be (re-)enabled through 2089 reverting the network/installation into non-productive state. Such 2090 security designs would not only allow to provide additional layers of 2091 protection against configuration attacks, but would foremost protect 2092 the active production process from such configuration attacks. 2094 8. IANA Considerations 2096 This document requests no action by IANA. 2098 9. Acknowledgements 2100 The authors would like to thank Greg Shepherd, Ijsbrand Wijnands, 2101 Neale Ranns, Dirk Trossen, Sandy Zheng, Lou Berger, Jeffrey Zhang, 2102 Alvaro Retana and Wolfgang Braun for their reviews and suggestions. 2104 10. Change log [RFC Editor: Please remove] 2106 draft-ietf-bier-te-arch: 2108 10: AD review Alvaro Retana, summary: 2110 Note: rfcdiff shows more changes than actually exist because text 2111 moved around. 2113 Summary: 2115 1. restructuring: merged all controller sections under common 2116 controller ops main section, moved unfitting stuff out to 2117 other parts of doc. Split Intro section into Overview and 2118 Intro. Shortened Abstract, moved text into Overview, added 2119 sections overview. 2121 2. enhanced/rewrote: 2.3 Comparison with -> Relationship to BIER- 2122 TE 2124 3. enhanced/rewrote: 3.2 BIER-TE controller -> BIER-TE control 2125 plane, 3.2.1 BIER-TE controller, for consistency with rfc8279 2127 4. additional subsections for Alvaros asks 2129 5. added to: 3.3 BIER-TE forwarding plane (consistency with 2130 rfc8279) 2132 6. Enhanced description of 4.3/encap considerations to better 2133 explain how BIER/BIER-TE can run together. 2135 Notation: Markers (a),(b),... at end of points are references from 2136 the review discussion with Alvaro to the changes made. 2138 Details:. 2140 Throughout text: changed term spelling to rfc8279 - bit positions, 2141 sub-domain, ... (i). 2143 Reset changed to clear, also DNR changed to DNC (Do Not Clear) 2144 (q). 2146 Abstract: Shortened. Removed name explanation note (Tree 2147 Engineering), (a). 2149 1. Introduction -> Overview: Moved important explanation 2150 paragraph from abstract to Introduction. Fixed text, (a). 2152 Added bullet point list explanation of structure of document (e). 2154 Renamed to Overview because that is now more factually correct. 2156 1.1. Fixed bug in example adding bit p15.(l). 2158 2. (New - Introduction): Moved section 1.1 - 1.3 (examples, 2159 comparison with BIER-TE) from Introduction into new "Overview" 2160 section. Primarily so that "requirements language" section (at 2161 end of Introduction) is not in middle of document after all the 2162 Introduction. 2164 2.1 Removed discussion of encap, moved to 4.2.2 (m). 2166 2.2 enhanced paragraph suggesting native/overlay topology types, 2167 also sugest type hybrid (n). 2169 2.3 Overhauled comparison text BIER/BIER-TE, structured into 2170 common, different, not-required-by-te, integration-bier-bier-te. 2171 Changed title to "Relationship" to allow including last point. 2172 (f). 2174 2.4 moved Hardware forwarding comparison section into section 2 to 2175 allow coalescing of sections into section 5 about the controller 2176 operations (hardware forwarding was in the middle of it, wrong 2177 place). Shortened/improved third paragraph by pointing to BIFT as 2178 deciding element for selection between BIER/BIER-TE. Removed 2179 notion of experimentation (this now targets standard) (g). 2181 3. (Components): Aligned component name and descriptions better 2182 with RFC8279. Now describe exactly same three layers. BIER layer 2183 constituted from BIER-TE control plane and BIER-TE forwarding 2184 plane. BIER-TE controller is now simply component of BIER-TE 2185 control plane. (b). 2187 3.1. shortened/improved paragraph explaining use of SI:BP instead 2188 of also bfr-id as index into BIFT, rewrote paragraph talking about 2189 reuse of BPs(o). 2191 3.2. rewrote explanation of BIER-TE control plane in the style of 2192 RFC8729 Section 4.2 (BIER layer) with numbered points. Note that 2193 RFC8729 mixes control and forwarding plane bullet points (this doc 2194 does not). Merged text from old sections 2.2.1 and 2.2.3 into 2195 list. (b). 2197 3.2.1. Expanded/improved explanation of BIER-TE Controller (b). 2199 3.2.1.1. Added subsection for topology discovery and creation 2200 (d). 2202 3.2.1.2. Added subsection for engineered BitStrings as key novel 2203 aspect not found in BIER. (X). 2205 3.3. Added numbered list for components of BIER-TE forwarding 2206 plane (completing the comparable text from RFC8729 Section 4.2). 2208 3.4 Alvaro does not mind additional example, fixed bugs. 2210 3.5 Removed notion about using IGP BIER extensions for BIER-TE, 2211 such as BIFT address ranges. After -10 making use of BIFT 2212 clearer, it now looks to authors as if use of IGP extensions would 2213 not be beneficial, as long as we do need to use the BIER-TE 2214 controller, e.g. unlike in BIER, a BFR could not learn from the 2215 IGP information what traffic to send towards a particular BIFT-ID, 2216 but instead that is the core of what the controller needs to 2217 provide. 2219 4.2.2 Improved text to explain requirement to identify BIER-TE in 2220 the tunnel encap and compress description of use-cases (m). 2222 4.2.3 enhanced ECMP text (p). 2224 4.3. rewrote most of Encapsulation Considerations to better 2225 explain to Alvaros question re sharing or not sharing SD via BIER/ 2226 BIER-TE. Added reference to I-D.ietf-bier-non-mpls-bift-encoding 2227 as a very helpful example. (f). 2229 4.3 Renamed title to "...Co-Existence with BIER" as this is what 2230 it is about and to help finding it from abstract/intro ("co- 2231 exist") (j). 2233 4.4. Moved BIER-TE Forwarding Pseudocode here to coalesce text 2234 logically. Changed text to better compare with BIER pseudo 2235 forwarding code. Numerical list of how F-BM works for BIER-TE. 2236 Removed efficiency comparison with BIER (too difficult to provide 2237 sufficient justification, derails from focus of section) (j). 2239 4.6. (Requirements) Restructured: Removed notion of "basic" BIER- 2240 TE forwarding, simply referring to it now as "mandatory" BIER-TE 2241 forwarding. Cleaned up text to have requirements for different 2242 adjacencies in different paragraphs. (c). 2244 5. Created new main section "BIER-TE Controller operational 2245 considerations", coalesced old sections 4., 5., 7. into this new 2246 main section. No text changes. (k). 2248 5.1.9 Added new separate picture instead of referring to a picture 2249 later in text, adjusted text (r). 2251 5.3.2 Changed title to not include word "comparison" to avoid this 2252 being accounted against Alvaros concern about scattering 2253 comparison (IMHO text already has little comparison, so title was 2254 misleading) (h). 2256 co-authors internal review: 2258 4.4 Added xref to Figure 5. 2260 5.2.1 Duplicated ring picture, added visuals for described 2261 miswiring (s). 2263 5.2.2 replace "topology" with graph (wrong word). 2265 5.3.3 rewrote explanation of how to map BFR-id to SI:BP and assign 2266 them, clarified BFR-id is option. Retitled to better explain 2267 scope of section. 2269 5.3.4 Removed considerations in 5.3.4 for sharing BFR-id across 2270 BIER/BIER-TE (t), changed title to explain how BFIR/BIER-TE 2271 controller interactions need some form of identifying BFR but this 2272 does not have to be BFR-id. 2274 7. Added new security considerations (u). 2276 09: Incorporated fixes for feedback from Shepherd (Xuesong Geng). 2278 Added references for Bloom Filters and Rate Controlled Service 2279 Disciplines. 2281 1.1 Fixed numbering of example 1 topology explanation. Improved 2282 language on second example (less abbreviating to avoid confusion 2283 about meaning). 2285 1.2 Improved explanation of BIER-TE topology, fixed terminology of 2286 graphs (BIER-TE topology is a directed graph where the edges are 2287 the adjacencies). 2289 2.4 Fixed and amended routing underlay explanations: detailed why 2290 no need for BFER routing underlay routing protocol extensions, but 2291 potential to re-use BIER routing underlay routing protocol 2292 extensions for non-BFER related extensions. 2294 3.1 Added explanation for VRF and its use in adjacencies. 2296 08: Incorporated (with hopefully acceptable fixes) for Lou 2297 suggested section 2.5, TE considerations. 2299 Fixes are primarily to the point to a) emphasize that BIER-TE does 2300 not depend on the routing underlay unless forward_routed() 2301 adjacencies are used, and b) that the allocation and tracking of 2302 resources does not explicitly have to be tied to BPs, because they 2303 are just steering labels. Instead, it would ideally come from 2304 per-hop resource management that can be maintained only via local 2305 accounting in the controller. 2307 07: Further reworking text for Lou. 2309 Renamed BIER-PE to BIER-TE standing for "Tree Engineering" after 2310 votes from BIER WG. 2312 Removed section 1.1 (introduced by version 06) because not 2313 considered necessary in this doc by Lou (for framework doc). 2315 Added [RFC editor pls. remove] Section to explain name change to 2316 future reviewers. 2318 06: Concern by Lou Berger re. BIER-TE as full traffic engineering 2319 solution. 2321 Changed title "Traffic Engineering" to "Path Engineering" 2323 Added intro section of relationship BIER-PE to traffic 2324 engineering. 2326 Changed "traffic engineering" term in text" to "path engineering", 2327 where appropriate 2329 Other: 2331 Shortened "BIER-TE Controller Host" to "BIER-TE Controller". 2332 Fixed up all instances of controller to do this. 2334 05: Review Jeffrey Zhang. 2336 Part 2: 2338 4.3 added note about leaf-BFER being also a propery of routing 2339 setup. 2341 4.7 Added missing details from example to avoid confusion with 2342 routed adjacencies, also compressed explanatory text and better 2343 justification why seed is explicitly configured by controller. 2345 4.9 added section discussing generic reuse of BP methods. 2347 4.10 added section summarizing BP optimizations of section 4. 2349 6. Rewrote/compressed explanation of comparison BIER/BIER-TE 2350 forwarding difference. Explained benefit of BIER-TE per-BP 2351 forwarding being independent of forwarding for other BPs. 2353 Part 1: 2355 Explicitly ue forwarded_connected adjcency in ECMP adjcency 2356 examples to avoid confusion. 2358 4.3 Add picture as example for leav vs. non-leaf BFR in topology. 2359 Improved description. 2361 4.5 Exampe for traffic that can be broadcast -> for single BP in 2362 hub&spoke. 2364 4.8.1 Simplified example picture for routed adjacency, explanatory 2365 text. 2367 Review from Dirk Trossen: 2369 Fixed up explanation of ICC paper vs. bloom filter. 2371 04: spell check run. 2373 Addded remaining fixes for Sandys (Zhang Zheng) review: 2375 4.7 Enhance ECMP explanations: 2377 example ECMP algorithm, highlight that doc does not standardize 2378 ECMP algorithm. 2380 Review from Dirk Trossen: 2382 1. Added mentioning of prior work for traffic engineered paths 2383 with bloom filters. 2385 2. Changed title from layers to components and added "BIER-TE 2386 control plane" to "BIER-TE Controller" to make it clearer, what it 2387 does. 2389 2.2.3. Added reference to I-D.ietf-bier-multicast-http-response 2390 as an example solution. 2392 2.3. clarified sentence about resetting BPs before sending copies 2393 (also forgot to mention DNR here). 2395 3.4. Added text saying this section will be removed unless IESG 2396 review finds enough redeeming value in this example given how -03 2397 introduced section 1.1 with basic examples. 2399 7.2. Removed explicit numbers 20%/80% for number of topology bits 2400 in BIER-TE, replaced with more vague (high/low) description, 2401 because we do not have good reference material Added text saying 2402 this section will be removed unless IESG review finds enough 2403 redeeming value in this example given how -03 introduced section 2404 1.1 with basic examples. 2406 many typos fixed. Thanks a lot. 2408 03: Last call textual changes by authors to improve readability: 2410 removed Wolfgang Braun as co-authors (as requested). 2412 Improved abstract to be more explanatory. Removed mentioning of 2413 FRR (not concluded on so far). 2415 Added new text into Introduction section because the text was too 2416 difficult to jump into (too many forward pointers). This 2417 primarily consists of examples and the early introduction of the 2418 BIER-TE Topology concept enabled by these examples. 2420 Amended comparison to SR. 2422 Changed syntax from [VRF] to {VRF} to indicate its optional and to 2423 make idnits happy. 2425 Split references into normative / informative, added references. 2427 02: Refresh after IETF104 discussion: changed intended status back 2428 to standard. Reasoning: 2430 Tighter review of standards document == ensures arch will be 2431 better prepared for possible adoption by other WGs (e.g. DetNet) 2432 or std. bodies. 2434 Requirement against the degree of existing implementations is self 2435 defined by the WG. BIER WG seems to think it is not necessary to 2436 apply multiple interoperating implementations against an 2437 architecture level document at this time to make it qualify to go 2438 to standards track. Also, the levels of support introduced in -01 2439 rev. should allow all BIER forwarding engines to also be able to 2440 support the base level BIER-TE forwarding. 2442 01: Added note comparing BIER and SR to also hopefully clarify 2443 BIER-TE vs. BIER comparison re. SR. 2445 - added requirements section mandating only most basic BIER-TE 2446 forwarding features as MUST. 2448 - reworked comparison with BIER forwarding section to only 2449 summarize and point to pseudocode section. 2451 - reworked pseudocode section to have one pseudocode that mirrors 2452 the BIER forwarding pseudocode to make comparison easier and a 2453 second pseudocode that shows the complete set of BIER-TE 2454 forwarding options and simplification/optimization possible vs. 2455 BIER forwarding. Removed MyBitsOfInterest (was pure 2456 optimization). 2458 - Added captions to pictures. 2460 - Part of review feedback from Sandy (Zhang Zheng) integrated. 2462 00: Changed target state to experimental (WG conclusion), updated 2463 references, mod auth association. 2465 - Source now on http://www.github.com/toerless/bier-te-arch 2467 - Please open issues on the github for change/improvement requests 2468 to the document - in addition to posting them on the list 2469 (bier@ietf.). Thanks!. 2471 draft-eckert-bier-te-arch: 2473 06: Added overview of forwarding differences between BIER, BIER- 2474 TE. 2476 05: Author affiliation change only. 2478 04: Added comparison to Live-Live and BFIR to FRR section 2479 (Eckert). 2481 04: Removed FRR content into the new FRR draft [I-D.eckert-bier- 2482 te-frr] (Braun). 2484 - Linked FRR information to new draft in Overview/Introduction 2486 - Removed BTAFT/FRR from "Changes in the network topology" 2488 - Linked new draft in "Link/Node Failures and Recovery" 2490 - Removed FRR from "The BIER-TE Forwarding Layer" 2492 - Moved FRR section to new draft 2494 - Moved FRR parts of Pseudocode into new draft 2496 - Left only non FRR parts 2497 - removed FrrUpDown(..) and //FRR operations in 2498 ForwardBierTePacket(..) 2500 - New draft contains FrrUpDown(..) and ForwardBierTePacket(Packet) 2501 from bier-arch-03 2503 - Moved "BIER-TE and existing FRR to new draft 2505 - Moved "BIER-TE and Segment Routing" section one level up 2507 - Thus, removed "Further considerations" that only contained this 2508 section 2510 - Added Changes for version 04 2512 03: Updated the FRR section. Added examples for FRR key concepts. 2513 Added BIER-in-BIER tunneling as option for tunnels in backup 2514 paths. BIFT structure is expanded and contains an additional 2515 match field to support full node protection with BIER-TE FRR. 2517 03: Updated FRR section. Explanation how BIER-in-BIER 2518 encapsulation provides P2MP protection for node failures even 2519 though the routing underlay does not provide P2MP. 2521 02: Changed the definition of BIFT to be more inline with BIER. 2522 In revs. up to -01, the idea was that a BIFT has only entries for 2523 a single BitString, and every SI and sub-domain would be a 2524 separate BIFT. In BIER, each BIFT covers all SI. This is now 2525 also how we define it in BIER-TE. 2527 02: Added Section 5.3 to explain the use of SI, sub-domains and 2528 BFR-id in BIER-TE and to give an example how to efficiently assign 2529 bits for a large topology requiring multiple SI. 2531 02: Added further detailed for rings - how to support input from 2532 all ring nodes. 2534 01: Fixed BFIR -> BFER for section 4.3. 2536 01: Added explanation of SI, difference to BIER ECMP, 2537 consideration for Segment Routing, unicast FRR, considerations for 2538 encapsulation, explanations of BIER-TE Controller and CLI. 2540 00: Initial version. 2542 11. References 2544 11.1. Normative References 2546 [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate 2547 Requirement Levels", BCP 14, RFC 2119, 2548 DOI 10.17487/RFC2119, March 1997, 2549 . 2551 [RFC8174] Leiba, B., "Ambiguity of Uppercase vs Lowercase in RFC 2552 2119 Key Words", BCP 14, RFC 8174, DOI 10.17487/RFC8174, 2553 May 2017, . 2555 [RFC8279] Wijnands, IJ., Ed., Rosen, E., Ed., Dolganow, A., 2556 Przygienda, T., and S. Aldrin, "Multicast Using Bit Index 2557 Explicit Replication (BIER)", RFC 8279, 2558 DOI 10.17487/RFC8279, November 2017, 2559 . 2561 11.2. Informative References 2563 [Bloom70] Bloom, B., "Space/time trade-offs in hash coding with 2564 allowable errors", Comm. ACM 13(7):422-6, July 1970. 2566 [I-D.eckert-bier-te-frr] 2567 Eckert, T., Cauchie, G., Braun, W., and M. Menth, 2568 "Protection Methods for BIER-TE", draft-eckert-bier-te- 2569 frr-03 (work in progress), March 2018. 2571 [I-D.ietf-bier-multicast-http-response] 2572 Trossen, D., Rahman, A., Wang, C., and T. Eckert, 2573 "Applicability of BIER Multicast Overlay for Adaptive 2574 Streaming Services", draft-ietf-bier-multicast-http- 2575 response-05 (work in progress), January 2021. 2577 [I-D.ietf-bier-non-mpls-bift-encoding] 2578 Wijnands, I., Mishra, M., Xu, X., and H. Bidgoli, "An 2579 Optional Encoding of the BIFT-id Field in the non-MPLS 2580 BIER Encapsulation", draft-ietf-bier-non-mpls-bift- 2581 encoding-03 (work in progress), November 2020. 2583 [I-D.ietf-bier-te-yang] 2584 Zhang, Z., Wang, C., Chen, R., Hu, F., Sivakumar, M., and 2585 H. Chen, "A YANG data model for Traffic Engineering for 2586 Bit Index Explicit Replication (BIER-TE)", draft-ietf- 2587 bier-te-yang-02 (work in progress), August 2020. 2589 [I-D.ietf-roll-ccast] 2590 Bergmann, O., Bormann, C., Gerdes, S., and H. Chen, 2591 "Constrained-Cast: Source-Routed Multicast for RPL", 2592 draft-ietf-roll-ccast-01 (work in progress), October 2017. 2594 [I-D.ietf-teas-rfc3272bis] 2595 Farrel, A., "Overview and Principles of Internet Traffic 2596 Engineering", draft-ietf-teas-rfc3272bis-11 (work in 2597 progress), April 2021. 2599 [ICC] Reed, M., Al-Naday, M., Thomos, N., Trossen, D., 2600 Petropoulos, G., and S. Spirou, "Stateless multicast 2601 switching in software defined networks", IEEE 2602 International Conference on Communications (ICC), Kuala 2603 Lumpur, Malaysia, 2016, May 2016, 2604 . 2606 [RCSD94] Zhang, H. and D. Domenico, "Rate-Controlled Service 2607 Disciplines", Journal of High-Speed Networks, 1994, May 2608 1994, . 2610 [RFC4253] Ylonen, T. and C. Lonvick, Ed., "The Secure Shell (SSH) 2611 Transport Layer Protocol", RFC 4253, DOI 10.17487/RFC4253, 2612 January 2006, . 2614 [RFC4655] Farrel, A., Vasseur, J., and J. Ash, "A Path Computation 2615 Element (PCE)-Based Architecture", RFC 4655, 2616 DOI 10.17487/RFC4655, August 2006, 2617 . 2619 [RFC5440] Vasseur, JP., Ed. and JL. Le Roux, Ed., "Path Computation 2620 Element (PCE) Communication Protocol (PCEP)", RFC 5440, 2621 DOI 10.17487/RFC5440, March 2009, 2622 . 2624 [RFC6241] Enns, R., Ed., Bjorklund, M., Ed., Schoenwaelder, J., Ed., 2625 and A. Bierman, Ed., "Network Configuration Protocol 2626 (NETCONF)", RFC 6241, DOI 10.17487/RFC6241, June 2011, 2627 . 2629 [RFC7589] Badra, M., Luchuk, A., and J. Schoenwaelder, "Using the 2630 NETCONF Protocol over Transport Layer Security (TLS) with 2631 Mutual X.509 Authentication", RFC 7589, 2632 DOI 10.17487/RFC7589, June 2015, 2633 . 2635 [RFC7752] Gredler, H., Ed., Medved, J., Previdi, S., Farrel, A., and 2636 S. Ray, "North-Bound Distribution of Link-State and 2637 Traffic Engineering (TE) Information Using BGP", RFC 7752, 2638 DOI 10.17487/RFC7752, March 2016, 2639 . 2641 [RFC8253] Lopez, D., Gonzalez de Dios, O., Wu, Q., and D. Dhody, 2642 "PCEPS: Usage of TLS to Provide a Secure Transport for the 2643 Path Computation Element Communication Protocol (PCEP)", 2644 RFC 8253, DOI 10.17487/RFC8253, October 2017, 2645 . 2647 [RFC8296] Wijnands, IJ., Ed., Rosen, E., Ed., Dolganow, A., 2648 Tantsura, J., Aldrin, S., and I. Meilik, "Encapsulation 2649 for Bit Index Explicit Replication (BIER) in MPLS and Non- 2650 MPLS Networks", RFC 8296, DOI 10.17487/RFC8296, January 2651 2018, . 2653 [RFC8345] Clemm, A., Medved, J., Varga, R., Bahadur, N., 2654 Ananthakrishnan, H., and X. Liu, "A YANG Data Model for 2655 Network Topologies", RFC 8345, DOI 10.17487/RFC8345, March 2656 2018, . 2658 [RFC8401] Ginsberg, L., Ed., Przygienda, T., Aldrin, S., and Z. 2659 Zhang, "Bit Index Explicit Replication (BIER) Support via 2660 IS-IS", RFC 8401, DOI 10.17487/RFC8401, June 2018, 2661 . 2663 [RFC8444] Psenak, P., Ed., Kumar, N., Wijnands, IJ., Dolganow, A., 2664 Przygienda, T., Zhang, J., and S. Aldrin, "OSPFv2 2665 Extensions for Bit Index Explicit Replication (BIER)", 2666 RFC 8444, DOI 10.17487/RFC8444, November 2018, 2667 . 2669 Authors' Addresses 2671 Toerless Eckert (editor) 2672 Futurewei Technologies Inc. 2673 2330 Central Expy 2674 Santa Clara 95050 2675 USA 2677 Email: tte+ietf@cs.fau.de 2678 Gregory Cauchie 2679 Bouygues Telecom 2681 Email: GCAUCHIE@bouyguestelecom.fr 2683 Michael Menth 2684 University of Tuebingen 2686 Email: menth@uni-tuebingen.de