idnits 2.17.00 (12 Aug 2021) /tmp/idnits24617/draft-ietf-bier-te-arch-04.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- ** The abstract seems to contain references ([RFC8279]), which it shouldn't. Please replace those with straight textual mentions of the documents in question. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year -- The document date (October 22, 2019) is 942 days in the past. Is this intentional? Checking references for intended status: Proposed Standard ---------------------------------------------------------------------------- (See RFCs 3967 and 4897 for information about using normative references to lower-maturity documents in RFCs) -- Looks like a reference, but probably isn't: '2' on line 1107 -- Looks like a reference, but probably isn't: '1' on line 1117 == Missing Reference: 'SI' is mentioned on line 1154, but not defined == Missing Reference: 'I' is mentioned on line 1161, but not defined == Missing Reference: 'VRF' is mentioned on line 1590, but not defined == Outdated reference: A later version (-06) exists of draft-ietf-bier-multicast-http-response-01 Summary: 1 error (**), 0 flaws (~~), 5 warnings (==), 3 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 Network Working Group T. Eckert, Ed. 3 Internet-Draft Futurewei 4 Intended status: Standards Track G. Cauchie 5 Expires: April 24, 2020 Bouygues Telecom 6 M. Menth 7 University of Tuebingen 8 October 22, 2019 10 Traffic Engineering for Bit Index Explicit Replication (BIER-TE) 11 draft-ietf-bier-te-arch-04 13 Abstract 15 This memo introduces per-packet stateless strict and loose path 16 engineered replication and forwarding for Bit Index Explicit 17 Replication packets ([RFC8279]). This is called BIER-TE. 19 BIER-TE leverages the BIER architecture ([RFC8279]) and extends it 20 with a new semantic for bits in the bitstring. BIER-TE can leverage 21 BIER forwarding engines with little or no changes. 23 In BIER, the BitPositions (BP) of the packets bitstring indicate BIER 24 Forwarding Egress Routers (BFER), and hop-by-hop forwarding uses a 25 Routing Underlay such as an IGP. 27 In BIER-TE, BitPositions indicate adjacencies. The BIFT of each BFR 28 are only populated with BPs that are adjacent to the BFR in the BIER- 29 TE topology. The BIER-TE topology can consist of layer 2 or remote 30 (route) adjacencies. The BFR then replicates and forwards BIER 31 packets to those adjacencies. This results in the aforementioned 32 strict and loose path forwarding. 34 BIER-TE can co-exist with BIER forwarding in the same domain, for 35 example by using separate sub-domains. In the absence of routed 36 adjacencies, BIER-TE does not require a BIER routing underlay, and 37 can then be operated without requiring an IGP routing protocol. 39 BIER-TE operates without explicit in-network tree-building and 40 carries the multicast distribution tree in the packet header. It can 41 therefore be a good fit to support multicast path steering in Segment 42 Routing (SR) networks. 44 Status of This Memo 46 This Internet-Draft is submitted in full conformance with the 47 provisions of BCP 78 and BCP 79. 49 Internet-Drafts are working documents of the Internet Engineering 50 Task Force (IETF). Note that other groups may also distribute 51 working documents as Internet-Drafts. The list of current Internet- 52 Drafts is at https://datatracker.ietf.org/drafts/current/. 54 Internet-Drafts are draft documents valid for a maximum of six months 55 and may be updated, replaced, or obsoleted by other documents at any 56 time. It is inappropriate to use Internet-Drafts as reference 57 material or to cite them other than as "work in progress." 59 This Internet-Draft will expire on April 24, 2020. 61 Copyright Notice 63 Copyright (c) 2019 IETF Trust and the persons identified as the 64 document authors. All rights reserved. 66 This document is subject to BCP 78 and the IETF Trust's Legal 67 Provisions Relating to IETF Documents 68 (https://trustee.ietf.org/license-info) in effect on the date of 69 publication of this document. Please review these documents 70 carefully, as they describe your rights and restrictions with respect 71 to this document. Code Components extracted from this document must 72 include Simplified BSD License text as described in Section 4.e of 73 the Trust Legal Provisions and are provided without warranty as 74 described in the Simplified BSD License. 76 Table of Contents 78 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 3 79 1.1. Basic Examples . . . . . . . . . . . . . . . . . . . . . 4 80 1.2. BIER-TE Topology and adjacencies . . . . . . . . . . . . 7 81 1.3. Comparison with BIER . . . . . . . . . . . . . . . . . . 8 82 1.4. Requirements Language . . . . . . . . . . . . . . . . . . 8 83 2. Components . . . . . . . . . . . . . . . . . . . . . . . . . 8 84 2.1. The Multicast Flow Overlay . . . . . . . . . . . . . . . 9 85 2.2. The BIER-TE Controller Host . . . . . . . . . . . . . . . 9 86 2.2.1. Assignment of BitPositions to adjacencies of the 87 network topology . . . . . . . . . . . . . . . . . . 10 88 2.2.2. Changes in the network topology . . . . . . . . . . . 10 89 2.2.3. Set up per-multicast flow BIER-TE state . . . . . . . 10 90 2.2.4. Link/Node Failures and Recovery . . . . . . . . . . . 11 91 2.3. The BIER-TE Forwarding Layer . . . . . . . . . . . . . . 11 92 2.4. The Routing Underlay . . . . . . . . . . . . . . . . . . 11 93 3. BIER-TE Forwarding . . . . . . . . . . . . . . . . . . . . . 11 94 3.1. The Bit Index Forwarding Table (BIFT) . . . . . . . . . . 11 95 3.2. Adjacency Types . . . . . . . . . . . . . . . . . . . . . 13 96 3.2.1. Forward Connected . . . . . . . . . . . . . . . . . . 13 97 3.2.2. Forward Routed . . . . . . . . . . . . . . . . . . . 13 98 3.2.3. ECMP . . . . . . . . . . . . . . . . . . . . . . . . 13 99 3.2.4. Local Decap . . . . . . . . . . . . . . . . . . . . . 14 100 3.3. Encapsulation considerations . . . . . . . . . . . . . . 14 101 3.4. Basic BIER-TE Forwarding Example . . . . . . . . . . . . 14 102 3.5. Forwarding comparison with BIER . . . . . . . . . . . . . 17 103 3.6. Requirements . . . . . . . . . . . . . . . . . . . . . . 17 104 4. BIER-TE Controller Host BitPosition Assignments . . . . . . . 18 105 4.1. P2P Links . . . . . . . . . . . . . . . . . . . . . . . . 18 106 4.2. BFER . . . . . . . . . . . . . . . . . . . . . . . . . . 18 107 4.3. Leaf BFERs . . . . . . . . . . . . . . . . . . . . . . . 18 108 4.4. LANs . . . . . . . . . . . . . . . . . . . . . . . . . . 19 109 4.5. Hub and Spoke . . . . . . . . . . . . . . . . . . . . . . 19 110 4.6. Rings . . . . . . . . . . . . . . . . . . . . . . . . . . 19 111 4.7. Equal Cost MultiPath (ECMP) . . . . . . . . . . . . . . . 20 112 4.8. Routed adjacencies . . . . . . . . . . . . . . . . . . . 23 113 4.8.1. Reducing BitPositions . . . . . . . . . . . . . . . . 23 114 4.8.2. Supporting nodes without BIER-TE . . . . . . . . . . 23 115 5. Avoiding loops and duplicates . . . . . . . . . . . . . . . . 24 116 5.1. Loops . . . . . . . . . . . . . . . . . . . . . . . . . . 24 117 5.2. Duplicates . . . . . . . . . . . . . . . . . . . . . . . 24 118 6. BIER-TE Forwarding Pseudocode . . . . . . . . . . . . . . . . 24 119 7. Managing SI, subdomains and BFR-ids . . . . . . . . . . . . . 27 120 7.1. Why SI and sub-domains . . . . . . . . . . . . . . . . . 28 121 7.2. Bit assignment comparison BIER and BIER-TE . . . . . . . 29 122 7.3. Using BFR-id with BIER-TE . . . . . . . . . . . . . . . . 29 123 7.4. Assigning BFR-ids for BIER-TE . . . . . . . . . . . . . . 30 124 7.5. Example bit allocations . . . . . . . . . . . . . . . . . 31 125 7.5.1. With BIER . . . . . . . . . . . . . . . . . . . . . . 31 126 7.5.2. With BIER-TE . . . . . . . . . . . . . . . . . . . . 32 127 7.6. Summary . . . . . . . . . . . . . . . . . . . . . . . . . 33 128 8. BIER-TE and Segment Routing (SR) . . . . . . . . . . . . . . 33 129 9. Security Considerations . . . . . . . . . . . . . . . . . . . 34 130 10. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 35 131 11. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . 35 132 12. Change log [RFC Editor: Please remove] . . . . . . . . . . . 35 133 13. References . . . . . . . . . . . . . . . . . . . . . . . . . 39 134 13.1. Normative References . . . . . . . . . . . . . . . . . . 39 135 13.2. Informative References . . . . . . . . . . . . . . . . . 39 136 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 40 138 1. Introduction 140 BIER-TE shares architecture, terminology and packet formats with BIER 141 as described in [RFC8279] and [RFC8296]. This document describes 142 BIER-TE in the expectation that the reader is familiar with these two 143 documents. 145 In BIER-TE, BitPositions (BP) indicate adjacencies. The BIFT of each 146 BFR is only populated with BP that are adjacent to the BFR in the 147 BIER-TE Topology. Other BPs are left without adjacency. The BFR 148 replicate and forwards BIER packets to adjacent BPs that are set in 149 the packet. BPs are normally also reset upon forwarding to avoid 150 duplicates and loops. This is detailed further below. 152 Note that related work [ICC], [I-D.ietf-roll-ccast] uses bloom 153 filters to represent leaves or edges of the intended delivery tree. 154 Bloom filters can support larger trees with fewer addressing bits, 155 but they introduce the heuristic risk of false positives and cannot 156 reset bits in the bitstring during forwarding to avoid loops. For 157 these reasons, BIER-TE does not use bloom filters, but explicit 158 bitstrings like BIER. 160 1.1. Basic Examples 162 BIER-TE forwarding is best introduced with simple examples. 164 BIER-TE Topology: 166 Diagram: 168 p5 p6 169 --- BFR3 --- 170 p3/ p13 \p7 171 BFR1 ---- BFR2 BFR5 ----- BFR6 172 p1 p2 p4\ p14 /p10 p11 p12 173 --- BFR4 --- 174 p8 p9 176 (simplified) BIER-TE Bit Index Forwarding Tables (BIFT): 178 BFR1: p1 -> local_decap 179 p2 -> forward_connected to BFR2 181 BFR2: p1 -> forward_connected to BFR1 182 p5 -> forward_connected to BFR3 183 p8 -> forward_connected to BFR4 185 BFR3: p3 -> forward_connected to BFR2 186 p7 -> forward_connected to BFR5 187 p13 -> local_decap 189 BFR4: p4 -> forward_connected to BFR2 190 p10 -> forward_connected to BFR5 191 p14 -> local_decap 193 BFR5: p6 -> forward_connected to BFR3 194 p9 -> forward_connected to BFR4 195 p12 -> forward_connected to BFR6 197 BFR6: p11 -> forward_connected to BFR5 198 p12 -> local_decap 200 Figure 1: BIER-TE basic example 202 Consider the simple network in the above BIER-TE overview example 203 picture with 6 BFRs. p1...p14 are the BitPositions (BP) used. All 204 BFRs can act as ingress BFR (BFIR), BFR1, BFR3, BFR4 and BFR6 can 205 also be egress BFR (BFER). Forward_connected is the name for 206 adjacencies that are representing subnet adjacencies of the network. 207 Local_decap is the name of the adjacency to decapsulate BIER-TE 208 packets and pass their payload to higher layer processing. 210 Assume a packet from BFR1 should be sent via BFR4 to BFR6. This 211 requires a bitstring (p2,p8,p10,p12). When this packet is examined 212 by BIER-TE on BFR1, the only BitPosition from the bitstring that is 213 also set in the BIFT is p2. This will cause BFR1 to send the only 214 copy of the packet to BFR2. Similarly, BFR2 will forward to BFR4 215 because of p8, BFR4 to BFR5 because of p10 and BFR5 to BFR6 because 216 of p12. p12 also makes BFR6 receive and decapsulate the packet. 218 To send in addition to BFR6 via BFR4 also a copy to BFR3, the 219 bitstring needs to be (p2,p5,p8,p10,p12,p13). When this packet is 220 examined by BFR2, p5 causes one copy to be sent to BFR3 and p8 one 221 copy to BFR4. When BFR3 receives the packet, p13 will cause it to 222 receive and decapsulate the packet. 224 If instead the bitstring was (p2,p6,p8,p10,p12,p13), the packet would 225 be copied by BFR5 towards BFR3 because p6 instead of BFR2 to BFR5 226 because of p6 in the prior case. This is showing the ability of the 227 shown BIER-TE Topology to make the traffic pass across any possible 228 path and be replicated where desired. 230 BIER-TE has various options to minimize BP assignments, many of which 231 are based on assumptions about the required multicast traffic paths 232 and bandwidth consumption in the network. 234 The following picture shows a modified example, in which Rtr2 and 235 Rtr5 are assumed not to support BIER-TE, so traffic has to be unicast 236 encapsulated across them. Unicast tunneling of BIER-TE packets can 237 leverage any feasible mechanism such as MPLS or IP, these 238 encapsulations are out of scope of this document. To emphasize non- 239 native forwarding of BIER-TE packets, these adjacencies are called 240 "forward_routed", but otherwise there is no difference in their 241 processing over the aforementioned "forward_connected" adjacencies. 243 In addition, bits are saved in the following example by assuming that 244 BFR1 only needs to be BFIR but not BFER or transit BFR. 246 BIER-TE Topology: 248 Diagram: 250 p1 p3 p7 251 ....> BFR3 <.... p5 252 ........ ........> 253 BFR1 (Rtr2) (Rtr5) BFR6 254 ........ ........> 255 ....> BFR4 <.... p6 256 p2 p4 p8 258 (simplified) BIER-TE Bit Index Forwarding Tables (BIFT): 260 BFR1: p1 -> forward_routed to BFR3 261 p2 -> forward_routed to BFR4 263 BFR3: p3 -> local_decap 264 p5 -> forward_routed to BFR6 266 BFR4: p4 -> local_decap 267 p6 -> forward_routed to BFR6 269 BFR6: p5 -> local_decap 270 p6 -> local_decap 271 p7 -> forward_routed to BFR3 272 p8 -> forward_routed to BFR4 274 Figure 2: BIER-TE basic overlay example 276 To send a BIER-TE packet from BFR1 via BFR3 to BFR6, the bitstring is 277 (p1,p5). From BFR1 via BFR4 to BFR6 it is (p2,p6). A packet from 278 BFR1 to BFR3,BFR4 and BFR6 can use (p1,p2,p3,p4,p5) or 279 (p1,p2,p3,p4,p6), or via BFR6 (p2,p3,p4,p6,p7) or (p1.p3,p4,p5,p8). 281 1.2. BIER-TE Topology and adjacencies 283 The key new component in BIER-TE to control where replication can or 284 should happens and how to minimize the required BP for segments is - 285 as shown in these two examples - the BIER-TE topology. 287 The BIER-TE Topology effectively consists of the BIFT of all the BFR 288 and can also be expressed in a diagram as a graph where the edges are 289 the adjacencies between the BFR. Adjacencies are naturally 290 unidirectional. BP can be reused across multiple adjacencies as long 291 as this does not lead to undesired duplicates or loops as explained 292 further down in the text. 294 If the BIER-TE topology represents the underlying (layer 2) topology 295 of the network, this is called "native" BIER-TE as shown in the first 296 example. This can be freely mixed with "overlay" BIER-TE, in 297 "forward_routed" adjacencies are used. 299 1.3. Comparison with BIER 301 The key differences over BIER are: 303 o BIER-TE replaces in-network autonomous path calculation by 304 explicit paths calculated off-path by the BIER-TE controller host. 306 o In BIER-TE every BitPosition of the BitString of a BIER-TE packet 307 indicates one or more adjacencies - instead of a BFER as in BIER. 309 o BIER-TE in each BFR has no routing table but only a BIER-TE 310 Forwarding Table (BIFT) indexed by SI:BitPosition and populated 311 with only those adjacencies to which the BFR should replicate 312 packets to. 314 BIER-TE headers use the same format as BIER headers. 316 BIER-TE forwarding does not require/use the BFIR-ID. The BFIR-ID can 317 still be useful though for coordinated BFIR/BFER functions, such as 318 the context for upstream assigned labels for MPLS payloads in MVPN 319 over BIER-TE. 321 If the BIER-TE domain is also running BIER, then the BFIR-ID in BIER- 322 TE packets can be set to the same BFIR-ID as used with BIER packets. 324 If the BIER-TE domain is not running full BIER or does not want to 325 reduce the need to allocate bits in BIER bitstrings for BFIR-ID 326 values, then the allocation of BFIR-ID values in BIER-TE packets can 327 be done through other mechanisms outside the scope of this document, 328 as long as this is appropriately agreed upon between all BFIR/BFER. 330 1.4. Requirements Language 332 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 333 "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this 334 document are to be interpreted as described in RFC 2119 [RFC2119]. 336 2. Components 338 End to end BIER-TE operations consists of four mayor components: The 339 "Multicast Flow Overlay", the "BIER-TE control plane" consisting of 340 the "BIER-TE Controller Host" and its signaling channels to the BFR, 341 the "Routing Underlay" and the "BIER-TE forwarding layer". The Bier- 342 TE Controller Host is the new architectural component in BIER-TE 343 compared to BIER. 345 Picture 2: Components of BIER-TE 347 <------BGP/PIM-----> 348 |<-IGMP/PIM-> multicast flow <-PIM/IGMP->| 349 overlay 351 [BIER-TE Controller Host] <=> [BIER-TE Topology] 352 BIER-TE control plane 353 ^ ^ ^ 354 / | \ BIER-TE control protocol 355 | | | e.g. Netconf/Restconf/Yang 356 v v v 357 Src -> Rtr1 -> BFIR-----BFR-----BFER -> Rtr2 -> Rcvr 359 |<----------------->| 360 BIER-TE forwarding layer 362 |<- BIER-TE domain->| 364 |<--------------------->| 365 Routing underlay 367 Figure 3: BIER-TE architecture 369 2.1. The Multicast Flow Overlay 371 The Multicast Flow Overlay operates as in BIER. See [RFC8279]. 372 Instead of interacting with the BIER forwarding layer (as in BIER), 373 it interacts with the BIER-TE Controller Host. 375 2.2. The BIER-TE Controller Host 377 The BIER-TE controller host is representing the control plane of 378 BIER-TE. It communicates two sets of information with BFRs: 380 During initial provisioning or modifications of the network topology, 381 the controller discovers the network topology and creates the BIER-TE 382 topology from it: determine which adjacencies are required/desired 383 and assign BitPositions to them. Then it signals the resulting of 384 BitPositions and their adjacencies to each BFR to set up their BIER- 385 TE BIFTs. 387 During day-to-day operations of the network, the controller signals 388 to BFIRs what multicast flows are mapped to what BitStrings. 390 Communications between the BIER-TE controller host to BFRs is ideally 391 via standardized protocols and data-models such as Netconf/Restconf/ 392 Yang. This is currently outside the scope of this document. Vendor- 393 specific CLI on the BFRs is also a possible stopgap option (as in 394 many other SDN solutions lacking definition of standardized data 395 model). 397 For simplicity, the procedures of the BIER-TE controller host are 398 described in this document as if it is a single, centralized 399 automated entity, such as an SDN controller. It could equally be an 400 operator setting up CLI on the BFRs. Distribution of the functions 401 of the BIER-TE controller host is currently outside the scope of this 402 document. 404 2.2.1. Assignment of BitPositions to adjacencies of the network 405 topology 407 The BIER-TE controller host tracks the BFR topology of the BIER-TE 408 domain. It determines what adjacencies require BitPositions so that 409 BIER-TE explicit paths can be built through them as desired by 410 operator policy. 412 The controller then pushes the BitPositions/adjacencies to the BIFT 413 of the BFRs, populating only those SI:BitPositions to the BIFT of 414 each BFR to which that BFR should be able to send packets to - 415 adjacencies connecting to this BFR. 417 2.2.2. Changes in the network topology 419 If the network topology changes (not failure based) so that 420 adjacencies that are assigned to BitPositions are no longer needed, 421 the controller can re-use those BitPositions for new adjacencies. 422 First, these BitPositions need to be removed from any BFIR flow state 423 and BFR BIFT state, then they can be repopulated, first into BIFT and 424 then into the BFIR. 426 2.2.3. Set up per-multicast flow BIER-TE state 428 The BIER-TE controller host interacts with the multicast flow overlay 429 to determine what multicast flow needs to be sent by a BFIR to which 430 set of BFER. It calculates the desired distribution tree across the 431 BIER-TE domain based on algorithms outside the scope of this document 432 (e.g. CSFP, Steiner Tree, ...). It then pushes the calculated 433 BitString into the BFIR. 435 See [I-D.ietf-bier-multicast-http-response] for a solution describing 436 this interaction. 438 2.2.4. Link/Node Failures and Recovery 440 When link or nodes fail or recover in the topology, BIER-TE can 441 quickly respond with the optional FRR procedures described in [I- 442 D.eckert-bier-te-frr]. It can also more slowly react by 443 recalculating the BitStrings of affected multicast flows. This 444 reaction is slower than the FRR procedure because the controller 445 needs to receive link/node up/down indications, recalculate the 446 desired BitStrings and push them down into the BFIRs. With FRR, this 447 is all performed locally on a BFR receiving the adjacency up/down 448 notification. 450 2.3. The BIER-TE Forwarding Layer 452 When the BIER-TE Forwarding Layer receives a packet, it simply looks 453 up the BitPositions that are set in the BitString of the packet in 454 the Bit Index Forwarding Table (BIFT) that was populated by the BIER- 455 TE controller host. For every BP that is set in the BitString, and 456 that has one or more adjacencies in the BIFT, a copy is made 457 according to the type of adjacencies for that BP in the BIFT. Before 458 sending any copy, the BFR resets all BP in the BitString of the 459 packet for which the BFR has one or more adjacencies in the BIFT, 460 except when the adjacency indicates "DoNotReset" (DNR, see 461 Section 3.2.1). This is done to inhibit that packets can loop. 463 2.4. The Routing Underlay 465 BIER-TE is sending BIER packets to directly connected BIER-TE 466 neighbors as L2 (unicasted) BIER packets without requiring a routing 467 underlay. BIER-TE forwarding uses the Routing underlay for 468 forward_routed adjacencies which copy BIER-TE packets to not- 469 directly-connected BFRs (see below for adjacency definitions). 471 If the BFR intends to support FRR for BIER-TE, then the BIER-TE 472 forwarding plane needs to receive fast adjacency up/down 473 notifications: Link up/down or neighbor up/down, e.g. from BFD. 474 Providing these notifications is considered to be part of the routing 475 underlay in this document. 477 3. BIER-TE Forwarding 479 3.1. The Bit Index Forwarding Table (BIFT) 481 The Bit Index Forwarding Table (BIFT) exists in every BFR. For every 482 subdomain in use, it is a table indexed by SI:BitPosition and is 483 populated by the BIER-TE control plane. Each index can be empty or 484 contain a list of one or more adjacencies. 486 BIER-TE can support multiple subdomains like BIER. Each one with a 487 separate BIFT 489 In the BIER architecture, indices into the BIFT are explained to be 490 both BFR-id and SI:BitString (BitPosition). This is because there is 491 a 1:1 relationship between BFR-id and SI:BitString - every bit in 492 every SI is/can be assigned to a BFIR/BFER. In BIER-TE there are 493 more bits used in each BitString than there are BFIR/BFER assigned to 494 the bitstring. This is because of the bits required to express the 495 (traffic engineered) path through the topology. The BIER-TE 496 forwarding definitions do therefore not use the term BFR-id at all. 497 Instead, BFR-ids are only used as required by routing underlay, flow 498 overlay of BIER headers. Please refer to Section 7 for explanations 499 how to deal with SI, subdomains and BFR-id in BIER-TE. 501 ------------------------------------------------------------------ 502 | Index: | Adjacencies: | 503 | SI:BitPosition | or one or more per entry | 504 ================================================================== 505 | 0:1 | forward_connected(interface,neighbor,DNR) | 506 ------------------------------------------------------------------ 507 | 0:2 | forward_connected(interface,neighbor,DNR) | 508 | | forward_connected(interface,neighbor,DNR) | 509 ------------------------------------------------------------------ 510 | 0:3 | local_decap({VRF}) | 511 ------------------------------------------------------------------ 512 | 0:4 | forward_routed({VRF,}l3-neighbor) | 513 ------------------------------------------------------------------ 514 | 0:5 | | 515 ------------------------------------------------------------------ 516 | 0:6 | ECMP({adjacency1,...adjacencyN}, seed) | 517 ------------------------------------------------------------------ 518 ... 519 | BitStringLength | ... | 520 ------------------------------------------------------------------ 521 Bit Index Forwarding Table 523 Figure 4: BIFT adjacencies 525 The BIFT is programmed into the data plane of BFRs by the BIER-TE 526 controller host and used to forward packets, according to the rules 527 specified in the BIER-TE Forwarding Procedures. 529 Adjacencies for the same BP when populated in more than one BFR by 530 the controller does not have to have the same adjacencies. This is 531 up to the controller. BPs for p2p links are one case (see below). 533 3.2. Adjacency Types 535 3.2.1. Forward Connected 537 A "forward_connected" adjacency is towards a directly connected BFR 538 neighbor using an interface address of that BFR on the connecting 539 interface. A forward_connected adjacency does not route packets but 540 only L2 forwards them to the neighbor. 542 Packets sent to an adjacency with "DoNotReset" (DNR) set in the BIFT 543 will not have the BitPosition for that adjacency reset when the BFR 544 creates a copy for it. The BitPosition will still be reset for 545 copies of the packet made towards other adjacencies. This can be 546 used for example in ring topologies as explained below. 548 3.2.2. Forward Routed 550 A "forward_routed" adjacency is an adjacency towards a BFR that is 551 not a forward_connected adjacency: towards a loopback address of a 552 BFR or towards an interface address that is non-directly connected. 553 Forward_routed packets are forwarded via the Routing Underlay. 555 If the Routing Underlay has multiple paths for a forward_routed 556 adjacency, it will perform ECMP independent of BIER-TE for packets 557 forwarded across a forward_routed adjacency. 559 If the Routing Underlay has FRR, it will perform FRR independent of 560 BIER-TE for packets forwarded across a forward_routed adjacency. 562 3.2.3. ECMP 564 The ECMP mechanisms in BIER are tied to the BIER BIFT and are 565 therefore not directly useable with BIER-TE. The following 566 procedures describe ECMP for BIER-TE that we consider to be 567 lightweight but also well manageable. It leverages the existing 568 entropy parameter in the BIER header to keep packets of the flows on 569 the same path and it introduces a "seed" parameter to allow 570 engineering traffic to be polarized or randomized across multiple 571 hops. 573 An "Equal Cost Multipath" (ECMP) adjacency has a list of two or more 574 adjacencies included in it. It copies the BIER-TE to one of those 575 adjacencies based on the ECMP hash calculation. The BIER-TE ECMP 576 hash algorithm must select the same adjacency from that list for all 577 packets with the same "entropy" value in the BIER-TE header if the 578 same number of adjacencies and same seed are given as parameters. 579 Further use of the seed parameter is explained below. 581 3.2.4. Local Decap 583 A "local_decap" adjacency passes a copy of the payload of the BIER-TE 584 packet to the packets NextProto within the BFR (IPv4/IPv6, 585 Ethernet,...). A local_decap adjacency turns the BFR into a BFER for 586 matching packets. Local_decap adjacencies require the BFER to 587 support routing or switching for NextProto to determine how to 588 further process the packet. 590 3.3. Encapsulation considerations 592 Specifications for BIER-TE encapsulation are outside the scope of 593 this document. This section gives explanations and guidelines. 595 Because a BFR needs to interpret the BitString of a BIER-TE packet 596 differently from a BIER packet, it is necessary to distinguish BIER 597 from BIER-TE packets. This is subject to definitions in BIER 598 encapsulation specifications. 600 MPLS encapsulation [RFC8296] for example assigns one label by which 601 BFRs recognizes BIER packets for every (SI,subdomain) combination. 602 If it is desirable that every subdomain can forward only BIER or 603 BIER-TE packets, then the label allocation could stay the same, and 604 only the forwarding model (BIER/BIER-TE) would have to be defined per 605 subdomain. If it is desirable to support both BIER and BIER-TE 606 forwarding in the same subdomain, then additional labels would need 607 to be assigned for BIER-TE forwarding. 609 "forward_routed" requires an encapsulation permitting to unicast 610 BIER-TE packets to a specific interface address on a target BFR. 611 With MPLS encapsulation, this can simply be done via a label stack 612 with that addresses label as the top label - followed by the label 613 assigned to (SI,subdomain) - and if necessary (see above) BIER-TE. 614 With non-MPLS encapsulation, some form of IP tunneling (IP in IP, 615 LISP, GRE) would be required. 617 The encapsulation used for "forward_routed" adjacencies can equally 618 support existing advanced adjacency information such as "loose source 619 routes" via e.g. MPLS label stacks or appropriate header extensions 620 (e.g. for IPv6). 622 3.4. Basic BIER-TE Forwarding Example 624 [RFC Editor: remove this section.] 626 THIS SECTION TO BE REMOVED IN RFC BECAUSE IT WAS SUPERCEEDED BY 627 SECTION 1.1 EXAMPLE - UNLESS REVIEWERS CHIME IN AND EXPRESS DESIRE TO 628 KEEP THIS ADDITIONAL EXAMPLE SECTION. 630 Step by step example of basic BIER-TE forwarding. This does not use 631 ECMP or forward_routed adjacencies nor does it try to minimize the 632 number of required BitPositions for the topology. 634 [Bier-Te Controller Host] 635 / | \ 636 v v v 638 | p13 p1 | 639 +- BFIR2 --+ | 640 | | p2 p6 | LAN2 641 | +-- BFR3 --+ | 642 | | | p7 p11 | 643 Src -+ +-- BFER1 --+ 644 | | p3 p8 | | 645 | +-- BFR4 --+ +-- Rcv1 646 | | | | 647 | | 648 | p14 p4 | 649 +- BFIR1 --+ | 650 | +-- BFR5 --+ p10 p12 | 651 LAN1 | p5 p9 +-- BFER2 --+ 652 | +-- Rcv2 653 | 654 LAN3 656 IP |..... BIER-TE network......| IP 658 Figure 5: BIER-TE Forwarding Example 660 pXX indicate the BitPositions number assigned by the BIER-TE 661 controller host to adjacencies in the BIER-TE topology. For example, 662 p9 is the adjacency towards BFR5 on the LAN connecting to BFER2. 664 BIFT BFIR2: 665 p13: local_decap() 666 p2: forward_connected(BFR3) 668 BIFT BFR3: 669 p1: forward_connected(BFIR2) 670 p7: forward_connected(BFER1) 671 p8: forward_connected(BFR4) 673 BIFT BFER1: 674 p11: local_decap() 675 p6: forward_connected(BFR3) 676 p8: forward_connected(BFR4) 678 Figure 6: BIER-TE Forwarding Example Adjacencies 680 ...and so on. 682 For example, we assume that some multicast traffic seen on LAN1 needs 683 to be sent via BIER-TE by BFIR2 towards Rcv1 and Rcv2. The 684 controller determines it wants it to pass this traffic across the 685 following paths: 687 -> BFER1 ---------------> Rcv1 688 BFIR2 -> BFR3 689 -> BFR4 -> BFR5 -> BFER2 -> Rcv2 691 Figure 7: BIER-TE Forwarding Example Paths 693 These paths equal to the following BitString: p2, p5, p7, p8, p10, 694 p11, p12. 696 This BitString is assigned by BFIR2 to the example multicast traffic 697 received from LAN1. 699 Then BFIR2 forwards this multicast traffic with BIER-TE based on that 700 BitString. The BIFT of BFIR2 has only p2 and p13 populated. Only p2 701 is in the BitString and this is an adjacency towards BFR3. BFIR2 702 therefore resets p2 in the BitString and sends a copy towards BFR2. 704 BFR3 sees a BitString of p5,p7,p8,p10,p11,p12. It is only interested 705 in p1,p7,p8. It creates a copy of the packet to BFER1 (due to p7) 706 and one to BFR4 (due to p8). It resets p7, p8 before sending. 708 BFER1 sees a BitString of p5,p10,p11,p12. It is only interested in 709 p6,p7,p8,p11 and therefore considers only p11. p11 is a "local_decap" 710 adjacency installed by the BIER-TE controller host because BFER1 711 should pass packets to IP multicast. The local_decap adjacency 712 instructs BFER1 to create a copy, decapsulate it from the BIER header 713 and pass it on to the NextProtocol, in this example IP multicast. IP 714 multicast will then forward the packet out to LAN2 because it did 715 receive PIM or IGMP joins on LAN2 for the traffic. 717 Further processing of the packet in BFR4, BFR5 and BFER2 accordingly. 719 3.5. Forwarding comparison with BIER 721 Forwarding of BIER-TE is designed to allow common forwarding hardware 722 with BIER. In fact, one of the main goals of this document is to 723 encourage the building of forwarding hardware that cannot only 724 support BIER, but also BIER-TE - to allow experimentation with BIER- 725 TE and support building of BIER-TE control plane code. 727 The pseudocode in Section 6 shows how existing BIER/BIFT forwarding 728 can be amended to support basic BIER-TE forwarding, by using BIER 729 BIFT's F-BM. Only the masking of bits due to avoid duplicates must 730 be skipped when forwarding is for BIER-TE. 732 Whether to use BIER or BIER-TE forwarding can simply be a configured 733 choice per subdomain and accordingly be set up by a BIER-TE 734 controller host. The BIER packet encapsulation [RFC8296] too can be 735 reused without changes except that the currently defined BIER-TE ECMP 736 adjacency does not leverage the entropy field so that field would be 737 unused when BIER-TE forwarding is used. 739 3.6. Requirements 741 Basic BIER-TE forwarding MUST support to configure Subdomains to use 742 basic BIER-TE forwarding rules (instead of BIER). With basic BIER-TE 743 forwarding, every bit MUST support to have zero or one adjacency. It 744 MUST support the adjacency types forward_connected without DNR flag, 745 forward_routed and local_decap. All other BIER-TE forwarding 746 features are optional. These basic BIER-TE requirements make BIER-TE 747 forwarding exactly the same as BIER forwarding with the exception of 748 skipping the aforementioned F-BM masking on egress. 750 BIER-TE forwarding SHOULD support the DNR flag, as this is highly 751 useful to save bits in rings (see Section 4.6). 753 BIER-TE forwarding MAY support more than one adjacency on a bit and 754 ECMP adjacencies. The importance of ECMP adjacencies is unclear when 755 traffic engineering is used because it may be more desirable to 756 explicitly steer traffic across non-ECMP paths to make per-path 757 traffic calculation easier for controllers. Having more than one 758 adjacency for a bit allows further savings of bits in hub&spoke 759 scenarios, but unlike rings it is less "natural" to flood traffic 760 across multiple links unconditional. Both ECMP and multiple 761 adjacencies are forwarding plane features that should be possible to 762 support later when needed as they do not impact the basic BIER-TE 763 replication loop. This is true because there is no inter-copy 764 dependency through resetting of F-BM as in BIER. 766 4. BIER-TE Controller Host BitPosition Assignments 768 This section describes how the BIER-TE controller host can use the 769 different BIER-TE adjacency types to define the BitPositions of a 770 BIER-TE domain. 772 Because the size of the BitString is limiting the size of the BIER-TE 773 domain, many of the options described exist to support larger 774 topologies with fewer BitPositions (4.1, 4.3, 4.4, 4.5, 4.6, 4.7, 775 4.8). 777 4.1. P2P Links 779 Each P2p link in the BIER-TE domain is assigned one unique 780 BitPosition with a forward_connected adjacency pointing to the 781 neighbor on the p2p link. 783 4.2. BFER 785 Every BFER is given a unique BitPosition with a local_decap 786 adjacency. 788 4.3. Leaf BFERs 790 Leaf BFERs are BFERs where incoming BIER-TE packets never need to be 791 forwarded to another BFR but are only sent to the BFER to exit the 792 BIER-TE domain. For example, in networks where PEs are spokes 793 connected to P routers, those PEs are Leaf BFIRs unless there is a 794 U-turn between two PEs. 796 All leaf-BFER in a BIER-TE domain can share a single BitPosition. 797 This is possible because the BitPosition for the adjacency to reach 798 the BFER can be used to distinguish whether or not packets should 799 reach the BFER. 801 This optimization will not work if an upstream interface of the BFER 802 is using a BitPosition optimized as described in the following two 803 sections (LAN, Hub and Spoke). 805 4.4. LANs 807 In a LAN, the adjacency to each neighboring BFR on the LAN is given a 808 unique BitPosition. The adjacency of this BitPosition is a 809 forward_connected adjacency towards the BFR and this BitPosition is 810 populated into the BIFT of all the other BFRs on that LAN. 812 BFR1 813 |p1 814 LAN1-+-+---+-----+ 815 p3| p4| p2| 816 BFR3 BFR4 BFR7 818 Figure 8: LAN Example 820 If Bandwidth on the LAN is not an issue and most BIER-TE traffic 821 should be copied to all neighbors on a LAN, then BitPositions can be 822 saved by assigning just a single BitPosition to the LAN and 823 populating the BitPosition of the BIFTs of each BFRs on the LAN with 824 a list of forward_connected adjacencies to all other neighbors on the 825 LAN. 827 This optimization does not work in the face of BFRs redundantly 828 connected to more than one LANs with this optimization because these 829 BFRs would receive duplicates and forward those duplicates into the 830 opposite LANs. Adjacencies of such BFRs into their LANs still need a 831 separate BitPosition. 833 4.5. Hub and Spoke 835 In a setup with a hub and multiple spokes connected via separate p2p 836 links to the hub, all p2p links can share the same BitPosition. The 837 BitPosition on the hubs BIFT is set up with a list of 838 forward_connected adjacencies, one for each Spoke. 840 This option is similar to the BitPosition optimization in LANs: 841 Redundantly connected spokes need their own BitPositions. 843 4.6. Rings 845 In L3 rings, instead of assigning a single BitPosition for every p2p 846 link in the ring, it is possible to save BitPositions by setting the 847 "Do Not Reset" (DNR) flag on forward_connected adjacencies. 849 For the rings shown in the following picture, a single BitPosition 850 will suffice to forward traffic entering the ring at BFRa or BFRb all 851 the way up to BFR1: 853 On BFRa, BFRb, BFR30,... BFR3, the BitPosition is populated with a 854 forward_connected adjacency pointing to the clockwise neighbor on the 855 ring and with DNR set. On BFR2, the adjacency also points to the 856 clockwise neighbor BFR1, but without DNR set. 858 Handling DNR this way ensures that copies forwarded from any BFR in 859 the ring to a BFR outside the ring will not have the ring BitPosition 860 set, therefore minimizing the chance to create loops. 862 v v 863 | | 864 L1 | L2 | L3 865 /-------- BFRa ---- BFRb --------------------\ 866 | | 867 \- BFR1 - BFR2 - BFR3 - ... - BFR29 - BFR30 -/ 868 | | L4 | | 869 p33| p15| 870 BFRd BFRc 872 Figure 9: Ring Example 874 Note that this example only permits for packets to enter the ring at 875 BFRa and BFRb, and that packets will always travel clockwise. If 876 packets should be allowed to enter the ring at any ring BFR, then one 877 would have to use two ring BitPositions. One for clockwise, one for 878 counterclockwise. 880 Both would be set up to stop rotating on the same link, e.g. L1. 881 When the ingress ring BFR creates the clockwise copy, it will reset 882 the counterclockwise BitPosition because the DNR bit only applies to 883 the bit for which the replication is done. Likewise for the 884 clockwise BitPosition for the counterclockwise copy. In result, the 885 ring ingress BFR will send a copy in both directions, serving BFRs on 886 either side of the ring up to L1. 888 4.7. Equal Cost MultiPath (ECMP) 890 The ECMP adjacency allows to use just one BP per link bundle between 891 two BFRs instead of one BP for each p2p member link of that link 892 bundle. In the following picture, one BP is used across L1,L2,L3 and 893 BFR1/BFR2 have for the BP 894 --L1----- 895 BFR1 --L2----- BFR2 896 --L3----- 898 BIFT entry in BFR1: 899 ------------------------------------------------------------------ 900 | Index | Adjacencies | 901 ================================================================== 902 | 0:6 | ECMP({L1-to-BFR2,L2-to-BFR2,L3-to-BFR2}, seed) | 903 ------------------------------------------------------------------ 905 BIFT entry in BFR2: 906 ------------------------------------------------------------------ 907 | Index | Adjacencies | 908 ================================================================== 909 | 0:6 | ECMP({L1-to-BFR1,L2-to-BFR1,L3-to-BFR1}, seed) | 910 ------------------------------------------------------------------ 912 Figure 10: ECMP Example 914 This document does not standardize any ECMP algorithm because it is 915 sufficient for implementations to document their freely chosen ECMP 916 algorithm. This allows the BIER-TE controller host to calculate ECMP 917 paths and seeds. The following picture shows an example ECMP 918 algorithm: 920 forward(packet, ECMP(adj(0), adj(1),... adj(N-1), seed)): 921 i = (packet(bier-header-entropy) XOR seed) % N 922 forward packet to adj(i) 924 Figure 11: ECMP algorithm Example 926 In the following example, all traffic from BFR1 towards BFR10 is 927 intended to be ECMP load split equally across the topology. This 928 example is not meant as a likely setup, but to illustrate that ECMP 929 can be used to share BPs not only across link bundles, and it 930 explains the use of the seed parameter. 932 BFR1 933 / \ 934 /L11 \L12 935 BFR2 BFR3 936 / \ / \ 937 /L21 \L22 /L31 \L32 938 BFR4 BFR5 BFR6 BFR7 939 \ / \ / 940 \ / \ / 941 BFR8 BFR9 942 \ / 943 \ / 944 BFR10 946 BIFT entry in BFR1: 947 ------------------------------------------------------------------ 948 | 0:6 | ECMP({L11-to-BFR2,L12-to-BFR3}, seed1) | 949 ------------------------------------------------------------------ 951 BIFT entry in BFR2: 952 ------------------------------------------------------------------ 953 | 0:6 | ECMP({L21-to-BFR4,L22-to-BFR5}, seed1) | 954 ------------------------------------------------------------------ 956 BIFT entry in BFR3: 957 ------------------------------------------------------------------ 958 | 0:6 | ECMP({L31-to-BFR6,L32-to-BFR7}, seed1) | 959 ------------------------------------------------------------------ 961 Figure 12: Polarization Example 963 With the setup of ECMP in above topology, traffic would not be 964 equally load-split. Instead, links L22 and L31 would see no traffic 965 at all: BFR2 will only see traffic from BFR1 for which the ECMP hash 966 in BFR1 selected the first adjacency in the list of 2 adjacencies 967 given as parameters to the ECMP. It is link L11-to-BFR2. BFR2 968 performs again ECMP with two adjacencies on that subset of traffic 969 using the same seed1, and will therefore again select the first of 970 its two adjacencies: L21-to-BFR4. And therefore L22 and BFR5 sees no 971 traffic. Likewise for L31 and BFR6. 973 To resolve this issue, the ECMP adjacency on BFR1 simply needs to be 974 set up with a different seed2 than the ECMP adjacencies on BFR2/BFR3. 975 ECMP in BFR2 could use the same seed2 to avoid its issue. 977 This issue in BFR2/BFR3 is called polarization. It depends on the 978 ECMP hash. Instead of explicitly setting up different seeds in 979 consecutive BFR in a topology subject to polarization, it is possible 980 to build ECMP that does not have polarization, for example by taking 981 entropy from the actual adjacency members into account such as the 982 next-hop identifiers like L11-to-BFR2 and, but that can make it 983 harder to achieve evenly balanced load-splitting on all BFR without 984 making the ECMP hash algorithm potentially too complex for fast 985 forwarding in the BFRs. In addition, these type of polarization free 986 ECMP algorithms likely make it harder for a BIER-TE controller host 987 to calculate entropy fields for BIER-TE headers that would flow on 988 the same or different ECMP paths. With polarizing algorithms, this 989 is typically easier. 991 4.8. Routed adjacencies 993 4.8.1. Reducing BitPositions 995 Routed adjacencies can reduce the number of BitPositions required 996 when the traffic engineering requirement is not hop-by-hop explicit 997 path selection, but loose-hop selection. 999 ............... ............... 1000 BFR1--... Redundant ...--L1-- BFR2... Redundant ...--- 1001 \--... Network ...--L2--/ ... Network ...--- 1002 BFR4--... Segment 1 ...--L3-- BFR3... Segment 2 ...--- 1003 ............... ............... 1005 Figure 13: Routed Adjacencies Example 1007 Assume the requirement in above network is to explicitly engineer 1008 paths such that specific traffic flows are passed from segment 1 to 1009 segment 2 via link L1 (or via L2 or via L3). 1011 To achieve this, BFR1 and BFR4 are set up with a forward_routed 1012 adjacency BitPosition towards an address of BFR2 on link L1 (or link 1013 L2 BFR3 via L3). 1015 For paths to be engineered through a specific node BFR2 (or BFR3), 1016 BFR1 and BFR4 are set up with a forward_routed adjacency BitPosition 1017 towards a loopback address of BFR2 (or BFR3). 1019 4.8.2. Supporting nodes without BIER-TE 1021 Routed adjacencies also enable incremental deployment of BIER-TE. 1022 Only the nodes through which BIER-TE traffic needs to be steered - 1023 with or without replication - need to support BIER-TE. Where they 1024 are not directly connected to each other, forward_routed adjacencies 1025 are used to pass over non BIER-TE enabled nodes. 1027 5. Avoiding loops and duplicates 1029 5.1. Loops 1031 Whenever BIER-TE creates a copy of a packet, the BitString of that 1032 copy will have all BitPositions cleared that are associated with 1033 adjacencies in the BFR. This inhibits looping of packets. The only 1034 exception are adjacencies with DNR set. 1036 With DNR set, looping can happen. Consider in the ring picture that 1037 link L4 from BFR3 is plugged into the L1 interface of BFRa. This 1038 creates a loop where the rings clockwise BitPosition is never reset 1039 for copies of the packets traveling clockwise around the ring. 1041 To inhibit looping in the face of such physical misconfiguration, 1042 only forward_connected adjacencies are permitted to have DNR set, and 1043 the link layer destination address of the adjacency (e.g. MAC 1044 address) protects against closing the loop. Link layers without port 1045 unique link layer addresses should not be used with the DNR flag set. 1047 5.2. Duplicates 1049 Duplicates happen when the topology of the BitString is not a tree 1050 but redundantly connecting BFRs with each other. The controller must 1051 therefore ensure to only create BitStrings that are trees in the 1052 topology. 1054 When links are incorrectly physically re-connected before the 1055 controller updates BitStrings in BFIRs, duplicates can happen. Like 1056 loops, these can be inhibited by link layer addressing in 1057 forward_connected adjacencies. 1059 If interface or loopback addresses used in forward_routed adjacencies 1060 are moved from one BFR to another, duplicates can equally happen. 1061 Such re-addressing operations must be coordinated with the 1062 controller. 1064 6. BIER-TE Forwarding Pseudocode 1066 The following simplified pseudocode for BIER-TE forwarding is using 1067 BIER forwarding pseudocode of [RFC8279], section 6.5 with the one 1068 modification necessary to support basic BIER-TE forwarding. Like the 1069 BIER pseudo forwarding code, for simplicity it does hide the details 1070 of the adjacency processing inside PacketSend() which can be 1071 forward_connected, forward_routed or local_decap. 1073 void ForwardBitMaskPacket_withTE (Packet) 1074 { 1075 SI=GetPacketSI(Packet); 1076 Offset=SI*BitStringLength; 1077 for (Index = GetFirstBitPosition(Packet->BitString); Index ; 1078 Index = GetNextBitPosition(Packet->BitString, Index)) { 1079 F-BM = BIFT[Index+Offset]->F-BM; 1080 if (!F-BM) continue; 1081 BFR-NBR = BIFT[Index+Offset]->BFR-NBR; 1082 PacketCopy = Copy(Packet); 1083 PacketCopy->BitString &= F-BM; [2] 1084 PacketSend(PacketCopy, BFR-NBR); 1085 // The following must not be done for BIER-TE: 1086 // Packet->BitString &= ~F-BM; [1] 1087 } 1088 } 1090 Figure 14: Simplified BIER-TE Forwarding Pseudocode 1092 The difference is that in BIER-TE, step [1] must not be performed. 1094 In BIER, this step is necessary to avoid duplicates when two or more 1095 BFER are reachable via the same neighbor. The F-BM of all those BFER 1096 bits will indicate each other's bits, and step [1] will reset all 1097 these bits on the first copy made for the first of those BFER bits 1098 set in the BitString, hence skipping any further copies to that 1099 neighbor. 1101 Whereas in BIER, the F-BM of bits toward a specific neighbor contain 1102 only the bits of those BFER destined to be forwarded across this 1103 neighbor, in BIER-TE the F-BM for a neighbor needs to have all bits 1104 set except all those bits that are actual (non-empty) adjacencies of 1105 this BFR. Step [2] will reset those adjacency bits to avoid loops, 1106 but all the other bits that are not adjacencies of this BFR need to 1107 stay untouched by [2] so that they can be processed by further BFR 1108 along the path. If [1] was performed as in BIER, then those non- 1109 adjacency bits would erroneously get reset during replication. 1111 To support the DNR (Do Not Reset) flag of forward_connected() 1112 adjacencies, the F-BM must also have its own bit set in the F-BM of 1113 such an adjacency , so that for the packet copy made for this 1114 adjacency the bit stays on, whereas it will not be set in the F-BM of 1115 other bits so that it will be reset for any other packet copy made. 1117 Eliminating the need to perform [1] also makes processing of bits in 1118 the BIER-TE bitstring independent of processing other bits, which may 1119 also simplify forwarding plane implementations. 1121 The following pseudocode is comprehensive: 1123 o This pseudocode eliminates per-bit F-BM, therefore reducing state 1124 by BitStringLength^2*SI and eliminating the need for per-packet- 1125 copy masking operation except for adjacencies with DNR flag set: 1127 * AdjacentBits[SI] are bits with a non-empty list of adjacencies. 1128 This can be computed whenever the BIER-TE controller host 1129 updates the adjacencies. 1131 * Only the AdjacentBits need to be examined in the loop for 1132 packet copies. 1134 * The packets BitString is masked with those AdjacentBits on 1135 ingress to avoid packets looping. 1137 o The code loops over the adjacencies because there may be more than 1138 one adjacency for a bit. 1140 o When an adjacency has the DNR bit, the bit is set in the packet 1141 copy (to save bits in rings for example). 1143 o The ECMP adjacency is shown. Its parameters are a 1144 ListOfAdjacencies from which one is picked. 1146 o The forward_local, forward_routed, local_decap adjacencies are 1147 shown with their parameters. 1149 void ForwardBitMaskPacket_withTE (Packet) 1150 { 1151 SI=GetPacketSI(Packet); 1152 Offset=SI*BitStringLength; 1153 AdjacentBitstring = Packet->BitString &= ~AdjacentBits[SI]; 1154 Packet->BitString &= AdjacentBits[SI]; 1155 for (Index = GetFirstBitPosition(AdjacentBits); Index ; 1156 Index = GetNextBitPosition(AdjacentBits, Index)) { 1157 foreach adjacency BIFT[Index+Offset] { 1158 if(adjacency == ECMP(ListOfAdjacencies, seed) ) { 1159 I = ECMP_hash(sizeof(ListOfAdjacencies), 1160 Packet->Entropy, seed); 1161 adjacency = ListOfAdjacencies[I]; 1162 } 1163 PacketCopy = Copy(Packet); 1164 switch(adjacency) { 1165 case forward_connected(interface,neighbor,DNR): 1166 if(DNR) 1167 PacketCopy->BitString |= 2<<(Index-1); 1168 SendToL2Unicast(PacketCopy,interface,neighbor); 1170 case forward_routed({VRF},neighbor): 1171 SendToL3(PacketCopy,{VRF,}l3-neighbor); 1173 case local_decap({VRF},neighbor): 1174 DecapBierHeader(PacketCopy); 1175 PassTo(PacketCopy,{VRF,}Packet->NextProto); 1176 } 1177 } 1178 } 1179 } 1181 Figure 15: BIER-TE Forwarding Pseudocode 1183 7. Managing SI, subdomains and BFR-ids 1185 When the number of bits required to represent the necessary hops in 1186 the topology and BFER exceeds the supported bitstring length, 1187 multiple SI and/or subdomains must be used. This section discusses 1188 how. 1190 BIER-TE forwarding does not require the concept of BFR-id, but 1191 routing underlay, flow overlay and BIER headers may. This section 1192 also discusses how BFR-ids can be assigned to BFIR/BFER for BIER-TE. 1194 7.1. Why SI and sub-domains 1196 For BIER and BIER-TE forwarding, the most important result of using 1197 multiple SI and/or subdomains is the same: Packets that need to be 1198 sent to BFER in different SI or subdomains require different BIER 1199 packets: each one with a bitstring for a different (SI,subdomain) 1200 bitstring. Each such bitstring uses one bitstring length sized SI 1201 block in the BIFT of the subdomain. We call this a BIFT:SI (block). 1203 For BIER and BIER-TE forwarding itself there is also no difference 1204 whether different SI and/or sub-domains are chosen, but SI and 1205 subdomain have different purposes in the BIER architecture shared by 1206 BIER-TE. This impacts how operators are managing them and how 1207 especially flow overlays will likely use them. 1209 By default, every possible BFIR/BFER in a BIER network would likely 1210 be given a BFR-id in subdomain 0 (unless there are > 64k BFIR/BFER). 1212 If there are different flow services (or service instances) requiring 1213 replication to different subsets of BFER, then it will likely not be 1214 possible to achieve the best replication efficiency for all of these 1215 service instances via subdomain 0. Ideal replication efficiency for 1216 N BFER exists in a subdomain if they are split over not more than 1217 ceiling(N/bitstring-length) SI. 1219 If service instances justify additional BIER:SI state in the network, 1220 additional subdomains will be used: BFIR/BFER are assigned BFIR-id in 1221 those subdomains and each service instance is configured to use the 1222 most appropriate subdomain. This results in improved replication 1223 efficiency for different services. 1225 Even if creation of subdomains and assignment of BFR-id to BFIR/BFER 1226 in those subdomains is automated, it is not expected that individual 1227 service instances can deal with BFER in different subdomains. A 1228 service instance may only support configuration of a single subdomain 1229 it should rely on. 1231 To be able to easily reuse (and modify as little as possible) 1232 existing BIER procedures including flow-overlay and routing underlay, 1233 when BIER-TE forwarding is added, we therefore reuse SI and subdomain 1234 logically in the same way as they are used in BIER: All necessary 1235 BFIR/BFER for a service use a single BIER-TE BIFT and are split 1236 across as many SI as necessary (see below). Different services may 1237 use different subdomains that primarily exist to provide more 1238 efficient replication (and for BIER-TE desirable traffic engineering) 1239 for different subsets of BFIR/BFER. 1241 7.2. Bit assignment comparison BIER and BIER-TE 1243 In BIER, bitstrings only need to carry bits for BFER, which leads to 1244 the model that BFR-ids map 1:1 to each bit in a bitstring. 1246 In BIER-TE, bitstrings need to carry bits to indicate not only the 1247 receiving BFER but also the intermediate hops/links across which the 1248 packet must be sent. The maximum number of BFER that can be 1249 supported in a single bitstring or BIFT:SI depends on the number of 1250 bits necessary to represent the desired topology between them. 1252 "Desired" topology because it depends on the physical topology, and 1253 on the desire of the operator to allow for explicit traffic 1254 engineering across every single hop (which requires more bits), or 1255 reducing the number of required bits by exploiting optimizations such 1256 as unicast (forward_route), ECMP or flood (DNR) over "uninteresting" 1257 sub-parts of the topology - e.g. parts where different trees do not 1258 need to take different paths due to traffic-engineering reasons. 1260 The total number of bits to describe the topology vs. the BFER in a 1261 BIFT:SI can range widely based on the size of the topology and the 1262 amount of alternative paths in it. The higher the percentage, the 1263 higher the likelihood, that those topology bits are not just BIER-TE 1264 overhead without additional benefit, but instead that they will allow 1265 to express desirable traffic-engineering path alternatives. 1267 7.3. Using BFR-id with BIER-TE 1269 Because there is no 1:1 mapping between bits in the bitstring and 1270 BFER, BIER-TE cannot simply rely on the BIER 1:1 mapping between bits 1271 in a bitstring and BFR-id. 1273 In BIER, automatic schemes could assign all possible BFR-ids 1274 sequentially to BFERs. This will not work in BIER-TE. In BIER-TE, 1275 the operator or BIER-TE controller host has to determine a BFR-id for 1276 each BFER in each required subdomain. The BFR-id may or may not have 1277 a relationship with a bit in the bitstring. Suggestions are detailed 1278 below. Once determined, the BFR-id can then be configured on the 1279 BFER and used by flow overlay, routing underlay and the BIER header 1280 almost the same as the BFR-id in BIER. 1282 The one exception are application/flow-overlays that automatically 1283 calculate the bitstring(s) of BIER packets by converting BFR-id to 1284 bits. In BIER-TE, this operation can be done in two ways: 1286 "Independent branches": For a given application or (set of) trees, 1287 the branches from a BFIR to every BFER are independent of the 1288 branches to any other BFER. For example, shortest part trees have 1289 independent branches. 1291 "Interdependent branches": When a BFER is added or deleted from a 1292 particular distribution tree, branches to other BFER still in the 1293 tree may need to change. Steiner tree are examples of dependent 1294 branch trees. 1296 If "independent branches" are sufficient, the BIER-TE controller host 1297 can provide to such applications for every BFR-id a SI:bitstring with 1298 the BIER-TE bits for the branch towards that BFER. The application 1299 can then independently calculate the SI:bitstring for all desired 1300 BFER by OR'ing their bitstrings. 1302 If "interdependent branches" are required, the application could call 1303 a BIER-TE controller host API with the list of required BFER-id and 1304 get the required bitstring back. Whenever the set of BFER-id 1305 changes, this is repeated. 1307 Note that in either case (unlike in BIER), the bits in BIER-TE may 1308 need to change upon link/node failure/recovery, network expansion and 1309 network load by other traffic (as part of traffic engineering goals). 1310 Interactions between such BFIR applications and the BIER-TE 1311 controller host do therefore need to support dynamic updates to the 1312 bitstrings. 1314 7.4. Assigning BFR-ids for BIER-TE 1316 For a non-leaf BFER, there is usually a single bit k for that BFER 1317 with a local_decap() adjacency on the BFER. The BFR-id for such a 1318 BFER is therefore most easily the one it would have in BIER: SI * 1319 bitstring-length + k. 1321 As explained earlier in the document, leaf BFERs do not need such a 1322 separate bit because the fact alone that the BIER-TE packet is 1323 forwarded to the leaf BFER indicates that the BFER should decapsulate 1324 it. Such a BFER will have one or more bits for the links leading 1325 only to it. The BFR-id could therefore most easily be the BFR-id 1326 derived from the lowest bit for those links. 1328 These two rules are only recommendations for the operator or BIER-TE 1329 controller assigning the BFR-ids. Any allocation scheme can be used, 1330 the BFR-ids just need to be unique across BFRs in each subdomain. 1332 It is not currently determined if a single subdomain could or should 1333 be allowed to forward both BIER and BIER-TE packets. If this should 1334 be supported, there are two options: 1336 A. BIER and BIER-TE have different BFR-id in the same subdomain. 1337 This allows higher replication efficiency for BIER because their BFR- 1338 id can be assigned sequentially, while the bitstrings for BIER-TE 1339 will have also the additional bits for the topology. There is no 1340 relationship between a BFR BIER BFR-id and BIER-TE BFR-id. 1342 B. BIER and BIER-TE share the same BFR-id. The BFR-id are assigned 1343 as explained above for BIER-TE and simply reused for BIER. The 1344 replication efficiency for BIER will be as low as that for BIER-TE in 1345 this approach. Depending on topology, only the same 20%..80% of bits 1346 as possible for BIER-TE can be used for BIER. 1348 7.5. Example bit allocations 1350 7.5.1. With BIER 1352 Consider a network setup with a bitstring length of 256 for a network 1353 topology as shown in the picture below. The network has 6 areas, 1354 each with ca. 170 BFR, connecting via a core with some larger (core) 1355 BFR. To address all BFER with BIER, 4 SI are required. To send a 1356 BIER packet to all BFER in the network, 4 copies need to be sent by 1357 the BFIR. On the BFIR it does not make a difference how the BFR-id 1358 are allocated to BFER in the network, but for efficiency further down 1359 in the network it does make a difference. 1361 area1 area2 area3 1362 BFR1a BFR1b BFR2a BFR2b BFR3a BFR3b 1363 | \ / \ / | 1364 ................................ 1365 . Core . 1366 ................................ 1367 | / \ / \ | 1368 BFR4a BFR4b BFR5a BFR5b BFR6a BFR6b 1369 area4 area5 area6 1371 Figure 16: Scaling BIER-TE bits by reuse 1373 With random allocation of BFR-id to BFER, each receiving area would 1374 (most likely) have to receive all 4 copies of the BIER packet because 1375 there would be BFR-id for each of the 4 SI in each of the areas. 1376 Only further towards each BFER would this duplication subside - when 1377 each of the 4 trees runs out of branches. 1379 If BFR-id are allocated intelligently, then all the BFER in an area 1380 would be given BFR-id with as few as possible different SI. Each 1381 area would only have to forward one or two packets instead of 4. 1383 Given how networks can grow over time, replication efficiency in an 1384 area will also easily go down over time when BFR-id are network wide 1385 allocated sequentially over time. An area that initially only has 1386 BFR-id in one SI might end up with many SI over a longer period of 1387 growth. Allocating SIs to areas with initially sufficiently many 1388 spare bits for growths can help to alleviate this issue. Or renumber 1389 BFR-id after network expansion. In this example one may consider to 1390 use 6 SI and assign one to each area. 1392 This example shows that intelligent BFR-id allocation within at least 1393 subdomain 0 can even be helpful or even necessary in BIER. 1395 7.5.2. With BIER-TE 1397 In BIER-TE one needs to determine a subset of the physical topology 1398 and attached BFER so that the "desired" representation of this 1399 topology and the BFER fit into a single bitstring. This process 1400 needs to be repeated until the whole topology is covered. 1402 Once bits/SIs are assigned to topology and BFER, BFR-id is just a 1403 derived set of identifiers from the operator/BIER-TE controller as 1404 explained above. 1406 Every time that different sub-topologies have overlap, bits need to 1407 be repeated across the bitstrings, increasing the overall amount of 1408 bits required across all bitstring/SIs. In the worst case, random 1409 subsets of BFER are assigned to different SI. This is much worse 1410 than in BIER because it not only reduces replication efficiency with 1411 the same number of overall bits, but even further - because more bits 1412 are required due to duplication of bits for topology across multiple 1413 SI. Intelligent BFER to SI assignment and selecting specific 1414 "desired" subtopologies can minimize this problem. 1416 To set up BIER-TE efficiently for above topology, the following bit 1417 allocation methods can be used. This method can easily be expanded 1418 to other, similarly structured larger topologies. 1420 Each area is allocated one or more SI depending on the number of 1421 future expected BFER and number of bits required for the topology in 1422 the area. In this example, 6 SI, one per area. 1424 In addition, we use 4 bits in each SI: bia, bib, bea, beb: bit 1425 ingress a, bit ingress b, bit egress a, bit egress b. These bits 1426 will be used to pass BIER packets from any BFIR via any combination 1427 of ingress area a/b BFR and egress area a/b BFR into a specific 1428 target area. These bits are then set up with the right 1429 forward_routed adjacencies on the BFIR and area edge BFR: 1431 On all BFIR in an area j, bia in each BIFT:SI is populated with the 1432 same forward_routed(BFRja), and bib with forward_routed(BFRjb). On 1433 all area edge BFR, bea in BIFT:SI=k is populated with 1434 forward_routed(BFRka) and beb in BIFT:SI=k with 1435 forward_routed(BFRkb). 1437 For BIER-TE forwarding of a packet to some subset of BFER across all 1438 areas, a BFIR would create at most 6 copies, with SI=1...SI=6, In 1439 each packet, the bits indicate bits for topology and BFER in that 1440 topology plus the four bits to indicate whether to pass this packet 1441 via the ingress area a or b border BFR and the egress area a or b 1442 border BFR, therefore allowing path engineering for those two 1443 "unicast" legs: 1) BFIR to ingress are edge and 2) core to egress 1444 area edge. Replication only happens inside the egress areas. For 1445 BFER in the same area as in the BFIR, these four bits are not used. 1447 7.6. Summary 1449 BIER-TE can like BIER support multiple SI within a sub-domain to 1450 allow re-using the concept of BFR-id and therefore minimize BIER-TE 1451 specific functions in underlay routing, flow overlay methods and BIER 1452 headers. 1454 The number of BFIR/BFER possible in a subdomain is smaller than in 1455 BIER because BIER-TE uses additional bits for topology. 1457 Subdomains can in BIER-TE be used like in BIER to create more 1458 efficient replication to known subsets of BFER. 1460 Assigning bits for BFER intelligently into the right SI is more 1461 important in BIER-TE than in BIER because of replication efficiency 1462 and overall amount of bits required. 1464 8. BIER-TE and Segment Routing (SR) 1466 Segment Routing (SR ([RFC8402])) aims to enable lightweight path 1467 engineering via loose source routing. Compared to its more heavy- 1468 weight predecessor RSVP-TE ([RFC3209]), SR does for example not 1469 require per-path signaling to each of these hops. 1471 BIER-TE supports the same design philosophy for multicast. Like in 1472 SR, it relies on source-routing - via the definition of a BitString. 1473 Like SR, it only requires to consider the "hops" on which either 1474 replication has to happen, or across which the traffic should be 1475 steered (even without replication). Any other hops can be skipped 1476 via the use of routed adjacencies. 1478 BIER-TE BitPosition (BP) can be understood as the BIER-TE equivalent 1479 of "forwarding segments" in SR, but they have a different scope than 1480 SR forwarding segments. Whereas forwarding segments in SR are global 1481 or local, BPs in BIER-TE have a scope that is the group of BFR(s) 1482 that have adjacencies for this BP in their BIFT. This can be called 1483 "adjacency" scoped forwarding segments. 1485 Adjacency scope could be global, but then every BFR would need an 1486 adjacency for this BP, for example a forward_routed adjacency with 1487 encapsulation to the global SR SID of the destination. Such a BP 1488 would always result in ingress replication though. The first BFR 1489 encountering this BP would directly replicate to it. Only by using 1490 non-global adjacency scope for BPs can traffic be steered and 1491 replicated on non-ingress BFR. 1493 SR can naturally be combined with BIER-TE and help to optimize it. 1494 For example, instead of defining BitPositions for non-replicating 1495 hops, it is equally possible to use segment routing encapsulations 1496 (eg: MPLS label stacks) for the encapsulation of "forward_routed" 1497 adjacencies. 1499 Note that BIER itself can also be seen to be similar to SR. BIER BPs 1500 act as global destination Node-SIDs and the BIER bitstring is simply 1501 a highly optimized mechanism to indicate multiple such SIDS and let 1502 the network take care of effectively replicating the packet hop-by- 1503 hop to each destination Node-SID. What BIER does not allow is to 1504 indicate intermediate hops, or terms of SR the ability to indicate a 1505 sequence of SID to reach the destination. This is what BIER-TE and 1506 its adjacency scoped BP enables. 1508 Both BIER and BIER-TE allow BFIR to "opportunistically" copy packets 1509 to a set of desired BFER on a packet-by-packet basis. In BIER, this 1510 is done by OR'ing the BP for the desired BFER. In BIER-TE this can 1511 be done by OR'ing for each desired BFER a bitstring using the 1512 "independent branches" approach described in Section 7.3 and 1513 therefore also indicating the engineered path towards each desired 1514 BFER. This is the approach that 1515 [I-D.ietf-bier-multicast-http-response] relies on. 1517 9. Security Considerations 1519 The security considerations are the same as for BIER with the 1520 following differences: 1522 BFR-ids and BFR-prefixes are not used in BIER-TE, nor are procedures 1523 for their distribution, so these are not attack vectors against BIER- 1524 TE. 1526 10. IANA Considerations 1528 This document requests no action by IANA. 1530 11. Acknowledgements 1532 The authors would like to thank Greg Shepherd, Ijsbrand Wijnands and 1533 Neale Ranns for their extensive review and suggestions. 1535 12. Change log [RFC Editor: Please remove] 1537 draft-ietf-bier-te-arch: 1539 04: spell check run. 1541 Addded remaining fixes for Sandys (Zhang Zheng) review: 1543 4.7 Enhance ECMP explanations: 1545 example ECMP algorithm, highlight that doc does not standardize 1546 ECMP algorithm. 1548 Review from Dirk Trossen: 1550 1. Added mentioning of prior work for traffic engineered paths 1551 with bloom filters. 1553 2. Changed title from layers to components and added "BIER-TE 1554 control plane" to "BIER-TE controller host" to make it clearer, 1555 what it does. 1557 2.2.3. Added reference to I-D.ietf-bier-multicast-http-response 1558 as an example solution. 1560 2.3. clarified sentence about resetting BPs before sending copies 1561 (also forgot to mention DNR here). 1563 3.4. Added text saying this section will be removed unless IESG 1564 review finds enough redeeming value in this example given how -03 1565 introduced section 1.1 with basic examples. 1567 7.2. Removed explicit numbers 20%/80% for number of topology bits 1568 in BIER-TE, replaced with more vague (high/low) description, 1569 because we do not have good reference material Added text saying 1570 this section will be removed unless IESG review finds enough 1571 redeeming value in this example given how -03 introduced section 1572 1.1 with basic examples. 1574 many typos fixed. Thanks a lot. 1576 03: Last call textual changes by authors to improve readability: 1578 removed Wolfgang Braun as co-authors (as requested). 1580 Improved abstract to be more explanatory. Removed mentioning of 1581 FRR (not concluded on so far). 1583 Added new text into Introduction section because the text was too 1584 difficult to jump into (too many forward pointers). This 1585 primarily consists of examples and the early introduction of the 1586 BIER-TE Topology concept enabled by these examples. 1588 Amended comparison to SR. 1590 Changed syntax from [VRF] to {VRF} to indicate its optional and to 1591 make idnits happy. 1593 Split references into normative / informative, added references. 1595 02: Refresh after IETF104 discussion: changed intended status back 1596 to standard. Reasoning: 1598 Tighter review of standards document == ensures arch will be 1599 better prepared for possible adoption by other WGs (e.g. DetNet) 1600 or std. bodies. 1602 Requirement against the degree of existing implementations is self 1603 defined by the WG. BIER WG seems to think it is not necessary to 1604 apply multiple interoperating implementations against an 1605 architecture level document at this time to make it qualify to go 1606 to standards track. Also, the levels of support introduced in -01 1607 rev. should allow all BIER forwarding engines to also be able to 1608 support the base level BIER-TE forwarding. 1610 01: Added note comparing BIER and SR to also hopefully clarify 1611 BIER-TE vs. BIER comparison re. SR. 1613 - added requirements section mandating only most basic BIER-TE 1614 forwarding features as MUST. 1616 - reworked comparison with BIER forwarding section to only 1617 summarize and point to pseudocode section. 1619 - reworked pseudocode section to have one pseudocode that mirrors 1620 the BIER forwarding pseudocode to make comparison easier and a 1621 second pseudocode that shows the complete set of BIER-TE 1622 forwarding options and simplification/optimization possible vs. 1623 BIER forwarding. Removed MyBitsOfInterest (was pure 1624 optimization). 1626 - Added captions to pictures. 1628 - Part of review feedback from Sandy (Zhang Zheng) integrated. 1630 00: Changed target state to experimental (WG conclusion), updated 1631 references, mod auth association. 1633 - Source now on http://www.github.com/toerless/bier-te-arch 1635 - Please open issues on the github for change/improvement requests 1636 to the document - in addition to posting them on the list 1637 (bier@ietf.). Thanks!. 1639 draft-eckert-bier-te-arch: 1641 06: Added overview of forwarding differences between BIER, BIER- 1642 TE. 1644 05: Author affiliation change only. 1646 04: Added comparison to Live-Live and BFIR to FRR section 1647 (Eckert). 1649 04: Removed FRR content into the new FRR draft [I-D.eckert-bier- 1650 te-frr] (Braun). 1652 - Linked FRR information to new draft in Overview/Introduction 1654 - Removed BTAFT/FRR from "Changes in the network topology" 1656 - Linked new draft in "Link/Node Failures and Recovery" 1658 - Removed FRR from "The BIER-TE Forwarding Layer" 1660 - Moved FRR section to new draft 1662 - Moved FRR parts of Pseudocode into new draft 1664 - Left only non FRR parts 1666 - removed FrrUpDown(..) and //FRR operations in 1667 ForwardBierTePacket(..) 1668 - New draft contains FrrUpDown(..) and ForwardBierTePacket(Packet) 1669 from bier-arch-03 1671 - Moved "BIER-TE and existing FRR to new draft 1673 - Moved "BIER-TE and Segment Routing" section one level up 1675 - Thus, removed "Further considerations" that only contained this 1676 section 1678 - Added Changes for version 04 1680 03: Updated the FRR section. Added examples for FRR key concepts. 1681 Added BIER-in-BIER tunneling as option for tunnels in backup 1682 paths. BIFT structure is expanded and contains an additional 1683 match field to support full node protection with BIER-TE FRR. 1685 03: Updated FRR section. Explanation how BIER-in-BIER 1686 encapsulation provides P2MP protection for node failures even 1687 though the routing underlay does not provide P2MP. 1689 02: Changed the definition of BIFT to be more inline with BIER. 1690 In revs. up to -01, the idea was that a BIFT has only entries for 1691 a single bitstring, and every SI and subdomain would be a separate 1692 BIFT. In BIER, each BIFT covers all SI. This is now also how we 1693 define it in BIER-TE. 1695 02: Added Section 7 to explain the use of SI, subdomains and BFR- 1696 id in BIER-TE and to give an example how to efficiently assign 1697 bits for a large topology requiring multiple SI. 1699 02: Added further detailed for rings - how to support input from 1700 all ring nodes. 1702 01: Fixed BFIR -> BFER for section 4.3. 1704 01: Added explanation of SI, difference to BIER ECMP, 1705 consideration for Segment Routing, unicast FRR, considerations for 1706 encapsulation, explanations of BIER-TE controller host and CLI. 1708 00: Initial version. 1710 13. References 1712 13.1. Normative References 1714 [RFC8279] Wijnands, IJ., Ed., Rosen, E., Ed., Dolganow, A., 1715 Przygienda, T., and S. Aldrin, "Multicast Using Bit Index 1716 Explicit Replication (BIER)", RFC 8279, 1717 DOI 10.17487/RFC8279, November 2017, 1718 . 1720 [RFC8296] Wijnands, IJ., Ed., Rosen, E., Ed., Dolganow, A., 1721 Tantsura, J., Aldrin, S., and I. Meilik, "Encapsulation 1722 for Bit Index Explicit Replication (BIER) in MPLS and Non- 1723 MPLS Networks", RFC 8296, DOI 10.17487/RFC8296, January 1724 2018, . 1726 13.2. Informative References 1728 [I-D.ietf-bier-multicast-http-response] 1729 Trossen, D., Rahman, A., Wang, C., and T. Eckert, 1730 "Applicability of BIER Multicast Overlay for Adaptive 1731 Streaming Services", draft-ietf-bier-multicast-http- 1732 response-01 (work in progress), June 2019. 1734 [I-D.ietf-roll-ccast] 1735 Bergmann, O., Bormann, C., Gerdes, S., and H. Chen, 1736 "Constrained-Cast: Source-Routed Multicast for RPL", 1737 draft-ietf-roll-ccast-01 (work in progress), October 2017. 1739 [ICC] Reed, M., Al-Naday, M., Thomos, N., Trossen, D., 1740 Petropoulos, G., and S. Spirou, "Stateless multicast 1741 switching in software defined networks", IEEE 1742 International Conference on Communications (ICC), Kuala 1743 Lumpur, Malaysia, 2016, May 2016, 1744 . 1746 [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate 1747 Requirement Levels", BCP 14, RFC 2119, 1748 DOI 10.17487/RFC2119, March 1997, 1749 . 1751 [RFC3209] Awduche, D., Berger, L., Gan, D., Li, T., Srinivasan, V., 1752 and G. Swallow, "RSVP-TE: Extensions to RSVP for LSP 1753 Tunnels", RFC 3209, DOI 10.17487/RFC3209, December 2001, 1754 . 1756 [RFC8402] Filsfils, C., Ed., Previdi, S., Ed., Ginsberg, L., 1757 Decraene, B., Litkowski, S., and R. Shakir, "Segment 1758 Routing Architecture", RFC 8402, DOI 10.17487/RFC8402, 1759 July 2018, . 1761 Authors' Addresses 1763 Toerless Eckert (editor) 1764 Futurewei Technologies Inc. 1765 2330 Central Expy 1766 Santa Clara 95050 1767 USA 1769 Email: tte+ietf@cs.fau.de 1771 Gregory Cauchie 1772 Bouygues Telecom 1774 Email: GCAUCHIE@bouyguestelecom.fr 1776 Michael Menth 1777 University of Tuebingen 1779 Email: menth@uni-tuebingen.de