idnits 2.17.00 (12 Aug 2021) /tmp/idnits47215/draft-rosen-rfc2547bis-03.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- ** Looks like you're using RFC 2026 boilerplate. This must be updated to follow RFC 3978/3979, as updated by RFC 4748. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- == No 'Intended status' indicated for this document; assuming Proposed Standard Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- ** The document seems to lack an IANA Considerations section. (See Section 2.2 of https://www.ietf.org/id-info/checklist for how to handle the case when there are no actions for IANA.) ** The document seems to lack separate sections for Informative/Normative References. All references will be assumed normative when checking for downward references. -- The abstract seems to indicate that this document obsoletes RFC2547, but the header doesn't have an 'Obsoletes:' line to match this. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the RFC 3978 Section 5.4 Copyright Line does not match the current year -- The document seems to lack a disclaimer for pre-RFC5378 work, but may have content which was first submitted before 10 November 2008. If you have contacted all the original authors and they are all willing to grant the BCP78 rights to the IETF Trust, then this is fine, and you can ignore this comment. If not, you may need to add the pre-RFC5378 disclaimer. (See the Legal Provisions document at https://trustee.ietf.org/license-info for more information.) -- Couldn't find a document date in the document -- date freshness check skipped. Checking references for intended status: Proposed Standard ---------------------------------------------------------------------------- (See RFCs 3967 and 4897 for information about using normative references to lower-maturity documents in RFCs) == Unused Reference: 'IPSEC' is defined on line 1759, but no explicit reference was found in the text == Unused Reference: 'MPLS-ARCH' is defined on line 1762, but no explicit reference was found in the text ** Obsolete normative reference: RFC 2858 (ref. 'BGP-MP') (Obsoleted by RFC 4760) -- Possible downref: Non-RFC (?) normative reference: ref. 'BGP-EXTCOMM' -- Possible downref: Non-RFC (?) normative reference: ref. 'BGP-ORF' ** Obsolete normative reference: RFC 2796 (ref. 'BGP-RR') (Obsoleted by RFC 4456) ** Obsolete normative reference: RFC 2401 (ref. 'IPSEC') (Obsoleted by RFC 4301) -- Possible downref: Non-RFC (?) normative reference: ref. 'MPLS-BGP' ** Obsolete normative reference: RFC 3036 (ref. 'MPLS-LDP') (Obsoleted by RFC 5036) -- Possible downref: Non-RFC (?) normative reference: ref. 'MPLS-RSVP' ** Downref: Normative reference to an Informational RFC: RFC 2430 (ref. 'PASTE') -- Possible downref: Non-RFC (?) normative reference: ref. 'VPN-OSPF' Summary: 8 errors (**), 0 flaws (~~), 4 warnings (==), 8 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 Network Working Group Eric C. Rosen 3 Internet Draft Cisco Systems, Inc. 4 Expiration Date: August 2001 Yakov Rekhter 5 Juniper Networks, Inc. 7 Tony Bogovic Stephen John Brannon 8 Ravichander Vaidyanathan Monique Jeanne Morrow 9 Telcordia Technologies Swisscom AG 11 Marco Carugi Christopher J. Chase 12 France Telecom Luyuan Fang 13 ATT 15 Ting Wo Chung Jeremy De Clercq 16 Bell Nexxia Alcatel 18 Eric Dean Paul Hitchin 19 Global One Adrian Smith 20 BT 22 Manoj Leelanivas Dave Marshall 23 Juniper Networks, Inc. Worldcom 25 Luca Martini Vijay Srinivasan 26 Level 3 Communications, LLC CoSine Communications 28 Alain Vedrenne 29 SITA EQUANT 31 February 2001 33 BGP/MPLS VPNs 35 draft-rosen-rfc2547bis-03.txt 37 Status of this Memo 39 This document is an Internet-Draft and is in full conformance with 40 all provisions of Section 10 of RFC2026. 42 Internet-Drafts are working documents of the Internet Engineering 43 Task Force (IETF), its areas, and its working groups. Note that 44 other groups may also distribute working documents as Internet- 45 Drafts. 47 Internet-Drafts are draft documents valid for a maximum of six months 48 and may be updated, replaced, or obsoleted by other documents at any 49 time. It is inappropriate to use Internet-Drafts as reference 50 material or to cite them other than as "work in progress." 52 The list of current Internet-Drafts can be accessed at 53 http://www.ietf.org/ietf/1id-abstracts.txt. 55 The list of Internet-Draft Shadow Directories can be accessed at 56 http://www.ietf.org/shadow.html. 58 Copyright Notice 60 Copyright (C) The Internet Society (2000). All Rights Reserved. 62 Abstract 64 This document describes a method by which a Service Provider may use 65 an IP backbone to provide VPNs for its customers. MPLS is used for 66 forwarding packets over the backbone, and BGP is used for 67 distributing routes over the backbone. The primary goal of this 68 method is to support the case in which a client obtains IP backbone 69 services from a Service Provider or Service Providers with which it 70 maintains contractual relationships. The client may be an 71 enterprise, a group of enterprises which need an extranet, an 72 Internet Service Provider, an application service provider, another 73 VPN Service Provider which uses this same method to offer VPNs to 74 clients of its own, etc. The method makes it very simple for the 75 client to use the backbone services. It is also very scalable and 76 flexible for the Service Provider, and allows the Service Provider to 77 add value. 79 This document obsoletes RFC 2547. 81 Table of Contents 83 1 Introduction ....................................... 4 84 1.1 Virtual Private Networks ........................... 4 85 1.2 Edge Devices ....................................... 5 86 1.3 Multiple Forwarding Tables in PEs .................. 6 87 1.4 VPNs with Overlapping Address Spaces ............... 6 88 1.5 VPNs with Different Routes to the Same System ...... 7 89 1.6 SP Backbone Routers ................................ 7 90 1.7 Security ........................................... 8 91 2 Sites and CEs ...................................... 8 92 3 VRFs: Per-Site Forwarding Tables in the PEs ........ 9 93 4 VPN Route Distribution via BGP ..................... 11 94 4.1 The VPN-IPv4 Address Family ........................ 11 95 4.2 Encoding of Route Distinguishers ................... 13 96 4.3 Controlling Route Distribution ..................... 13 97 4.3.1 The Route Target Attribute ......................... 14 98 4.3.2 Route Distribution Among PEs by BGP ................ 15 99 4.3.3 Use of Route Reflectors ............................ 17 100 4.3.4 How VPN-IPv4 NLRI is Carried in BGP ................ 19 101 4.3.5 Building VPNs using Route Targets .................. 19 102 4.3.6 Route Distribution Among VRFs in a Single PE ....... 20 103 5 Forwarding Across the Backbone ..................... 20 104 6 Maintaining Proper Isolation of VPNs ............... 22 105 7 How PEs Learn Routes from CEs ...................... 22 106 8 How CEs learn Routes from PEs ...................... 25 107 9 Carriers' Carriers ................................. 26 108 10 Inter-Provider Backbones ........................... 27 109 11 Accessing the Internet from a VPN .................. 29 110 12 Management VPNs .................................... 31 111 13 Security ........................................... 32 112 13.1 Data Plane ......................................... 32 113 13.2 Control Plane ...................................... 34 114 13.3 Security of P and PE devices ....................... 34 115 14 Quality of Service ................................. 34 116 15 Scalability ........................................ 34 117 16 Intellectual Property Considerations ............... 35 118 17 Acknowledgments .................................... 35 119 18 Authors' Addresses ................................. 36 120 19 References ......................................... 39 121 20 Full Copyright Statement ........................... 40 123 1. Introduction 125 1.1. Virtual Private Networks 127 Consider a set of "sites" which are attached to a common network 128 which we may call the "backbone". Let's apply some policy to create a 129 number of subsets of that set, and let's impose the following rule: 130 two sites may have IP interconnectivity over that backbone only if at 131 least one of these subsets contains them both. 133 The subsets we have created are "Virtual Private Networks" (VPNs). 134 Two sites have IP connectivity over the common backbone only if there 135 is some VPN which contains them both. Two sites which have no VPN in 136 common have no connectivity over that backbone. 138 If all the sites in a VPN are owned by the same enterprise, the VPN 139 is a corporate "intranet". If the various sites in a VPN are owned 140 by different enterprises, the VPN is an "extranet". A site can be in 141 more than one VPN; e.g., in an intranet and in several extranets. In 142 general, when we use the term VPN we will not be distinguishing 143 between intranets and extranets. 145 We wish to consider the case in which the backbone is owned and 146 operated by one or more Service Providers (SPs). The owners of the 147 sites are the "customers" of the SPs. The policies that determine 148 whether a particular collection of sites is a VPN are the policies of 149 the customers. Some customers will want the implementation of these 150 policies to be entirely the responsibility of the SP. Other 151 customers may want to implement these policies themselves, or to 152 share with the SP the responsibility for implementing these policies. 153 In this document, we are primarily discussing mechanisms that may be 154 used to implement these policies. The mechanisms we describe are 155 general enough to allow these policies to be implemented either by 156 the SP alone, or by a VPN customer together with the SP. Most of the 157 discussion is focused on the former case, however. 159 The mechanisms discussed in this document allow the implementation of 160 a wide range of policies. For example, within a given VPN, we can 161 allow every site to have a direct route to every other site ("full 162 mesh"), or we can restrict certain pairs of sites from having direct 163 routes to each other ("partial mesh"). 165 In this document, we are interested in the case where the common 166 backbone offers an IP service. We are NOT focused on the case where 167 the common backbone is part of the public Internet, but rather on the 168 case where it is the backbone network of an SP or set of SPs with 169 which the customer maintains contractual relationships. That is, the 170 customer is explicitly purchasing VPN service from the SP, rather 171 than purchasing Internet access from it. (The customer may or may 172 not be purchasing Internet access from the same SP as well.) 174 The customer itself may be a single enterprise, a set of enterprises 175 needing an extranet, an Internet Service Provider, an application 176 service provider, or even another SP which offers the same kind of 177 VPN service to its own customers. 179 In the rest of this introduction, we specify some properties which 180 VPNs should have. The remainder of this document outlines a VPN 181 model which has all these properties. 183 1.2. Edge Devices 185 We suppose that at each site, there are one or more Customer Edge 186 (CE) devices, each of which is "attached" via some sort of data link 187 (e.g., PPP, ATM, ethernet, Frame Relay, GRE tunnel, etc.) to one or 188 more Provider Edge (PE) routers. Routers in the Provider's network 189 which do not attach to CE devices are known as "P routers". 191 If a particular site has a single host, that host may be the CE 192 device. If a particular site has a single subnet, the CE device may 193 be a switch. In general, the CE device can be expected to be a 194 router, which we call the CE router. 196 We will say that a PE router is attached to a particular VPN if it is 197 attached to a CE device which is in that VPN. Similarly, we will say 198 that a PE router is attached to a particular site if it is attached 199 to a CE device which is in that site. 201 When the CE device is a router, it is a routing peer of the PE(s) to 202 which it is attached, but it is NOT a routing peer of CE routers at 203 other sites. Routers at different sites do not directly exchange 204 routing information with each other; in fact, they do not even need 205 to know of each other at all. As a consequence, the customer has no 206 backbone or "virtual backbone" to manage, and does not have to deal 207 with any inter-site routing issues. In other words, in the scheme 208 described in this document, a VPN is NOT an "overlay" on top of the 209 SP's network. 211 With respect to the management of the edge devices, clear 212 administrative boundaries are maintained between the SP and its 213 customers. Customers are not required to access the PE or P routers 214 for management purposes, nor is the SP required to access the CE 215 devices for management purposes. 217 1.3. Multiple Forwarding Tables in PEs 219 Each PE router maintains a number of separate forwarding tables. 220 Every site to which the PE is attached must be mapped to one of those 221 forwarding tables. When a packet is received from a particular site, 222 the forwarding table associated with that site is consulted in order 223 to determine how to route the packet. The forwarding table 224 associated with a particular site S is populated ONLY with routes 225 that lead to other sites which have at least one VPN in common with 226 S. This prevents communication between sites which have no VPN in 227 common. 229 A PE router is attached to a site by virtue of being the endpoint of 230 an interface or "sub-interface" (e.g., PVC, VLAN, GRE tunnel, etc.) 231 whose other endpoint is a CE device. If there are multiple 232 attachments between a site and a PE router, all the attachments may 233 be mapped to the same forwarding table, or different attachments may 234 be mapped to different forwarding tables. When a PE router receives 235 a packet from a CE device, it knows the interface or sub-interface 236 over which the packet arrived, and this determines the forwarding 237 table used for processing that packet. The choice of forwarding 238 table is NOT determined by the user content of the packet. 240 Different sites can be mapped to the same forwarding table, but ONLY 241 if they have all their VPNs in common. 243 A PE router will also have a "default forwarding table," which is not 244 associated with any particular VPN site or sites. The default 245 forwarding table is used for handling traffic which is not VPN 246 traffic, as well as for VPN traffic which is simply transiting this 247 router (i.e., traffic which was not received over a sub-interface 248 whose other endpoint is a CE device, and which is not being sent over 249 a sub-interface whose other endpoint is a CE device. 251 1.4. VPNs with Overlapping Address Spaces 253 If two VPNs have no sites in common, then they may have overlapping 254 address spaces. That is, a given address might be used in VPN V1 as 255 the address of system S1, but in VPN V2 as the address of a 256 completely different system S2. This is a common situation when the 257 VPNs each use an RFC1918 private address space. (In fact, two VPNs 258 which do have sites in common may have overlapping address spaces, as 259 long as the overlapping part of the address space does not belong to 260 any of the sites which the two VPNs have in common.) 262 The fact that sites in different VPNs are mapped to different 263 forwarding tables makes it possible for different VPNs to have 264 overlapping address spaces, without creating any ambiguity. 266 1.5. VPNs with Different Routes to the Same System 268 Although a site may be in multiple VPNs, it is not necessarily the 269 case that the route to a given system at that site should be the same 270 in all the VPNs. Suppose, for example, we have an intranet 271 consisting of sites A, B, and C, and an extranet consisting of A, B, 272 C, and the "foreign" site D. Suppose that at site A there is a 273 server, and we want clients from B, C, or D to be able to use that 274 server. Suppose also that at site B there is a firewall. We want 275 all the traffic from site D to the server to pass through the 276 firewall, so that traffic from the extranet can be access controlled. 277 However, we don't want traffic from C to pass through the firewall on 278 the way to the server, since this is intranet traffic. 280 This means that it needs to be possible to set up two routes to the 281 server. One route, used by sites B and C, takes the traffic directly 282 to site A. The second route, used by site D, takes the traffic 283 instead to the firewall at site B. If the firewall allows the 284 traffic to pass, it then appears to be traffic coming from site B, 285 and follows the route to site A. 287 1.6. SP Backbone Routers 289 The SP's backbone consists of the PE routers, as well as other 290 routers ("P routers") which do not attach to CE devices. 292 If every router in an SP's backbone had to maintain routing 293 information for all the VPNs supported by the SP, this model would 294 have severe scalability problems; the number of sites that could be 295 supported would be limited by the amount of routing information that 296 could be held in a single router. It is important therefore that the 297 routing information about a particular VPN is only required to be 298 present in those PE routers which attach to that VPN. In particular, 299 the P routers should not need to have ANY per-VPN routing information 300 whatsoever. (This condition may need to be relaxed somewhat when 301 multicast routing is considered. This is not considered further in 302 this paper.) 304 So just as the VPN owners do not have a backbone or "virtual 305 backbone" to administer, the SPs themselves do not have a separate 306 backbone or "virtual backbone" to administer for each VPN. Site-to- 307 site routing in the backbone is optimal (within the constraints of 308 the policies used to form the VPNs), and is not constrained in any 309 way by an artificial "virtual topology" of tunnels. 311 VPNs may span multiple service providers. There are a number of 312 possible methods for implementing this, which shall be discussed 313 later. 315 1.7. Security 317 VPNs of the sort being discussed here, even without making use of 318 cryptographic security measures, are intended to provide a level of 319 security equivalent to that obtainable when a level 2 backbone (e.g., 320 Frame Relay) is used. That is, in the absence of misconfiguration or 321 deliberate interconnection of different VPNs, it is not possible for 322 systems in one VPN to gain access to systems in another VPN. This is 323 discussed in more detail in section 13. 325 2. Sites and CEs 327 From the perspective of a particular backbone network, a set of IP 328 systems constitutes a site if those systems have mutual IP 329 interconnectivity, and communication among them occurs without use of 330 the backbone. In general, a site will consist of a set of systems 331 which are in geographic proximity. However, this is not universally 332 true. If two geographic locations are connected via a leased line, 333 over which OSPF is running, and if that line is the preferred way of 334 communicating between the two locations, then the two locations can 335 be regarded as a single site, even if each location has its own CE 336 router. (This notion of "site" is topological, rather than 337 geographical. If the leased line goes down, or otherwise ceases to 338 be the preferred route, but the two geographic locations can continue 339 to communicate by using the VPN backbone, then one site has become 340 two.) 342 A CE device is always regarded as being in a single site (though as 343 we shall see, a site may consist of multiple "virtual sites"). A 344 site, however, may belong to multiple VPNs. 346 A PE router may attach to CE devices in any number of different 347 sites, whether those CE devices are in the same or in different VPNs. 348 A CE device may, for robustness, attach to multiple PE routers, of 349 the same or of different service providers. If the CE device is a 350 router, the PE router and the CE router will appear as router 351 adjacencies to each other. 353 While the basic unit of interconnection is the site, the architecture 354 described herein allows a finer degree of granularity in the control 355 of interconnectivity. For example, certain systems at a site may be 356 members of an intranet as well as members of one or more extranets, 357 while other systems at the same site may be restricted to being 358 members of the intranet only. 360 In some cases, a particular site may be divided by the customer into 361 several "virtual sites", perhaps by the use of VLANs. Each virtual 362 site may be a member of a different set of VPNs. For example, if a CE 363 supports VLANs, and wants each VLAN mapped to a separate VPN, the 364 packets sent between CE and PE could be contained in the site's VLAN 365 encapsulation. Then the VLAN tag could be used by the PE, along with 366 the interface over which the packet is received, to assign the packet 367 to a particular VPN. 369 Alternatively, one could divide the interface into multiple "sub- 370 interfaces" (particularly if the interface is Frame Relay or ATM), 371 and assign the packet to a VPN based on the sub-interface over which 372 it arrives. Or one could simply use a different interface for each 373 virtual site. In any case, only one CE router is ever needed per 374 site, even if there are multiple virtual sites. Of course, a 375 different CE router could be used for each virtual site, if that is 376 desired. 378 Note that in all these cases, the mechanisms, as well as the policy, 379 for controlling which traffic is in which VPN are in the hand of the 380 customer. 382 If it is desired to have a particular host be in multiple virtual 383 sites, then that host must determine, for each packet, which virtual 384 site the packet is associated with. It can do this, e.g., by sending 385 packets from different virtual sites on different VLANs, or out 386 different network interfaces. 388 3. VRFs: Per-Site Forwarding Tables in the PEs 390 Each PE router maintains one or more "per-site forwarding tables." 391 These are known as VRFs, or "VPN Routing and Forwarding" tables. 392 Every site to which the PE router is attached is associated with one 393 of these tables. A particular packet's IP destination address is 394 looked up in a particular VRF only if that packet has arrived 395 directly from a site which is associated with that table. 397 It would in fact be more precise to say that in the PE router, 398 - sub-interfaces may be mapped to VRFs, 399 - the mapping is many-to-one, 400 - the VRF in which a packet's destination address is looked up is 401 determined by the sub-interface over which it is received, and 403 - two sub-interfaces may not be mapped to the same VRF unless the 404 same set of routes is meant to be available to packets received 405 over either sub-interface. 407 A sub-interface which is mapped to a VRF may be referred to as a "VRF 408 sub-interface". 410 How are the VRFs populated? 412 As an example, let PE1, PE2, and PE3 be three PE routers, and let 413 CE1, CE2, and CE3 be three CE routers. Suppose that PE1 learns, from 414 CE1, the routes which are reachable at CE1's site. If PE2 and PE3 415 are attached respectively to CE2 and CE3, and there is some VPN V 416 containing CE1, CE2, and CE3, then PE1 uses BGP to distribute to PE2 417 and PE3 the routes which it has learned from CE1. PE2 and PE3 use 418 these routes to populate the VRFs which they associate respectively 419 with the sites of CE2 and CE3. Routes from sites which are not in 420 VPN V do not appear in these VRFs, which means that packets from CE2 421 or CE3 cannot be sent to sites which are not in VPN V. 423 If a site is in multiple VPNs, the VRF associated with that site 424 contains routes from the full set of VPNs of which the site is a 425 member. 427 A PE generally associates only one VRF to each site, even if it is 428 multiply connected to that site. However, different sites can share 429 the same VRF if (and only if) they are meant to use exactly the same 430 set of routes. 432 When a PE receives a packet from a directly attached site, it always 433 looks up the packet's destination address in the VRF which is 434 associated with that site. However, when a PE receives a packet 435 which is destined to go to a particular directly attached site, it 436 does not necessarily need to lookup the packet's destination address 437 in the VRF (or anywhere else). The packet may already be carrying 438 enough information (in the form of an MPLS label, see section 5) to 439 determine the packet's outgoing sub-interface. That is, the packet's 440 exit point from the backbone may be completely determined by the 441 information in the VRF associated with its entry point to the 442 backbone. 444 This allows the backbone to support multiple different routes to the 445 same system, where the route followed by a given packet is determined 446 by the site from which the packet enters the backbone. E.g., one may 447 have one route to a given system for packets from the extranet (where 448 the route leads to a firewall), and a different route to the same 449 system for packets from the intranet (including packets that have 450 already passed through the firewall). 452 A PE router also contains a "default forwarding table", which is not 453 a VRF. The default forwarding table is used for forwarding packets 454 that arrive on sub-interfaces which are not associated with any VRF, 455 and which are not destined to be sent on sub-interfaces associated 456 with a VRF. The default forwarding table is populated in the normal 457 way by the routing algorithm of the SP network; it does not contain 458 routes from the VPNs. 460 4. VPN Route Distribution via BGP 462 PE routers use BGP to distribute VPN routes to each other (more 463 accurately, to cause VPN routes to be distributed to each other). 465 We allow each VPN to have its own address space, which means that a 466 given address may denote different systems in different VPNs. If two 467 routes, to the same IP address prefix, are actually routes to 468 different systems, it is important to ensure that BGP not treat them 469 as comparable. Otherwise BGP might choose to install only one of 470 them, making the other system unreachable. Further, we must ensure 471 that POLICY is used to determine which packets get sent on which 472 routes; given that several such routes are installed by BGP, only one 473 such must appear in any particular VRF. 475 We meet these goals by the use of a new address family, as specified 476 below. 478 4.1. The VPN-IPv4 Address Family 480 The BGP Multiprotocol Extensions [BGP-MP] allow BGP to carry routes 481 from multiple "address families". We introduce the notion of the 482 "VPN-IPv4 address family". A VPN-IPv4 address is a 12-byte quantity, 483 beginning with an 8-byte "Route Distinguisher (RD)" and ending with a 484 4-byte IPv4 address. If two VPNs use the same IPv4 address prefix, 485 the PEs translate these into unique VPN-IPv4 address prefixes. This 486 ensures that if the same address is used in two different VPNs, it is 487 possible to install two completely different routes to that address, 488 one for each VPN. 490 The RD does not by itself impose any semantics; it contains no 491 information about the origin of the route or about the set of VPNs to 492 which the route is to be distributed. The purpose of the RD is 493 solely to allow one to create distinct routes to a common IPv4 494 address prefix. Other means are used to determine where to 495 redistribute the route (see section 4.3). 497 The RD can also be used to create multiple different routes to the 498 very same system. In section 3, we gave an example where the route 499 to a particular server had to be different for intranet traffic than 500 for extranet traffic. This can be achieved by creating two different 501 VPN-IPv4 routes that have the same IPv4 part, but different RDs. 502 This allows BGP to install multiple different routes to the same 503 system, and allows policy to be used (see section 4.3.5) to decide 504 which packets use which route. 506 The RDs are structured so that every service provider can administer 507 its own "numbering space" (i.e., can make its own assignments of 508 RDs), without conflicting with the RD assignments made by any other 509 service provider. An RD consists of a two-byte type field, an 510 administrator field, and an assigned number field. The value of the 511 type field determines the lengths of the other two fields, as well as 512 the semantics of the administrator field. The administrator field 513 identifies an assigned number authority, and the assigned number 514 field contains a number which has been assigned, by the identified 515 authority, for a particular purpose. For example, one could have an 516 RD whose administrator field contains an Autonomous System number 517 (ASN), and whose (4-byte) number field contains a number assigned by 518 the SP to whom that ASN belongs (having been assigned to that SP by 519 the appropriate authority). 521 RDs are given this structure in order to ensure that an SP which 522 provides VPN backbone service can always create a unique RD when it 523 needs to do so. However, the structuring provides no semantics. When 524 BGP compares two such address prefixes, it ignores the structure 525 entirely. 527 Note that VPN-IPv4 addresses and IPv4 addresses are always 528 considered by BGP to be incomparable. 530 A VRF may have multiple VPN-IPv4 routes for a single IPv4 address 531 prefix. When a packet's destination address is matched against a 532 VPN-IPv4 route, only the IPv4 part is actually matched. 534 A PE needs to be configured such that routes which lead to particular 535 CE become associated with a particular RD. The configuration may 536 cause all routes leading to the same CE to be associated with the 537 same RD, or it may be cause different routes to be associated with 538 different RDs, even if they lead to the same CE. 540 4.2. Encoding of Route Distinguishers 542 As stated, a VPN-IPv4 address consists of an 8-byte Route 543 Distinguisher followed by a 4-byte IPv4 address. The RDs are encoded 544 as follows: 546 - Type Field: 2 bytes 547 - Value Field: 6 bytes 549 The interpretation of the Value field depends on the value of the 550 Type field. At the present time, two values of the type field are 551 defined: 0 and 1. 553 - Type 0: The Value field consists of two subfields: 555 * Administrator subfield: 2 bytes 556 * Assigned Number subfield: 4 bytes 558 The Administrator subfield must contain an Autonomous System 559 number. If this ASN is from the public ASN space, it must have 560 been assigned by the appropriate authority (use of ASN values 561 from the private ASN space is strongly discouraged). The 562 Assigned Number subfield contains a number from a numbering space 563 which is administered by the enterprise to which the ASN has been 564 assigned by an appropriate authority. 566 - Type 1: The Value field consists of two subfields: 568 * Administrator subfield: 4 bytes 569 * Assigned Number subfield: 2 bytes 571 The Administrator subfield must contain an IP address. If this IP 572 address is from the public IP address space, it must have been 573 assigned by an appropriate authority (use of addresses from the 574 private IP address space is strongly discouraged). The Assigned 575 Number sub-field contains a number from a numbering space which 576 is administered by the enterprise to which the IP address has 577 been assigned. 579 4.3. Controlling Route Distribution 581 In this section, we discuss the way in which the distribution of the 582 VPN-IPv4 routes is controlled. 584 4.3.1. The Route Target Attribute 586 Every VRF is associated with one or more "Route Target" attributes. 588 When a VPN-IPv4 route is created by a PE router, it is associated 589 with one or more "Route Target" attributes. These are carried in BGP 590 as attributes of the route. 592 Any route associated with Route Target T must be distributed to every 593 PE router that has a VRF associated with Route Target T. When such a 594 route is received by a PE router, it is eligible to be installed 595 those of the PE's VRFs which are associated with Route Target T. 596 (Whether it actually gets installed depends on the outcome of the BGP 597 decision process.) 599 A Route Target attribute can be thought of as identifying a set of 600 sites. (Though it would be more precise to think of it as 601 identifying a set of VRFs.) Associating a particular Route Target 602 attribute with a route allows that route to be placed in the VRFs 603 that are used for routing traffic which is received from the 604 corresponding sites. 606 There is a set of Route Targets that a PE router attaches to a route 607 received from site S; these may be called the "Export Targets". And 608 there is a set of Route Targets that a PE router uses to determine 609 whether a route received from another PE router could be placed in 610 the VRF associated with site S; these may be called the "Import 611 Targets". The two sets are distinct, and need not be the same. Note 612 that a particular VPN-IPv4 route is only eligible for installation in 613 a particular VRF if there is some Route Target which is both one of 614 the route's Route Targets and one of the VRF's Import Targets. 616 The function performed by the Route Target attribute is similar to 617 that performed by the BGP Communities Attribute. However, the format 618 of the latter is inadequate for present purposes, since it allows 619 only a two-byte numbering space. It is desirable to structure the 620 format, similar to what we have described for RDs (see section 4.2), 621 so that a type field defines the length of an administrator field, 622 and the remainder of the attribute is a number from the specified 623 administrator's numbering space. This can be done using BGP Extended 624 Communities. The Route Targets discussed herein are encoded as BGP 625 Extended Community Route Targets [BGP-EXTCOMM]. 627 When a BGP speaker has received more than one route to the same VPN- 628 IPv4 prefix, the BGP rules for route preference are used to choose 629 which route are installed. 631 Note that a route can only have one RD, but it can have multiple 632 Route Targets. In BGP, scalability is improved if one has a single 633 route with multiple attributes, as opposed to multiple routes. One 634 could eliminate the Route Target attribute by creating more routes 635 (i.e., using more RDs), but the scaling properties would be less 636 favorable. 638 How does a PE determine which Route Target attributes to associate 639 with a given route? There are a number of different possible ways. 640 The PE might be configured to associate all routes that lead to a 641 particular site with a particular Route Target. Or the PE might be 642 configured to associate certain routes leading to a particular site 643 with one Route Target, and certain with another. Or the CE router, 644 when it distributes these routes to the PE (see section 7), might 645 specify one or more Route Targets for each route. The latter method 646 shifts the control of the mechanisms used to implement the VPN 647 policies from the SP to the customer. If this method is used, it may 648 still be desirable to have the PE eliminate any Route Targets that, 649 according to its own configuration, are not allowed, and/or to add in 650 some Route Targets that according to its own configuration are 651 mandatory. 653 4.3.2. Route Distribution Among PEs by BGP 655 If two sites of a VPN attach to PEs which are in the same Autonomous 656 System, the PEs can distribute VPN-IPv4 routes to each other by means 657 of an IBGP connection between them. Alternatively, each can have an 658 IBGP connection to a route reflector. 660 When a PE router distributes a VPN-IPv4 route via BGP, it uses its 661 own address as the "BGP next hop". This address is encoded as a 662 VPN-IPv4 address with an RD of 0. ([BGP-MP] requires that the next 663 hop address be in the same address family as the NLRI.) It also 664 assigns and distributes an MPLS label. (Essentially, PE routers 665 distribute not VPN-IPv4 routes, but Labeled VPN-IPv4 routes. Cf. 666 [MPLS-BGP]). When the PE processes a received packet that has this 667 label at the top of the stack, the PE will pop the stack, and process 668 the packet appropriately. 670 The PE may distribute the exact set of routes that appears in the 671 VRF, or it may perform summarization and distribute aggregates of 672 those routes, or it may do some of one and some of the other. 674 Suppose that a PE has assigned label L to route R, and has 675 distributed this label mapping via BGP. If R is an aggregate of a 676 set of routes in the VRF, the PE will know that packets from the 677 backbone which arrive with this label must have their destination 678 addresses looked up in a VRF. When the PE looks up the label in its 679 Label Information Base, it learns which VRF must be used. On the 680 other hand, if R is not an aggregate, then when the PE looks up the 681 label, it learns the output sub-interface and the data link 682 encapsulation header for the packet. In this case, no lookup in the 683 VRF is done. 685 We would expect that the most common case would be the case where the 686 route is NOT an aggregate. The case where it is an aggregate can be 687 very useful though if the VRF contains a large number of host routes 688 (e.g., as in dial-in), or if the VRF has an associated LAN interface 689 (where there is a different outgoing layer 2 header for each system 690 on the LAN, but a route is not distributed for each such system). 691 However, we do not consider this further in this paper. 693 Note that the use of BGP-distributed MPLS labels is only possible if 694 there is a label switched path between the PE router that installs 695 the BGP-distributed route and PE router which is the BGP next hop of 696 that route. This label switched path may follow a "best effort" 697 route, or it may follow a traffic engineered route. Between a 698 particular PE router and its BGP next hop for a particular route 699 there may be one label switched path, or there may be several, 700 perhaps with different QoS characteristics. All that matters for the 701 VPN architecture is that some label switched path between the router 702 and its BGP next hop exists. However, to ensure interoperability 703 among systems which implement this VPN architecture, all such systems 704 must support LDP [MPLS-LDP]. 706 A PE router, UNLESS it is a Route Reflector (see section 4.3.3) 707 should not install a VPN-IPv4 route unless it has at least one VRF 708 with an Import Target identical to one of the route's Route Target 709 attributes. Inbound filtering should be used to cause such routes to 710 be discarded. If a new Import Target is later added to one of the 711 PE's VRFs (a "VPN Join" operation), it must then acquire the routes 712 it may previously have discarded. This can be done using the refresh 713 mechanism described in [BGP-RFSH]. The outbound route filtering 714 mechanism of [BGP-ORF] can also be used to advantage to make the 715 filtering more dynamic. 717 Similarly, if a particular Import Target is no longer present in any 718 of a PE's VRFs (as a result of one or more "VPN Prune" operations), 719 the PE may discard all routes which, as a result, no longer have any 720 of the PE's VRF's Import Targets as one of their Route Target 721 Attributes. 723 A router which is not attached to any VPN, and which is not a Route 724 Reflector (i.e., a P router), never installs any VPN-IPv4 routes at 725 all. 727 Note that VPN Join and Prune operations are non-disruptive, and do 728 not require any BGP connections to be brought down, as long as the 729 refresh mechanism of [BGP-RFSH] is used. 731 As a result of these distribution rules, no one PE ever needs to 732 maintain all routes for all VPNs; this is an important scalability 733 consideration. 735 4.3.3. Use of Route Reflectors 737 Rather than having a complete IBGP mesh among the PEs, it is 738 advantageous to make use of BGP Route Reflectors [BGP-RR] to improve 739 scalability. All the usual techniques for using route reflectors to 740 improve scalability, e.g., route reflector hierarchies, are 741 available. 743 Route reflectors are the only systems which need to have routing 744 information for VPNs to which they are not directly attached. 745 However, there is no need to have any one route reflector know all 746 the VPN-IPv4 routes for all the VPNs supported by the backbone. 748 We outline below two different ways to partition the set of VPN-IPv4 749 routes among a set of route reflectors. 751 1. Each route reflector is preconfigured with a list of Route 752 Targets. For redundancy, more than one route reflector may be 753 preconfigured with the same list. A route reflector uses the 754 preconfigured list of Route Targets to construct its inbound 755 route filtering. The route reflector may use the techniques of 756 [BGP-ORF] to install on each of its peers (regardless of 757 whether the peer is another route reflector, or a PE) the set 758 of "Outbound Route Filters" (ORFs) that contain the list of its 759 preconfigured Route Targets. Note that route reflectors should 760 accept ORFs from other route reflectors, which means that route 761 reflectors should advertise the ORF capability to other route 762 reflectors. 764 A service provider may modify the list of preconfigured Route 765 Targets on a route reflector. When this is done, the route 766 reflector modifies the ORFs it installs on all of its IBGP 767 peers. To reduce the frequency of configuration changes on 768 route reflectors, each route reflector may be preconfigured 769 with a block of Route Targets. This way, when a new Route 770 Target is needed for a new VPN, there is already one or more 771 route reflectors that are (pre)configured with this Route 772 Target. 774 Unless a given PE is a client of all route reflectors, when a 775 new VPN is added to the PE ("VPN Join"), it will need to become 776 a client of the route reflector(s) that maintain routes for 777 that VPN. Likewise, deleting an existing VPN from the PE ("VPN 778 Prune") may result in a situation where the PE no longer need 779 to be a client of some route reflector(s). In either case, the 780 Join or Prune operation is non-disruptive (as long as [BGP- 781 RFSH] is used, and never requires a BGP connection to be 782 brought down, only to be brought right back up. 784 (By "adding a new VPN to a PE", we really mean adding a new 785 import Route Target to one of its VRFs, or adding a new VRF 786 with an import Route Target not had by any of the PE's other 787 VRFs.) 789 2. Another method is to have each PE be a client of some subset of 790 the route reflectors. A route reflector is not preconfigured 791 with the list of Route Targets, and does not perform inbound 792 route filtering of routes received from its clients (PEs); 793 rather it accepts all the routes received from all of its 794 clients (PEs). The route reflector keeps track of the set of 795 the Route Targets carried by all the routes it receives. When 796 the route reflector receives from its client a route with a 797 Route Target that is not in this set, this Route Target is 798 immediately added to the set. On the other hand, when the route 799 reflector no longer has any routes with a particular Route 800 Target that is in the set, the route reflector should delay (by 801 a few hours) the deletion of this Route Target from the set. 803 The route reflector uses this set to form the inbound route 804 filters that it applies to routes received from other route 805 reflectors. The route reflector may also use ORFs to install 806 the appropriate outbound route filtering on other route 807 reflectors. Just like with the first approach, a route 808 reflector should accept ORFs from other route reflectors. To 809 accomplish this, a route reflector advertises ORF capability to 810 other route reflectors. 812 When the route reflector changes the set, it should immediately 813 change its inbound route filtering. In addition, if the route 814 reflector uses ORFs, then the ORFs have to be immediately 815 changed to reflect the changes in the set. If the route 816 reflector doesn't use ORFs, and a new Route Target is added to 817 the set, the route reflector, after changing its inbound route 818 filtering, must issue BGP Refresh to other route reflectors. 820 The delay of "a few hours" mentioned above allows a route 821 reflector to hold onto routes with a given RT, even after it 822 loses the last of its clients which are interested in such 823 routes. This protects against the need to reacquire all such 824 routes if the clients' "disappearance" is only temporary. 826 With this procedure, VPN Join and Prune operations are also 827 non-disruptive. 829 In both of these procedures, a PE router which attaches to a 830 particular VPN "auto-discovers" the other PEs which attach to the 831 same VPN. When a new PE router is added, or when an existing PE 832 router attaches to a new VPN, no reconfiguration of other PE routers 833 is needed. 835 Just as there is no one PE router that needs to know all the VPN-IPv4 836 routes that are supported over the backbone, these distribution rules 837 ensure that there is no one RR which needs to know all the VPN-IPv4 838 routes that are supported over the backbone. As a result, the total 839 number of such routes that can be supported over the backbone is not 840 bounded by the capacity of any single device, and therefore can 841 increase virtually without bound. 843 4.3.4. How VPN-IPv4 NLRI is Carried in BGP 845 The BGP Multiprotocol Extensions [BGP-MP] are used to encode the 846 NLRI. If the AFI field is set to 1, and the SAFI field is set to 847 128, the NLRI is an MPLS-labeled VPN-IPv4 address. AFI 1 is used 848 since the network layer protocol associated with the NLRI is still 849 IP. Note that this VPN architecture does not require the capability 850 to distribute unlabeled VPN-IPv4 addresses. 852 In order for two BGP speakers to exchange labeled VPN-IPv4 NLRI, they 853 must use BGP Capabilities Negotiation to ensure that they both are 854 capable of properly processing such NLRI. This is done as specified 855 in [BGP-MP], by using capability code 1 (multiprotocol BGP), with an 856 AFI of 1 and an SAFI of 128. 858 The labeled VPN-IPv4 NLRI itself is encoded as specified in [MPLS- 859 BGP], where the prefix consists of an 8-byte RD followed by an IPv4 860 prefix. 862 4.3.5. Building VPNs using Route Targets 864 By setting up the Import Targets and Export Targets properly, one can 865 construct different kinds of VPNs. 867 Suppose it is desired to create a a fully meshed closed user group, 868 i.e., a set of sites where each can send traffic directly to the 869 other, but traffic cannot be sent to or received from other sites. 870 Then each site is associated with a VRF, a single Route Target 871 attribute is chosen, that Route Target is assigned to each VRF as 872 both the Import Target and the Export Target, and that Route Target 873 is not assigned to any other VRFs as either the Import Target or the 874 Export Target. 876 Alternatively, suppose one desired, for whatever reason, to create a 877 "hub and spoke" kind of VPN. This could be done by the use of two 878 Route Target values, one meaning "Hub" and one meaning "Spoke". At 879 the VRFs attached to the hub sites, "Hub" is the Export Target and 880 "Spoke" is the Import Target. At the VRFs attached to the spoke 881 site, "Hub" is the Import Target and "Spoke" is the Export Target. 883 Thus the methods for controlling the distribution of routing 884 information among various sets of sites are very flexible, which in 885 turn provides great flexibility in constructing VPNs. 887 4.3.6. Route Distribution Among VRFs in a Single PE 889 It is possible to distribute routes from one VRF to another, even if 890 both VRFs are in the same PE, even though in this case one cannot say 891 that the route has been distributed by BGP. Nevertheless, the 892 decision to distribute a particular route from one VRF to another 893 within a single PE is the same decision that would be made if the 894 VRFs were on different PEs. That is, it depends on the route target 895 attribute which is assigned to the route (or would be assigned if the 896 route were distributed by BGP), and the import target of the second 897 VRF. 899 5. Forwarding Across the Backbone 901 If the intermediate routers in the backbone do not have any 902 information about the routes to the VPNs, how are packets forwarded 903 from one VPN site to another? 905 This is done by means of MPLS with a two-level label stack. 907 PE routers (and ASBRs which redistribute VPN-IPv4 addresses) need to 908 insert /32 address prefixes for themselves into the IGP routing 909 tables of the backbone. This enables MPLS, at each node in the 910 backbone network, to assign a label corresponding to the route to 911 each PE router. To ensure interoperability among different 912 implementations, it is required to support LDP for setting up the 913 label switched paths across the backbone. However, other methods of 914 setting up these label switched paths are also possible. (Some of 915 these other methods may not require the presence of the /32 address 916 prefixes in the IGP.) 918 When a PE receives a packet from a CE device, it chooses a particular 919 VRF in which to look up the packet's destination address. This 920 choice is based on the packet's incoming sub-interface. Assume that 921 a match is found. As a result we learn a "next hop" and an "outgoing 922 sub-interface". 924 If the packet's outgoing sub-interface is associated with a VRF, then 925 the next hop is a CE device. The packet is sent directly to the CE 926 device. However, if the outgoing sub-interface and the incoming sub- 927 interface are associated with different VRFs, and the route which 928 best matches the destination address in the incoming sub-interface's 929 VRF is an aggregate of several routes in the outgoing sub-interface's 930 VRF, it may be necessary to look up the packet's destination address 931 in the VRF of the outgoing interface as well. 933 If the packet's outgoing sub-interface is NOT associated with a VRF, 934 then the packet must travel at least one hop through the backbone. 935 The packet thus has a "BGP Next Hop", and the BGP Next Hop will have 936 assigned a label for the route which best matches the packet's 937 destination address. This label is pushed onto the packet's label 938 stack, and becomes the bottom label. The packet will also have an 939 "IGP Next Hop", which is the next hop along the IGP route to the BGP 940 Next Hop. The IGP Next Hop will have assigned a label for the route 941 which best matches the address of the BGP Next Hop. This label gets 942 pushed on as the packet's top label. The packet is then forwarded to 943 the IGP next hop. (Of course, if the BGP Next Hop and the IGP Next 944 Hop are the same, and if penultimate hop popping is used, the packet 945 may be sent with only the BGP-supplied label.) 947 MPLS will then carry the packet across the backbone. The egress PE 948 router's treatment of the packet will depend on the label that was 949 first pushed on by the ingress PE. In many cases, the PE will be 950 able to determine, from this label, the sub-interface over which the 951 packet should be transmitted (to a CE device), as well as the proper 952 data link layer header for that interface. In other cases, the PE 953 may only be able to determine that the packet's destination address 954 needs to be looked up in a particular VRF before being forwarded to a 955 CE device. Information in the MPLS header itself, and/or information 956 associated with the label, may also be used to provide QoS on the 957 interface to the CE. In any event, when the packet finally gets to a 958 CE device, it will again be an ordinary unlabeled IP packet. 960 Note that it is the two-level labeling that makes it possible to keep 961 all the VPN routes out of the P routers, and this in turn is crucial 962 to ensuring the scalability of the model. The backbone does not even 963 need to have routes to the CEs, only to the PEs. 965 If it is necessary to carry VPN packets through a sequence of P 966 routers which do not support MPLS, the top label (which represents a 967 route to the BGP next hop) could in theory be replaced with an "MPLS 968 in IP (or in GRE or in IPsec, etc.)" encapsulation, where the IP 969 destination address is the address of the BGP next hop. The use of 970 such techniques is for further study. 972 6. Maintaining Proper Isolation of VPNs 974 To maintain proper isolation of one VPN from another, it is important 975 that no router in the backbone accept a labeled packet from any 976 adjacent non-backbone device unless the following two conditions 977 hold: 979 1. the label at the top of the label stack was actually 980 distributed by that backbone router to that non-backbone 981 device, and 983 2. the backbone router can determine that use of that label will 984 cause the packet to leave the backbone before any labels lower 985 in the stack will be inspected, and before the IP header will 986 be inspected. 988 The first condition ensure that any labeled packets received from 989 non-backbone routers have a legitimate and properly assigned label at 990 the top of the label stack. The second condition ensures that the 991 backbone routers will never look below that top label. Of course, 992 the simplest way to meet these two conditions is just to have the 993 backbone devices refuse to accept labeled packets from non-backbone 994 devices. 996 7. How PEs Learn Routes from CEs 998 The PE routers which attach to a particular VPN need to know, for 999 each of that VPN's sites, which addresses in that VPN are at each 1000 site. 1002 In the case where the CE device is a host or a switch, this set of 1003 addresses will generally be configured into the PE router attaching 1004 to that device. In the case where the CE device is a router, there 1005 are a number of possible ways that a PE router can obtain this set of 1006 addresses. 1008 The PE translates these addresses into VPN-IPv4 addresses, using a 1009 configured RD. The PE then treats these VPN-IPv4 routes as input to 1010 BGP. Routes from a site are not leaked into the backbone's IGP. 1012 Exactly which PE/CE route distribution techniques are possible 1013 depends on whether a particular CE is in a "transit VPN" or not. A 1014 "transit VPN" is one which contains a router that receives routes 1015 from a "third party" (i.e., from a router which is not in the VPN, 1016 but is not a PE router), and that redistributes those routes to a PE 1017 router. A VPN which is not a transit VPN is a "stub VPN". The vast 1018 majority of VPNs, including just about all corporate enterprise 1019 networks, would be expected to be "stubs" in this sense. 1021 The possible PE/CE distribution techniques are: 1023 1. Static routing (i.e., configuration) may be used. (This is 1024 likely to be useful only in stub VPNs.) 1026 2. PE and CE routers may be RIP peers, and the CE may use RIP to 1027 tell the PE router the set of address prefixes which are 1028 reachable at the CE router's site. When RIP is configured in 1029 the CE, care must be taken to ensure that address prefixes from 1030 other sites (i.e., address prefixes learned by the CE router 1031 from the PE router) are never advertised to the PE. More 1032 precisely: if a PE router, say PE1, receives a VPN-IPv4 route 1033 R1, and as a result distributes an IPv4 route R2 to a CE, then 1034 R2 must not be distributed back from that CE's site to a PE 1035 router, say PE2, (where PE1 and PE2 may be the same router or 1036 different routers), unless PE2 maps R2 to a VPN-IPv4 route 1037 which is different than (i.e., contains a different RD than) 1038 R1. 1040 3. The PE and CE routers may be OSPF peers. A PE router which is 1041 an OSPF peer of a CE router appears, to the CE router, to be an 1042 area 0 router. If a PE router is an OSPF peer of CE routers 1043 which are in distinct VPNs, the PE must of course be running 1044 multiple instances of OSPF. 1046 IPv4 routes which the PE learns from the CE via OSPF are 1047 redistributed into BGP as VPN-IPv4 routes. Extended community 1048 attributes are used to carry, along with the route, all the 1049 information needed to enable the route to be distributed to 1050 other CE routers in the VPN in the proper type of OSPF LSA. 1051 OSPF route tagging is used to ensure that routes received from 1052 the MPLS/BGP backbone are not sent back into the backbone. 1054 Specification of the complete set of procedures for the use of 1055 OSPF between PE and CE can be found in [VPN-OSPF]. 1057 4. The PE and CE routers may be BGP peers, and the CE router may 1058 use BGP (in particular, EBGP to tell the PE router the set of 1059 address prefixes which are at the CE router's site. (This 1060 technique can be used in stub VPNs or transit VPNs.) 1062 This technique has a number of advantages over the others: 1064 a) Unlike the IGP alternatives, this does not require the PE 1065 to run multiple routing algorithm instances in order to 1066 talk to multiple CEs 1068 b) BGP is explicitly designed for just this function: 1069 passing routing information between systems run by 1070 different administrations 1072 c) If the site contains "BGP backdoors", i.e., routers with 1073 BGP connections to routers other than PE routers, this 1074 procedure will work correctly in all circumstances. The 1075 other procedures may or may not work, depending on the 1076 precise circumstances. 1078 d) Use of BGP makes it easy for the CE to pass attributes of 1079 the routes to the PE. A complete specification of the 1080 set of attributes and their use is outside the scope of 1081 this document. However, some examples of the way this 1082 may be used are the following: 1084 - The CE may suggest a particular Route Target for each 1085 route, from among the Route Targets that the PE is 1086 authorized to attach to the route. The PE would then 1087 attach only the suggested Route Target, rather than 1088 the full set. This gives the CE administrator some 1089 dynamic control of the distribution of routes from 1090 the CE. 1092 - Additional types of Extended Community attributes may 1093 be defined, where the intention is to have those 1094 attributes passed transparently (i.e., without being 1095 changed by the PE routers) from CE to CE. This would 1096 allow CE administrators to implement additional route 1097 filtering, beyond that which is done by the PEs. 1098 This additional filtering would not require 1099 coordination with the SP. 1101 On the other hand, using BGP is likely to be something new for 1102 the CE administrators, except in the case where the customer 1103 itself is already an Internet Service Provider (ISP), or where 1104 the CE devices are managed by the SP. 1106 If a site is not in a transit VPN, note that it need not have a 1107 unique Autonomous System Number (ASN). Every CE whose site 1108 which is not in a transit VPN can use the same ASN. This can 1109 be chosen from the private ASN space, and it will be stripped 1110 out by the PE. Routing loops are prevented by use of the Site 1111 of Origin Attribute (see below). 1113 What if a set of sites constitute a transit VPN? This will 1114 generally be the case only if the VPN is itself an ISP's 1115 network, where the ISP is itself buying backbone services from 1116 another SP. The latter SP may be called a "Carrier's Carrier". 1117 In this case, the best way to provide the VPN is to have the CE 1118 routers support MPLS, and to use the technique described in 1119 section 9. 1121 When we do not need to distinguish among the different ways in which 1122 a PE can be informed of the address prefixes which exist at a given 1123 site, we will simply say that the PE has "learned" the routes from 1124 that site. 1126 Before a PE can redistribute a VPN-IPv4 route learned from a site, it 1127 must assign a Route Target attribute (see section 4.3.1) to the 1128 route, and it may assign a Site of Origin attribute to the route. 1130 The Site of Origin attribute, if used, is encoded as a Route Origin 1131 Extended Community [BGP-EXTCOMM]. The purpose of this attribute is 1132 to uniquely identify the set of routes learned from a particular 1133 site. This attribute is needed in some cases to ensure that a route 1134 learned from a particular site via a particular PE/CE connection is 1135 not distributed back to the site through a different PE/CE 1136 connection. It is particularly useful if BGP is being used as the 1137 PE/CE protocol, but different sites have not been assigned distinct 1138 ASNs. 1140 8. How CEs learn Routes from PEs 1142 In this section, we assume that the CE device is a router. 1144 If the PE places a particular route in the VRF it uses to route 1145 packets received from a particular CE, then in general, the PE may 1146 distribute that route to the CE. Of course the PE may distribute 1147 that route to the CE only if this is permitted by the rules of the 1148 PE/CE protocol. (For example, if a particular PE/CE protocol has 1149 "split horizon", certain routes in the VRF cannot be redistributed 1150 back to the CE.) We add one more restriction on the distribution of 1151 routes from PE to CE: if a route's Site of Origin attribute 1152 identifies a particular site, that route must never be redistributed 1153 to any CE at that site. 1155 In most cases, however, it will be sufficient for the PE to simply 1156 distribute the default route to the CE. (In some cases, it may even 1157 be sufficient for the CE to be configured with a default route 1158 pointing to the PE.) This will generally work at any site which does 1159 not itself need to distribute the default route to other sites. 1160 (E.g., if one site in a corporate VPN has the corporation's access to 1161 the Internet, that site might need to have default distributed to the 1162 other site, but one could not distribute default to that site 1163 itself.) 1165 Whatever procedure is used to distribute routes from CE to PE will 1166 also be used to distribute routes from PE to CE. 1168 9. Carriers' Carriers 1170 Sometimes a VPN may actually be the network of an ISP, with its own 1171 peering and routing policies. Sometimes a VPN may be the network of 1172 an SP which is offering VPN services in turn to its own customers. 1173 VPNs like these can also obtain backbone service from another SP, the 1174 "carrier's carrier", using essentially the same methods described in 1175 this document. In particular: 1177 - The CE routers should distribute to the PE routers ONLY those 1178 routes which are internal to the VPN. This allows the VPN to be 1179 handled as a stub VPN. 1181 - The CE routers should support MPLS, in that they should be able 1182 to receive labels from the PE routers, and send labeled packets 1183 to the PE routers. They do not need to distribute labels of 1184 their own though. 1186 - The PE routers should distribute, to the CE routers, labels for 1187 the routes they distribute to the CE routers. 1189 - Routers at the different sites should establish BGP connections 1190 among themselves for the purpose of exchanging external routes 1191 (i.e., routes which lead outside of the VPN). 1193 - All the external routes must be known to the CE routers. 1195 Then when a CE router looks up a packet's destination address, the 1196 routing lookup will resolve to an internal address, usually the 1197 address of the packet's BGP next hop. The CE labels the packet 1198 appropriately and sends the packet to the PE. 1200 In the above procedure, the CE routers are the only routers in the 1201 VPN which need to support MPLS. If, on the other hand, all the 1202 routers at a particular VPN site support MPLS, then it is no longer 1203 required that the CE routers know all the external routes. All that 1204 is required is that the external routes be known to whatever routers 1205 are responsible for putting the label stack on a hitherto unlabeled 1206 packet, and that there be label switched path that leads from those 1207 routers to their BGP peers at other sites. In this case, for each 1208 internal route that a CE router distributes to a PE router, it must 1209 also distribute a label. 1211 10. Inter-Provider Backbones 1213 What if two sites of a VPN are connected to different Autonomous 1214 Systems (e.g., because the sites are connected to different SPs)? 1215 The PE routers attached to that VPN will then not be able to maintain 1216 IBGP connections with each other, or with a common route reflector. 1217 Rather, there needs to be some way to use EBGP to distribute VPN-IPv4 1218 addresses. 1220 There are a number of different ways of handling this case, which we 1221 present in order of increasing scalability. 1223 a) VRF-to-VRF connections at the AS border routers. 1225 In this procedure, a PE router in one AS attaches directly to a 1226 PE router in another. The two PE routers will be attached by 1227 multiple sub-interfaces, at least one for each of the VPNs 1228 whose routes need to be passed from AS to AS. Each PE will 1229 treat the other as if it were a CE router. That is, the PEs 1230 associate each such sub-interface with a VRF, and use EBGP to 1231 distribute unlabeled IPv4 addresses to each other. 1233 This is a procedure that "just works", and that does not 1234 require MPLS at the border between ASes. However, it does not 1235 scale as well as the other procedures discussed below. 1237 b) EBGP redistribution of labeled VPN-IPv4 routes from AS to 1238 neighboring AS. 1240 In this procedure, the PE routers use IBGP to redistribute 1241 labeled VPN-IPv4 routes either to an Autonomous System Border 1242 Router (ASBR), or to a route reflector of which an ASBR is a 1243 client. The ASBR then uses EBGP to redistribute those labeled 1244 VPN-IPv4 routes to an ASBR in another AS, which in turn 1245 distributes them to the PE routers in that AS, or perhaps to 1246 another ASBR which in turn distributes them ... 1248 When using this procedure, VPN-IPv4 routes should only be 1249 accepted on EBGP connections at private peering points, as part 1250 of a trusted arrangement between SPs. VPN-IPv4 routes should 1251 neither be distributed to nor accepted from the public 1252 Internet, or from any BGP peers which are not trusted. An ASBR 1253 should never accept a labeled packet from an EBGP peer unless 1254 it has actually distributed the top label to that peer. 1256 If there are many VPNs having sites attached to different 1257 Autonomous Systems, there does not need to be a single ASBR 1258 between those two ASes which holds all the routes for all the 1259 VPNs; there can be multiple ASBRs, each of which holds only the 1260 routes for a particular subset of the VPNs. 1262 This procedure requires that there be a label switched path 1263 leading from a packet's ingress PE to its egress PE. Hence the 1264 appropriate trust relationships must exist between and among 1265 the set of ASes along the path. Also, there must be agreement 1266 among the set of SPs as to which border routers need to receive 1267 routes with which Route Targets. 1269 c) Multihop EBGP redistribution of labeled VPN-IPv4 routes between 1270 source and destination ASes, with EBGP redistribution of 1271 labeled IPv4 routes from AS to neighboring AS. 1273 In this procedure, VPN-IPv4 routes are neither maintained nor 1274 distributed by the ASBRs. An ASBR must maintain labeled IPv4 1275 /32 routes to the PE routers within its AS. It uses EBGP to 1276 distribute these routes to other ASes. ASBRs in any transit 1277 ASes will also have to use EBGP to pass along the labeled /32 1278 routes. This results in the creation of a label switched path 1279 from the ingress PE router to the egress PE router. Now PE 1280 routers in different ASes can establish multi-hop EBGP 1281 connections to each other, and can exchange VPN-IPv4 routes 1282 over those connections. 1284 If the /32 routes for the PE routers are made known to the P 1285 routers of each AS, everything works normally. If the /32 1286 routes for the PE routers are NOT made known to the P routers 1287 (other than the ASBRs), then this procedure requires a packet's 1288 ingress PE to put a three label stack on it. The bottom label 1289 is assigned by the egress PE, corresponding to the packet's 1290 destination address in a particular VRF. The middle label is 1291 assigned by the ASBR, corresponding to the /32 route to the 1292 egress PE. The top label is assigned by the ingress PE's IGP 1293 Next Hop, corresponding to the /32 route to the ASBR. 1295 To improve scalability, one can have the multi-hop EBGP 1296 connections exist only between a route reflector in one AS and 1297 a route reflector in another. (However, when the route 1298 reflectors distribute routes over this connection, they do not 1299 modify the BGP next hop attribute of the routes.) The actual 1300 PE routers would then only have IBGP connections to the route 1301 reflectors in their own AS. 1303 This procedure is very similar to the "Carrier's Carrier" 1304 procedures described in section 9. Like the previous procedure, 1305 it requires that there be a label switched path leading from a 1306 packet's ingress PE to its egress PE. 1308 11. Accessing the Internet from a VPN 1310 Many VPN sites will need to be able to access the public Internet, as 1311 well as to access other VPN sites. The following describes some of 1312 the alternative ways of doing this. 1314 1. In some VPNs, one or more of the sites will obtain Internet 1315 Access by means of an "Internet gateway" (perhaps a firewall) 1316 attached to a non-VRF interface to an ISP. The ISP may or may 1317 not be the same organization as the SP which is providing the 1318 VPN service. Traffic to/from the Internet gateway would then 1319 be routed according to the PE router's default forwarding 1320 table. 1322 In this case, the sites which have Internet Access may be 1323 distributing a default route to their PEs, which in turn 1324 redistribute it to other PEs and hence into other sites of the 1325 VPN. This provides Internet Access for all of the VPN's sites. 1327 In order to properly handle traffic from the Internet, the ISP 1328 must distribute, to the Internet, routes leading to addresses 1329 that are within the VPN. This is completely independent of any 1330 of the route distribution procedures described in this 1331 document. The internal structure of the VPN will in general 1332 not be visible from the Internet; such routes would simply lead 1333 to the non-VRF interface that attaches to the VPN's Internet 1334 gateway. 1336 In this model, there is no exchange of routes between a PE 1337 router's default forwarding table and any of its VRFs. VPN 1338 route distribution procedures and Internet route distribution 1339 procedures are completely independent. 1341 Note that although some sites of the VPN use a VRF interface to 1342 communicate with the Internet, ultimately all packets to/from 1343 the Internet traverse a non-VRF interface before 1344 leaving/entering the VPN, so we refer to this as "non-VRF 1345 Internet Access". 1347 Note that the PE router to which the non-VRF interface attaches 1348 does not necessarily need to maintain all the Internet routes 1349 in its default forwarding table. The default forwarding table 1350 could have as few as one route, "default", which leads to 1351 another router (probably an adjacent one) which has the 1352 Internet routes. A variation of this scheme is to tunnel 1353 packets received over the non-VRF interface from the PE router 1354 to another router, where this other router maintains the full 1355 set of Internet routes. 1357 2. Some VPNs may obtain Internet access via a VRF interface ("VRF 1358 Internet Access"). If a packet is received by a PE over a VRF 1359 interface, and if the packet's destination address does not 1360 match any route in the VRF, then it may be matched against the 1361 PE's default forwarding table. If a match is made there, the 1362 packet can be forwarded natively through the backbone to the 1363 Internet, instead of being forwarded by MPLS. 1365 In order for traffic to flow natively in the opposite direction 1366 (from Internet to VRF interface), some of the routes from the 1367 VRF must be exported to the Internet forwarding table. 1368 Needless to say, any such routes must correspond to globally 1369 unique addresses. 1371 In this scheme, the default forwarding table might have the 1372 full set of Internet routes, or it might have a little as a 1373 single default route leading to another router which does have 1374 the full set of Internet routes in its default forwarding 1375 table. 1377 3. Suppose the PE has the capability to store "non-VPN routes" in 1378 a VRF. If a packet's destination address matches a "non-VPN 1379 route", then the packet is transmitted natively, rather than 1380 being transmitted via MPLS. If the VRF contains a non-VPN 1381 default route, all packets for the public Internet will match 1382 it, and be forwarded natively to the default route's next hop. 1383 At that next hop, the packets' destination addresses will be 1384 looked up in the default forwarding table, and may match more 1385 specific routes. 1387 This technique would only be available if none of the CE 1388 routers is distributing a default route. 1390 4. It is also possible to obtain Internet access via a VRF 1391 interface by having the VRF contain the Internet routes. 1392 Compared with model 2, this eliminates the second lookup, but 1393 it has the disadvantage of requiring the Internet routes to be 1394 replicated in each such VRF. 1396 If this technique is used, the SP may want to make its 1397 interface to the Internet be a VRF interface, and to use the 1398 techniques of section 4 to distribute Internet routes, as VPN- 1399 IPv4 routes, to other VRFs. 1401 It should be clearly understood that by default, there is no exchange 1402 of routes between a VRF and the default forwarding table. This is 1403 done ONLY upon agreement between a customer and a SP, and only if it 1404 suits the customer's policies. 1406 12. Management VPNs 1408 This specification does not require that the sub-interface connecting 1409 a PE router and a CE router be a "numbered" interface. If it is a 1410 numbered interface, this specification allows the addresses assigned 1411 to the interface to come from either the address space of the VPN or 1412 the address space of the SP. 1414 If a CE router is being managed by the Service Provider, then the 1415 Service Provider will likely have a network management system which 1416 needs to be able to communicate with the CE router. In this case, 1417 the addresses assigned to the sub-interface connecting the CE and PE 1418 routers should come from the SP's address space, and should be unique 1419 within that space. The network management system should itself 1420 connect to a PE router (more precisely, be at a site which connects 1421 to a PE router) via a VRF interface. The address of the network 1422 management system will be exported to all VRFs which are associated 1423 with interfaces to CE routers that are managed by the SP. The 1424 addresses of the CE routers will be exported to the VRF associated 1425 with the Network Management system, but not to any other VRFs. 1427 This allows communication between CE and Network Management system, 1428 but does not allow any undesired communication to or among the CE 1429 routers. 1431 One way to ensure that the proper route import/exports are done is to 1432 use two Route Targets, call them T1 and T2. If a particular VRF 1433 interface attaches to a CE router that is managed by the SP, then 1434 that VRF is configured to: 1436 - import routes that have T1 attached to them, and 1438 - attach T2 to addresses assigned to each end of its VRF 1439 interfaces. 1441 If a particular VRF interface attaches to the SP's Network Management 1442 system, then that VRF is configured to attach T1 to the address of 1443 that system, and to import routes that have T2 attached to them. 1445 13. Security 1447 13.1. Data Plane 1449 By security in the "data plane", we mean protection against the 1450 following possibilities: 1452 - Packets from within a VPN travel to a site outside the VPN, other 1453 than in a manner consistent with the policies of the VPN. 1455 - Packets from outside a VPN enter one of the VPN's sites, other 1456 than in a manner consistent with the policies of the VPN. 1458 Under the following conditions: 1460 1. a backbone router does not accept labeled packets over a 1461 particular data link, unless it is known that that data link 1462 attaches only to trusted systems, or unless it is known that 1463 such packets will leave the backbone before the IP header or 1464 any labels lower in the stack will be inspected, and 1466 2. labeled VPN-IPv4 routes are not accepted from untrusted or 1467 unreliable routing peers, 1469 3. no successful attacks have been mounted on the control plane, 1471 the data plane security provided by this architecture is virtually 1472 identical to that provided to VPNs by Frame Relay or ATM backbones. 1473 If the devices under the control of the SP are properly configured, 1474 data will not enter or leave a VPN unless authorized to do so. 1476 Condition 1 above can be stated more precisely. One should discard a 1477 labeled packet received from a particular neighbor unless one of the 1478 following two conditions holds: 1480 - the packet's top label has a label value which the receiving 1481 system has distributed to that neighbor, or 1483 - the packet's top label has a label value which the receiving 1484 system has distributed to a system beyond that neighbor (i.e., 1485 when it is known that the path from the system to which the label 1486 was distributed to the receiving system may be via that 1487 neighbor). 1489 Condition 2 above is of most interest in the case of inter-provider 1490 VPNs (see section 10). For inter-provider VPNs constructed according 1491 to scheme b) of section 10, condition 2 is easily checked. (The 1492 issue of security when scheme c) of section 10 is used is for further 1493 study.) 1495 It is worth noting that the use of MPLS makes it much simpler to 1496 provide data plane security than might be possible if one attempted 1497 to use some form of IP tunneling in place of the MPLS outer label. 1498 It is a simple matter to have one's border routers refuse to accept a 1499 labeled packet unless the first of the above conditions applies to 1500 it. It is rather more difficult to configure a router to refuse to 1501 accept an IP packet if that packet is an IP tunnelled packet whose 1502 destination address is that of a PE router; certainly this is not 1503 impossible to do, but it has both management and performance 1504 implications. 1506 Note that if the PE routers support any "MPLS in IP" or "MPLS in GRE" 1507 or similar encapsulations, security is compromised unless either any 1508 such packets are filtered at the borders, or else some acceptable 1509 means of authentication (e.g., IPsec authentication) is carried in 1510 the packet itself. 1512 In the case where a number of CE routers attach to a PE router via a 1513 LAN interface, to ensure proper security, one of the following 1514 conditions must hold: 1516 1. All the CE routers on the LAN belong to the same VPN, or 1518 2. A trusted and secured LAN switch divides the LAN into multiple 1519 VLANs, with each VLAN containing only systems of a single VPN; 1520 in this case the switch will attach the appropriate VLAN tag to 1521 any packet before forwarding it to the PE router. 1523 Cryptographic privacy is not provided by this architecture, nor by 1524 Frame Relay or ATM VPNs. These architectures are all compatible with 1525 the use of cryptography on a CE-CE basis, if that is desired. 1527 The use of cryptography on a PE-PE basis is for further study. 1529 13.2. Control Plane 1531 The data plane security of the previous section depends on the 1532 security of the control plane. To ensure security, neither BGP nor 1533 LDP connections should be made with untrusted peers. The TCP/IP MD5 1534 authentication option should be used with both these protocols. The 1535 routing protocol within the SP's network should also be secured in a 1536 similar manner. 1538 13.3. Security of P and PE devices 1540 If the physical security of these devices is compromised, data plane 1541 security may also be compromised. 1543 The usual steps should be take to ensure that IP traffic from the 1544 public Internet cannot be used to modify the configuration of these 1545 devices, or to mount Denial of Service attacks on them. 1547 14. Quality of Service 1549 Although not the focus of this paper, Quality of Service is a key 1550 component of any VPN service. In MPLS/BGP VPNs, existing L3 QoS 1551 capabilities can be applied to labeled packets through the use of the 1552 "experimental" bits in the shim header [MPLS-ENCAPS], or, where ATM 1553 is used as the backbone, through the use of ATM QoS capabilities. 1554 The traffic engineering work discussed in [MPLS-RSVP] is also 1555 directly applicable to MPLS/BGP VPNs. Traffic engineering could even 1556 be used to establish label switched paths with particular QoS 1557 characteristics between particular pairs of sites, if that is 1558 desirable. Where an MPLS/BGP VPN spans multiple SPs, the 1559 architecture described in [PASTE] may be useful. An SP may apply 1560 either intserv or diffserv capabilities to a particular VPN, as 1561 appropriate. 1563 15. Scalability 1565 We have discussed scalability issues throughout this paper. In this 1566 section, we briefly summarize the main characteristics of our model 1567 with respect to scalability. 1569 The Service Provider backbone network consists of (a) PE routers, (b) 1570 BGP Route Reflectors, (c) P routers (which are neither PE routers nor 1571 Route Reflectors), and, in the case of multi-provider VPNs, (d) 1572 ASBRs. 1574 P routers do not maintain any VPN routes. In order to properly 1575 forward VPN traffic, the P routers need only maintain routes to the 1576 PE routers and the ASBRs. The use of two levels of labeling is what 1577 makes it possible to keep the VPN routes out of the P routers. 1579 A PE router maintains VPN routes, but only for those VPNs to which it 1580 is directly attached. 1582 Route reflectors can be partitioned among VPNs so that each partition 1583 carries routes for only a subset of the VPNs supported by the Service 1584 Provider. Thus no single route reflector is required to maintain 1585 routes for all VPNs. 1587 For inter-provider VPNs, if the ASBRs maintain and distribute VPN- 1588 IPv4 routes, then the ASBRs can be partitioned among VPNs in a 1589 similar manner, with the result that no single ASBR is required to 1590 maintain routes for all the inter-provider VPNs. If multi-hop EBGP 1591 is used, then the ASBRs need not maintain and distribute VPN-IPv4 1592 routes at all. 1594 As a result, no single component within the Service Provider network 1595 has to maintain all the routes for all the VPNs. So the total 1596 capacity of the network to support increasing numbers of VPNs is not 1597 limited by the capacity of any individual component. 1599 16. Intellectual Property Considerations 1601 Cisco Systems may seek patent or other intellectual property 1602 protection for some of all of the technologies disclosed in this 1603 document. If any standards arising from this document are or become 1604 protected by one or more patents assigned to Cisco Systems, Cisco 1605 intends to disclose those patents and license them on reasonable and 1606 non-discriminatory terms. 1608 17. Acknowledgments 1610 Significant contributions to this work have been made by Ravi 1611 Chandra, Dan Tappan and Bob Thomas. 1613 We also wish to thank Shantam Biswas for his review and 1614 contributions. 1616 18. Authors' Addresses 1618 Eric C. Rosen 1619 Cisco Systems, Inc. 1620 250 Apollo Drive 1621 Chelmsford, MA, 01824 1622 E-mail: erosen@cisco.com 1624 Yakov Rekhter 1625 Juniper Networks 1626 1194 N. Mathilda Avenue 1627 Sunnyvale, CA 94089 1628 E-mail: yakov@juniper.net 1630 Tony Bogovic 1631 Telcordia Technologies 1632 445 South Street, Room 1A264B 1633 Morristown, NJ 07960 1634 E-mail: tjb@research.telcordia.com 1636 Stephen John Brannon 1637 Swisscom AG 1638 Postfach 1570 1639 CH-8301 1640 Glattzentrum (Zuerich), Switzerland 1641 E-mail: stephen.brannon@swisscom.com 1643 Marco Carugi 1644 France Telecom / CNET Research Centre 1645 IP networks and services 1646 CNET/DAC/NTR 1647 Technopole Anticipa 1648 2, av. P. Marzin 1649 22307 Lannion 1650 E-mail: marco.carugi@cnet.francetelecom.fr 1652 Christopher J. Chase 1653 AT&T 1654 200 Laurel Ave 1655 Middletown, NJ 07748 1656 USA 1657 E-mail: chase@att.com 1658 Ting Wo Chung 1659 Bell Nexxia 1660 181 Bay Street 1661 Suite 350 1662 Toronto, Ontario 1663 M5J2T3 1664 E-mail: ting_wo.chung@bellnexxia.com 1666 Eric Dean 1667 Global One 1668 12490 Sunrise Valley Dr. 1669 Reston, VA 20170 USA 1670 E-mail: edean@gip.net 1672 Jeremy De Clercq 1673 Alcatel Network Strategy Group 1674 Francis Wellesplein 1 1675 2018 Antwerp, Belgium 1676 E-mail: jeremy.de_clercq@alcatel.be 1678 Luyuan Fang 1679 AT&T 1680 IP Backbone Architecture 1681 200 Laurel Ave. 1682 Middletown, NJ 07748 1683 E-mail: luyuanfang@att.com 1685 Paul Hitchin 1686 BT 1687 BT Adastral Park 1688 Martlesham Heath, 1689 Ipswich IP5 3RE 1690 UK 1691 E-mail: paul.hitchen@bt.com 1693 Manoj Leelanivas 1694 Juniper Networks, Inc. 1695 385 Ravendale Drive 1696 Mountain View, CA 94043 USA 1697 E-mail: manoj@juniper.net 1698 Dave Marshall 1699 Worldcom 1700 901 International Parkway 1701 Richardson, Texas 75081 1702 E-mail: dave.marshall@wcom.com 1704 Luca Martini 1705 Level 3 Communications, LLC. 1706 1025 Eldorado Blvd. 1707 Broomfield, CO, 80021 1708 E-mail: luca@level3.net 1710 Monique Jeanne Morrow 1711 Swisscom AG 1712 Postfach 1570 1713 CH-8301 1714 Glattzentrum (Zuerich), Switzerland 1715 E-mail: monique.morrow@swisscom.com 1717 Ravichander Vaidyanathan 1718 Telcordia Technologies 1719 445 South Street, Room 1C258B 1720 Morristown, NJ 07960 1721 E-mail: vravi@research.telcordia.com 1723 Adrian Smith 1724 BT 1725 BT Adastral Park 1726 Martlesham Heath, 1727 Ipswich IP5 3RE 1728 UK 1729 E-mail: adrian.ca.smith@bt.com 1731 Vijay Srinivasan 1732 1200 Bridge Parkway 1733 Redwood City, CA 94065 1734 E-mail: vsriniva@cosinecom.com 1735 Alain Vedrenne 1736 SITA EQUANT 1737 3100 Cumberland Blvd, Suite 200 1738 Atlanta, GA, 30339 USA 1739 Email:Alain.Vedrenne@sita.int 1740 Alain.Vedrenne@equant.com 1742 19. References 1744 [BGP-MP] Bates, Chandra, Katz, and Rekhter, "Multiprotocol Extensions 1745 for BGP4", June 2000, RFC 2858 1747 [BGP-EXTCOMM] Ramachandra, Tappan, "BGP Extended Communities 1748 Attribute", January 2001, work in progress 1750 [BGP-ORF] Chen, Rekhter, "Cooperative Route Filtering Capability for 1751 BGP-4", November 2000, work in progress 1753 [BGP-RFSH] Chen, "Route Refresh Capability for BGP-4", March 2000, 1754 RFC 2918 1756 [BGP-RR] Bates and Chandrasekaran, "BGP Route Reflection: An 1757 alternative to full mesh IBGP", RFC 2796, April 2000 1759 [IPSEC] Kent and Atkinson, "Security Architecture for the Internet 1760 Protocol", November 1998, RFC 2401 1762 [MPLS-ARCH] Rosen, Viswanathan, and Callon, "Multiprotocol Label 1763 Switching Architecture", RFC 3031, January 2001 1765 [MPLS-BGP] Rekhter and Rosen, "Carrying Label Information in BGP4", 1766 January 2001, work in progress 1768 [MPLS-LDP] Andersson, Doolan, Feldman, Fredette, Thomas, "LDP 1769 Specification", RFC 3036, January 2001 1771 [MPLS-ENCAPS] Rosen, Rekhter, Tappan, Farinacci, Fedorkow, Li, and 1772 Conta, "MPLS Label Stack Encoding", RFC 3032, January 2001 1774 [MPLS-RSVP] Awduche, Berger, Gan, Li, Srinavasan, Swallow, 1775 "Extensions to RSVP for LSP Tunnels", August 2000, work in progress 1777 [PASTE] Li and Rekhter, "A Provider Architecture for Differentiated 1778 Services and Traffic Engineering (PASTE)", RFC 2430, October 1998. 1780 [VPN-OSPF] Rosen and Psenak, "OSPF as the PE/CE Protocol in BGP/MPLS 1781 VPNs", February 2001, work in progress 1783 20. Full Copyright Statement 1785 Copyright (C) The Internet Society (2000). All Rights Reserved. 1787 This document and translations of it may be copied and furnished to 1788 others, and derivative works that comment on or otherwise explain it 1789 or assist in its implementation may be prepared, copied, published 1790 and distributed, in whole or in part, without restriction of any 1791 kind, provided that the above copyright notice and this paragraph are 1792 included on all such copies and derivative works. However, this 1793 document itself may not be modified in any way, such as by removing 1794 the copyright notice or references to the Internet Society or other 1795 Internet organizations, except as needed for the purpose of 1796 developing Internet standards in which case the procedures for 1797 copyrights defined in the Internet Standards process must be 1798 followed, or as required to translate it into languages other than 1799 English. 1801 The limited permissions granted above are perpetual and will not be 1802 revoked by the Internet Society or its successors or assigns. 1804 This document and the information contained herein is provided on an 1805 "AS IS" basis and THE INTERNET SOCIETY AND THE INTERNET ENGINEERING 1806 TASK FORCE DISCLAIMS ALL WARRANTIES, EXPRESS OR IMPLIED, INCLUDING 1807 BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE INFORMATION 1808 HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED WARRANTIES OF 1809 MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE.