idnits 2.17.00 (12 Aug 2021) /tmp/idnits35907/draft-km-industrial-internet-requirements-00.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- No issues found here. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year == Line 280 has weird spacing: '...unction code ...' -- The document date (June 10, 2021) is 338 days in the past. Is this intentional? Checking references for intended status: Informational ---------------------------------------------------------------------------- == Outdated reference: A later version (-03) exists of draft-jia-intarea-scenarios-problems-addressing-00 -- Obsolete informational reference (is this intentional?): RFC 4995 (ref. 'ROHC') (Obsoleted by RFC 5795) Summary: 0 errors (**), 0 flaws (~~), 3 warnings (==), 2 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 Independent Submission K. Makhijani 3 Internet-Draft L. Dong 4 Intended status: Informational Futurewei 5 Expires: December 12, 2021 June 10, 2021 7 Requirements and Scenarios for Industry Internet Addressing 8 draft-km-industrial-internet-requirements-00 10 Abstract 12 Industry Control Networks host a diverse set of non-internet 13 protocols for different purposes. Even though they operate in a 14 controlled environment, one end of industrial control applications 15 run over internet technologies (IT) and another over operational 16 technology (OT) protocols. This memo discusses the challenges and 17 requirements relating to converegence of OT and IT networks. One 18 particular problem in convergence is figuring out reachability 19 between the these networks. 21 Status of This Memo 23 This Internet-Draft is submitted in full conformance with the 24 provisions of BCP 78 and BCP 79. 26 Internet-Drafts are working documents of the Internet Engineering 27 Task Force (IETF). Note that other groups may also distribute 28 working documents as Internet-Drafts. The list of current Internet- 29 Drafts is at https://datatracker.ietf.org/drafts/current/. 31 Internet-Drafts are draft documents valid for a maximum of six months 32 and may be updated, replaced, or obsoleted by other documents at any 33 time. It is inappropriate to use Internet-Drafts as reference 34 material or to cite them other than as "work in progress." 36 This Internet-Draft will expire on December 12, 2021. 38 Copyright Notice 40 Copyright (c) 2021 IETF Trust and the persons identified as the 41 document authors. All rights reserved. 43 This document is subject to BCP 78 and the IETF Trust's Legal 44 Provisions Relating to IETF Documents 45 (https://trustee.ietf.org/license-info) in effect on the date of 46 publication of this document. Please review these documents 47 carefully, as they describe your rights and restrictions with respect 48 to this document. Code Components extracted from this document must 49 include Simplified BSD License text as described in Section 4.e of 50 the Trust Legal Provisions and are provided without warranty as 51 described in the Simplified BSD License. 53 Table of Contents 55 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 2 56 2. Terminology . . . . . . . . . . . . . . . . . . . . . . . . . 3 57 2.1. Acronymns . . . . . . . . . . . . . . . . . . . . . . . . 4 58 3. Industrial Network Reference Architecture . . . . . . . . . . 4 59 3.1. Communication Patterns . . . . . . . . . . . . . . . . . 5 60 3.2. Industry Control Network Nuances (current state) . . . . 5 61 4. Problem Statement . . . . . . . . . . . . . . . . . . . . . . 6 62 4.1. Heterogenity . . . . . . . . . . . . . . . . . . . . . . 7 63 4.2. Automation Impact . . . . . . . . . . . . . . . . . . . . 7 64 4.2.1. Scale . . . . . . . . . . . . . . . . . . . . . . . . 8 65 4.2.2. Stretch Control Fabric to Edge and Cloud . . . . . . 8 66 4.2.3. Reliability . . . . . . . . . . . . . . . . . . . . . 8 67 4.2.4. Resilience . . . . . . . . . . . . . . . . . . . . . 8 68 4.3. OT/IT Convergence . . . . . . . . . . . . . . . . . . . . 8 69 4.4. Data oriented networking . . . . . . . . . . . . . . . . 9 70 4.5. Virtualization . . . . . . . . . . . . . . . . . . . . . 9 71 5. Address Space Requirements . . . . . . . . . . . . . . . . . 9 72 5.1. Short Device Addressing . . . . . . . . . . . . . . . . . 9 73 5.2. Meaningful Addresses . . . . . . . . . . . . . . . . . . 10 74 5.3. Device name based Addresses . . . . . . . . . . . . . . 10 75 5.4. Adoption of Lean Network Layer . . . . . . . . . . . . . 10 76 5.5. Multi-semantic behavior . . . . . . . . . . . . . . . . . 10 77 5.6. Interoperability with IP-world machines . . . . . . . . . 11 78 6. Relationship with Activities in IETF . . . . . . . . . . . . 11 79 6.1. Deterministic Networks (DetNet WG) . . . . . . . . . . . 11 80 6.2. IoT OPS . . . . . . . . . . . . . . . . . . . . . . . . . 11 81 6.3. LPWAN . . . . . . . . . . . . . . . . . . . . . . . . . . 11 82 6.4. Recent Addressing related work . . . . . . . . . . . . . 12 83 7. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 12 84 8. Security Considerations . . . . . . . . . . . . . . . . . . . 12 85 9. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . 12 86 10. Informative References . . . . . . . . . . . . . . . . . . . 12 87 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 13 89 1. Introduction 91 An industry control network interconnects devices used to operate, 92 control and monitor physical equipment in industrial environments. 93 These networks are increasingly becoming complex as the emphasis on 94 convergence of OT/IT grows to improve the automation. On one side of 95 Industrial internet are the inventory management, supply chain and 96 simulation software and the other side are the control devices 97 operating on machines. Operational Technologies (OT) networks are 98 more often tied to set of non-internet protocols such as Modbus, 99 Profibus, CANbus, Profinet, etc. There are more than 100 different 100 protocols each with it's own packet format and are used in the 101 industry. 103 It is expected that integration between the IT and OT will provide 104 numerous benefits in terms of improved productivity, efficiency of 105 operations by providing end to end visibility and control. Industry 106 control applications also expect to operate at cloud scale by 107 virtualization of several modules (especially PLCs) leading to new 108 set of network requirements. 110 One aspect of industry control is the delivery of data associated 111 with the Real-time, deterministic and reliability characteristics 112 over local-area and wide-area networks. This type of inter- 113 operability functionality and study is already covered in DETNET 114 working group. The other aspect is rachability and interconnection 115 keeping heterogenity of communication interfaces and a variety of 116 services in mind. This doument focuses on the latter part only. 118 OT networks have been traditionally separate from the IT networks. 119 It allowed OT network experts to manage and control proceses without 120 much dependency on changes in the external networks. This is an 121 important to consideration when dealing with the industry control 122 networks to maintain them in a controlled environment leveraging the 123 limited-domain networks [LDN] concept for an independent network 124 control. 126 The purpose of this document is to discuss the reachability and 127 interconnection characteristics, challenges and new requirements 128 emerging from large-scale integration of IT and OT. 130 2. Terminology 132 o Industrial Control Networks: The indutrial control networks are 133 interconnection of equipments used for the operation, control or 134 monitoring of machines in the industry environment. It involves 135 different level of communications - between fieldbus devices, 136 digital controllers and software applications 138 o Industry Automation: Mechansims that enable machine to machine 139 communication by use of technologies that enable automatic control 140 and operation of industrial devices and processes leading to 141 minimizing human intervention. 143 o Human Machine Interface: An interface between the operator and the 144 machine. The communication interface relays I/O data back and 145 forth between an oeprator's terminal anf HMI software to control 146 and monitor equipment. 148 2.1. Acronymns 150 o HMI: Human Machine Interface 152 3. Industrial Network Reference Architecture 154 In the scope of this document the following reference industrial 155 network will be used to provide structure to the discussion. In the 156 Fig. Figure 1 below, a hierarchy of communications is shown. At the 157 lowest level, PLCs operate and control field devices; above that 158 Human Machine Interface (HMI) interconnects with different PLCs to 159 program and control underlying field devices. HMI itself, sends data 160 up to applications for consumption in that industry vertical. 162 +-+-+-+-+-+-+ 163 ^ | Data Apps |.... External business logic network 164 : +-+-+-+-+-+-+ : 165 : | : 166 v +-+-+-+-+-+-+ +-+-+-+-+--+ 167 | vendor A | |vendor B | Interconnection of 168 | controller| |controller| controllers (system integrators) 169 ^ +-+-+-+-+-+-+ +-+-+-+-+-+ 170 : | | 171 : +-+-+-+-+ +-+-++-+ 172 : | Net X | | Net Y| 173 v | PLCs | | PLCs |--+ device-controllers 174 ^ +-+-+-+-+ +-+-+--+ | 175 : | | | 176 : +-+-+ +-+-+ +-+-+ 177 v | | | | | | Field level devices 178 +-+-+ +-+-+ +-+-+ 180 Figure 1: Hierarchy of Functions Industrial Control Networks 182 Unlike commercial networks that uniformly run IP protocols, the 183 communication links run different protocols at along the different 184 level of the hierarchy. One of the key requirement from new 185 industrial applications is the integration of different types of 186 communication protocols including Modbus, Profinet, Profibus, 187 ControlNet, CANOpen etc. 189 A vertically integration system involves a network between the 190 external business applications and higher controllers (for e.g. 191 SCADA, HMI, or system integrators) is IP based. The second level of 192 networks between the controllers can be either IP or non-IP 193 (Profibus, BACNet, etc.). The lowest field-level networks between 194 industrial controllers and field-level may be any of the fieldbus or 195 device control protocols (More details of the industry networks can 196 be found in [SURV]). 198 3.1. Communication Patterns 200 The following communication patterns are commonly observed: 202 o controller to controller: A communication between multi-vendor 203 controller maybe required by system integrators to work in complex 204 systems. 206 o controller to field level devices: This is a fieldbus 207 communication between device such as I/O modules, motors, 208 controllers. This communication represent. 210 o Device to device: allows direct communication between wired 211 industrial devices and wireless devices to enhance automation use 212 cases. For an exmaple, use of camera to visually monitor and 213 detect anamolies in other devices. 215 o controller to compute: vertical communication between a controller 216 and compute integrates IP-based technologies with non-IP since OT 217 product systems and solutions are not connected with IP based 218 networks. 220 A certain level of inter-operability is required to exchange data 221 between the above endpoints from different vendors. One of the 222 challange is that Ethernet (which unifies IT standards) that's not 223 always possible in Industry networks. 225 3.2. Industry Control Network Nuances (current state) 227 The Industry control networks are engineered for the idustry 228 verticals they belong to and depict unique properties as below: 230 o location bound: The Control Device's location or the network they 231 are attached to is predetermined and changes rarely. However, the 232 network resources may not get efficiently utilized to avoid 233 contention between them. 235 o security by separation: Typically, security is enhanced by keeping 236 IT/OTnetworks separate. The operators control how data goes in 237 and out of a site through firewalls and policies. 239 o data growth: Even though the size of network remains the same, 240 data generated is much higher. For example, cameras installed for 241 visual inspection to determine the quality of manufactured product 242 generates a high bandwidth demand. 244 o Wired device constraints: A bulk of machines are over wired 245 network, their constraints vary from LPWAN and IoT devices which 246 is an active area of standardization work. device lifetime, or 247 power-requirements are not typical constraints. Instead direct 248 process control mechanisms are more important. 250 o Real-time behavior: The control devices require realtime as well 251 as deterministic behavior between a controller (such as an HMI 252 station) to the equipment. The DetNet working group covers 253 several aspects. 255 The goal of the document is not to reinvent the Industry control 256 infrastructure. See section Section 6 on related standards work. It 257 is meant to exclude the already covered by other WGs. 259 Since a device connects to network through its address, the document 260 explores different address specific nuances in control devies - such 261 as management, device discovery and integration requirements. This 262 document concerns with the identification of and role networks, 263 specifically from the organization of industry control devices. 265 The goal of this document is to outline some of the challenges and 266 improvement of connectivity aspects of Industry control networks. 268 4. Problem Statement 270 In industrial networks, a good number of devices still communicate 271 over a serial or field bus (although Ethernet is being gradually 272 adopted). The operations on these devices are performed by writing 273 provide direct access to operation-control. i.e what operation to 274 perform is embedded in the type of interface itself. For instance, 275 Profibus, Modbus networks are implicitly latency sensitive and short 276 control-command based. 278 ModBus 279 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 280 | address | Function code | data| 281 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 283 CANBus 285 +-+-+-+-+-+-+-+-+-+-+--+-+-+ 286 | message id | data | 287 +-+-+-+-+-+-+-+-+-+-+-++-+-+ 288 Profibus - todo. 290 Since they are localized in an area such as factory floor or a site, 291 such networks have evolved independently and are seperated from the 292 IT applications. The emerging trend requires a seamless integration 293 with intelligent software, sophisticated compute platforms and other 294 operational aspects as highlighted below: 296 4.1. Heterogenity 298 A typical industry control network has devices of different 299 communication interfaces such as Fieldbus (PROFIBUS, Modbus, and 300 HART), Ethernet (generic Ethernet/IP, PROFINET, and Modbus-TCP), and 301 also wireless (Bluetooth, Wireless HART, and IoT). These interfaces 302 vary at the physical and link layers and because they integrate with 303 their own application technologies providing interoperability between 304 these devices remains a challenge. This also makes difficult to 305 adopt to modern integration technologies. 307 Fieldbus client-server architecture is widely deployed. It delivers 308 commands deterministically from a controller to the device and vice- 309 a-versa. Interfaces of this kind have typically shorter addresses 310 (upto 256 devices on a single bus in Modbus). 312 Some of the servers also behave as protocol gateways and interconnect 313 different type of protocols. For example when a modbus device is 314 being controlled by a profinet server, an gateway function will 315 translate modbus data or encapsulate it over IP (if the controller 316 supports it). 318 In a Gateway-centric approach, gateways are in charge of protocol 319 translations between the devices with different interfaces. This 320 requires packing and unpacking of data in the source and destination 321 formats at the attached gateways. Note: As an example, a Modbus 322 device does not know whether to send command to Profibus PLC or 323 Modbus PLC. The gateway device attaches to performs the translation. 324 This is even worse with encapsulations, where the entire frame is 325 carried over IP. 327 This is not ideal for latency sensitive applications. Although 328 hardware wise, gateways need to be equipped with all the interface, 329 it is more efficient to only perform data link conversion. 331 4.2. Automation Impact 333 Automation of processes in industry relies on control sophisticated 334 technologies such as machine learning, big data, etc. with minimal 335 human intervention. Automation needs to support scale, reliability 336 and resiliance at large-scale. 338 4.2.1. Scale 340 Automation control at small scale applications with well defined task 341 has been possible. In order to improve production, and eliminate 342 stoppages and minimizing human intervention. 344 When the number or density of devices, and processes increase there 345 is a need to schedule, route, and coordinate over multiple control 346 environments. 348 4.2.2. Stretch Control Fabric to Edge and Cloud 350 The industry control networks can be extended to cloud or edge 351 compute platforms. Since these networks are not equipped with 352 compute intensive servers. Now extending the communication to the 353 edge and cloud nodes increases the distance requiring traditional L2 354 networks to be adopted to L3 network designs. 356 Design decisions will require to choose different transit strategies 357 (this maybe layer 1, 2, 3 technologies or even network slices). It 358 also influence the security policies. 360 4.2.3. Reliability 362 Production efficiency is inversely related to number of defects in a 363 process. System reliability is determined through measurements of 364 its instantaneous state. 366 Automation processes need to ensure that system is performing in an 367 expected state and is capable of reporting anamolies fast and 368 accurately (i.e. packet drops or jitter leading to poor quality 369 product). 371 4.2.4. Resilience 373 TBD. 375 4.3. OT/IT Convergence 377 Most of the factory floors are not equipped with IT servers to 378 perform compute intensive tasks. Yet an IP-based device need to 379 connect with non-IP interface to control those devices. 381 Often real-time response is necessary for example, in closed-loop 382 control systems direct communication is desired to avoid any 383 additional packet processing delay or overheades at the source and 384 destination gateways, equipping IP to all OT devices and abandoning 385 the existing investment and depolyment could result in the following 386 obvious problems. 388 o Many of the standard IP based protocols maybe too much overhead 389 for OT devices. 391 o Cannot preserve communication characteristics of devices 392 (different device addressing scheme, realtime, IRT, message 393 identifiers, Bus-like properties). 395 o It relies heavily on hierarchy network stack (network layer, 396 transport layer, application), where as OT devices do not have 397 any, they generally operate at data link layer or below. 399 4.4. Data oriented networking 401 Industry verticals keep data and control on the manufacturing floor, 402 on a closed system. There is no easy way to forward this data to 403 enterprise level software. On premise micro data centers or edge 404 computing are new infrastructure pieces that wil impact the design of 405 current industrial networks. 407 4.5. Virtualization 409 Traditional Industry control infrastructure is not virtualized. 410 Virtualization will enable deployment of new functionality in a 411 flexible manner. 413 o Virtual PLCs are considered an important component functionality 414 customization of digital-twin realization. 416 o virtualization enables edge and cloud native computing by moving 417 and instantiating workflows at different locations. 419 Implications that PLCs are no longer one-hop away. 421 5. Address Space Requirements 423 5.1. Short Device Addressing 425 Shorter addresses are inherent to industry control systems to provide 426 implicit determinism. 428 Note: The motivation for short address is to preseve the legacy 429 attributes of fieldbus control devices. It is not related low-power 430 or resource constraints. 432 A large volume of the messages are of sizes shorter than the size of 433 IP headers (v4, v6) themselves. The header tax will be very high 434 over industry control networks. 436 5.2. Meaningful Addresses 438 The industry control floors are built bottom-up. The devices are 439 carefully wired and connected to controllers. In a hierarchical 440 network design, a particular type of machine can be reached in a 441 structured manner by adding subnet or location to the address 442 structures. 444 5.3. Device name based Addresses 446 HMI might require human readable address that is undertandable to 447 human operators or application end users. For example, a device 448 address could be associated with its location, type of applications, 449 attached objects etc. The network needs to support the resolution 450 and routing based on such device addresses, which is more user 451 friendly. On the other hand, grouping devices based on their 452 addresses shall be easily implemented to enable group operation and 453 communication. 455 5.4. Adoption of Lean Network Layer 457 Challenge of Industrial network device address is that it 458 communicates to a physical device address. Traditionally, in a 459 limited environment there was no need for network layer or expressing 460 network specific service, access control. 462 o If a sensor is broken, it will require reprogramming of controller 463 and re-aligning with the new address. The benefit of network 464 layer, removes this restriction. 466 o Note that, using IP stack is not suitable because these devices 467 perform specific functions and any overhead in transport or large 468 addressing can add to processing delays. 470 o Several other IP suite protocols such as device discovery should 471 be revisited. 473 5.5. Multi-semantic behavior 475 OT networks, at least at site level are organized at much smaller 476 scale than typical IP-capable networks. There is in turn a fixed 477 hierarchy of networks w.r.t. location in a plant. 479 5.6. Interoperability with IP-world machines 481 To develop further on different type of address format support. From 482 smaller address of legacy devices to IT based applications with IP 483 address. 485 (OT-Address )--->(Industry Control)--->(IP-Address) 486 (control dev) ( network ) (application) 488 Preferably allow OT devices to understand IP-addresses for the 489 servers they connect to. 491 6. Relationship with Activities in IETF 493 6.1. Deterministic Networks (DetNet WG) 495 The Deterministic Networking (DetNet) [DETNET-ARCH] is working on 496 using IP for long-range connectivity with bounded latency in industry 497 control networks . Its data plane [DETNET-DP] takes care of 498 forwarding aspects and most close to Industry control networks but 499 the focus is on the controlled latency, low packet loss & delay 500 variation, and high reliability functions. Not dealing with 501 interconnection of devices. 503 In layer 2 domain, similar functionalty is convered by TSN Ethernet 504 [IEEE802.1TSNTG]. 506 6.2. IoT OPS 508 IoT operations group discusses device security, privacy, and 509 bootstrapping and device onboarding concepts. Among the device 510 provisioning one of the object is network identifier. We understand 511 that the IoT OPs does not exclude evaluation of industry IoT or 512 control devices requirements. Given the specific functions described 513 above it maybe necessary to configure more than an identifier, i.e. 514 server or controller information or specific address scope and 515 structure. 517 6.3. LPWAN 519 The LPWAN has focussed on low-power and constrained devices. There 520 are compression related approaches that may apply are [SCHC] or 521 [ROHC]. To be evaluated for process control devices. 523 6.4. Recent Addressing related work 525 Some of the work initiated on the addressing include solutions such 526 as [FlexIP], [Flexible_IP], [FHE], and [SOIP]. 528 Recently, a broader area of problem statement and challenges in 529 [CHALLEN]. 531 7. IANA Considerations 533 This document requires no actions from IANA. 535 8. Security Considerations 537 This document introduces no new security issues. 539 9. Acknowledgements 541 10. Informative References 543 [CHALLEN] Jia, Y., Trossen, D., Iannone, L., 3rd, D. E. E., and P. 544 Liu, "Challenging Scenarios and Problems in Internet 545 Addressing", draft-jia-intarea-scenarios-problems- 546 addressing-00 (work in progress), February 2021. 548 [DETNET-ARCH] 549 Finn, N., Thubert, P., Varga, B., and J. Farkas, 550 "Deterministic Networking Architecture", RFC 8655, 551 DOI 10.17487/RFC8655, October 2019, 552 . 554 [DETNET-DP] 555 Varga, B., Ed., Farkas, J., Berger, L., Fedyk, D., and S. 556 Bryant, "Deterministic Networking (DetNet) Data Plane: 557 IP", RFC 8939, DOI 10.17487/RFC8939, November 2020, 558 . 560 [FHE] Jiang, S., Li, G., and B. Carpenter, "Asymmetric IPv6 for 561 Resource-constrained IoT Networks", draft-jiang- 562 asymmetric-ipv6-04 (work in progress), November 2020. 564 [Flexible_IP] 565 Jia, Y., Chen, Z., and S. Jiang, "Flexible IP: An 566 Adaptable IP Address Structure", draft-jia-flex-ip- 567 address-structure-00 (work in progress), October 2020. 569 [FlexIP] Moskowitz, R., Li, G., and S. Ren, "FlexIP Addressing", 570 draft-moskowitz-flexip-addressing-00 (work in progress), 571 January 2019. 573 [IEEE802.1TSNTG] 574 "IEEE, "Time-Sensitive Networking (TSN) Task Group"", 575 2018, . 577 [LDN] Carpenter, B. and B. Liu, "Limited Domains and Internet 578 Protocols", RFC 8799, DOI 10.17487/RFC8799, July 2020, 579 . 581 [ROHC] Jonsson, L-E., Pelletier, G., and K. Sandlund, "The RObust 582 Header Compression (ROHC) Framework", RFC 4995, 583 DOI 10.17487/RFC4995, July 2007, 584 . 586 [SCHC] Minaburo, A., Toutain, L., Gomez, C., Barthel, D., and JC. 587 Zuniga, "SCHC: Generic Framework for Static Context Header 588 Compression and Fragmentation", RFC 8724, 589 DOI 10.17487/RFC8724, April 2020, 590 . 592 [SOIP] Carpenter, B., Jiang, S., and G. Li, "Service Oriented 593 Internet Protocol", draft-jiang-service-oriented-ip-03 594 (work in progress), May 2020. 596 [SURV] Galloway, B. and G. Hancke, "Introduction to Industrial 597 Control Networks", IEEE Communications Surveys & 598 Tutorials Vol. 15, pp. 860-880, 599 DOI 10.1109/surv.2012.071812.00124, 2013. 601 Authors' Addresses 603 Kiran Makhijani 604 Futurewei 606 Email: kiran.ietf@gmail.com 608 Lijun Dong 609 Futurewei 610 Central Expy 611 Santa Clara, CA 95050 612 United States of America 614 Email: lijun.dong@futurewei.com