CCIE R&S 400-101 V5.1 Exam: Describe basic software architecture differences between IOS and IOS XEPaul Adam
Overall Network Principles consists of three broad topics ie network theory the one that you’re looking at right now network implementation and operation and network troubleshooting now in this article I will be covering network theory related topics such as IOS versus IOS XE architecture so let’s dive into it!
- Describe basic software architecture differences between IOS and IOS XE
- Control plane and Forwarding plane
- Impact to troubleshooting and performances
- Excluding specific platform’s architecture
Cisco classic IOS has always had a monolithic software architecture going back 20-25 years which means that it is both downloaded and run as a single binary image where all processes share the same memory address space now monolithic and non-modular architecture leads to obviously no memory protection between processes as a result software defects in classic IOS code can potentially corrupt data used by other processes and this used to happen all the time going back again to decades it also has a run to completion scheduler for the processes which means that the kernel does not preempt a running process the process must take or must make a kernel call before other processes can be scheduled and get a chance to run now what that means this means that if a corrupt process gets scheduled and order or a process that’s already scheduled gets corrupted somehow you’re looking at basically a core dump and then IOS classic is essentially taking an exception likely and then rebooting the entire outer which means bringing down the entire device right so now you can understand obviously what led to the IOS XE as far as cisco is concerned now.
If you compare this with juniper or Junos OS then you know the Junos OS always been both modular and non model monolithic now if you when I look at it visually then referred to the diagram on the right so you can see looking at it from bottoms up you have a kernel on the classic IOS side then you have drivers for different modules then you have common infrastructure and then you have features such as net IPSec etc on the right side if you look at the IOS XE you can see that there is a kernel obviously there is a Linux kernel and then you got the module drivers on the end the common infrastructure but the thing to note here is that IOS now no longer in control of the hardware that it is running on IOS becomes IOSd or IOS demon in IOS XE and run simply as another process alongside everything else all of the other infrastructure software and it basically is this provides you know CLI look and feel as well as the common code features that we’re going to talk about later in the coming slides so it talks it does all of that but it is no longer in control of the hardware now that’s magical because as soon as you do that your platform becomes very stable you can restart processes without taking the IOS down and so and so forth and obviously you can host apps like for example you can see the LXE you can run a Linux container you can run a VM whatever you want to do because now you got a blank sheet where you have a kernel and then you can run whatever you want on top of it all right so that’s the big difference from where we have come from the classic IOS to IOS XE.
Just historically speaking IOS XE was not done in house inside Cisco it was actually in acquisition the Cisco made back in two thousand four or five time frame and from and there was called BCN networks now routing and other protocols run as IOS processes and contribute to to the formation of Reb routing information base this is process to generate to the final IP routing table which means fab forwarding information base and then obviously which is used subsequently for the actual forwarding function for the router to take place right this is again this is how it happens in the classic IOS and more or less the same in concept for IOS XE as well but where these how the RIB is it is derived or how the FIB is derived obviously is different because classic IOS is not modular and IOS XE is fully modular or at least modular enough that you can do ISSU etc now on routing platform with software based forwarding for example going back all the way to Cisco 7200 or 7500 or Cisco is ours the classic is ours are used to run classic IOS image most traffic handling used to be done at the interrupt level using Cisco CEF or Cisco Express forwarding in routers with hardware based forwarding such as the Cisco ASR 1000 or even a SR nine thousand or CRS 1 or NCS series the the IOS computes the FIB in software running on the route processor hardware which generally, is x86 CPUs that they run the the IOS there like as in case of IOS XE you’re talking about entire IOS blob is together and runs as a process on top of the Linux kernel in case of IOS XE are for example things are very different because IOS XR takes the modularity to the next level it actually breaks it down at the subsystem level which means OSPF is different from BGP and BGP is different from is alright so you can actually have an issue in IOS XE are in BGP that would definitely not impact the is or LSP of databases so there is no possibility of that in case of IOS XE conceptually that can still happen now those route processors are the one they where’re IOS is running computers of fib and then in case of hardware platform where the control and data planes are physically separated the fib is downloaded into the forwarding hardware which could be a network processor for example in case of Cisco ASR 1000 or it could also be an ASIC right as in the case may be is a 9000.
IOS XE is a POSIX based environment along with tons of other open-source software for the that basically makeup of the common drivers tools utilities that need needed to manage the system now in addition to the standard set of off-the-shelf drivers Iowa XE also includes a set of Cisco specific drivers and associated chassis platform management modules because IOS XE now at Cisco runs on a number of platform pretty much everything right other than the platform that runs IOS XE are for example Cisco ASR 9000 or crs-1 or NCS now on top of the base operating system the Linux and drivers IOS XE provides a comprehensive set of infrastructure modules which define how software is installed how processes are started and sequenced and when the booting is happening for example how HA the high availability and software upgrades are performed for example the ISSU you in service software upgrade the core application that runs on top of this new infrastructure as I mentioned earlier is the IOS demon or IOS feature said that and the CLI is running now as the process on top of the Linux kernel and no longer running as the OS on the entire hardware which used to happen as we talked back in the days with the software based browser like Cisco 7200 Cisco 7500 in you know and another is ISR G2 for example generation 2 platforms that used to run classic IOS.
By running Cisco IOS as a process obviously, the products are running it the benefit because you get the 20 years of the innovation that Cisco has done in terms of the look and feel of the CLI as well as the feature development at least at the common code level or the control plane and the service plane or non data plane level right extensive feature set for routing and switching platforms but at the same time you get to have the ISS you and the HA benefits of the hardware based platform so it is a good happy medium now IOS XE is not a new network OS per se it is basically an incarnation of classic IOS where role of classic IOS has been reduced to an application running on top of Linux kernel as you can see on the right side top the blue box now this approach obviously allows building a routing switching platform that used a variety of different data plane Hardware right the ASIC or Cisco’s QFP processor or CPP right you can use in-house processors and for data plane forwarding like you see the in the orange block I have the QFP the client drivers stuff and then you have the forwarding software and then running on its own kernel likewise the spa on the left side the yellow block you have a spa interface processor or sip and that is running its own kernel its own infrastructure software and then running each SPA Bay the full height is running its own driver so obviously this abstraction that IOS XE brought to the table back in the days with ASR 1000 platform opened up a whole set of new possibilities that were not possible with classic IOS in terms of the separation of the control and data plane and the IOS plane not just logically but physically separating those planes.
If you look at the blue block at the top the route processor you can also see now because IOS is simply running as a process you can actually run another IOS process next to it right and do an active and standby between those two IOS processes all right then obviously if you do if you take the IOS or remove a you know convert IOS into a process and then remove the drivers from the IOS and give IOS no more control of the actual underlying hardware then you need to have some kind of middleware software that manages your platform right at the route processor level and likewise all of those as you can see boards route processor or embedded service processor is spa the IOS the data and the control and they are connected together what you see as control messaging is basically happening on a geeky Gigabit Ethernet switch right that’s inside the system is called EOBC or Ethernet out-of-band Channel right so that’s where it’s basically nothing but a switch with hard-coded obviously MAC addresses and an IP addresses and they that switch is a gigabit switch and it allows the system components RP ESP or spa or even in other cases another platform either Nexus series or other platform.
EOBC allows all those components to do inter process communication alright now going into details a little bit if you look at an IOS XE package you can see the it basically has about six seven eight packages or sub packages so RP base anything that starts with RP obviously run on the RP board so RP base provides the operating system software for the route processor RP control is the control blame process that runs interface between Cisco IOS D and rest of the platform now RP access is about the the software that requires for router access right so it obviously the the logic to to kind of delineate the the access from the control is because that you can have a non canine and a canine version because canine obviously is restricted in some of the territories so you can have non canine because of the Department of Commerce restriction RPI OS provides the Cisco IOS software kernel which is we’re Cisco IOS software features are stored and run each consolidated package obviously has a different RP IOS sub package alright so RP IOS is basically where the IOSD blobs blob lives.
ESP is the forwarding processor software and control processes where everything is bundled together sip spa and sip base obviously they are for the IOS and they can they include the driver as well as the SIP you know the session initiation not the session initiation protocol bubble more the actual IO board right so the carrier card the actual card operating system and control process it’s not the it’s not the s sip as in the cisco voice or or open source voice protocol but it is the actual board that you variable you you plug in your SPAs right you know in for example in Cisco 7200 platform now obviously if you ever heard about Cisco routers or any routers for that matter you have heard about control plane and data plane now IOS XE obviously allows development of data plane ASICs outside the IOS instance and that allows you to you know program this whole thing in a standard ap is and then you can have a complete and clean control and data plane separation now it a cheese control and data plane separation through the introduction of some infrastructure software for example forwarding and feature manager FFM and also standard interface to the forwarding engine driver FED right from the forth at the data plane FFM provides a number of api’s to control plane processes now FFM programs the data plan using the FED and maintains forwarding state for the entire system the FED is the instantiation of the hardware driver for the data plane all right now back again this time we look at both the IOS classic IOS and the IOS XE.
Looking at them side by side to see what changed right because you already have gone over quite a bit about IOS XE and IOS so is classic IOS obviously is monolithic which means everything is one single blob if something happens any anything goes wrong inside one of the process for example run to completion model right if the process does not want to leave the the scheduler scheduled process doesn’t want to leave the system then you have an issue where eventually they’re going to be a watchdog kick off and then you’re gonna see the whole system reboot and this used to happen a lot with software forwarding platforms such as that now in case of IOS XE obviously that that doesn’t happen because IOS D which is equivalent of IOS in some ways as long as the control as far as CLI control plane and that management plane let’s say are concerned those are not monolithic control and data plane separation obviously exists in Cisco IOS XE and all the platform that runs is why IOS XE have a clean separation of control and data plane now the control and plane and data plane separation on the basis of IOS XE obviously they’re not just in software but as I said it also happens in hardware as well because all those platforms the run is XE can be considered as hardware based forwarding platform now more or less there is a feature parity in case of IOS XE version versus the IOS right because classic always lived in T train and M train but not right now in case of IOS XE obviously you have single train more or less obviously there is a different image for IOS XE if you’re running ASR 1000 or ASR 900 or whatever another newer IOS our platform there on IOS XE.
Now, in terms of troubleshooting and performance obviously there is the big difference when it comes to classic IOS versus IOS XE all right because now since the two platform are different that lead to architecture really different the classic IOS has versus the Iowa succeed that lead to significant difference in how you troubleshoot the platforms that run these either classic IOS versus the IOS XE so we can break down those differences in a few larger areas methodology so all software based platforms use obviously single set of troubleshooting approach gives of Io XE obviously you’re gonna have a different platform hardware and that’s the beauty of IOS XE that allows you to run different data plane hardware and because of that you’re going to have different show and debug platform see allies because platform can be different from one to another in case of classic IOS obviously you would have same show and debug commands and that’s fine across all of the platform now CLI look and feel of classic IOS is very much intact which means like you could run is our classic or you you’d be running the seven two hundred you wouldn’t be able to tell the difference in case of obviously I have a IOS XE there will be some platform specific see allies which will be different from one platform to the other and there may also be some difference in the how the show commands output is laid out to you as when you do a show or a debug in some cases files and locations obviously they’re very different so if you looking for crash dump that knows classic IOS used to go to internal boot flash by default now in case of IOS XE you can obviously configure it but there are there are big differences right the if they if the ASR 1000 you have a dump the spa driver crash temp will go to hard disk by default for example and then you have the separation of control data and the forwarding planes obviously they all those boards are separate physically as well so obviously the the dump will go to a local hard drive generally by default right and obviously IOS XE also has when it was introduced it carried a newer software package naming convention right so that’s there was some learning curve in terms of performing troubleshooting and knowing where the files are located what is the directory structure with the where the folders are etcetera right as opposed to classic IOS now level of difficulty obviously classic IOS was very well understood and it was you can consider that to be an upside I guess for software based platforms but in case of IOS XE initially when IOS XE was introduced obviously the level of difficulty was higher and there was a downside or cost of modularity right you can basically think of this as since you’re getting so much obviously you’re gonna have to take some kind of a step function right you’re gonna have to live with some sort of change so that was the case with IOS XE obviously the you get enormous benefit of very strong control planes that are running on multi-core x86 CPUs which was never possible with classic IOS and also running a different kind of forwarding plan which could be either net for processor or a sig base a lot of flexibility there which means you can have significant amount of packet per second you know gigabits per second kind of performance out of a platform but downside obviously a little bit of complexity that comes with the modular fully modular and non monolithic IOS.
Now, with IOS XE obviously there is a platform independent features as well so what that means is that there are features that are exactly same regardless of which IOS or IOS II platform you’re on historically inside Cisco the development of IOS was divided between two different groups right one group which used to be known as n s T G for example that when a network services Technology Group there used to develop the platform independent code which means the common code for example net or IOS you know firewall zone based firewall whatever some basic stuff that’s common SNMP whatever that kind of stuff and then you have platform dependent code which was obviously the routing protocol I think is another good example of platform independent code or features but on the PD side or platform dependent code you’re talking about code that’s where the feature is only available for some platform and not available for everybody else so what Cisco did is they divided those teams and obviously the business unit that ran or came up with those platform used to write or still ride the platform dependent code but the platform independent code is written by the centralized software entity inside Cisco so this was the structure now is XE allows the platform dependent code to be extracted out of a single monolithic image right so that simplified the life of the people on both sides because the independent platform independent quarters don’t need to do anything that platform dependent coders are doing right because they are just they know their focus is on control CLI look and feel the parts are whatever that kind of stuff right the actual control or that the actual data forwarding is going to happen for that given features going to be done by the platform guys so as long as the two teams are talking to each other it’s beautiful it works and not only Cisco but even other companies like vendors like Juniper same thing right you have a platform independent teams of the coders that are in a centralized entity and then you have platform dependent teams which are obviously separated out and scattered across the business unit so iris becomes IOSD and runs simply as a process top of linux kernel and there’s something we talked already now this provides obviously a more efficient software delivery model for both decentralized IOS common code teams as well as the platform developers since that the two can be developed package and release pretty much independently right now iris XE CLI looks and feels exactly like IOS if you have run it you know with some exceptions as we talk like show platform or debug platform CLI in that case obviously you’re gonna have to look at the actual platform nuances and that’s where the two kind of differ now in some cases show CLI output also reflects some behind-the-scene.
In terms of exam essentials, please remember that IOS XE allowed the development of data ASIC outside the IOS and that you can use standard APIs which can basically enforce control and data plane processing separation IOS XE accomplishes the complete separation of control and data plane through FFM and FED IOS XE can obviously host application outside the IOS context in fact the all of the infrastructure software is already running like applications outside the IOS right on top of the Linux kernel directly just like the IOSD or I was daemon so you can obviously run a VM or you can run docker container whatever you want on top of that in that environment now IOS XE control and data plan can utilize multi-core processors and this is something I covered earlier that I was XE among other things brought to the table that now you can have you no longer need to have a single core cpu for controlling data plane for example you can actually have multi-core CPUs x86 CPU for control which means now your all of your control processes and all of your internet routing tables etc you can you can have really massive stuff like you can have 32 gigabyte of DRAM with for core or eight core processors running the IOSD which is which is beautiful because now that allows you to really scale your control plane running for example RIP or you know EIGRP OSPF BGP etc and then at the same time on that the data plane you can utilize multi-core network processors either built in-house like CPP or taken or taken from a company like for example for routing platform taking from a company like Cavium right so you can have those off-the-shelf network processors as well.
Please post your questions or suggestions in the comments. I will be answering them in the next few days.