Makefile Quickies: How I Make Charts

My current general purpose workhorse is a Chromebook, a first generation Pixel to be exact. As my buddy Goekesmi likes to say, a Chromebook is a knife, an extremely good knife. Unfortunately, it isn’t much else, if I need tcpdump, I drop down to my FreeBSD beast, or my Kali rig, both of which weigh over 4 times as much as my beloved Pixel. Both are toolboxes, very excellent toolboxes, and they indeed contain very excellent knives, but they weigh a great deal. Since 90% of the work I do is on a computer geographically not near me, a Chromebook with an SSH client is more often than not, a very sufficient, and appreciably lightweight tool.

But I am also a huge fan of the UNIX command line way of doing things. I use LaTeX to make most of my documentation, and graphviz to make network diagrams and flowcharts. A Chromebook doesn’t exactly have a full UNIX userland installed, so I can’t just run dot locally. With my pension for working on remote machines in mind, here is how I make charts; I resort to a web server and a Makefile.

Makefiles are great for a variety of tasks, with the added benefit of allowing you to not remember much of anything, you just type make when you want a result. With that in mind, lets examine what led me to this course of action.

Since I cannot run the graphviz suite of awesome locally, I cannot expect to use a local image viewer to view my handiwork. And even if I could, viewing local content on a Chromebook is….. meh at best. It really is designed to be online, all the time, and it shows. The logical solution is to present the images via a web browser, because that is basically what a Chromebook is. I could manually run dot for each chart I make, and enable directory indexes on my apache server, so I can easily find and view them, one by one. Urgh, who wants to type that much, and click that much? Lets create a Makefile that not only generates all my charts, but also creates a simple webpage, so I can F5 to my heart’s content!

So here it is, quite simple, but I think it illustrates an excellent use of a Makefile that is slightly outside the box from their usual task.

OUTPUT="/path/to/html/folder/here"
all:
 /bin/rm -f $(OUTPUT)/*.png
 echo "<html><head><title>Graphs!</title></head><body>" > $(OUTPUT)/index.html
 @for iii in *.dot ; do \
 dot -Tpng $$iii > $(OUTPUT)/`basename -s .dot $$iii`.png ; \
 echo "<br><img src=`basename -s .dot $$iii`.png>" >> $(OUTPUT)/index.html ; \
 done
 echo "</body></html>" >> $(OUTPUT)/index.html \

That is all there is to it, this is how I make charts and diagrams. I tend to symlink this Makefile into whatever directory I am working on currently, so if I want to view older graphs, I just enter a project directory, and type ‘make’. It cleans up the old mess, and generates images for all the *.dot files in the current working directory. This also means that my base Makefile is consistent across uses, as I have been known to introduce, and usually remove, features from this little bit of scripty fun. At one point, I had it traversing sub directories, and generating a hierarchical series of html documents and images, but it was honestly overkill for my needs.

Hope you found this useful, maybe you’ve decided that the lowly Makefile deserves a closer look. I have found little tricks like this save me a great deal of time. Lets face it, time is the one commodity none of us will ever get back.

Semper Fi, kthanxbai!

Home Network of Doom Part II: VPLS for Maniacs

Now that you have your new mess of tangled wires, and miles of configuration out of the way, it is time to start firing up some useful services. My entire initial reason for learning about MPLS was to get VPLS. As a consequence, this entire series of articles revolves around building a giant, virtual switch that spans multiple L3 devices. This particular article will focus on getting basic VPLS working. I will go into filtering, load balancing, and other such complex topics at a later time. Conceptually, the end result of this article can be compared to a VLAN aware switch nestled comfortably on top of an existing IP network. Why this is awesome is that, once built, you have all the advantages of an Ethernet network delivered to the edge, with the ability to use layer 3 tools and techniques for management of the core.

Warning: SRX Limitations

When delivering Ethernet services end to end, a service provider would(should) concern themselves with loop detection. The most common way to achieve this would be to run rapid spanning tree protocol (RSTP). Another key piece would be dropping inbound BPDU frames from customer devices, or disabling the port entirely when BPDU frames are detected. This is done to prevent a customer from joining your spanning tree network, as spanning tree has no security options. RSTP over VPLS is not supported by the branch SRX platform, nor can you configure bpdu-block in any useful way for VPLS.

What you can do is configure a firewall rule to act as a basic BPDU Filter.

nuintari@garage-mpls4> show configuration firewall  
family vpls {
    filter flood-ctrl {
        term stp {
            from {
                destination-mac-address {
                    01:80:c2:00:00:00/48;
                }
            }
            then {
                count stp;
                discard;
            }
        }
        term a-stp {
            from {
                destination-mac-address {
                    01:80:c2:00:00:00/44;
                }
            }
            then {
                count a-stp;
                discard;
            }
        }
        term pvst {
            from {
                destination-mac-address {
                    01:00:0c:cc:cc:cd/48;
                }
            }
            then {
                count pvst;
                discard;
            }
        }
        term cdp {
            from {
                destination-mac-address {
                    01:00:0c:cc:cc:cc/48;
                }
            }
            then {
                count cdp;
                discard;
            }
        }
        term vlan-br {
            from {
                destination-mac-address {
                    01:00:0c:cd:cd:ce/48;
                }
            }
            then {
                count vlan-br;
                discard;
            }
        }
        term stp-upfast {
            from {
                destination-mac-address {
                    01:00:0c:cd:cd:cd/48;
                }
            }
            then {
                count stp-upfast;
                discard;
            }
        }
        term default {
            then accept;
        }
    }
}

Special thanks to Mike for this tip. I have included the set statements for this firewall here.

Simply apply it to a VPLS routing instance.

set routing-instances <instance-name> forwarding-options family vpls flood input flood-ctrl

With a little JunOS scripting, one could conceivably turn this into a form of BPDU Guard as well. Sadly, this does not address loop detection and protection. The other method to protect against loops would be to configure mac move protections, which again, is not supported by the Branch SRX platform. In short, don’t create any loops!

The Basic Layout

To the outside observer, my entire network behaves like a giant VLAN aware switch. I’ll demonstrate my configurations for a few devices spanning three of my more important VLANs. The whole thing looks something like this:

VLAN 1000 is my general purpose, internet access network. My workstations live here, the wifi access lives here, my various gadgets including my Roku and Chromecast live here. If I had a single LAN home like most people, it would be this network.

But since I am insane, I also maintain a network for servers on VLAN 1005. Okay, I actually maintain several, but VLAN 1005 is for all the basic services that the rest of the network needs to exist, such as DNS and DHCP.

I also maintain shared storage for servers, and my one UNIX desktop over a third VLAN, 1009. Basically a network of NFS and iSCSI traffic. This network is not handed up to the IP router, as there is no need.

I maintain about a dozen other VLANs, but for simplicity sake, I’ll be cutting them out of configurations, hopefully I don’t miss anything.

Before I even get into the config, yes, my home network is named ‘assylum,’ I name all my computers after mental disorders. Also yes, I am aware that it is not the correct spelling for asylum, it is a joke, try not to think too hard to get it. My proofreader mother thought it was hilarious.

Interface Configuration

The configurations for this entire mess are substantially shorter than the previous post, and I won’t be displaying each and every device’s configuration this time around. Interface examples will start with the IP router, and a few examples of PE interfaces.

For clarity sake, lets take a look at my IP Gateway’s configuration.

nuintari@headend-mpls3> show configuration interfaces ge-0/0/1
description core-mpls5:ge-0/0/0;
vlan-tagging;
unit 1000 {
    description assylum-nat-1000;
    vlan-id 1000;
    family inet {
        no-redirects;
        address 192.168.119.254/24 {
            primary;
            preferred;
        }
    }
}
unit 1005 {
    description assylum-srv-1005;
    vlan-id 1005;
    family inet {
        no-redirects;
        address 192.168.250.254/24 {
            primary;
            preferred;
        }
    }
}

Nothing special, just VLANs and IP. The router effectively acts as a CE device in MPLS parlance. I’ll skip any real explanation here, and assume you know how a router basically works and is configured.

Exactly opposite on that same wire is core-mpls5, the, “entrance” to my VPLS switching layer.

description headend-mpls3:ge-0/0/1;
flexible-vlan-tagging;
mtu 1624;
encapsulation vlan-vpls;
unit 1000 {
    description assylum-nat-1000;
    encapsulation vlan-vpls;
    vlan-id 1000;
    family vpls;
}
unit 1005 {
    description assylum-srv-1005;
    encapsulation vlan-vpls;
    vlan-id 1005;
    family vpls;
}

This is performing the role of a PE device in MPLS-ese. The only real thing you need to take in, is that we are declaring an encapsulation of vlan-vpls at both the interface, and the sub-interface level, as well as declaring family vpls on each sub-interface. This is all required, on platforms with greater interface flexibility such as the MX, you can have different types of sub-interfaces on the same physical port. Sadly, the Branch SRX platform isn’t that awesome. We have also declared flexible-vlan-tagging for the interface itself, which becomes more important later. This port, as it stands right now, could operate with just “vlan-tagging” declared on the interface, but that would constrain us later on, which I will touch on in a bit. In short, if you don’t need any particular configuration for any particular reason, stick with flexible, because it is just that, flexible.

For purposes of wrapping your head around it, think of it like a switch performing as a trunk port. For a counter-example, here is an EX2200 switch passing the same VLANs (plus my storage network) in a pure L2 setting:

nuintari@core-sw0> show configuration interfaces ge-0/0/0
description core-mpls5:ge-0/0/1;
unit 0 {
    family ethernet-switching {
        port-mode trunk;
        vlan {
            members [ assylum-nat-1000 assylum-srv-1005 assylum-nfs-1009 ];
        }
    }
}

Directly opposite that port on core-mpls5 looks like this:

nuintari@core-mpls5> show configuration interfaces
ge-0/0/2 {
    description core-sw0:ge-0/0/0;
    flexible-vlan-tagging;
    mtu 1624;
    encapsulation vlan-vpls;
    unit 1000 {
        description assylum-nat-1000;
        encapsulation vlan-vpls;
        vlan-id 1000;
        family vpls;
    }
    unit 1005 {
        description assylum-srv-1005;
        encapsulation vlan-vpls;
        vlan-id 1005;
        family vpls;
    }
    unit 1009 {
        description assylum-nfs-1009;
        encapsulation vlan-vpls;
        vlan-id 1009;
        family vpls;
    }
}

Pretty standard config so far, but what happens if you want to present an untagged ethernet frame directly on the PE device? My wireless access points are dumb, they don’t understand VLANs (but they have killer antenna), so I need to somehow configure a VPLS speaking interface to act like an access port on a VLAN aware switch. Question is, how do I do that?

The short answer is, you don’t. Asking VPLS to act exactly like a VLAN aware switch belies their true nature in that they aren’t exactly mirror clones. With that in mind, you can fake an access port reasonably well.

nuintari@garage-mpls4> show configuration interfaces
ge-0/0/2 {
    description assylum-wifi;
    flexible-vlan-tagging;
    native-vlan-id 1000;
    mtu 1624;
    encapsulation vlan-vpls;
    unit 1000 {
        description assylum-nat-1000;
        encapsulation vlan-vpls;
        vlan-id 1000;
        family vpls;
    }
}

In reality, this is no different than any other VPLS speaking interface, except we have added the ‘native-vlan-id’ statement.  There is another piece to this puzzle which we will cover in a bit, but what you are now performing is called ‘VLAN normalization’, and only works on interfaces in ‘flexible-vlan-tagging’ mode. The other important piece is in the routing instance configuration. This particular trick only works in VPLS instances where we declare a single VLAN. Fortunately, you can do this on a per case basis. In most provider networks, it would be most advantageous to provide a simple tube, whatever the customer puts on the wire, tags or no tags, comes out the other end the same. If you want to be play fake access port, you lose this ability. I think, this might be way wrong, I wrote this article, stopped fact checking, and posted it a month and a half later……

Routing Instances

The actual work of gluing the interfaces together across the network falls to the routing instances. There are three and a half major pieces that require a bit of explanation. The route-distinguisher, vrf-target, and site-range and site-identifier. The rest of the configuration is fairly self explanatory, this is the routing instance for vlan 1000, my general purpose network.

nuintari@garage-mpls4> show configuration routing-instances
assylum-nat-1000 {
    instance-type vpls;
    vlan-id 1000;
    interface ge-0/0/1.1000;
    interface ge-0/0/2.1000;
    route-distinguisher 10.0.7.4:1000;
    vrf-target target:65001:1000;
    forwarding-options {
        family vpls {
            flood {
                input flood-ctrl;
            }
        }
    }
    protocols {
        vpls {
            site-range 10;
            no-tunnel-services;
            site garage-mpls4:assylum-nat-1000:pe4 {
                site-identifier 4;
            }
        }
    }
}

The vrf-target is essentially, the identifier for the l2vpn. Each device taking part in the same VPLS VRF should have the same vrf-target. They take the form of target:<asn>:<uid>. Since I have been creating single vlan pseudowires, I have taken to using the VLAN I am flinging around as the identifier, but as single VLANs are not all VPLS can do, this is hardly a requirement. The ASN I used from the previous article in this series was 65001, so we use that here as well.

The route-distinguisher takes the form of <any ipv4 addr>:<uid>, but the common convention is to use this router’s loopback address, and again, I use the VLAN I am moving around on the pseudowire for the uid. The best way I can explain the difference between vrf-target and the route-distinguisher is, the vrf-target defines a VPLS instance, network wide, the route-distinguisher helps identify the individual members. With all analogies, there is more to it than that, but its a pretty small piece of VPLS. The RD matters much more when you are talking about MPLS l3vpn, and moving the same address space for disparate customers. If you elected to forego BGP signaling, and are doing all your signaling with LDP, this option is completely unnecessary.

The final pairing, site-range and site-identifier, serve to identify the individual members of the VPLS VRF internally, as well as dictate if and when pseudowires are formed. The site-range dictates the highest site-identifier that this instance will form a pseudowire against…… What? Okay, there are three common scenarios where this either really matters, or means next to nothing.

The most obvious is a point to point connection. With only two members, the site-range could be 2, with site-identifiers 1 and 2. Nice and self documenting, this VRF is a tube, what goes in one end comes out the other.

The next is the full mesh, where these settings matter the least. In a scenario where full pseudowires  between all sites are desired, one could opt to accept the default site-range (its 65,534) and use automatic-site-id in lieu of outright declaring anything. For the purposes of actually learning anything, I like to adjust all these options, lest I never figure out why you would ever want to.

Where this really matters is when a complex topology of hubs and spokes is the desired behavior. Edge devices that only connect to one or two devices in the core should not be wasting resources forming pseudowires across the network. I don’t own enough hardware to build this scenario(yet), but when I do, I’ll post some examples.

So this all being said, I standardized on a site-range of 10 for everything except the instance that carries my internet traffic. My cable modem sits in my family room near the TV, and is carried over my VPLS network to my core router in the server room, so a true point to point link, where I used a site-range of 2.  With that one exception in mind, I used my loopback addresses to determine site-identifiers. This won’t scale far, but helps keep me sane at home.

Diagnostics and a Rosetta Stone

Going along with our analogy that this is just one big VLAN aware switch, we will need some basic commands to verify the insanity we have created.

First of all, we want to see if our pseudowires have actually come up, which is pretty easy:

show vpls connections

Yes, for some reason, this one singular JunOS command comes with a built in legend. Easily ignored over SSH, really friggin annoying at 9600 bauds of serial cable love…… The most important thing to look for here is Remote PE: entries. A mis-configured instance, or a bifurcated network will instead show entries with things like “No connections found.”

Good:

Instance: assylum-nat-1000
Edge protection: Not-Primary
  Local site: core-mpls5:assylum-nat-1000:pe0 (5)
    connection-site Type St Time last up # Up trans
    1 rmt Up Mar 27 17:49:01 2017 1
      Remote PE: 10.0.7.1, Negotiated control-word: No
      Incoming label: 262153, Outgoing label: 262149
      Local interface: lsi.1048576, Status: Up, Encapsulation: VPLS
        Description: Intf - vpls assylum-nat-1000 local site 5 remote site 1
   2 rmt Up Mar 27 17:49:09 2017 1
     Remote PE: 10.0.7.2, Negotiated control-word: No
     Incoming label: 262154, Outgoing label: 262157
     Local interface: lsi.1048578, Status: Up, Encapsulation: VPLS
       Description: Intf - vpls assylum-nat-1000 local site 5 remote site 2
  4 rmt Up Mar 29 11:23:53 2017 1
    Remote PE: 10.0.7.4, Negotiated control-word: No
    Incoming label: 262156, Outgoing label: 262157
    Local interface: lsi.1048586, Status: Up, Encapsulation: VPLS
      Description: Intf - vpls assylum-nat-1000 local site 5 remote site 4

Bad:

Instance: assylum-sdc-1006
Edge protection: Not-Primary
  Local site: core-mpls5:assylum-sdc-1006:pe5 (5)
  Local site: core-mpls5:assylum-sdc-1006:pe10 (10)
No connections found.

How fortuitous! A VLAN for a dead project I have long since forgotten about, but have failed to rip out of the network layer! This also shows that a local instance can be more than a single site, more on that later….

The next thing you might care about is whether or not these instances are actually moving any real traffic, we have a command for that:

show vpls statistics

Basic counters, bits in, bits out, zeros probably mean bits are not moving. If I get to this point and get zeros from this command, it invariably means I have a typo somewhere, usually a mismatched VLAN.

Since this mess is effectively a giant, virtualized switch, you will probably want to view the MAC address table. If this was a pure L2 environment, this would suffice:

show ethernet-switching table

Sadly, this gets us nothing in a VPLS enviroment. Instead, you need to type out this handful:

show route forwarding-table family vpls

If you all you care about is MAC address entries, be prepared to visually filter a bit. Sometimes you just want to know the answer to a simple question like, “DO YOU FRIGGIN SEE THIS MAC?!?!?!” Plenty of other, occasionally very useful, information is presented here. This is another fine example of something that doesn’t paste well, so here is a text dump.

Some MAC entries have next-hops listed, others do not. In case you haven’t figured it out, this tells you if the MAC is directly attached, or comes form somewhere else in the VPLS network. Either way, it is kind enough to tell you which interface it came from. This is a fine time to yank a cable out, assuming you constructed a redundant path, and re-check the output of this command. If you have a mesh or any redundant sort, MACs will move, otherwise, they will time out and disappear.

Moof

I hate writing wrap ups. We have gone down the rabbit hole on MPLS/VPLS a bit further, and have something that is actually usable. In the next piece in this series, we will talk about load balancing, fault tolerance, and using L3 to troubleshoot L2, because directly troubleshooting L2 sucks, believe me I know. *mental note, write post on that subject, link it here (I’ll never remember that second part).*

 

Home Network of Doom Part I: MPLS Fundies

Before I can take you down the ridiculous rabbit hole that is my home network’s more hilarious features and failings, we must get some basics out of the way. I am not going to bore you with yet another explanation on P/PE/CE devices, and the various often vaguely described reasons for MPLS to even exist. You can google MPLS and read just about anything else on the subject if you want that information. Rather than reinventing that wheel here, my goal is to be a little entertaining, a little informative, and a great deal different from your average technology blog. When we are done with this three part series, you will have practical knowledge of how to build and maintain MPLS networks.

My home network consists of five major devices relevant to this article, all Juniper SRX Branch platform, which can be found for dirt cheap at any number of online marketplaces. Dear My Juniper sales rep, please don’t kill me, no one wants to buy a support contract that costs more than the hardware itself for a home lab. A fully functional SRX300 can be had for roughly $190 from a variety of sources. Four of my devices, a pair of SRX100s and two SRX300s, form a simple loop, and my sole SRX210 serves as my core router and firewall. All wired together, it looks like this:

The SRX210, labeled headend-mpls3, does not actually take part in any MPLS signaling(it used to), but is still connected to the MPLS network via OSPF for management purposes. Due to the per interface limitations of the branch SRX series, the simplest way to attach this device to the rest of the network was via a separate interface from the ‘NNI'(Network to Network Interface). No biggie, I have fastEthernet ports coming out my ears.

A Word on Numbers

You will notice that each device has a name, and a number, and that number coincides with its assigned loopback address. This is by design, and is the result of a decent amount of effort being expended so that I can be a little lazy. All point to point links in this diagram are /24 networks in the 10.0.0.0/8 address space. The final digit of each IP address in the MPLS mesh always ends in the same digit as the device’s loopback address. This allows me to speed read various command outputs, which may become more clear as this post, and subsequent articles are written. Suffice to say, this would never scale in a large network, but in a small to medium sized one, allows you to be a little extra lazy when interpreting command outputs. I actually came up with this approach for dealing with those god awful impossible to read IPv6 addresses, and “back ported” the concept. Like everything, it almost scales better in IPv6. It really doesn’t matter much, but it makes outputs of various diagnostic commands slightly easier to parse quickly. Your mileage will vary.

Limitations of the Branch SRX series

Sadly, the branch SRX series has a few limitations, I will point them out briefly whenever we hit them through this series of articles. The biggest one you must concern yourself with is that SRX devices do not ship with MPLS enabled. From the factory default, the SRX is first and foremost a firewall. Sadly, in order to make the SRX behave as an MPLS speaking device, you have to disable its primary role. To achieve this, delete your entire security stanza, and replace it with this configuration:

nuintari@core-mpls5> show configuration security
forwarding-options {
    family {
        inet6 {
            mode packet-based;
        }
        mpls {
            mode packet-based;
        }
        iso {
            mode packet-based;
        }
    }
}

The impact is that nothing else underneath the security stanza is available if you wish to go down the MPLS path. This is why my SRX210 no longer takes part in any MPLS speaking madness, its job is pure L3 router, firewall, and NAT. Trying to shoehorn it into being both a security device, and an MPLS device was a nightmare on its own. An additional limitation of the branch SRX platform is that the VPLS routing-instances cannot contain an Integrated Routing and Bridging interface (IRB). In simpler terms, you can’t drop IP services into a VPLS instance on a single SRX device, without doing some wackiness in terms of cabling. In even more simple terms, you can’t make a VPLS routing-instance behave like a routed trunk port feeding into a VLAN aware switch. My solution was to buy more hardware.

Thing 1: L3/IP

You can’t have an MPLS network without some layer 3. So, first things first, lets stand up the basic IP networks between our four MPLS speaking devices. Same base diagram as before, but with link/subnet IP address info:

From my selected subnets you can clearly see that I, like most humans, am a terrible source of entropy. I seem to like the number seven an awful lot.

My MPLS speaking interfaces look like this, and I’ll include the loopback interfaces for you:

fam-mpls1

fe-0/0/4 {
    description garage-mpls4:ge-0/0/5;
    vlan-tagging;
    mtu 1624;
    unit 23 {
        description mpls-lan-23;
        vlan-id 23;
        family inet {
            no-redirects;
            address 10.23.23.1/24 {
                primary;
                preferred;
            }
        }
        family mpls;
    }
}
fe-0/0/5 {
    description core-mpls2:fe-0/0/5;
    vlan-tagging;
    mtu 1624;   
    unit 90 {                           
        description mpls-wifi-90;       
        vlan-id 90;                     
        family inet {                   
            no-redirects;               
            address 10.77.0.1/24 {      
                primary;                
                preferred;              
            }                           
        }                               
        family mpls;                    
    }                                   
}
lo0 {
    unit 0 {
        description fam-mpls1;
        family inet {
            address 10.0.7.1/32 {
                primary;
                preferred;
            }
        }
        family mpls;
    }
}

core-mpls2

fe-0/0/5 {
    description fam-mpls1:fe-0/0/5
    vlan-tagging;
    mtu 1624;
    unit 90 {
        description mpls-wifi-90;
        vlan-id 90;
        family inet {
            address 10.77.0.2/24 {
                primary;
                preferred;
            }
        }
        family mpls;
    }
}

fe-0/0/7 {
    description core-mpls5:ge-0/0/5;
    vlan-tagging;
    mtu 1624;
    unit 19 {
        description mpls-ospf-19;
        vlan-id 19;
        family inet {
            no-redirects;
            address 10.19.19.2/24 {
                primary;
                preferred;
            }                           
        }                               
        family mpls;                    
    }                                   
}
lo0 {
    unit 0 {
        description core-mpls2;
        family inet {
            address 10.0.7.2/32 {
                primary;
                preferred;
            }
        }
        family mpls;
    }
}

headend-mpls3

headend-mpls3, despite its name, no longer speaks MPLS. For brevity’s sake, what little is left that is, lets skip it.

garage-mpls4

Yes, I have an SRX300 in my garage, do you not?

ge-0/0/4 {                              
    description core-mpls5:ge-0/0/1; 
    vlan-tagging;                       
    mtu 1624;                           
    unit 16 {                           
        description mpls-ospf-16;       
        vlan-id 16;                     
        family inet {                   
            no-redirects;               
            address 10.16.16.4/24 {     
                primary;                
                preferred;              
            }                           
        }                               
        family mpls;                    
    }                                   
}
ge-0/0/5 {                              
    description fam-mpls1:fe-0/0/4;     
    vlan-tagging;                       
    mtu 1624;                           
    unit 23 {                           
        description mpls-lan-23;        
        vlan-id 23;                     
        family inet {                   
            no-redirects;               
            address 10.23.23.4/24 {     
                primary;                
                preferred;              
            }                           
        }                               
        family mpls;                    
    }                                   
}
lo0 {                                   
    unit 0 {                            
        description garage-mpls4;       
        family inet {                   
            no-redirects;               
            address 10.0.7.4/32 {       
                primary;                
                preferred;              
            }                           
        }                               
        family mpls;                    
    }                                   
}

core-mpls5

ge-0/0/1 {
    description garage-mpls4:ge-0/0/4;
    vlan-tagging;
    mtu 1624;
    unit 16 {
        description mpls-ospf-16;
        vlan-id 16;
        family inet {
            no-redirects;
            address 10.16.16.5/24 {
                primary;
                preferred;
            }
        }
        family mpls;
    }
}
ge-0/0/5 {                              
    description core-mpls2:fe-0/0/7;    
    vlan-tagging;                       
    mtu 1624;                           
    unit 19 {                           
        description mpls-ospf-19;       
        vlan-id 19;                     
        family inet {                   
            no-redirects;               
            address 10.19.19.5/24 {     
                primary;                
                preferred;              
            }                           
        }                               
        family mpls;                    
    }                                   
}
lo0 {                                   
    unit 0 {                            
        description core-mpls5;         
        family inet {                   
            no-redirects;               
            address 10.0.7.5/32 {       
                primary;                
                preferred;              
            }                           
        }                               
        family mpls;                    
    }                                   
}

The observant reader will notice two key items here. The altered MTU, and MPLS signaling. Since MPLS incurs some packet size overhead, you need to either increase your MPLS speaking interface MTU, or reduce the MTU everywhere else. Trust me, do everything in your power to do the former, not the latter. Since my house is a mix of gigE and fastE, I selected an MTU of 1624, since that is the highest my various fastE devices could all agree on. If you can enable true jumbo frames, I say go nuts, but any increase with sufficient overhead for MPLS will suffice. If you are insane, you could do a mix, just understand that fragmentation is not a thing in a VPLS network. Solutions to this exist, they are not simple. You have been warned, keep your MTUs on the MPLS transport side high enough to fling the largest, maximum sized ethernet frame generated by any of your non MPLS speaking devices. O’Reilly Juniper Networks Warrior chapter four, is a great read on this very subject.

You will also, for obvious reasons, need to enable MPLS signaling on each interface. Do this now with the ‘family mpls’ statement. If you forget just one in even a fairly simple network, your eyes will be bleeding by the time you find the source of the mystery issue. Trust me, been there, done that, type it now.

Thing 2: Choose you Interior Gateway Protocol

You will need an IGP. Okay fine, you don’t technically need one, but if you decide to stand up an MPLS network with static routes, you are not only a fool, but you deserve all the pain you will receive. You could stand up RIP, but again, the pain, you deserve it. Not to mention the scorn of every real router jockey on the planet. EIGRP is Cisco proprietary, and should never be deployed unless you are bolting onto someone else’s EIGRP mess. IS-IS is fine, except that Juniper likes to charge for the license to light it on almost anything, and no real open source solutions exist to my knowledge. Besides, simply typing IS-IS is going to get this blog reviewed by the CIA. Imagine explaining IS-IS in your google search history to some NSA goons who already know about your secret ladyboy porn collection and addiction to My Little Pony.

IS-IS, ISIS, Isis, isis…….

That leaves OSPF as your only real choice for an IGP. That is okay, OSPF is easy and powerful. At least I think so, but I know the functional difference between a Stub and a NSSA area, and have been known to deploy both. Don’t worry, you only need area 0.0.0.0 today.  Here is my OSPF config on all four MPLS speakers:

fam-mpls1

traffic-engineering;
area 0.0.0.0 {
    interface lo0.0 {
        passive;
    }
    interface fe-0/0/5.90 {
        metric 300;
        priority 240;
    }
    interface fe-0/0/4.23 {
        metric 150;
        priority 240;
    }
}

core-mpls2

(ignore interface fe-0/0/6.70, its that management interface we keep not talking about)

traffic-engineering;
area 0.0.0.0 {
    interface lo0.0 {
        passive;
    }
    interface fe-0/0/5.90 {
        metric 300;
        priority 245;
    }
    interface fe-0/0/6.70 {
        metric 150;
        priority 245;
    }
    interface fe-0/0/7.19 {
        metric 150;
        priority 245;
    }
}

garage-mpls4

traffic-engineering;
area 0.0.0.0 {
    interface lo0.0 {
        passive;
    }
    interface ge-0/0/4.16 {
        metric 50;
        priority 248;
    }
    interface ge-0/0/5.23 {
        metric 300;
        priority 248;
    }
}

core-mpls5

traffic-engineering;
area 0.0.0.0 {
    interface lo0.0 {
        passive;
    }
    interface ge-0/0/1.16 {
        metric 50;
        priority 250;
    }
    interface ge-0/0/5.19 {
        metric 150;
        priority 250;
    }
}

Just a typical, single area OSPF setup.  The only MPLS specific thing here is the traffic-engineering statement. In a nutshell, this statement makes OSPF “MPLS aware.” What it really does is allow label switched path (LSP) metrics to be advertised via OSPF link state announcements (LSA). So WTF does that mean? The most functional meaning is that when used in conjunction with RSVP, RSVP signaled bandwidth reservations are flooded within the OSPF link state database. There are additional knobs under the traffic-engineering statement, but for right now, you probably just want the base level statement enabled on every MPLS speaking device.

Verification of OSPF

OSPF diagnostics are fairly simple:

nuintari@core-mpls5> show ospf neighbor
Address     Interface      State      ID         Pri   Dead
10.16.16.4  ge-0/0/1.16    Full       10.0.7.4   248   33
10.19.19.2  ge-0/0/5.19    Full       10.0.7.2   245   34

This command will tell you if an adjacency has been successfully formed or not. You are looking for Full states. If you have created some multi-point OSPF networks, the Designated Router (DR) and Backup Designated Router (BDR) will show a status of Full, anything else will report itself in 2-Way. If you get something other than this, bust out the OSPF docs and figure out what basic thing you did wrong. Helpful hint: You are mucking with MTUs to make room for MPLS overhead, if an OSPF adjacency won’t form, double check your MTU settings. OSPF insists on a consistent MTU.

Thing 3: BGP Signaling

Just like you can’t make a proper Italian sub without salami, you cannot make a true MPLS network without Border Gateway Protocol. Go ahead, google it, you’ll find discussion forums filled with experts telling someone with a bad idea not to follow through with their madness. And if I ever find someone selling Italian subs without salami, no jury would ever convict me.

If you plan to deliver L3VPN over your MPLS network, BGP is the only sane way to exchange routes with other networks. No one in their right mind will have any interest in running an IGP across autonomous network lines. For fine grained control of VPLS pseudowires, the flexibility afforded by the various knobs and buttons BGP provides is hard to beat.

The configuration is very simple, a standard internal BGP mesh, using one of the reserved ASNs. When using type internal, it isn’t nessesary to include the neighboring devices’ peer-as, but it doesn’t hurt anything either. Like my rock and roll, I like my configurations to be very explicit.

fam-mpls1

nuintari@fam-mpls1> show configuration protocols bgp
group mpls {
    type internal;
    local-address 10.0.7.1;
    family inet {
        unicast;
    }
    family l2vpn {
        signaling;
    }
    local-as 65001;
    neighbor 10.0.7.2 {
        peer-as 65001;
    }
    neighbor 10.0.7.4 {
        peer-as 65001;
    }
    neighbor 10.0.7.5 {
        peer-as 65001;
    }
}

core-mpls2

group mpls {
    type internal;
    local-address 10.0.7.2;
    family inet {
        unicast;
    }
    family l2vpn {
        signaling;
    }
    local-as 65001;
    neighbor 10.0.7.1 {
        peer-as 65001;
    }
    neighbor 10.0.7.4 {
        peer-as 65001;
    }
    neighbor 10.0.7.5 {
        peer-as 65001;
    }
}

garage-mpls4

group mpls {
    type internal;
    local-address 10.0.7.4;
    family inet {
        unicast;
    }
    family l2vpn {
        signaling;
    }
    local-as 65001;
    neighbor 10.0.7.1 {
        peer-as 65001;
    }
    neighbor 10.0.7.2 {
        peer-as 65001;
    }
    neighbor 10.0.7.5 {
        peer-as 65001;
    }
}

core-mpls5

group mpls {
    type internal;
    local-address 10.0.7.5;
    family inet {
        unicast;
    }
    family l2vpn {
        signaling;
    }
    neighbor 10.0.7.1 {
        peer-as 65001;
    }
    neighbor 10.0.7.4 {
        peer-as 65001;
    }
    neighbor 10.0.7.2 {
        peer-as 65001;
    }
}

The only non-conventional BGP thing here is enabling l2vpn signaling, since that is what I use MPLS for in my home. Many other options exist, but if you intend to follow this series of articles to completion, you will need l2vpn signaling enabled.

Verification of BGP

Much like OSPF, BGP is very easy to query, but has a singularly stupid piece of language that catches everyone.

nuintari@core-mpls5> show bgp summary

Oh dear lord, this output will never format well......

Okay, nothing I can paste into html formatting short of going all tables will do anything except make blood shoot out your nose. So here is a text dump of the output. A state of Establ(Established) is good…. a state of Active is not. I have no idea how we came to agree that Active means not working. I guess it is “actively looking for its peer”? All I know is, I have had to answer the question, “Its shows its active, how come it isn’t working?” far too often to junior networking initiates.

Thing 4: Labels

Now we come to the moment of decision, Resource Reservation Protocol (RSVP), or Label Distribution Protocol (LDP). LDP is basically low on features, but has a small configuration footprint. It is essentially label distribution autopilot. RSVP has a configuration complexity requirement due to the fact that Label Switched Paths (LSPs) are not automagically created for you. Ergo, as your RSVP network grows, the number of LSPs you will need to configure also rises. But at the same time, RSVP affords you control over how LSPs are created and utilized.

Larger networks reconcile this configuration scaling issue by running RSVP in the core, where the fine control is needed, and running LDP out on the edges. This level of complexity is beyond the scope of this series of articles. Google LDP Tunneling for more information.

I recommend starting with RSVP, as you will learn more. But for the sake of comparison, and that one idiot who plans on not listening to me, I will highlight what configuration options can be skipped when using LDP.

It should also be noted that the MPLS configuration in this section is not truly part of the RSVP configuration. Yet they are so closely intertwined, I find it useful to think of them at the same time.

Basic RSVP configuration itself is simple, any interface that is speaking family mpls at the interface layer, should be included in the RSVP configuration. The base configuration only requires interface statements, but I wanted to explain my internal network a bit better with bandwidth statements. Bandwidth statements will make more sense in part three of this series, when I get into advanced RSVP topics. For right now, consider them informational:

fam-mpls1

nuintari@fam-mpls1> show configuration protocols rsvp
interface lo0.0 {
    aggregate;
}
interface fe-0/0/5.90 {
    aggregate;
    bandwidth 54m;
}
interface fe-0/0/4.23 {
    aggregate;
    bandwidth 100m;
}

core-mpls2

nuintari@core-mpls2> show configuration protocols rsvp
interface lo0.0 {
    aggregate;
}
interface fe-0/0/7.19 {
    aggregate;
    bandwidth 100m;
}
interface fe-0/0/5.90 {
    aggregate;
    bandwidth 54m;
}

garage-mpls4

nuintari@garage-mpls4> show configuration protocols rsvp
interface lo0.0 {
    aggregate;
}
interface ge-0/0/4.16 {
    aggregate;
    bandwidth 1g;
}
interface ge-0/0/5.23 {
    aggregate;
    bandwidth 100m;
}

core-mpls5

nuintari@core-mpls5> show configuration protocols rsvp
interface lo0.0 {
    aggregate;
}
interface ge-0/0/1.16 {
    aggregate;
    bandwidth 1g;
}
interface ge-0/0/5.19 {
    aggregate;
    bandwidth 100m;
}

Most of my links are either fastE or gigE speeds, but I still have one link being handled by a piece of crap 802.11g ptp link provided by a cheap ass vendor whose name shall remain ubiquitous. It was free, okay? Once we go further down the MPLS rabbit hole, bandwidth statements in the RSVP stanza will make more sense.

The aggregate statement enables all RSVP refresh reduction features on the interface. RFC 2961 specifies some RSVP timer reduction methods, which modern JunOS completely supports. Most other vendors have also implemented these extensions. Unless you know for a fact you are a dealing with something that is either ancient, or just pathetic, you should be enabling aggregation, it does good things.

Verification of RSVP

One simple command for querying RSVP:

show rsvp neighbor
nuintari@core-mpls5> show rsvp neighbor
RSVP neighbor: 2 learned
Address       Idle Up/Dn  LastChange  HelloInt  HelloTx/Rx  MsgRcvd
10.16.16.4    5     2/1   1d 21:05:30         9  64263/64253 87912
10.19.19.2    5     1/0   6d 17:56:52         9  64399/64399 100936

All you care about is that all the neighbors you expect to see are indeed up. The basic configuration is so simple, if everything else works, and RSVP is still showing down, go find the typo.

For that one jerk who insists on running LDP instead, just set the interfaces with no additional configuration options, something like this:

nuintari@not-a-real-device> show configuration protocols ldp
interface lo0.0;
interface fe-1/2/3.45;
interface ge-6/7/8.90;

Thing 4.5: MPLS

The MPLS configuration when using LDP is almost identical, every interface we have created that will be taking part in the MPLS network must be listed here.

nuintari@not-a-real-device> show configuration protocols mpls
interface lo0.0;
interface fe-1/2/3.45;
interface ge-6/7/8.90;

Sound familiar? Didn’t you already light MPLS during the interface configuration steps? You did, but you still need to do it here as well. There is a reason for this, it is very, very outside the scope of this article, so lets not get lost down that path, okay?

If you have decided to listen to me, and begin your learning with RSVP signaling, you will not only need to declare MPLS speaking interfaces, but also every labeled switch path you might need. Everything we have done up to this point has been about how the network is physically constructed. LSPs allow you to control how that physical network is utilized by creating virtual channels that traverse the network according to your configuration. I will get into more detail in a later post.

To facilitate us hitting the ground running, I recommend creating a simple mesh of LSPs among your MPLS routers. No hard defined paths, no strict or loose statements, just connect every MPLS device to every other MPLS device with a single LSP via loopback addresses. Think of it sort of like a BGP mesh, just keep in the back of your mind that LSPs are unidirectional tubes, that will matter later on.

fam-mpls1

nuintari@fam-mpls1> show configuration protocols mpls
revert-timer 60;
label-switched-path fam-mpls1:core-mpls2 {
    from 10.0.7.1;
    to 10.0.7.2;
    link-protection;
}
label-switched-path fam-mpls1:garage-mpls4 {
    from 10.0.7.1;
    to 10.0.7.4;
    link-protection;
}
label-switched-path fam-mpls1:core-mpls5 {
    from 10.0.7.1;
    to 10.0.7.5;
    link-protection;
}
interface lo0.0;
interface fe-0/0/5.90;
interface fe-0/0/4.23;

core-mpls2

nuintari@core-mpls2> show configuration protocols mpls 
revert-timer 60;
label-switched-path core-mpls2:fam-mpls1 {
    from 10.0.7.2;
    to 10.0.7.1;
    link-protection;
}
label-switched-path core-mpls2:garage-mpls4 {
    from 10.0.7.2;
    to 10.0.7.4;
    link-protection;
}
label-switched-path core-mpls2:core-mpls5 {
    from 10.0.7.2;
    to 10.0.7.5;
    link-protection;
}
interface lo0.0;
interface fe-0/0/5.90;
interface fe-0/0/7.19;

garage-mpls4

nuintari@garage-mpls4> show configuration protocols mpls  
revert-timer 60;
label-switched-path garage-mpls4:core-mpls2 {
    from 10.0.7.4;
    to 10.0.7.2;
    link-protection;
}
label-switched-path garage-mpls4:fam-mpls1 {
    from 10.0.7.4;
    to 10.0.7.1;
    link-protection;
}
label-switched-path garage-mpls4:core-mpls5 {
    from 10.0.7.4;
    to 10.0.7.5;
    link-protection;
}
interface lo0.0;
interface ge-0/0/4.16;
interface ge-0/0/5.23;

core-mpls5

nuintari@core-mpls5> show configuration protocols mpls
revert-timer 60;
label-switched-path core-mpls5:core-mpls2 {
    from 10.0.7.5;
    to 10.0.7.2;
    link-protection;
}
label-switched-path core-mpls5:fam-mpls1 {
    from 10.0.7.5;
    to 10.0.7.1;
    link-protection;
}
label-switched-path core-mpls5:garage-mpls4 {
    from 10.0.7.5;
    to 10.0.7.4;
    link-protection;
}
interface ge-0/0/1.16;
interface lo0.0;
interface ge-0/0/5.19;

I actually have created several other LSPs that accurately describe my internal routing and switching policy, and I have policy statements that change how they are utilized. But in the name of baby steps, I have removed them from my displayed configurations. I do not recommend trying to go from zero to eleven in one day.

Verification of MPLS

The most basic question to ask MPLS is the state of LSPs, which can be easily queried with the following command:

show mpls lsp

This one doesn’t format well either, so here is the text version. There are three kinds of LSPs, ingress, egress, and transit. Ingress LSPs are LSPs that are originating from the current device. Ingress means entering the MPLS network, not entering the device. Egress LSPs are ones that are leaving the the MPLS network on this device. Transit LSPs are simply passing through. If you have taken my advice thus far, you will likely see a hodgepodge of all three types on most of your devices. We haven’t gotten into how IGP metrics affect any of this mania just yet, nor have we created any path statements to force any LSPs to go any particular direction. For right now, all you should concern yourself with, is that every LSP you created on each router is accounted for, and up.

The Final Anaysis

If you have come this far, you have a way cool network that delivers absolutely nothing useful. But you have built a foundation. One that can be used to stand up a variety of useful services. For now, I suggest a beer, I like IPAs, go get one of those. In part two of this series we will jump into basic VPLS, or, how to make your house into one giant, virtual switch.

The Triumphant Return of Nuintari the Great

In a discussion with some of my hacker friends, I mentioned that my home network is a giant virtualized switch in the form of VPLS. The ensuring discussion of Roku issues I was having due to my over-engineered house got me this reaction:

The ensuing discussion lead me to the decide to start blogging again. This of course lead me down a huge rabbit hole of securing a decent hosting option. Obnoxious, since I no longer work for the Internet, and no longer get these things for free. But I digress, talking about networking is pretty much why I got this site back up and running. I also have some infosec and sysadmin topics in the works, but I wanted to get the ball rolling with my true passion, which is networks. Unlike a lot of technical blogs, I intend for this to be as entertaining as it is informative. I also hope to avoid getting into the rut of just posting the steps. I intend to explain why you need to do various things, or at least why you should.

None of this was made any easier by the day job, which had me furiously working away at something huge. Said something huge is nearly finished, but I still can’t talk about it. Since my life has quieted down significantly at least in the short term, I thought I should get a few articles written. Part one of the hacker house of MPLS doom is in the pipe right after this meta post, so stay tuned!

@lil_lost

My dear friend Lauren needs a bio for her upcoming stint at BSides Nashville. Here is my attempt to commit greatness such as her own, into mere mortal words.

Her Ladyship Baroness Lauren Abrielle Cherlene Griddlebone McHackingston ne Bonaparte I, Esquire. With an ancestry to match her impressive moniker, Lauren was born to Marie Curie and Napoleon Bonaparte during the former’s brief tenure as a theoretical temporal scientist. Raised by a cadre of unicorns and LISP programmers, Lauren grew into one of the world’s foremost experts in the fields of fluffy pink pastel fashion, doom enhanced information security, and equestrian para-genetics.

A member of many esteemed societies, Lauren holds many titles. The Almighty Church of Wifi has proclaimed her The Most Fabulous Unholy Unicorn Empress. She is an honorary member of The Esteemed Order of Awesomeness, Second Class. She is also a a 774th Degree Dark Paladin of the Fabled Unicerate Equestrian UNIX Order. During the mid 1990s, Lauren briefly performed with famous rockers AC/DC, while bassist Cliff Williams was recovering from a bout of hysterical pregnancy.

Lauren is best known for her ongoing work to breed a race of cybernetically enhanced, hacker unicorns. One day, we can all expect this harbinger of infosec doom to rule over hackerkind with her legion of Metasploit enhanced, cyber pink ponies. I, for one, say that day cannot arrive soon enough. Hail Pink Unicorns!

My Adoring Public Has Spoken

The Twitter poll garnered a bit more than the 30 or so votes I was expecting.

Apparently, people want Infosec now now now. I also had a few requests for specific subjects, most relating to my extensive time working in the ISP world, as well as one suggestion that I rant aimlessly with little to no editorial process behind it. I can do these! I appreciate all the input, and votes. Thanks Twitter peeps!