Junos Groups Part I: Basics

On my many IT adventures, I see issues, one of the biggest, is lack of network consistency. Network ports configured one way, others configured another, VLANs trunked to parts unknown, none alike, even if they share the same basic role.

Juniper Junos has a wonderful tool, that seems incredibly underused, that largely resolves this: groups. Groups are awesome, they enforce consistency, they reduce typing, and they make configs shorter, and easier to read.  Here is a quick example of how to use them to manage VLANs on a switch.

Groups are basically templates of configuration settings that can be layered on top of any section of the Junos configuration with the apply-groups statement. There is some globbing support, allowing you to fine grain control when groups are applied. For more on that, check this Juniper article.

Groups essentially follow any subsection of the Junos configuration stanza, with matching patterns in place of variable data, such as interface names, ASNs, OSPF areas, etc. Here are two groups that apply Ethernet settings, namely member VLANs, and port mode, to an any interface they are applied to.

 

In order to apply these groups, we can apply them directly to a few interfaces with the apply-groups statement:

 

In order to see and verify the applied group settings, we pipe the show configuration command to display inheritance:

 

Using groups, one can greatly simplify the configuration of a Junos device, while at the same time enforcing consistency. Groups are not limited to interfaces, and can be applied to virtually any section of the Junos configuration.  In the next part of this series, I will display some more complex examples. Please check back soon!

The Lawnmower/Stereo/Bikini/Chicken/Sig Sauer/Beer Incident

I have told this story many times over the years, but never actually tried to write it down.

Framing Elements

A long time ago, before portable compute devices with more horsepower than anyone ever needed were commonplace, and long before the first Bluetooth speakers, there was a man with a dream. Or maybe it was a woman, I suppose you really need to figure out what I was wearing the day I came up with this idea, and what you think of gender identity issues, and whatever, it was me, okay? I had the dream. It wasn’t a grand dream, it will never change the world, but it was fun, and it was mine!

The dream was a riding lawnmower with a kick ass stereo system, that had wireless, and streamed MP3s from my home file server, and operated completely hands free.

The hands free part was either a design goal, or just me admitting that I had no desire to get X working on a portable LCD screen, much less pay for such a beast in 2003, back when computer hardware was far less disposable, and rarely cheap. At this point, I think I still had 486s in service at home, and at $dayJob. In fact I know I did, the original billmax.wedroppackets.net was an AMD 486 piece of shit from hell.

The lawn mower was, and is to this day, a Toro Z4200 Timecutter. A zero turn radius, 42 inch deck, gas guzzling monster, that cuts through my 1 acre property in less than an hour……. I call her Rachel.

The Stereo

So, with the design goals in mind, prepare to be amazed with my oh so awesome solution. The stereo itself was a pair of cheap speakers, requisitioned from my first dual cassette tape deck, circa 1989. The compute workhorse was a Soekris Engineering Net4501, with a MiniPCI 802.11b wireless card, and a 3.3v PCI sound card.

For those that remember such hell, the Soekris boards were notorious for not actually providing 3.3v on their single PCI slot, they were also notoriously interupt craaaazy. Finding a sound card that actually didn’t wig out at being under-powered…. and still worked under OpenBSD, took some doing. Yes folks, I was over Linux even back in those days. Linux sucks. I wish I could even remember the make and model, because I went through hell, as did my wallet to find such a beast. Trust me, it exists, it was not easy to find.

A little electrical glue provided the rest, a dc-to-dc step down converter got me the power I needed from the mower battery. My childhood as the son of a journeymen electrician has been good for a few things in my life. Operation was simple, a simple quick release wire snap provided the connectivity to the battery, it was technically possible to run the stereo without the mower running, but why would I ever do that? The modus operandi was to connect the power, go inside, grab a beer, come out, fire up Rachel.

Meanwhile, OpenBSD booted, hopped on the household wireless, mounted $fileServer0:/home/nuintari/media/tunes via NFS, read only, of course, and grabbed a playlist. From there it was just mpg123 (or was it mpg321? I forget).  Tunes soon started flying out the cheap cassette deck speakers, and yours truly would proceed to enjoy a relaxing hour or so of yard work and beer.

Rock and Roll!

Pre, The Incident

My wife is afraid of birds, royally terrified of birds. Have you ever seen how I react to spiders? Imagine that, but with birds, it is that level of terror. Actually it isn’t, my wife isn’t the bloomin’ coward I am in the face of her fears. But, she is not a fan of them to say the least.

We live in the country, or….. maybe right on the edge of the country. As country as Northwest Ohio ever gets is the point. Country enough that the neighbors raise chickens. Chickens that are mostly free to wander, and return to the hen house at the end of the day. How they weren’t all eaten by foxes, I will never know. But, they did seem to have a thing for my lilac bushes. They would wander across the street, and nest in my lilacs. My wife hated this, she’d be out in the yard, and a chicken would appear out of nowhere, and my young, young, gorgeous lady would lose her shit and run inside. I would of course, be dispatched, usually with some kind of makeshift polearm, to shoo them away.

Occasionally, I would notice them while mowing the lawn. Rachel has some oomph behind her, and if you kill the blades, and pull the deck all the way up, you can move at a solid 12+ MPH….. with the wind. Fast enough to chase chickens. Not fast enough to catch them, not that I ever wanted to, but fast enough, and loud enough, to chase them away. Also, good for a solid laugh.

We were the new couple in the neighborhood, and the farm across the street was our only real neighbor. Turns out, they had a daughter graduating high school. We were invited to the party, which we wholeheartedly accepted on the assumption that there was likely to be beer. And, I guess we should get to know the neighbors or something.

Over the course of a fine afternoon, the father approached me, and informed me that, “I see you chasing my chickens, they give you any trouble, just shoot em, they’re good eating!”

I should point out that I live NORTH of US-6….. which anyone from Ohio will recognize as the actual Mason Dixon line of demarcation between civilization and Hicksville, USA. Someone will hate this bit, but I don’t care. South of Six Hicks are a thing, and we were a solid 40 miles North of their territory, spooky.

I should note that this phenomenon exists only in Ohio. Once you reach Kentucky, the hick meter resets back to a sane level, people are way nicer, and supremely less racist. South Ohio sucks ass.

Now, I have zero interest in shooting a chicken. For starters, I own a few guns, none of them suitable for avians. Can you imagine actually hitting a chicken with a 12 gauge? Or a 7.62 SKS? It’d be feathers and a fine mist. But, even assuming I killed it, and left it intact, who wants to clean it? My old man took me hunting a few times, cleaning the carcass is the nasty part I never want to experience again.

The Incident

This part is actually pretty short, the lead up is what makes the story funny.

The stage is set, Nuintari, the man with a dream, is riding a hacked up stereo laden lawn mower, listening to classic thunder, and of course, I have a beer , and I am wearing daisy duke shorts, and a bikini top. It is either truly awesome, or truly awful to live next door to me, even if the houses are fairly far apart.

A chicken waddles over the street, through my side yard, and right into my lilac bush.

It should be noted that at this time in my life, I had come into possession of two key items relevant to this story. A Sig Sauer, P229 9mm handgun, and a pile of 9mm blanks. Remember, I don’t actually want to kill the chicken, I just wanna fuck with it. Also, I am drinking.

I know, I know, I know, I shouldn’t mix beer and guns….. It hasn’t happened since…… that I can recall.

So, naturally, inside I go, grab the gun, a fresh beer (I know, I know), and load the weapon with blanks. Upon returning to Rachel, the stereo is now beginning to play Wagner’s Ride of The Valkyries. It was so on. Deck up, blades off, LETS GET THOSE CHICKENS!

The next few minutes is basically me, in a Bikini top, daisy duke shorts, driving a zero turn radius mower, with a beer in one hand, a blanks loaded 9mm handgun in the other, rocking out to classical German musical great Wagner, chasing a chicken around my yard, occasionally taking potshots at it with the blanks…… and of course, laughing like an idiot the entire time.

At one point, I caught a look from the farmer across the street, who was basically, as the kids say, “losing his shit.”

The Legacy

The stereo blew up. A victim of a replacement battery, and operator failure to observe reversed poles….. oops. It has since been replaced with a smart phone, a bluetooth headset, and Pandora. Not as sexy, but it works. The neighbor moved away, the chickens are all gone, the farm is largely empty these days, some days, I can chase a Killdeer around a bit, but it just isn’t the same. Killdeer fight back.

That Time an IT Emergency Made Me Sneeze Blood

Due to popular Twitter demand, you all apparently want to hear this tale. Warning, it really isn’t all that gruesome, but should probably serve as a cautionary fable for anyone who has decided to get into the magical world of consulting. I am also under an NDA, so the names have been changed to protect the grotesquely stupid, and I sadly, do not have any photos.

The Situation

This is a client I started working for about a year ago, mostly network stuff. They brought me in to reign in the insanity that is intrinsic to small <redacted> industry IT (hint, all IT sucks). One of the first things I did was whip out a label maker, and label patch cables everywhere I could find them, this saved my butt in putting this all back together later.

This particular small shop had a single rack for their IT assets, tucked into a back store room. This rack had many, many, many issues. I guess it is time for a bullet list.

  • 23 inch rack, nothing in the rack was wider than 19 inches. So, multiple 2 inch spacers on each side, top to bottom.
  • Two post rack, plenty of stuff that really screams four post. At least it was all at the bottom.
  • Cheap, flimsy construction, this thing would wobble even without the 2 inch spacers.
  • Filled to the brim, stuffed.
  • Bolted to the floor, a badly poured concrete slab that had clearly been laid down in winter. Stomping your feet made dust appear.
  • A stiff breeze caused this rack to wobble in all directions.

Between the shaky rack, and the shitty foundation, it doesn’t take an idiot to realize that the bolts holding this thing down were slowly wiggling themselves free. I told them a year ago, this is going to fall, and it is going to suck. They dismissed my warnings. Oh, I should have walked then.

Friday

They call in the AM. “We are completely down, our rack of servers fell over!”

“Yup, lemme grab my drill and my crimpers, I’ll be right in.” I replied.

Coffee to go would have been appropriate, but I had a cup of traditional at home first. My E-rate doesn’t start until I arrive, and I warned them, I fucking warned them.

Also, I knew what I would have to do.

Sure enough, the rack had ripped the bolts straight out from the floor, and collapsed. One Dell something or other is not in good shape, as it took the brunt of the fall. The rest looks like it might be alive.

I tasked one of their underlings with testing cables, anything that cleared gigE/voip on the the Paladin was re-usable, at least for now. I got to work on the rack itself. Fortunately, I only had to make six new cables by the end of this mess. No, underling didn’t know how to do that, and I am not a teacher when shit is hitting the fan right after shit has hit the fan.

So, four big ass 3/4 inch bolt holes in the floor, blasted out to all hell and back like incels think happens to lady bits if they dare have sex with someone not them. Yes, this rack really needed something bigger, 1 1/4 would be a solid minimum, but, I don’t carry concrete bolts in my Network/Systems/Security IT kit. Shit, I don’t have those in my house. But I do own a drill that can eat concrete. Thank you very much DeWalt for making a beast of a monster that I can afford. Also, my years in WISP land left me with a collection of masonry bits. LETS DO THIS.

Relocate the rack a few feet over, and mark out my holes. “NUMBER ONE, ENGAGE!”

This is where the shitty foundation starts to matter. In addition to not typically carrying concrete lags in my standard IT kit, I don’t normally bring a hazmat mask. This concrete slab had clearly been poured in the winter. For those not familiar with construction, masonry, or physics, water freezes, water is a critical component of concrete. When you lay concrete in sub zero temperatures, you get some bad shit, like a lot of dust, uneven level, and an overall shit pour.

I spent the next forty five minutes creating dust storms in my face, drilling out four holes in shit concrete with my barely adequate DeWalt Doomhammer.

I inhaled a small quantity of dust, it sucked. Then I had to make six replacement cables, and trace out shitloads of stuff that had come loose, and test. I was there for just short of three hours. Maybe 2 hours, 40 minutes. We got it done, they didn’t lose an entire day. I’m good, yo.

The Aftermath

I felt like shit, I had clearly inhaled a great deal of dust. But the next morning…… Dear god. Sneezing up blood, repeatedly. That was not fun. I still feel like ass, my nose and throat are clearly irritated beyond belief.

The client has already contested my bill. My emergency rate is always in hour increments, rounded up, no exceptions.  This particular client has a signed contract stating this, so I will get my money. But, that isn’t the point. The three hours of E-rate have no chance of addressing any possible health complications I might encounter because of this mess. Yet, here they are trying to claim they only owe me for two and a half hours, not a solid three. Now, I find myself looking for a legal way to make them responsible for the hell that is my lungs right now.

The Moral

And the moral of this motherfuckah is, ladies make em……. no wait, that is Prince.

Don’t let a company fuck with your health, they will happily do so to get what they want. I am currently updating my contracts to include personal health and danger clauses.

Organizations will not look out for you, you have to make sure you are looking out for yourself. Do not make your health a lower priority than your dedication. It isn’t worth it.

The Iconic USMC Moment

Today is a significant day in history, an iconic day for the United States Marine Corps; the day the Marines took Mount Suribachi, and performed the  now famous raising of the flag. Now, I know almost everyone at this point knows that the event was at least partially staged, but that is not the point. A lot of Marines died taking that mountain. By this time in 1945, support for the war back home was tenuous at best. A great photo, propaganda that it may be, was what the home front needed to revitalize support for continued warfare. Furthermore, a metric shitload of good US Marines died to make that staged photo happen. Today, it is emblematic of the Corps, one cannot imagine rough and tough Marines without eventually seeing this image in your mind. But I am not going to debate the merits of wartime propaganda, I was hoping to instill a bit of my historical knowledge on this subject.

Mount Suribachi sits at the southwestern most corner of the island, at a point known as Point Tobiishi. Elements from the 3rd, 4th, and 5th Marine Divisions were landed at two beaches, on the southern, and western edges of the island. Being a prominent high point on the island, Japanese positions had full view of both beaches, and the vast majority of the island. Marines were under artillery, mortar, and machine gun fire before they even hit the beaches, yet they pressed on.

Mount Suribachi is a honeycomb of caves, and the defenders took excellent advantage of this. Despite extended aerial bombardment by the US Army Air Corps, non-stop naval bombardment from the US 5th fleet, and close air support from Navy and Marine pilots, the enemy resisted, and held the peak for five days. All the time, devil dogs on the ground fought for every inch of land, under constant enemy fire.

Anyone who has ever seen an Iowa class battleship, or a B-24 Liberator, would have a hard time imaging how anything could survive the sheer onslaught of destructive force these weapons of war could bring to bear. Yet the Japanese defenders did exactly this, and continued to effectively wage war. Tobiishi point wasn’t won with air power, it wasn’t won with artillery and naval gunfire support. It was won with tenacity. Marines, in the blood soaked volcanic ash, with Garands and grenades, fought for that key position. They did the job, they fought for their buddies, they fought for each other, and in the end, they reigned supreme.

The battle for Iwo Jima would rage on for another month, with US Marines engaging a well prepared, well entrenched, and very desperate enemy. The securing of Mount Suribachi meant that, in this hellish landscape, Marines fighting to secure the rest of the island had one less place where death could reign down upon them. We will never know how many lives were ultimately saved by the taking of that tiny piece of land. Staged as it may have been, the photo now immortalized in Arlington is a true reminder of the values of the Marine Corps. They fought, they fought for their country, they fought for each other, and they got the job done.

In the aftermath of the battle, a US Army Air Corps base was established so P-51 fighter pilots could launch escort missions alongside B-29 bomber missions over mainland Japan. Mustang escort was crucial to the saving of countless air crews, and the emergency landing point afforded by the airfields at Iwo Jima saved many more.

I am not a nationalist, nationalism is the sentiment that brought us conflicts like the second world war. The notion that might makes right, and that certain people are less valuable because of their ideology, religion, or the color of their skin is poison to the peace loving people of this world. I find these notions repulsive, and anyone who uses any reason to justify such thoughts is equally abhorrent. I am, however, a patriot, and have the utmost respect for anyone who puts on the uniform in the defense of freedom, justice, and liberty. In the ashes of World War II, racial and national hatreds were eventually set aside, at least to some degree, and an important understanding was established between the former Allied and AXIS powers. That is the true legacy of those that fought in WW2, on any side, they fought, they bled, and they died so we could, as a human race, realize that this cannot happen again.

Today marks the 73rd anniversary of the raising of the US Flag over Mount Suribachi. Sadly, not many of those that fought to make this happen are still with us. I invite you to honor them, as I do, in the solemn hope that one day there will be no need for Marines. But until that time comes, I am very glad that when the chips are down, there are men and women still willing to rise to the challenge.

Semper Fi.

Custom MMC Console for Active Directory Management of External Domains

Ugh, what a title…..

The Client

A client of mine is on the road to recovery. I have thus far, taken them from about 1998, to roughly mid 2000s status in terms on IT practices. I like working for this client, they are a quirky bunch of people, and have managed to create one of the finest examples of wildly unkempt, organic IT growth I have ever seen. They have survived thus far by paying so called professionals to put out bush fires. They simply had no idea any other alternative existed. I have convinced them that IT doesn’t have to be so painful.

The Problem

It is time to roll out Active Directory. The vast majority of their machines are home versions of windows, so they won’t be joining the domain any time soon, but we can at the very least bring some sanity to the file server environment. Right now, they have two file servers, and employees named Steve log in with usernames like Brittany, who hasn’t worked for the org in three years. No one knows how to change passwords, nor create new accounts. At the same time, I am rolling out useful internal tools such as a wiki,  and a trouble ticketing system, all authenticating against AD/LDAP. Less passwords would be great here, this place is awash in a veritable sea of sticky notes.

A few of the employees are proficient enough that I can grant them the ability to manage basic AD functions, such as account creation, and password resets. However, they all have machines that cannot join the AD domain due to them all being home versions. Sadly, that is not going to change for some time. Baby steps here, folks, baby steps. So, I need a way for them to authenticate against the AD domain, launch MMC, and retain saved settings for AD management.

The Solution

The first issue is that MMC requires an account with local admin privileges to even start. Firing it up locally presents us with the friendly UAC. Fine, great. So, I snap in the AD controls, it gripes because I am not a member of a domain, so I tell it to change domain to my client’s (Via a VPN, don’t panic, I’m not grotesquely stupid). I am informed that my username or password are incorrect. This is because MMC is running as the local privileged account, not one that was successfully authenticated against the remote AD domain. We can use runas to resolve this:

So, we can just make a bat or ps1 file, and have the user run that, right? Wrong!

Open a powershell prompt, and try this, it will fail. You will be informed that the operation requires privilege elevation. Start a powershell prompt as an administrator, and try again, it will work fine.

But I want to make this into a button that a non-technical end user can click. I can train them how to change passwords, I will not be able to teach them command line anything. They’ll write it down, and then never do it, opting to instead, call me every single time.

Okay, so I’ll just go into the shortcut settings, and tell it to run as Administrator. Except, Windows won’t let me check that option in this particular case. I have no idea why, and now that I have a workaround, I don’t much care.

First you need to prepare the MMC console, as one spawned naked isn’t useful to a non-technical user. Launch an administrative powershell prompt, run the little ditty from above, and snap in all the appropriate tools. Connect them all to the correct domains. Make sure you select all the check boxes that say, “Save this domain setting for the current console.” Then save the console settings somewhere reasonable. This makes sure your end user won’t have to do this work every time.

Now create a ps1 file that looks like this:

Save that somewhere sane, create a shortcut somewhere that makes sense for the end user, and then be really nice and edit the friggin registry to make “Open” actually execute ps1 scripts, and edit send them to notepad. Why this isn’t the default, I have no idea. Here is how you do that btw:

Then get all fancy, and change the icon of the shortcut, and there you have it, problem solved. Non-technical users can now be easily trained to reset passwords, and have a button they click on that lets them do so. Wheeee!

 

The Night After X-Mas

Twas the night after x-mas, post-consumerist boom,
Not a synapse was stirring, and this makes me fume.

Debt was amassed as gadgets were bought,
And the fury of installation would soon be wrought,

Upon our humble narrator, he fixes all things,
Like the stupidity of all the world’s ding a lings.

Like little Suzy’s iPOD, it played no new jazz,
For she had not read the manual, what a stupid little spaz.

She lamented and cried, and let loose a shriek,
Without my new iPod, I can’t be unique!

She dashed to her phone, and my digits she dialed,
As I answered the phone, my fury ran wild.

Tech support I answered, how can I help you this day?
You fix my iPod mister, I demand things my way!

You fix my new toy, or I’ll cancel my service,
I could tell from her voice she was a bit nervous.

I let out a sigh, and I said, do you suppose,
You forgot the power cable – it needs one of those?

Silence I heard, and then a slight scuffle,
Then bad music, some ghetto-rap shuffle.

You fixed my iPod! I love you to death!
You are so welcome! “Fucking idiot,” under my breath.

I hung up the phone, but it rang much, much more,
and from all this, there is one thing I adore.

Self sufficient people, and instruction manual readers,
To me, they alone should be allowed to be breeders.

So if you have ever called my number, which I suppose is your right,
Eat shit, goto hell, and I hope you die this very night!

Home Network of Doom Part 2.37: LDP Tunneling

/* This is fluff

This is going to be short and sweet, because it was as easy as tasty, apple pie. I’d been wanting to do this for a while, but working for yourself doesn’t generate tons and tons of disposable income. Being frugal sucks for the mad hacker in me, but I had been holding off on purchasing the last piece of hardware I needed. Enter the deadbeat client. Long story short, sold them on a network overhaul, fixed a few things for them. They bounced a few checks, and I am left holding a new Juniper SRX300. Gee, darn. What they paid that didn’t bounce mostly covered it, so I paid about fifteen bucks for a super nice new router.

Also, I paid for the lifetime Pastebin account, and installed a wordpress plugin to embed them here, I’ll be trying that out for configs.

end of fluff */

If you will think back to the first part of this series, my network was a ring of four major devices, 2 SRX300s, and 2 SRX100s(And the head end SRX210 that doesn’t actually speak MPLS). It looked like this:

Well, I can finally say goodbye to fastE ports in the core. I rewired the whole thing to incorporate the new SRX300, and pushed both of the SRX100s out to the edge, because I do own a lot of gadgets that only eat fastE. Also, I wanted to try out LDP tunneling. So the new network looks like this:

When I rolled this out, I deployed RSVP signaling everywhere, which meant bulk adding new LSPs manually to every existing device just to get the thing working. RSVP signaling lets you do some cool stuff,which I will cover if I ever decide to write part three of this series, but scaling it is indeed a pain. My brute force method lacked any proper planning, and left me with four ingress LSPs and at least 4 egress and transit LSPs on each device. In a proper network, you would clean this up, but I did this live while my wife was streaming Netflix(And she never noticed!). If it was this annoying at my tiny lab scale, imagine what it would be like on a large network.

Enter LDP, which has basically no options, no knobs, and has one truly endearing quality: Configuration scaling. It is so dirt simple, it is not a pain to deploy en masse. Since my SRX100s are now single homed to my core network, RSVP grade decision making is no longer needed, we just need to make sure labels get flung around properly.

As I tweak my graphviz skills, here is a quick and dirty best effort conceptual view of what I am about to describe.

Better image when I figure out how to make graphviz a bit more obedient.

Making LDP work over RSVP is called LDP Tunneling, and is very, very simple. The single homed devices simply need to run LDP and MPLS on their single line back to the core. Since we aren’t running RSVP, no label switched path statements are needed.

 

That is it, easy peasy. On the other sides of those links, the RSVP core speakers need to be talking LDP on their ports out to these edge devices:

 

Now, in order to get LDP signaled paths across the RSVP signaled core, we add the ldp-tunneling option to our LSP statements:

 

On a device that is not speaking LDP, in my case, garage-mpls4, ldp-tunneling is not required on the LSP statements(But it doesn’t hurt anything either). Commit confirmed, and you should be good to go. Don’t forget to commit!

As you can see, this greatly reduced the complexity of my LSPs:

 

Only time I see transit LSPs in my core is when I have a down link, or I am screwing around with wackiness.

Kthanxbai!

Makefile Quickies: How I Make Charts

My current general purpose workhorse is a Chromebook, a first generation Pixel to be exact. As my buddy Goekesmi likes to say, a Chromebook is a knife, an extremely good knife. Unfortunately, it isn’t much else, if I need tcpdump, I drop down to my FreeBSD beast, or my Kali rig, both of which weigh over 4 times as much as my beloved Pixel. Both are toolboxes, very excellent toolboxes, and they indeed contain very excellent knives, but they weigh a great deal. Since 90% of the work I do is on a computer geographically not near me, a Chromebook with an SSH client is more often than not, a very sufficient, and appreciably lightweight tool.

But I am also a huge fan of the UNIX command line way of doing things. I use LaTeX to make most of my documentation, and graphviz to make network diagrams and flowcharts. A Chromebook doesn’t exactly have a full UNIX userland installed, so I can’t just run dot locally. With my pension for working on remote machines in mind, here is how I make charts; I resort to a web server and a Makefile.

Makefiles are great for a variety of tasks, with the added benefit of allowing you to not remember much of anything, you just type make when you want a result. With that in mind, lets examine what led me to this course of action.

Since I cannot run the graphviz suite of awesome locally, I cannot expect to use a local image viewer to view my handiwork. And even if I could, viewing local content on a Chromebook is….. meh at best. It really is designed to be online, all the time, and it shows. The logical solution is to present the images via a web browser, because that is basically what a Chromebook is. I could manually run dot for each chart I make, and enable directory indexes on my apache server, so I can easily find and view them, one by one. Urgh, who wants to type that much, and click that much? Lets create a Makefile that not only generates all my charts, but also creates a simple webpage, so I can F5 to my heart’s content!

So here it is, quite simple, but I think it illustrates an excellent use of a Makefile that is slightly outside the box from their usual task.

 

That is all there is to it, this is how I make charts and diagrams. I tend to symlink this Makefile into whatever directory I am working on currently, so if I want to view older graphs, I just enter a project directory, and type ‘make’. It cleans up the old mess, and generates images for all the *.dot files in the current working directory. This also means that my base Makefile is consistent across uses, as I have been known to introduce, and usually remove, features from this little bit of scripty fun. At one point, I had it traversing sub directories, and generating a hierarchical series of html documents and images, but it was honestly overkill for my needs.

Hope you found this useful, maybe you’ve decided that the lowly Makefile deserves a closer look. I have found little tricks like this save me a great deal of time. Lets face it, time is the one commodity none of us will ever get back.

Semper Fi, kthanxbai!

Home Network of Doom Part II: VPLS for Maniacs

Now that you have your new mess of tangled wires, and miles of configuration out of the way, it is time to start firing up some useful services. My entire initial reason for learning about MPLS was to get VPLS. As a consequence, this entire series of articles revolves around building a giant, virtual switch that spans multiple L3 devices. This particular article will focus on getting basic VPLS working. I will go into filtering, load balancing, and other such complex topics at a later time. Conceptually, the end result of this article can be compared to a VLAN aware switch nestled comfortably on top of an existing IP network. Why this is awesome is that, once built, you have all the advantages of an Ethernet network delivered to the edge, with the ability to use layer 3 tools and techniques for management of the core.

Warning: SRX Limitations

When delivering Ethernet services end to end, a service provider would(should) concern themselves with loop detection. The most common way to achieve this would be to run rapid spanning tree protocol (RSTP). Another key piece would be dropping inbound BPDU frames from customer devices, or disabling the port entirely when BPDU frames are detected. This is done to prevent a customer from joining your spanning tree network, as spanning tree has no security options. RSTP over VPLS is not supported by the branch SRX platform, nor can you configure bpdu-block in any useful way for VPLS.

What you can do is configure a firewall rule to act as a basic BPDU Filter.

nuintari@garage-mpls4> show configuration firewall  
family vpls {
    filter flood-ctrl {
        term stp {
            from {
                destination-mac-address {
                    01:80:c2:00:00:00/48;
                }
            }
            then {
                count stp;
                discard;
            }
        }
        term a-stp {
            from {
                destination-mac-address {
                    01:80:c2:00:00:00/44;
                }
            }
            then {
                count a-stp;
                discard;
            }
        }
        term pvst {
            from {
                destination-mac-address {
                    01:00:0c:cc:cc:cd/48;
                }
            }
            then {
                count pvst;
                discard;
            }
        }
        term cdp {
            from {
                destination-mac-address {
                    01:00:0c:cc:cc:cc/48;
                }
            }
            then {
                count cdp;
                discard;
            }
        }
        term vlan-br {
            from {
                destination-mac-address {
                    01:00:0c:cd:cd:ce/48;
                }
            }
            then {
                count vlan-br;
                discard;
            }
        }
        term stp-upfast {
            from {
                destination-mac-address {
                    01:00:0c:cd:cd:cd/48;
                }
            }
            then {
                count stp-upfast;
                discard;
            }
        }
        term default {
            then accept;
        }
    }
}

Special thanks to Mike for this tip. I have included the set statements for this firewall here.

Simply apply it to a VPLS routing instance.

set routing-instances <instance-name> forwarding-options family vpls flood input flood-ctrl

With a little JunOS scripting, one could conceivably turn this into a form of BPDU Guard as well. Sadly, this does not address loop detection and protection. The other method to protect against loops would be to configure mac move protections, which again, is not supported by the Branch SRX platform. In short, don’t create any loops!

The Basic Layout

To the outside observer, my entire network behaves like a giant VLAN aware switch. I’ll demonstrate my configurations for a few devices spanning three of my more important VLANs. The whole thing looks something like this:

VLAN 1000 is my general purpose, internet access network. My workstations live here, the wifi access lives here, my various gadgets including my Roku and Chromecast live here. If I had a single LAN home like most people, it would be this network.

But since I am insane, I also maintain a network for servers on VLAN 1005. Okay, I actually maintain several, but VLAN 1005 is for all the basic services that the rest of the network needs to exist, such as DNS and DHCP.

I also maintain shared storage for servers, and my one UNIX desktop over a third VLAN, 1009. Basically a network of NFS and iSCSI traffic. This network is not handed up to the IP router, as there is no need.

I maintain about a dozen other VLANs, but for simplicity sake, I’ll be cutting them out of configurations, hopefully I don’t miss anything.

Before I even get into the config, yes, my home network is named ‘assylum,’ I name all my computers after mental disorders. Also yes, I am aware that it is not the correct spelling for asylum, it is a joke, try not to think too hard to get it. My proofreader mother thought it was hilarious.

Interface Configuration

The configurations for this entire mess are substantially shorter than the previous post, and I won’t be displaying each and every device’s configuration this time around. Interface examples will start with the IP router, and a few examples of PE interfaces.

For clarity sake, lets take a look at my IP Gateway’s configuration.

nuintari@headend-mpls3> show configuration interfaces ge-0/0/1
description core-mpls5:ge-0/0/0;
vlan-tagging;
unit 1000 {
    description assylum-nat-1000;
    vlan-id 1000;
    family inet {
        no-redirects;
        address 192.168.119.254/24 {
            primary;
            preferred;
        }
    }
}
unit 1005 {
    description assylum-srv-1005;
    vlan-id 1005;
    family inet {
        no-redirects;
        address 192.168.250.254/24 {
            primary;
            preferred;
        }
    }
}

Nothing special, just VLANs and IP. The router effectively acts as a CE device in MPLS parlance. I’ll skip any real explanation here, and assume you know how a router basically works and is configured.

Exactly opposite on that same wire is core-mpls5, the, “entrance” to my VPLS switching layer.

description headend-mpls3:ge-0/0/1;
flexible-vlan-tagging;
mtu 1624;
encapsulation vlan-vpls;
unit 1000 {
    description assylum-nat-1000;
    encapsulation vlan-vpls;
    vlan-id 1000;
    family vpls;
}
unit 1005 {
    description assylum-srv-1005;
    encapsulation vlan-vpls;
    vlan-id 1005;
    family vpls;
}

This is performing the role of a PE device in MPLS-ese. The only real thing you need to take in, is that we are declaring an encapsulation of vlan-vpls at both the interface, and the sub-interface level, as well as declaring family vpls on each sub-interface. This is all required, on platforms with greater interface flexibility such as the MX, you can have different types of sub-interfaces on the same physical port. Sadly, the Branch SRX platform isn’t that awesome. We have also declared flexible-vlan-tagging for the interface itself, which becomes more important later. This port, as it stands right now, could operate with just “vlan-tagging” declared on the interface, but that would constrain us later on, which I will touch on in a bit. In short, if you don’t need any particular configuration for any particular reason, stick with flexible, because it is just that, flexible.

For purposes of wrapping your head around it, think of it like a switch performing as a trunk port. For a counter-example, here is an EX2200 switch passing the same VLANs (plus my storage network) in a pure L2 setting:

nuintari@core-sw0> show configuration interfaces ge-0/0/0
description core-mpls5:ge-0/0/1;
unit 0 {
    family ethernet-switching {
        port-mode trunk;
        vlan {
            members [ assylum-nat-1000 assylum-srv-1005 assylum-nfs-1009 ];
        }
    }
}

Directly opposite that port on core-mpls5 looks like this:

nuintari@core-mpls5> show configuration interfaces
ge-0/0/2 {
    description core-sw0:ge-0/0/0;
    flexible-vlan-tagging;
    mtu 1624;
    encapsulation vlan-vpls;
    unit 1000 {
        description assylum-nat-1000;
        encapsulation vlan-vpls;
        vlan-id 1000;
        family vpls;
    }
    unit 1005 {
        description assylum-srv-1005;
        encapsulation vlan-vpls;
        vlan-id 1005;
        family vpls;
    }
    unit 1009 {
        description assylum-nfs-1009;
        encapsulation vlan-vpls;
        vlan-id 1009;
        family vpls;
    }
}

Pretty standard config so far, but what happens if you want to present an untagged ethernet frame directly on the PE device? My wireless access points are dumb, they don’t understand VLANs (but they have killer antenna), so I need to somehow configure a VPLS speaking interface to act like an access port on a VLAN aware switch. Question is, how do I do that?

The short answer is, you don’t. Asking VPLS to act exactly like a VLAN aware switch belies their true nature in that they aren’t exactly mirror clones. With that in mind, you can fake an access port reasonably well.

nuintari@garage-mpls4> show configuration interfaces
ge-0/0/2 {
    description assylum-wifi;
    flexible-vlan-tagging;
    native-vlan-id 1000;
    mtu 1624;
    encapsulation vlan-vpls;
    unit 1000 {
        description assylum-nat-1000;
        encapsulation vlan-vpls;
        vlan-id 1000;
        family vpls;
    }
}

In reality, this is no different than any other VPLS speaking interface, except we have added the ‘native-vlan-id’ statement.  There is another piece to this puzzle which we will cover in a bit, but what you are now performing is called ‘VLAN normalization’, and only works on interfaces in ‘flexible-vlan-tagging’ mode. The other important piece is in the routing instance configuration. This particular trick only works in VPLS instances where we declare a single VLAN. Fortunately, you can do this on a per case basis. In most provider networks, it would be most advantageous to provide a simple tube, whatever the customer puts on the wire, tags or no tags, comes out the other end the same. If you want to be play fake access port, you lose this ability. I think, this might be way wrong, I wrote this article, stopped fact checking, and posted it a month and a half later……

Routing Instances

The actual work of gluing the interfaces together across the network falls to the routing instances. There are three and a half major pieces that require a bit of explanation. The route-distinguisher, vrf-target, and site-range and site-identifier. The rest of the configuration is fairly self explanatory, this is the routing instance for vlan 1000, my general purpose network.

nuintari@garage-mpls4> show configuration routing-instances
assylum-nat-1000 {
    instance-type vpls;
    vlan-id 1000;
    interface ge-0/0/1.1000;
    interface ge-0/0/2.1000;
    route-distinguisher 10.0.7.4:1000;
    vrf-target target:65001:1000;
    forwarding-options {
        family vpls {
            flood {
                input flood-ctrl;
            }
        }
    }
    protocols {
        vpls {
            site-range 10;
            no-tunnel-services;
            site garage-mpls4:assylum-nat-1000:pe4 {
                site-identifier 4;
            }
        }
    }
}

The vrf-target is essentially, the identifier for the l2vpn. Each device taking part in the same VPLS VRF should have the same vrf-target. They take the form of target:<asn>:<uid>. Since I have been creating single vlan pseudowires, I have taken to using the VLAN I am flinging around as the identifier, but as single VLANs are not all VPLS can do, this is hardly a requirement. The ASN I used from the previous article in this series was 65001, so we use that here as well.

The route-distinguisher takes the form of <any ipv4 addr>:<uid>, but the common convention is to use this router’s loopback address, and again, I use the VLAN I am moving around on the pseudowire for the uid. The best way I can explain the difference between vrf-target and the route-distinguisher is, the vrf-target defines a VPLS instance, network wide, the route-distinguisher helps identify the individual members. With all analogies, there is more to it than that, but its a pretty small piece of VPLS. The RD matters much more when you are talking about MPLS l3vpn, and moving the same address space for disparate customers. If you elected to forego BGP signaling, and are doing all your signaling with LDP, this option is completely unnecessary.

The final pairing, site-range and site-identifier, serve to identify the individual members of the VPLS VRF internally, as well as dictate if and when pseudowires are formed. The site-range dictates the highest site-identifier that this instance will form a pseudowire against…… What? Okay, there are three common scenarios where this either really matters, or means next to nothing.

The most obvious is a point to point connection. With only two members, the site-range could be 2, with site-identifiers 1 and 2. Nice and self documenting, this VRF is a tube, what goes in one end comes out the other.

The next is the full mesh, where these settings matter the least. In a scenario where full pseudowires  between all sites are desired, one could opt to accept the default site-range (its 65,534) and use automatic-site-id in lieu of outright declaring anything. For the purposes of actually learning anything, I like to adjust all these options, lest I never figure out why you would ever want to.

Where this really matters is when a complex topology of hubs and spokes is the desired behavior. Edge devices that only connect to one or two devices in the core should not be wasting resources forming pseudowires across the network. I don’t own enough hardware to build this scenario(yet), but when I do, I’ll post some examples.

So this all being said, I standardized on a site-range of 10 for everything except the instance that carries my internet traffic. My cable modem sits in my family room near the TV, and is carried over my VPLS network to my core router in the server room, so a true point to point link, where I used a site-range of 2.  With that one exception in mind, I used my loopback addresses to determine site-identifiers. This won’t scale far, but helps keep me sane at home.

Diagnostics and a Rosetta Stone

Going along with our analogy that this is just one big VLAN aware switch, we will need some basic commands to verify the insanity we have created.

First of all, we want to see if our pseudowires have actually come up, which is pretty easy:

show vpls connections

Yes, for some reason, this one singular JunOS command comes with a built in legend. Easily ignored over SSH, really friggin annoying at 9600 bauds of serial cable love…… The most important thing to look for here is Remote PE: entries. A mis-configured instance, or a bifurcated network will instead show entries with things like “No connections found.”

Good:

Instance: assylum-nat-1000
Edge protection: Not-Primary
  Local site: core-mpls5:assylum-nat-1000:pe0 (5)
    connection-site Type St Time last up # Up trans
    1 rmt Up Mar 27 17:49:01 2017 1
      Remote PE: 10.0.7.1, Negotiated control-word: No
      Incoming label: 262153, Outgoing label: 262149
      Local interface: lsi.1048576, Status: Up, Encapsulation: VPLS
        Description: Intf - vpls assylum-nat-1000 local site 5 remote site 1
   2 rmt Up Mar 27 17:49:09 2017 1
     Remote PE: 10.0.7.2, Negotiated control-word: No
     Incoming label: 262154, Outgoing label: 262157
     Local interface: lsi.1048578, Status: Up, Encapsulation: VPLS
       Description: Intf - vpls assylum-nat-1000 local site 5 remote site 2
  4 rmt Up Mar 29 11:23:53 2017 1
    Remote PE: 10.0.7.4, Negotiated control-word: No
    Incoming label: 262156, Outgoing label: 262157
    Local interface: lsi.1048586, Status: Up, Encapsulation: VPLS
      Description: Intf - vpls assylum-nat-1000 local site 5 remote site 4

Bad:

Instance: assylum-sdc-1006
Edge protection: Not-Primary
  Local site: core-mpls5:assylum-sdc-1006:pe5 (5)
  Local site: core-mpls5:assylum-sdc-1006:pe10 (10)
No connections found.

How fortuitous! A VLAN for a dead project I have long since forgotten about, but have failed to rip out of the network layer! This also shows that a local instance can be more than a single site, more on that later….

The next thing you might care about is whether or not these instances are actually moving any real traffic, we have a command for that:

show vpls statistics

Basic counters, bits in, bits out, zeros probably mean bits are not moving. If I get to this point and get zeros from this command, it invariably means I have a typo somewhere, usually a mismatched VLAN.

Since this mess is effectively a giant, virtualized switch, you will probably want to view the MAC address table. If this was a pure L2 environment, this would suffice:

show ethernet-switching table

Sadly, this gets us nothing in a VPLS enviroment. Instead, you need to type out this handful:

show route forwarding-table family vpls

If you all you care about is MAC address entries, be prepared to visually filter a bit. Sometimes you just want to know the answer to a simple question like, “DO YOU FRIGGIN SEE THIS MAC?!?!?!” Plenty of other, occasionally very useful, information is presented here. This is another fine example of something that doesn’t paste well, so here is a text dump.

Some MAC entries have next-hops listed, others do not. In case you haven’t figured it out, this tells you if the MAC is directly attached, or comes form somewhere else in the VPLS network. Either way, it is kind enough to tell you which interface it came from. This is a fine time to yank a cable out, assuming you constructed a redundant path, and re-check the output of this command. If you have a mesh or any redundant sort, MACs will move, otherwise, they will time out and disappear.

Moof

I hate writing wrap ups. We have gone down the rabbit hole on MPLS/VPLS a bit further, and have something that is actually usable. In the next piece in this series, we will talk about load balancing, fault tolerance, and using L3 to troubleshoot L2, because directly troubleshooting L2 sucks, believe me I know. *mental note, write post on that subject, link it here (I’ll never remember that second part).*

 

Home Network of Doom Part I: MPLS Fundies

Before I can take you down the ridiculous rabbit hole that is my home network’s more hilarious features and failings, we must get some basics out of the way. I am not going to bore you with yet another explanation on P/PE/CE devices, and the various often vaguely described reasons for MPLS to even exist. You can google MPLS and read just about anything else on the subject if you want that information. Rather than reinventing that wheel here, my goal is to be a little entertaining, a little informative, and a great deal different from your average technology blog. When we are done with this three part series, you will have practical knowledge of how to build and maintain MPLS networks.

My home network consists of five major devices relevant to this article, all Juniper SRX Branch platform, which can be found for dirt cheap at any number of online marketplaces. Dear My Juniper sales rep, please don’t kill me, no one wants to buy a support contract that costs more than the hardware itself for a home lab. A fully functional SRX300 can be had for roughly $190 from a variety of sources. Four of my devices, a pair of SRX100s and two SRX300s, form a simple loop, and my sole SRX210 serves as my core router and firewall. All wired together, it looks like this:

The SRX210, labeled headend-mpls3, does not actually take part in any MPLS signaling(it used to), but is still connected to the MPLS network via OSPF for management purposes. Due to the per interface limitations of the branch SRX series, the simplest way to attach this device to the rest of the network was via a separate interface from the ‘NNI'(Network to Network Interface). No biggie, I have fastEthernet ports coming out my ears.

A Word on Numbers

You will notice that each device has a name, and a number, and that number coincides with its assigned loopback address. This is by design, and is the result of a decent amount of effort being expended so that I can be a little lazy. All point to point links in this diagram are /24 networks in the 10.0.0.0/8 address space. The final digit of each IP address in the MPLS mesh always ends in the same digit as the device’s loopback address. This allows me to speed read various command outputs, which may become more clear as this post, and subsequent articles are written. Suffice to say, this would never scale in a large network, but in a small to medium sized one, allows you to be a little extra lazy when interpreting command outputs. I actually came up with this approach for dealing with those god awful impossible to read IPv6 addresses, and “back ported” the concept. Like everything, it almost scales better in IPv6. It really doesn’t matter much, but it makes outputs of various diagnostic commands slightly easier to parse quickly. Your mileage will vary.

Limitations of the Branch SRX series

Sadly, the branch SRX series has a few limitations, I will point them out briefly whenever we hit them through this series of articles. The biggest one you must concern yourself with is that SRX devices do not ship with MPLS enabled. From the factory default, the SRX is first and foremost a firewall. Sadly, in order to make the SRX behave as an MPLS speaking device, you have to disable its primary role. To achieve this, delete your entire security stanza, and replace it with this configuration:

nuintari@core-mpls5> show configuration security
forwarding-options {
    family {
        inet6 {
            mode packet-based;
        }
        mpls {
            mode packet-based;
        }
        iso {
            mode packet-based;
        }
    }
}

The impact is that nothing else underneath the security stanza is available if you wish to go down the MPLS path. This is why my SRX210 no longer takes part in any MPLS speaking madness, its job is pure L3 router, firewall, and NAT. Trying to shoehorn it into being both a security device, and an MPLS device was a nightmare on its own. An additional limitation of the branch SRX platform is that the VPLS routing-instances cannot contain an Integrated Routing and Bridging interface (IRB). In simpler terms, you can’t drop IP services into a VPLS instance on a single SRX device, without doing some wackiness in terms of cabling. In even more simple terms, you can’t make a VPLS routing-instance behave like a routed trunk port feeding into a VLAN aware switch. My solution was to buy more hardware.

Thing 1: L3/IP

You can’t have an MPLS network without some layer 3. So, first things first, lets stand up the basic IP networks between our four MPLS speaking devices. Same base diagram as before, but with link/subnet IP address info:

From my selected subnets you can clearly see that I, like most humans, am a terrible source of entropy. I seem to like the number seven an awful lot.

My MPLS speaking interfaces look like this, and I’ll include the loopback interfaces for you:

fam-mpls1

fe-0/0/4 {
    description garage-mpls4:ge-0/0/5;
    vlan-tagging;
    mtu 1624;
    unit 23 {
        description mpls-lan-23;
        vlan-id 23;
        family inet {
            no-redirects;
            address 10.23.23.1/24 {
                primary;
                preferred;
            }
        }
        family mpls;
    }
}
fe-0/0/5 {
    description core-mpls2:fe-0/0/5;
    vlan-tagging;
    mtu 1624;   
    unit 90 {                           
        description mpls-wifi-90;       
        vlan-id 90;                     
        family inet {                   
            no-redirects;               
            address 10.77.0.1/24 {      
                primary;                
                preferred;              
            }                           
        }                               
        family mpls;                    
    }                                   
}
lo0 {
    unit 0 {
        description fam-mpls1;
        family inet {
            address 10.0.7.1/32 {
                primary;
                preferred;
            }
        }
        family mpls;
    }
}

core-mpls2

fe-0/0/5 {
    description fam-mpls1:fe-0/0/5
    vlan-tagging;
    mtu 1624;
    unit 90 {
        description mpls-wifi-90;
        vlan-id 90;
        family inet {
            address 10.77.0.2/24 {
                primary;
                preferred;
            }
        }
        family mpls;
    }
}

fe-0/0/7 {
    description core-mpls5:ge-0/0/5;
    vlan-tagging;
    mtu 1624;
    unit 19 {
        description mpls-ospf-19;
        vlan-id 19;
        family inet {
            no-redirects;
            address 10.19.19.2/24 {
                primary;
                preferred;
            }                           
        }                               
        family mpls;                    
    }                                   
}
lo0 {
    unit 0 {
        description core-mpls2;
        family inet {
            address 10.0.7.2/32 {
                primary;
                preferred;
            }
        }
        family mpls;
    }
}

headend-mpls3

headend-mpls3, despite its name, no longer speaks MPLS. For brevity’s sake, what little is left that is, lets skip it.

garage-mpls4

Yes, I have an SRX300 in my garage, do you not?

ge-0/0/4 {                              
    description core-mpls5:ge-0/0/1; 
    vlan-tagging;                       
    mtu 1624;                           
    unit 16 {                           
        description mpls-ospf-16;       
        vlan-id 16;                     
        family inet {                   
            no-redirects;               
            address 10.16.16.4/24 {     
                primary;                
                preferred;              
            }                           
        }                               
        family mpls;                    
    }                                   
}
ge-0/0/5 {                              
    description fam-mpls1:fe-0/0/4;     
    vlan-tagging;                       
    mtu 1624;                           
    unit 23 {                           
        description mpls-lan-23;        
        vlan-id 23;                     
        family inet {                   
            no-redirects;               
            address 10.23.23.4/24 {     
                primary;                
                preferred;              
            }                           
        }                               
        family mpls;                    
    }                                   
}
lo0 {                                   
    unit 0 {                            
        description garage-mpls4;       
        family inet {                   
            no-redirects;               
            address 10.0.7.4/32 {       
                primary;                
                preferred;              
            }                           
        }                               
        family mpls;                    
    }                                   
}

core-mpls5

ge-0/0/1 {
    description garage-mpls4:ge-0/0/4;
    vlan-tagging;
    mtu 1624;
    unit 16 {
        description mpls-ospf-16;
        vlan-id 16;
        family inet {
            no-redirects;
            address 10.16.16.5/24 {
                primary;
                preferred;
            }
        }
        family mpls;
    }
}
ge-0/0/5 {                              
    description core-mpls2:fe-0/0/7;    
    vlan-tagging;                       
    mtu 1624;                           
    unit 19 {                           
        description mpls-ospf-19;       
        vlan-id 19;                     
        family inet {                   
            no-redirects;               
            address 10.19.19.5/24 {     
                primary;                
                preferred;              
            }                           
        }                               
        family mpls;                    
    }                                   
}
lo0 {                                   
    unit 0 {                            
        description core-mpls5;         
        family inet {                   
            no-redirects;               
            address 10.0.7.5/32 {       
                primary;                
                preferred;              
            }                           
        }                               
        family mpls;                    
    }                                   
}

The observant reader will notice two key items here. The altered MTU, and MPLS signaling. Since MPLS incurs some packet size overhead, you need to either increase your MPLS speaking interface MTU, or reduce the MTU everywhere else. Trust me, do everything in your power to do the former, not the latter. Since my house is a mix of gigE and fastE, I selected an MTU of 1624, since that is the highest my various fastE devices could all agree on. If you can enable true jumbo frames, I say go nuts, but any increase with sufficient overhead for MPLS will suffice. If you are insane, you could do a mix, just understand that fragmentation is not a thing in a VPLS network. Solutions to this exist, they are not simple. You have been warned, keep your MTUs on the MPLS transport side high enough to fling the largest, maximum sized ethernet frame generated by any of your non MPLS speaking devices. O’Reilly Juniper Networks Warrior chapter four, is a great read on this very subject.

You will also, for obvious reasons, need to enable MPLS signaling on each interface. Do this now with the ‘family mpls’ statement. If you forget just one in even a fairly simple network, your eyes will be bleeding by the time you find the source of the mystery issue. Trust me, been there, done that, type it now.

Thing 2: Choose you Interior Gateway Protocol

You will need an IGP. Okay fine, you don’t technically need one, but if you decide to stand up an MPLS network with static routes, you are not only a fool, but you deserve all the pain you will receive. You could stand up RIP, but again, the pain, you deserve it. Not to mention the scorn of every real router jockey on the planet. EIGRP is Cisco proprietary, and should never be deployed unless you are bolting onto someone else’s EIGRP mess. IS-IS is fine, except that Juniper likes to charge for the license to light it on almost anything, and no real open source solutions exist to my knowledge. Besides, simply typing IS-IS is going to get this blog reviewed by the CIA. Imagine explaining IS-IS in your google search history to some NSA goons who already know about your secret ladyboy porn collection and addiction to My Little Pony.

IS-IS, ISIS, Isis, isis…….

That leaves OSPF as your only real choice for an IGP. That is okay, OSPF is easy and powerful. At least I think so, but I know the functional difference between a Stub and a NSSA area, and have been known to deploy both. Don’t worry, you only need area 0.0.0.0 today.  Here is my OSPF config on all four MPLS speakers:

fam-mpls1

traffic-engineering;
area 0.0.0.0 {
    interface lo0.0 {
        passive;
    }
    interface fe-0/0/5.90 {
        metric 300;
        priority 240;
    }
    interface fe-0/0/4.23 {
        metric 150;
        priority 240;
    }
}

core-mpls2

(ignore interface fe-0/0/6.70, its that management interface we keep not talking about)

traffic-engineering;
area 0.0.0.0 {
    interface lo0.0 {
        passive;
    }
    interface fe-0/0/5.90 {
        metric 300;
        priority 245;
    }
    interface fe-0/0/6.70 {
        metric 150;
        priority 245;
    }
    interface fe-0/0/7.19 {
        metric 150;
        priority 245;
    }
}

garage-mpls4

traffic-engineering;
area 0.0.0.0 {
    interface lo0.0 {
        passive;
    }
    interface ge-0/0/4.16 {
        metric 50;
        priority 248;
    }
    interface ge-0/0/5.23 {
        metric 300;
        priority 248;
    }
}

core-mpls5

traffic-engineering;
area 0.0.0.0 {
    interface lo0.0 {
        passive;
    }
    interface ge-0/0/1.16 {
        metric 50;
        priority 250;
    }
    interface ge-0/0/5.19 {
        metric 150;
        priority 250;
    }
}

Just a typical, single area OSPF setup.  The only MPLS specific thing here is the traffic-engineering statement. In a nutshell, this statement makes OSPF “MPLS aware.” What it really does is allow label switched path (LSP) metrics to be advertised via OSPF link state announcements (LSA). So WTF does that mean? The most functional meaning is that when used in conjunction with RSVP, RSVP signaled bandwidth reservations are flooded within the OSPF link state database. There are additional knobs under the traffic-engineering statement, but for right now, you probably just want the base level statement enabled on every MPLS speaking device.

Verification of OSPF

OSPF diagnostics are fairly simple:

nuintari@core-mpls5> show ospf neighbor
Address     Interface      State      ID         Pri   Dead
10.16.16.4  ge-0/0/1.16    Full       10.0.7.4   248   33
10.19.19.2  ge-0/0/5.19    Full       10.0.7.2   245   34

This command will tell you if an adjacency has been successfully formed or not. You are looking for Full states. If you have created some multi-point OSPF networks, the Designated Router (DR) and Backup Designated Router (BDR) will show a status of Full, anything else will report itself in 2-Way. If you get something other than this, bust out the OSPF docs and figure out what basic thing you did wrong. Helpful hint: You are mucking with MTUs to make room for MPLS overhead, if an OSPF adjacency won’t form, double check your MTU settings. OSPF insists on a consistent MTU.

Thing 3: BGP Signaling

Just like you can’t make a proper Italian sub without salami, you cannot make a true MPLS network without Border Gateway Protocol. Go ahead, google it, you’ll find discussion forums filled with experts telling someone with a bad idea not to follow through with their madness. And if I ever find someone selling Italian subs without salami, no jury would ever convict me.

If you plan to deliver L3VPN over your MPLS network, BGP is the only sane way to exchange routes with other networks. No one in their right mind will have any interest in running an IGP across autonomous network lines. For fine grained control of VPLS pseudowires, the flexibility afforded by the various knobs and buttons BGP provides is hard to beat.

The configuration is very simple, a standard internal BGP mesh, using one of the reserved ASNs. When using type internal, it isn’t nessesary to include the neighboring devices’ peer-as, but it doesn’t hurt anything either. Like my rock and roll, I like my configurations to be very explicit.

fam-mpls1

nuintari@fam-mpls1> show configuration protocols bgp
group mpls {
    type internal;
    local-address 10.0.7.1;
    family inet {
        unicast;
    }
    family l2vpn {
        signaling;
    }
    local-as 65001;
    neighbor 10.0.7.2 {
        peer-as 65001;
    }
    neighbor 10.0.7.4 {
        peer-as 65001;
    }
    neighbor 10.0.7.5 {
        peer-as 65001;
    }
}

core-mpls2

group mpls {
    type internal;
    local-address 10.0.7.2;
    family inet {
        unicast;
    }
    family l2vpn {
        signaling;
    }
    local-as 65001;
    neighbor 10.0.7.1 {
        peer-as 65001;
    }
    neighbor 10.0.7.4 {
        peer-as 65001;
    }
    neighbor 10.0.7.5 {
        peer-as 65001;
    }
}

garage-mpls4

group mpls {
    type internal;
    local-address 10.0.7.4;
    family inet {
        unicast;
    }
    family l2vpn {
        signaling;
    }
    local-as 65001;
    neighbor 10.0.7.1 {
        peer-as 65001;
    }
    neighbor 10.0.7.2 {
        peer-as 65001;
    }
    neighbor 10.0.7.5 {
        peer-as 65001;
    }
}

core-mpls5

group mpls {
    type internal;
    local-address 10.0.7.5;
    family inet {
        unicast;
    }
    family l2vpn {
        signaling;
    }
    neighbor 10.0.7.1 {
        peer-as 65001;
    }
    neighbor 10.0.7.4 {
        peer-as 65001;
    }
    neighbor 10.0.7.2 {
        peer-as 65001;
    }
}

The only non-conventional BGP thing here is enabling l2vpn signaling, since that is what I use MPLS for in my home. Many other options exist, but if you intend to follow this series of articles to completion, you will need l2vpn signaling enabled.

Verification of BGP

Much like OSPF, BGP is very easy to query, but has a singularly stupid piece of language that catches everyone.

nuintari@core-mpls5> show bgp summary

Oh dear lord, this output will never format well......

Okay, nothing I can paste into html formatting short of going all tables will do anything except make blood shoot out your nose. So here is a text dump of the output. A state of Establ(Established) is good…. a state of Active is not. I have no idea how we came to agree that Active means not working. I guess it is “actively looking for its peer”? All I know is, I have had to answer the question, “Its shows its active, how come it isn’t working?” far too often to junior networking initiates.

Thing 4: Labels

Now we come to the moment of decision, Resource Reservation Protocol (RSVP), or Label Distribution Protocol (LDP). LDP is basically low on features, but has a small configuration footprint. It is essentially label distribution autopilot. RSVP has a configuration complexity requirement due to the fact that Label Switched Paths (LSPs) are not automagically created for you. Ergo, as your RSVP network grows, the number of LSPs you will need to configure also rises. But at the same time, RSVP affords you control over how LSPs are created and utilized.

Larger networks reconcile this configuration scaling issue by running RSVP in the core, where the fine control is needed, and running LDP out on the edges. This level of complexity is beyond the scope of this series of articles. Google LDP Tunneling for more information.

I recommend starting with RSVP, as you will learn more. But for the sake of comparison, and that one idiot who plans on not listening to me, I will highlight what configuration options can be skipped when using LDP.

It should also be noted that the MPLS configuration in this section is not truly part of the RSVP configuration. Yet they are so closely intertwined, I find it useful to think of them at the same time.

Basic RSVP configuration itself is simple, any interface that is speaking family mpls at the interface layer, should be included in the RSVP configuration. The base configuration only requires interface statements, but I wanted to explain my internal network a bit better with bandwidth statements. Bandwidth statements will make more sense in part three of this series, when I get into advanced RSVP topics. For right now, consider them informational:

fam-mpls1

nuintari@fam-mpls1> show configuration protocols rsvp
interface lo0.0 {
    aggregate;
}
interface fe-0/0/5.90 {
    aggregate;
    bandwidth 54m;
}
interface fe-0/0/4.23 {
    aggregate;
    bandwidth 100m;
}

core-mpls2

nuintari@core-mpls2> show configuration protocols rsvp
interface lo0.0 {
    aggregate;
}
interface fe-0/0/7.19 {
    aggregate;
    bandwidth 100m;
}
interface fe-0/0/5.90 {
    aggregate;
    bandwidth 54m;
}

garage-mpls4

nuintari@garage-mpls4> show configuration protocols rsvp
interface lo0.0 {
    aggregate;
}
interface ge-0/0/4.16 {
    aggregate;
    bandwidth 1g;
}
interface ge-0/0/5.23 {
    aggregate;
    bandwidth 100m;
}

core-mpls5

nuintari@core-mpls5> show configuration protocols rsvp
interface lo0.0 {
    aggregate;
}
interface ge-0/0/1.16 {
    aggregate;
    bandwidth 1g;
}
interface ge-0/0/5.19 {
    aggregate;
    bandwidth 100m;
}

Most of my links are either fastE or gigE speeds, but I still have one link being handled by a piece of crap 802.11g ptp link provided by a cheap ass vendor whose name shall remain ubiquitous. It was free, okay? Once we go further down the MPLS rabbit hole, bandwidth statements in the RSVP stanza will make more sense.

The aggregate statement enables all RSVP refresh reduction features on the interface. RFC 2961 specifies some RSVP timer reduction methods, which modern JunOS completely supports. Most other vendors have also implemented these extensions. Unless you know for a fact you are a dealing with something that is either ancient, or just pathetic, you should be enabling aggregation, it does good things.

Verification of RSVP

One simple command for querying RSVP:

show rsvp neighbor
nuintari@core-mpls5> show rsvp neighbor
RSVP neighbor: 2 learned
Address       Idle Up/Dn  LastChange  HelloInt  HelloTx/Rx  MsgRcvd
10.16.16.4    5     2/1   1d 21:05:30         9  64263/64253 87912
10.19.19.2    5     1/0   6d 17:56:52         9  64399/64399 100936

All you care about is that all the neighbors you expect to see are indeed up. The basic configuration is so simple, if everything else works, and RSVP is still showing down, go find the typo.

For that one jerk who insists on running LDP instead, just set the interfaces with no additional configuration options, something like this:

nuintari@not-a-real-device> show configuration protocols ldp
interface lo0.0;
interface fe-1/2/3.45;
interface ge-6/7/8.90;

Thing 4.5: MPLS

The MPLS configuration when using LDP is almost identical, every interface we have created that will be taking part in the MPLS network must be listed here.

nuintari@not-a-real-device> show configuration protocols mpls
interface lo0.0;
interface fe-1/2/3.45;
interface ge-6/7/8.90;

Sound familiar? Didn’t you already light MPLS during the interface configuration steps? You did, but you still need to do it here as well. There is a reason for this, it is very, very outside the scope of this article, so lets not get lost down that path, okay?

If you have decided to listen to me, and begin your learning with RSVP signaling, you will not only need to declare MPLS speaking interfaces, but also every labeled switch path you might need. Everything we have done up to this point has been about how the network is physically constructed. LSPs allow you to control how that physical network is utilized by creating virtual channels that traverse the network according to your configuration. I will get into more detail in a later post.

To facilitate us hitting the ground running, I recommend creating a simple mesh of LSPs among your MPLS routers. No hard defined paths, no strict or loose statements, just connect every MPLS device to every other MPLS device with a single LSP via loopback addresses. Think of it sort of like a BGP mesh, just keep in the back of your mind that LSPs are unidirectional tubes, that will matter later on.

fam-mpls1

nuintari@fam-mpls1> show configuration protocols mpls
revert-timer 60;
label-switched-path fam-mpls1:core-mpls2 {
    from 10.0.7.1;
    to 10.0.7.2;
    link-protection;
}
label-switched-path fam-mpls1:garage-mpls4 {
    from 10.0.7.1;
    to 10.0.7.4;
    link-protection;
}
label-switched-path fam-mpls1:core-mpls5 {
    from 10.0.7.1;
    to 10.0.7.5;
    link-protection;
}
interface lo0.0;
interface fe-0/0/5.90;
interface fe-0/0/4.23;

core-mpls2

nuintari@core-mpls2> show configuration protocols mpls 
revert-timer 60;
label-switched-path core-mpls2:fam-mpls1 {
    from 10.0.7.2;
    to 10.0.7.1;
    link-protection;
}
label-switched-path core-mpls2:garage-mpls4 {
    from 10.0.7.2;
    to 10.0.7.4;
    link-protection;
}
label-switched-path core-mpls2:core-mpls5 {
    from 10.0.7.2;
    to 10.0.7.5;
    link-protection;
}
interface lo0.0;
interface fe-0/0/5.90;
interface fe-0/0/7.19;

garage-mpls4

nuintari@garage-mpls4> show configuration protocols mpls  
revert-timer 60;
label-switched-path garage-mpls4:core-mpls2 {
    from 10.0.7.4;
    to 10.0.7.2;
    link-protection;
}
label-switched-path garage-mpls4:fam-mpls1 {
    from 10.0.7.4;
    to 10.0.7.1;
    link-protection;
}
label-switched-path garage-mpls4:core-mpls5 {
    from 10.0.7.4;
    to 10.0.7.5;
    link-protection;
}
interface lo0.0;
interface ge-0/0/4.16;
interface ge-0/0/5.23;

core-mpls5

nuintari@core-mpls5> show configuration protocols mpls
revert-timer 60;
label-switched-path core-mpls5:core-mpls2 {
    from 10.0.7.5;
    to 10.0.7.2;
    link-protection;
}
label-switched-path core-mpls5:fam-mpls1 {
    from 10.0.7.5;
    to 10.0.7.1;
    link-protection;
}
label-switched-path core-mpls5:garage-mpls4 {
    from 10.0.7.5;
    to 10.0.7.4;
    link-protection;
}
interface ge-0/0/1.16;
interface lo0.0;
interface ge-0/0/5.19;

I actually have created several other LSPs that accurately describe my internal routing and switching policy, and I have policy statements that change how they are utilized. But in the name of baby steps, I have removed them from my displayed configurations. I do not recommend trying to go from zero to eleven in one day.

Verification of MPLS

The most basic question to ask MPLS is the state of LSPs, which can be easily queried with the following command:

show mpls lsp

This one doesn’t format well either, so here is the text version. There are three kinds of LSPs, ingress, egress, and transit. Ingress LSPs are LSPs that are originating from the current device. Ingress means entering the MPLS network, not entering the device. Egress LSPs are ones that are leaving the the MPLS network on this device. Transit LSPs are simply passing through. If you have taken my advice thus far, you will likely see a hodgepodge of all three types on most of your devices. We haven’t gotten into how IGP metrics affect any of this mania just yet, nor have we created any path statements to force any LSPs to go any particular direction. For right now, all you should concern yourself with, is that every LSP you created on each router is accounted for, and up.

The Final Anaysis

If you have come this far, you have a way cool network that delivers absolutely nothing useful. But you have built a foundation. One that can be used to stand up a variety of useful services. For now, I suggest a beer, I like IPAs, go get one of those. In part two of this series we will jump into basic VPLS, or, how to make your house into one giant, virtual switch.