Please bear with me while I settle on a template that is both tolerable and functional and also find a way to cleanup the horrendus markup/highlighting of code in posts and comments. I also understand there is some additional problem with formatting in comments. There’s a ton of info I’d like to start dumping up here, but need things functional first. If anyone is aware of any clean/easy way to callout/highlight/markup text/code/etc without blowing other formatting all to hell, I’m all ears!
One stumbling block I had transition from IOS to JunOS was the way juniper does policy based routing, or PBR. Juniper refers to this as “Filter Based Forwarding”, and it’s a bit more involved to setup than a basic PBR on IOS.
I’ve come across plenty of documentation referencing /how/ to get this working, but I have yet to see a clear explanation of the rib-group import, even from JTAC. I’ve never seen much more than “you need to do this for FBF to work”.
Maybe I’ve just missed it, maybe I’m ignorant, but a coworker and I finally realized /why/ you need to do this for FBF to function properly.
First lets start with a very basic FBF setup. I’ll use a recent example where I’ve had to forward all web traffic to a web filter appliance. Only relevant config is included.
routing-options {
interface-routes {
rib-group inet webfilter;
}
rib-groups {
webfilter {
import-rib [ inet.0 webfilter.inet.0 ];
}
}
}
routing-instances {
webfilter {
instance-type forwarding;
routing-options {
static {
route 0.0.0.0/0 next-hop 192.168.1.10;
}
}
}
}
firewall {
family inet {
filter webfilter {
term 1 {
from {
destination-port [ http https ];
}
then {
routing-instance webfilter;
}
term 2 {
then {
accept;
}
}
}
}
}
interfaces {
fe-0/0/0 {
unit 0 {
family inet {
filter {
input webfilter;
}
address 192.168.1.1/24;
}
}
}
}
So, let’s see what we’ve done here…
- We’ve created a routing instance “webfilter” of type forwarding with a route for 0/0 pointing to our webfilter.
- We’ve written a firewall filter to match http and https traffic and put the traffic into our “webfilter” forwarding instance.
- We’ve attached that filter to an interface so any web traffic passing that interface gets re-routed to our web filter.
So far pretty basic, but we’ve also created a “rib-group” and imported some stuff into it. This is the part I’ve never found a clear explanation on. But here’s what we’ve done and why:
- The routing instance we’re using for our FBF setup is type “forwarding”. This means its’ sole purpose is to forward traffic where we tell it to. It does not, and can not, have any interfaces assigned to it. If there’s no interfaces in the routing instance, there’s no directly attached networks. There’s no path to any next-hop we might define in the routing instance. This is why we need to create the rib group and the imports.
- First we create the new rib-group and set it to import the default inet.0 routing table as well as the webfilter.inet.0 routing table from our forwarding instance. (My understanding is that this happens automatically without rib-groups defined, but as soon as you define a rib group to import interface-routes to, you also have to tell the rib-group to import inet.0 and the parent routing-instance’s inet.0 instance as well.)
rib-groups {
webfilter {
import-rib [ inet.0 webfilter.inet.0 ];
}
}
-
Once the rib-group exists, we pump interface routes into it so it knows how to get to the next-hop we define for forwarding.
routing-options {
interface-routes {
rib-group inet webfilter;
}
}
And THAT is the key – the forwarding instance has no interfaces, and therefore also lacks any local or directly connected networks. The rib-group is configured to import the interface-routes so the forwarding instance knows how to send traffic to the next-hop we define.
As always, comments, corrections, etc always appreciated !
~pf
We’ve had many an issue with NSM upgrades and config import/export changes, unexpected behaviour, NSM thinking it knows better and screwing up our configs (especially NAT), and much other fun.
Because of this, in prep for rebuilding our NSM appliance (which is an absolute mess from a once steady stream of upgrade/rollback cycles attempting to find a version that worked best with our configs), we wanted to test the latest versions without impacting our production instance. So I started looking at creating a virtualized instance of NSM for testing.
Note – This is NOT supported by Juniper or JTAC, you will NOT receive assistance or support for doing this.
When I started upon this little adventure I had found a great resource that detailed the few tweaks needed to make NSM install happily on a CentOS base (since only RHEL and Solaris are officially supported). Unfortunately, when I got around to actually attacking this, I found that resource to have disappeared.
All credit for the CentOS tweaks and base install go to the great (and unfortunately neglected?) Juniper Hacks Blog. Unfortunately the author of that blog chose to store many of his articles outside his WordPress instance and instead used Google’s “knol” service which has been retired. You can find a cached/snapshot copy of hackjuniper’s original NSM on CentOS article via the Internet Archive’s WayBack Machine here.
Now, on to the fun.
Notes:
- CentOS 5.2 seems to be the cleanest base. I had some weird package version conflicts and issues on 5.8. You can get the 5.2 install media from http://vault.centos.org/5.2/isos/i386/. For the base install you’ll need CD’s 1, 2, and 3 (why they couldn’t put the entire base install on cd1 is beyond me). There’s also a .torrent link in that directory for the DVD iso.
- I have successfully installed pretty much every NSM version from 2008 up to the latest 2011.4s2 on the CentOS 5.2 base. All install and run cleanly. However, for some reason, I simply *cannot* get the web-based administration page to work properly. You can get the basic client download page, but /administration never comes up even though webproxy appears to be running correctly. This isn’t a huge issue for my testing purposes, but I would love to get it working anyway. If anyone has any idea how to fix this, I’d love to hear it.
- Again, this is completely UNSUPPORTED by Juniper or JTAC. Additionally, NSM is a pig – it runs horribly on dedicated high-end hardware, and it’s worse in a VM. I would certainly never attempt to use a virtualized instance for anything outside of basic testing and functional validation.
- I’ve used this on VMWare Workstation/Player, ESX, and VirtualBox. No issues with any of them.
And, without further ado…..
Create your VM
- 1 CPU
- 2GB RAM (At least, it’ll run a bit smoother with 4)
- 40-50GB HD (You won’t use most of this, but it will ensure the NSM installers don’t complain about free disk)
Install CentOS 5.2
- Boot your install media iso and choose text mode install (type ‘linux text’ and hit enter). Skip the disk check that takes forever.
- Take the defaults for most of the install. Let the OS take care of disk partitioning with the default layout.
- For the software selection, on the first page deselect all options and check the box at the bottom for ‘customize selection’
- On the customize selection page, ensure everything but ‘base’ is unchecked.
- Proceed with the install. You’ll need the images for discs 2 and 3 as well if you’re using the CD images rather than the DVD.
- Reboot when complete
Prep CentOS for NSM
- On first boot log in as root.
- Disable iptables (optional – you can create an appropriate iptables policy to allow NSM to function, but that is outside the scope of this article)
- ‘/etc/init.d/iptables stop; chkconfig –level 12345 iptables off’
- ‘/etc/init.d/ip6tables.stop; chkconfig –level 12345 ip6tables off’
- Update the system to appear to be RHEL5
- ‘vi /etc/redhat-release’
- Delete the existing release name and replace with the string below, exactly, without quotes
- “Redhat Enterprise Linux Server release 5”
- Disable selinux (probably optional; NSM expects selinux to be present, but I had issues at times with the selinux policy preventing or blocking things. Because of this, and because this system is meant for testing only in my case, I’ve configured selinux to run in “permissive” mode so it logs without blocking).
- ‘vi /etc/selinux/config’
- Set SELINUX=permissive
Download and Install NSM
- I have successfully installed and run 2008, 2009, 2010, and 2011 with various revisions. I am currently running 2011.4s2, the latest release as of 5.28.12.
- You’ll need two files from the JTAC software download site (you’ll need a JTAC account and support contract to access the downloads).
- For your chosen version of NSM, download both the appropriate systemupdate_linux package for your release as well as the linux_servers package.
- Once downloaded, use SCP or ftp to transfer the two files to your VM. Move them both to /var/tmp
- Unzip the systemupdate file and you’ll end up with 2 .tar archives. Remove the archive for ES4, and extract the archive for ES5. You’ll be left with a es5 directory. Drop into there where we will run the systemupdate script momentarily.
- Install a few additional packages then update the system and all packages
- ‘yum install gnupg rsync xorg-x11-font-utils vim http’
- ‘yum update’
- Run the NSM systemupdate script to install the NSM preferred package versions
- ‘sh /var/tmp/es5/rhes5.sh’
- Let Yum update the system and packages once more. This will override a few of the NSM provided packages, but running this update/systemupdate/update cycle was the only way I could seem to get all the packages into a happy state for everything to install cleanly.
- ‘yum update’
- Extract the NSM installer and you’ll be left with a very large .sh script. Don’t fret, it’s huge because most of it is a binary blob containing the NSM installers.
- Now we’ll run the installer. The command line below is for the release I’m using, 2011.4s2. The -niAPPLIANCE=n tag at the end tells it that we are installing on a full Linux system and NOT an NSMXpress appliance.
- ‘sh /var/tmp/nsm2011.4s2_servers_linux_x86.sh -niAPPLIANCE=n’
- There’s nothing too special in the installer if you’ve installed NSM previously. Can mostly take the defaults and adjust as appropriate for your environment.
- Once the NSM installer is complete, you’re done!
- Point a web browser to https://<your.nsm.vm.ip>:8443 and download the client and test it out.
That’s pretty much it. Really the only special part is setting the redhat-release so that NSM believes it’s installing on RHEL, and getting the packages straight.
If anyone else has any other experience running NSM in a VM or on CentOS, I’d love to hear about it. Corrections, updates, etc are always welcome!
~pf
The Crystal Method – Community Service 05-23-2012 | Mixjunkies.
Sick and Tasty. Still killing it. <3 Crystal Method
(re: Crystal Method – one of my favorite games long ago in the days of the original playstation was n2o: Nitrous Oxide, a tunnel shooter scored entirely by TCM. I just found it’s available on psn for the ps3…score!)
Welcome to packetfail’s first post.
I’m not quite sure exactly what this blog will turn out to be, but my initial intent is just to collect information I find useful and think others might find useful as well.
Over the last 2 years or so I’ve taken a dive headlong into Juniper Networks’ JunOS platform and gear. At my place of employment, we’ve completely replaced all layer 3 with SRX/JunOS (both HE and branch), our DC switching is all EX/JunOS, and we’re mid-stride of a complete access layer replacement with, you guessed it, EX/JunOS. We’ve also got Juniper all over the ancilliaries – UAC(nac), IVE(sslvpn), and a little NSM and STRM for good measure.
In my adventures getting to know the gear and JunOS itself, I’ve found one of the most valuable resources to be a few hidden corners of web – small blogs, random forum postings, etc, and the community itself.
I’ve found that many of the resources I’ve used haven’t been updated or touched in quite some time, and some have completely disappeared. Today I went to stand up a CentOS VM to run an instance of NSM for testing only to find the resources I had previously referenced for this are gone. Poof.
So, the initial intent of this site will be to collect and archive useful bits and pieces of info and resources relating to networking in general, and Juniper/JunOS specifically. The first (real) post will to write-up my VM NSM setup and notes. Beyond that, who knows – but no goatse, I promise!
~pf