SWY's technical notes

Relevant mostly to OS X admins

Category Archives: Networking

Sonicwall “Error: Index of the interface.: Transparent Range not in WAN subnet”

I recently needed to put an internal server in my org’s DMZ, because the service didn’t play nicely with 1:1 NAT, returning unusable data to remote IP phones.  To configure my Sonicwall, I started following a blog post by guru-corner.com, as I often find what is documented by an outsider as more complete than the manufacturer docs (YMMV).  However, my efforts to set up both the X4 and a vlan in transparent mode were rejected with a “Error: Index of the interface.: Transparent Range not in WAN subnet” alert.  It wasn’t until I read Dell SonicWall’s documentation that I focused in on the one key word: primary.

We have 3 WAN links here, 2 I use for traffic, and a small link only suitable for “when all else fails”.  SonicWall devices give a number of options for failover and load balancing:

  • Basic Failover: Always route traffic out primary connection, secondary quietly waits to take over iff primary fails
  • Round Robin: Cycle through the outbound links for each new connection, maintaining an approximately equal number of connections through each.
  • Spill-over: Use the first defined interface for traffic, until a certain bandwidth usage is reached.  Once that happens, use round robin logic across the remaining link(s).  If there’s only a secondary link, then once primary hits the usage threshold, all subsequent requests go out the 2nd link.  (I’ve not found it documented for what duration traffic must exceed the threshold. 1 second? 1 minute? Rolling average over $time?)
  • Ratio: Round Robin, but instead of equal distribution, it can be weighted.  I see using this when there are multiple links, you’d like to use them all, but they’re not equal bandwidth.  You could set a ratio of use proportional to their percent of their bandwidth contribution to the whole.

Since the device was brought online, I’ve defined our cable modem as the primary link, with a Spill-over at 85% of inbound capacity to the 2nd link.  This has worked well, but it’s what tripped me up: our cable provides a single IP, while my 2nd link routes a /28 network.  It was one of these /28 I wished to apply transparent IP mode to, but since it wasn’t defined as the primary in my load balancing configuration, my change was rejected. After redefining the load balancing group to have X2 as the primary with a low exceeds value, I was then able to define the transparent mode as desired.

Additionally, when configuring failover criteria for SonicWall links, you want to set up multiple conditions with “Probe succeeds when either a Main Target or Alternate Target responds”, with a very reliable external host as the Alternate Target- I use ICMP to http://www.google.com, with 3 DNS servers on 3 networks configured under Network:DNS.  Default SonicWall conditions for the probes that monitor “is this link up?” is to connect to responder.global.sonicwall.com:5000.  This is fine for one of the criteria, but consider what would happen if you have multiple links, all only asking “is responder.global.sonicwall.com up?” and something happens to the service, which has happened.  Both links simultaneously and erroneously say “probe failed, therefore the WAN link failed. I’m supposed to shut down”, and dutifully do so- unnecessarily taking that location offline.  Not fun.

This configuration is found under Network: Failover and LB: [expand the group]: click [configure] for each link member.

 

Advertisements

Repackaging NetExtender- updated method

While my earlier blogged method for repackaging SonicWall NetExtender gave solid results, I’d rather learn to use The Luggage, due to it being easier to repeat consistent results.  The issue to solve with NetExtender is that while Dell provides a drag and drop .app that’s simple to dump into the Applications folder, it’s not ready to run.  Without adjustments, on first launch, it makes this request from the user:Screen Shot 2013-01-28 at 1.01.02 PM

That wasn’t going to work in 2013, and it’s still not in 2014.  Approving this request leads to an authentication dialog, and once authenticated, “magic happens”, and NetExtender is happy, probably until the next MacOS update, where the dialog will return.  Therefore, the question was to determine what sort of “magic” happens there.

Enter fseventer.  Like opensnoop, it will answer the question “what file(s) are being modified?”  The answer I came up with was a group in /usr/sbin and in /etc/ppp (which was consistent with my Composer work last year)

Next was to gather these files into The Luggage, and create a makefile.  My first attempts to build the package included many more cp, chown, chmod steps than necessary, but with some help from @chilcote and @mikeymikey, the following was created:

USE_PKGBUILD=1
include /usr/local/share/luggage/luggage.make

TITLE=NetExtender_7.5.757
PACKAGE_NAME=NetExtender_7.5.757
REVERSE_DOMAIN=com.hiebing
PAYLOAD=\
    unbz2-applications-NetExtender.app \
    pack-script-postinstall \
    pack-Library-LaunchAgents-com.hiebing.netextender.plist \
    pack-usr-sbin-netExtender \
    pack-usr-sbin-nxMonitor \
    pack-usr-sbin-uninstallNetExtender \
    pack-config \
    pack-man1-netExtender.1 \
    pack-ppp \
    fix-perms

pack-config: netextender_config.sh l_Library
    @sudo mkdir -p ${WORK_D}/Library/Hiebing/Scripts
    @sudo chown -R root:wheel ${WORK_D}/Library/Hiebing
    @sudo chmod -R 755 ${WORK_D}/Library/Hiebing
    @sudo ${INSTALL} -m 755 -g wheel -o root "netextender_config.sh" ${WORK_D}/Library/Hiebing/Scripts

pack-ppp: ppp.tar.bz2 l_private_etc
    @sudo ${TAR} xjf ppp.tar.bz2 -C ${WORK_D}/private/etc
    @sudo chown -R root:wheel ${WORK_D}/private/etc/ppp
    @sudo chmod -R 755 ${WORK_D}/private/etc/ppp
    @sudo chmod 644 ${WORK_D}/private/etc/ppp/peers/sslvpn
    @sudo chmod 744 ${WORK_D}/private/etc/ppp/sslvpnroute
    @sudo chmod 666 ${WORK_D}/private/etc/ppp/netextenderppp.pid
    @sudo chmod 666 ${WORK_D}/private/etc/ppp/netextender.pid
    @sudo chmod 644 ${WORK_D}/private/etc/ppp/options

fix-perms:
    @sudo chmod u+s ${WORK_D}/usr/sbin/uninstallNetExtender
    @sudo chmod 744 ${WORK_D}/usr/sbin/nxMonitor

Postinstall: a 1 liner to suid /usr/sbin/pppd.  In retrospect- that could have gone in the fix-perms statement.

com.hiebing.netextender.plist: a LaunchAgent that runs a configuration script, defined in pack-config

netExtender, nxmonitor and uninstallNetExtender need to go in /usr/sbin, so pack-usr-sbin-<item> handles that

pack-config: the script, which checks to see if ~/.netextender exists, and if not, creates the appropriate config file, based on my account config.

pack-man1 handles the man file

pack-ppp unarchives the contents of /etc/ppp, and ensures ownership and permission match the source, as NetExtender is checking these on first launch.

fix-perms does just that.

After running make pkg to build the pkg, I have a nice installer to push to clients, and a LaunchDaemon to configure the connection for each user login.  While this handles packaging for install, there’s one more aspect to keeping a healthy NetExtender: MacOS updates may remove the suid on /usr/sbin/pppd.  It happened on 10.8.5, 10.9, and 10.9.2.  One way to handle this is puppet, another is an installer in munki- mine has the following components:

An installcheck_script that queries the permissions on /usr/sbin/pppd:

#!/bin/bash
#installcheck for /usr/sbin/pppd permissions

proper=-r-sr-xr-x
current=`ls -al /usr/sbin/pppd | cut -c1-10`

if [ $current == $proper ] ; then
    exit 1    
else
    exit 0
fi

 

A postinstall script that is run if the permissions vary:

#!/bin/sh
chmod u+s /usr/sbin/pppd

exit 0

Net result is that after any update that alters the suid of pppd, on next munki run, it will be reset, and users will not be asked to “do maintenance tasks”- and more importantly, not asked to authenticate.

TFTP: mind your modes

In order to update the firmware on my stack of Dell 5548P switches, I needed to set up a TFTP server, and serve up some files 80’s style.  A quick Google search told me that OpenTFTP Server appeared to be a fine choice, so I grabbed it, and put it on a Windows 2K8R2 server.  It took me a little exploring to realize that everything I really needed to learn about configuring and launching it was living in the OpenTFTPServerMT.ini file.  Digging through that .ini I answered my initial question of “what directory are you serving from?” and found all the other configuration options adequately documented within.  I ran the OpenTFTPMTInstallService.exe file, saw I had a new service running, and tried to connect via TFTP on my ML laptop.

Nothin’.

Oh yes… the server level firewall.  TFTP runs on port 69, UDP protocol.  Open that up, and hurray, we can talk.

Next, let’s move a file.  There’s no directory listing in TFTP, you have to know your filenames, and the Dell firmware and boot code are relatively long, so I create a 2 line test.text file, and ask my ML laptop to GET it:

tftp> get test.txt
sent RRQ <file=test.txt, mode=netascii>
received DATA <block=1, 54 bytes>
Received 54 bytes in 0.1 seconds

Perfect!  The Hello World test passes, now onto the real stuff.

tftp> get powerconnect_55xx-4108.ros
sent RRQ <file=powerconnect_55xx-4108.ros, mode=netascii>
received DATA <block=1, 512 bytes>
sent ACK <block=1>
received DATA <block=2, 96 bytes>
Received 608 bytes in 0.1 seconds

No… the file isn’t 608 bytes.  I test a quick test.zip file: also ends much before it should. I experimented with a few options (blksize, getting more feedback via verbose and tsize) before it occurred to me: in TFTP, one must treat ascii and binary files differently, and set the appropriate mode:

tftp> binary

Oh, yes.  That changes everything.  Since the real use case here is for moving the files to the switch stack, I’m going to guess that binary is its default condition.  There’s no mention of needing to select that in the documentation.

I have to wonder if low level switch management will ever leave serial ports, 9600,8,N,1 and TFTP behind…

802.3ad (LACP) Bonding for fun

I happened to have a 15″ Retina, 2 Thunderbolt<->Ethernet adapters, a Synology DS1812+ NAS, a stack of 3 Dell 5548P switches, few extra minutes, and some curiosity at my disposal today, so I decided to see what sort of real-world numbers I could push between these 2 devices over 2 bonded Ethernet connections.

First up: the MBP. To create a Virtual Bonded Interface, start by plugging in the Thunderbolt-Ethernet adapters- the OS will need to see the interfaces present before they can be bonded.  Then, from the Network System Preferences, click on the [+] at the bottom, and select Manage Virtual Interfaces:

Screen Shot 2013-02-14 at 5.16.25 PM

In the new window, click [+] again, and select New Link Aggregate…

Screen Shot 2013-02-14 at 5.16.38 PM

Select the Ethernet interfaces to bond, and give them a name.

Screen Shot 2013-02-14 at 5.16.56 PM

Before these are connected to the network, you need to configure a pair of switch ports to also participate in the bond. In my Dells, that’s configured via Switching: Link Aggregation: LAG Membership, where LAG stands for Link Aggregation Group. Enter the Edit mode (vs the Summary they start in), and indicate which ports will be in your currently indicated LAG. Click first in the LAG row to bring them into the group, then the LACP row above it.  After saving the changes, you should be able to connect both Ethernet connections to your specified switch ports, and bring up a bonded connection.

LAG MembershipThis bonded connection isn’t too useful on its own without another bonded device. Next step is to configure LAG2 for the NAS in the same way.

Finally, we configure the NAS to do 802.3ad. In the Synology, that’s configured via Control Panel: Network: Network Interface: Create. Choose the default 802.3ad link aggregation, both interfaces, any VLAN options, and apply.  A couple of pings will drop, but the new bonded connection will be set up. Attach both Ethernet cables to the LAG2 configured ports, and it should be back on your network, with twice the data path.

To measure the value of LACP, I tested some Finder file copies to and from the NAS. I picked a 12 Gig folder of .DMG files- many large files, as to have minimal file creation overhead. Naturally, it helps to have fast disks at both ends- the MBP’s SSD can keep up far faster than this, and it looks like the NAS configuration of 8 drives with dual parity can also saturate 2 GigE links.

Screen Shot 2013-02-15 at 2.31.42 PM

The writes aren’t quite as steady as I’d hoped for, but holding in the 180+MB/sec range for network writes from a laptop is pretty nifty. There’s also the benefit of redundancy- one side of the bond can be lost (cable out, switch down (presuming the LAG spans multiple switches)), and the network connection isn’t broken. And, if multiple, non-LAG clients are addressing the NAS, it does a great job of providing high performance to both clients.

The price you pay in this configuration is in flexibility- you can’t just plug in one of these 2 adapters to an arbitrary network port like you can out of the box- those 2 Tbolt<->GigE adapters are now part of that bond, not separate devices to be used as single links. You’ll need a 3rd adapter if you wanted the ability to have both the high performance bond AND a standard, not-bound-to-the-LACP-configured-ports Ethernet connection- perhaps a “at my desk” vs “anywhere else” use case.

 

Addendum1 20160304:

I’ve realized my testing methods above are flawed.  I only used the data from Activity Monitor to answer the question “how fast are we transferring data?”, the answer above is 194MB/sec.

This value is not a measure of true throughput.  I revisited this topic today, timing how long a specific file transfer took with 2 bonded interfaces and an LACP configuration on my switch, vs how long it took using a single port, no bonding.  Answer: exactly the same file transfer speeds.  Despite Activity Monitor (now on 10.11.3) still reporting 2x the transfer rate under the bond, the actual time to transfer a 1.5GB .dmg was unchanged by bonding ports.  Sorry to say that despite the visual feedback in Activity Monitor, port bonding cannot double your transfer rates from a single source.

So what is it good for?  Maximizing performance from a server to multiple clients.  In this case, if you’re using a MBP as a server… well, more power to ya.

Repackaging SonicWall NetExtender.

One of the projects I’m starting in early 2013 is to get all of the OS X machines I manage at work on Mountain Lion. In my upgrade testing, I found that a machine upgraded from 10.6 would not properly run the NetExtender VPN client from SonicWall- on first use, a user would get interrupted with this alert in a log window:

01/28/2013 12:55:00.545 [general fatal     508] FATAL: You don't have permission to read/execute '/etc/ppp/peers'
01/28/2013 12:55:00.546 [general error     508] pppd permissions are invalid
01/28/2013 12:55:00.554 [gui     info      508] Failed to connect - check log for details

I can’t expect my users to tweak permissions on /etc/ppp/peers, and through previous upgrades, I’d also become aware that the NetExtender releases through 6.0.726 have DNS issues under Mountain Lion, so I also I needed to both upgrade, and tweak ppp permissions.

At first, the upgrade side would look simple, NetExtender installer provides an app to drop into /Applications.  But if one only does that, they get this dialog on first launch:

Screen Shot 2013-01-28 at 1.01.02 PM

That’s not going to fly for my non-admin users (nor should I expect my admins to do this), so I turned to JAMF’s Composer to repackage. The process was pretty simple- I copied the latest NetExtender .dmg and Composer to a basic OS X build, which had never had NetExtender installed. I copied NetExtender to the Applications folder, then started Composer, selecting a new Snapshot, by monitoring file system changes.

Screen Shot 2013-01-27 at 8.22.23 PM

After starting up the monitoring, I started up NetExtender, provided the expected authentication, let it connect, and then ended the Composer capture. The capture then only needed a little tweaking to omit the changes to my home directory that had been noted, and created via Build as PKG.

With a package built, I could then turn my attention to avoiding the pppd permissions error. Since I install most of my software via munki, it was easy to add that as a postflight script.

#!/bin/sh
chmod u+s /usr/sbin/pppd
exit 0

My 10.6 to 10.8 upgrades are running a number of steps via a second DeployStudio workflow, done after the install, as we’re changing machine naming and AD bind conventions, doing a Profile Manager enrollment, and many other steps. To force a reinstall of NetExtender, I added a DS “generic” workflow step that includes rm -r /Applications/NetExtender.app, and touch /Users/Shared/ .com.googlecode.munki.checkandinstallatstartup file, forcing Munki to see that NetExtender is absent, and run the installer again to adjust the permissions.