Solving Second Hop Restriction...

On of the benefits of my job, is that I get to handle some really cool problems in a nuts-and-bolts-solutions way that correct and issue for the enterprise. Maybe you think that doesn't sound too exciting, and if that's the case, feel fortunate you don't have my job. A lot of my current responsibilities revolve around picking up the slack where machine and automation fall short. Got a package that wasn't deployed to half the enterprise and we aren't sure why? Let's try to get it out by Close of Business. Got a new pilot program that's destroying the user data of everyone who's assigned to it? Let's do some heuristic safeguards, try to reverse-engineer the problem, and go for a deep dive into the vendor KBs and message boards to figure out what's going on. There's always something new going on, and finding an effective solution to the existing problem always involves sharp diagnostic and critical thinking skills.
Here we go:

What does that have to do with second hops???

First off, let me explain what second-hop restriction is. This is an (apparently) very old "safety feature" of Microsoft systems that prevents me from sending a command to a remote machine to send a command to a remote machine. Basically preventing what's going on in this diagram:

(image credit: https://devblogs.microsoft.com/scripting/enable-powershell-second-hop-functionality-with-credssp/)

When Might this Matter?

I lived for many professional years blissfully ignorant of the now forever-burned-in-my-brain second-hop restriction. It wasn't until I had a very specific problem with remote machines that it started to really get in my way. Without getting too specific, let's say you run a few thousand endpoints on an enterprise network. For an undetermined reason, several of the endpoints are losing connection with their management server and software distribution point, we'll call it an imaginary service named "XCCM." Your job, lucky lucky you, is to retroactively repair every machine that the management point of "XCCM" loses connection with. The management server actually spits out a pretty nice list of all the machines, so you just need to automate a simple way to have the machines download and install their programs. Simple enough, right? Well, I hate to tell you but if you try to

foreach ($Computer in $Remotecomputers) {
New-PSSession -ComputerName $Computer
$PSSession = Get-PSSession
foreach ($Session in $PSSession) {
Invoke-Command -Session $ Session {//Path/to/server/executable.exe -force}}

You're gonna get a biiiiiiiiiiiiiiig 'ol pile of nope back from Powershell?
Why? Because of that Second-hop safety feature, which prevents you from calling a process from another remote host while you're logged into a remote host.

Isn't this a big deal for everyone?

Surprisingly, no. I run Unix, Linux, and MacOS operating systems at home, and I had NEVER heard of a second hop restriction before. To be fair, I don't often second hop with my hosts, but as a "network guy" who owns dozens of routers and switches, I can tell you there are no feasible hop restrictions that you'll ever encounter when you work on networking hardware. In fact, you'll often ssh over to another devicdfsge as a superuser to administer a secure network section.

Image Consider trying to administer even this VERY basic network remotely (from the internet). No lunatic, no matter how bad a network administrator they may be, is going to leave a port open in their firewall to allow direct router access inside their network. How would you get in? Simple, use a second (or third) hop! Set up a VPN connection point somewhere in the firewall so you can join the network at the firewall edge. Then, once you're authenticated into the firewall, use your protocol of choice to login to the router as an administrator and do whatever you need to do. Very simple, very effective, but the whole concept kind of falls apart in your hands if you aren't allowed to make any hops after your first hop. In fact, the very concept of a Second-hop restriction is so absurd that I physically lol'd when I read about it for the sheer preposterousness. Only Microsoft would make that a production standard!

Was it solved?


Indeed it was!
I'd say it took nary an hour of tinkering to realize the simplest and most effective solution to this problem... *become* the second hop!
Image Instead of telling the remote host to copy it's files from the server, just copy all the server files to my local machine, and have the remote machine copy the XCCM files FROM me, then run the install. By condensing this task into a looped job that ran against a text file generated by the XCCM Management Console, I was able to run reinstall commands against hundreds of enterprise machines in a matter of minutes.
I hope one day you, too, have a need to circumvent the ridiculous second hop restrictions put in place by Microsoft, and that this post helps you to do so.


I don't have a comments section because I don't run a social media page, but feel free to email or call if you have any thoughts on this post!