The Longest Op of My Life, Part 2: The Long Way In
If you missed Part 1, the short version is that this was MWX, the Marine Air Ground Task Force Warfighting Exercise. I was leading the Marine Corps Red Team cyber cell for the op, and we had another DoD red team riding along with us. The planning phase had gone the way planning phases tend to go when leadership above you does not have a working knowledge of cyber. We had pushed back where we could, agreed to a plan we could live with, and started the work.
This part is about how we actually got in.
The Hotel
Most of the early execution did not happen on base. It happened in a hotel.
We had a room set up with the basics. A few power strips chained together, a handful of laptops spread across every flat surface, personal gear shoved into a corner, and a coffee maker that ran more or less continuously. This was the workshop for the first stretch of the op, and we lived in it for days.
Four of us worked in that room most nights. A lot of it was quiet. Everyone on their own laptop, heads down, trying to make something work. Then someone would find something or hit a wall, and two or three of us would crowd around one screen trying to figure out what we were looking at. We had no whiteboard, so we drew things out on whatever paper was lying around or just talked through it out loud. We were trying to build payloads, draft phishing emails, stand up new command and control infrastructure, and figure out how all of it fit together, all at the same time. The internet situation was hotel wifi tunneled through a VPN back to the infrastructure we needed, and the speeds were exactly what you would expect from that arrangement. Slow on a good night. Unusable on a bad one.
There were breaks. We would stop working and just talk about nothing for a while. One of the guys heated up a can of beans on a small hot plate and called it dinner. Nobody had thought through the food situation, which is how we ended up at Del Taco at two in the morning because it was the only thing still open. We bought enough cheap chicken tacos to have an eating contest, because when you have been staring at a screen for ten hours and your payload still is not working, the logical next step is apparently competitive taco consumption. Those breaks kept us sane. The work was grinding, and you cannot grind all night without coming up for air.
The drive from the hotel back to base was long enough that, when we ran late, it was not worth making the trip. We slept where we worked. On a couple of nights I slept on the floor near the AC, with a curtain pulled off the rod and used as a blanket, because one of my Marines needed a better spot and the spot next to the bed was warmer. Pete was the one who, more than once, looked up from his laptop and said we needed to stop and go get real food.
I would wrap up around 0400 and get a few hours of sleep. I would wake up with just enough time to make myself look presentable and make the hour drive back to base for the morning meeting at 0800. My guys got to sleep in a bit longer while I took care of that piece. They had earned the sack time. The briefing was on me, and on the days I could shoulder it alone, I did. That cycle ran for the first several days of the op.
Remote Browser Isolation
The technical problem at the front of the op was the new web proxy.
I mentioned in Part 1 that the Marine Corps had stood up a remote browser isolation capability in front of their web traffic, and that it was breaking the techniques we had planned to use. The short, generalized version of the problem is that RBI changes the trust model for a phishing payload. Things you used to be able to do on a target’s browser, you can no longer do, because the browser is not actually running on the target. The payload has to take a different shape to survive the trip, and the delivery has to account for what the proxy is willing to pass through to the user.
A lot of those late nights in the hotel were spent on this problem specifically. We would build something, test it against the proxy, watch it fail, figure out why, and start over. The first several attempts did not get close. The next few got closer but still broke in ways we did not expect. We were doing all of this over hotel wifi through a VPN that dropped whenever it felt like it, which added its own layer of frustration to every test cycle. Eventually we had something that worked, and we kept that capability in the toolbox afterward.
Lance and I wrote up one bypass approach as a public blog post after the op was done. If you want the technical details, you can read it here: Calling Home, Get Your Callbacks Through RBI. OPSEC Disclaimer: The blog is “a” method to get past some types of browser isolation. It was not the method we used here and would not have worked here.
The Other Team
There was another DoD red team with us for this op, and the contrast between how the two teams approached the work was, in retrospect, instructive.
I am not going to name them or describe them in any way that would identify them. What I will say is that, from the conversations I had with their members over the course of the op, the way they typically operated was different from how we operated. Most of their engagements involved being handed credentials and being given access to a network, and then running the operation from that starting point. That is a legitimate way to do a particular kind of offensive security work, and there is value in it. It is also a fundamentally different muscle than what we were doing on MWX, which was treating the target the way an external adversary would treat it. You fight for every inch. You assume nothing.
They sent a close access element first. Good at what close access teams do, but the people they put in the hotel with us had almost no computer experience. Most of their time during the day was spent sitting on the end of a bed watching us work. We did not have payloads ready for them to go do close access work with, and close access was all they knew, so there was not much for them to do. They would go out occasionally to try things, but without the tools ready on our end, there was nothing to plug in on their end.
When they rotated in their more technical people, it got a little better and a little worse at the same time. The second crew had some familiarity with Cobalt Strike, which we did not have available for this op. They did not have admin rights on their own laptops, so they could not install their own tooling. They had not brought the infrastructure they were used to working with. And from what I could tell in the conversations I had with them, their understanding of red teaming was more procedural than conceptual. They knew which commands to run. They did not always know what the commands were actually doing or why. A pretty novice team, by their own admission in some cases.
The friction was less about any single incident and more about the daily reality of it. Multiple members of their team hovering over different members of mine, giving suggestions that did not fit the situation or asking what was happening every few minutes. My guys were trying to solve hard problems under time pressure, and having someone who could not contribute standing over your shoulder offering commentary was not helping. It wore on morale fast.
I am not going to pretend that did not get to my Marines. The frustration of working a hard problem with limited time and limited sleep while a parallel team watches from the bed and occasionally offers input that does not apply is real, and we had to manage it internally. Part of my job for the op was making sure my Marines did not put hands on anyone from the other team out of pure aggravation. That was not a hypothetical. That was a real management problem I worked on every day.
Getting In
We got in.
I am not going to describe how. I will say that it took longer than it should have, that the new tooling we built held up, and that the moment a payload finally landed on the target side of the proxy, the room went quiet for about two seconds. Then it was chaos. Cheering, high fives, the kind of noise that comes out of people who have been sleeping on hotel room floors for days and just watched something finally work. If you have done this work, you know the feeling. If you have not, the closest analogy I can offer is the moment a long-running build finally compiles clean.
We did some initial enumeration. The interesting thing about that environment was that we were inside a network everyone used, but the parts of it that mattered for the exercise were not really there. The exercise was running on a separate network, and the network we had compromised was, in the end, just a stepping stone toward the network that actually mattered.
The Defenders Took the Network Down
At some point, somewhere in the defender community, word got out that the red team was in for the exercise. The response on their end was to bring large portions of the network down. Not segment it. Not harden it. Bring it down. To keep it safe.
From our side, what that looked like was all of our agents dying at once. One moment we had access. The next moment, nothing. We spent time trying to troubleshoot, which is a difficult thing to do when your only visibility into the network just went dark. I got a call from our point of contact asking if we had done anything to the network, if we had caused the outage. I told them no, we had just lost access ourselves and were trying to figure out what happened. It was not until a meeting later that we found out the defenders had cut the network off deliberately.
I have been doing this long enough to have a lot of opinions about what good defense looks like. None of those opinions include “take the network offline to keep the adversary out.” If your defensive posture is to remove the asset you are defending, the adversary has already won. That is the win condition. That is the outcome we are trying to produce when we plan a denial-of-service operation. You did it to yourself.
I have never seen a red team bring a network down the way a paranoid defender will bring it down. I do not say that to be cute. I say it because it is something I think about every time I work with an internal defense team and the conversation turns to incident response posture. There is a version of defense that protects the mission and a version of defense that protects the network at the expense of the mission. The second one looks the same as a successful attack from the user’s seat.
Cede and Move
The hotel work ended in a way that was difficult but not unexpected. We had actually raised this during planning. The first network was not the real target. The exercise ran on a separate, isolated network, and the two did not connect. We had questioned early on why we were spending time on the first one when we knew where we actually needed to be, but that was the plan we had been given.
To move into the next phase of the exercise, we had to give up the access we had built and move on-site to the part of the base the exercise was being run from. The reason was security. The exercise network was isolated. Anyone touching it had to do it from the right physical environment, on the right equipment, under the right oversight. That meant new laptops, new infrastructure, and starting a lot of things over from the inside.
It was a success for us either way. Everything we built and learned in those hotel nights paid off later. The tooling worked. The techniques transferred. The time we spent getting through RBI was time well spent regardless of which network we were pointed at. And honestly, for all the sleep deprivation and floor-sleeping and two AM taco runs, we had a good time doing it. There were a lot of laughs in that hotel room between the frustration. That is the part people do not see from the outside. The work is hard and the hours are stupid, but you are doing it with people you trust, solving problems nobody has solved before, and that combination produces a kind of energy that is hard to find anywhere else.
So we grabbed our gear, drove out, and got to work.
That is where Part 3 picks up.
This was a wargame, not an operation. Everything described is notional, generalized, and presented at a level intended to convey the experience and the lessons rather than the tradecraft. No personnel beyond those who have written publicly under their own names are identifiable. No units, dates, locations, techniques, or outcomes are presented in a way that would identify specific systems or people. Where details have been generalized, they have been generalized on purpose.