Vol 3 was about the people. This one is about what the people and I actually did together. some of the operations, the engagements with units that did not want us there, the engagements with units that did want us there, the moments that taught me what good defense looks like, and the moments that taught me what bad defense looks like dressed up as good defense. I will dig into many of these operations in the future to provide more than here but I feel this is a sufficient overview of many of the trends we saw and what some of the work looked like.
Same OPSEC posture as the rest of the series. Details generalized on purpose.
The Mix of Work
The operational portfolio for the Marine Corps Red Team was broader than a lot of people outside DoD would assume.
We did long-term operations against the Marine Corps enterprise network. Sustained, slow, methodical engagements designed to assess whether the institution could detect and respond to an advanced adversary inside its own infrastructure over time. Those were among the most valuable engagements we ran, because they exercised the full defensive stack against the kind of patient threat actor that real adversaries actually are.
We did tactical exercises. MWX is the obvious example, and it will get its own series on this site, but there were others. The exercises put us in support of conventional or special operations forces and asked us to deliver cyber effects in the context of a broader operational picture. Some of those went very well. Some of them were among the longest stretches of my career. Some of them were disappointments. All of them taught something.
We did acquisitions penetration tests, which is the somewhat unglamorous label for the work of evaluating the security of new systems and capabilities before they were fielded. The goal was to find the problems early, when there was still time to fix them, instead of late, when fixing them would mean ripping things out of a deployed environment. That work was, in many ways, the most leverage-positive thing the team did. A finding caught at acquisitions saved orders of magnitude more effort than the same finding caught after fielding.
We did other things in support of US Cyber Command and Naval Special Warfare Command that I am not going to describe in any detail. The general shape of the work involved being asked to operate at the high end of what was possible for our size of team, against targets that justified the level of effort. Some of those engagements are among the things I am most proud of from the entire career. None of them are going on a blog. This picture is what you are going to get.
The Units That Did Not Want Us There
The pattern that came up repeatedly, especially with tactical engagements, was the mismatch between who invited us and who actually had to host us.
The way these engagements often worked was that a request would come down from a command several echelons above the unit we were going to. A general officer, or someone working directly for one, would decide that a particular subordinate unit needed to be exercised. The request would flow down the chain. By the time it reached the unit, the unit had not asked for us, did not particularly want us, and had no real ability to refuse.
You can predict what happened next. The unit treated the engagement as an inspection rather than a training event. The framing was adversarial from the moment we arrived. People we were supposed to be working with administratively were not enthusiastic about supporting our work. Information that we needed to operate effectively had to be pried loose. Coordination was minimal. The unit’s instinct was to protect itself rather than to learn from the exercise.
I understand the instinct. I do not agree with it, but I understand where it comes from. Marines have careers. Careers are damaged by reports that say the unit did not do well. The natural defensive posture, when someone shows up to evaluate you against your will, is to limit their ability to evaluate. I have seen the playbook many times. Make the visitors uncomfortable. Slow-walk the support. Be technically helpful but never volunteer anything. Make sure the report has the smallest possible surface area.
This was, in my experience, the worst kind of engagement to run. Not because it was technically difficult. The technical work was usually the same as any other engagement. It was the worst kind because the value of the engagement collapses when the host unit decides not to engage with it. The point of a red team engagement is the conversation that happens afterward. The findings are not the deliverable. The conversation about the findings is the deliverable, because that is where the actual security improvement happens. A unit that has framed the engagement as an inspection has already opted out of the conversation. The report goes into a folder. Nothing changes.
We did the work anyway and delivered the reports to the best of our ability. Some of the units took the reports seriously and some did not. The ones who did not, I think about sometimes, because the vulnerabilities we found were likely still there long after we left. The network is often build and torn down for each exercise, but it is often from a standard template or process.
The Cell That Got Unplugged
There is one engagement in particular that I think about whenever this topic comes up. I was not on this op personally. One of our cells was.
The setup was the usual one. A request from a higher command to assess a subordinate unit that did not want us there. Our cell deployed and was given a single room from which they were authorized to work, on the grounds that all red team activity needed to be confined to a controlled space.
The defenders’ response was to pull the plug on the room.
I want to be precise about what I mean by that, because the precision is the point They did not implement network controls that limited what the room could reach, they did not segment the network to constrain the team’s ability to operate, they did not change a firewall rule, push a policy, or do anything to the network that they were supposed to be defending. They walked into the room our team was sitting in and physically disconnected our team from the network. That was the defense. Unplug the red team so the red team cannot assess the network. Leave the network in whatever state it was in. Declare victory.
From the team’s perspective, they had been put in a closet and the door had been locked. The network on the other side of the wall was, as far as anyone could prove, exactly as secure or insecure as it had been on the day the team arrived. Nothing had changed about it. The only thing that had changed was the team’s ability to look at it.
When the engagement ended and the defenders briefed up the result, they framed it as a defensive win. They had successfully prevented the red team from causing any harm to the network. They gave themselves awards for excellence in defense.
I am not making this up. I wish I were.
The reason this story sticks with me is not that it was funny, although it was a little funny in retrospect. The reason it sticks with me is that it is the cleanest possible illustration of the failure mode I wrote about in the compliance post. The unit had not defended the network. The unit had defended itself from being assessed. Those are not the same activity, and conflating them is exactly how an organization ends up secure on paper and exposed in reality. The vulnerabilities the team would have found were still there when the team left. The only thing the unit had proven was that it could prevent a friendly red team from telling it where they were.
The other thing the story illustrates is the danger of letting the host unit grade itself. If the framing of the engagement allows the defenders to declare victory by removing the red team from the network, the engagement is not measuring defense. It is measuring how willing the unit is to break its own infrastructure to make the visitors go away. Those are very different things.
The Programs of Record
One of the recurring themes in the operational work was what I came to call the “it works so do not touch it” problem.
Many of the systems we evaluated were not custom builds. They were commercial products or programs of record, set up at some point in the past by an installer who configured them with the vendor defaults and then walked away. The defaults made the system work. The defaults did not make the system secure. The defaults are designed by vendors to maximize the probability that a new installation will function out of the box, because a system that does not function out of the box generates support tickets and reputational damage for the vendor. Security is a secondary consideration in default configurations. It always has been.
The pattern we saw, again and again, was the system installed at defaults, the system working, and nobody touching the configuration after that because there was no incentive to. Touching a working system risks breaking it. The administrator who breaks a working system has a problem. The administrator who leaves a working system alone has no problem until an adversary shows up. The cost of inaction is invisible. The cost of action is immediate. The math is bad.
This is, I think, the single largest source of preventable vulnerability in real-world networks. It is not zero-days. It is not exotic tradecraft. It is systems sitting at defaults because the people responsible for them are reasonably risk-averse about modifying anything that is currently working. The fix is not technical. The fix is cultural. You have to create incentives for administrators to harden their configurations and remove the incentives that punish them for breaking things while they are doing it. Most organizations do not. Ours did not. The vulnerabilities accumulate.
Becoming Chief
In 2021, my friend rotated out of the chief seat and I rotated up into it.
The transition was less dramatic than it sounds. I had been doing a significant portion of the chief’s work as the engineering cell lead anyway, because the engineering cell lead is where a lot of the operational decisions land in a small team. Taking the title formalized what had already been happening. What changed was the surface area. The chief is the public face of the team to leadership, to other units, to other red teams, and to the inspection apparatus. The chief is also the person who carries the team’s culture, the team’s tempo, and the team’s standards. Those responsibilities did not exist in the engineering cell lead seat. They existed now.
The team I inherited was a team I knew. I had built half of its infrastructure and worked alongside the rest of its operators for years. The transition was clean and the team kept moving. The chief seat came with new conversations, new visibility, and a different kind of pressure than the engineering work had. Some of those conversations went well. Some of them went the way a particular conversation with a Commanding General would later go during MWX, which is to say I told people things they had not been told before and I lived with the consequences. That one is its own story and will get its own post in the MWX series.
That stretch is where MWX happened, which I will write up in its own series. It is also where the funding situation got fully clarified, in a way that produced the unwritten agreement I want to close this post with.
The Unwritten Agreement
I mentioned in the intro post on this site that I paid for infrastructure and domain registrations out of my own pocket because the team had no budget line that fit. I want to give the fuller version of that here, because it is part of what the chief job actually was, and because it illustrates how a small team makes do.
The team needed things. Domains, mostly. Hosting, occasionally. Small purchases that were operationally necessary and that, under the institution’s procurement rules, were essentially impossible to acquire through legitimate channels in any useful timeframe. The procurement process was designed for predictable expenses planned a year in advance. Red team operations do not work that way. When you need a domain for a campaign that is launching in three weeks, you do not have a year to wait for the procurement system to produce one.
I saw the leadership of our broader section, the same leadership we fell under, dress down a federal civilian for spending five dollars on fishing line that was needed for a physical operation. Five dollars for fishing line. The deputy who handled the team’s compliance with procurement rules was, in theory, supposed to enforce the same standards on me.
He and I had an unwritten agreement.
I was military. He was federal civilian. The chain of command meant that his ability to enforce procurement rules against me was structurally weaker than his ability to enforce them against another civilian. We both knew it. The agreement, never spoken aloud, was that I would make sure the team had what it needed and he would not ask questions about how. I bought what needed to be bought. He never inquired about the line items he was not seeing. The mission moved forward.
I am not going to romanticize this. I should not have been spending my own money on operational infrastructure for the United States Marine Corps. The institution should have figured out how to fund its own red team. The fact that it did not, and that the team produced anyway, is not a triumph of grit. It is a failure of resourcing that the team papered over because the alternative was to fail the mission. I did what I did because the people depending on me were going to depend on me regardless of what the procurement system did or did not do, and the only acceptable outcome was that the team had what it needed when it needed it.
The lesson, looking back, is not that you should pay for your unit’s infrastructure. The lesson is that if the institution does not fund the work, the work either does not happen or it happens on the backs of the people doing it. Both of those outcomes are bad. I laid much of the groundwork to finally getting new gear by jumping through the many, many hoops in military procurement. Some of it even started to arrive shortly after I left the team and they have started getting more gear since, thankfully.
The End of the Stretch
There are more operations from that period than I can fit into a single post, and I am going to keep writing about them as I think of the right framing for each one. The longest op of my life is the next series I plan to publish on this site. There are others that deserve the same treatment.
What I want to leave this post on is the same throughline that has been running through the whole memoir series. The work was hard. The conditions were not the conditions you would design from scratch. The team was smaller than it should have been and resourced less than it should have been. The org chart worked against the work. And we produced anyway, because the people were the people, and the friendships were the friendships, and the mission was the mission.
Eventually I rotated off and out of the Marines. The reasons are in the intro post. The work I did at MCCOG with the team I worked with is the foundation of everything I have done since. I owe that team a debt I am going to spend the rest of my career trying to pay back. This series of posts is the start of it.
There will be more.
This was a stretch of my career working inside the Department of Defense, and the descriptions in this post have been generalized to keep the focus on the experience and the lessons rather than the tradecraft. No personnel beyond those who already publish under their own names are identifiable. No units, dates, locations, techniques, or outcomes are presented in a way that would identify specific systems or people. Where details have been generalized, they have been generalized on purpose.