Drawing from this conversation, I think, if you were taking @Modiphius-Nathan’s position, if wouldn’t be a test to simply move your character from one zone to another unless you were doing it boldly or subtly.
I see your point though. If you’re in a skirmish level conflict and you move into an enemy’s zone, do any defensive assets hinder you? It doesn’t seem like they should. In your example, say all 4 of those NPCs have shields and rapiers (each at Quality 1 just to keep things easy). That’s 8 defensive assets in the zone. Therefore moving a character or asset into that zone would be +8 wouldn’t it? That’s what you’re questioning, right?
It does seem like the defensive assets increasing Difficulty for moving into zones really only makes sense in a duel to me which means that something is probably off. I know the systems were designed to be pretty much the same no matter what scope you’re playing in.