That 0.1% Overlap in Your A/A Test Is Not a Bug. It Is the Best Feature You Own.

A client called us last week with a very reasonable question. They had built swimlanes in Adobe Target using profile attributes, ran an A/A test to validate the setup, and pulled the numbers. The split was beautiful. Roughly 273,000 visitors in one lane, roughly 272,000 in the other. Textbook.

Then they broke each lane down by the experience that was actually served, and found 148 visitors in Lane A who had also seen Lane B, and 148 visitors in Lane B who had also seen Lane A.

Their first instinct was that something was broken. It is a fair instinct. But the symmetry is the giveaway. A broken profile script does not produce identical counts on both sides at one tenth of one percent. What they were looking at was not a leak. It was Adobe Target recognizing a person and doing exactly what you would want it to do.

Here is what is happening, and why it is worth being excited about rather than worried.

Two IDs, one person

Target keeps a profile for every visitor. By default that profile is keyed to a device level ID that Target generates on its own. If you clear your cache, reinstall the app, or open the site on your laptop instead of your phone, you get a new one. That ID knows nothing about you.

The second key is the one that changes everything. It is called mbox3rdPartyId, and despite the unfortunate name it has nothing to do with third party cookies. It is the opposite. It is the slot where you hand Target your own customer ID. Your loyalty number. Your CRM ID. Your hashed email. Whatever your business already uses to know who someone is.

When you pass that value, you are telling Target: this device belongs to this person.

The merge is immediate, and that is the whole point

This is the part people underestimate. When Target sees a first party ID it has encountered before, it does not queue up a job. It does not wait for an overnight sync or a batch profile update. The merge happens inside that request, at the edge, before the response comes back.

The known profile is found, the two profiles are reconciled into one, profile scripts re-evaluate against the merged data, and the personalized content that ships back to the browser or the app already reflects everything you knew about that person.

Think about what that means in practice. A customer who has never touched this device before signs in, and on the very next content call your site already knows their loyalty tier, their lifetime value, the category they were browsing on a different device last Tuesday, and which experiment lane they belong to. No cookies. No probabilistic matching. No waiting.

That is cookieless personalization working the way it is supposed to work, and it is happening at edge speed on infrastructure that sits close enough to the user that nobody notices the lookup happened at all.

So where does the 148 come from?

Now the overlap makes sense.

A visitor opens the app. They have not signed in yet, so no first party ID is on the call. Target has never seen this device, creates a fresh profile, runs the profile script, and assigns the visitor to Lane A. That impression is logged and reported.

Seconds or minutes later, the visitor signs in. Now the first party ID comes through. Target finds a profile it already had for this person, one that was pinned to Lane B from a previous visit or a different device, and merges immediately. The swimlane attribute flips to B. Every hit from that moment forward reports B.

One human being. Two experiences on the record. Nothing you can do to unwind the first impression, because it already happened.

Why the report shows both

There are two different things being measured here, and once you see the difference the numbers stop being confusing.

The profile holds one value at a time. When the merge overwrote the swimlane attribute, the old value was gone. That is why the two top level segments add up cleanly to your total visitor count. The segment resolves to whatever the profile says today.

The hit history is different. Every content delivery stamped the experience that was actually served at that moment, and those stamps are permanent. The breakdown is not asking what the profile says now. It is asking what was served then. For 148 people, the answer was both.

Is 0.1% a problem?

No. For most implementations it is the expected cost of personalizing traffic that starts out anonymous and finishes authenticated, across multiple devices, in a world where people reinstall apps and clear caches.

A few things quietly stack on top of the same root cause. App reinstalls mint a new device ID while the customer ID survives. Profiles that time out for infrequent visitors get reassigned when those visitors come back. Any profile script whose inputs only exist after login will return a different answer once the merged data arrives.

If you want to confirm the diagnosis rather than take my word for it, look at whether those 148 cluster around sign in events, and check whether the flips run mostly in one direction in time. Anonymous lane first, authenticated lane second. If they do, this is a merge, not a defect.

If you really want it closer to zero

The lever is entry timing, not the script. Hold activity entry until the delivery call carries the first party ID, or only set the swimlane attribute when that ID is present and let pre-authentication traffic fall through unassigned.

You will trade sample size and pre-login coverage for a cleaner lane. For A/A validation that trade is rarely worth it. For a persistent swimlane that downstream personalization depends on, it usually is.

The takeaway

The client came in worried about a rounding error. What they had actually built was a system that recognizes a real human across devices, collapses their history into one profile the instant they identify themselves, and puts every attribute you own to work in the same request.

The 148 is not the story. The 545,000 correctly identified visitors underneath it are.

If you are running swimlanes on profile attributes, go pull this breakdown on your own activities. Not because something is wrong, but because you should know what your overlap looks like and why. A tenth of a percent tells you identity resolution is working. Five percent tells you something about your entry timing deserves a look. Zero tells you nobody is authenticating, which is its own conversation.

If you want a second set of eyes on your Target profile architecture, your first party ID strategy, or how any of this lands in your reporting, that is the work we do every day. Reach out and we will take a look.