Brian Hawkins, Author at MiaProva

Author: Brian Hawkins Page 1 of 3

Adobe Target Audiences and Locations

Over the years, we have gotten many requests from customers asking for clarity around how best to use Adobe Target Audiences and Locations when setting up Activities via the Visual Experience Composer (VEC) or Form-based creation.

This video will share best practices and explain how best to leverage the functionality.

Video Transcript

Adobe Target and Local Storage

Spoiler Alert: If you don’t want to know today’s Wordle answer or how to access the solution without playing, please do not watch the video below.

Local Storage

Most websites widely use local Storage to store and access data. This data doesn’t expire like cookies and can hold considerably more data than cookies. Because of this, it is no surprise that there can be rich data to arm a testing and personalization engine like Adobe Target.

This video shows how easy it is to get data from Local Storage and make it available to Adobe Target for first impression targeting.

Adobe Target targetPageParams documentation:

Code used to grab the Wordle solution from Local Storage to Adobe Target:

function targetPageParams() { 

		var str = window.localStorage.getItem("gameState");
		var parsed = JSON.parse(str);
		var solution = parsed.solution;

		return ("solution=" + solution); 


Activity Conversions Use Case

This week, we had a client reach out to us with an interesting use case with one of their Activities in Adobe Target.

The Activity was a test in their cart targeted to visitors who were making their third visit to that location. Adobe Target profiles easily support targeting such an audience but the Analysts needed to only count purchases if they happened during the same session as that third visit.

This video highlights how you can use Adobe Target ‘events’ and data like Profile Attributes as Activity Goals and still have the data in Adobe Analytics via A4T.

Video Transcript

And screenshots of the “Primary Goal Configuration” to automate the flow of this data into Analytics:

Adobe Target Host Groups

One of our customers here at MiaProva is expanding their testing efforts and use of Adobe Target well beyond their www and mobile site. They are bringing Adobe Target to their Email Platform, and we are helping them do so.

The magic that is making this possible is Adobe Target’s Delivery API. You have to call Adobe Target before emails get sent because you can’t put mboxes or Adobe Target server calls in email. The Delivery API does precisely this. It calls Adobe Target with data, just as a client-side mbox does on the site, and it gets a response based on any Activities and Audiences configured in Adobe Target.

Host Groups

The challenge this organization is dealing with was permissions within Adobe Target. The team developing this incredible capability for email testing with Adobe Target didn’t have the necessary Approver rights. Given they were new to Target and this organization has hundreds of live activities in place, handing out Approver rights wasn’t an initial option.

Host Groups was the fix, and after talking to a few other companies today, I realized a video might be helpful to the Target community, so here you go:

Video Transcript

Amplitude and Adobe Target

Amplitude is a Digital Optimization System that we have been asked about quite a bit lately. It turns out, MiaProva has some friends there from our Omniture and Adobe days, and so we got a much closer look at the solution and some of the use cases that organizations are using it for.

They offer a free trial that allows you to send 10,000,000 hits per month, and so we decided to kick the tires and see if we could create an integration for those organizations that use Amplitude and Adobe Target.

Image from:

Amplitude Data Consumption

We are by no means experts and have minimal experience with Amplitude, but from what we figured out so far, there are many different ways to collect data via SDK Libraries. Given that most organizations using Adobe Target leverages Javascript, we decided to create our Adobe Target custom integration based on their Javascript SDK.

Amplitude SDKs

Users and Events

Greatly simplifying things, with Amplitude, you send in a Visitors ID (or use theirs), pass in any Properties related to that ID, and any Events as you wish.

Here is an elementary example of passing to Amplitude that a Visitor is a “Cubs” fan:

var userProperties = {
    favorite_baseball_team: "Cubs"

This data would then be correlated with the User that was initially defined. An event represents something that has happened like ‘clicked,’ ‘searched,’ ‘hovered,’ ‘started a video,’ etc… Really anything that all and these Events can be Arrays and have Properties similar to how User data is passed. Here is what an Event call looks like:

var event = "MiaProva_trial";

We’ve configured this event to fire when a visitor arrives on our trial page: as a straightforward example. This documentation goes into much more advanced approaches as well.

Passing in your Adobe Target data

Adobe Target has some wonderful mechanisms to make data available for other solutions like Amplitude. Those organizations that use our popular Chrome Extension get to see Adobe Target Response tokens as one way to make the Activity meta-data visible. We did something similar with our approach to Amplitude.

We first had to decide whether we wanted to push the data as an “Event” Property or “User” Property. This decision has some significant implications for the Analysis and the management of data. While folks can debate the approaches, the best approach is a “User Property.” Adobe Target defaults to the Visitor counting methodology, and for excellent reason. Participating in a test is not like clicking a button or visiting a page, but rather something tied to the User so that it can be correlated to Events after entering into the Activity.

With that in mind, this is the single step needed to push your Adobe Target test data to Amplitude. Keep in mind; these steps assume you have Amplitude snippet in place or the npm/yarn installation. Modifications can be made as needed depending on how you manage Adobe Target’s deployment – we are using Adobe Launch.

  1. You need to add this code to the bottom of your Adobe Target at.js file. This code can be added via the Adobe Target UI before your download the at.js file or it can be custom code added via Launch to execute after Target is Loaded.
     document.addEventListener(, function (e) {
         if (e.detail.responseTokens) {
             var rt = e.detail.responseTokens,
                 ids = [];
             window.targetExperienceList = '';
             for (var i = 0; i < rt.length; i++) {
                 var inList = false;
                 for (var j = 0; j < ids.length; j++) {
                     if (ids[j] == rt[i]['']) {
                         inList = true;
                 if (!inList) {
                     targetExperienceList += (targetExperienceList ? ',' : '') + rt[i][''] + ':' + rt[i][''] + ':' + rt[i]['experience.trafficAllocationType'];
         if (window.targetExperienceList) {

             var values = [targetExperienceList];
             var identify = new amplitude.Identify().set('Activity:Experience', values);



Pretty easy, right!? Let us break things down so all can see what is going on with this code because developers can use this code to push data to any solution, including CDPs, data layers, etc…

Line 1: This is an event listener that will execute the code within it after a successful Adobe Target server call (mbox call:) has taken place.

Lines 2 – 19: We are creating a window scoped variable called “targetExperienceList” that will have a value that will equal a concatenation of all “Activities, Activity Experience, and Activity Allocation” that were tied to the response of the successful Adobe Target server call (mbox call:). On, this is what the value looks like:

You won’t typically see this sort of thing, but we have quite a few Activities running here for demonstration purposes. For those of you not familiar with “Activity Allocation,” that value is only pertinent to Automated Personalization and Auto-Target, where Adobe has Control and Targeted Groups. All other Activity types will have ‘undefined’ as a value.

Lines 20 – 26: Here, we are first checking to see if there is a value for “targetExperienceList”. This value would exist if the Adobe Target server call (mbox call:) had Adobe Target Response Tokens that showed that the Visitor was in a live Activity. If so, then it would execute Lines 22-24, which pushes all the Adobe Target data to Amplitude as User Properties. This could differ from the above User Properties because we treat the values like an array if the Visitor gets into more than one Adobe Target Activity on a single Adobe Target server call (mbox call:).

And there you have it:

Then you can analyze as you wish, create user cohorts (not available in free trial), create dashboards, visualize paths, etc…

Happy testing, folks!

Why are we paying for Adobe Target Premium?

Optimization Management is so much more than managing the intake and output of tests. A true Optimization Management Platform enables the organization to monitor tests or groups of tests and answers the big questions about their testing program as a whole and their investments in resources and technologies.

MiaProva automatically stitches the organizational meta-data related to Activities to data from Adobe Target and Adobe Analytics. This approach allows organizations to use this data to analyze and manage their tests and what offline variables are impacting the program – similar to how organizations use eVars and list variables in Adobe Analytics.

One of the many ways organizations use this is to understand how Adobe Target Premium is adding value to their Optimization Programs. MiaProva automatically classifies Activities that are specific to Adobe Target Premium as a dimension for Program Analysis. Doing so exposes a potential problem that needs addressing if limited or no value is being realized or creates an opportunity to expand on any value that is being realized.

MiaProva just onboarded two organizations that have made significant investments with Adobe and are heavy users of Adobe Target. Both organizations have launched more than 100 Activities launched since January of 2021. While mapping test tickets or test design documents to Adobe Activities to put all that data to life, MiaProva was able to immediately highlight for both of these organizations that they were not receiving the amount of value they should be with Adobe Target Premium.

In this blog post, we are going to highlight how we were able to identify this opportunity (and others) for these two organizations, what is being done to change things, and a great example of another MiaProva client who sees considerable value in Premium and MiaProva has used the ROI models to expand on their use of the Automated Personalization.

Optimization Program Macro-Analysis

MiaProva automatically creates dimensions related to the Adobe data such as “Adobe Target Premium vs. Standard,” A4T vs. non-A4T, Activity Types, Activity Sources, etc… Organizations use this data to manage live Activities and for all historical tests. While this is powerful (and very helpful for these two new MiaProva clients), the strategic value lies in tying the organization’s internal data to their optimizations efforts.

With Adobe Target, organizations have the tools they need to analyze individual tests. Organizations can go into Adobe Target and understand how the test impacts any metrics configured in Adobe Target. Lift, Confidence, segments, Success Metrics, etc…..all the great things you need to analyze an individual test.

And with, Analytics for Target (A4T), the analysis capabilities are exponentially more powerful by automating the inclusion of any metric, full use of organizationally accepted segments, metrics, calculated metrics, etc… Organizations can also look at all Activities in aggregate and use Activities and Activity Experiences as Segments which is a massive capability in and of itself that can’t be understated.

What is missing, though, is all the great data that is driving the Activity or test, to begin with. That is where MiaProva comes in by automating the process of mapping data from test tickets or test design documents to the rich data collected and managed by Adobe. MiaProva does this by way of MiaProva Programs – customize templates for their test design documents.

MiaProva Programs allow organizations to ‘dimensionalize’ any internal data related to tests just like MiaProva is ‘dimensionalizing’ the Adobe data like how we are distinguishing Activities that are part of ‘Adobe Target Premium.’ The use cases are limitless, but here are some that are being used by our clients today:

  • What idea source is leading to the greatest number of tests? the greatest lift against metric x, y, or z?
  • How are the tests distributed across organizational locations?
  • What audiences are being used across test locations?
  • How does our feature flaging solution (that we capture via an Adobe Analytics eVar) compare in terms of impact to revenue compared to tests that were executed via Adobe Target?
  • What value is Adobe Target Premium giving us? What is our win rate with Premium Activities vs. Standard Activites?
  • What is our ratio of Premium Activites vs. Standard Activities? What business units are seeing value from Premium and which ones are not using Premium?
  • What Product Managers are runing the most tests?
  • What is the distribution of Activities to Analysts?
  • Are HIPPO driven tests as effective as data-driven tests?
  • What is the ratio of Adobe Target ‘personalization’ efforts (pushing the winner or just targeting content) to actual tests

and we can go on and on. Any data that an organization would like to use is available for management for any Live Activities and all historical ones. This data can come by way of our APIs, JIRA integration, or as users supply tickets as part of their test management process.

Our approach to macro-analysis of optimization programs has enabled organizations to learn fascinating things and exponentially increase the realized value of investments in optimization. It is this approach that these two new organizations to MiaProva started to ask, “Why are we paying for Adobe Target Premium”?

Automated Personalization Playbook

Both of these new MiaProva organizations reached out to us to better understand what they saw in terms of our Program Analysis regarding their investments in Adobe Target Premium. It turns out the value they were realizing was non-existent or low was because they were not using the Activity features that come with Adobe Target Premium (Automated Personalization, Recommendations, and Auto-Target) correctly or in a minimal capacity despite licensing them for a considerable period of time.

Diving deeper into their use cases and their understanding of the Premium Activity types, it became clear that their understanding of Automated Personalization, how it worked, and how best to apply it was incorrect or less than ideal. We dove into the other Premium Activity types, but the investments made for Adobe Target Premium were tied to Automated Personalization.

We tried to point them towards documentation that could help, including a co-authored book on Adobe Target. Still, there was nothing out there that provided a soup-to-nuts approach to how best to use Automated Personalization. So, for these two organizations, and anyone else that may want to learn about Automated Personalization, we are committing ourselves to author what will be referred to as the Automated Personalization Playbook.

Give us a follow below to follow this series over the next month or two automatically:

This Automated Personalization Playbook will go very deep into what Automated Personalization is, how to give you the best chance to ensure success, and outline the best-in-class:

  • Framework
  • Execution
  • Strategy
  • Analysis

Stay tuned, as this will be a pretty powerful tool for those organizations that license Adobe Target Premium. It will also be exciting to possibly report on the change in ROI of these two organizations if they can adapt the strategy as outlined in this playbook!

Automated Personalization for the win

Another organization that leverages MiaProva is seeing significant ROI with investments with Adobe Target Premium. This company had applied a macro-view of optimization way before adopting MiaProva, which was helpful, but now MiaProva automates much of this effort.

This organization invested more than $3MM on an internally built engine that sits on top of its content management system. This system enabled business teams to establish rules for sections of their home page – very similar to how Adobe’s Experience Manager works. The team made heavy use of this solution, leading to significant increases in engagement and impact on key organizational metrics.

In an ongoing effort to continue optimizing, they tested this solution against Automated Personalization (using strategies outlined in the above-mentioned playbook). This effort lasted more than six months and spanned dozens of iterations. Without going into the details, here were the high-level results:

This effort clearly showed that humans are clearly smarter when it comes to applying rules to digital experiences based on data. This effort also showed the value of removing the cognitive bias that humans bring to decision-making and the value that AI could bring.

This ROI is managed by MiaProva and clearly visible in the MiaProva dashboard as a target that they continue to try to hit. This visibility has enabled this organization to demonstrate the value of automation, which has led to its operationalization to several other key areas for its digital consumers.

MiaProva Activity Sources

We added something pretty significant to MiaProva this week. Activity Sources.

As you may know, MiaProva is the only Optimization Management Solution to integrate with Adobe Target and Adobe Analytics fully. Any Activity type or optimization effort in Adobe Target is fully available for management within MiaProva. Any of your Adobe Analytics Segments, Metrics, Calculated Metrics, Events, etc….are also fully available as well. We’ve expanded our deep integration with Adobe I/O to now give our users Activity Sources.

Activity Sources represent different sources for optimization or testing related events. Adobe Target is a wonderful example of an Activity Source, and now MiaProva has broadened Optimization Management to include any data from any dimension in Adobe Analytics.

At MiaProva, our most successful customers have shown us that:

  • Data collected and reported on should have more of an optimization focus with ROI and learning management
  • Considerable opportunities exist within Adobe Analytics to expand on optimization and personalization over and above Adobe Target
  • Organizations need a centralized platform to manage all of their optimization efforts along with the efforts that stem from their testing and optimization solution

Adobe Analytics Dimensions

Adobe Analytics provides organizations with 75 Props, 250 eVars, and 3 List Variables. These variables allow organizations to report and analyze any data that is associated with them. MiaProva now treats each of these as configuration sources for Optimization Related efforts.

These additional sources also inherit the same MiaProva benefits that Activities from Adobe Target get including:

  • Real-time and automated reporting
  • Custom alerting and notifications based on any conversion event in Adobe Analytics based on defined thresholds and confidence levels
  • Centralized management with any other source, including Adobe Target
  • Program Dashboards and ROI models
  • Automatic creation of a knowledge library
  • Collaboration and risk mitigation
  • And everything else available within MiaProva…

Given the limitless ways organizations can use their Adobe Analytics Variables, the opportunities to treat the data collected by them like optimization initiatives are essentially limitless. Here are a few examples to give you a sense of the power this Activity Sources feature enables:

  • All acquisition initiatives (SEM, Email, Affiliate, etc.….) – huge opportunities here!
  • Product Finding Method eVar
  • Form type if B2B
  • Custom feature flagging
  • Internal server-side testing
  • Loyal program management
  • Page Categories
  • Search
  • Custom multi-armed bandits
  • Other testing solutions

Looking to the future, as organizations adopt and make use of Adobe’s Experience Platform (AEP), this Activity Sources capability by MiaProva will be even more valuable given the power AEP has to collect and manage data from many sources. We’ve built this capability for today with the future in mind, given the considerable opportunities that exist with Adobe’s Journey Optimizer.

Configuration in MiaProva

Organizations can easily configure Activity Sources right within MiaProva’s Admin Console as seen below.

MiaProva Activity Sources Configuration

Here organizations can add any of their Adobe Analytics dimensions (eVars, list variables, sProps, etc…) as an Activity Source.

Simply provide a name, the dimension ID, and a value pattern and MiaProva will immediately take over and provide management of these sources along with your Adobe Target Activities!

Below highlights how one can Access or manage these different sources within MiaProva’s Live Dashboard.


Adobe Target and Customer Data Platforms (CDPs)

If your organization leverages Adobe Target for their optimization and personalization efforts and you have not joined your internal Visitor ID to Adobe Target’s visitor ID, consider making that a top priority. This capability has existed for quite some time but is now getting a renewed focus mainly due to ITP, browsers like Chrome are changing how cookies are managed, and Customer Data Platforms (CDPs).

Joining your internal Visitor ID to Adobe Target’s ID takes just a few minutes to complete, costs you absolutely nothing, immediately creates cross-device test coordination, and opens the door wide open to use ANY (except PII, of course:) of the data that you have mapped to that ID. We recommended this blog post or this article by Adobe to see how to do this.

Adobe Target 1st party data, 2nd party data, and 3rd party data

This effort is, by far, the most valuable and highly strategic component of Adobe Target. Doing this converts your optimization program from a 2nd Party one to a 1st Party one where all the data is based on YOUR organization’s ID, not just Adobe’s. All of your Adobe IDs across your various browsers and devices will now map to your organization’s 1st Party ID.

Once this exercise is complete, your organization has a 1st-party optimization program which means ITP and changes to cookies are managed within browsers will not impact test execution and analysis. In fact, a 1st-party optimization program not only future proofs investments in testing and personalization but also enables your organizations to expand their efforts considerably with customer journey optimization.

Okay, so now you have the 1st Party ID in place, now what…

Configuring Adobe Target to become a 1st-Party Optimization program is the easy part. Historically, and still quite common, the more challenging part is figuring out where internally at your organization the 1st Party data is located that is mapped to that visitor ID that you now have mapped to Adobe Target. Then once you know that, the challenge then becomes how best to get it to Adobe Target.

The where…

Organizations vary considerably in terms of where this rich 1st-Party data exists. The data could be located across several different systems and spread across an array of different teams. The data might be in data-lakes, data layers, cookies, completely offline, and in some cases not joined to the Visitor ID you used but rather other definitions of a Visitor ID. Users of Adobe Target, those that execute the Activities, may also not have access to the systems or teams that manage this 1st Party data.

Data compliance is another important factor concerning where the data is located. Data compliance is the practice of making sure that the data being managed is guarded against misuse. Depending on where the data is located, it may or may not be data compliant – safe to use for testing and personalization within Adobe Target.

As you can see, finding safe-to-use data can be a challenge. Thankfully, some organizations have overcome this challenge and established governance and processes to get the data. This then leaves them with the “how” to get this data into Adobe Target for their personalization and testing needs.

The how…

Once you know where the data is, Adobe Target provides quite a few options to get the data into Adobe Target. Depending on the format your data is in, you may also have to modify the format to leverage one of the options available. Here are the main mechanisms organizations get data into Adobe Target for personalization and targeting Activities:

  • Adobe Target API – here you can update single Profile attributes or use Adobe Target batch files to upload as many as 1,000,000 Visitors per day (I know of several organizations that exceed this considerably and it works wonderfuly)
  • Adobe Target adbox – this is an old-school tool that many of us old-timers use from time to time. It is simply a pixel that passes data to Target. Very helpful from taking data from email platforms and passing it to Adobe Target upon email open.
  • Client-side or Server-side via parameter value pairs – just like your Analytics tags collect data that Adobe Target mbox calls as well. This is a very popular approach given the use of local stroage and datalayers and Adobe Target’s ability to easily consume that information.
  • Adobe Analytics Customer Attributes – those organizations that make strategic use of Customer Attributes are using this approach to very creatively to get 1st party data into Adobe Target. Adobe converts these attribute to Adobe Target Profile Attributes automatically.
  • Adobe Audience Manager – within Audience Manager, organizations can create segments as they wish and share them to the Experience Cloud, which makes them available within Adobe Target. Quite a few of the organizations that use MiaProva, leverage this approach.

As you can see, the highways are built and are ready to use. Adobe has done a fantastic job giving organizations different options to get the data that they need to Adobe Target for personalization and testing. Despite these mechanisms, organizations still struggle to establish the infrastructure to automate and maintain the data transfer despite the considerable value.

The value…

Out of the clients that use MiaProva and the organizations that we work with at Demystified, it is safe to say that only about 45% of them leverage the functionality that enables 1st Party optimization programs. The reality of it all is that while the functionality is there and the value is massive, organizations often struggle to take advantage of these capabilities. Learning the capabilities, connecting the pipes, establishing process and governance, etc… are barriers that many organizations can’t overcome.

Those organizations that have been able to overcome the barriers realize significant value. Here are just a handle of many examples of organizations and what they are doing with their 1st Party Optimization Programs:

  • For a $1.2B retailer, weekly (on Sundays), they use Adobe Target’s batch processing for Profile attributes processing in-store transaction data so they can be used in key Activities by the various testing teams for customer journey optimization.
  • A $900M retailer, before each email blast, they pass key profile attributes via a pixel call to Adobe Target that is specific to that email in the event the visitor “view throughs” to the website vs. clicking through. This effort was instrumental to coordinate pricing threshold messaging offline and online.
  • For a $7B retailer, 1st Party data is made available as Response Tokens so that their CMS can use the data for content decionsing and rules.
  • For finance, a $20B financial institution, using Audience Manager and Adobe Target batch APIs is allowing them to give Automated Personalization very rich 1st Party data for modeling and content decisioning. Additionally, considerable personalization efforts via Target are rooted in this data.
  • A $39B financial instituation is using client-side and server-side parameter passing Tag Management for very advanced use of Automated Personalization along with rich analyis via A4T. Additionally, this approach enables their internal call centers to mimic experiences of the digital consumers by way of the cross-device test coordination.

The challenges around “where” the data is and “how” to use it are not specific to personalization and optimization efforts. The increasing demand to make data actionable, connect multiple data sources, and provide data compliance expands well beyond testing to all marketing and data science components.

This perfect storm has created a vacuum that is being filled by something called Customer Data Platforms (CDPs).


What is a CDP?  This is not a simple answer by any means, as CDPs providers have a wide range of features and capabilities that align and manage consumer data for organizations.   CDPs can make the “where” and the “how” considerably easier than it has to be, and some CDPs can automate those efforts.

We are not going to go into all that a CDP is here. At a high level, a CDP is all about managing an individual customer with a single profile. CDPs have visitor stitching algorithms, provide data compliances, and make the data available to different departments and solutions like Adobe Target, for example.


MiaProva customers vary considerably in terms of CDP use with testing and optimization. Those that do can automate and accelerate their use of 1st Party data for testing and optimization. Our clients work with several different CDPs, but I wanted to call out ACTIONIQ specifically because they work with Adobe Target to bring the 1st party data to life. 

I’ve highlighted above how organizations have to establish processes and governance around the “where” and the “how”, but here is how ACTIONIQ automates that process seamlessly, specifically for Adobe Target, so that testing teams can focus on strategy and execution. 

  1. Real-time syncing – as this first-party data gets updates, these guys update the Adobe Target Profile as events take place across other channels 
  2. Batch synching – weekly, every 10-days, Sunday nights, they can automate the process of porting Adobe Target Profile attributes for personalization and testing.  Much easier than gathering a file internally and uploading it from your PC. 
  3. Porting Adobe Target Activity Meta-data to other solutions.  CDPs shouldn’t be limited to feeding Adobe Target data about the digital consumers, but they can also consume what Activities or tests your digital consumers are members of.  Consider how helpful it would be for internal Call Centers to have visibility into this information?  Or to coordinate these onsite personalizations with initiatives with Social or with Acquisition initiatives?  
  4. It is related to #3 above, impression management.  If you are using Adobe Target to show digital consumers an offer through an Activity, you can control in Target how many times they see it.  This is better served in a CDP where you can coordinate impressions of ‘offers’ or content at a macro level across channels and mediums.  

MiaProva and CDPs

The customers of MiaProva that use CDPs as part of their personalization and testing efforts are increasingly using the data the facilitate Customer Journey Optimization. Customer Journey Optimization is where organizations have testing roadmaps with targeted testing and personalization, advancing customers across their various customer journeys.

MiaProva supports those efforts by way of our MiaProva workstreams, where we automate the management of metrics, segments, tickets, activities, and ROI based on these workstreams – making it more efficient to advance the throughput of these efforts.

Sample Ratio Mismatch (SRM)

Sample Ratio Mismatch (SRM) is a thing, a real thing that optimization programs need to concern themselves about. At a high level, SRM means that the variants (experiences) of an A/B test have a traffic distribution (sample) that differs in a statistically significant way from what you expected the allocation of traffic or the sample to be.

This blog post and this blog post are great as they dive into what it is, how scary it can be, and what to do about it. Please do check out both those posts as they drive home just how important of a topic this is for optimization programs.

Sample Ratio Mismatch and MiaProva

About 5-6 weeks ago, MiaProva quietly released a new Alert to our array of alerts we provide for testing programs. This Alert will notify MiaProva customers that use Adobe Target automatically when SRM has been detected. This approach automatically applies a Chi-Squared calculation to every single A/B Activity currently Live in their Adobe Target accounts.

When MiaProva detects these issues, we will immediately notify stakeholders via email and highlight the Alert in the Live Dashboard and the daily Aggregate email that summarizes all Activity-related Alerts.

MiaProva Alert Management Dashboard

In the last month, we have had SRM detected a handful of times across many hundreds of Live Activities from different organizations. With SRM, the most important thing is identifying the root cause of the mismatch. Two out of the five are still being investigated, but the other three were quickly diagnosed. The statistically significant issues were:

  • During QA, an internal team became members of one of the variants and maintained that membership when the test went live. Incorrect targeting used in the Activity setup further exacerbated this SRM issue.
  • A bot had gotten into one of the variants.
  • Improper use of the Advanced settings upon Conversion, the selected Action below, combined with the audience criteria, caused an imbalance of sample distribution.
Adobe Target Conversion Advanced Settings

Had MiaProva not detected and alerted people to this key issue, organizations could have found themselves in a situation where they were making decisions on statistically invalid test results. Furthermore, because MiaProva detected SRM, the organizations could remedy the underlying causes, thus preventing additional reporting issues.

Below represents what SRM alerts look like in context to Live Activities that are from Adobe Target. MiaProva also provides an SRM calculator with each Live Adobe Target Activity that is an A/B Activity. Automated Personalization, Experience Targeting, and Auto-Allocate Activity types have logic to promote differences in sample distribution across Activity variants. MiaProva provides this service for all A/B Activities, Multivariate Activities, and Adobe Recommendations.

MiaProva alerts on Live Dashboard

And lastly, here is what the calculator looks like and how users can adjust their Expected Visitor distribution as needed.

MiaProva SRM calculator

Using Adobe Target to test Adobe Experience Manager

I imagine many folks may find this post fascinating given the amount of Adobe Experience Manager (AEM) adoption as of late. It seems like about half the companies I work with are either being sold AEM, are onboarding it, or have already rolled it out.

As an Adobe Target practitioner, I can definitely appreciate and value the integration between AEM and Adobe Target. Especially helpful are:

  • Experience Fragments – AEM can automatically export content to have available in Adobe Target as offers. Very helpful for the highly democratized organizations with limited technical expertise. This solution provides a wonderful way to mitigate risk as well. This approach helps organizations that want to move away from modular mboxes to the global mbox approach to manage all test-related content.
  • Productization (operationalizing test winners)- When Adobe Target finds a winner, you don’t want to rely on Adobe Target to maintain that experience for that audience. The Audiences in the Activities are often pretty sophisticated, and AEM does a pretty good job of mirroring rules used in Audiences.

Deploying AEM via Adobe Target

One of our MiaProva’s customers decided to contract Analytics Demystified to provide Adobe Target training and strategic support to their growing optimization practice. During this engagement, this customer also licensed AEM and asked if we could provide support and a model to leverage Adobe Target to throttle to AEM and test the impact of AEM against their organizational KPIs, and so we did. We evaluated their use of Tridion, provided support for Adobe Target server-side, created the Activity and offers, and provided the monitoring via MiaProva and Adobe Analytics.

AEM vs. Tridion

The test design is actually pretty straightforward. The approach that was decided on provided a great introduction to the organization to the server-side testing capabilities of Adobe Target. The following video shows the approach and how you can easily use Adobe Target to quantify the impact of AEM and leverage Adobe Target to throttle AEM’s rollout.

Steps taken in video:

  1. Leverage Adobe Target server-side capabilities (video uses the delivery API as an example)
  2. Configure an Activity in Adobe Target using JSON offer type
  3. Configure the default CMS or server to trigger AEM or other CMS based on the server-side JSON response from Adobe Target

Code used:

Experience A:

"redirect": "false",
"domain": ""

Experience B:

"redirect": "true",

Page 1 of 3