WIth Adobe Target, you can have two Activities work together to assign users to groups after some behavior has occurred or any other event that you would like. This video outlines this very helpful approach.
okay so we have another very cool
question from measure slack
regarding um
having visitors via adobe target be
assigned to either a test or control
group
after a certain behavior has done on the
site and so i thought this was very cool
the approach i’m recommending here is
something i use all the time there’s a
few different ways to do it depending on
what the behavior is uh so if there’s
more details could be provided than you
know mousetrak and what have you um so
essentially how i would approach this
sort of thing is having a parent-child
relationship so you would set up an
activity in adobe target using the
global mbox and that would have code in
it to do your experience a or experience
b
then what you’ll want to do is include
this code here something similar to it
this is a nested m boxes or additional
server calls to adobe target so
experience a
whatever your offer code is whatever you
want you know for whatever behavior
you’re tracking if you want to put
people in group a or group b
same thing down here for experience b so
you have this mbox name which is the
same but then you have an mbox parameter
called experience a and experience b
then you go into adobe target and you
set up the child activity so this will
be the child activity
that you will use and you can support a
for t because it’s server to server so
even if this unbox call fires after the
analytics called it you’re fine
and then just add in audience
refinements not here
that’s for the location you don’t have
to worry about it because technically
this mbox should only fire from the
parent activity but add a change in
audience here and then you would add
experience targeting here and this would
be your
code that
box parameter
experience or exp equals a but you can
change this to anything that you like so
yeah it’s a quick way to put people in
groups depending on behavior
lots of applications for all sorts of
things that can happen
after the pageview has taken place so
hopefully this helps
the other day i posted a just a real
quick two minute video uh regarding mia
provo’s live dashboard and i got these
uh comment earlier today some some great
questions and i thought it’d be easier
uh to get together a quick video to
address uh each of these great questions
here
so first question uh and thank you for
the comments please hit me up with
questions making videos is now really
easy and i enjoy doing them i can turn
them around pretty quickly so keep keep
stuff like this coming please uh is this
dashboard basically an improved
dashboard or over the native adobe
target one so the video had to do with
the mia provo live dashboard um it’s
different uh the adobe target
activities list page has a list of all
the different activities i’ll quickly go
to it really quick here so adobe target
you go to the activities list page so
here you can see all of the different
activities you can create activities see
by works uh workspace
mia provo what we only do is import any
live activities we want to keep things
short sweet and keep things manageable
and very clean uh the one of the main
points to me of proba is the ability
to add all of these to the knowledge
libraries so we don’t want like poc or
junk activities uh in media provo so
we only deal with live activities
organizations can also add filters
remove report suites uh so all of that
control actually jump over to a
different account here really quick to
show you that this account’s actually
connected to adobe
so within here you can you can control
like not only
what report suites are used for a4t
dimensions but you can configure how you
want the reporting to look this is a
huge thing around our journeys
configuration which i plan to do some
videos hopefully later this week with
that but you can define this how you
want to consume the data we
automatically calculate the a for t
calculations that sort of thing so it is
similar to the adobe dashboard but it’s
only for live activities and the default
view
mia provo also allows on the live
dashboard the ability to quickly add
programs like categorize it because we
provide program analytics like how are
you using adobe target i think adobe
target is probably one of the best
content management systems out there
the
majority of our activities in the me
approval world are not really tasked
with targeted messaging and so we allow
organizations to categorize it and
provide reporting around that as such
um okay the great question uh does the
does it have features like notify when
someone created a new activity
yes absolutely so i’ll jump over to this
account here as well
so these are all the live activities so
just by turning it on in mia prova
or turning it on in adobe target it’ll
show up in mia proba it’ll initially
show up as yellow which means that it
doesn’t have a ticket associated with it
uh green means that it’s met its preset
sample size one of the big things we’re
proud of is our throughput uh we’re able
to look at before mia provo and after
mia provo and because we notify you know
when things have met sample size and
when their statistical confidence uh we
highlight all of that as well you can
see some of the alerts uh within there
as well you can see the audiences you
can see one of our alerts about
performance again all customizable by
journey
um but to the question regarding
notifications yes we uh we have a huge
alerting i’m going to work on a video to
show that because we’ve made some
updates to that these are all the
different alerts that we do and you can
see who can see it by
user role within me at provo but the
outbound notifications are exactly that
so when ideas are added when a ticket’s
been created when it’s ready for qa move
to under submitted all of the typical
workflow including like activities
launching reaching statistical
confidence
you can get them either in slack via
email and based off of different roles
within the appropriate we manage within
here
we also send out real-time alerts when
we detect a change in the code
when the code conflicts with where the
test is running as well as
kpi metrics
but below or above predefined thresholds
in a different part of the ui and then
every day
around three o’clock in the afternoon us
time we send out an aggregate email
about all the different activities and
and how they’re doing uh within mia
provo and then all of that is obviously
reflected uh within the live dashboard
view here as well as the kanban board
where you can see across the various
states where this column here is the
column that gets uh
triggered from adobe where you can see
all the different alerts and all of
those things within there so great
question there
can it raise some kind of alarm if
someone uh adds a new activity without
an end date um yes um
without an end date uh so we highlight
that that’s a great question so
we automatically
we automatically import activities if
they’re live and they don’t meet the
particular thresholds uh but let me jump
over to this account here but this
calendar view
this calendar view is based off of the
activity dates uh configured within the
adobe target activity so
um this first is if there’s got a preset
sample uh preset date which none of my
activities do but the evergreen
activities represent that there’s no uh
end date and so you would see that here
in this particular view of the calendar
um
you can also create a field within the
ticket uh as well but you can see here
the evergreens do not have an end date
um if we also have the under development
period of time which is back to the end
of the under development ticket window
um
so to answer your question you get
visibility into that but we don’t have
an op on alert if it doesn’t have an end
date it’s something we can easily add
though because we have that information
if you needed to be notified
can it map activities to a user yes
absolutely so that’s this is something
that we do so mia provo um the live
dashboard as well as the kanban viewed
and the program overview
it’s all about managing your efforts in
adobe target or data from adobe
analytics and coupling it with your your
organizational metadata so you had a
question about the live dashboard and
who the
um
managing by who who created it so these
are the fields you get from adobe i
highlighted this in the in the original
video about the priority
activity name last modified all that
good stuff these are other things from
adobe whether it’s a for t not a for t
what type of activity is this is mia
prova uh because mia provo allows for
multiple sources to contribute to what
shows up in the live dashboard your
question though about who started it yes
so um all of these activities the ones
that are not yellow
haven’t have a ticket associated with it
and we call those programs so you can
create
any dimension that you want and
customize these any which way that you’d
like we import them from jira we’ve got
a really super awesome integration with
workfront coming that will automatically
import them
but here’s an example of where companies
are including product managers users you
know other different dimensions a lot of
companies are doing like um
idea source so really any organizational
metadata that you’d like to include
including the
creator can absolutely be used you can
filter on it within here you can also
filter on that within the kanban board
across all of the different various
states uh and then you can actually
generate graphs reporting around the um
the count uh of users as as well that
are creating particular activities so uh
thank you rafael good questions if
there’s any more please let me know and
if anyone else has any questions shoot
them my way i’m happy to answer them
right away
MiaProva’s Live Dashboard provides organizations with various tools to manage any Activities in flight. This quick 2-minute video gives you an overview of some of the functionality available here.
okay here’s a super short and super
sweet uh overview of mia provo’s live
dashboard so
when we connect to adobe io
whether you’re using adobe target or
data that’s going to adobe analytics if
it’s live if it’s in flight we are
automatically importing it here within
mia provo on the live dashboard what i
want to highlight was just some of the
features
that we make available here obviously
you can sort and refine with very
similar
refinements that you have within adobe
target but i also want to kind of
highlight you can sort by obviously last
modified activity id
one thing that we added recently last
year was priority uh the activity
priority which is pretty neat and uh all
of the data whether it’s a for t or non
a for t is in real time you can actually
refresh the data we automatically do a
day over date graph for all the data as
well this is a demo account
but you get a sense of some of the data
that we include we also automatically on
the live dashboard highlight the
audiences that are being used as well as
any alerts
that’s represented here by the
exclamation point as well
and then screenshots are available here
as well but i also want to highlight the
search functionality so not only can you
search by
the activity name the adobe target id
which is this this value here and also
the value that we highlight here in our
chrome extension
this is the activity ids all of these
activities are running here
but you can also search on where the
test is running so for example if test
is running on our home page you can
search and all of the activities that
are running there will quickly and
easily show up you can see it’s part of
the audiences within there you can also
search for any of the offer code that’s
actually being used in any of the
activities so you can see here i’ve got
an alert and all of those are showing up
automatically so pretty cool
functionality but you also can
incorporate any of the custom filters
that any of your mi approval program
tickets
and any of the mia provo metadata such
as the programs the labels uh things
along those lines so that’s it quick
overview in terms of what the live
dashboard is and how it works
MiaProva regularly makes product releases once or twice a month on average. This video outlines some of the updates made this week.
Monday, August 22, 2022 Product Release
– UI updates to the Idea Lab
– Adding Adobe Target Audiences and Adobe Target Offers to Program fields
– Bug Fix: Removed Alerts sent via email from disabled Report Suites
– Various architectural and performance enhancements
– New Filter on Awaiting Summary
– Updates to naming conventions under Notifications
As always, please reach out with any questions and, Customers, if there are features that would be helpful, please let us know at support@miaprova.com
hello very excited to report some
product enhancements that took place
over the weekend and so this video is
for today monday august 22nd 2022.
uh we’ve made some various improvements
in terms of how we’re going to be
rolling out these communications because
we do roll out quite a bit of features a
couple times a month sometimes more than
that we’ve got some really exciting
features coming up that we’re going to
make sure that the our community as well
as the adobe target community are well
aware of here are the updates for this
week
first one ui updates on the ideas lab so
i’m going to be jumping around mia
probably here really quick to show you
this so this is our ideas lab we made
various improvements to make it easier
to kind of prioritize uh different ideas
by labels uh move it over to here to
this kind of pre-test ticket queue so we
made various improvements there we also
fixed a minor css bug in here for uh one
of our clients had a very very lengthy
label name these are different labels
within here
but we also built on making it easier to
create a ticket directly from uh this
idea lab where you just go right in
create a particular ticket the test
hypothesis moves automatically over
there you can also update what program
is being applied that sort of thing
makes it very very easy so that was a
cool enhancement that we rolled out
next we added adobe target audiences and
adobe target offers to program fields so
we’re going to be expanding on our
knowledge library and we’re going to
make it public so so people can learn
more about our
solutions so
on the live dashboard we’ve been doing
this for for several months now we had a
good suggestion from a customer uh to
apply in
audiences and offers so now on our live
dashboard not only do we provide like
that quick insight in terms of how the
test is doing as well as real-time
reporting and a default day over day
graph
of what’s going on um you can see the
audience that’s being used and what mbox
is is is you know using that particular
audience very very similar to the swim
lane management view that is very
popular
but we brought it up on a proactivity
level and what we wanted to do is based
off of customer feedback is give
organizations the ability to manage this
data across their various me approva
programs
for those of you not familiar with
programs mia provo offers lots of
different ways to customize the intake
of the different
test requests that come in
and so organizations will customize this
any which way that they’d like we
provide full functionality based off of
any type of
you know field and field type that you
can imagine so we provide a default
template here mia provo default and that
has it the adobe target audiences as
well as offers but i’ll jump down to a a
different uh ticket that’s or different
program and so you can see some of the
built-in different types within here so
i’ll just add in audiences add in that
field and so you can see we’re quickly
adding it there
i’ll go ahead and quickly add in adobe
target offers which is nice and so
what’s very helpful here especially in
finance
for regulatory reasons they want to keep
track of everything now when they
archive an activity within mia provo uh
along with the data that’ll be a
permanent record created we’ll also
create a record of the audiences that
were being used and any of the offer
code we are we already did this but now
we put this in the ui to make it a
little bit easier for organizations to
manage so simply do that
now whenever that
program is used and tied to an activity
boom it’ll automatically have that data
with it
uh we had a small bug fix uh so mia
prova as you know does a lot of awesome
reporting
and alerts based off of different things
and so
we also allow organizations to limit
what activities show up for management
within mia proba and so you can see here
you can disable certain report suites
especially if all if your a for t
configuration has many report suites
within it um but we were still alerting
for some of the different alerts we we
sent out a couple emails um by way of
our notifications as well as aggregate
alerts when when different things happen
and and some of these were triggered
from activities that were disabled
within mia provo but we were still doing
that back-end alerting so we disabled
that service because it made sense if
you want to disable the service that
means the alerts as well which is cool
uh we also had some various
architectural and performance
enhancements
my engineering team which way smarter
than i can speak to details about that
but i understand they were pretty
significant and they took some time but
uh we’re very very happy with the
outcome of that and
users should see that in the
using the platform as well uh last two
things here uh i’ll actually come with
this one really quick and then jump to
this one at the end because i think this
one’s pretty exciting so um
perfect i’m here so our notifications we
send out notifications based off of
events or alerts or things along those
lines um historically mia provo reported
you know
called its tickets like tests so if you
submit a ticket you’re submitting a test
request if a test request moved to under
development or awaiting summary that was
different events that would happen
um
we had some discussions talked to some
clients so we’ve rebranded that to be
ticket created i think it makes a lot of
sense um you know a ticket was moved to
a submitted state that sort of thing
so we made some updates there uh the
last thing i want to report on very
quickly is
an update to the awaitings final summary
so let me just clear all the filters
here um i should update this we we have
the functionality persist filters that
was recently added uh uh some time ago
uh but on this particular state and you
can actually update and see this state
within here
to do so this is the status board or
kanban board which gives you a wider
view and everything but i’ll jump back
into here
we added this new filter so in this
state this is where those tickets live
when
activities have been recently launched
or there they’ve recently come down or
they they just haven’t been um
the learning hasn’t been applied to it
and so some of our clients had some
backlog uh here you you know with a few
dozen tickets that that haven’t been
archived um and so in talking with them
we wanted to make it easier to filter on
that so activities that are on that or
if activities have recently ended you
might want to prioritize uh working on
uh getting those particular activities
archived uh and then we also have the
ability if there’s no activity that is
paired with it you can easily access
that as well uh which we highlight as
yellow which makes it easier to say okay
these tickets are out there
and they haven’t been associated with a
particular activity at this time so
thank you we’ll have a lot more updates
to come and if there’s any questions let
us know and as always if there’s any
features or functionality to be helpful
certainly let us know we have a feedback
throughout our entire ui thank you
Amplitude is a Digital Optimization System that we have been asked about quite a bit lately. It turns out, MiaProva has some friends there from our Omniture and Adobe days, and so we got a much closer look at the solution and some of the use cases that organizations are using it for.
They offer a free trial that allows you to send 10,000,000 hits per month, and so we decided to kick the tires and see if we could create an integration for those organizations that use Amplitude and Adobe Target.
We are by no means experts and have minimal experience with Amplitude, but from what we figured out so far, there are many different ways to collect data via SDK Libraries. Given that most organizations using Adobe Target leverages Javascript, we decided to create our Adobe Target custom integration based on their Javascript SDK.
Amplitude SDKs
Users and Events
Greatly simplifying things, with Amplitude, you send in a Visitors ID (or use theirs), pass in any Properties related to that ID, and any Events as you wish.
Here is an elementary example of passing to Amplitude that a Visitor is a “Cubs” fan:
var userProperties = {
favorite_baseball_team: "Cubs"
};
amplitude.getInstance().setUserProperties(userProperties);
This data would then be correlated with the User that was initially defined. An event represents something that has happened like ‘clicked,’ ‘searched,’ ‘hovered,’ ‘started a video,’ etc… Really anything that all and these Events can be Arrays and have Properties similar to how User data is passed. Here is what an Event call looks like:
var event = "MiaProva_trial";
amplitude.getInstance().logEvent(event);
We’ve configured this event to fire when a visitor arrives on our trial page: https://www.miaprova.com/trial as a straightforward example. This documentation goes into much more advanced approaches as well.
Passing in your Adobe Target data
Adobe Target has some wonderful mechanisms to make data available for other solutions like Amplitude. Those organizations that use our popular Chrome Extension get to see Adobe Target Response tokens as one way to make the Activity meta-data visible. We did something similar with our approach to Amplitude.
We first had to decide whether we wanted to push the data as an “Event” Property or “User” Property. This decision has some significant implications for the Analysis and the management of data. While folks can debate the approaches, the best approach is a “User Property.” Adobe Target defaults to the Visitor counting methodology, and for excellent reason. Participating in a test is not like clicking a button or visiting a page, but rather something tied to the User so that it can be correlated to Events after entering into the Activity.
With that in mind, this is the single step needed to push your Adobe Target test data to Amplitude. Keep in mind; these steps assume you have Amplitude snippet in place or the npm/yarn installation. Modifications can be made as needed depending on how you manage Adobe Target’s deployment – we are using Adobe Launch.
You need to add this code to the bottom of your Adobe Target at.js file. This code can be added via the Adobe Target UI before your download the at.js file or it can be custom code added via Launch to execute after Target is Loaded.
document.addEventListener(adobe.target.event.REQUEST_SUCCEEDED, function (e) {
if (e.detail.responseTokens) {
var rt = e.detail.responseTokens,
ids = [];
window.targetExperienceList = '';
for (var i = 0; i < rt.length; i++) {
var inList = false;
for (var j = 0; j < ids.length; j++) {
if (ids[j] == rt[i]['activity.id']) {
inList = true;
break;
}
}
if (!inList) {
ids.push(rt[i]['activity.id']);
targetExperienceList += (targetExperienceList ? ',' : '') + rt[i]['activity.name'] + ':' + rt[i]['experience.name'] + ':' + rt[i]['experience.trafficAllocationType'];
}
}
}
if (window.targetExperienceList) {
var values = [targetExperienceList];
var identify = new amplitude.Identify().set('Activity:Experience', values);
amplitude.getInstance().identify(identify);
}
});
Pretty easy, right!? Let us break things down so all can see what is going on with this code because developers can use this code to push data to any solution, including CDPs, data layers, etc…
Line 1: This is an event listener that will execute the code within it after a successful Adobe Target server call (mbox call:) has taken place.
Lines 2 – 19: We are creating a window scoped variable called “targetExperienceList” that will have a value that will equal a concatenation of all “Activities, Activity Experience, and Activity Allocation” that were tied to the response of the successful Adobe Target server call (mbox call:). On MiaProva.com, this is what the value looks like:
You won’t typically see this sort of thing, but we have quite a few Activities running here for demonstration purposes. For those of you not familiar with “Activity Allocation,” that value is only pertinent to Automated Personalization and Auto-Target, where Adobe has Control and Targeted Groups. All other Activity types will have ‘undefined’ as a value.
Lines 20 – 26: Here, we are first checking to see if there is a value for “targetExperienceList”. This value would exist if the Adobe Target server call (mbox call:) had Adobe Target Response Tokens that showed that the Visitor was in a live Activity. If so, then it would execute Lines 22-24, which pushes all the Adobe Target data to Amplitude as User Properties. This could differ from the above User Properties because we treat the values like an array if the Visitor gets into more than one Adobe Target Activity on a single Adobe Target server call (mbox call:).
And there you have it:
Then you can analyze as you wish, create user cohorts (not available in free trial), create dashboards, visualize paths, etc…
Optimization Management is so much more than managing the intake and output of tests. A true Optimization Management Platform enables the organization to monitor tests or groups of tests and answers the big questions about their testing program as a whole and their investments in resources and technologies.
MiaProva automatically stitches the organizational meta-data related to Activities to data from Adobe Target and Adobe Analytics. This approach allows organizations to use this data to analyze and manage their tests and what offline variables are impacting the program – similar to how organizations use eVars and list variables in Adobe Analytics.
One of the many ways organizations use this is to understand how Adobe Target Premium is adding value to their Optimization Programs. MiaProva automatically classifies Activities that are specific to Adobe Target Premium as a dimension for Program Analysis. Doing so exposes a potential problem that needs addressing if limited or no value is being realized or creates an opportunity to expand on any value that is being realized.
MiaProva just onboarded two organizations that have made significant investments with Adobe and are heavy users of Adobe Target. Both organizations have launched more than 100 Activities launched since January of 2021. While mapping test tickets or test design documents to Adobe Activities to put all that data to life, MiaProva was able to immediately highlight for both of these organizations that they were not receiving the amount of value they should be with Adobe Target Premium.
In this blog post, we are going to highlight how we were able to identify this opportunity (and others) for these two organizations, what is being done to change things, and a great example of another MiaProva client who sees considerable value in Premium and MiaProva has used the ROI models to expand on their use of the Automated Personalization.
Optimization Program Macro-Analysis
MiaProva automatically creates dimensions related to the Adobe data such as “Adobe Target Premium vs. Standard,” A4T vs. non-A4T, Activity Types, Activity Sources, etc… Organizations use this data to manage live Activities and for all historical tests. While this is powerful (and very helpful for these two new MiaProva clients), the strategic value lies in tying the organization’s internal data to their optimizations efforts.
With Adobe Target, organizations have the tools they need to analyze individual tests. Organizations can go into Adobe Target and understand how the test impacts any metrics configured in Adobe Target. Lift, Confidence, segments, Success Metrics, etc…..all the great things you need to analyze an individual test.
And with, Analytics for Target (A4T), the analysis capabilities are exponentially more powerful by automating the inclusion of any metric, full use of organizationally accepted segments, metrics, calculated metrics, etc… Organizations can also look at all Activities in aggregate and use Activities and Activity Experiences as Segments which is a massive capability in and of itself that can’t be understated.
What is missing, though, is all the great data that is driving the Activity or test, to begin with. That is where MiaProva comes in by automating the process of mapping data from test tickets or test design documents to the rich data collected and managed by Adobe. MiaProva does this by way of MiaProva Programs – customize templates for their test design documents.
MiaProva Programs allow organizations to ‘dimensionalize’ any internal data related to tests just like MiaProva is ‘dimensionalizing’ the Adobe data like how we are distinguishing Activities that are part of ‘Adobe Target Premium.’ The use cases are limitless, but here are some that are being used by our clients today:
What idea source is leading to the greatest number of tests? the greatest lift against metric x, y, or z?
How are the tests distributed across organizational locations?
What audiences are being used across test locations?
How does our feature flaging solution (that we capture via an Adobe Analytics eVar) compare in terms of impact to revenue compared to tests that were executed via Adobe Target?
What value is Adobe Target Premium giving us? What is our win rate with Premium Activities vs. Standard Activites?
What is our ratio of Premium Activites vs. Standard Activities? What business units are seeing value from Premium and which ones are not using Premium?
What Product Managers are runing the most tests?
What is the distribution of Activities to Analysts?
Are HIPPO driven tests as effective as data-driven tests?
What is the ratio of Adobe Target ‘personalization’ efforts (pushing the winner or just targeting content) to actual tests
and we can go on and on. Any data that an organization would like to use is available for management for any Live Activities and all historical ones. This data can come by way of our APIs, JIRA integration, or as users supply tickets as part of their test management process.
Our approach to macro-analysis of optimization programs has enabled organizations to learn fascinating things and exponentially increase the realized value of investments in optimization. It is this approach that these two new organizations to MiaProva started to ask, “Why are we paying for Adobe Target Premium”?
Automated Personalization Playbook
Both of these new MiaProva organizations reached out to us to better understand what they saw in terms of our Program Analysis regarding their investments in Adobe Target Premium. It turns out the value they were realizing was non-existent or low was because they were not using the Activity features that come with Adobe Target Premium (Automated Personalization, Recommendations, and Auto-Target) correctly or in a minimal capacity despite licensing them for a considerable period of time.
Diving deeper into their use cases and their understanding of the Premium Activity types, it became clear that their understanding of Automated Personalization, how it worked, and how best to apply it was incorrect or less than ideal. We dove into the other Premium Activity types, but the investments made for Adobe Target Premium were tied to Automated Personalization.
We tried to point them towards documentation that could help, including a co-authored book on Adobe Target. Still, there was nothing out there that provided a soup-to-nuts approach to how best to use Automated Personalization. So, for these two organizations, and anyone else that may want to learn about Automated Personalization, we are committing ourselves to author what will be referred to as the Automated Personalization Playbook.
Give us a follow below to follow this series over the next month or two automatically:
This Automated Personalization Playbook will go very deep into what Automated Personalization is, how to give you the best chance to ensure success, and outline the best-in-class:
Framework
Execution
Strategy
Analysis
Stay tuned, as this will be a pretty powerful tool for those organizations that license Adobe Target Premium. It will also be exciting to possibly report on the change in ROI of these two organizations if they can adapt the strategy as outlined in this playbook!
Automated Personalization for the win
Another organization that leverages MiaProva is seeing significant ROI with investments with Adobe Target Premium. This company had applied a macro-view of optimization way before adopting MiaProva, which was helpful, but now MiaProva automates much of this effort.
This organization invested more than $3MM on an internally built engine that sits on top of its content management system. This system enabled business teams to establish rules for sections of their home page – very similar to how Adobe’s Experience Manager works. The team made heavy use of this solution, leading to significant increases in engagement and impact on key organizational metrics.
In an ongoing effort to continue optimizing, they tested this solution against Automated Personalization (using strategies outlined in the above-mentioned playbook). This effort lasted more than six months and spanned dozens of iterations. Without going into the details, here were the high-level results:
This effort clearly showed that humans are clearly smarter when it comes to applying rules to digital experiences based on data. This effort also showed the value of removing the cognitive bias that humans bring to decision-making and the value that AI could bring.
This ROI is managed by MiaProva and clearly visible in the MiaProva dashboard as a target that they continue to try to hit. This visibility has enabled this organization to demonstrate the value of automation, which has led to its operationalization to several other key areas for its digital consumers.
We added something pretty significant to MiaProva this week. Activity Sources.
As you may know, MiaProva is the only Optimization Management Solution to integrate with Adobe Target and Adobe Analytics fully. Any Activity type or optimization effort in Adobe Target is fully available for management within MiaProva. Any of your Adobe Analytics Segments, Metrics, Calculated Metrics, Events, etc….are also fully available as well. We’ve expanded our deep integration with Adobe I/O to now give our users Activity Sources.
Activity Sources represent different sources for optimization or testing related events. Adobe Target is a wonderful example of an Activity Source, and now MiaProva has broadened Optimization Management to include any data from any dimension in Adobe Analytics.
At MiaProva, our most successful customers have shown us that:
Data collected and reported on should have more of an optimization focus with ROI and learning management
Considerable opportunities exist within Adobe Analytics to expand on optimization and personalization over and above Adobe Target
Organizations need a centralized platform to manage all of their optimization efforts along with the efforts that stem from their testing and optimization solution
Adobe Analytics Dimensions
Adobe Analytics provides organizations with 75 Props, 250 eVars, and 3 List Variables. These variables allow organizations to report and analyze any data that is associated with them. MiaProva now treats each of these as configuration sources for Optimization Related efforts.
These additional sources also inherit the same MiaProva benefits that Activities from Adobe Target get including:
Real-time and automated reporting
Custom alerting and notifications based on any conversion event in Adobe Analytics based on defined thresholds and confidence levels
Centralized management with any other source, including Adobe Target
Program Dashboards and ROI models
Automatic creation of a knowledge library
Collaboration and risk mitigation
And everything else available within MiaProva…
Given the limitless ways organizations can use their Adobe Analytics Variables, the opportunities to treat the data collected by them like optimization initiatives are essentially limitless. Here are a few examples to give you a sense of the power this Activity Sources feature enables:
Looking to the future, as organizations adopt and make use of Adobe’s Experience Platform (AEP), this Activity Sources capability by MiaProva will be even more valuable given the power AEP has to collect and manage data from many sources. We’ve built this capability for today with the future in mind, given the considerable opportunities that exist with Adobe’s Journey Optimizer.
Configuration in MiaProva
Organizations can easily configure Activity Sources right within MiaProva’s Admin Console as seen below.
MiaProva Activity Sources Configuration
Here organizations can add any of their Adobe Analytics dimensions (eVars, list variables, sProps, etc…) as an Activity Source.
Simply provide a name, the dimension ID, and a value pattern and MiaProva will immediately take over and provide management of these sources along with your Adobe Target Activities!
Below highlights how one can Access or manage these different sources within MiaProva’s Live Dashboard.
Sample Ratio Mismatch (SRM) is a thing, a real thing that optimization programs need to concern themselves about. At a high level, SRM means that the variants (experiences) of an A/B test have a traffic distribution (sample) that differs in a statistically significant way from what you expected the allocation of traffic or the sample to be.
This blog post and this blog post are great as they dive into what it is, how scary it can be, and what to do about it. Please do check out both those posts as they drive home just how important of a topic this is for optimization programs.
Sample Ratio Mismatch and MiaProva
About 5-6 weeks ago, MiaProva quietly released a new Alert to our array of alerts we provide for testing programs. This Alert will notify MiaProva customers that use Adobe Target automatically when SRM has been detected. This approach automatically applies a Chi-Squared calculation to every single A/B Activity currently Live in their Adobe Target accounts.
When MiaProva detects these issues, we will immediately notify stakeholders via email and highlight the Alert in the Live Dashboard and the daily Aggregate email that summarizes all Activity-related Alerts.
MiaProva Alert Management Dashboard
In the last month, we have had SRM detected a handful of times across many hundreds of Live Activities from different organizations. With SRM, the most important thing is identifying the root cause of the mismatch. Two out of the five are still being investigated, but the other three were quickly diagnosed. The statistically significant issues were:
During QA, an internal team became members of one of the variants and maintained that membership when the test went live. Incorrect targeting used in the Activity setup further exacerbated this SRM issue.
A bot had gotten into one of the variants.
Improper use of the Advanced settings upon Conversion, the selected Action below, combined with the audience criteria, caused an imbalance of sample distribution.
Adobe Target Conversion Advanced Settings
Had MiaProva not detected and alerted people to this key issue, organizations could have found themselves in a situation where they were making decisions on statistically invalid test results. Furthermore, because MiaProva detected SRM, the organizations could remedy the underlying causes, thus preventing additional reporting issues.
Below represents what SRM alerts look like in context to Live Activities that are from Adobe Target. MiaProva also provides an SRM calculator with each Live Adobe Target Activity that is an A/B Activity. Automated Personalization, Experience Targeting, and Auto-Allocate Activity types have logic to promote differences in sample distribution across Activity variants. MiaProva provides this service for all A/B Activities, Multivariate Activities, and Adobe Recommendations.
MiaProva alerts on Live Dashboard
And lastly, here is what the calculator looks like and how users can adjust their Expected Visitor distribution as needed.
Many optimization programs find themselves having a requirement to limit the number of times an offer from Adobe Target gets shown to visitors. The offer could be anything really but this type of ‘offer capping’ is usually applied to overlays or modals of some sort. Organizations would like to have the offer displayed two or three times to the user and if the user doesn’t interact, stop the offer from showing.
Here we dive into a generic profile that sets an impression value independent of any activity or test. Then we show how to use the same logic at an Activity or offer level.
At the earlier stage of optimization programs, MiaProva provides a standard test ticket in-take form that is based on industry best practice. Our hope is to guide users to carefully consider all factors that can impact their tests downstream.
As most of our clients reach a level of maturity and sophistication in their testing program, the need to allow flexibility has become clearly pressing. MiaProva is pleased to share that we just released a new capability, where different teams in one organization can setup different test intake forms that is most relevant to their individual needs.
Let’s walk through in some details!
To create a new test ticket form, the test manager with admin access can go to the new TEAMS tab under admin console. Here you can setup team specific work streams, as well as customizable ticket intake form.
You can set default form for the ‘submit ticket’ page to load, as well as create and customize any team-specific new form.
MiaProva’s ticket goes through different states in its life cycle. Here are the key states and in which order they occur:
Intake > Submitted > Under Development > Awaiting Summary > Finalized
You can decide if a specific field is required on each state, or if it’s visible on a given state. Please note however since these states are sequential in nature, MiaProva would require you to enter the data in a prior state IF that field is required in the subsequent state.
In addition to add/remove fields from a default form, you can also add additional MiaProva suggested fields or customize your own fields.
To add a MiaProva pre-configured, most frequently used field:
To add a new custom field, first click the ‘add field’ button, then:
Once you finished configuring an existing form or create a new one, find it as the first item on any ticket submission form. Here is an example under the ‘submit ticket’ section:
On this form, choose the form that you wish to use, and all your custom fields will show up accordingly. Note that this form will govern the remainder of the ticket life cycle! So consider carefully please when you set the ‘required’ and ‘visibility’ options when customizing the form. For instance, if you wish to have a simple form to encourage ease of submission, but want to provide very rich test information in the test summary stage, you might want to consider setting ‘test hypothesis’ as NOT required on the input form state, but add it as ‘required’ on the ‘Awaiting Summary’ and ‘Finalized’ stage.
A note about filters:
Note that with these changes, we added custom filters by team as well – since filters are defined by the fields specified for an individual team. Here is an example:
A note about Jira users’ integration with MiaProva
If your organization uses Jira and has an integration with MiaProva, please note that the form that you set as ‘DEFAULT’ will be the one used to integrate with Jira. This is because Jira does not support the concept of different intake forms by different teams.
