The editorial team compares two property campaigns. Campaign A has a smaller overall share of qualified follow-up contacts than Campaign B. After splitting into two regions it suddenly looks reversed: within each region A is ahead of B. One of the tables therefore does not have to be miscalculated.
This own calculation workshop for Austria and Germany uses entirely fictional cases. It shows a possible effect of different composition. It does not evaluate any real platform, any actual target group or any real property campaign. In particular, the example is not a statistical proof of a causal advertising effect.
Define the counted action before the calculation
In the model a follow-up contact is a specially defined and confirmed further enquiry about an offer. Each considered person is counted at most once in the specified model period. The definition is an assumption of this workshop and not a general industry standard.
For each campaign 50 recorded contacts are considered. These are distributed differently across the fictional regions Nord and Süd. Group assignment is fixed before the comparison; it is not changed afterwards to produce a desired ranking.
The initial data are:
- A – Region: Nord; Considered contacts: 40; Qualified follow-up contacts: 4; Share: 10 percent
- A – Region: Süd; Considered contacts: 10; Qualified follow-up contacts: 9; Share: 90 percent
- B – Region: Nord; Considered contacts: 10; Qualified follow-up contacts: 0; Share: 0 percent
- B – Region: Süd; Considered contacts: 40; Qualified follow-up contacts: 32; Share: 80 percent
Each subgroup rate is correct, and the total rate as well
In Nord the observed rate for A is four divided by 40, i.e. 10 percent. For B it is zero of 10, i.e. 0 percent. In Süd 90 percent for A are compared with 80 percent for B. Within both named regions A therefore has the observed rate higher.
Now the totals are added correctly. A has 13 follow-up contacts out of 50 contacts in total; that gives 26 percent. B has 32 out of 50, i.e. 64 percent. In the overall view B is ahead. No value was swapped for this and no mean of the two regional percentages was formed.
The decisive difference lies in which quantities the subgroup rates are carried by. A has many contacts in the region with low observed shares, B many in the region with high shares. The overall rate therefore does not contain the same mixture.
What question should the ranking actually answer?
For the question “How many specified follow-up contacts were there in total?” the actual sums are relevant. Whoever, however, wants to examine the two campaigns under a regionally comparable presumed composition asks a different question. The appropriate analysis must be clarified before a conclusion.
One could, for a purely additional model calculation, mentally weight both regions equally. The mean of the regional rates would then be 50 percent for A and 40 percent for B. However, that is a deliberately differently defined summary. It does not retroactively describe the observed total amount.
One representation is therefore not generally “the right one”, while all others would be meaningless. It must match the reader’s question and disclose its weighting. Without this information a mathematically correct metric can suggest an inappropriate decision.
The comparison does not prove an effect of the creative
From the tables it does not follow that the creative of A caused a better effect in every region. For that the fictional case lacks information on the actual comparability of the contacts, on the process and on other influences. Small subgroups with very few events are not presented as stable general success rates.
A regional split does not automatically make observational data a controlled test either. The office should therefore not conclude solely on the basis of the arithmetic example that a particular creative would produce more qualified enquiries in future.
The benefit is more limited and yet important: before a blanket assessment it becomes apparent that composition explains part of the apparent contradiction. The subsequent professional or methodological investigation can then be commissioned more precisely.
Leave a usable evaluation note
Our editorial recommendation is to keep the actually used numerators, denominators and key subgroups traceable for every aggregated rate. The public does not need a personal contact list for this. An approved suitable summary is sufficient for the stated explanation.
If two reports show different rankings, first check whether definition, period and grouping match. Only then should it be decided whether a calculation error, a different analytical question or a different composition is present.
The result of the workshop is not a new victory announcement. It is a more precise question to the campaign evaluation: Are we comparing the same conditions, and what actual composition is behind the overall rate? Whoever discloses this question does not have to explain a mathematically possible difference with an invented success story.