Marianne Jaen · Navigate case study
More attention.
Not a better return.
Use the commute-labelled ad (H1) as the stronger commercial reference. Investigate the morning-labelled ad (H3) before deciding which message to develop next.
An analysis of historical simulated results for a fictional insulated-mug brand. Recommendations have not been implemented.
My role
I analyzed the simulated ad results, separated performance findings from creative hypotheses, and developed prioritized recommendations and a client-facing report.
The project used simulated data with AI-assisted analysis and report production. Recommendations remain proposed and have not been implemented.
The commercial signal favours H1.
H3 recorded the higher link click-through rate (CTR), while H1 returned more attributed revenue per advertising dollar.
ROAS: return on ad spend
H1 is above the 2.0 scenario target; H3 is below. This is an observed comparison, not a causal winner or proof of profitability.
Validate the outcome.
Then investigate the message.
H1: 15 purchases at a $30.00 cost per purchase (CPA) and $75.00 average order value (AOV).
H3: 6 purchases at $36.67 CPA and $55.00 average order value.
Lower purchase cost and higher order value account for H1’s stronger ROAS. The data does not isolate the creative or delivery factors behind those differences.
The scenario uses a 2.0 ROAS target and a $200 review threshold. Meeting the spend threshold qualifies an execution for analysis; it does not establish a winner.
Selected-ad scorecard
H1 + H3 testing only · H2 and all scaling excluded · not account-wide
What this means — The selected pair clears the scenario target in aggregate, but the blend hides H3’s weaker return. Decisions still need execution-level evidence.
*Revenue is reconstructed from displayed purchases and AOV; rounding in source inputs may affect precision. Attribution settings are unavailable. This is not incremental revenue, store-wide sales or profit.
Show scorecard calculations
| Calculation | Result |
|---|---|
| H1 revenue: 15 × $75 | $1,125 |
| H3 revenue: 6 × $55 | $330 |
| Blended ROAS: ($1,125 + $330) ÷ ($450 + $220) | 2.1716 → 2.17× |
| Purchase CPA: $670 ÷ 21 | $31.90 |
| AOV: $1,455 ÷ 21 | $69.29 |
Totals are recomputed from numerators and denominators. A simple average of 2.50 and 1.50 would incorrectly yield 2.00 ROAS. Aggregate CTR is unavailable without total clicks and impressions.
Clicks need a purchase-outcome check.
| Testing execution | Attention · Link CTR | Commercial outcome | Interpretation |
|---|---|---|---|
| H1 · “Commute” | 1.5% | 15 purchases · 2.50 ROAS | Stronger observed return at greater spend. |
| H3 · “Morning” | 3.0% | 6 purchases · 1.50 ROAS | Higher click-through rate; weaker purchase return. |
What this means — H3 attracts a higher rate of clicks, but H1 delivers the stronger purchase return. Click and impression totals are unavailable, preventing a click-to-purchase comparison.
“Commute” and “Morning” are supplied context labels. They do not verify the full hook, image, video sequence, pacing or format.
Read purchase cost and order value together.
CPA: cost per purchase
H1 spends less per purchase. This is purchase efficiency, not verified new-customer acquisition cost.
The relationship behind ROAS
AOV ÷ purchase CPA = ROAS when all inputs refer to the same purchases and attribution scope.
Use unrounded CPA for calculations. Audience, placement, offer, landing page, tracking, attribution, delivery and order mix remain possible explanations—not established causes.
Performance table
| Execution | Spend | Purchases | Link CTR | CPA | AOV | ROAS |
|---|---|---|---|---|---|---|
| H1 · Testing “Commute” context | $450 | 15 | 1.5% | $30.00 | $75.00 | 2.50× |
| H3 · Testing “Morning” context | $220 | 6 | 3.0% | $36.67 | $55.00 | 1.50× |
Showing H1 + H3 testing.
What this means — H1 leads commercially in this selected pair. H3’s six purchases are too sparse to establish a reliable winner or loser; its higher CTR alone does not support a budget increase. Statistical significance has not been established.
A numerical difference
H1 has lower CPA, higher AOV and higher ROAS than H3. H3 has the higher supplied link CTR.
Context may matter
A specific usage context could influence relevance. Basket mix or delivery conditions could also explain the commercial gap.
The actual execution
Finished ads are unavailable. Visuals, pacing and message execution need review before recommending specific creative edits.
H2, scaling and the limits of coverage
H2 testing: $180 spend and 2 purchases, below the $200 review floor. Reported 4.00 ROAS implies $720 revenue and $360 AOV. Those purchase values remain unverified; the apparent return is provisional, not proof of a winning hook.
H1 scaling: the original analysis lists 1.20 ROAS, while an earlier project record reports 1.50. Spend, revenue and purchases are missing. The conflicting figures cannot be reconciled with the available records. Scaling is excluded from charts and totals.
Coverage: H1 + H3 represent $670 of $850 supplied testing spend (78.8%). This denominator excludes scaling and is not the whole account. Including H2 would provisionally yield $2,175 inferred revenue and 2.56 ROAS, but depends on its unverified values; it is not the headline scorecard.
Turn the pattern into a question worth testing.
Does a specific usage context improve relevance?
Consider a commute-specific versus general morning message test after reviewing the final ads. This is a proposed hypothesis; the historical comparison does not isolate a message effect.
Rule out material differences first.
Verify delivery, attribution and order values. Separately inspect offer accuracy, page access and checkout continuity. A confirmed measurement or journey issue may change which creative question is worth pursuing.
Evidence → recommendation
| Observation | Possible explanations | Verification needed | Recommended next step |
|---|---|---|---|
| H1 has lower CPA and higher AOV. | Audience or order mix, offer, delivery and attribution may differ. A message effect remains a hypothesis. | Matched delivery records; purchase values and products; actual assets and offers. | Use H1 as the commercial reference for validation—not a causal winner. |
| H3 has higher CTR but weaker purchase return. | Expectation mismatch, placement mix, page friction, offer differences or tracking could contribute. | Clicks and page arrivals; final ad-to-page continuity; checkout terms; event definitions. | Diagnose before selecting a corrective creative change. |
| H2 reports 4.00 ROAS from two purchases below the review floor. | Larger orders could be real; currency, duplicate events or value errors are alternatives. | The two order values and corresponding purchase events. | Keep H2 provisional; do not rank its hook or fund it simply to cross the threshold. |
| H1 scaling evidence is incomplete and conflicting. | The records may cover different delivery contexts or reporting scopes; the discrepancy remains unresolved. | Resolve 1.20 versus 1.50 ROAS; obtain spend, purchases, revenue and matching dates. | Keep scaling out of totals and do not infer scale readiness. |
What this means — Validate the source of the performance gap before deciding whether the next step is a creative test, a delivery adjustment or a customer-journey fix.
Prioritized actions and review conditions
| Priority | Action | Reason | Dependency or review condition |
|---|---|---|---|
| 01 · Verify Measurement & delivery | Reconcile H2 values and H1 scaling records; align attribution and delivery definitions. | Unverified inputs can distort both ranking and recommendations. | Order/event records, campaign data and current statuses; recalculate if inputs change. |
| 02 · Inspect Creative & customer journey | Retrieve final ads and inspect message, offer and landing-page continuity. | Labels and briefs do not establish what audiences actually saw. | Exact creative versions, product claims and offer terms; correct verified mismatches before new production. |
| 03 · Prioritize Creative question | Assess whether a specific usage context is worth testing against general morning messaging. | H1 is the stronger observed commercial reference, but its mechanism is unproven. | An inspected-asset rationale and a feasible comparison; otherwise revise the hypothesis. |
| 04 · Review Reporting & budget decision | In a live application, maintain H1’s approved delivery only if current economics and risk limits support it. Validate the evidence before expansion; H3’s CTR alone does not justify more spend. | Protect the distinction between observed return and reliable future performance. | Current economics, mature attribution, comparable delivery and approved risk limits. Any restart or budget change requires a separate decision. |
What this means — Resolve the decision-critical evidence gaps before commissioning new creative. These are proposed actions.
Agree on ownership, budget and review conditions before proceeding. Review timing should reflect delivery volume and conversion lag.
Validate H1. Investigate H3.
H1 returned more per dollar.
In the selected testing pair, H1 recorded 2.50 ROAS versus H3’s 1.50, despite H3’s higher click-through rate. Together they produced 21 purchases on $670 spend.
Attention is not the commercial verdict.
H1’s lower purchase cost and higher order value account for its stronger return. The comparison does not isolate a message effect or establish performance at a higher budget.
Validate H1; diagnose H3.
Use H1 as the validation reference. Any continued delivery depends on approved economics and risk limits. Review H3’s assets, purchase values and customer journey before choosing a corrective message test.
Better evidence—or a corrected input.
Reconciled order values, mature comparable delivery, actual creative inspection or a verified offer/tracking issue could change the ranking or next question. Any pause, restart or budget change needs current context and authorization.
A clearer direction for the next creative decision.
What this analysis produced
- A selected-ad comparison with consistent H1/H3 inputs and separate disclosure of unresolved source conflicts.
- A distinction between numerical findings and creative hypotheses.
- Prioritized verification needs and recommendations.
- A concise recommendation summary with conditions for revisiting the decision.
Related next step: creative iteration
The findings can inform a separate brief comparing usage-context messaging while controlling other material changes.
The companion creative-iteration case study develops a matched concept pair and test plan. Production, launch and performance validation have not taken place.
Outcome — The analysis identified H1 as the stronger observed execution and defined the evidence needed before making a creative change or expanding delivery.
Sources and methodology
Source records, labels and conflicts
| Source | Role in this case study | Important limitation |
|---|---|---|
| Cozy Cup reporting summary · October 7, 2026 | Execution labels, reported metrics and recommendations. | Reconstructed revenue and selected-ad scope; no live export. |
| Cozy Cup creative analysis · simulated dataset | Underlying simulated rows; H2 context; target and floor; incomplete scaling evidence. | Synthetic IDs; no final assets; H1 scaling reported as 1.20. |
| Earlier project records | Supporting context and source reconciliation. | Scaling ROAS of 1.50 conflicts with the analysis figure of 1.20; excluded pending reconciliation. |
The reporting summary and simulated dataset agree on H1/H3 metrics. H1 and H3 are historical scenario labels; “Commute” and “Morning” describe their supplied message contexts. Scenario hook text is available, but final ad assets were not inspected, so visuals, sequence, pacing and format remain unverified.
Calculations and metric definitions
CPA (cost per purchase) = spend ÷ purchases. AOV (average order value) = attributed purchase revenue ÷ purchases. ROAS (return on ad spend) = attributed purchase revenue ÷ spend. All aggregates use summed inputs, not averaged ratios.
H1: $450 ÷ 15 = $30 CPA; 15 × $75 = $1,125 reconstructed revenue; $1,125 ÷ $450 = 2.50 ROAS. H3: $220 ÷ 6 = $36.6667 CPA; 6 × $55 = $330 reconstructed revenue; $330 ÷ $220 = 1.50 ROAS. Displayed figures are rounded after calculation.
Displayed AOVs may already be rounded. Reconstructed revenue therefore indicates scenario-consistent amounts, not independently verified transaction precision. Neither source supplies order records to remove that limitation.
Attributed revenue is purchase value credited under a reporting rule. It is not automatically incremental revenue, total store sales or profit. Purchase counts do not identify verified new customers. AOV minus advertising CPA is not gross profit.
Link CTR (click-through rate) is used as supplied. Aggregate CTR, CPC (cost per click) and click-to-purchase conversion are unavailable because click and impression totals are absent. Missing values are not treated as zero.
Unavailable evidence and confidence
Account timezone, attribution settings, extraction timestamp, active delivery days, current statuses, actual Meta IDs, final media, matched audiences/placements, full account totals, prior-period baseline and cost model are unavailable. The scenario stipulates period eligibility; first-ever launch was not independently verified.
Arithmetic can be verified against the simulated inputs. Creative causation, significance, profitability, scaling transferability and future performance cannot. An observation of 15 versus 6 purchases does not itself establish confidence or a causal winner.