Lessons from a Horse Products Audience Test

Note: All use cases utilize illustrative revenue and customer figures for demonstration purposes. Percentage lifts, experimentation methodologies, strategies, and outcomes are based on real-world testing frameworks and accurately reflect the types of results and insights brands can uncover through the LXRInsights experimentation process. The examples are intended to communicate the value and impact of structured experimentation rather than represent actual client performance data.

‍

Is The Control Audience Is Better?

When running experiments, the goal isn't always to find a test campaign that outperforms the control. Sometimes the most valuable outcome is confirming that your current strategy is already performing at a high level. Those insights help eliminate assumptions, validate optimization efforts, and provide confidence that your campaigns are working as efficiently as possible, even when powered by LXRInsights audiences and additional testing variables.

In this experiment, we found that the test audience for horse products generated lower acquisition costs and higher click volume than the control audience. However, those advantages did not translate into stronger business outcomes. The control campaign ultimately delivered higher revenue and better ROAS, indicating it was more effective at identifying high-value buyers. Meanwhile, the test audience appears better suited for driving entry-level purchases, lower-cost acquisition, and top-of-funnel traffic.

So what did we learn? Lower CPCs and higher click-through rates do not always equal better business performance. This experiment reinforced that audience quality matters more than traffic volume, while also uncovering a potential new audience segment that can be leveraged for awareness and customer acquisition objectives.

What We Learned: Lower Cost Does Not Always Mean Better Performance

At first glance, the test audience appears to be winning. It generates more clicks and acquires customers at a lower cost than the control audience. However, when we look at the metrics that matter most, revenue and return on ad spend, the control audience continues to outperform.

That does not make the test a failure. In fact, it is exactly why experimentation is valuable.

One of the goals of testing is not always to find a new winner. Sometimes the goal is to challenge assumptions and validate that your current strategy is already optimized. In this case, the experiment confirmed that while the test audience is effective at driving lower-cost traffic and engagement, the control audience is still better at finding higher-value customers and generating stronger business outcomes.

This distinction is especially important for horse product brands. Not every customer carries the same value. A customer purchasing a small grooming accessory is very different from a customer purchasing premium tack, recurring supplements, or larger equipment purchases. More clicks and lower acquisition costs do not automatically translate into more revenue.

‍

The Experiment Is Still Early

The dashboard marks the experiment as Flat (Early), meaning performance is about the same right now and the test should keep running. That is not exciting, but it is the correct conclusion. The confidence level is only 24%, which means the data is not strong enough yet to make a final decision.

At this stage, the difference between the test and control could still be normal campaign noise. Early performance can lie. A few strong days, a seasonal spike, or a short burst of lower-cost orders can make a test audience look better than it really is.

For a horse products brand, this can happen fast. A spring sale, show season demand, weather-related blanket purchases, or a popular tack item can distort the read before the experiment has enough data. So the first lesson is restraint: do not scale the test audience yet, do not kill it yet, and keep watching until the confidence level becomes more reliable. Marketing teams hate waiting, which is adorable, since they also hate being wrong.

Revenue Tells a Different Story

The test audience generated $103.1K in revenue, while the control audience generated $112.8K, an 8% advantage for the control campaign.

This is where the experiment becomes interesting. While the test audience delivered lower acquisition costs and stronger engagement, those gains did not translate into higher revenue. The control audience continued to generate more value for the business.

That does not mean the test audience is ineffective. It suggests the audience may be serving a different purpose. The test audience appears well-suited for driving lower-cost traffic and attracting new customers, while the control audience is currently better at identifying higher-value buyers and generating stronger revenue outcomes.

For horse product brands, this distinction matters. Not every customer has the same value. A shopper purchasing a grooming brush, treats, or basic care products is very different from a customer purchasing premium tack, riding apparel, recurring supplements, or larger equipment purchases.

The key takeaway is not that the test audience lost. It is that the experiment helped clarify where each audience may create the most value. One audience appears stronger for efficient customer acquisition, while the other remains stronger for revenue generation. Both insights can be useful when building a balanced growth strategy.

CPA Is the Bright Spot, But Context Matters

The strongest positive signal in this experiment is CPA. The test audience achieved a CPA of $8.10 compared to $14.60 for the control audience, a 45% improvement.

This tells us the test audience is significantly cheaper to acquire and appears more likely to take action. For horse product brands, that could indicate stronger responsiveness to entry-level products, seasonal purchases, first-time buyer offers, grooming supplies, treats, fly protection products, or other lower-friction purchases.

However, CPA is only one piece of the story.

While the test audience is generating customers at a lower cost, it is also producing less revenue and a lower return on ad spend than the control audience. This suggests that although customers are easier and cheaper to acquire, they may not currently be generating the same business value as the customers coming from the control audience.

That does not diminish the value of the audience. Instead, it helps clarify how it may be best used. The test audience may be a strong candidate for customer acquisition, awareness campaigns, or lower-cost growth initiatives, while the control audience remains more effective at driving revenue and maximizing return on ad spend.

This is exactly why experimentation matters. Looking at CPA alone would suggest the test audience is the clear winner. Looking at revenue, ROAS, and CPA together provides a more complete picture and helps teams understand where each audience can contribute most effectively within their marketing strategy.

CTR Shows Strong Interest

The test audience also has a stronger CTR. Test CTR is 3.81%, while control CTR is 3.12%. That is a 22% lift, which tells us the test audience is more likely to click.

The messaging, creative, or product offer is catching their attention. That matters because attention is the first step in the funnel. If people are not clicking, nothing else happens.

But clicks are not the final goal. The test audience is showing interest, but the revenue numbers suggest that interest is not turning into stronger purchase value. The audience may be curious, engaged, and easy to move onto the site, but not necessarily ready to buy higher-value horse products.

That creates a possible funnel split: the test audience is stronger at the top of the funnel, while the control audience is stronger closer to purchase value. That is the real story.

Conversion Rate Suggests the Audience Is Not More Qualified

The test campaign generated a 2.70% conversion rate compared to 2.85% for the control campaign, a modest 5% decline.

While the difference is relatively small, it is directionally important. If the test audience were identifying more qualified buyers, we would typically expect conversion rate to improve alongside the stronger CTR and lower CPA. Instead, the audience is clicking more often but converting at a slightly lower rate.

This suggests the test audience may be more interested in the offer, but not necessarily more ready to purchase. There appears to be a gap between engagement and buying intent.

That insight is valuable because it helps explain the broader results. The audience is responding to the ads and driving efficient traffic, but that traffic is not translating into stronger revenue, ROAS, or conversion performance.

The experiment does not yet tell us exactly why. The audience may be broader than the control group, may be better suited for awareness and acquisition, or may simply require more time and data before a clear pattern emerges. What it does tell us is that stronger engagement alone is not enough to determine success.

Taken together, the CTR, CPA, CVR, revenue, and ROAS results point to the same conclusion: the test audience is effective at generating interest and lower-cost traffic, while the control audience continues to generate stronger business outcomes.

What the Test Audience May Be Best Suited For

Based on the results so far, the test audience appears to be strongest for efficient customer acquisition. The significantly lower CPA and higher CTR suggest this audience can bring new customers into the brand at a lower cost, making it a potential fit for customer growth initiatives and top-of-funnel campaigns.

The audience may also respond well to introductory or lower-commitment purchases. While revenue and ROAS trail the control audience, the engagement metrics indicate there is interest in the products and messaging being presented.

Additionally, the stronger CTR suggests this audience could be valuable for awareness, product discovery, and customer acquisition efforts. The audience is engaging with the ads and taking action, even if that activity has not yet translated into stronger revenue performance.

Most importantly, the experiment has helped identify where this audience may create value within the broader marketing strategy. Not every audience needs to be optimized for the same objective. Some audiences are better suited for efficient acquisition, while others are better suited for revenue generation.

What the Test Audience Has Not Proven Yet

At this stage, the test audience has not demonstrated stronger revenue performance than the control audience. Revenue remains lower, and ROAS continues to trail the control campaign, indicating that the audience is not currently generating stronger business outcomes.

The results also do not suggest that the audience is identifying higher-value customers. While acquisition costs are lower, those efficiencies have not translated into higher revenue or improved return on ad spend.

For that reason, it would be premature to shift significant budget away from the control audience. The experiment is still early, and the confidence level is not yet high enough to justify major scaling decisions.

This does not mean the audience lacks value. It simply means the audience has not yet demonstrated that it can outperform the existing strategy on the metrics that matter most. For now, the test audience appears better suited for efficient acquisition and awareness objectives, while the control audience remains the stronger driver of revenue and profitabilit

‍

The Control Audience Is Still Delivering the Stronger Business Outcome

At first glance, the test audience appears to be the more exciting result. It delivers a lower CPA, generates a higher CTR, and drives strong engagement. Those are all positive signals.

However, when we look at the metrics most closely tied to business performance, revenue and ROAS, the control audience continues to lead.

The control campaign generated more revenue and a higher return on ad spend, suggesting it is doing a better job identifying customers who are ready to purchase and generating greater value for the business. While the audience may not be producing as many low-cost conversions, the customers it does acquire appear to be more valuable overall.

This is exactly why experimentation matters. If we only looked at CTR or CPA, the test audience would appear to be the clear winner. The control audience provides the context needed to understand whether those engagement gains are actually translating into business impact.

In this case, the experiment revealed an important distinction. The test audience appears stronger at driving engagement and efficient acquisition, while the control audience remains stronger at generating revenue and maximizing return on ad spend.

That does not make one audience good and the other bad. It simply helps clarify where each audience may be most effective within the overall marketing strategy. More importantly, it reinforces that success should be measured by business outcomes, not just engagement metrics.

Recommendation and Next Steps

At this stage, the recommendation is not to scale the test audience. The experiment is still early, with a confidence level of only 24%, and there is not enough data to conclude that the test audience can outperform the control on the metrics that matter most.

Instead, the focus should shift to understanding customer quality. The next step is determining whether the customers acquired through the test audience differ meaningfully from those acquired through the control campaign.

Key questions include:

• Are test audience customers generating lower average order values?
• Are they purchasing different product categories than the control audience?
• Are they primarily first-time buyers or returning customers?
• Are they more likely to purchase discounted products?
• Do they return and purchase again over time?
• Which products and categories are driving the majority of conversions?

The answers will help determine whether this audience represents a valuable acquisition opportunity or simply lower-cost traffic with lower long-term value.

Opportunities for Future Testing

One of the biggest learnings from this experiment is that audience segmentation may need to become more specific.

Rather than testing a broad horse products audience, future experiments could focus on distinct customer groups such as high-value purchasers, repeat buyers, seasonal purchasers, cart abandoners, customers who have not purchased recently, or shoppers who have previously purchased specific product categories.

More focused audience segments often produce clearer insights because they isolate customer behavior and intent more effectively. The current audience may simply be too broad, which could explain why engagement metrics are improving while revenue metrics remain weaker than the control.

The Bigger Takeaway

This experiment highlights why testing audiences against a true control group is so important.

The test audience demonstrated real strengths. It generated a lower CPA, produced a higher CTR, and showed the ability to drive efficient customer acquisition. Those are meaningful outcomes.

At the same time, the control audience continued to generate stronger revenue and a higher return on ad spend. That insight is equally valuable because it helps validate the effectiveness of the current strategy while identifying where the test audience may fit within the broader marketing mix.

Not every experiment uncovers a new winner. Sometimes the greatest value comes from confirming what is already working and identifying new opportunities that can be refined and tested further. In this case, the test audience shows promise as an acquisition-focused audience, while the control audience remains the stronger driver of revenue and profitability.

‍