Marketers are always looking to optimize their strategies. We A/B test everything from email subject lines to landing pages, so why wouldn’t we also A/B test our app messaging?

Click-through rates are a vanity metric. They’re not telling the whole story. Think about how many push notifications you get throughout the day. It’s possible that a user reads your message, but didn’t have time to click-through.

That’s why it’s essential to look beyond click-through rates and measure the downstream impact of your messages on KPIs that matter like engagement, revenue, and conversion frequency:

But how do you know if this messaging is truly what’s driving the user to action? Queue control groups. Control groups give mobile marketers the ability to see if their messaging is truly effective by leaving a small group of users out of the send. By having control groups in your A/B testing, you’re able to clearly see if the message you sent was beneficial or detrimental to KPIs: