How to run A/B tests in Framer

Nov 15, 2023

Posted by

On this page

Framer is a great tool for building marketing websites. However, sometimes you may be unsure if a change will actually improve your conversion rate. This is where A/B testing is helpful. It enables you to test and compare your changes to systemically improve your site.

This tutorial shows you how to set up A/B tests with Framer and PostHog to get the most out of your website.

Adding PostHog to your Framer site

First, sign up to PostHog. Then, go to your project settings and copy your web snippet:

PostHog web snippet

With the snippet copied, go to your Framer project settings by clicking the gear in the top right. If you haven’t already, sign up for the "Mini" site plan. This enables you to add custom code.

Go to the "General" tab in site settings and scroll down to the "Custom Code" section. Under "End of <head> tag", paste your PostHog snippet. Make sure to press "Save" next to custom code.

Adding a custom event

We're going to create an A/B test comparing how the text of a button affects it's click-through rate.

To do this, first we'll create a button using a custom code component, and then we capture an event whenever the button is clicked. We'll use this event as our goal metric for our A/B test.

Go to the "Assets" tab in the top left of your Framer project, and click the plus icon next to the "Code" tab. This will show a pop up to create a new code file. Name the file CaptureButton, set it as a "New component" and press "Create."

In the new code file, delete the existing code and replace it with the following:

JavaScript
export default function CaptureButton() {
    const handleClick = () => {
        window.posthog.capture("clicked_homepage_button", {
            $set_once: { clicked_homepage_button: true },
        })
    }


    return (
        <button id="capture-button" onClick={handleClick}>
            Click me
        </button>
    )
}

Press Cmd/Ctrl + s to save your changes. Then press the "Home" button to go back to the home page. Add your new CaptureButton to your page by dragging it from the Code tab.

Publish your site and then click your new button to see the event in PostHog.

View captured events in PostHog

Create an A/B test in PostHog

The next step is to set up an A/B test (we call them "experiments" in PostHog).

Go to the Experiments tab in PostHog and click "New experiment". Add the following details to your experiment:

  1. Name it "Home button experiment".
  2. Set "Feature flag key" to home-button-test.
  3. Under the experiment goal, select the clicked_homepage_button we created in the previous step.
  4. Use the default values for all other fields.

Click "Save as draft" and then click "Launch".

View captured events in PostHog

Implement the A/B test in Framer

The final step is to add the experiment code in Framer. Go back to the code file for CaptureButton. Here we'll add code that does the following:

  1. Fetch the home-button-test flag.
  2. Change the title of the button based on the value of the flag (control or test).

To do this, we'll add another script. Go to your Framer settings and under the section start of <body> tag add the following code:

JavaScript
<script>
  posthog.onFeatureFlags(() => {
   const button = document.getElementById('capture-button')
   if (button) {
      const flagValue = posthog.getFeatureFlag('home-button-test')
      if (flagValue === 'control') {
        button.text = "This is the title for CONTROL group"
      } else if (flagValue === 'test') {
         button.text = "This is the title for TEST group"
      }
   }
  });
</script>

Click "Save" and then publish your site.

That's it! Your A/B test is now live. Half your users will see the updated button text and PostHog will track whether it has an impact on your clickthrough rate. You can view your test results on the experiment page.

If you want to view both variants of your experiment to make sure they are working correctly, add the line posthog.featureFlags.override({'home-button-test': 'test'}) to your code:

JavaScript
<script>
  posthog.onFeatureFlags(() => {
   posthog.featureFlags.override({'home-button-test': 'test'}) // or 'control' 


   const button = document.getElementById('capture-button')
   if (button) {
      const flagValue = posthog.getFeatureFlag('home-button-test')
      if (flagValue === 'control') {
        button.text = "This is the title for CONTROL group"
      } else if (flagValue === 'test') {
         button.text = "This is the title for TEST group"
      }
   }
  });
</script>

Further reading

Comments