Do1e

Do1e

github
email
HomeArchivesHomepageAbout @Do1e

Reverse proxy to avoid Microsoft Clarity being blocked by advertising rules

#Blog14
AI Translation
This post is translated from Chinese into English through AI.View Original
AI-generated summary
Microsoft Clarity is a tool for analyzing user behavior on websites. It is easy to use by adding a single line of code to the HTML. However, many ad blockers may prevent Clarity from collecting user data. To overcome this, one can set up an Nginx reverse proxy to their own domain. The process involves creating a subdomain (e.g., clarity.example.com) and saving the Clarity script to a server. The original Clarity links must be replaced with the new domain links. The Nginx configuration is then set up to proxy requests to the appropriate Clarity endpoints. After updating the script in the website's HTML to use the new domain, users can monitor real-time traffic. However, a downside is that all geographic data will appear to come from the Nginx server. Additionally, the script does not automatically update with new versions of Clarity, but a custom script can be written to handle updates.

This article is synchronized and updated to xLog by Mix Space
For the best browsing experience, it is recommended to visit the original link
https://www.do1e.cn/posts/code/avoid-clarity-blocked

Introduction#

Microsoft Clarity can analyze user behavior on websites, and this site has integrated it.
I find it quite useful, and I can check the website's traffic when I have time.
It's also very simple to use; just add a line of code provided by the backend in the <head> tag.

image

However, many ad rules will block Clarity's domain, so some user connection information may be missing. But this can be avoided by proxying through nginx to your own domain. Here are the specific steps for reference.

Analyzing Requests#

Below is an example of the code added in the <head> tag.

<script type="text/javascript">
    (function(c,l,a,r,i,t,y){
        c[a]=c[a]||function(){(c[a].q=c[a].q||[]).push(arguments)};
        t=l.createElement(r);t.async=1;t.src="https://www.clarity.ms/tag/"+i;
        y=l.getElementsByTagName(r)[0];y.parentNode.insertBefore(t,y);
    })(window, document, "clarity", "script", "abcdefg");
</script>

Alternatively, you can open the browser's developer tools to capture packets, and you will see that the relevant code first requests https://www.clarity.ms/tag/abcdefg, and after downloading, it is a js script:

(function (c, l, a, r, i, t, y) {
    function sync() {
        (new Image()).src = "https://c.clarity.ms/c.gif";
    }

    if ("complete" == document.readyState) {
        sync();
    } else {
        window.addEventListener("load", sync);
    }

    if (a[c].v || a[c].t) {
        return a[c]("event", c, "dup." + i.projectId);
    }

    a[c].t = true;
    t = l.createElement(r);
    t.async = true;
    t.src = "https://www.clarity.ms/s/0.8.13-beta/clarity.js";

    y = l.getElementsByTagName(r)[0];
    y.parentNode.insertBefore(t, y);

    a[c]("start", i);
    a[c].q.unshift(a[c].q.pop());
    a[c]("set", "C_IS", "0");
})(
    "clarity",
    document,
    window,
    "script",
    {
        projectId: "abcdefg",
        upload: "https://z.clarity.ms/collect",
        expire: 365,
        cookies: ["_uetmsclkid", "_uetvid"],
        track: true,
        content: true,
        unmask: ["body"],
        dob: 2002
    }
);

There are 3 links:

  1. https://c.clarity.ms/c.gif
  2. https://www.clarity.ms/s/0.8.13-beta/clarity.js
  3. https://z.clarity.ms/collect

These links need to be replaced one by one.

Replacing Links and Proxying#

Assuming your domain is example.com, you can create a subdomain clarity.example.com and resolve it to a public nginx server.

At this point, you can save the script requested from https://www.clarity.ms/tag/abcdefg to /var/www/html/tag/abcdefg.

mkdir -p /var/www/html/tag
wget https://www.clarity.ms/tag/abcdefg -O /var/www/html/tag/abcdefg

Then replace the above 3 links as follows:

  1. https://clarity.example.com/c.gif
  2. https://clarity.example.com/s/0.8.13-beta/clarity.js
  3. https://clarity.example.com/collect

Next, you also need to download https://www.clarity.ms/s/0.8.13-beta/clarity.js to /var/www/html/s/0.8.13-beta/clarity.js. The version may be different when you check, so modify accordingly. The above /var/www/html directory can also be modified, but be careful with permissions to avoid the nginx process being unable to access it. The directory structure is as follows:

> tree /var/www/html
/var/www/html
├── clarity.js
├── s
│   ├── 0.8.13-beta
│   │   └── clarity.js
│   └── 0.8.9
│       └── clarity.js
└── tag
    ├── abcdefg
    └── hijklmn

5 directories, 5 files

nginx proxy configuration reference:

server {
    listen 80;
    listen [::]:80;
    listen 443 ssl;
    listen [::]:443 ssl;
    server_name clarity.example.com;

    if ($scheme = http) {
        return 301 https://$host$request_uri;
    }
    location /c.gif {
        proxy_pass https://c.clarity.ms$request_uri;
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection "upgrade";
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Real-PORT $remote_port;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    }
    location /collect {
        proxy_pass https://z.clarity.ms$request_uri;
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection "upgrade";
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Real-PORT $remote_port;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    }
    root /var/www/html;
}

Reconfiguring the Website#

Replace www.clarity.ms in the script added to the website's <head> with clarity.example.com. At this point, the task is complete. Try accessing your website and see if there are any online real-time users in the backend!

Disadvantages#

  1. The downside is that the backend will show that all access geographic locations come from the nginx server. It would be better if Microsoft could determine geographic location based on request headers.
  2. It cannot automatically follow Microsoft in upgrading clarity.js, but you can also write a script to achieve this automatically. Here’s a reference code:
import os
import re
import requests
from dotenv import load_dotenv

load_dotenv()

PROJECT_ID = os.getenv("PROJECT_ID")
BASE_DIR = os.getenv("BASE_DIR", "/var/www/html/clarity")
CUSTOM_DOMAIN = os.getenv("CUSTOM_DOMAIN")
NGINX_CONF = os.getenv("NGINX_CONF", "/etc/nginx/conf.d/clarity.conf")
if not PROJECT_ID:
    raise ValueError("PROJECT_ID is not set in the environment variables.")
if not CUSTOM_DOMAIN:
    raise ValueError("CUSTOM_DOMAIN is not set in the environment variables.")

def get_index(try_times = 5):
    resp = requests.get(f'https://www.clarity.ms/tag/{PROJECT_ID}')
    if resp.status_code != 200:
        if try_times > 0:
            return get_index(try_times - 1)
        raise RuntimeError(f"Failed to fetch data from Clarity. Status code: {resp.status_code}\n{resp.text}")
    return resp

def get_script(url, try_times = 5):
    resp = requests.get(url)
    if resp.status_code != 200:
        if try_times > 0:
            return get_script(url, try_times - 1)
        raise RuntimeError(f"Failed to fetch Clarity script. Status code: {resp.status_code}\n{resp.text}")
    return resp

resp = get_index()
script_content = resp.text

clarity_js_url = re.search(r'https://www.clarity.ms/s/[^"]+\.js', script_content)
if not clarity_js_url:
    raise RuntimeError("Clarity script URL not found in the response.")
clarity_js_url = clarity_js_url.group(0)
upload_url = re.search(r'"upload":"([^"]+)"', script_content)
if not upload_url:
    raise RuntimeError("Upload URL not found in the response.")
upload_url = upload_url.group(1)

clarity_js_version = clarity_js_url.split('/')[-2]
clarity_js_path = os.path.join(BASE_DIR, 's', clarity_js_version, 'clarity.js')
if not os.path.exists(clarity_js_path):
    resp = get_script(clarity_js_url)
    os.makedirs(os.path.dirname(clarity_js_path), exist_ok=True)
    with open(clarity_js_path, 'wb') as f:
        f.write(resp.content)

with open(NGINX_CONF, 'w') as f:
    f.write("""server {
    listen 80;
    listen [::]:80;
    listen 443 ssl;
    listen [::]:443 ssl;
    server_name %s;
    if ($scheme = http) {
        return 301 https://$host$request_uri;
    }
    location /c.gif {
        proxy_pass https://c.clarity.ms$request_uri;
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection "upgrade";
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Real-PORT $remote_port;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    }
    location /collect {
        proxy_pass https://%s$request_uri;
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection "upgrade";
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Real-PORT $remote_port;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    }
    root %s;
}
""" % (CUSTOM_DOMAIN, upload_url.split('/')[2], BASE_DIR))

os.system('nginx -s reload')

new_script_content = script_content.replace(
    'www.clarity.ms',
    CUSTOM_DOMAIN
).replace(
    upload_url.split('/')[2],
    CUSTOM_DOMAIN
).replace(
    'c.clarity.ms',
    CUSTOM_DOMAIN
)

with open(os.path.join(BASE_DIR, 'tag', PROJECT_ID), 'w') as f:
    f.write(new_script_content)
Loading...
Ownership of this post data is guaranteed by blockchain and smart contracts to the creator alone.