Building a custom CDP and enriching event data using server-side GTM

Before making a purchase, customers interact with brands through various channels, both online and offline. This can make it difficult for marketers and analytics specialists to keep track of these interactions in one place. Standard tools like Google Analytics or Matomo server-side tracking often cannot track offline events. To solve this problem, people usually use Customer Data Platforms (CDP). 

One way to use a CDP is to purchase a pre-built tool, which can be expensive. Another option is to build your CDP. In our blog post, we will show how to build a simple version of CDP using the server Google Tag Manager and database. 

This setup allows you to combine all your customer information in one place, making it easier to understand and reach out to them more effectively. Whether you are new to tracking or an experienced specialist, this guide can help you create a CDP that significantly improves how you manage customer data.

The server Google Tag Manager container will receive requests on all user interactions, whether online or offline. You can use standard clients like GA4 or Data Client to receive online events from web GTM. To retrieve offline events, you can use webhooks. Later in the blog, I will show how to send and debug webhook data inside sGTM.

No matter what type of data sGTM receives, it will store the data in the database. There are several databases that sGTM can be integrated with, for example, Firebase, Supabase (much cheaper than Firestore), or Data Store. Here I will use the Stape Data Store since it is a built-in functionality inside Stape and is free of charge for Stape-paid customers (Pro plans or higher).

Besides the sGTM and store, we will also utilize a unique user identifier, which will help store data for one user in the same document. 

1. Data source (CRM, SDK, CMS, mobile app, website, etc).

2. Server GTM container.

3. Database (I will use Data Store in this guide. You can also utilize FireStore or Supabase, for which Stape created the Supabase writer tag).

4. To read/write data from the database these sGTM templates can be used:

• Stape Store writer tag
• Stape Store lookup variable
• Stape reStore variable
• Firestore Writer tag (in case you use Firestore)
• Firestore reStore variable
• Supabase Writer tag (in case you use Supabase)
• Supabase Lookup variable

5. Unique user identifiers. In this guide, I will use a Unique user ID or WordPress user ID that Stape WordPress plugin generates. 

This is how user data flows from the data source (CRM) to CDP:

1. 1.
   Every time a user purchases, CRM sends a webhook to sGTM. 
2. 2.
   Data Client in sGTM processes the incoming request in sGTM and maps it to sGTM format. 
3. 3.
   In the server GTM container, we use the reStore variable to identify if a user who made a purchase already exists in the database. If a user exists, we check if the request contains new user information, and then we merge the latest data with the one in the database. After that, write enriched information. If a user does not exist, just create a new customer record. 
4. 4.
   Merge user data in the incoming request with the user data in the database. For example, if the city parameter is missing in the CRM webhook,  there is no field city, then this field will be used from the Store. If incoming requests have a field city but the Store does not have it, or it is different, then data from the request for this field will be written to the store.
5. 5.
   Use a complete customer profile to send it to the vendor through a server-to-server connection.

1. 1.
   Website. The two most popular ways of sending data from the website to sGTM are by setting up server-side GA4 and Data Tag.
2. 2.
   CRM webhooks (Hubspot, SalesForce, Jira, etc).
3. 3.
   CMS. You can use Shopify server-side tracking, WordPress, or Magento 2 Google Tag Manager plugins to send refunds and purchase webhooks to sGTM.
4. 4.
   SDK and Mobile application. Stape created iOS, Android, PHP SDK, Node.js SDK.
5. 5.
   Payment processors (Stripe, Braintree, etc).

The method of receiving data depends on how you send data to sGTM. These are the three ways of obtaining data:

1. 1.
   GA4 client. Works if you use GA4 to deliver data to sGTM.
2. 2.
   Measurement protocol client. It can be used to control the type of request sent to sGTM and make it in measurement protocol standard. Usually, this is your application code.
3. 3.
   Data Client. It can be used when you do not control incoming requests since, in Data Client, you can modify the request path, for example, Stripe webhook.

This is the most tricky and crucial part. To build a proper CDP, you must have precise data on users and their actions with your brand. The CDP should contain up-to-date information and clear, not duplicated, data. You need a proper user ID to build such a system of user profiles. 

For identified users, you may use an email or internal ID that can be used across different systems. If a user is not identified, it can be a Stape User ID consisting of a hash of:

• Client IP (IPv4/IPv6)
• Browser User-Agent
• Browser TLS settings
• Hostname

If, due to privacy compliance, you want to make your User ID more unique, you can add a dependable variable. It can be a day, month, week, or year. It all depends on how unique the ID should be. 

We will use either the reStore variable or the Stape Store Writer Tag and Stape Store Lookup to merge and stitch user profiles from the requests and database. Copies of these templates are also available on Stape GitHub.

Writer tag and lookup variables are pretty self-explanatory. These two templates allow you to write and read data from the Data Store. 

The restore variable will first identify if the user exists in the database. At first, the variable gets all the user identifiers from our event.  (email, phone, or client ID.) Then, it sends a query to the Store to select customers that have one of such identifiers. If there is information about the customer in the database, it will return data about this customer.

When a variable gets the response from the Store, it merges it with an incoming request. For example, if no field city exists, this field will be used from the Store. If incoming requests have a field city but the Store does not have it, or it is different, then data from the request for this field will be written to the store. This variable creates a new entity if a customer is partially found in the store.

Once the data in sGTM is processed and formatted into the sGTM format, you can use it to send to any vendor you want, such as Meta, TikTok, Google Ads, or Analytics. Be aware of the user data that is sent in the requests. 

Debugging in sGTM is difficult, but offline events can be even trickier. We will discuss how to use the following:

• sGTM debugger
• Access/Request/Response Stape logs
• Logger tag
• sGTM preview header

I will present two use cases to demonstrate how the Data Store can be used. We have found that these are the most popular scenarios for Stape customers.

1. Configure user ID. I will use the Stape User ID power-up that will help to merge users to customers.

a. Log in to your Stape account and enable the power-up. To do so, open the sGTM container, click on Power up the tab, open the user ID, switch the toggle on, and save.

b. In the Server GTM container create a variable that will retrieve the user ID from the request headers in the server.

2. Write data to the database on the purchase event. Ensure you are adding a Stape user ID to the event data. 

a. Add a Stape store writer tag to the sGTM templates gallery (download it from GitHub and import it into sGTM tag template section) and create a tag to write event data on a purchase event. I’m writing all event data and adding a Stape user ID to the store record. We will need it to merge users on a pageview event.

b. If the tag worked correctly, you will see a record in your database similar to the screenshot below.

3. Enrich pageview event data with the stored data in the Data Store.

a. To merge users from the page view event and the one we store in the database, we will again use the Stape user ID and Stape Store lookup variable that can retrieve data from the store. Add the Stape Store Lookup variable to the sGTM variable template gallery. To download the template from the gitHub, go to the sGTM templates section. Click new in the variable sections, then import.

b. Create Stape Store Lookup variable for the parameters you want to restore: ad document path and path name. 

c. Add a Lookup variable to the pageview tag. 

4. Test the setup. You should see enriched user data in the tag triggered in the sGTM container. 

I want to send offline event data to an advertising network in this use case. When sending conversion data to the advertiser platform, the critical element is sending a click ID. Click ID is an identifier added to the URL or cookies when the user clicks on the advertising link. With the help of Click ID, the advertising network can easily attribute conversion to the ad campaign. 

The Click IDs are available for all the web events, but if you want to use conversion data from the CRM or CMS to send to the advertising network, in most cases, click IDs are not available.

For configuring this option, I will use the Stape restore variable to write data to the Data Store on pageview and restore data on purchase events. Besides that, I will need transformation in the sGTM container to augment the event sent to Facebook. Another critical element is matching users on pageview events to users in offline purchase webhooks. To do so you must use a user identifier that will remain the same on web pageview events and offline purchases. 

With standard CMSs, it´s easy to achieve because they usually have this type of identifier in cookies. You will need to ask developers to add these parameters for customer websites. My setup is done on a WordPress site. Stape WordPress plugin sends a unique user ID on web events and adds it to the webhook. Using this user ID, I can match users on a pageview and purchase webhooks in the database. 

1. Receive purchase webhook from offline data source inside sGTM. 

a. Send a webhook from the CRM. I will use the Stape WordPress app for this. Go to the Webhooks tab and add the Server GTM container URL. This is a domain you use for your server GTM container. We recommend setting up a custom domain within your website domain since it will allow setting up 1st party cookies. Don't forget to tick the checkboxes. 

b. Add Data client to the sGTM clients template and configure it to retrieve webhooks you are sending from the CRM. To do so, create a Data Client template and add a Webhook path.

c. Configure sGTM Preview header config power up on Stape to debug incoming webhooks in sGTM. Go to sGTM preview, click three dots in the top right corner, and click send requests manually. Copy the X-Gtm-Server-Preview HTTP header and paste it into the sGTM Preview header config power up.

d. Open the sGTM debugger and test that it received webhooks. Once the webhook is triggered, you should see the incoming request inside sgtm and that the data client claimed that request.

2. Write fbc, fbp, and client ID parameters on the pageviews event to the data to the database.

In this step, I will show you how to write fbc and fbp parameters to the database using the reStore variable. To do so, we will use a Facebook pageview tag to trigger pageview events and a transformation that will write data to the store using the restore variable.

a. Add a restore variable to the sGTM variable using this link. You would need to download it from GitHub, create a new variable template in sGTM, and click import.

b. Configure the restore variable. It should use a wordpress user ID and an identifier and write fbp, fbc and WordPress user ID to the store.

c. Create a transformation that will augment even data on Facebook pageview events using the restore variable and add this transformation to your FB pageview tag.

d. Test the setup. If all is configured correctly you should see data recorded in the Data Store. 

e. Set up a Facebook purchase event that will trigger every time a webhook is received. To augment this event with the data from the Data Store, we would need to use a transformation, and the same restore variable that we used for the transformation on a FB pageview event. This will extract the required parameter on a purchase event. But to send the data in the form that Facebook will recognize we will need to use an object extractor variable that extracts the required object from the restore variable.

f. Test your setup. If everything works correctly, you should see that fbc and fbp parameters were successfully sent to Facebook or on a purchase offline event.

With the help of Data Store, you can create a customer database or CDP. It provides an easy way to collect and store all the information you can get from your customers, which can help you build better analytics forms or improve the results of your paid campaigns. Augmenting and enriching your customer data can help you gain insights into their behavior and preferences, which in turn can help you improve your marketing strategies.