Top cloud databases for server GTM explained

A server GTM container functions as a reverse proxy. The drawback of such an approach is that there is no data storage. However, there are situations when you would need temporary data storage to use it in your server GTM. In such cases, a cloud database will serve.

In this article, we will review the most popular databases for storing data. We will highlight each database's killer features to make your decision-making as easy as possible.

In case you want to create a customer database or use Customer Data Platforms (CDP) you will need a database. It helps efficiently gather and store all the information about your customers/website visitors.

Among the popular scenarios for using a cloud database are:

• Enrich the pageview event with user data from the purchase event.
• Save the click ID and use it for events when the click ID is not available.

Let's consider how data flow works.

1. Whenever the user makes a purchase, CRM sends a webhook to the server GTM (in case you use webhooks).

2. The Data Client processes the requests and makes them in sGTM format.

3. In server GTM, the configured variable looks for the user who purchased the database:

• If the user exists - the variable takes the new data from the request (if there is such) and merges all the data.
• If no such user exists in the database, - it creates a new record.

4. A customer profile with enriched data is sent to the vendor (for example, Google Analytics, Facebook Ads) through a server-to-server connection.

There are relational databases (SQL-based) and non-relational databases (aka NoSQL). The difference is in how they are structured. Relational ones use tables with predefined schemas (like spreadsheets), while non-relational databases are more flexible and can store data as documents, key-value pairs, or graphs.

We recommend using NoSQL databases, as their horizontal scale design makes them better for data scaling. They can handle large volumes of event data, such as numerous impressions, page views, and clicks. In the next section, we review such databases.

If you have integrations with Stape, Google Cloud, Amazon, or Azure, we suggest paying attention to Stape Store, Firebase, DynamoDB, or Cosmos DB accordingly. This way, the setup process may be easier for you or for the developers you ask to configure data storage.

To pick the right database, you need to understand the approximate volume of traffic that needs to be processed and stored. For websites or apps with many concurrent users and massive real-time updates, it is better to consider databases such as DynamoDB or Cosmos DB, as they can effectively handle up to 100 million daily requests.

Some databases may be completely free (such as Stape Store for users with Pro or higher plan) in this case, skip this criteria. Usually, databases stick to specific plans (like Firestore does) or on-demand capacity (like Amazon DynamoDB or Azure Cosmos DB).

You may benefit from an on-demand capacity pricing model if you have a predictable request volume.

Below, we review top databases, highlighting what purpose they will serve the best, but here is a quick recap, and you can jump directly to the section with the detailed database description.

Stape Store will be the most handy and effective solution for those who already use Stape or are going to start using it. The solution was developed specifically to store data from server GTM, so you can avoid any errors due to complicated configuration. You can store JSON data organised in documents (1mb is the maximum size of a document).

The database is seamlessly integrated with Stape hosting for server GTM, and no additional setup is required. We provide all the required instructions and documentation on how to use Stape Store, if you have any problems you can contact our support team. 

Besides, our solution is completely free for those who use Pro or higher plan, which makes it stand out among other databases.

We have created solutions to write, retrieve and sync data to work with Stape Store:

• Stape Store Writer tag - developed to store authorisation keys and other valuable data for tracking.
• Stape Store Lookup variable - lookup the necessary data from the doc.
• Stape restore variable - write and get data at the same time.

Firestore is a good option for those hosting server-side GTM on the Google Cloud Platform, providing seamless integration with other Firebase and GCP products. In terms of functionality and interface, Stape Store and Firestore are pretty similar.

Firestore is a document-based (JSON) database; it works excellently for offline-first websites due to its native cache implementation. When integrated with Google Firebase, Firestore can use web socket connections for data exchange. This allows Firestore to build real-time applications.

Firestore offers a free trial of 50,000 reads, 20,000 writes, and 20,000 deletes per day. Depending on the number of operations, it can charge money according to its pricing.

At Stape, we developed templates for Firestore that will help to write and retrieve data in your server GTM:

• Firestore Writer tag - use it to record and write data from the server Google Tag Manager to Firestore.
• Firestore reStore Variable - store data inside Firebase and restore data values in case they are found by identifiers.
• Firestore Request Delay tag - manage the timing of event processing in sGTM by delaying, replaying, or storing events in Firebase Firestore.

If Firestore works for you, please see our instructions on how to set up the solutions developed by Stape:

• How to install Firestore Request Delay tag for Google Tag Manager server container
• How to write data to Firestore from server Google Tag Manager

Amazon DynamoDB is a low-latency database that can effectively process a large number of requests per day (up to 100 million requests). A low-latency database is a management system optimised for extremely fast response times. It ensures that when a user requests data, the system retrieves and returns it with minimal delay or lag. So, Amazon DynamoDB is a good fit for high-traffic websites or apps, including those running Amazon conversion tracking via server GTM, where storing and enriching conversion event data at scale is essential.

Also, Amazon DynamoDB is a popular database, and most back-end developers have dealt with it in one way or another, making it easier for your developer to work with it.

DynamoDB charges clients based on an on-demand capacity model, meaning you pay for the actual number of read and write requests you made. This can be more cost-efficient if your usage patterns are consistent.

Like Amazon DynamoDB, Azure Cosmos DB is a low-latency database that can process a large number of requests per day.

Cosmos DB provides a high level of flexibility and is known as a multi-model database service. It supports document, key-value, wide-column, and graph-based databases.

Azure's pricing system is based on consumed Request Units (RUs), which makes Azure Cosmos DB's pricing similar to Amazon DynamoDB's.

A server-side Google Tag Manager has a limitation: it lacks data storage. However, there are cases when you may need to use a temporary data storage solution (e.g. if you want to create a customer database), and the cloud databases can help here. Among the frequently used databases are Stape Store, Firestore, Amazon DynamoDB, and Azure Cosmos DB.

Each database has advantages and can be a great fit for specific cases. To pick the right one, we advise looking for NoSQL databases as they are better for data scaling, pay attention to the integrations you already have as it may be easier to set data storage and know your website or app traffic volume.

As a quick recap, we have a kind of "winning categories" for each database to make it easy for you to make a decision:

• Stape Store - seamless integration with Stape; it provides free use for those with a Pro or higher plan; you don't need to set up a database additionally, and there are no data storage limits.
• Google Firestore - best for those hosting server-side tagging on Google Cloud Platform and offline-first websites. Firestore provides separate plans according to the number of requests.
• Amazon DynamoDB - a popular database for websites that receive numerous daily requests (up to 100 million requests). It charges clients based on an on-demand capacity model.
• Azure Cosmos DB - a good option for large websites that need comprehensive flexibility.