How to Create Scalable Zaps with Dynamic Data

Are your Zaps just not zapping like you want them to? 

‍

If you've dabbled in apps like Zapier and Make, you might have hit a wall trying to take your automations to the next level. As you try to move past simple workflows, you may be struggling to build automations that find all the data you need, and you may find that your Zaps or Scenarios end up becoming far too long and difficult to maintain. 

‍

Fortunately, there’s a reliable technique that can untangle your automation woes and help you build better, more consistent workflows. It's all about diving into the world of no-code databases, particularly in apps like Airtable and SmartSuite. 

‍

In this post, we're going to walk you through how leveraging an Operational Database in Airtable or SmartSuite can revolutionize your approach to automations, and will let you build systems that support multiple teams and departments. 

‍

As we go along, we’ll share some real-world examples from our own workflow toolkit so you can see exactly how it all works. 

‍

The problem: linear design

If you’re getting frustrated with Zapier or Make, you’re not alone. At XRay, we constantly hear from people who are struggling to connect their apps the way they want to with no-code automation providers. 

‍

In many cases, we find that the issue comes down to one critical concept: linear design.

‍

For new builders, it can be difficult to break out of a linear design pattern, especially since automation providers typically describe their product using this type of pattern. Most of the automation descriptions on Zapier and Make’s websites rely on a “When A, then B” format, like in the examples below. 

‍

‍

However, this setup is just the beginning, and if you keep trying to use it for every automation, you’ll quickly find that it becomes limiting.

‍

For example, let’s say you want to build an automation that creates a task in Asana (or any other task management app) whenever you add an emoji to a Slack message. 

Tip: if you'd like to learn how to add custom emojis to your Slack workspace, explore Slack's help docs here.

‍

‍

‍

It’s easy enough to set up a trigger in Zapier or Make that watches for new emoji reactions, and filters for custom “team task” emojis. Then, you can add an action that creates a task in your preferred software.

‍

‍

Unfortunately, the trigger doesn’t actually retrieve all the useful, dynamic data that might be needed when creating tasks. 

‍

For instance, how do you reliably attach the task to the appropriate project? The title of the channel might be a good starting point, but the title of your “Marketing” channel in Slack might not exactly match your “Marketing” project in your task management app, so adding a search step based on this data won’t be a reliable approach. 

‍

‍

‍

A single mismatched hyphen or underscore could prevent the automation from working correctly. 

‍

‍

And what if you want to automatically create tasks from multiple sources, like email or forms? If you want to avoid building redundant automations, you would need to mess around with things like Sub-Zaps or Webhooks, which can get a little complicated.

The solution: supporting your automations with an Operational Database (ODb)

There are many ways you could potentially solve automation design problems like these, but in our experience building no-code automations for our members and for our internal team, there’s one technique that consistently stands out.

‍

The trick is to connect all of your automations to what we at XRay call an “Operational database,” or ODb.

‍

While this term might sound like technical jargon, a well-designed operational database is arguably the single most important asset you’ll create as you’re building automations in the long term.

‍

It will enable automations that support your entire team with the dynamic data that Zapier, Make, and other similar platforms struggle to find on their own. 

‍

We’ll show you how it works and how it can fix all of these problems we’ve outlined and more.

Using an ODb to store and retrieve any data

Below is an example ODb that we could use to support the Task Creation automations we’ve described.

‍

‍

This is a pretty simple no-code database built in Airtable, which is what we typically use at XRay, although you can also accomplish something very similar with SmartSuite. 

‍

This database is a repository for all of the information that our automations use. We can match usernames to their emails and IDs in various apps, their job role, and a lot more. That way, we can create truly dynamic automations that suit an entire team instead of just an individual.

‍

We can assign tasks or send messages to anyone with a single automation that searches for a record in this database, because all of the info we need is present. 

Integrating an ODb into automated workflows

Now, let’s take a look at some updated automations that connect to this ODb instead of using a standard, linear design.

‍

By supporting the automations with an ODb, we can make sure that every task is assigned to the right person and connected to the right project.

‍

In the “people” table, we’ve stored the names of the custom emojis that represent each team member, as well as their IDs in Slack and Asana.

‍

‍

After the automation is triggered by a new emoji reaction in Sack, it searches in our Airtable database for a matching emoji name, which is guaranteed to find an exact match. 

‍

‍

The Asana ID found in this Airtable record can then be used to assign the right task. When a “mattj” emoji is added to a Slack message, this search step will find Matt J’s Asana ID. When a “tom” emoji is added, it will find Tom’s Asana ID, and so on. 

‍

‍

Dynamically selecting an Asana project with an ODb

With an ODb, you can use largely the same process to attach the task to the right project in Asana. The table pictured below lists all of the projects at the company, and includes their exact names and IDs in Slack and Asana. 

‍

‍

With this information, we can be confident that lookups will always work correctly. 

‍

Using an ODb as an advanced lookup table

With an Operational Database, you don’t just go from App A to App B. You can search in the database for any data you need, send it to other apps, and update the database with the latest information. 

‍

When you use an ODb, you don’t need to worry about necessary data missing from a Zapier trigger, action, or search. Just store any data you need in your ODb to conveniently access it whenever you want from any automation.

‍

Additionally, by leveraging linked records in your ODB, you can associate data in one table with data in another table. Even as you add more and more information, you can keep the whole database neat and organized with one table for “People,” one for “Tasks,” one for “Projects,” and so on. 

‍

‍

You don’t need to worry about a given Zapier trigger or action not having the right data, because you can find it all with a simple search step for the right record in your database. 

‍

And unlike a simple lookup table in Zapier, you can refer to this database from any number of automations, on any number of automation platforms, and include as many attributes as you want. Plus, you don’t need an expensive subscription like Zapier Tables just to put this database together.

Controlling automated activity and building modular automations with an ODb

However, an Operational Database is more than just an advanced lookup table. With views and filters, you can use an ODb as a control center for your automations.

‍

For example, one useful aspect of an ODb in Airtable is that you can launch automations whenever a record enters a specified view or meets certain conditions.

‍

‍

This is a very flexible automation trigger that gives you a lot of options for controlling your workflows, and can help you to build modular automations that avoid redundancy. 

‍

For instance, if we wanted to expand our task creation automation and establish multiple ways to create tasks, we could use our ODb to create a system that funnels data from several sources to a single endpoint. 

‍

Our main task creation Zap, pictured below, triggers in Zapier whenever a record enters the “Create Task” view in our database. 

‍

‍

So whenever we want to add a new method for creating tasks, we don’t need to add steps that create the task every single time. 

‍

All we need to do is get the information into Airtable, and then the main automation that we already built will take over. 

‍

This means that we can have one automation that collects data from Slack, one that collects data from a form, and one that collects data from an email inbox.

‍

‍

They all send their data to Airtable, which launches a single automation that actually creates the task. 

‍

This is a much more scalable, maintainable approach than building three fully separate automations, and you don’t need to mess around with sub-Zaps or webhooks to do it. 

Additional benefits of using an Operational Database

In addition to all of the advantages that we’ve described so far, you can also use your database to create indefinite delays, as we demonstrated in this earlier Zapier tutorial.

‍

With an ODb, you can also create several smaller automations that run on demand, or transform data however you want with formula fields, and much more. 

‍

We can’t cover every feature and benefit in a single blog post, but if you’d like to learn more about designing scalable and dynamic automations with an Operational Database, you can sign up for our Workflow Design Course. 

Going beyond linear design to build better automations

Building simple, straightforward automations in Zapier or Make is a good way to get started, but creating great automated workflows requires a bit more. It's all about thinking beyond the linear and embracing the power of a well-crafted database.

‍

With an Operational Database in Airtable or SmartSuite, you can get all of the data you need in one place to build dynamic automations that support entire teams. You can control when each automated action runs, and you can avoid creating redundant automations that all do basically the same thing. 

‍

Essentially, with an Operational Database, you can build complete systems instead of just building automations.

If you’d like to learn more about designing and implementing automated workflows with no-code tools, be sure to check out the other posts on our blog or our YouTube channel. You can also follow XRay on Twitter, Facebook, or LinkedIn.