# Scheduled sync > Run an integration guide's code once for history, then every 15 minutes, on AWS Lambda, cron or GitHub Actions. Sending one message to the tracker is a single `POST`. Doing it reliably for a live system needs a small piece of infrastructure: a job that runs on a schedule, pulls what's new, and sends it. You host and run this job yourself. This page gives you one runner that works with every [integration guide](/integrations). ## How it fits together Every universal API guide exports one function with the same shape: ```js export async function syncZendesk({ since, historical = false }) { /* ... */ return newSince; } ``` - `since` is Unix time in milliseconds. The function sends everything from that point on. - `historical: true` sends everything to `/trackhistorical`. Without it, each message goes to `/track`, unless it is more than an hour old when the job sees it, in which case it goes to `/trackhistorical`. - It returns the new `since`: store it, and pass it to the next run. That stored value is the **cursor**. The runner loads the cursor, calls the guide's function, and saves the cursor. If a run fails, the cursor isn't saved, so the next run tries again from the same point. That is safe: duplicates are removed automatically, and the helper already retries the tracker's `429` and `5xx` responses. ## Your project Set up the project as in [Set up your project](/api/tracker#set-up-your-project), then add the guide's code and the two runner files. For Zendesk: ```text dl-sync/ package.json "type": "module" dimensionlabs.mjs the helper, downloaded zendesk.mjs the code from the guide's "Send it" step run.mjs runs it from your computer, cron or GitHub Actions handler.mjs runs it on AWS Lambda ``` In both runner files, change the one import line to your guide, for example `import { syncQualtrics as sync } from './qualtrics.mjs';`. Each guide's opening comment lists the environment variables it reads. ## Load history first `run.mjs` runs a guide from your computer or a server. With a date, it loads history; without one, it picks up from the saved cursor. ```js // run.mjs. History: node run.mjs 2026-01-01 Since the last run: node run.mjs const FILE = new URL('./cursor.json', import.meta.url); const arg = process.argv[2]; let since, historical = false; if (arg) { since = Date.parse(arg); if (Number.isNaN(since)) throw new Error(`Can't read the date "${arg}". Use YYYY-MM-DD.`); historical = true; } else { try { ({ since } = JSON.parse(await readFile(FILE, 'utf8'))); } catch { since = Date.now() - 60 * 60 * 1000; } // first run: start an hour ago } const next = await sync({ since, historical }); await writeFile(FILE, JSON.stringify({ since: next })); console.log(`Sent everything from ${new Date(since).toISOString()}. The next run starts at ${new Date(next).toISOString()}.`); ``` To load history: 1. Set the environment variables: `export DL_API_KEY=...` and the vendor's variables from the guide. 2. Run `node run.mjs 2026-01-01` with the oldest date you want. Everything goes to `/trackhistorical`. A long history can take hours: the code waits whenever the vendor or the tracker asks it to. 3. When it finishes, start a [backfill](/dimensions/backfill) from the **Backfill** tab. A backfill typically shows up within the hour, depending on how much data is being enriched. Start the scheduled job before or after the history load: where the two overlap, the duplicates are removed. Some vendors cap how far back their API reaches (the guide says so); for anything older, [upload a file](/data/upload-a-file) instead. ## Run it every 15 minutes on AWS Lambda `handler.mjs` keeps the cursor in AWS Systems Manager Parameter Store: ```js // handler.mjs. Lambda handler: handler.handler const ssm = new SSMClient({}); const Name = process.env.CURSOR_PARAM; // for example /dimensionlabs/zendesk/since export const handler = async () => { let since; try { since = Number((await ssm.send(new GetParameterCommand({ Name }))).Parameter.Value); } catch (err) { if (err.name !== 'ParameterNotFound') throw err; since = Date.now() - 60 * 60 * 1000; // first run: start an hour ago } const next = await sync({ since }); await ssm.send(new PutParameterCommand({ Name, Value: String(next), Type: 'String', Overwrite: true })); return { since, next }; }; ``` ### 1. Package the code In your project folder: ```bash npm install @aws-sdk/client-ssm zip -r function.zip . -x "cursor.json" -x ".git/*" ``` The zip holds `package.json` (with `"type": "module"`), `dimensionlabs.mjs`, your guide's file, `handler.mjs` and `node_modules`. ### 2. Create the function 1. In the AWS console, open **Lambda** and select **Create function**. 2. Choose **Author from scratch**, enter a name such as `dl-sync-zendesk`, and pick a **Node.js** runtime (Node.js 22.x, or the newest offered). Select **Create function**. 3. On the **Code** tab, choose **Upload from > .zip file** and upload `function.zip`. 4. Under **Runtime settings**, select **Edit** and set **Handler** to `handler.handler` (the file `handler.mjs`, the export `handler`). ### 3. Configure it On the **Configuration** tab: 1. **General configuration > Edit.** Raise **Timeout** from the 3-second default to 5 minutes (Lambda allows up to 15 minutes). Set **Memory** to 256 MB. Save. 2. **Environment variables > Edit.** Add `DL_API_KEY`, `CURSOR_PARAM` (for example `/dimensionlabs/zendesk/since`) and the vendor variables from the guide's opening comment. Lambda encrypts environment variables at rest. 3. **Permissions.** Open the execution role (the link under **Role name**), choose **Add permissions > Create inline policy**, switch to **JSON**, and paste the following, with your region and account id. Name it and create it. ```json { "Version": "2012-10-17", "Statement": [{ "Effect": "Allow", "Action": ["ssm:GetParameter", "ssm:PutParameter"], "Resource": "arn:aws:ssm:YOUR_REGION:YOUR_ACCOUNT_ID:parameter/dimensionlabs/*" }] } ``` Guides that read from AWS (Amazon Connect's S3 bucket, Redshift's Data API) need their own permissions on this role too. 4. **Concurrency > Edit.** Choose **Reserve concurrency** and enter `1`, so two runs never overlap. 5. **Asynchronous invocation > Edit.** Set **Retry attempts** to `0`. The next scheduled run picks up anything a failed run missed. On the **Test** tab, select **Test** to run it once. The result shows `since` and `next`; the log shows any error from the vendor or the tracker. ### 4. Schedule it 1. In the AWS console, open **Amazon EventBridge**. Under **Scheduler** in the left menu, choose **Schedules**, then **Create schedule**. 2. Enter a name. Under **Schedule pattern**, choose **Recurring schedule**, then **Rate-based schedule**, and enter `15` **minutes**. Set **Flexible time window** to **Off**. Select **Next**. 3. Under **Target detail**, choose **AWS Lambda Invoke** and pick your function. Select **Next**. 4. Under **Permissions**, choose **Create new role for this schedule**. EventBridge Scheduler creates a role that is allowed to invoke the function. Select **Next**, then **Create schedule**. ### 5. Check the first runs 1. In the function's **Monitor** tab, select **View CloudWatch logs** and look at the first few runs for errors. 2. Ask the agent *"How many messages arrived in this source today?"* and compare with your source system. If a run hits the timeout, it saves no cursor and the next run starts from the same point. That only happens when a backlog is too big for one run: catch up from your computer with `node run.mjs` and a date, then let the schedule carry on. ## Run it with cron On a Linux server, `run.mjs` keeps the cursor in `cursor.json` next to it. Put the environment variables in `env.sh` (`export DL_API_KEY=...` and so on), then add this line with `crontab -e`: ```bash */15 * * * * cd /opt/dl-sync && . ./env.sh && flock -n /tmp/dl-sync.lock node run.mjs >> sync.log 2>&1 ``` `flock -n` skips a run while the last one is still going. ## Run it with GitHub Actions Put your project in a private repository with a `package-lock.json` (from `npm install`), and add the API keys under **Settings > Secrets and variables > Actions**. Don't commit `cursor.json`: the workflow keeps it in the Actions cache. ```yaml # .github/workflows/dl-sync.yml name: Dimension Labs sync on: schedule: - cron: '*/15 * * * *' workflow_dispatch: concurrency: dl-sync jobs: sync: runs-on: ubuntu-latest timeout-minutes: 30 steps: - uses: actions/checkout@v4 - uses: actions/setup-node@v4 with: node-version: 22 - run: npm ci - uses: actions/cache/restore@v4 with: path: cursor.json key: dl-cursor-${{ github.run_id }} restore-keys: dl-cursor- - run: node run.mjs env: DL_API_KEY: ${{ secrets.DL_API_KEY }} ZENDESK_SUBDOMAIN: ${{ secrets.ZENDESK_SUBDOMAIN }} ZENDESK_EMAIL: ${{ secrets.ZENDESK_EMAIL }} ZENDESK_API_TOKEN: ${{ secrets.ZENDESK_API_TOKEN }} - uses: actions/cache/save@v4 with: path: cursor.json key: dl-cursor-${{ github.run_id }} ``` GitHub runs scheduled workflows on a best-effort basis, so runs can start late. A late run still picks up from the saved cursor. If the cache is ever lost, the next run starts an hour back; run `node run.mjs` with a date to fill a longer gap. On Google Cloud, the same `run.mjs` can run as a Cloud Run job on a Cloud Scheduler trigger; see Google's [Execute jobs on a schedule](https://cloud.google.com/run/docs/execute/jobs-on-schedule). Keep the cursor somewhere that outlives the job, such as a Cloud Storage object. ## Make it safe to re-run - Use your system's own conversation ids for `sessionId` and `conversationId`, so a re-sent message lands in the same conversation. The guides do this. - Re-sending is safe: duplicates are removed automatically. - Retries are built in: the helper retries the tracker, and each guide's `get()` retries the vendor's `429` and `5xx` responses, honoring `Retry-After`. Any other error stops the run, and the next run tries again. - Keep the clock in milliseconds and in UTC. `toMillis` reads dates without a time zone as UTC. - Redact personal information in the function, before you send. See [PII redaction](/data/pii-redaction). Stuck? Email support@dimensionlabs.io with the source name, the error from the log, and one sample payload (with keys removed).