Before you send it

A converted file can look right and still be wrong in ways that only show up weeks later as odd numbers. Run these checks every time, in this order.

1. Run the file checker

Drop the output into the file checker. It runs in your browser and finds the format problems behind most failed uploads: empty text, TRUE/FALSE in capitals, timestamps in seconds or scientific notation, colons in ids, line breaks in text, blank userIds, messages less than a second apart. Fix what it reports, or take its fixed copy. Codes are explained in Upload errors.

2. Compare the counts

Write these down for the export and for the converted file. If you used the toolkit, most of them are in its report.

Count	What you expect
Rows in the export	Matches what the source system says it exported
Rows written	Rows in, minus rows dropped (for one row per message); more than rows in for transcripts and surveys
Rows dropped, by reason	Every reason makes sense: empty text, system notes, unanswered questions
Distinct sessionIds	About the number of conversations, tickets, calls or respondents in the source
Rows per session	No session with one huge count (a sign that the session key is too broad, such as a customer id instead of a ticket id) and not every session with exactly one row (a sign that the transcript wasn’t split)
incoming: true share	Plausible for the channel: reviews are all true except owner replies; surveys are half true, one answer per question; chats and tickets have both sides
First and last timestamp	The export’s date range, in the right year, not 1970
Rows per day	No single day holding everything (a sign that times were missing and filled with one date)

3. Read ten conversations

Pick ten sessions at random and read them top to bottom in the converted file. Check that:

• each message is on the right side (incoming is true only for the customer);
• the messages are in the order they happened;
• no message is cut in half or glued to the next one;
• the text has no HTML, quoted replies or placeholder text;
• the extra columns hold the values you expect.

4. Check the ids

• sessionId is the ticket, chat, call or response id, and it is the same on every rerun.
• No sessionId or conversationId contains a colon.
• In bot and agent conversations, the replies carry the customer’s userId, not the bot’s or agent’s.
• No id is an email address, phone number or name.
• Long ids are whole numbers, not 1.70324E+12 or 12345.0.
• If you will join business data, the join key has the same format as in the business file: the file checker’s join test shows the match rate.

5. Check what you’re sending

• No personal data you don’t need in the extra columns: emails, phone numbers, addresses, names, access tokens. See PII redaction.
• Extra column names are consistent across files and months. Each one becomes a field the agent can filter on, so Store, store and store_name would be three different fields.
• Values in extra columns use one encoding: Yes, yes and 1 are three different values (conversation format).

6. After the upload

Ask the agent “How many rows are in my new source, and what date range does it cover?” and compare with your counts. A file can land partly without an error (Upload a file). An uploaded file is not enriched until you start a backfill from the Backfill tab.

The mistakes conversions make most

These come from real conversion scripts that each worked once:

Mistake	What it looks like afterwards	Prevention
Columns read by position	Text in the timestamp field, or ids where the text should be, after the export added a column	Map columns by name
Direction inverted, or guessed from “who spoke first”	Customers appear to say what agents said; complaint Dimensions land on agents	Map from a role column or a list of your people; read ten conversations
Random ids for users or sessions	Every rerun creates new users and new copies of the same conversations, which duplicate removal can’t catch	Build ids from ids in the export
Month off by one in hand-built dates	Everything a month late	Parse dates with a library, or check the first and last timestamp
Clock times read in the machine’s time zone	Times off by hours, different on each computer	Name the time zone
Timestamps rewritten to be one second apart across a whole history	Weeks between messages become seconds; time-based questions go wrong	Keep real times; only move messages that collide
Placeholder text for empty answers or bodies	[No Response] enriched as if a customer wrote it	Drop empty rows
One session per customer instead of per ticket	A few giant conversations	Use the ticket, chat or call id
Business records sent as messages	Orders and CRM notes enriched as customer text	Upload them as business data or as extra columns
Excel resaved as CSV	Long ids and timestamps in scientific notation, digits lost	Read the .xlsx directly
Output appended to last run’s files	Your counts don’t match the export, and last month’s rows go up again	Write to a fresh output folder
Personal data left in metadata, or used as an id	Emails and addresses stored with every message	Drop them, or hash a key you need to join on

Sending through the API instead

If you send the converted rows to the universal API rather than uploading a file:

• Send anything older than an hour to /trackhistorical, and run a backfill from the Backfill tab afterwards.
• Test with a handful of messages first and check they arrived before sending the rest.
• Send sequentially or with low concurrency. Rate limits are not published.
• Process rows in a fixed order and log what you’ve sent, so a run that stops halfway can resume where it stopped. Re-sending is safe: duplicates are removed when they are ingested.
• Retry only failures that a retry can fix (network timeouts, HTTP 429 and 5xx responses), with exponential backoff, and wait as long as a Retry-After header says. Any other 4xx response means the request itself was rejected: fix the message, don’t resend it unchanged.
• Write every message that still failed to a file, so you can resend exactly those.

Still stuck

Email [email protected] with the details listed in Get help. Dimension Labs technical support can run large or complex transformations for you.