Skip to content
DIMENSION LABS Docs

02 Get your data in

The toolkit

One configurable script, the toolkit, that turns CSV, TSV, JSONL, JSON or XLSX exports into upload-ready files, with a report of every row it dropped or fixed.

One-off conversion scripts are easy to write and hard to reuse: column positions, sheet names and file paths end up hard-coded, and next month’s export needs a new script. The toolkit, to-dimensionlabs.mjs, keeps everything that changes in a small JSON mapping file. You write the mapping once per export type and rerun it every time.

For a quick one-time conversion without code, the in-browser converter may be all you need. Use the toolkit when the conversion repeats, or you need HTML stripping, skipped system rows, transcript splitting, a sheet other than the first, a header that isn’t in row 1, a time zone the converter doesn’t list, nested JSON paths or split output.

What it does

Section titled “What it does”
  • Reads CSV, TSV, JSONL, JSON or XLSX (the format is picked from the file extension).
  • Maps your columns to text, userId, incoming, sessionId, conversationId and dimensionlabs_timestamp, and keeps the other columns as metadata (or the ones you list).
  • Sets incoming from value lists ("customer" is true, "agent" and "bot" are false), or a constant.
  • Fills a blank userId with the sessionId, and gives the agent’s or bot’s replies the customer’s userId, so both sides of a conversation carry the end user’s id.
  • Reads timestamps as Unix milliseconds, Unix seconds, ISO 8601, clock times in a named time zone, or Excel serial dates.
  • Replaces colons in ids, flattens line breaks in text, optionally strips HTML, drops rows with empty text.
  • Splits transcripts with a speaker pattern, and reshapes wide surveys into question and answer rows, taking the question wording from the export’s own wording row.
  • Removes duplicate messages, keeps messages in a session at least 1,000 ms apart, and splits output into parts of about N rows without cutting a session.
  • Writes a report: rows in, messages out, every dropped row counted by reason, every fix counted, and a note when a kept column looks like personal data.

It writes CSV by default, or JSONL. A JSONL line has the same shape as a universal API message: extra fields go in a nested platformJson object and intent is an object ({ "name": "...", "confidence": 0.94 }). Uploads accept that shape, so either format works for a file upload.

Install and run

Section titled “Install and run”

Node 18 or later.

Terminal window
npm install papaparse      # always
npm install xlsx           # only if you read .xlsx files
node to-dimensionlabs.mjs --config mapping.json --input export.csv --out out/export

--input and --out override input.path and output.path in the mapping. Output files are <out>-1.csv, <out>-2.csv and so on, plus <out>.report.json. The script holds the whole input in memory; for an input larger than your machine’s memory, split it first or ask support.

Example: a help-desk export

Section titled “Example: a help-desk export”

Example input, lumen-tickets.csv:

lumen-tickets.csv
ticket_id,author_id,author_type,body,created_at,store_name,customer_email,customer_phone,priority
4471,c-88,end-user,"<div>My card was charged twice<br>for one flat white.</div>",1/3/2026 9:12 AM,Portland Pearl,[email protected],555-0100,high
4471,a-2,agent,Refund is on its way.,1/3/2026 9:20 AM,Portland Pearl,[email protected],555-0100,high
4471,sys,system,Ticket closed,1/3/2026 9:21 AM,Portland Pearl,[email protected],555-0100,high

The example mapping, to-dimensionlabs.example.json. Keys that start with _ are notes and are ignored:

to-dimensionlabs.example.json
{
  "_about": "Example mapping for to-dimensionlabs.mjs: a Lumen Coffee help-desk export, one row per message. Keys that start with _ are ignored. Docs: https://docs.dimensionlabs.io/transform/toolkit",


  "input": {
    "format": "auto",
    "_format": "auto picks from the extension: .csv, .tsv, .jsonl, .json, .xlsx",
    "sheet": "Messages",
    "headerRow": 1,
    "skipRowsAfterHeader": 0,
    "_skipRowsAfterHeader": "survey tools often add a row of question text under the column names: set 1 to skip it (Qualtrics adds two: set 2)"
  },


  "shape": "messages",
  "_shape": "messages = one row per message; transcript = one row per conversation with the whole transcript in a cell; survey = one row per respondent with one column per question",


  "columns": {
    "text": "body",
    "userId": "author_id",
    "sessionId": "ticket_id",
    "conversationId": "ticket_id",
    "timestamp": "created_at",
    "_columns": "source column names; use a dot path such as sender.id for nested JSON. Never use an email, phone number or name as userId or sessionId. Optional: intent and intentConfidence, for an intent-based chatbot"
  },


  "userIdForReplies": "customer",
  "_userIdForReplies": "customer (default) gives the agent's and bot's replies the end user's userId, so both sides of a conversation carry the customer's id; own keeps each row's userId",


  "incoming": {
    "column": "author_type",
    "true": ["customer", "end-user"],
    "false": ["agent", "bot", "system"],
    "_incoming": "values are matched case-insensitively; rows with any other value are dropped and counted. Use \"incoming\": true when every row is from the customer"
  },


  "timestamp": {
    "format": "auto",
    "timeZone": "America/Los_Angeles",
    "dayFirst": false,
    "_timestamp": "auto reads Unix ms, Unix seconds, ISO 8601 and Excel serial dates. timeZone applies to times written without a zone, such as 1/3/2026 9:12 AM"
  },


  "clean": {
    "flattenNewlines": true,
    "stripHtml": true,
    "replaceColonsWith": "-"
  },


  "metadata": {
    "keep": "unmapped",
    "drop": ["customer_email", "customer_phone"],
    "rename": { "store_name": "store" },
    "_metadata": "keep: unmapped (every column not used above), none, or a list of column names. Drop personal data you don't need"
  },


  "constants": { "channel": "email" },


  "skip": [
    { "column": "author_type", "values": ["system"] }
  ],
  "_skip": "rows whose column holds one of these values are dropped and counted, for example automatic ticket-closed notes",


  "dedupe": true,
  "spaceMessagesMs": 1000,


  "transcript": {
    "_used_when": "shape is transcript",
    "column": "transcript",
    "speakerPattern": "^(Agent|Customer):\\s*",
    "incomingSpeakers": ["Customer"],
    "outgoingSpeakers": ["Agent"],
    "secondsApart": 1,
    "userId": "session"
  },


  "survey": {
    "_used_when": "shape is survey",
    "questionsFromRow": 2,
    "_questionsFromRow": "the file row (counting from 1) that holds the question wording; it must be one of the rows skipped by input.skipRowsAfterHeader. A question's own text, if set, wins",
    "questions": [
      { "column": "why_score", "text": "Why did you give that score?" },
      { "column": "improve", "text": "What one thing should we improve?" }
    ]
  },


  "output": {
    "format": "csv",
    "path": "out/lumen-tickets",
    "chunkRows": 50000,
    "_output": "csv or jsonl. chunkRows 0 writes one file; otherwise files of about that many rows, never splitting a session"
  }
}
Terminal window
node to-dimensionlabs.mjs --config to-dimensionlabs.example.json --input lumen-tickets.csv
Rows in: 3  Messages out: 2  Sessions: 1  Dropped: 1
  dropped 1: skipped: author_type is system
  fixed 1: html removed from text
  fixed 1: line breaks in text replaced with spaces
  fixed 1: replies given the customer's userId
  wrote out/lumen-tickets-1.csv (2 rows)
  report: out/lumen-tickets.report.json
out/lumen-tickets-1.csv
sessionId,userId,incoming,text,dimensionlabs_timestamp,conversationId,store,priority,channel
4471,c-88,true,My card was charged twice for one flat white.,1767460320000,4471,Portland Pearl,high,email
4471,c-88,false,Refund is on its way.,1767460800000,4471,Portland Pearl,high,email

The email and phone columns were dropped, store_name was renamed, the automatic “Ticket closed” note was skipped, channel was added to every row, and the agent’s reply carries the customer’s id, c-88.

Mapping reference

Section titled “Mapping reference”
KeyWhat it does
input.formatauto (default, from the extension), csv, tsv, jsonl, json or xlsx.
input.sheetXLSX: the sheet name. Default: the first sheet.
input.headerRowThe row that holds the column names, counting from 1. Default 1.
input.skipRowsAfterHeaderRows to skip under the header, such as a row of question wording. Default 0.
input.delimiterForce a delimiter. Default: detected (tab for .tsv).
input.recordsPathJSON: the key that holds the array of records, if the file isn’t an array. Default data.
shapemessages (one row per message, default), transcript (one row per conversation, transcript in a cell) or survey (one row per respondent).
columns.text, .userId, .sessionId, .conversationId, .timestampSource column names. Dot paths such as sender.id read nested JSON. conversationId defaults to the sessionId. Never map an email address, phone number or name to userId or sessionId: use the tool’s internal ids.
columns.intent, .intentConfidenceOnly for an intent-based chatbot: the intent name and its confidence. JSONL output writes them as "intent": { "name": "...", "confidence": 0.94 }.
userIdForRepliescustomer (default): every outgoing message in a session takes the userId of the session’s first incoming message (or the sessionId if there is none), so the bot’s and agent’s replies carry the end user’s id. own keeps each row’s own userId.
incoming{ "column": "...", "true": [...], "false": [...] }, matched case-insensitively. With only one list, every other value is the other direction. A value in neither list drops the row. Or true / false for the whole file.
timestamp.formatauto (default), ms, s or excel. auto reads 13-digit milliseconds, 10-digit seconds, numbers below 1,000,000 as Excel serial dates, and date text. Values in scientific notation are dropped, because their digits are lost.
timestamp.timeZoneThe zone of clock times without one, such as America/Chicago. Default UTC, with a note in the report.
timestamp.dayFirsttrue when 03/01/2026 means 3 January. Default false.
clean.flattenNewlinesReplace line breaks in text with spaces. Default true.
clean.stripHtmlRemove HTML tags and decode entities. Default false.
clean.replaceColonsWithReplacement for : in sessionId and conversationId. Default -.
metadata.keepunmapped (default: every column not used above), none, or a list of columns.
metadata.dropColumns never to keep, such as emails and phone numbers.
metadata.rename{ "source name": "new name" }.
constantsColumns added to every row, such as { "channel": "email" }.
skip[{ "column": "...", "values": [...] }]: rows to drop and count, such as system notes.
dedupetrue removes repeated messages (same session, direction, time and text). Default false.
spaceMessagesMsMinimum gap between messages in a session; later messages are moved. Default 1000. 0 turns it off.
transcript.column, .speakerPattern, .incomingSpeakers, .outgoingSpeakers, .secondsApart, .userId, .stripQuotesFor shape: transcript. The pattern’s first group is the speaker label. Lines without a label join the turn above. userId: "speaker" gives each speaker their own id; the default keeps the row’s userId (or the session id) on every turn.
survey.questionsFor shape: survey: a list of question columns, as names (["why_score", "improve"]) or as { "column": "...", "text": "the question as asked" }. Each answered question becomes an outgoing question row and an incoming answer row. Blank answers are skipped.
survey.questionsFromRowThe file row, counting from 1, that holds the question wording, such as 2 when the export puts it under the column names. It must be one of the rows skipped by input.skipRowsAfterHeader. A question’s own text wins over it; with neither, the column name is used and the report says so.
output.formatcsv (default) or jsonl (one universal API message per line, metadata nested in platformJson). Both upload.
output.pathOutput prefix. Default: next to the input.
output.chunkRowsSplit output into parts of about this many rows, never cutting a session. Default 0 (one file).

Tested examples

Section titled “Tested examples”

Each example below was run exactly as shown.

Value lists, HTML, colons, seconds, duplicates and empty rows

Section titled “Value lists, HTML, colons, seconds, duplicates and empty rows”
tickets.csv
ticket_id,author,author_type,body,created_at,store,customer_email
LC:1001,cust-17,Customer,"<p>My oat latte was cold &amp; late.</p>",2026-01-03T17:12:00Z,Portland Pearl,[email protected]
LC:1001,agent-4,Agent,"Sorry about that!
We've refunded you.",2026-01-03T17:12:00Z,Portland Pearl,[email protected]
LC:1001,cust-17,Customer,,2026-01-03T17:14:00Z,Portland Pearl,[email protected]
LC:1002,,customer,Do you sell decaf beans?,1767460320,Austin Eastside,[email protected]
LC:1002,agent-9,Bot,Yes: our Night Owl decaf.,1767460380000,Austin Eastside,[email protected]
LC:1002,agent-9,Bot,Yes: our Night Owl decaf.,1767460380000,Austin Eastside,[email protected]
LC:1003,cust-22,Visitor,Is the Pearl store open on Sunday?,2026-01-04T10:00:00Z,Portland Pearl,[email protected]
tickets.json
{
  "input": { "format": "auto" },
  "shape": "messages",
  "columns": { "text": "body", "userId": "author", "sessionId": "ticket_id", "timestamp": "created_at" },
  "incoming": { "column": "author_type", "true": ["customer"], "false": ["agent", "bot"] },
  "timestamp": { "format": "auto", "timeZone": "UTC" },
  "clean": { "stripHtml": true, "flattenNewlines": true, "replaceColonsWith": "-" },
  "metadata": { "keep": "unmapped", "drop": ["customer_email"] },
  "constants": { "channel": "email" },
  "dedupe": true,
  "spaceMessagesMs": 1000,
  "output": { "format": "csv", "chunkRows": 0 }
}
Terminal window
node to-dimensionlabs.mjs --config tickets.json --input tickets.csv --out out/tickets
Rows in: 7  Messages out: 4  Sessions: 2  Dropped: 3
  dropped 1: empty text
  dropped 1: incoming value not recognized: "visitor"
  dropped 1: duplicate message
  fixed 3: ids with colons rewritten
  fixed 1: html removed from text
  fixed 1: line breaks in text replaced with spaces
  fixed 1: timestamps in seconds converted to milliseconds
  fixed 1: blank userId filled from sessionId
  fixed 2: replies given the customer's userId
  fixed 1: messages moved to be 1000 ms apart
  wrote out/tickets-1.csv (4 rows)
  report: out/tickets.report.json
out/tickets-1.csv
sessionId,userId,incoming,text,dimensionlabs_timestamp,conversationId,store,channel
LC-1001,cust-17,true,My oat latte was cold & late.,1767460320000,LC-1001,Portland Pearl,email
LC-1001,cust-17,false,Sorry about that! We've refunded you.,1767460321000,LC-1001,Portland Pearl,email
LC-1002,LC-1002,true,Do you sell decaf beans?,1767460320000,LC-1002,Austin Eastside,email
LC-1002,LC-1002,false,Yes: our Night Owl decaf.,1767460380000,LC-1002,Austin Eastside,email

The agent’s and the bot’s replies carry the customer’s id: cust-17 in LC-1001, and in LC-1002, where the customer’s id was blank, the session id.

Clock times in a named time zone, TSV in, JSONL out

Section titled “Clock times in a named time zone, TSV in, JSONL out”
chat.tsv (tab-separated)
conv  sender  who  message  sent
c1  u1  customer  Hi, is the Pearl store open?  1/3/2026 9:12 AM
c1  bot  bot  Yes, until 6 PM.  1/3/2026 9:12 AM
c2  u2  customer  Summer hours?  7/4/2026 12:30 PM
c3  u3  customer  Loyalty card lost  Oct 3 2025  9:12AM
chat.json
{
  "columns": { "text": "message", "userId": "sender", "sessionId": "conv", "timestamp": "sent" },
  "incoming": { "column": "who", "true": ["customer"] },
  "timestamp": { "timeZone": "America/Los_Angeles" },
  "output": { "format": "jsonl" }
}
Terminal window
node to-dimensionlabs.mjs --config chat.json --input chat.tsv --out out/chat
Rows in: 4  Messages out: 4  Sessions: 3  Dropped: 0
  fixed 1: replies given the customer's userId
  fixed 1: messages moved to be 1000 ms apart
  wrote out/chat-1.jsonl (4 rows)
  report: out/chat.report.json
out/chat-1.jsonl
{"text":"Loyalty card lost","userId":"u3","incoming":true,"sessionId":"c3","conversationId":"c3","dimensionlabs_timestamp":1759507920000}
{"text":"Hi, is the Pearl store open?","userId":"u1","incoming":true,"sessionId":"c1","conversationId":"c1","dimensionlabs_timestamp":1767460320000}
{"text":"Yes, until 6 PM.","userId":"u1","incoming":false,"sessionId":"c1","conversationId":"c1","dimensionlabs_timestamp":1767460321000}
{"text":"Summer hours?","userId":"u2","incoming":true,"sessionId":"c2","conversationId":"c2","dimensionlabs_timestamp":1783193400000}

The bot’s reply carries the customer’s id, u1, not bot. 1/3/2026 9:12 AM in Los Angeles is 1767460320000 (17:12 UTC, standard time); 7/4/2026 12:30 PM is 1783193400000 (19:30 UTC, daylight saving time).

Excel: a named sheet, a title row, serial dates, long ids and an error cell

Section titled “Excel: a named sheet, a title row, serial dates, long ids and an error cell”

The workbook chats.xlsx has a Read me sheet and a Chats sheet with a title in row 1. Build it with:

make-xlsx.mjs
import XLSX from 'xlsx';
import * as fs from 'node:fs';
XLSX.set_fs(fs);
const rows = [
  ['Lumen Coffee chat export'],
  ['Chat ID', 'Visitor', 'Role', 'Message', 'Sent at', 'Order no'],
  [880011223344, 'v-1', 'Visitor', 'Can I change my pickup store?', 46025.38333333333, 4400123456789],
  [880011223344, '', 'Barista', 'Sure, which store?', 46025.38333333333, 4400123456789],
  [880011223345, 'v-2', 'Visitor', 'Gift card balance?', 46026.5, { t: 'e', v: 0x2A }],
];
const ws = XLSX.utils.aoa_to_sheet(rows);
// mark the date column as a date format so it looks like a real export
for (const r of [3, 4, 5]) { const c = ws['E' + r]; c.z = 'm/d/yyyy h:mm'; }
ws['F5'] = { t: 'e', v: 0x2A, w: '#N/A' };
const wb = XLSX.utils.book_new();
XLSX.utils.book_append_sheet(wb, XLSX.utils.aoa_to_sheet([['notes']]), 'Read me');
XLSX.utils.book_append_sheet(wb, ws, 'Chats');
XLSX.writeFile(wb, 'chats.xlsx');
console.log('ok');
xlsx.json
{
  "input": { "sheet": "Chats", "headerRow": 2 },
  "columns": { "text": "Message", "userId": "Visitor", "sessionId": "Chat ID", "timestamp": "Sent at" },
  "incoming": { "column": "Role", "true": ["visitor"], "false": ["barista"] },
  "timestamp": { "format": "auto", "timeZone": "America/Chicago" },
  "metadata": { "rename": { "Order no": "order_number" } }
}
Terminal window
node make-xlsx.mjs
node to-dimensionlabs.mjs --config xlsx.json --input chats.xlsx --out out/xlsx
Rows in: 3  Messages out: 3  Sessions: 2  Dropped: 0
  fixed 3: excel serial dates converted
  fixed 1: blank userId filled from sessionId
  fixed 1: replies given the customer's userId
  fixed 1: messages moved to be 1000 ms apart
  note: read sheet "Chats"
  wrote out/xlsx-1.csv (3 rows)
  report: out/xlsx.report.json
out/xlsx-1.csv
sessionId,userId,incoming,text,dimensionlabs_timestamp,conversationId,order_number
880011223344,v-1,true,Can I change my pickup store?,1767453120000,880011223344,4400123456789
880011223344,v-1,false,"Sure, which store?",1767453121000,880011223344,4400123456789
880011223345,v-2,true,Gift card balance?,1767549600000,880011223345,

The order number in the last row was a #N/A cell and is left empty. The barista’s reply had no id and now carries the visitor’s, v-1.

Transcripts

Section titled “Transcripts”
calls.csv
call_id,started_at,store,transcript
call-501,2026-02-10T15:00:00Z,Austin Eastside,"Agent: Thanks for calling Lumen Coffee, this is Sam.
Customer: Hi, my mobile order never showed up.
It was a cold brew and a scone.
Agent: I'm sorry. Let me look that up.
Customer: Thanks."
call-502,2026-02-11T16:30:00Z,Portland Pearl,"Customer: Do you roast on site?
Agent: We do, every Tuesday."
calls.json
{
  "shape": "transcript",
  "columns": { "sessionId": "call_id", "timestamp": "started_at" },
  "transcript": { "column": "transcript", "speakerPattern": "^(Agent|Customer):\\s*", "incomingSpeakers": ["Customer"], "secondsApart": 5 },
  "constants": { "channel": "phone" }
}
Terminal window
node to-dimensionlabs.mjs --config calls.json --input calls.csv --out out/calls

Transcript turns all carry the row’s userId, or the session id when there is none, so both sides already share one id.

out/calls-1.csv
sessionId,userId,incoming,text,dimensionlabs_timestamp,conversationId,store,channel,speaker
call-501,call-501,false,"Thanks for calling Lumen Coffee, this is Sam.",1770735600000,call-501,Austin Eastside,phone,Agent
call-501,call-501,true,"Hi, my mobile order never showed up. It was a cold brew and a scone.",1770735605000,call-501,Austin Eastside,phone,Customer
call-501,call-501,false,I'm sorry. Let me look that up.,1770735610000,call-501,Austin Eastside,phone,Agent
call-501,call-501,true,Thanks.,1770735615000,call-501,Austin Eastside,phone,Customer
call-502,call-502,true,Do you roast on site?,1770827400000,call-502,Portland Pearl,phone,Customer
call-502,call-502,false,"We do, every Tuesday.",1770827405000,call-502,Portland Pearl,phone,Agent

A wide survey with a row of question wording

Section titled “A wide survey with a row of question wording”

The export puts the question wording in row 2, under the column names. questionsFromRow takes the question text from that row, and skipRowsAfterHeader keeps it from being read as a respondent.

survey.csv
response_id,submitted_at,nps,why_score,improve
Response ID,Submitted,How likely are you to recommend Lumen Coffee?,Why did you give that score?,What one thing should we improve?
R-1,2026-03-01 08:15,9,Friendly baristas.,Faster mobile pickup.
R-2,2026-03-01 09:40,4,,More oat milk options.
R-3,2026-03-02 18:05,7,,
survey.json
{
  "input": { "skipRowsAfterHeader": 1 },
  "shape": "survey",
  "columns": { "sessionId": "response_id", "timestamp": "submitted_at" },
  "timestamp": { "timeZone": "America/New_York" },
  "survey": { "questionsFromRow": 2, "questions": ["why_score", "improve"] }
}
Terminal window
node to-dimensionlabs.mjs --config survey.json --input survey.csv --out out/survey
Rows in: 3  Messages out: 6  Sessions: 2  Dropped: 3
  dropped 3: blank survey answer (question and answer skipped)
  fixed 6: blank userId filled from sessionId
  note: question wording read from row 2
  note: some respondents answered none of the listed questions and produce no rows
  wrote out/survey-1.csv (6 rows)
  report: out/survey.report.json
out/survey-1.csv
sessionId,userId,incoming,text,dimensionlabs_timestamp,conversationId,nps,question
R-1,R-1,false,Why did you give that score?,1772370900000,R-1,9,why_score
R-1,R-1,true,Friendly baristas.,1772370901000,R-1,9,why_score
R-1,R-1,false,What one thing should we improve?,1772370902000,R-1,9,improve
R-1,R-1,true,Faster mobile pickup.,1772370903000,R-1,9,improve
R-2,R-2,false,What one thing should we improve?,1772376000000,R-2,4,improve
R-2,R-2,true,More oat milk options.,1772376001000,R-2,4,improve

A Qualtrics CSV has two rows under the column names: the question wording, then a row of {"ImportId":...} values. Use "skipRowsAfterHeader": 2 and "questionsFromRow": 2.

Nested JSONL and split output

Section titled “Nested JSONL and split output”
messages.jsonl (the last line is deliberately broken)
{"ticket":{"id":"T-1"},"sender":{"type":"customer","id":"u-301"},"body":"Where is my bag of Night Owl?","created":"2026-04-01T10:00:00Z","tags":["shipping"]}
{"ticket":{"id":"T-1"},"sender":{"type":"agent","id":"a-12"},"body":"It ships today.","created":"2026-04-01T10:05:00Z","tags":["shipping"]}
{"ticket":{"id":"T-2"},"sender":{"type":"customer","id":"u-302"},"body":"Cancel my subscription","created":"2026-04-02T11:00:00Z","tags":[]}
{"ticket":{"id":"T-3"},"sender":{"type":"customer","id":"u-303"},"body":"Thanks!","created":"2026-04-03T11:00:00Z","tags":[]}
{"ticket":{"id":"T-3"},"sender":{"type":"agent","id":"a-12"},"body":"Anytime.","created":"2026-04-03T11:01:00Z","tags":[]}
not json
jsonl.json
{
  "columns": { "text": "body", "userId": "sender.id", "sessionId": "ticket.id", "timestamp": "created" },
  "incoming": { "column": "sender.type", "true": ["customer"], "false": ["agent"] },
  "metadata": { "keep": ["tags"] },
  "output": { "format": "csv", "chunkRows": 2 }
}
Terminal window
node to-dimensionlabs.mjs --config jsonl.json --input messages.jsonl --out out/nested
Rows in: 6  Messages out: 5  Sessions: 3  Dropped: 1
  dropped 1: line 6 is not valid JSON
  fixed 2: replies given the customer's userId
  wrote out/nested-1.csv (2 rows)
  wrote out/nested-2.csv (1 row)
  wrote out/nested-3.csv (2 rows)
  report: out/nested.report.json

With chunkRows set to 2, ticket T-1 (2 rows), T-2 (1 row) and T-3 (2 rows) each land in their own part: no part can take the next ticket without going over 2 rows.

Read the report

Section titled “Read the report”

Every run writes <out>.report.json:

out/lumen-tickets.report.json
{
  "input": "lumen-tickets.csv",
  "rowsIn": 3,
  "messagesOut": 2,
  "sessions": 1,
  "dropped": {
    "skipped: author_type is system": 1
  },
  "fixed": {
    "html removed from text": 1,
    "line breaks in text replaced with spaces": 1,
    "replies given the customer's userId": 1
  },
  "notes": [],
  "files": [
    {
      "file": "out/lumen-tickets-1.csv",
      "rows": 2
    }
  ],
  "firstTimestamp": "2026-01-03T17:12:00.000Z",
  "lastTimestamp": "2026-01-03T17:20:00.000Z"
}
  • rowsIn and messagesOut should make sense together. For messages, out is in minus dropped. Transcripts and surveys write several rows per input row.
  • dropped lists every reason a row (or a survey answer) was left out. An unexpected incoming value not recognized means a value is missing from your lists.
  • fixed counts each fix. Large numbers are fine when you expect them (seconds converted to milliseconds on every row of a file that uses seconds); a surprise is worth a look.
  • notes flags things to check, such as times read as UTC because no time zone was set, or kept columns that look like personal data (add those to metadata.drop; see PII redaction).
  • firstTimestamp and lastTimestamp should match the date range of your export.

Then run the file checker on the output and go through Before you send it.

Monthly exports

Section titled “Monthly exports”

Keep one mapping file per export type next to your data, under a name such as helpdesk-tickets.mapping.json. Next month, run the same command on the new file and upload it to the same source. If the new export overlaps last month’s, that’s fine: Dimension Labs removes duplicate messages when it ingests them. If the export’s columns change, the run tells you: a missing column shows up as dropped rows (no sessionId, empty text) instead of quietly shifted data.

What it doesn’t do

Section titled “What it doesn’t do”