GitHub Issues and Discussions

This brings in each issue in a GitHub repository as one conversation: the issue’s title and body, then its comments, from your community and from your maintainers. It’s built for a public or customer-facing repository where people file bugs and requests. You run it on your side: a small script pulls from the GitHub REST API and posts each message to the universal API. Discussions are only in GitHub’s GraphQL API: see Discussions below.

What you need

• A GitHub account that can read the repository’s issues.
• A fine-grained personal access token with the Issues repository permission set to read. For a public repository the API works without a token, but with much lower limits.
• A Dimension Labs API key for a new Universal API source. See Your API key.

1. Get API access in GitHub

1. In the upper-right corner of any page on GitHub, click your profile picture, then Settings.
2. In the left sidebar, click Developer settings, then under Personal access tokens click Fine-grained tokens.
3. Click Generate new token. Enter a name, pick an expiration and the resource owner.
4. Under Repository access, choose the repository. Under Permissions, set the Issues repository permission to read. Generate the token and copy it (managing your personal access tokens).
5. Send it as Authorization: Bearer YOUR_TOKEN.

2. Pull new issues and comments

List repository issues updated since your cursor, and list issue comments for the repository the same way:

curl -H "Authorization: Bearer $GITHUB_TOKEN" -H "Accept: application/vnd.github.text+json" \
  "https://api.github.com/repos/OWNER/REPO/issues?state=all&sort=updated&direction=asc&since=2026-10-01T00:00:00Z&per_page=100"
curl -H "Authorization: Bearer $GITHUB_TOKEN" -H "Accept: application/vnd.github.text+json" \
  "https://api.github.com/repos/OWNER/REPO/issues/comments?sort=updated&direction=asc&since=2026-10-01T00:00:00Z&per_page=100"

• since (ISO 8601, YYYY-MM-DDTHH:MM:SSZ) returns only what was updated after that time. state=all includes closed issues; the default is open only.
• per_page goes up to 100. Follow the rel="next" URL in the link header for the next page (pagination).
• The application/vnd.github.text+json media type adds body_text, a plain-text version of the Markdown body.
• GitHub counts every pull request as an issue. The issues list marks them with a pull_request key, and the comments list includes pull request comments too: the code skips both.

Rate limits, per GitHub: 5,000 requests per hour with a personal access token, 60 without one. Going over returns 403 or 429. Wait for retry-after seconds if it’s there, or until x-ratelimit-reset (UTC epoch seconds) when x-ratelimit-remaining is 0 (rate limits).

Discussions

GitHub Discussions aren’t in the REST API. They’re in the GraphQL API, at POST https://api.github.com/graphql. This query reads the most recently updated discussions with their comments and replies:

query ($owner: String!, $name: String!, $after: String) {
  repository(owner: $owner, name: $name) {
    discussions(first: 25, after: $after, orderBy: { field: UPDATED_AT, direction: DESC }) {
      pageInfo { hasNextPage endCursor }
      nodes {
        number title bodyText createdAt updatedAt authorAssociation
        comments(first: 50) {
          nodes {
            bodyText createdAt authorAssociation
            replies(first: 50) { nodes { bodyText createdAt authorAssociation } }
          }
        }
      }
    }
  }
}

Stop paging once updatedAt is older than your cursor. Map each discussion the same way as an issue: one session per discussion, the post and every comment and reply as a message, with authorAssociation deciding direction. The code below covers issues only. For private repositories, GitHub’s guide says access tokens need the repo scope.

3. Map the fields

Dimension Labs	GitHub	Notes
text	issue title and body_text; comment body_text	Falls back to body (Markdown) if body_text is missing. Skip empty bodies.
userId	user.id	GitHub’s numeric user id, never the login. On your maintainers’ replies, use the issue author’s id.
incoming	author_association, user.type	OWNER, MEMBER and COLLABORATOR are your side, and so is any author whose user.type is Bot. Everyone else (CONTRIBUTOR, FIRST_TIME_CONTRIBUTOR, NONE and so on) is incoming: true.
sessionId	issue id	Prefix it, for example github-1296269. Comments find their issue through issue_url.
conversationId	issue id	Same value as sessionId.
dimensionlabs_timestamp	created_at	ISO 8601. Convert with toMillis.
platformJson	repository, issue number, state, labels, author_association; comment id	Labels are the quickest way to filter bugs from feature requests.

Personal data. Don’t send login, names or emails. Issues often carry log output, stack traces and config snippets with emails, tokens or hostnames: redact them before you send. See PII redaction.

4. Send it

// github.mjs. Environment: GITHUB_TOKEN, GITHUB_REPO (owner/name), DL_API_KEY.
import { sendToDimensionLabs, toMillis } from './dimensionlabs.mjs';

const HOUR = 60 * 60 * 1000;
const sleep = (ms) => new Promise((r) => setTimeout(r, ms));
const base = `https://api.github.com/repos/${process.env.GITHUB_REPO}`;
// The text media type adds body_text: the Markdown body as plain text.
const headers = { Authorization: `Bearer ${process.env.GITHUB_TOKEN}`, Accept: 'application/vnd.github.text+json' };
const TEAM = new Set(['OWNER', 'MEMBER', 'COLLABORATOR']);

// Waits out GitHub's rate limits (403 or 429), using Retry-After or x-ratelimit-reset, and retries 5xx.
async function get(url) {
  for (let attempt = 0; ; attempt++) {
    const res = await fetch(url, { headers });
    if (res.ok) return { data: await res.json(), next: res.headers.get('link')?.match(/<([^>]+)>;\s*rel="next"/)?.[1] ?? null };
    const limited = res.status === 429 || (res.status === 403 && (res.headers.has('retry-after') || res.headers.get('x-ratelimit-remaining') === '0'));
    if ((!limited && res.status < 500) || attempt === 5) throw new Error(`GitHub returned ${res.status}: ${await res.text()}`);
    const after = Number(res.headers.get('retry-after'));
    const reset = Number(res.headers.get('x-ratelimit-reset')) * 1000 - Date.now();
    await sleep(after > 0 ? after * 1000 : reset > 0 ? reset + 1000 : 60000 * 2 ** Math.min(attempt, 2));
  }
}

async function list(url) {
  const items = [];
  for (let next = url; next; ) {
    const page = await get(next);
    items.push(...page.data);
    next = page.next;
  }
  return items;
}

// Sends every issue and issue comment written at or after `since` (Unix ms). Pull requests are skipped.
export async function syncGitHub({ since, historical = false }) {
  let newest = since;
  const iso = new Date(since).toISOString().replace(/\.\d{3}Z$/, 'Z');
  const issues = new Map();
  for (const i of await list(`${base}/issues?state=all&sort=updated&direction=asc&since=${iso}&per_page=100`)) {
    if (!i.pull_request) issues.set(i.number, i);
  }
  const messages = [...issues.values()].map((i) => ({ issue: i, by: i, text: [i.title, i.body_text ?? i.body].filter(Boolean).join('\n\n') }));
  for (const c of await list(`${base}/issues/comments?sort=updated&direction=asc&since=${iso}&per_page=100`)) {
    const issue = issues.get(Number(c.issue_url.split('/').pop()));
    if (issue) messages.push({ issue, by: c, text: c.body_text ?? c.body, comment_id: c.id }); // no issue: a pull request comment
  }
  for (const { issue, by, text, comment_id } of messages) {
    const ts = toMillis(by.created_at);
    if (ts < since || !text?.trim()) continue;
    const sessionId = `github-${issue.id}`;
    const incoming = !TEAM.has(by.author_association) && by.user?.type !== 'Bot';
    await sendToDimensionLabs({
      text,
      userId: String((incoming ? by.user?.id : issue.user?.id) ?? sessionId),
      incoming,
      sessionId,
      conversationId: sessionId,
      dimensionlabs_timestamp: ts,
      platformJson: { repo: process.env.GITHUB_REPO, issue_number: issue.number, state: issue.state, labels: issue.labels?.map((l) => (typeof l === 'string' ? l : l.name)), author_association: by.author_association, comment_id },
    }, { historical: historical || ts < Date.now() - HOUR });
    newest = Math.max(newest, ts);
  }
  return newest;
}

5. Run it

Save the code as github.mjs in a project set up as in Set up your project, and run it with the scheduled sync runner: import { syncGitHub as sync } from './github.mjs';.

• History first: node run.mjs 2025-01-01 sends every issue and comment written since that date to /trackhistorical. Then start a backfill from the Backfill tab.
• Then every 15 minutes: the scheduled job sends new issues and comments to /track, and they are enriched automatically every day. A message that is more than an hour old when the job sees it goes to /trackhistorical, and needs a backfill.
• Re-runs are safe: duplicates are removed automatically.

Issues and comments come from two repository-wide lists, so a run costs one call per 100 items, well inside GitHub’s hourly limit. The code waits whenever GitHub asks it to. For an older archive, you can also upload a file of issues (check it with the file checker), then run a backfill.

GitHub reference

• REST API endpoints for issues
• REST API endpoints for issue comments
• Using pagination in the REST API
• Rate limits for the REST API
• Managing your personal access tokens
• Using the GraphQL API for Discussions
• Forming calls with GraphQL

Checked against GitHub’s documentation on 2026-10-26.