# GitHub Issues and Discussions

> Send GitHub issues and their comments to Dimension Labs with the universal API.

This brings in each issue in a GitHub repository as one conversation: the issue's title and body, then its comments, from your community and from your maintainers. It's built for a public or customer-facing repository where people file bugs and requests. You run it on your side: a small script pulls from the GitHub REST API and posts each message to the [universal API](/api/tracker). Discussions are only in GitHub's GraphQL API: see [Discussions](#discussions) below.

## What you need

- A GitHub account that can read the repository's issues.
- A fine-grained personal access token with the **Issues** repository permission set to read. For a public repository the API works without a token, but with much lower limits.
- A Dimension Labs API key for a new **Universal API** source. See [Your API key](/api/tracker#your-api-key).

## 1. Get API access in GitHub

1. In the upper-right corner of any page on GitHub, click your profile picture, then **Settings**.
2. In the left sidebar, click **Developer settings**, then under **Personal access tokens** click **Fine-grained tokens**.
3. Click **Generate new token**. Enter a name, pick an expiration and the resource owner.
4. Under **Repository access**, choose the repository. Under **Permissions**, set the **Issues** repository permission to read. Generate the token and copy it ([managing your personal access tokens](https://docs.github.com/en/authentication/keeping-your-account-and-data-secure/managing-your-personal-access-tokens)).
5. Send it as `Authorization: Bearer YOUR_TOKEN`.

## 2. Pull new issues and comments

[List repository issues](https://docs.github.com/en/rest/issues/issues) updated since your cursor, and [list issue comments for the repository](https://docs.github.com/en/rest/issues/comments) the same way:

```bash
curl -H "Authorization: Bearer $GITHUB_TOKEN" -H "Accept: application/vnd.github.text+json" \
  "https://api.github.com/repos/OWNER/REPO/issues?state=all&sort=updated&direction=asc&since=2026-10-01T00:00:00Z&per_page=100"
curl -H "Authorization: Bearer $GITHUB_TOKEN" -H "Accept: application/vnd.github.text+json" \
  "https://api.github.com/repos/OWNER/REPO/issues/comments?sort=updated&direction=asc&since=2026-10-01T00:00:00Z&per_page=100"
```

- `since` (ISO 8601, `YYYY-MM-DDTHH:MM:SSZ`) returns only what was updated after that time. `state=all` includes closed issues; the default is open only.
- `per_page` goes up to 100. Follow the `rel="next"` URL in the `link` header for the next page ([pagination](https://docs.github.com/en/rest/using-the-rest-api/using-pagination-in-the-rest-api)).
- The `application/vnd.github.text+json` media type adds `body_text`, a plain-text version of the Markdown body.
- GitHub counts every pull request as an issue. The issues list marks them with a `pull_request` key, and the comments list includes pull request comments too: the code skips both.

Rate limits, per GitHub: 5,000 requests per hour with a personal access token, 60 without one. Going over returns `403` or `429`. Wait for `retry-after` seconds if it's there, or until `x-ratelimit-reset` (UTC epoch seconds) when `x-ratelimit-remaining` is `0` ([rate limits](https://docs.github.com/en/rest/using-the-rest-api/rate-limits-for-the-rest-api)).

### Discussions

GitHub Discussions aren't in the REST API. They're in the [GraphQL API](https://docs.github.com/en/graphql/guides/using-the-graphql-api-for-discussions), at `POST https://api.github.com/graphql`. This query reads the most recently updated discussions with their comments and replies:

```graphql
query ($owner: String!, $name: String!, $after: String) {
  repository(owner: $owner, name: $name) {
    discussions(first: 25, after: $after, orderBy: { field: UPDATED_AT, direction: DESC }) {
      pageInfo { hasNextPage endCursor }
      nodes {
        number title bodyText createdAt updatedAt authorAssociation
        comments(first: 50) {
          nodes {
            bodyText createdAt authorAssociation
            replies(first: 50) { nodes { bodyText createdAt authorAssociation } }
          }
        }
      }
    }
  }
}
```

Stop paging once `updatedAt` is older than your cursor. Map each discussion the same way as an issue: one session per discussion, the post and every comment and reply as a message, with `authorAssociation` deciding direction. The code below covers issues only. For private repositories, GitHub's guide says access tokens need the `repo` scope.

## 3. Map the fields

| Dimension Labs | GitHub | Notes |
|---|---|---|
| `text` | issue `title` and `body_text`; comment `body_text` | Falls back to `body` (Markdown) if `body_text` is missing. Skip empty bodies. |
| `userId` | `user.id` | GitHub's numeric user id, never the `login`. On your maintainers' replies, use the issue author's id. |
| `incoming` | `author_association`, `user.type` | `OWNER`, `MEMBER` and `COLLABORATOR` are your side, and so is any author whose `user.type` is `Bot`. Everyone else (`CONTRIBUTOR`, `FIRST_TIME_CONTRIBUTOR`, `NONE` and so on) is `incoming: true`. |
| `sessionId` | issue `id` | Prefix it, for example `github-1296269`. Comments find their issue through `issue_url`. |
| `conversationId` | issue `id` | Same value as `sessionId`. |
| `dimensionlabs_timestamp` | `created_at` | ISO 8601. Convert with `toMillis`. |
| `platformJson` | repository, issue `number`, `state`, `labels`, `author_association`; comment `id` | Labels are the quickest way to filter bugs from feature requests. |

**Personal data.** Don't send `login`, names or emails. Issues often carry log output, stack traces and config snippets with emails, tokens or hostnames: redact them before you send. See [PII redaction](/data/pii-redaction).

## 4. Send it

```js
// github.mjs. Environment: GITHUB_TOKEN, GITHUB_REPO (owner/name), DL_API_KEY.


const HOUR = 60 * 60 * 1000;
const sleep = (ms) => new Promise((r) => setTimeout(r, ms));
const base = `https://api.github.com/repos/${process.env.GITHUB_REPO}`;
// The text media type adds body_text: the Markdown body as plain text.
const headers = { Authorization: `Bearer ${process.env.GITHUB_TOKEN}`, Accept: 'application/vnd.github.text+json' };
const TEAM = new Set(['OWNER', 'MEMBER', 'COLLABORATOR']);

// Waits out GitHub's rate limits (403 or 429), using Retry-After or x-ratelimit-reset, and retries 5xx.
async function get(url) {
  for (let attempt = 0; ; attempt++) {
    const res = await fetch(url, { headers });
    if (res.ok) return { data: await res.json(), next: res.headers.get('link')?.match(/<([^>]+)>;\s*rel="next"/)?.[1] ?? null };
    const limited = res.status === 429 || (res.status === 403 && (res.headers.has('retry-after') || res.headers.get('x-ratelimit-remaining') === '0'));
    if ((!limited && res.status < 500) || attempt === 5) throw new Error(`GitHub returned ${res.status}: ${await res.text()}`);
    const after = Number(res.headers.get('retry-after'));
    const reset = Number(res.headers.get('x-ratelimit-reset')) * 1000 - Date.now();
    await sleep(after > 0 ? after * 1000 : reset > 0 ? reset + 1000 : 60000 * 2 ** Math.min(attempt, 2));
  }
}

async function list(url) {
  const items = [];
  for (let next = url; next; ) {
    const page = await get(next);
    items.push(...page.data);
    next = page.next;
  }
  return items;
}

// Sends every issue and issue comment written at or after `since` (Unix ms). Pull requests are skipped.
export async function syncGitHub({ since, historical = false }) {
  let newest = since;
  const iso = new Date(since).toISOString().replace(/\.\d{3}Z$/, 'Z');
  const issues = new Map();
  for (const i of await list(`${base}/issues?state=all&sort=updated&direction=asc&since=${iso}&per_page=100`)) {
    if (!i.pull_request) issues.set(i.number, i);
  }
  const messages = [...issues.values()].map((i) => ({ issue: i, by: i, text: [i.title, i.body_text ?? i.body].filter(Boolean).join('\n\n') }));
  for (const c of await list(`${base}/issues/comments?sort=updated&direction=asc&since=${iso}&per_page=100`)) {
    const issue = issues.get(Number(c.issue_url.split('/').pop()));
    if (issue) messages.push({ issue, by: c, text: c.body_text ?? c.body, comment_id: c.id }); // no issue: a pull request comment
  }
  for (const { issue, by, text, comment_id } of messages) {
    const ts = toMillis(by.created_at);
    if (ts < since || !text?.trim()) continue;
    const sessionId = `github-${issue.id}`;
    const incoming = !TEAM.has(by.author_association) && by.user?.type !== 'Bot';
    await sendToDimensionLabs({
      text,
      userId: String((incoming ? by.user?.id : issue.user?.id) ?? sessionId),
      incoming,
      sessionId,
      conversationId: sessionId,
      dimensionlabs_timestamp: ts,
      platformJson: { repo: process.env.GITHUB_REPO, issue_number: issue.number, state: issue.state, labels: issue.labels?.map((l) => (typeof l === 'string' ? l : l.name)), author_association: by.author_association, comment_id },
    }, { historical: historical || ts < Date.now() - HOUR });
    newest = Math.max(newest, ts);
  }
  return newest;
}
```

## 5. Run it

Save the code as `github.mjs` in a project set up as in [Set up your project](/api/tracker#set-up-your-project), and run it with the [scheduled sync](/api/scheduled-sync) runner: `import { syncGitHub as sync } from './github.mjs';`.

- **History first:** `node run.mjs 2025-01-01` sends every issue and comment written since that date to `/trackhistorical`. Then start a [backfill](/dimensions/backfill) from the **Backfill** tab.
- **Then every 15 minutes:** the scheduled job sends new issues and comments to `/track`, and they are enriched automatically every day. A message that is more than an hour old when the job sees it goes to `/trackhistorical`, and needs a backfill.
- **Re-runs are safe:** duplicates are removed automatically.

Issues and comments come from two repository-wide lists, so a run costs one call per 100 items, well inside GitHub's hourly limit. The code waits whenever GitHub asks it to. For an older archive, you can also [upload a file](/data/upload-a-file) of issues (check it with the [file checker](/data/check-a-file)), then run a backfill.

## GitHub reference

- [REST API endpoints for issues](https://docs.github.com/en/rest/issues/issues)
- [REST API endpoints for issue comments](https://docs.github.com/en/rest/issues/comments)
- [Using pagination in the REST API](https://docs.github.com/en/rest/using-the-rest-api/using-pagination-in-the-rest-api)
- [Rate limits for the REST API](https://docs.github.com/en/rest/using-the-rest-api/rate-limits-for-the-rest-api)
- [Managing your personal access tokens](https://docs.github.com/en/authentication/keeping-your-account-and-data-secure/managing-your-personal-access-tokens)
- [Using the GraphQL API for Discussions](https://docs.github.com/en/graphql/guides/using-the-graphql-api-for-discussions)
- [Forming calls with GraphQL](https://docs.github.com/en/graphql/guides/forming-calls-with-graphql)

Checked against GitHub's documentation on 2026-10-26.