Index documentation straight from a git repository. Contextator keeps a shallow, single-branch checkout on the server and fetches the branch tip at the start of every index run — or the moment someone pushes, if you add a webhook.
Works with any HTTPS git server: GitHub, GitLab, Bitbucket, Gitea, Forgejo, Codeberg and self-hosted instances.
Adding one
Add source → Git repository
| Field | Example | Notes |
|---|---|---|
| Name | api-docs |
The mount prefix. Immutable |
| Repository URL | https://github.com/org/repo.git |
HTTP(S) only, HTTPS strongly recommended — SSH remotes are not supported |
| Branch | main |
The single branch that is checked out |
| Subdirectory | docs |
Optional. Index only this folder of the repository |
| Access token | – | Only for private repositories. Stored encrypted, never shown again |
| Username | – | Only needed for a GitLab deploy token (gitlab+deploy-token-N) or a Bitbucket Cloud API token (your Bitbucket username or x-bitbucket-api-token-auth) |
| File types | .md, .mdx |
.txt, .html/.htm, .csv, .docx and .pdf can be selected too |
Once the source is saved, press Test on its row (or Test connection in its Edit dialog): it lists the remote’s refs without cloning and answers with the branch’s current commit, so a wrong URL, branch or token is obvious. It tests the source as saved, not the unsaved form: after changing the token or branch in Edit, save first, then test.
With a subdirectory set, a file at docs/guides/install.md in the repository is indexed as
api-docs/guides/install.md — the subdirectory itself does not appear in the path.
A git server on the internal network
The repository URL is typed by an editor and the server connects to it, so where a git source may connect is bounded. Every request the source makes — the ref list that Test reads, the clone, each update and every redirect — goes through the same address check as a Confluence source, run on the address the connection actually uses, after DNS:
- Always refused: loopback and link-local addresses (the cloud metadata address
169.254.169.254included), unspecified and multicast addresses, and the reserved240.0.0.0/4block. No setting lifts these. An IPv6 address that carries an IPv4 one (IPv4-mapped, translated, compatible and NAT64 forms) is judged as that IPv4 address. - Refused unless allowed: private addresses —
10.0.0.0/8,172.16.0.0/12,192.168.0.0/16,fc00::/7, the deprecated site-localfec0::/10,100.64.0.0/10and198.18.0.0/15.
At most 5 redirects are followed, and the access token is dropped when a redirect leads to another origin (scheme, host or port).
Upgrading to 0.3.1. Before 0.3.1 a git source could clone from any address. A git server on an internal address — a
self-hosted Gitea, for example — stops working after the upgrade until its host is listed in
GIT_ALLOWED_HOSTS (host names, comma-separated, no scheme, port or path), set in .env, and the instance is restarted:
GIT_ALLOWED_HOSTS=git.corp.example
A refusal names the rule on the source’s row and in the Test message, for example:
refused: `git.corp.example` resolves to a private address; list `git.corp.example` in GIT_ALLOWED_HOSTS to allow it
No request is sent in that case, so no token leaves. The list lifts only the private-address rule, and only
for git sources; SOURCE_ALLOWED_HOSTS and WEB_ALLOWED_HOSTS are separate lists and do not admit a host
here. On Helm the key is config.gitAllowedHosts (see Running on Kubernetes).
How large a response may be
Each response body the git client reads is limited, so a remote that answers with far more than expected cannot
exhaust the instance’s memory. The limit applies to the ref list and to the pack of a clone or update, each
on its own, not to a whole sync together. It is set by GIT_MAX_PACK_BYTES, an integer number of bytes:
- Default:
268435456(256 MiB). The lowest accepted value is1048576(1 MiB); a smaller one is a configuration error at start-up. - Memory: the git library holds a pack in memory and copies it twice, so a fetch can use about twice the limit (roughly 512 MiB at the default) on top of what the instance already uses. The limit caps the response, not the memory.
- When it is exceeded: the response is cut off, the sync fails, and the error on the source’s row names the setting:
git remote response is larger than GIT_MAX_PACK_BYTES (268435456 bytes); raise GIT_MAX_PACK_BYTES or point the source at a smaller repository
A large repository needs a higher value, and the memory limit of the container has to go up with it: keep it well
above twice the new value plus the instance’s own footprint — twice is a lower bound, because parsing the pack
adds to it. Set it in .env and restart the instance:
GIT_MAX_PACK_BYTES=536870912
On Helm the key is config.gitMaxPackBytes; leaving it null keeps the default (see
Running on Kubernetes). Available from 0.3.2.
Private repositories
Paste an access token into Access token. It is encrypted with SECRET_KEY (AES-256-GCM) before it
is stored, is never returned by the API and is never shown again — the dialog only says that a token
exists. To replace it, paste a new one; to remove it, tick Remove the stored token.
SECRET_KEYmust be set (32+ characters, e.g.openssl rand -hex 32) before a source can store a token. See Configuration.
The username sent with the token depends on the provider and is detected from the URL:
| Provider | Username used | Token types |
|---|---|---|
| GitHub | x-access-token |
Classic PAT, fine-grained PAT, App installation token |
| GitLab | oauth2 |
OAuth tokens, personal and project access tokens |
| Bitbucket Cloud | x-token-auth |
Repository and workspace access tokens |
| Bitbucket Cloud (API token) | your Bitbucket username, or x-bitbucket-api-token-auth |
Put it in the Username field — the default x-token-auth is for repository/workspace access tokens only |
| Gitea / Forgejo / Codeberg / other | token, or whatever you type in Username |
Never paste credentials into the URL itself (https://user:token@host/…). They are not used to authenticate — a private repository with only a URL token fails to clone, as it would with no token at all. Since 0.3.1 the user:token@ part is also removed from the URL before it is stored, and from the source view and exports; migration 0020 cleans up rows stored earlier. Enter the token in the token field; that is the only place it is read from. A token that was ever pasted into a URL may still sit in a shell history or a log, so revoke it and issue a new one.
Recommended token scopes
Read access to the repository’s contents is enough — the narrowest credential each provider offers:
| Provider | Narrowest token | Username |
|---|---|---|
| GitHub | A fine-grained PAT limited to the repository with Contents: read, or a GitHub App installation token | leave empty (x-access-token) |
| GitLab | A project deploy token with read_repository |
the token’s generated username, e.g. gitlab+deploy-token-42 — the default oauth2 is refused for a deploy token |
| Bitbucket Cloud | A repository access token with Repositories: read | leave empty (x-token-auth) |
| Gitea / Forgejo | An access token with read scope on the repository | your username, or leave empty (token) |
A GitHub wiki
A GitHub wiki is a git repository of its own, so it can be a source. Its clone URL is the repository’s with
.wiki added:
https://github.com/YOUR-USERNAME/YOUR-REPOSITORY.wiki.git
Put that in Repository URL. GitHub creates the wiki repository when the first page is made, so the source has nothing to read before then. Set Branch to the wiki repository’s default branch — check it there rather than assume it. A private repository’s wiki is visible only to people with access to the repository, so the same Access token applies; a public one needs none.
Reaching a repository only available over SSH
Git is read over HTTP(S) only. SSH remotes (ssh://…, git@host:path) are not accepted and no SSH
key can be stored. For a repository reachable only over SSH, mirror it yourself: clone or
mirror it on the host, inside ALLOWED_DOC_ROOTS, keep it current on your own schedule (a cron job
running git pull, or your CI), and add that directory as a
Local Directory Source instead. It is then a folder like any other —
no push webhook, no branch or subdirectory setting, no Test connection — and exactly as fresh as your
schedule keeps it.
How syncing works
At the start of every index run:
- If there is no checkout yet, clone it: shallow (
depth=1), single branch, no tags. - Otherwise fetch the branch tip. If it moved, point the local branch at it and check out.
- If anything goes wrong, fall back to a fresh clone.
- Verify the subdirectory still exists on that branch.
The commit currently checked out is shown on the source row (main @ a1b2c3d). Checkouts live under
DATA_DIR, so they survive restarts and are deleted with the source.
Automatic re-indexing on push
Every git source gets its own webhook URL and secret, shown while editing the source:
POST http://<your-host>/api/webhooks/git/<source-id>
Add it as a push webhook in the repository settings with that secret, and every push to the branch queues a re-index. Full instructions and provider-by-provider screenshots of the fields: Push Webhooks.
Common problems
| Symptom | Cause and fix |
|---|---|
| The operation was canceled. on Test, or on the source’s row after a sync, when a token is set, or HTTP Error: 401 … when none is | The git host answered 401: the token is missing, rejected, expired, or lacking read access. Leave Username empty for a Bitbucket repository or workspace access token (the default x-token-auth is for those only); a Bitbucket Cloud API token needs your Bitbucket username or x-bitbucket-api-token-auth, and a GitLab deploy token its generated gitlab+deploy-token-N username |
| Connected, but branch “…” was not found on the remote (on Test) | The branch name is wrong, or the default branch is master rather than main |
| Subdirectory “…” does not exist in the repository | The path is relative to the repository root and is checked against the branch you configured |
| Documents disappeared after a failed sync | They do not — a source that cannot be read keeps its documents. Fix the cause and press Sync |
| git remote response is larger than GIT_MAX_PACK_BYTES … | The repository’s pack is over the response limit. Raise GIT_MAX_PACK_BYTES together with the container’s memory limit — see How large a response may be |
| SSH URL rejected | Only HTTP(S) is supported. Use the HTTPS URL with a token |
Notes and limits
- Only one branch per source. To index two branches, add two sources with different names.
- Very large repositories clone more slowly than the native
gitbinary would;depth=1, a single branch and a subdirectory keep it comfortable for documentation. - Submodules are not fetched.
- The repository is never written to; Contextator only ever fetches.