CHARS_PER_TOKEN
@forge/monorepo / backend/src / CHARS_PER_TOKEN
Variable: CHARS_PER_TOKEN
constCHARS_PER_TOKEN:4=4
Defined in: backend/src/core/tokens.ts:20
Characters over four — the industry rule of thumb for English text with a BPE tokeniser.
@forge/monorepo / backend/src / CHARS_PER_TOKEN
constCHARS_PER_TOKEN:4=4
Defined in: backend/src/core/tokens.ts:20
Characters over four — the industry rule of thumb for English text with a BPE tokeniser.