Interface: TokenizerProfile

Defined in: provider-sdk/src/tokenizer.ts:32

Local tokenizer profile registered by core or a process-isolated provider plugin.

Properties

approximate

readonly approximate: boolean

Defined in: provider-sdk/src/tokenizer.ts:38

False only for a deterministic exact tokenizer implementation.

---

id

readonly id: string

Defined in: provider-sdk/src/tokenizer.ts:34

Stable diagnostic id, for example openai:o200k_base.

---

priority?

readonly optional priority?: number

Defined in: provider-sdk/src/tokenizer.ts:36

Higher-priority profiles win when multiple profiles match a model.

Methods

count()

count(text): number | Promise<number>

Defined in: provider-sdk/src/tokenizer.ts:40

Parameters

text

string

Returns

number | Promise<number>

---

matches()

matches(model): boolean | Promise<boolean>

Defined in: provider-sdk/src/tokenizer.ts:39

Parameters

model

string

Returns

boolean | Promise<boolean>