Tokenizer | /tokenizers/mod.ts | netsaur@0.3.2-patch

Using Deno in production at your company? Earn free Deno merch.

Powerful machine learning, accelerated by WebGPU

Latest

method Tokenizer.prototype.decode

import { Tokenizer } from "https://deno.land/x/netsaur@0.3.2-patch/tokenizers/mod.ts";

decode(ids: Uint32Array, skipSpecialTokens?): string

Decode a sentence from its encoded tokens to a string

Parameters

ids: Uint32Array

tokens to decode

optional

skipSpecialTokens = [UNSUPPORTED]

string