Skip to main content
Deno 2 is finally here 🎉️
Learn more
Module

x/netsaur/tokenizers/mod.ts>Tokenizer#encode

Powerful machine learning, accelerated by WebGPU
Go to Latest
method Tokenizer.prototype.encode
import { Tokenizer } from "https://deno.land/x/netsaur@0.3.0/tokenizers/mod.ts";

Encode a sentence

Parameters

sentence: string

sentence to tokenize