gpt-tokenizer

a fast and user-friendly runtime for transformer inference (Bert, Albert, GPT2, Decoders, etc) on CPU and GPU.

Unsupervised text tokenizer focused on computational efficiency

Platform-Agnostic Security Tokens implementation in GO (Golang)

PyTorch CTC Decoder bindings

Connectionist Temporal Classification (CTC) decoder with dictionary and language model.

Transformer implemented in Keras

Fast and customizable text tokenization library with BPE and SentencePiece support

Ready-made tokenizer library for working with GPT and tiktoken

OysterKit is a framework that provides a native Swift scanning, lexical analysis, and parsing capabilities. In addition it provides a language that can be used to rapidly define the rules used by OysterKit called STLR

Logisim Italian Fork