rahul900day / gpt-3-encoder
GPT-3-Encoder is a PHP implementation of OpenAI's original python encoder/decoder.
Fund package maintenance!
Ko Fi
Installs: 17 992
Dependents: 0
Suggesters: 0
Security: 0
Stars: 2
Watchers: 2
Forks: 0
Open Issues: 2
Requires
- php: ^7.4 || ^8.0
- ext-mbstring: *
Requires (Dev)
- pestphp/pest: ^1.22
- phpstan/phpstan: ^1.10
- rector/rector: ^0.15.21
- symfony/var-dumper: ^5.4 || ^6.0
This package is auto-updated.
Last update: 2024-06-16 15:46:04 UTC
README
PHP BPE Encoder Decoder for GPT-2/GPT-3
About
GPT-2 and GPT-3 use byte pair encoding to turn text into a series of integers to feed into the model. This is a PHP implementation of OpenAI's original python encoder/decoder which can be found here.
Get Started
Requires PHP 7.4+
Install GPT-3-Encoder via the Composer package manager
composer require rahul900day/gpt-3-encoder
Usage
Encoding a text to tokens
use Rahul900day\Gpt3Encoder\Encoder; Encoder::encode("Your prompt.");
Decoding to text from tokens
use Rahul900day\Gpt3Encoder\Encoder; Encoder::decode([8582, 242, 98]);
Credits
This packages has some codes and test inspiration from node's gpt-3-encoder
License
This package is released under the MIT License.