English ↔ Roman Urdu
Loading the model
The weights download once and are then cached by the browser, so a second visit starts immediately.
How it works
Written out, not imported
Multi-head attention, sinusoidal positional encoding, residual connections, layer norm, the encoder and decoder stacks and the projection head are all written directly in PyTorch. No nn.Transformer, no AutoModel, no training framework.
The corpus was the hard part
No public English to Roman Urdu parallel corpus existed, so 1.81M sentence pairs were built for this project. Both languages share one 24k byte-level BPE vocabulary, because Roman Urdu is written in Latin script.
One model, both directions
A leading tag tells the model which language to emit, so a single set of weights translates either way and works out which one you want from what you typed. Arabic-script Urdu is romanised first.
Nothing you type leaves the tab
The model is exported to ONNX, quantised to int8 and decoded by onnxruntime-web. The weights stream from the Hugging Face CDN; the sentences never go anywhere. It also keeps working offline once cached.
Results
- 23.2
- BLEU en→ur
- 30.7
- BLEU ur→en
- 30
- epochs trained
- 1.81M
- sentence pairs
Validation loss improved at every one of the 30 epochs and early stopping never fired, so the model was still learning when it hit the cap. The published checkpoint is chosen on validation loss rather than on benchmark BLEU, because picking the epoch that happens to peak on the benchmark is how you quietly end up fitting it.