NbAiLab/nb-nbnn-lid

Bokmål-Nynorsk Language Detection

A sample Fasttext model for Bokmål-Nynorsk detection.

Trained on NBNN Language Detection Dataset

This model will only work on Norwegian Bokmål vs Norwegian Nynorsk. It has an accuracy or roughly 97.50% on the dataset above. For more finegrained language detection, please user nb-nordic-lid.

Usage

The model can be used through the Fasttext library. Make sure you do pip install fasttextand pip install huggingface-hubfirst. The code below loads the binary directly from this repo:

from huggingface_hub import hf_hub_download
import fasttext

model_name = "nb-nbnn-lid.ftz"
model = fasttext.load_model(hf_hub_download("NbAiLab/nb-nbnn-lid", model_name))

model.predict("Debatt er bra og sunt for demokratier, og en forutsetning for politikkutvikling.")
# (('__label__nob',), array([0.97784388]))