metadata
title: Word Length
emoji: 🤗
colorFrom: green
colorTo: purple
sdk: gradio
sdk_version: 3.0.2
app_file: app.py
pinned: false
tags:
- evaluate
- measurement
Measurement Card for Word Length
Measurement Description
The word_length measurement returns the word count of the input string, based on tokenization using NLTK word_tokenize.
How to Use
This measurement requires a list of strings as input:
>>> data = ["hello world"]
>>> wordlength = evaluate.load("word_length", module_type="measurement")
>>> results = wordlength.compute(data=data)
Inputs
- data (list of
str): The input list of strings for which the word length is calculated. - tokenizer (
Callable) : approach used for tokenizingdata(optional). The default tokenizer is NLTK'sword_tokenize. This can be replaced by any function that takes a string as input and returns a list of tokens as output.
Output Values
- average_word_length(
float): the average number of words in the input string(s).
Output Example(s):
{"average_word_length": 245}
This metric outputs a dictionary containing the number of words in the input string (word length).
Examples
Example for a single string
>>> data = ["hello sun and goodbye moon"]
>>> wordlength = evaluate.load("word_length", module_type="measurement")
>>> results = wordlength.compute(data=data)
>>> print(results)
{'average_length': 5}
Example for a multiple strings
>>> data = ["hello sun and goodbye moon", "foo bar foo bar"]
>>> wordlength = evaluate.load("word_length", module_type="measurement")
>>> results = wordlength.compute(data=text)
{'average_length': 4.5}