

TITLE
Allow to set truncation strategy for pipeline

COMMENTS
2

REACTIONS
+1: 1
-1: 0
laugh: 0
hooray: 0
heart: 0
rocket: 0
eyes: 0

BODY
# 🚀 Feature request

The highlevel pipeline function should allow to set the truncation strategy of the tokenizer in the pipeline.

## Motivation

Some models will crash if the input sequence has too many tokens and require truncation. Additionally available memory is limited and it is often useful to shorten the amount of tokens. Sadly this is currently not possible using the pipeline-API. One has to call the tokenizer manually to set the truncation strategy or hope that the task specific pipeline has truncation turned on by default (the summarization pipeline for example has not).

## Your contribution

I could potentially create a PR for this, but want to confirm first that the change is welcome.