text: 
  
  TITLE
  Sentencepiece dependency causing docker build to fail
  
  COMMENTS
  5
  
  REACTIONS
  +1: 0
  -1: 0
  laugh: 0
  hooray: 0
  heart: 0
  rocket: 0
  eyes: 0
  
  BODY
  ## Environment info
  <!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
       Don't forget to fill out the missing fields in that output! -->
       
  - `transformers` version:3.4.0
  - Platform:Ubuntu 18.04
  - Python version: 3.7
  - PyTorch version (GPU?): 1.7.0 no gpu
  - Tensorflow version (GPU?): N/A
  - Using GPU in script?: no
  - Using distributed or parallel set-up in script?: no
  
  
  ## Information
  
  Question answering pipelines is the feature I am using. There is a problem with the sentencepiece dependency of transformers. It cannot find the correct package when installing and causing the build process to fail
  
  The problem arises when using:
  Downloading the transformers library onto a docker container running ubuntu.
  
  The tasks I am working on is:
  Uploadding a transformers script to AWS fargate
  
  ## To reproduce
  
  Steps to reproduce the behavior:
  
  1. Create a project
  2. Try to create docker container using dockerfile attached below
  
  I have attached the relevant parts of my dockerfile below
  Dockerfile
  
  ```
  FROM ubuntu:18.04
  
  RUN mkdir /usr/app
  WORKDIR /usr/app
  
  # Add and install Python modules
  COPY requirements.txt ./
  RUN apt-get update
  RUN apt-get -y install python3
  RUN apt-get -y install python3-pip
  RUN pip3 install virtualenv
  ENV VIRTUAL_ENV=/venv
  RUN virtualenv venv -p python3
  ENV PATH="VIRTUAL_ENV/bin:$PATH"
  
  RUN pip3 install transformers[torch]
  RUN pip3 install torch==1.7.0+cpu torchvision==0.8.1+cpu torchaudio==0.7.0 -f https://download.pytorch.org/whl/torch_stable.html
  RUN pip3 install -r requirements.txt
  
  # Bundle app source
  COPY . ./
  # Expose
  EXPOSE 6000
  # Run
  CMD ["python", "app.py"]
  
  ```
  This is the stack trace that comes back from running and trying to build using this dockerfile
  
  ```
  [+] Building 571.8s (14/17)
   => [internal] load .dockerignore                                                                                             0.0s
   => => transferring context: 34B                                                                                              0.0s
   => [internal] load build definition from Dockerfile                                                                          0.0s
   => => transferring dockerfile: 835B                                                                                          0.0s
   => [internal] load metadata for docker.io/library/ubuntu:18.04                                                               0.8s
   => [internal] load build context                                                                                             0.1s
   => => transferring context: 139.29kB                                                                                         0.1s
   => [1/13] FROM docker.io/library/ubuntu:18.04@sha256:646942475da61b4ce9cc5b3fadb42642ea90e5d0de46111458e100ff2c7031e6        0.0s
   => CACHED [2/13] RUN mkdir /usr/app                                                                                          0.0s
   => CACHED [3/13] WORKDIR /usr/app                                                                                            0.0s
   => [4/13] COPY requirements.txt ./                                                                                           0.0s
   => [5/13] RUN apt-get update                                                                                                30.8s
   => [6/13] RUN apt-get -y install python3                                                                                    15.6s
   => [7/13] RUN apt-get -y install python3-pip                                                                               214.0s
   => [8/13] RUN pip3 install virtualenv                                                                                        4.5s
   => [9/13] RUN virtualenv venv -p python3                                                                                     0.8s
   => ERROR [10/13] RUN pip3 install transformers[torch]                                                                      305.1s
  ------
   > [10/13] RUN pip3 install transformers[torch]:
  #14 1.095 Collecting transformers[torch]
  #14 1.410   Downloading https://files.pythonhosted.org/packages/2c/4e/4f1ede0fd7a36278844a277f8d53c21f88f37f3754abf76a5d6224f76d4a/
  transformers-3.4.0-py3-none-any.whl (1.3MB)
  #14 1.897 Collecting numpy (from transformers[torch])
  #14 2.507   Downloading https://files.pythonhosted.org/packages/8f/40/ddb5109614aabad67e6fe426b3579a879b7b3cdd375eb27af467c4367ae0/
  numpy-1.19.3-cp36-cp36m-manylinux1_x86_64.whl (13.4MB)
  #14 5.947 Requirement already satisfied: filelock in /usr/local/lib/python3.6/dist-packages (from transformers[torch])
  #14 5.953 Collecting sentencepiece!=0.1.92 (from transformers[torch])
  #14 6.174   Downloading https://files.pythonhosted.org/packages/72/e0/57edbab017a204e9f39448c1717292437a45b5f7cf3a9dbf4a9c026b03c5/
  sentencepiece-0.1.94.tar.gz (507kB)
  #14 6.575 Collecting sacremoses (from transformers[torch])
  #14 6.714   Downloading https://files.pythonhosted.org/packages/7d/34/09d19aff26edcc8eb2a01bed8e98f13a1537005d31e95233fd48216eed10/
  sacremoses-0.0.43.tar.gz (883kB)
  #14 7.158 Collecting requests (from transformers[torch])
  #14 7.341   Downloading https://files.pythonhosted.org/packages/45/1e/0c169c6a5381e241ba7404532c16a21d86ab872c9bed8bdcd4c423954103/
  requests-2.24.0-py2.py3-none-any.whl (61kB)
  #14 7.404 Collecting packaging (from transformers[torch])
  #14 7.551   Downloading https://files.pythonhosted.org/packages/46/19/c5ab91b1b05cfe63cccd5cfc971db9214c6dd6ced54e33c30d5af1d2bc43/
  packaging-20.4-py2.py3-none-any.whl
  #14 7.579 Collecting dataclasses; python_version < "3.7" (from transformers[torch])
  #14 7.708   Downloading https://files.pythonhosted.org/packages/e1/d2/6f02df2616fd4016075f60157c7a0452b38d8f7938ae94343911e0fb0b09/
  dataclasses-0.7-py3-none-any.whl
  #14 7.725 Collecting tokenizers==0.9.2 (from transformers[torch])
  #14 8.019   Downloading https://files.pythonhosted.org/packages/7c/a5/78be1a55b2ac8d6a956f0a211d372726e2b1dd2666bb537fea9b03abd62c/
  tokenizers-0.9.2-cp36-cp36m-manylinux1_x86_64.whl (2.9MB)
  #14 8.732 Collecting regex!=2019.12.17 (from transformers[torch])
  #14 9.570   Downloading https://files.pythonhosted.org/packages/87/9f/aad666560082cb11331167cbb31cf0e8bd90af8ea4951436d1fcb2ddde44/
  regex-2020.10.28-cp36-cp36m-manylinux1_x86_64.whl (666kB)
  #14 9.756 Collecting protobuf (from transformers[torch])
  #14 10.02   Downloading https://files.pythonhosted.org/packages/30/79/510974552cebff2ba04038544799450defe75e96ea5f1675dbf72cc8744f/
  protobuf-3.13.0-cp36-cp36m-manylinux1_x86_64.whl (1.3MB)
  #14 10.36 Collecting tqdm>=4.27 (from transformers[torch])
  #14 10.57   Downloading https://files.pythonhosted.org/packages/93/3a/96b3dc293aa72443cf9627444c3c221a7ba34bb622e4d8bf1b5d4f2d9d08/
  tqdm-4.51.0-py2.py3-none-any.whl (70kB)
  #14 10.60 Collecting torch>=1.0; extra == "torch" (from transformers[torch])
  #14 10.79   Downloading https://files.pythonhosted.org/packages/80/2a/58f8078744e0408619c63148f7a2e8e48cf007e4146b74d4bb67c56d161b/
  torch-1.7.0-cp36-cp36m-manylinux1_x86_64.whl (776.7MB)
  #14 285.6 Collecting click (from sacremoses->transformers[torch])
  #14 292.4   Downloading https://files.pythonhosted.org/packages/d2/3d/fa76db83bf75c4f8d338c2fd15c8d33fdd7ad23a9b5e57eb6c5de26b430e/
  click-7.1.2-py2.py3-none-any.whl (82kB)
  #14 292.4 Collecting joblib (from sacremoses->transformers[torch])
  #14 292.6   Downloading https://files.pythonhosted.org/packages/fc/c9/f58220ac44a1592f79a343caba12f6837f9e0c04c196176a3d66338e1ea8/
  joblib-0.17.0-py3-none-any.whl (301kB)
  #14 292.8 Requirement already satisfied: six in /usr/lib/python3/dist-packages (from sacremoses->transformers[torch])
  #14 292.8 Collecting urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 (from requests->transformers[torch])
  #14 293.0   Downloading https://files.pythonhosted.org/packages/56/aa/4ef5aa67a9a62505db124a5cb5262332d1d4153462eb8fd89c9fa41e5d92/
  urllib3-1.25.11-py2.py3-none-any.whl (127kB)
  #14 293.0 Collecting chardet<4,>=3.0.2 (from requests->transformers[torch])
  #14 293.2   Downloading https://files.pythonhosted.org/packages/bc/a9/01ffebfb562e4274b6487b4bb1ddec7ca55ec7510b22e4c51f14098443b8/
  chardet-3.0.4-py2.py3-none-any.whl (133kB)
  #14 293.2 Collecting certifi>=2017.4.17 (from requests->transformers[torch])
  #14 293.4   Downloading https://files.pythonhosted.org/packages/5e/c4/6c4fe722df5343c33226f0b4e0bb042e4dc13483228b4718baf286f86d87/
  certifi-2020.6.20-py2.py3-none-any.whl (156kB)
  #14 293.4 Requirement already satisfied: idna<3,>=2.5 in /usr/lib/python3/dist-packages (from requests->transformers[torch])
  #14 293.4 Collecting pyparsing>=2.0.2 (from packaging->transformers[torch])
  #14 293.7   Downloading https://files.pythonhosted.org/packages/8a/bb/488841f56197b13700afd5658fc279a2025a39e22449b7cf29864669b15d/
  pyparsing-2.4.7-py2.py3-none-any.whl (67kB)
  #14 293.7 Requirement already satisfied: setuptools in /usr/lib/python3/dist-packages (from protobuf->transformers[torch])
  #14 293.7 Collecting future (from torch>=1.0; extra == "torch"->transformers[torch])
  #14 293.9   Downloading https://files.pythonhosted.org/packages/45/0b/38b06fd9b92dc2b68d58b75f900e97884c45bedd2ff83203d933cf5851c9/
  future-0.18.2.tar.gz (829kB)
  #14 294.7 Collecting typing-extensions (from torch>=1.0; extra == "torch"->transformers[torch])
  #14 294.9   Downloading https://files.pythonhosted.org/packages/60/7a/e881b5abb54db0e6e671ab088d079c57ce54e8a01a3ca443f561ccadb37e/
  typing_extensions-3.7.4.3-py3-none-any.whl
  #14 294.9 Building wheels for collected packages: sentencepiece, sacremoses, future
  #14 294.9   Running setup.py bdist_wheel for sentencepiece: started
  #14 295.6   Running setup.py bdist_wheel for sentencepiece: finished with status 'error'
  #14 295.6   Complete output from command /usr/bin/python3 -u -c "import setuptools, tokenize;__file__='/tmp/pip-build-o837wqyj/sent
  encepiece/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __
  file__, 'exec'))" bdist_wheel -d /tmp/tmprwjlzrwrpip-wheel- --python-tag cp36:
  #14 295.6   /usr/lib/python3.6/distutils/dist.py:261: UserWarning: Unknown distribution option: 'long_description_content_type'
  #14 295.6     warnings.warn(msg)
  #14 295.6   running bdist_wheel
  #14 295.6   running build
  #14 295.6   running build_py
  #14 295.6   creating build
  #14 295.6   creating build/lib.linux-x86_64-3.6
  #14 295.6   creating build/lib.linux-x86_64-3.6/sentencepiece
  #14 295.6   copying src/sentencepiece/__init__.py -> build/lib.linux-x86_64-3.6/sentencepiece
  #14 295.6   copying src/sentencepiece/sentencepiece_model_pb2.py -> build/lib.linux-x86_64-3.6/sentencepiece
  #14 295.6   copying src/sentencepiece/sentencepiece_pb2.py -> build/lib.linux-x86_64-3.6/sentencepiece
  #14 295.6   running build_ext
  #14 295.6   /bin/sh: 1: pkg-config: not found
  #14 295.6   ./build_bundled.sh: 8: ./build_bundled.sh: git: not found
  #14 295.6   ./build_bundled.sh: 10: ./build_bundled.sh: git: not found
  #14 295.6   ./build_bundled.sh: 12: cd: can't cd to sentencepiece
  #14 295.6   ./build_bundled.sh: 15: ./build_bundled.sh: cmake: not found
  #14 295.6   make: *** No targets specified and no makefile found.  Stop.
  #14 295.6   make: *** No rule to make target 'install'.  Stop.
  #14 295.6   env: 'pkg-config': No such file or directory
  #14 295.6   Failed to find sentencepiece pkg-config
  #14 295.6
  #14 295.6   ----------------------------------------
  #14 295.6   Failed building wheel for sentencepiece
  #14 295.6   Running setup.py clean for sentencepiece
  #14 295.8   Running setup.py bdist_wheel for sacremoses: started
  #14 296.3   Running setup.py bdist_wheel for sacremoses: finished with status 'done'
  #14 296.3   Stored in directory: /root/.cache/pip/wheels/29/3c/fd/7ce5c3f0666dab31a50123635e6fb5e19ceb42ce38d4e58f45
  #14 296.4   Running setup.py bdist_wheel for future: started
  #14 297.2   Running setup.py bdist_wheel for future: finished with status 'done'
  #14 297.2   Stored in directory: /root/.cache/pip/wheels/8b/99/a0/81daf51dcd359a9377b110a8a886b3895921802d2fc1b2397e
  #14 297.3 Successfully built sacremoses future
  #14 297.3 Failed to build sentencepiece
  #14 297.3 Installing collected packages: numpy, sentencepiece, click, joblib, regex, tqdm, sacremoses, urllib3, chardet, certifi, r
  equests, pyparsing, packaging, dataclasses, tokenizers, protobuf, future, typing-extensions, torch, transformers
  #14 301.3   Running setup.py install for sentencepiece: started
  #14 301.7     Running setup.py install for sentencepiece: finished with status 'error'
  #14 301.7     Complete output from command /usr/bin/python3 -u -c "import setuptools, tokenize;__file__='/tmp/pip-build-o837wqyj/se
  ntencepiece/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code,
  __file__, 'exec'))" install --record /tmp/pip-7ji4iyud-record/install-record.txt --single-version-externally-managed --compile:
  #14 301.7     /usr/lib/python3.6/distutils/dist.py:261: UserWarning: Unknown distribution option: 'long_description_content_type'
  #14 301.7       warnings.warn(msg)
  #14 301.7     running install
  #14 301.7     running build
  #14 301.7     running build_py
  #14 301.7     creating build
  #14 301.7     creating build/lib.linux-x86_64-3.6
  #14 301.7     creating build/lib.linux-x86_64-3.6/sentencepiece
  #14 301.7     copying src/sentencepiece/__init__.py -> build/lib.linux-x86_64-3.6/sentencepiece
  #14 301.7     copying src/sentencepiece/sentencepiece_model_pb2.py -> build/lib.linux-x86_64-3.6/sentencepiece
  #14 301.7     copying src/sentencepiece/sentencepiece_pb2.py -> build/lib.linux-x86_64-3.6/sentencepiece
  #14 301.7     running build_ext
  #14 301.7     /bin/sh: 1: pkg-config: not found
  #14 301.7     mkdir: cannot create directory 'bundled': File exists
  #14 301.7     ./build_bundled.sh: 8: ./build_bundled.sh: git: not found
  #14 301.7     ./build_bundled.sh: 10: ./build_bundled.sh: git: not found
  #14 301.7     ./build_bundled.sh: 12: cd: can't cd to sentencepiece
  #14 301.7     mkdir: cannot create directory 'build': File exists
  #14 301.7     ./build_bundled.sh: 15: ./build_bundled.sh: cmake: not found
  #14 301.7     make: *** No targets specified and no makefile found.  Stop.
  #14 301.7     make: *** No rule to make target 'install'.  Stop.
  #14 301.7     env: 'pkg-config': No such file or directory
  #14 301.7     Failed to find sentencepiece pkg-config
  #14 301.7
  #14 301.7     ----------------------------------------
  #14 302.4 Command "/usr/bin/python3 -u -c "import setuptools, tokenize;__file__='/tmp/pip-build-o837wqyj/sentencepiece/setup.py';f=
  getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" inst
  all --record /tmp/pip-7ji4iyud-record/install-record.txt --single-version-externally-managed --compile" failed with error code 1 in
   /tmp/pip-build-o837wqyj/sentencepiece/
  ------
  failed to solve with frontend dockerfile.v0: failed to build LLB: executor failed running [/bin/sh -c pip3 install transformers[tor
  ch]]: runc did not terminate sucessfully
  
  ```
  ## Expected behavior
  
  I expect transformers to be downloaded and allow me to access it from the docker container.
  