remove old readme and cleanup old readme

README.md

@@ -17,6 +17,8 @@ pinned: true
 
 View the demo at huggingface spaces:
 
+[DEMO](https://huggingface.co/spaces/RugNlpFlashcards/Speech_Language_Processing_Jurafsky_Martin)
+
 
 ## Dependencies
 
@@ -46,6 +48,11 @@ Next, run the following command **from the root of the repository**:
 docker cp es01:/usr/share/elasticsearch/config/certs/http_ca.crt .
 ```
 
+
+Alternatively, if docker is not available or feasable. It is possible to use a trail hosted version of Elasticsearch at: 
+
+https://www.elastic.co/cloud/
+
 ## Running
 
 To make sure we're using the dependencies managed by Poetry, run `poetry shell`
@@ -101,5 +108,4 @@ options:
 ```
 
 
-
 Check out the configuration reference at https://huggingface.co/docs/hub/spaces#reference

README.old.md

@@ -1,93 +0,0 @@
-# nlp-flashcard-project
-
-## Todo 2
-
-- [ ] Contexts preprocessing
-  - [ ] Formules enzo eruit filteren
-  - [ ] Splitsen op zinnen...?
-- [ ] Meer language models proberen
-- [X] Elasticsearch
-- [X] CLI voor vragen beantwoorden
-
-### Extra dingen
-
-- [X] Huggingface spaces demo
-- [ ] Question generation voor finetuning
-- [ ] Language model finetunen
-
-## Todo voor progress meeting
-
-- [ ] Data inlezen/Repo klaarmaken
-- [ ] Proof of concept met UnifiedQA
-- [ ] Standaard QA model met de dataset
-- [ ] Papers verzamelen/lezen
-- [ ] Eerder werk bekijken, inspiratie opdoen voor research richting
-
-## Overview
-
-De meeste QA systemen bestaan uit twee onderdelen:
-
-- Een retriever. Die haalt adhv de vraag _k_ relevante stukken context op, bv.
-  met `tf-idf`.
-- Een model dat het antwoord genereert. Wat je hier precies gebruikt hangt af
-  van de manier van question answering:
-  - Voor **extractive QA** gebruik je een reader;
-  - Voor **generative QA** gebruik je een generator.
-
-  Beide werken op basis van een language model.
-
-## Handige info
-
-- Huggingface QA tutorial: <https://huggingface.co/docs/transformers/tasks/question_answering#finetune-with-tensorflow>
-- Overview van open-domain question answering technieken: <https://lilianweng.github.io/posts/2020-10-29-odqa/>
-
-## Base model
-
-Tot nu toe alleen een retriever die adhv een vraag de top-k relevante documents
-ophaalt. Haalt voor veel vragen wel hoge similarity scores, maar de documents
-die die ophaalt zijn meestal niet erg relevant.
-
-```bash
-poetry shell
-cd base_model
-poetry run python main.py
-```
-
-### Voorbeeld
-
-"What is the perplexity of a language model?"
-
-> Result 1 (score: 74.10):  
-> Figure 10 .17 A sample alignment between sentences in English and French, with
-> sentences extracted from Antoine de Saint-Exupery's Le Petit Prince and a
-> hypothetical translation. Sentence alignment takes sentences e 1 , ..., e n ,
-> and f 1 , ..., f n and finds minimal > sets of sentences that are translations
-> of each other, including single sentence mappings like (e 1 ,f 1 ), (e 4 -f 3
-> ), (e 5 -f 4 ), (e 6 -f 6 ) as well as 2-1 alignments (e 2 /e 3 ,f 2 ), (e 7
-> /e 8 -f 7 ), and null alignments (f 5 ).
->
-> Result 2 (score: 74.23):  
-> Character or word overlap-based metrics like chrF (or BLEU, or etc.) are
-> mainly used to compare two systems, with the goal of answering questions like:
-> did the new algorithm we just invented improve our MT system? To know if the
-> difference between the chrF scores of two > MT systems is a significant
-> difference, we use the paired bootstrap test, or the similar randomization
-> test.
->
-> Result 3 (score: 74.43):  
-> The model thus predicts the class negative for the test sentence.
->
-> Result 4 (score: 74.95):  
-> Translating from languages with extensive pro-drop, like Chinese or Japanese,
-> to non-pro-drop languages like English can be difficult since the model must
-> somehow identify each zero and recover who or what is being talked about in
-> order to insert the proper pronoun.
->
-> Result 5 (score: 76.22):  
-> Similarly, a recent challenge set, the WinoMT dataset (Stanovsky et al., 2019)
-> shows that MT systems perform worse when they are asked to translate sentences
-> that describe people with non-stereotypical gender roles, like "The doctor
-> asked the nurse to help her in the > operation".
-
-
-## Setting up elastic search.