:pencil: [Doc] Readme for features, and comments for references

README.md

@@ -8,23 +8,21 @@ app_port: 23333
 ---
 
 ## HF-LLM-API
-API for LLM inference in Huggingface spaces.
-
+Huggingface LLM Inference API in OpenAI message format.
 
 ## Features
 
 ✅ Implemented:
 
 - Support Models
-  - `mixtral-8x7b`
+  - `mixtral-8x7b`, `mistral-7b`
 - Support OpenAI API format
   - Can use api endpoint via official `openai-python` package
-- Support stream response
-- Support infinite-round chat
-- Support Docker deployment
+- Stream response
+- Docker deployment
 
 🔨 In progress:
-- [ ] Support more models
+- [x] Support more models
 
 ## Run API service
 

networks/message_streamer.py

@@ -22,6 +22,7 @@ class MessageStreamer:
         else:
             self.model = "default"
         self.model_fullname = self.MODEL_MAP[self.model]
+        self.message_outputer = OpenaiStreamOutputer()
 
     def parse_line(self, line):
         line = line.decode("utf-8")
@@ -38,11 +39,11 @@ class MessageStreamer:
         stream: bool = True,
         yield_output: bool = False,
     ):
-        # https://huggingface.co/docs/text-generation-inference/conceptual/streaming#streaming-with-curl
+        # https://huggingface.co/docs/api-inference/detailed_parameters?code=curl
+        # curl --proxy http://<server>:<port> https://api-inference.huggingface.co/models/<org>/<model_name> -X POST -d '{"inputs":"who are you?","parameters":{"max_new_token":64}}' -H 'Content-Type: application/json' -H 'Authorization: Bearer <HF_TOKEN>'
         self.request_url = (
             f"https://api-inference.huggingface.co/models/{self.model_fullname}"
         )
-        self.message_outputer = OpenaiStreamOutputer()
         self.request_headers = {
             "Content-Type": "application/json",
         }
@@ -51,6 +52,7 @@ class MessageStreamer:
         #     class InferenceClient > def text_generation()
         #   huggingface_hub/inference/_text_generation.py:
         #     class TextGenerationRequest > param `stream`
+        # https://huggingface.co/docs/text-generation-inference/conceptual/streaming#streaming-with-curl
         self.request_body = {
             "inputs": prompt,
             "parameters": {