erax-ai
/

EraX-VL-7B-V1.0

@@ -1,199 +1,152 @@
 ---
 library_name: transformers
-tags: []
 ---
-# Model Card for Model ID
-<!-- Provide a quick summary of what the model is/does. -->
-## Model Details
 ### Model Description
-<!-- Provide a longer summary of what this model is. -->
-This is the model card of a 🤗 transformers model that has been pushed on the Hub. This model card has been automatically generated.
-- **Developed by:** [More Information Needed]
-- **Funded by [optional]:** [More Information Needed]
-- **Shared by [optional]:** [More Information Needed]
-- **Model type:** [More Information Needed]
-- **Language(s) (NLP):** [More Information Needed]
-- **License:** [More Information Needed]
-- **Finetuned from model [optional]:** [More Information Needed]
-### Model Sources [optional]
-<!-- Provide the basic links for the model. -->
-- **Repository:** [More Information Needed]
-- **Paper [optional]:** [More Information Needed]
-- **Demo [optional]:** [More Information Needed]
-## Uses
-<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
 ### Direct Use
-<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
-[More Information Needed]
-### Downstream Use [optional]
-<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
-[More Information Needed]
-### Out-of-Scope Use
-<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
-[More Information Needed]
-## Bias, Risks, and Limitations
-<!-- This section is meant to convey both technical and sociotechnical limitations. -->
-[More Information Needed]
-### Recommendations
-<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
-Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
-## How to Get Started with the Model
-Use the code below to get started with the model.
-[More Information Needed]
-## Training Details
-### Training Data
-<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
-[More Information Needed]
-### Training Procedure
-<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
-#### Preprocessing [optional]
-[More Information Needed]
-#### Training Hyperparameters
-- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
-#### Speeds, Sizes, Times [optional]
-<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
-[More Information Needed]
-## Evaluation
-<!-- This section describes the evaluation protocols and provides the results. -->
-### Testing Data, Factors & Metrics
-#### Testing Data
-<!-- This should link to a Dataset Card if possible. -->
-[More Information Needed]
-#### Factors
-<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
-[More Information Needed]
-#### Metrics
-<!-- These are the evaluation metrics being used, ideally with a description of why. -->
-[More Information Needed]
-### Results
-[More Information Needed]
-#### Summary
-## Model Examination [optional]
-<!-- Relevant interpretability work for the model goes here -->
-[More Information Needed]
-## Environmental Impact
-<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
-Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
-- **Hardware Type:** [More Information Needed]
-- **Hours used:** [More Information Needed]
-- **Cloud Provider:** [More Information Needed]
-- **Compute Region:** [More Information Needed]
-- **Carbon Emitted:** [More Information Needed]
-## Technical Specifications [optional]
-### Model Architecture and Objective
-[More Information Needed]
-### Compute Infrastructure
-[More Information Needed]
-#### Hardware
-[More Information Needed]
-#### Software
-[More Information Needed]
-## Citation [optional]
-<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
-**BibTeX:**
-[More Information Needed]
-**APA:**
-[More Information Needed]
-## Glossary [optional]
-<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
-[More Information Needed]
-## More Information [optional]
-[More Information Needed]
-## Model Card Authors [optional]
-[More Information Needed]
-## Model Card Contact
-[More Information Needed]

 ---
 library_name: transformers
+license: apache-2.0
+language:
+- vi
+- multilingual
+base_model:
+- Qwen/Qwen2-VL-7B-Instruct
 ---
+![image/jpeg](https://cdn-uploads.huggingface.co/production/uploads/66e93d483745423cbb14c5ff/fNxjr3en_onzbOv0sghpE.jpeg)
 ### Model Description
+We are happy to share the EraX-VL-7B-v1 model, a powerful multimodal model in OCR (optical character recognition) and VQA (video question-answering) across multiple languages but especially in Vietnamese.
+One of the strengths of the EraX-VL-7B is its ability to accurately recognize forms, invoices, bills of sale, quotes, and medical records, among other things, which we believe will be very convenient for Hospitals, Clinics, Insurance Companies, and other applications.
+EraX-VL-7B has been finetuned from the base Qwen/Qwen2-VL-7B-Instruct which we found to be of good quality and quite fluent in Vietnamese.
+We will continue to develop better versions, as well as share benchmarks in the near future.
+EraX-VL-7B-V1 is a young member of our EraX's LànhGPT repository of LLM models.
+- **Developed by:** Nguyễn A. Nguyên (Steve) and Dũng T. Hoàng and Thục D. Phạm and Nhật H. Phạm
+- **Funded by:** Bamboo Capital Group (https://bamboocap.com.vn) & EraX
+- **Model type:** Transformers' multi-modal, 7 billions parameters.
+- **Language(s) (NLP):** Vietnamese and Multilingual
+- **License:** apache-2.0
+- **Finetuned from model:** Qwen/Qwen2-VL-7B-Instruct
 ### Direct Use
+Please use this Colab example: https://colab.research.google.com/drive/1CnSxtWDLG48-NQh7wk9_z8WI7J4OY_Ci?usp=sharing
+## Ví dụ
+![image/jpeg](https://cdn-uploads.huggingface.co/production/uploads/66e93d483745423cbb14c5ff/Q5GkK8vuZ9zDVPkhwu4yH.jpeg)
+* Kết quả:
+*
+{
+    "document": {
+        "header": {
+            "title": "GIẤY HẸN KHÁM LẠI",
+            "organization": "SỞ Y TẾ NGHỆ AN\nBỆNH VIỆN UNG BƯỚU NGHỆ AN",
+            "address": "Võ Thị Sáu, Thủy Tùng - TP Vinh - Nghệ An"
+        },
+        "patient_info": {
+            "name": "NGUYỄN THỊ LUÂN",
+            "date_of_birth": "03/07/1976",
+            "gender": "40",
+            "address": "Xã Nghĩa Khánh-Huyện Nghĩa Đàn-Nghệ An",
+            "medical_card_number": "CN 3 40 40 168 60413",
+            "registration_date": "16/12/2016",
+            "admission_date": "Từ 01/03/2016",
+            "diagnosis": "C20-Bướu ac trực tràng",
+            "revisit_date": "17/01/2017"
+        },
+        "administrative_details": {
+            "department": "Trung tâm điều trị ung bướu",
+            "revisit_instruction": "vào ngày 17/01/2017, hoặc đến hết kỳ thời gian nếu nước ngoài hẹn khám lại nếu có dấu hiệu (triệu chứng)",
+            "note": "nếu KCB ban đầu: Trạm y tế xã Nghĩa Khánh",
+            "signature": "Trưởng khoa",
+            "doctor_signature": "Lâm Nguyễn Khang",
+            "revisiting_date_confirmation": "Ngày 16 tháng 12 năm 2016",
+            "confirmation_signature": "Bác sĩ điều trị",
+            "physician_signature": "Nguyễn Văn Việt"
+        }
+    }
+}
+![image/png](https://cdn-uploads.huggingface.co/production/uploads/66e93d483745423cbb14c5ff/IrX-QP67TZTcTl3vlp1uZ.png)
+* Kết quả:
+{
+  "header": {
+    "title": "PHIẾU KHÁM BỆNH",
+    "date": "Hà Nội, ngày 23 tháng 3 năm 2020",
+    "patient_info": {
+      "id": "HN011000002",
+      "name": "Vương Hồng Thắng - Năm sinh: 1978",
+      "address": "Số 10 tầng 2, TTTM V+, Số 505 Phố Minh Khai, Quận Hai Bà Trưng, Hà Nội",
+      "phone": "+0942116117",
+      "email": "bamu.vn@gmail.com"
+    },
+    "contact_info": {
+      "address": "Nhà Khoa Bamufit\nĐịa chỉ: 505, Phố Minh Khai, Hai Bà Trưng, Hà Nội, Việt Nam",
+      "phone": "0942484784",
+      "email": "info@bamufit.vn",
+      "website": "https://bamufit.vn"
+    }
+  },
+  "treatment_details": [
+    {
+      "visit_date": "13-09-2019",
+      "treatment_type": "Chẩn đoán: Abscess chẽ",
+      "procedure": "Cắt lợi bằng Laser r23",
+      "doctor": "THỊ HIEN",
+      "price": "500,000",
+      "quantity": "1",
+      "discounted_price": "0",
+      "total_cost": "500,000"
+    },
+    {
+      "visit_date": "13-09-2019",
+      "treatment_type": "Chẩn đoán: Abscess quanh chóp",
+      "procedure": "Bám gai xuống ở răng r23",
+      "doctor": "THỊ HIEN",
+      "price": "100,000",
+      "quantity": "1",
+      "discounted_price": "0",
+      "total_cost": "100,000"
+    }
+  ],
+  "financial_details": {
+    "total_cost": "600,000",
+    "discounted_total": "0",
+    "paid_amount": "1,114,000",
+    "remaining_balance": "1,714,000"
+  },
+  "notes": "- Kiêng ăn uống đồ để gây nhiễm mủ như chè, cà phê, thuốc lá, rượu vang đỏ .. và hạn chế dùng đồ quá nóng, quá lạnh sau khi tẩy trắng răng ít nhất 2 tuần.",
+  "footer": {
+    "customer_signature": "(Ký và ghi rõ họ tên)",
+    "accountant_signature": "(Ký và ghi rõ họ tên)",
+    "dentist_signature": "(Ký và ghi rõ họ tên)"
+  }
+}
+![image/jpeg](https://cdn-uploads.huggingface.co/production/uploads/66e93d483745423cbb14c5ff/pSSqEOXQCsvz9H76CGQXa.jpeg)
+* Kết quả:
+Hình ảnh là một biểu đồ thể hiện mối quan hệ giữa chỉ số BMI (Body Mass Index) và tuổi, được chia thành các mức độ khác nhau dựa trên phần trăm percentile. Trục hoành của biểu đồ đại diện cho tuổi từ 2 đến 20 năm, trong khi trục tung đại diện cho chỉ số BMI từ 10 đến 32. Biểu đồ này có ba khu vực chính: vùng màu đỏ ở phía dưới cùng đại diện cho mức béo phì với chỉ số BMI cao hơn 30; vùng màu vàng nằm giữa đại diện cho nguy cơ béo phì với chỉ số BMI từ khoảng 25 đến 30; và vùng màu xanh lá cây ở phía trên đại diện cho mức cân nặng khỏe mạnh hoặc thiếu cân với chỉ số BMI thấp hơn 25. Trên biểu đồ còn có đường cong màu xám chạy qua các mức độ BMI theo tuổi, đánh dấu các mức 5th, 50th, và 95th percentile. Văn bản trong hình gồm các cụm từ 'Béo phì', 'Nguy cơ béo phì', 'Sức khỏe dinh dưỡng tốt', và 'Thiếu cân' để mô tả từng khu vực tương ứng với chỉ số BMI.
+## Citation
+@misc{erax-multimodal-17sept2024,
+      title={EraX-VL-7B-V1: A Highly Efficient Multimodal Large Language Model for Vietnamese, especially for medical forms and bills.},
+      author={Nguyễn A. Nguyên (Steve) and Dũng T. Hoàng and Thục D. Phạm and Nhật H. Phạm},
+      helpers={Nguyễn Hồ Nam nd Khang Đoàn}
+      contact={nguyen@erax.ai}
+      organization={EraX}
+}
+## References
+[1] Yang, An, et al. "Qwen2 technical report." arXiv preprint arXiv:2407.10671 (2024).
+[2] Chen, Zhe, et al. "Internvl: Scaling up vision foundation models and aligning for generic visual-linguistic tasks." Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2024.
+[3] Chen, Zhe, et al. "How far are we to gpt-4v? closing the gap to commercial multimodal models with open-source suites." arXiv preprint arXiv:2404.16821 (2024).
+[4] Tran, Chi, and Huong Le Thanh. "LaVy: Vietnamese Multimodal Large Language Model." arXiv preprint arXiv:2404.07922 (2024).