|
--- |
|
language: |
|
- zh |
|
- en |
|
tags: |
|
- glm |
|
- chatglm |
|
- thudm |
|
base_model: THUDM/chatglm3-6b |
|
--- |
|
|
|
# chatglm3-6b-awq-w-int4-asym-gs128-a-fp16-onnx-ryzen-strix-hybrid |
|
- ## Introduction |
|
This model was created by applying [Quark](https://quark.docs.amd.com/latest/index.html) with calibration samples from Pile dataset, and applying [onnxruntime-genai model builder](https://github.com/microsoft/onnxruntime-genai/tree/main/src/python/py/models) to convert to ONNX. |
|
- ## Quantization Strategy |
|
- ***Quantized Layers***: All linear layers, including "transformer.output_layer" |
|
- ***Weight***: uint4 asymmetric per-group, with group size 128 |
|
- AWQ / Group 128 / Asymmetric / FP16 activations / INT4 weights |
|
- ## Quick Start |
|
For quickstart, refer to AMD [RyzenAI-SW-EA](https://account.amd.com/en/member/ryzenai-sw-ea.html) (to be updated) |
|
|
|
#### License |
|
Modifications copyright(c) 2024 Advanced Micro Devices,Inc. All rights reserved. |
|
|
|
Licensed under the Apache License, Version 2.0 (the "License"); |
|
you may not use this file except in compliance with the License. |
|
You may obtain a copy of the License at |
|
|
|
http://www.apache.org/licenses/LICENSE-2.0 |
|
|
|
Unless required by applicable law or agreed to in writing, software |
|
distributed under the License is distributed on an "AS IS" BASIS, |
|
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. |
|
See the License for the specific language governing permissions and |
|
limitations under the License. |
