大模型部署例程

AIBOX-1684X 的大模型部署示例代码请参考 LLM-TPU 项目。

LLM-TPU 项目简介

LLM-TPU 包含了各类开源生成式 AI 模型的移植部署例程，其中以LLM为主。

例程清单

Model	INT4	INT8	FP16/BF16	Huggingface Link
Baichuan2-7B		✔		LINK
ChatGLM3-6B	✔	✔	✔	LINK
CodeFuse-7B	✔	✔		LINK
DeepSeek-6.7B	✔	✔		LINK
Falcon-40B		✔	✔	LINK
Qwen-7B	✔	✔	✔	LINK
Qwen-14B	✔	✔	✔	LINK
Qwen1.5-0.5B	✔	✔	✔	LINK
Qwen1.5-1.8B	✔	✔	✔	LINK
Llama2-7B	✔	✔	✔	LINK
Llama2-13B		✔	✔	LINK
LWM-Text-Chat	✔	✔	✔	LINK
Mistral-7B-Instruct	✔	✔		LINK
Stable Diffusion			✔	LINK
Stable Diffusion XL			✔	LINK
WizardCoder-15B	✔			LINK
Yi-6B	✔	✔		LINK