--- title: MS-LongWriter-Qwen2.5-7B-Instruct-GPTQ-Int4 canonical_url: "https://www.modelscope.ai/models/swift/MS-LongWriter-Qwen2.5-7B-Instruct-GPTQ-Int4" md_url: "https://www.modelscope.ai/models/swift/MS-LongWriter-Qwen2.5-7B-Instruct-GPTQ-Int4.md" repository: swift/MS-LongWriter-Qwen2.5-7B-Instruct-GPTQ-Int4 last_updated: 2025-09-06 license: "Apache License 2.0" pipeline_tag: text-generation tasks: - text-generation model_type: - qwen2 library_name: - safetensors - pytorch frameworks: - Pytorch downloads: 124 stars: 0 --- # MS-LongWriter-Qwen2.5-7B-Instruct-GPTQ-Int4 > MS-LongWriter-Qwen2.5-7B-Instruct-GPTQ-Int4 - An open-source model by swift on ModelScope. MS-LongWriter-Qwen2.5-7B-Instruct-GPTQ-Int4 swift/MS-LongWriter-Qwen2.5-7B-Instruct-GPTQ-Int4 is a text-generation model on ModelScope. licensed under Apache License 2.0. - **Repository**: swift/MS-LongWriter-Qwen2.5-7B-Instruct-GPTQ-Int4 - **License**: Apache License 2.0 - **Tasks**: text-generation - **Downloads**: 124 - **Stars**: 0 - **Last updated**: 2025-09-06 Source: https://www.modelscope.ai/models/swift/MS-LongWriter-Qwen2.5-7B-Instruct-GPTQ-Int4 --- # MS-LongWriter-Qwen2.5-7B-Instruct-GPTQ-Int4

🤖 [LongWriter Dataset] • 💻 [Github Repo] • 📃 [LongWriter Paper]

GPTQ INT4量化版本。 您可以通过ModelScope命令行来下载模型： ```bash #安装ModelScope pip install modelscope ``` 使用command line下载： ```bash modelscope download --model swift/MS-LongWriter-Qwen2.5-7B-Instruct-GPTQ-Int4 --local_dir ./your_path ``` 您也可以下载整个repo, 参见模型下载页面。 以下为原版模型MS-LongWriter-Qwen2.5-7B-Instruct 的说明。 MS-LongWriter-Qwen2.5-7B-Instruct is trained based on [https://modelscope.cn/models/qwen/Qwen2.5-7B-Instruct](https://modelscope.cn/models/qwen/Qwen2.5-7B-Instruct), and is capable of generating 10,000+ words at once. MS-LongWriter-Qwen2.5-7B-Instruct begins training directly from the Qwen2.5-7B-Instruct, while performing significant distillation on the [LongWriter-6k](https://modelscope.cn/datasets/ZhipuAI/LongWriter-6k) to obtain 666 high-quality samples, which is [LongWriter-6k-filtered](https://modelscope.cn/datasets/swift/longwriter-6k-filtered) ## Datasets 1. [LongWriter-6k-filtered](https://modelscope.cn/datasets/swift/longwriter-6k-filtered), based on the [LongWriter-6k](https://modelscope.cn/datasets/ZhipuAI/LongWriter-6k) 2. [Magpie-Qwen2-Pro-200K-Chinese](https://modelscope.cn/datasets/AI-ModelScope/Magpie-Qwen2-Pro-200K-Chinese) , random sampling 6k examples. 3. [Magpie-Qwen2-Pro-200K-English](https://modelscope.cn/datasets/AI-ModelScope/Magpie-Qwen2-Pro-200K-English) , random sampling 6k examples. ## Model We use [ms-swift](https://github.com/modelscope/swift) to fine-tune the Qwen2-7B-Instruct model. 1. Installation ```python pip install ms-swift[llm] ``` 2. Fine-tuning Envs: ```text Nvidia A100(80G) x 4 ``` Run: ```shell CUDA_VISIBLE_DEVICES=0,1,2,3 swift sft \ --model_type qwen2_5-7b-instruct \ --dataset longwriter-6k-filtered#666 qwen2-pro-zh#6660 qwen2-pro-en#6660 \ --max_length 28672 \ --num_train_epochs 2 \ --eval_steps 200 \ --batch_size 1 \ --gradient_accumulation_steps 64 \ --gradient_checkpointing true \ --warmup_ratio 0.1 \ --learning_rate 1e-5 \ --sft_type full \ --loss_name long-ce \ --check_dataset_strategy warning \ --save_only_model false \ --save_total_limit -1 \ --lazy_tokenize true \ --dataloader_num_workers 1 \ --resume_only_model true \ --neftune_noise_alpha 5 \ --use_flash_attn true ``` 3. Fine-tuning with annealing The annealing strategy is used to improve the performance of the model during the post-training process. We leverage the LongWriter-6k-filtered dataset to fine-tune the model with annealing, and set the learning rate to 2e-6. Run: ```shell CUDA_VISIBLE_DEVICES=0,1,2,3 swift sft \ --model_type qwen2_5-7b-instruct \ --dataset longwriter-6k-filtered#666 \ --max_length 28672 \ --num_train_epochs 2 \ --eval_steps 200 \ --batch_size 1 \ --gradient_accumulation_steps 64 \ --gradient_checkpointing true \ --warmup_ratio 0.1 \ --learning_rate 2e-6 \ --sft_type full \ --loss_name long-ce \ --check_dataset_strategy warning \ --save_only_model false \ --save_total_limit -1 \ --lazy_tokenize true \ --dataloader_num_workers 1 \ --resume_only_model true \ --neftune_noise_alpha 5 \ --use_flash_attn true \ --resume_from_checkpoint {previous-checkpoint-path} ``` 4. GPTQ-Int4 Quantization To run following command, export the quantization model with GPTQ-Int4. Envs: ```text Nvidia A100(80G) x 1 ``` Run: ```shell CUDA_VISIBLE_DEVICES=0 swift export \ --ckpt_dir {previous-checkpoint-path} \ --quant_bits 4 \ --load_dataset_config true --quant_method gptq ``` Note: 1. The `--resume_from_checkpoint` parameter is used to specify the path of the previous checkpoint. (see the step2) ## Evaluation Refer to [LongWriter Evaluation](https://github.com/modelscope/evalscope/tree/main/evalscope/third_party/longbench_write) from the [EvalScope](https://github.com/modelscope/evalscope). ## Reference 量子位文章：[666条数据教会AI写万字长文！模型数据集都开源](https://mp.weixin.qq.com/s/LvWUSgIRO5HI5YSDRz7SxA)