作者： aigcrepo 来源： aigcrepo

《讲一讲现在最流行的微调方法LoRA》粗略谈了LoRA的技术原理，huggingface的transformers库集成了peft库，所以微调LoRA非常简单。

首先微调，先加载peft配置：

---

from peft import (
    get_peft_config,
    get_peft_model,
    get_peft_model_state_dict,
    set_peft_model_state_dict,
    LoraConfig,
    PeftType,
)
  

peft_config = LoraConfig(task_type="SEQ_CLS", inference_mode=False, r=8, lora_alpha=16, lora_dropout=0.1)

其中r表示Lora attention维度，lora_alpha表示缩放因子，还有一些参数，比如target_modules表示对那些层进行lora。

然后加载一个预训练模型，然后进行微调：

model = AutoModelForSequenceClassification.from_pretrained(model_name_or_path, return_dict=True)
model = get_peft_model(model, peft_config)

可以打印由多少参数是要训练的，下面显示只有0.5%的参数进行梯度：

---

model.print_trainable_parameters()
  

trainable params: 1,838,082 || all params: 357,199,876 || trainable%: 0.5146

另外也可以打印PeftModel模型相关内容：

---

(query): lora.Linear(
    (base_layer): Linear(in_features=1024, out_features=1024, bias=True)
    (lora_dropout): ModuleDict(
      (default): Dropout(p=0.1, inplace=False)
    )
    (lora_A): ModuleDict(
      (default): Linear(in_features=1024, out_features=8, bias=False)
    )
    (lora_B): ModuleDict(
      (default): Linear(in_features=8, out_features=1024, bias=False)
    )
    (lora_embedding_A): ParameterDict()
    (lora_embedding_B): ParameterDict()
    (lora_magnitude_vector): ModuleDict()
)

可以清晰看到分解的A和B矩阵。

训练完成推送到hub中，可以看到模型adapter_config.json ，表明它采用了LoRA微调：

---

  "peft_type": "LORA",
  "r": 8,
  "rank_pattern": {},
  "revision": null,
  "target_modules": [
    "query",
    "value"
  ],

可以基于微调模型的adapter_config.json找到base model进行推理：

---

peft_model_id = "John19870907/roberta-large-peft-lora"
config = PeftConfig.from_pretrained(peft_model_id)
inference_model = AutoModelForSequenceClassification.from_pretrained(config.base_model_name_or_path)
tokenizer = AutoTokenizer.from_pretrained(config.base_model_name_or_path)
  

# Load the Lora model
inference_model = PeftModel.from_pretrained(inference_model, peft_model_id)
  

inference_model.to(device)
inference_model.eval()

也可以使用adapter的方式加载：

---

model_id = "roberta-large"
adapter_model_id = "John19870907/roberta-large-peft-lora"
tokenizer = AutoTokenizer.from_pretrained(model_id)
text = "Hello"
inputs = tokenizer(text, return_tensors="pt")
model = AutoModelForCausalLM.from_pretrained(model_id)
peft_config = PeftConfig.from_pretrained(adapter_model_id)

甚至还能禁止adapter：

model.disable_adapters()
output = model.generate(**inputs)

最后也可以将微调后的模型融合到base模型，比如：

---

model = PeftModel.from_pretrained(base_model, peft_model_id)
merged_model = model.merge_and_unload()
merged_model.push_to_hub("John19870907/roberta-large-peft-lora")

References

[1]

https://github.com/ywdblog/nlp_colab/blob/main/HF_sequence_classification_Peft_LoRA.ipynb
[2]

https://huggingface.co/John19870907/roberta-large-peft-lora/tree/main

更多AI工具，参考Github-AiBard123，国内AiBard123