# WALL-OSS
WALL-OSS模型基于基模型进行微调，仅支持全参数微调（FFT），不支持LORA。
- conda环境：lerobot-wallx
- 默认超参数：batch_size=2、steps=100000、save_freq=10000、chunk_size=32、optimizer_lr=2e-5、scheduler_decay_lr=1e-6、scheduler_warmup_steps=1000、scheduler_decay_steps=100000
**训练启动命令（FFT）**
1. 复制基模型到输出目录
   ```
   cp -r ${BASE_MODEL} ${OUTPUT}/base
   ```
   

2. 启动训练
   ```
   conda activate lerobot-wallx && cd /opt/cloud/lerobot_ascend_notebook/lerobot && HF_HOME=${HF_CACHE} HF_HUB_OFFLINE=1 TRANSFORMERS_OFFLINE=1 accelerate launch --num_processes=${MA_NUM_GPUS} scripts/train.py --policy.type=wall_x --output_dir=${OUTPUT}/_ckpt --dataset.root=${DATASET} --dataset.repo_id=cloudrobo --policy.push_to_hub=false --wandb.enable=true --wandb.mode=offline --policy.pretrained_name_or_path=${BASE_MODEL} --policy.attn_implementation=eager --policy.prediction_mode=diffusion
   ```
   
**推理部署启动命令**
```
conda activate lerobot-wallx && cd /opt/cloud/lerobot_ascend_notebook/lerobot && HF_HUB_OFFLINE=1 TRANSFORMERS_OFFLINE=1 HF_HOME=/home/ma-user/.cache/huggingface/hf_cache python -m scripts.serve_policy --policy_type=wall_x --pretrained_name_or_path=${CHECKPOINT}/base --checkpoint_path=${CHECKPOINT}
```