nlp-deberta-classification
Fine-tunes DeBERTa-v3 for text classification tasks. Use when building text classifiers, sentiment analysis, or multi-label classification on domain-specific data.
SKILL.mdskills/nlp/deberta-classification/SKILL.md
GitHubDeBERTa Classification
Overview
DeBERTa-v3 (Decoding-enhanced BERT with disentangled attention) is a strong baseline for text classification. It outperforms BERT/RoBERTa on most NLU benchmarks while maintaining similar inference cost.
Quick Start
from transformers import AutoTokenizer, AutoModelForSequenceClassification, TrainingArguments, Trainer
model_name = "microsoft/deberta-v3-base"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForSequenceClassification.from_pretrained(model_name, num_labels=2)
Workflow
- Prepare dataset in HuggingFace Dataset format (text + label columns)
- Tokenize with
max_length=512, dynamic padding - Fine-tune with
TrainingArguments(lr=2e-5, epochs=3, warmup_ratio=0.1) - Evaluate with accuracy, F1, and confusion matrix
- Export model with
model.save_pretrained()
Key Decisions
- Base vs Large: Use
deberta-v3-base(86M params) first. Only upgrade tolarge(304M) if base plateaus. - Max length: 512 covers most tasks. Truncate long docs or use sliding window.
- Learning rate: 2e-5 is a safe default. Search [1e-5, 3e-5] if needed.