What are adapters and LoRA in the context of efficient fine-tuning?

• A. Adapters modify data; LoRA modifies architecture.
• B. Adapters are large pre-trained models; LoRA is data augmentation
• C. Adapters remove layers; LoRA increases parameter counts.
• D. Adapters are small extra layers inserted into models to enable task-specific fine-tuning with limited updates; LoRA decomposes weight updates into low-rank matrices to reduce trainable parameters.

Answer: (D)

Parameter-efficient fine-tuning focuses on adapting large pre-trained models by updating only a small portion of parameters. Adapters are small, trainable modules inserted into the model architecture—often after each layer—that learn task-specific adjustments. With adapters, the base model weights stay frozen and only the adapter parameters are updated, keeping memory and compute light while enabling quick switching between tasks.

LoRA takes a different route by keeping the original weights untouched and expressing their updates as products of low-rank matrices. The trainable part is just these low-rank factors, which approximate the needed changes to the weight matrices. This dramatically reduces the number of trainable parameters while retaining the model's capacity to adapt to new tasks.

So the statement is accurate: adapters are small extra layers for task-specific fine-tuning with limited updates, and LoRA decomposes weight updates into low-rank matrices to reduce trainable parameters. Other descriptions would mischaracterize these methods.