How do backpropagation and gradient descent relate to prompt tuning vs full-model fine-tuning?

• A. Backpropagation updates all weights; gradient descent updates only the prompt.
• B. Backpropagation computes gradients; gradient descent updates weights. Prompt tuning updates only soft prompt embeddings; full fine-tuning updates all model weights.
• C. Backpropagation is used only for training from scratch; gradient descent for inference.
• D. Prompt tuning uses no gradients.

Answer: (B)

Gradients tell us how to improve the model, and updates move the parameters in the direction that lowers the loss. Backpropagation is the algorithm that computes those gradients by applying the chain rule through the network. Gradient descent is the basic update rule that uses those gradients to adjust the parameters over training steps.

In prompt tuning, you freeze the main model and only train the soft prompt embeddings added to the input. The gradients flow through the full network, but the update step applies only to those prompt parameters, leaving the rest of the model fixed. In full-model fine-tuning, every weight in the model is trainable, so gradients are computed for all parameters and gradient descent updates all of them.

So the same pair—backpropagation for calculating gradients and gradient descent for updating parameters—applies, but the scope of what gets updated differs: prompt tuning updates a small, specific set of prompt embeddings, while full fine-tuning updates the entire model.