What is soft prompts and how do they enable prompt tuning without updating the base model?

• A. Soft prompts are fixed textual prompts.
• B. Soft prompts are trainable continuous embeddings prepended to inputs; the base model remains frozen; only prompt embeddings updated.
• C. Soft prompts replace the model.
• D. Soft prompts change model architecture.

Answer: (B)

Soft prompts are learned vectors in the model’s input embedding space that are prepended to the input sequence. Instead of changing the large model’s weights, you train a small set of these prompt embeddings to steer the model toward the desired task behavior. The base model stays frozen, meaning its parameters are not updated during training; only the prompt embeddings are updated through backpropagation. This conditioning guides the model’s hidden representations and outputs without altering the core parameters, making it a parameter-efficient way to adapt a single model to multiple tasks.

This is why the option describing trainable continuous embeddings added to the input—and the freezing of the base model while updating only those prompt embeddings—is the best fit. Fixed textual prompts would not adapt through learning, replacing the model would discard the learned capabilities, and changing the model’s architecture would go beyond prompting.