Compare BERT-style encoders with GPT-style decoders in terms of architecture and typical use cases.

• A. BERT is for generation; GPT is for classification.
• B. BERT uses bidirectional encoder (masked language modeling) for understanding tasks; GPT uses unidirectional decoder for generation; encoder-decoder like T5 handles seq2seq tasks.
• C. They have identical architectures.
• D. BERT is supervised; GPT is unsupervised.

Answer: (C)

Architecture differences between bidirectional encoders and autoregressive decoders drive their use in understanding versus generation tasks. BERT-style models stack transformer encoders that attend to tokens on both sides, producing rich, context-aware representations of the whole input. They’re typically trained with masked language modeling and next-sentence prediction, which makes them well suited for understanding tasks like classification, QA, and inference where the goal is to interpret existing text. GPT-style models use a transformer decoder with causal masking, so each token is generated based only on previously produced tokens. This unidirectional, autoregressive setup excels at generation, such as completing prompts or writing coherent continuations. Encoder-decoder architectures like T5 blend both components to handle seq2seq tasks (e.g., translation, summarization). So, claiming they have identical architectures isn’t accurate—their structures and resulting use cases are distinct.