GloVe

Discussion starters

• ▸What surprised you most about the ratio insight?
• ▸How would you explain the derivation to someone who hasn't taken linear algebra?
• ▸Try the RatioExplorer with different word pairs. What interesting patterns did you find?
• ▸Compare GloVe and Word2Vec embeddings on the same corpus. When do they agree? When do they differ?
• ▸Did you try training with different hyperparameters? What happened when you changed the window size or alpha?
• ▸The derivation predicted that biases encode frequency. Were you convinced before seeing the scatter plot in Chapter 6?