Test Machine Learning Foundations and Introduction to LLMs

[bogdanfazakas]

[ndrpp]

UPB - Transformer foundations

• 3 labs:
  • tokenization - demonstrates how BPE (GPT), WordPiece(BERT) and SentencePiece(T5) split tokens of the same phrases
  • attention - uses BERT-base to compute attention of each token (word in this case) generates a graph with the results (attention = ability of a token to get information about other nearby tokens)
  • embeddings - extracts BERT's static and contextual embeddings and demonstrates how different meanings of words are handled, computes similarity between sentences and displays a graph for semantic clusters
• they don't need GPU, they can be run locally on a CPU - completes in a few seconds

UPB - Clustering

• clustering = grouping data from a dataset in clusters based on similarity or common features - can be done with a variety of algorithms
• runs clustering algorithms on a dataset of wines (178 wines, 3 cultivars, 13 chemical features)
• generates metrics for each run to determine the best performing one - and builds plots with the data
• doesn't need GPU as well, runs in seconds