Tasks

10
TaskTopicsDifficultyEnvironment

Implement learn_bpe(word_counts, num_merges), which learns a sequence of BPE merges from a word-frequency corpus.

nlp · tokenization · bpe · algorithms
medium

Implement apply_bpe(word, merges). Start with the word's characters and apply merges strictly in the given order. At every step, merge all non-overlapping occurrences of the pair from left to right.…

nlp · tokenization · bpe
easy

Implement pad_sequences(sequences, pad_value=0), converting a list of integer-id sequences into a rectangular NumPy array.

numpy · nlp · padding · masking
easy

Implement TfidfVectorizerLite without sklearn. Tokenize with lowercase text and re.findall(r"\w+", text).

nlp · tfidf · numpy · feature-engineering
hard

Implement make_skipgram_pairs(sentences, window_size=2). Every sentence is already a sequence of integer token IDs. For every position, add (center_id, context_id) pairs for all neighbors no farther…

nlp · embeddings · word2vec · skipgram
easy

Implement NumPy causal scaled dot-product attention: causal_attention(q, k, v). The last two axes are [L, D]; all leading batch/head axes must support compatible broadcasting.

numpy · transformers · attention · causal-mask
hard

Implement make_lm_batch(data, starts, block_size). For every start index s, take x = data[s:s+block_size] and target y = data[s+1:s+block_size+1]. Return two NumPy arrays of shape [B, block_size].

numpy · language-modeling · batching · gpt
easy

Implement top_k_filter(logits, k): return a copy of the NumPy array that keeps exactly the k largest logits along the final axis and replaces all others with -inf.

numpy · language-modeling · sampling · top-k
medium

Implement LoRALinear(base, rank=4, alpha=8.0) in PyTorch. Wrap an existing nn.Linear, freeze all of its parameters, and add the low-rank update ΔW = B @ A * (alpha / rank).

pytorch · lora · fine-tuning · linear-algebra
hard

Implement clip_loss(a, b, temperature=0.07) for two [N, D] PyTorch embedding tensors whose equal-index rows form positive pairs.

pytorch · contrastive-learning · clip · infonce
medium