Exam Preparation

This page is based on the past exam in exams/DSMaster M2 Exam.pdf and the lesson pages in this study guide. Treat it as a preparation map, not a guarantee of the next paper.

Observed exam pattern

The past paper has three tasks and 90 total points:

Task	Weight	What it tested	Expected output
Databases and SQL	45	Load CSV files, inspect tables, move DataFrames into SQLite, write SQL aggregations and joins	Code, query results, short explanations
Clustering using k-means	30	Generate blob data, run k-means, plot clusters and centers, choose k, explain PCA/LDA relevance	Code, plot, method choice, conceptual explanation
Generative AI	15	Explain ML application workflow, place data cleansing and dimensionality reduction, summarize LLM construction and attention	Diagrams and prose

The exam style is practical-conceptual. It asks you to write small working code, then explain why that code or method is appropriate.

Likely variance

Expect the same skills to appear with different surface details:

• The SQL task may use another dataset or different tables, but the operations are likely to stay close to read_csv, schema inspection, to_sql, GROUP BY, ORDER BY, JOIN, COUNT, SUM, DISTINCT, and set overlap.
• The clustering task may change the generated dataset, number of clusters, or algorithm comparison. Be ready to explain k-means, elbow method, silhouette score, scaling, initialization, hierarchical clustering, and DBSCAN.
• The dimensionality question may ask whether PCA or LDA helps before clustering, classification, visualization, or model training. The key distinction is that PCA is unsupervised and preserves variance, while LDA is supervised and preserves class separation.
• The Generative AI task may swap "build an LLM by hand" for text preparation, PyTorch training loops, GPT blocks, pretraining, finetuning, RAG, evaluation, or inference-time scaling.
• A future exam may ask you to compare classic NLP with LLM text pipelines: bag-of-words and TF-IDF versus token IDs, embeddings, positional embeddings, sliding windows, next-token prediction, and attention.

Priority study route

1. Read SQL vs Pandas, Interfacing Python and MySQL, and Modern Data Management 2.
2. Read Dimensionality Reduction and Cluster Analysis.
3. Read Working with Text, Attention Mechanisms, Implementing a GPT Model, Pretraining on Unlabeled Data, Spam Classification Finetuning, and Instruction Finetuning.
4. If time remains, read Reasoning, RAG, and Agents, Generating Text with a Pre-Trained LLM, Evaluating Reasoning Models, and Improving Reasoning with Inference-Time Scaling.

Mock exams with answers

• Mock Exam 01: close to the past exam pattern, with Olympics SQL, k-means, and GPT basics.
• Mock Exam 02: SQL edge cases, noise in clustering, and classic NLP versus LLM preprocessing.
• Mock Exam 03: SQL/Pandas comparison, PCA versus LDA, and LLM system workflows.
• Mock Exam 04: data management safety, association rules, vector search, RAG, and inference-time scaling.
• Mock Exam 05: SQL inspection, uneven clusters, hierarchical clustering, and neural-network training loops.
• Mock Exam 06: SQL joins, PCA preprocessing, explained variance, and the LLM text pipeline.
• Mock Exam 07: generated-SQL safety, sentiment classification, spam finetuning, and GPT blocks.
• Mock Exam 08: date grouping, vector databases, approximate search, and reasoning evaluation.
• Mock Exam 09: ML workflow design, leakage, monitoring, and instruction finetuning.
• Mock Exam 10: broad review of SQL, model choice, attention, decoding, and KV caching.

SQL exam template

Use this skeleton when the exam gives a directory of CSV files:

from pathlib import Path
import sqlite3

import pandas as pd

data_dir = Path("Data")
dataframes = {}

for path in data_dir.glob("*.csv"):
    dataframes[path.stem] = pd.read_csv(path)

for name, df in dataframes.items():
    print(name, df.shape)
    print(df.columns.tolist())
    print(df.head(2))

conn = sqlite3.connect(":memory:")
for name, df in dataframes.items():
    df.to_sql(name, conn, index=False, if_exists="replace")

Then answer each question with one SQL query and one short explanation.

Common query shapes

Count rows per group:

SELECT country, COUNT(*) AS athlete_count
FROM athletes
GROUP BY country
ORDER BY athlete_count DESC;

Count distinct values per group:

SELECT discipline, COUNT(DISTINCT event) AS event_count
FROM events
GROUP BY discipline
ORDER BY event_count DESC;

Aggregate medal totals:

SELECT country, SUM("Gold Medal") AS gold, SUM("Silver Medal") AS silver,
       SUM("Bronze Medal") AS bronze, SUM(Total) AS total
FROM medals_total
GROUP BY country
ORDER BY total DESC;

Find countries appearing in two conditions:

SELECT DISTINCT m1.country
FROM medals AS m1
JOIN medals AS m2
  ON m1.country_code = m2.country_code
WHERE m1.discipline = 'Swimming'
  AND m2.discipline = 'Athletics'
ORDER BY m1.country;

Join related tables:

SELECT a.name, a.country, t.team, t.discipline
FROM athletes AS a
JOIN teams AS t
  ON t.athletes_codes LIKE '%' || a.code || '%'
LIMIT 20;

Use the last pattern carefully. If a column stores a list as text, a proper normalized join may not exist. Say that this is a pragmatic text-match workaround and explain its limits.

SQL traps to avoid

• Do not import unused libraries. If the task only needs CSV loading and SQLite, pandas, sqlite3, and Path are enough.
• Do not assume table names. Print the dictionary keys and column names first.
• Do not use COUNT(column) when missing values matter. COUNT(*) counts rows; COUNT(column) skips null values.
• Do not forget quotes around column names with spaces, such as "Gold Medal".
• Do not confuse medals with medals_total. One is medal-event level; the other is already aggregated by country.
• Do not explain only the result. Explain the SQL operation: grouping, joining, filtering, aggregation, or distinct counting.

Clustering answer template

Minimal k-means workflow:

import matplotlib.pyplot as plt
from sklearn.cluster import KMeans
from sklearn.datasets import make_blobs
from sklearn.metrics import silhouette_score

X, y_true = make_blobs(
    n_samples=300,
    centers=4,
    cluster_std=0.7,
    random_state=42,
)

model = KMeans(n_clusters=4, random_state=42, n_init=10)
labels = model.fit_predict(X)
centers = model.cluster_centers_

plt.scatter(X[:, 0], X[:, 1], c=labels, s=20, cmap="viridis")
plt.scatter(centers[:, 0], centers[:, 1], c="red", marker="X", s=180)
plt.show()

How to choose k:

• Elbow method: fit several k values and plot inertia. Choose the point where improvement starts flattening.
• Silhouette score: compare how well points fit their own cluster versus nearby clusters. Higher is usually better.
• Domain knowledge: if the problem has a natural number of groups, use it as evidence.
• Visual inspection: useful for 2D toy data, weak for high-dimensional real data.

Short answer for "Would PCA or LDA help?":

For this generated two-dimensional blob dataset, PCA is not necessary because the data is already in 2D and k-means can be plotted directly. PCA could help if the original data had many numeric features and we wanted a lower-dimensional visualization or noise reduction. LDA is not appropriate for ordinary unsupervised clustering because it needs class labels; clustering does not have labels before the model runs.

Use the eligibility vs usefulness frame:

eligibility = can this method logically be used here?
usefulness  = did it actually improve the result after checking evidence?

For a mock-exam answer, decide eligibility from the setup, then avoid overclaiming usefulness until you mention evidence such as explained variance, class separation, plots, or model performance.

Dimensionality reduction decision table

Situation	Use	Why
Many numeric features, no labels, want compression or visualization	PCA	Preserves high-variance directions without labels
Labeled classes, want class-separating lower-dimensional features	LDA	Uses labels to maximize between-class separation and reduce within-class spread
Already 2D clustering data	Neither is necessary for visualization	The data can already be plotted directly
Need original feature names for interpretation	Feature selection	PCA and LDA create new mixed features
Nonlinear manifold or curved structure	Consider nonlinear methods	PCA and LDA are linear transformations
Raw features have different units	Standardize first	Scale can dominate PCA and distance-based methods
Running k-means on many features	Maybe PCA first	Can reduce noise and make clusters easier to visualize, but may also remove useful structure

For the slower explanation, read Dimensionality Reduction: Why PCA and LDA Work Differently. For output interpretation, read PCA in Code and LDA in Code.

ML application workflow diagram

Use this when asked to "outline developing a machine learning application":

problem definition
-> data collection
-> data understanding
-> data cleansing
-> feature engineering / representation
-> optional dimensionality reduction
-> train / validation / test split
-> model selection
-> training
-> evaluation
-> tuning
-> deployment
-> monitoring and iteration

Data cleansing happens before serious modeling because missing values, duplicates, inconsistent labels, bad types, and outliers can damage all later steps.

Dimensionality reduction usually happens after cleaning and numeric feature preparation, often inside a pipeline after scaling and before a model. Use PCA when labels are unavailable or the goal is variance-preserving compression. Use LDA when labels are available and the goal is class separation.

LLM construction answer template

When asked for the main steps in building an LLM by hand, organize the answer like this:

raw text
-> tokenization
-> token IDs
-> sliding input-target windows
-> token embeddings + positional embeddings
-> transformer blocks
   -> masked multi-head self-attention
   -> feed-forward network
   -> layer normalization and shortcut connections
-> output logits over vocabulary
-> next-token prediction loss
-> pretraining on unlabeled text
-> optional finetuning for classification or instruction following
-> generation one token at a time

Attention mechanism:

Attention lets each token build a context-aware representation by comparing itself with other tokens, assigning attention weights, and mixing information from value vectors. In GPT-style models, causal masking prevents a token from using future tokens during next-token prediction.

Important terms:

Term	Exam-ready meaning
Tokenization	Convert text into pieces the model can index
Token ID	Integer representation of a token
Embedding	Learned vector looked up from a token ID
Positional embedding	Adds order information to token embeddings
Query and key	Vectors used to compute attention scores
Value	Vector containing the information that gets mixed
Attention score	Raw match strength before softmax
Attention weight	Normalized focus amount after softmax
Causal mask	Blocks attention to future tokens
Multi-head attention	Several attention mechanisms in parallel
Logits	Raw scores for possible next tokens
Cross-entropy loss	Training objective for next-token prediction or classification
Pretraining	Learn general language patterns from unlabeled text
Finetuning	Adapt a pretrained model to a task or instruction style

High-yield conceptual contrasts

Contrast	What to say
SQL vs Pandas	SQL pushes filtering, joining, and aggregation into a database engine. Pandas is strong after data is in Python for cleaning, exploration, reshaping, plotting, and model preparation.
SQL vs MongoDB	SQL uses relational tables and joins. MongoDB uses flexible documents, nested fields, and arrays.
Clustering vs classification	Clustering is unsupervised and finds groups without labels. Classification is supervised and learns from known labels.
PCA vs LDA	PCA ignores labels and preserves variance. LDA uses labels and preserves class separation.
Bag-of-words vs LLM tokens	Bag-of-words counts words and loses order. LLM token pipelines preserve sequence through token IDs, positional embeddings, and next-token targets.
Pretraining vs finetuning	Pretraining learns general patterns from broad data. Finetuning adapts the model to a narrower task or response style.
RAG vs finetuning	RAG retrieves external context at prompt time. Finetuning changes model weights.
Greedy decoding vs sampling	Greedy picks the highest-probability token. Sampling can produce more varied text using temperature, top-k, or top-p.

Mock exam

Task 1: Databases and SQL, 45 points

Use the Olympics data directory.

1. Import only the libraries needed to load CSV files, inspect them, create an in-memory SQLite database, and run SQL queries.
2. Load all top-level CSV files into a dictionary of DataFrames. Print each table name, shape, and columns.
3. Write SQL queries for:
4. The ten countries with the most athletes.
5. The number of medal events per discipline.
6. The top ten countries by total medals.
7. Countries with medals in both cycling road and swimming.
8. The number of male and female athletes per country for the ten largest delegations.
9. For each query, show the result and explain the SQL idea in one or two sentences.

Task 2: Clustering and dimensionality reduction, 30 points

1. Generate a 2D dataset with 400 points and either three or five visible blob areas.
2. Fit k-means for several possible k values.
3. Plot the chosen clustering result with cluster centers.
4. Explain how you chose k using elbow method, silhouette score, or both.
5. Explain whether PCA or LDA would help in this exact setting.
6. Explain one situation where DBSCAN would be better than k-means.

Task 3: Generative AI, 15 points

1. Draw the machine learning application workflow from problem definition to monitoring.
2. Extend the diagram by placing data cleansing, feature engineering, dimensionality reduction, and model evaluation.
3. Explain the main steps in preparing text for GPT-style pretraining.
4. Explain attention using query, key, value, scores, weights, causal mask, and multi-head attention.
5. Briefly distinguish pretraining, classification finetuning, instruction finetuning, RAG, and inference-time scaling.

Final self-check

You are ready when you can do the following without looking:

• Load a folder of CSVs into a dictionary keyed by file stem.
• Move DataFrames into SQLite and query them with pd.read_sql.
• Write GROUP BY, ORDER BY, COUNT, SUM, DISTINCT, and JOIN queries.
• Explain when SQL should happen before Pandas.
• Generate blobs, run k-means, plot labels and centers.
• Explain k-means assumptions and why scaling matters for distance-based methods.
• Choose k with elbow method or silhouette score.
• Explain why LDA needs labels and why PCA does not.
• Draw the ML workflow and place cleaning and dimensionality reduction correctly.
• Describe the GPT pipeline from text to logits and next-token loss.
• Explain attention in plain language and with query-key-value terminology.
• Distinguish pretraining, finetuning, RAG, and inference-time scaling.