\documentclass[11pt]{article}

\usepackage[letterpaper,margin=0.78in]{geometry}
\usepackage[T1]{fontenc}
\usepackage{lmodern}
\usepackage{amsmath,amssymb}
\usepackage{xcolor}
\usepackage{enumitem}
\usepackage{fancyhdr}
\usepackage{microtype}

\definecolor{titleblue}{HTML}{17365D}
\definecolor{ruleblue}{HTML}{5B9BD5}

\pagestyle{fancy}
\setlength{\headheight}{14pt}
\fancyhf{}
\lhead{CIS 5590: Vision-Language Models}
\rhead{Background Knowledge Diagnostic}
\cfoot{\thepage}
\renewcommand{\headrulewidth}{0.4pt}

\setlength{\parindent}{0pt}
\setlength{\parskip}{4pt}
\setlist[enumerate]{leftmargin=*,label=\textbf{\arabic*.},itemsep=0.85em,topsep=0.5em}

\begin{document}

{\LARGE\bfseries\color{titleblue} Background Knowledge Diagnostic}

{\large CIS 5590: Vision-Language Models}

\vspace{0.25em}
{\color{ruleblue}\rule{\textwidth}{1.2pt}}

\textbf{Name:} \rule{0.43\textwidth}{0.4pt}\hfill
\textbf{Date:} \rule{0.22\textwidth}{0.4pt}

\textit{Purpose:} These questions assess whether you have the mathematical and machine-learning foundations needed for the course. Concise answers are sufficient; no knowledge of the assigned papers is expected.

\begin{enumerate}

\item \textbf{Two-layer neural network.}
Write the formula for a two-layer feed-forward neural network that maps an input vector $\mathbf{x}\in\mathbb{R}^{d}$ to an output vector $\mathbf{y}$. Identify the weight matrices, bias vectors, and nonlinear activation function, and state their dimensions.

\item \textbf{Backpropagation and optimization.}
What is a gradient, and how is it used by gradient descent to update a neural-network parameter $\theta$? Write the basic update equation and explain the role of the learning rate.

\item \textbf{Softmax.}
Define the softmax function for a vector of logits $\mathbf{z}$. What properties make its output interpretable as a probability distribution?

\item \textbf{Cross-entropy loss.}
Write the cross-entropy loss for a multiclass classification problem with a one-hot target. How is it related to maximizing the likelihood of the correct class?

\item \textbf{Next-token prediction.}
Describe next-token prediction in an autoregressive language model. Given tokens $x_1,\ldots,x_T$, write the training loss and explain what probability distribution the model learns at every position.

\item \textbf{Embeddings and similarity.}
What is an embedding? Define cosine similarity between two embedding vectors and explain why cosine similarity is commonly used to compare image and text representations.

\item \textbf{Self-attention.}
Given an input matrix $X$, write the formulas for queries, keys, and values,
\[
Q=XW_Q,\qquad K=XW_K,\qquad V=XW_V,
\]
and then write the scaled dot-product self-attention operation. Explain why the factor $1/\sqrt{d_k}$ is used and what the attention weights represent.

\item \textbf{Images as transformer tokens.}
An RGB image of size $224\times224$ is divided into non-overlapping $16\times16$ patches. How many visual tokens are produced? Describe how each patch can be converted into a transformer input embedding.

\item \textbf{Evaluation metrics.}
Define precision, recall, and intersection over union (IoU). Give an example in which accuracy could be high even though a model performs poorly on the class of interest.

\item \textbf{Generalization and experimental design.}
Explain the difference among training, validation, and test sets. What is overfitting, and why must examples from the same image, video, patient, or data source sometimes be kept within only one split?

\end{enumerate}

\vfill
{\small\color{gray}This diagnostic is intended to identify topics that may benefit from review before or early in the course.}

\end{document}