Calculus

Calculus is essential for machine learning (ML), enabling optimization of models through derivatives and gradients. This section introduces key concepts—functions, derivatives, gradients, and optimization—with a Rust lab using nalgebra.

Functions and Derivatives

A function $f(x)$ maps inputs to outputs (e.g., $f(x)=x^2$). The derivative measures the rate of change of $f(x)$ with respect to $x$, denoted $f^{\prime }(x)$ or $\frac{df}{dx}$.

Example: For $f(x)=x^2$, the derivative is:

$$f^{\prime }(x)=2x$$

In ML, derivatives guide model updates (e.g., minimizing loss).

Gradients

For functions of multiple variables, $f(x_1,x_2,\dots ,x_n)$, the gradient is a vector of partial derivatives:

$$\mathrm{\nabla }f=\left[\begin{array}{c}\frac{\partial f}{\partial x_1},\frac{\partial f}{\partial x_2},\dots ,\frac{\partial f}{\partial x_n}\end{array}\right]$$

Gradients point in the direction of steepest ascent, used in gradient descent to minimize loss.

Optimization

ML models optimize a loss function $L(\mathbf{w})$ by adjusting weights $\mathbf{w}$ using gradient descent:

$$\mathbf{w}\leftarrow \mathbf{w}-\eta \mathrm{\nabla }L(\mathbf{w})$$

where $\eta$ is the learning rate.

Lab: Gradient Computation with nalgebra

You’ll compute the gradient of a simple function $f(x,y)=x^2+y^2$ using nalgebra.

1. Edit src/main.rs in your rust_ml_tutorial project:

   use nalgebra::Vector2;
   
   fn main() {
       // Define point (x, y)
       let point = Vector2::new(2.0, 3.0);
   
       // Compute gradient of f(x, y) = x^2 + y^2
       let gradient = Vector2::new(2.0 * point.x, 2.0 * point.y);
       println!("Gradient at ({}, {}): {:?}", point.x, point.y, gradient);
   }

2. Ensure Dependencies:

   • Verify Cargo.toml includes:

     [dependencies]
     nalgebra = "0.33.2"

   • Run cargo build.

3. Run the Program:

   cargo run

   Expected Output:

   Gradient at (2, 3): [4, 6]

Understanding the Results

• Function: $f(x,y)=x^2+y^2$ has partial derivatives $\frac{\partial f}{\partial x}=2x$, $\frac{\partial f}{\partial y}=2y$.
• Gradient: At $(x,y)=(2,3)$, the gradient is $[4,6]$, pointing toward the steepest ascent.
• ML Relevance: Gradients drive optimization in algorithms like linear regression.

This lab prepares you for gradient-based ML techniques.

Learning from Official Resources

Deepen Rust skills with:

• The Rust Programming Language (The Book): Free at doc.rust-lang.org/book.
• Programming Rust: By Blandy, Orendorff, and Tindall.

Next Steps

Continue to Probability for ML’s probabilistic foundations, or revisit Linear Algebra.

Further Reading

• Deep Learning by Goodfellow et al. (Chapter 4)
• An Introduction to Statistical Learning by James et al. (Prerequisites)
• nalgebra Documentation: nalgebra.org