Calculus

Calculus-based heuristics, like Newton's method or gradient descent, use principles of calculus to find solutions to complex problems. They involve calculating slopes or gradients to understand how a function changes. For instance, gradient descent moves towards the lowest point of a function, similar to finding the bottom of a valley - this point is often the best solution. These methods are powerful for optimizing problems, like in machine learning or economics, where finding the most efficient point is crucial.

Newton's method

Newton's Method is a calculus-based technique to find the roots of a function, where the function equals zero. It starts with a guess and repeatedly applies a formula to get closer to the root. It uses the function's slope (derivative) to improve each guess. It's effective for smooth, continuous functions but can fail for functions with no derivative, flat slopes, or multiple roots near each other. It's great for precise, math-heavy problems but not for erratic or non-differentiable functions.

Gradient descent

Gradient Descent is a method used to find the minimum of a function. Imagine walking downhill towards the lowest point in a valley—that's what this method does mathematically. It calculates the gradient (the slope) of the function and takes steps in the direction that decreases the function's value. It's powerful for optimizing in machine learning and economics. However, it struggles with functions having many valleys (local minima) or plateaus, and might not find the absolute lowest point (global minimum).