Design and Analysis of Algorithms

Course Features

Course Description

This is an intermediate algorithms course with an emphasis on teaching techniques for the design and analysis of efficient algorithms, emphasizing methods of application. Topics include divide-and-conquer, randomization, dynamic programming, greedy algorithms, incremental improvement, complexity, and cryptography.

Note on Previous Versions:

The Spring 2015 version of 6.046 contains substantially different content than the Spring 2005 version. The 2005 version was an introductory algorithms course assuming minimal previous experience, while the 2015 version is an intermediate course requiring a semester of introductory material found in 6.006.

Other OCW Versions

OCW has published multiple versions of this subject. Question_OVT logo

Best approximation theorem

Best approximation theorem


Let X be an inner product space with induced norm, and Asubseteq X a non-emptycomplete convex subset. Then, for all xin X, there exists a unique best approximation a0 to x in A.


Suppose x = 0 (if not the case, consider A − {x} instead) and let d=d(0,A)=inf_{ain A} ||a||. There exists a sequence (an) in Asuch that
  • ||a_n||to d.
We now prove that (an) is a Cauchy sequence. By the parallelogram rule, we get
  • ||frac{a_n-a_m}{2}||^2+||frac{a_n+a_m}{2}||^2=frac{1}{2}(||a_n||^2+||a_m||^2).
Since A is convexfrac{a_n+a_m}{2}in A so
  • underset{m,nin mathbb N}{forall }; ||frac{a_n+a_m}{2}||geq d.
  • ||frac{a_n-a_m}{2}||^2leq frac{1}{2}(||a_n||^2+||a_m||^2)-d^2to 0 as m,nto infty
which implies ||a_n-a_m||to 0 as m,nto infty. In other words, (an) is a Cauchy sequence. Since A is complete,
  • underset{a_0in A}{exists }; a_nto a_0.
Since a_0in A||a_0||geq d. Furthermore
  • ||a_0||leq ||a_0-a_n||+||a_n||to d as nto infty,
which proves | | a0 | | = d. Existence is thus proved. We now prove uniqueness. Suppose there were two distinct best approximations a0and a0 to x (which would imply | | a0 | | = | | a0‘ | | = d). By the parallelogram rule we would have
  • ||frac{a_0+a_0'}{2}||^2+||frac{a_0-a_0'}{2}||^2=frac{1}{2}(||a_0||^2+||a_0'||^2)=d^2.
  • ||frac{a_0+a_0'}{2}||^2<d^2
which cannot happen since A is convex, and as such frac{a_0+a_0'}{2}in A, which means ||frac{a_0+a_0'}{2}||^2geq d^2, thus completing the proof.

Fréchet derivative

 the Fréchet derivative is a derivative defined on Banach spaces. Named after Maurice Fréchet, it is commonly used to formalize the concept of the functional derivative used widely in the calculus of variations. Intuitively, it generalizes the idea of linear approximation from functions of one variable to functions on Banach spaces. The Fréchet derivative should be contrasted to the more general Gâteaux derivative which is a generalization of the classical directional derivative.
The Fréchet derivative has applications throughout mathematical analysis, and in particular to the calculus of variations and much of nonlinear analysis and nonlinear functional analysis. It has applications to nonlinear problems throughout the sciences.