analysis – Ware

Design and Analysis of Algorithms

Course Features

Course Description

This is an intermediate algorithms course with an emphasis on teaching techniques for the design and analysis of efficient algorithms, emphasizing methods of application. Topics include divide-and-conquer, randomization, dynamic programming, greedy algorithms, incremental improvement, complexity, and cryptography.

Note on Previous Versions:

The Spring 2015 version of 6.046 contains substantially different content than the Spring 2005 version. The 2005 version was an introductory algorithms course assuming minimal previous experience, while the 2015 version is an intermediate course requiring a semester of introductory material found in 6.006.

Other OCW Versions

OCW has published multiple versions of this subject.

Best approximation theorem

Theorem

Let X be an inner product space with induced norm, and $Asubseteq X$ a non-empty, complete convex subset. Then, for all $xin X$ , there exists a unique best approximation a₀ to x in A.

Proof

Suppose x = 0 (if not the case, consider A − {x} instead) and let $d=d(0,A)=inf_{ain A} ||a||$ . There exists a sequence (a_n) in Asuch that

$||a_n||to d$ .

We now prove that (a_n) is a Cauchy sequence. By the parallelogram rule, we get

$||frac{a_n-a_m}{2}||^2+||frac{a_n+a_m}{2}||^2=frac{1}{2}(||a_n||^2+||a_m||^2)$ .

Since A is convex, $frac{a_n+a_m}{2}in A$ so

$underset{m,nin mathbb N}{forall }; ||frac{a_n+a_m}{2}||geq d$ .

Hence

$||frac{a_n-a_m}{2}||^2leq frac{1}{2}(||a_n||^2+||a_m||^2)-d^2to 0$ as $m,nto infty$

which implies $||a_n-a_m||to 0$ as $m,nto infty$ . In other words, (a_n) is a Cauchy sequence. Since A is complete,

$underset{a_0in A}{exists }; a_nto a_0$ .

Since $a_0in A$ , $||a_0||geq d$ . Furthermore

$||a_0||leq ||a_0-a_n||+||a_n||to d$ as $nto infty$ ,

which proves | | a₀ | | = d. Existence is thus proved. We now prove uniqueness. Suppose there were two distinct best approximations a₀and a₀‘ to x (which would imply | | a₀ | | = | | a₀‘ | | = d). By the parallelogram rule we would have

$||frac{a_0+a_0'}{2}||^2+||frac{a_0-a_0'}{2}||^2=frac{1}{2}(||a_0||^2+||a_0'||^2)=d^2$ .

Then

$||frac{a_0+a_0'}{2}||^2<d^2$

which cannot happen since A is convex, and as such $frac{a_0+a_0'}{2}in A$ , which means $||frac{a_0+a_0'}{2}||^2geq d^2$ , thus completing the proof.

Fréchet derivative

http://en.wikipedia.org/wiki/Fr%C3%A9chet_derivative

the Fréchet derivative is a derivative defined on Banach spaces. Named after Maurice Fréchet, it is commonly used to formalize the concept of the functional derivative used widely in the calculus of variations. Intuitively, it generalizes the idea of linear approximation from functions of one variable to functions on Banach spaces. The Fréchet derivative should be contrasted to the more general Gâteaux derivative which is a generalization of the classical directional derivative.

The Fréchet derivative has applications throughout mathematical analysis, and in particular to the calculus of variations and much of nonlinear analysis and nonlinear functional analysis. It has applications to nonlinear problems throughout the sciences.