Notes-TheAlgorithmDesignManual/TheAlgorithmDesignManual.org

Notes & Exercises: The Algorith Design Manual

• Chapter 1
  • 1
  • 1.1 Robots
    • Implement NearestNeighbor
    • Implement ClosestPair
    • Implement OptimalTSP for N < 8
  • 1.2 Right Jobs
  • 1.3 Correctness
  • 1.4 Modeling
  • 1.5-1.6 War Story about Psychics
• Chapter 2
  • 2.1 RAM Model of Computation
  • 2.2 Big Oh
  • 2.3 Growth Rates and Dominance Relations
  • 2.4 Working with Big Oh
  • 2.5 Efficiency
    • Selection Sort
      • C
    • Insertion Sort
      • C
    • String Pattern Matching
    • Matrix Multiplication
  • 2.6 Logarithms
  • 2.7 Properties of Logarithms
  • 2.8 War Story Pyramids
  • 2.9 Advanced Analysis
• Chapter 3

Chapter 1

1

An algorithm is a procedure that takes any of the possible input instances and transforms it to the desired output.

1.1 Robots

The Robot arm problem is presented where it is trying to solder contact points and visit all points in the shortest path possible.

The first algorithm considered is NearestNeighbor. However this is a naïve, and the arm hopscotches around

Next we consider ClosestPair, but that too misses in certain instances.

Next is OptimalTSP, which will always give the correct result because it enumerates all possible combinations and return the one with the shortest path. For 20 points however, the algorithm grows at a right of O(n!). TSP stands for Traveling Salesman Problem.

TODO Implement NearestNeighbor

TODO Implement ClosestPair

TODO Implement OptimalTSP for N < 8

1.2 Right Jobs

Here we are introduced to the Movie Scheduling Problem where we try to pick the largest amount of mutually non-overlapping movies an actor can pick to maximize their time. Algorithms considered are EarliestJobFirst to start as soon as possible, and then ShortestJobFirst to be done with it the quickest, but both fail to find optimal solutions.

ExhaustiveScheduling grows at a rate of O(2ⁿ) which is much better than O(n!) as in the previous problem. Finally OptimalScheduling improves efficiency by first removing candidates that are overlapping such that it doesn't even compare them.

1.3 Correctness

It's important to be clear about the steps in pseudocode when designing algorithms on paper. There are important things to consider about algorithm correctness;

• Verifiability
• Simplicity
• Think small
• Think exhaustively
• Hunt for the weakness
• Go for a tie
• Seek extremes

Other tecniques include Induction, Recursion, and Summations.

1.4 Modeling

Most algorithms are designed to work on rigorously defined abstract structures. These fundamental structures include;

• Permutations
• Subsets
• Trees
• Graphs
• Points
• Polygons
• Strings

1.5-1.6 War Story about Psychics

Chapter 2

2.1 RAM Model of Computation

This is a simpler kind of Big Oh where

• Each simple operation is 1 step
• Loops and Subroutines are composition of simple operations
• Each memory access is one time step

Like flat earth theory, in practice we use it when engineering certain structures because we don't take into account the curvature of the Earth.

We can already apply the concept of worst, average, and best case to this model.

2.2 Big Oh

The previous model often requires concrete implementations to actually measure correctly, so instead Big Oh gives us a better, simpler framework for discussing the relative performance between algorithms. It ignores factors that don't impact how algorithms scale.

2.3 Growth Rates and Dominance Relations

These are the functions that occur in algorithm analyses;

• Constant O(1) Hashtable look up, array look up, consing a list
• Logarithmic O(log n) Binary Search
• Linear O(n) Iterating over a list
• Superlinear O(n log n) Quicksort and Mergesort
• Quadratic O(n²) Insertion Sort and Selection Sort
• Cubic O(n³) Some dynamic programming problems
• Exponential O(Cⁿ^{}) c for any constant c > 1 Enumerate all subsets
• Factorial O(n!) Generating all permutations or orderings

Notes:

• O(n!) algorithms become useless for anything n >= 20
• O(2ⁿ) algorithms become impractical for anything n > 40
• O(n²^{}) algorithms start deteriorating after n > 10,000, a million is hopeless
• O(n²^{}) and O(n log n) Are fine up to 1 billion

2.4 Working with Big Oh

Apparently you can do arithmetic on the Big Oh functions

2.5 Efficiency

Selection Sort

C

void print_nums(int *nums, int length) {
    for (int i = 0; i < length; i++) {
        printf("%d,", nums[i]);
    }
    printf("\n");
}

void selection_sort(int *nums, int length) {
    int i, j;
    int min_idx;
    for (i = 0; i < length; i++) {
        print_nums(nums, length);
        min_idx = i;
        for (j = i+1; j < length; j++) {
            if (nums[j] < nums[min_idx]) {
                min_idx = j;
            }
        }
        int temp = nums[min_idx];
        nums[min_idx] = nums[i];
        nums[i] = temp;
    }
}

int nums[9] = { 2, 4, 9, 1, 3, 8, 5, 7, 6 };
selection_sort(nums, 9);

2	4	9	1	3	8	5	7	6
1	4	9	2	3	8	5	7	6
1	2	9	4	3	8	5	7	6
1	2	3	4	9	8	5	7	6
1	2	3	4	9	8	5	7	6
1	2	3	4	5	8	9	7	6
1	2	3	4	5	6	9	7	8
1	2	3	4	5	6	7	9	8
1	2	3	4	5	6	7	8	9

Insertion Sort

C

void insertion_sort(int *nums, int len) {
    int i, j;
    for (i = 1; i < len; i++) {
        j = i;
        while (nums[j] < nums[j -1] && j > 0) {
            int temp = nums[j];
            nums[j] = nums[j - 1];
            nums[j - 1] = temp;
            j--;
        }
    }
}

int nums[8] = {1,4,5,2,8,3,7,9};
insertion_sort(nums, 8);
for (int i = 0; i < 8; i++) {
    printf("%d", nums[i]);
}

12345789

TODO String Pattern Matching

TODO Matrix Multiplication

2.6 Logarithms

Logarithms are the inverse of exponents. Binary search is great for sorted lists. There are applications related to fast exponentiation, binary trees, harmonic numbers, and criminal sentencing.

2.7 Properties of Logarithms

Common bases for logarithms include 2, e, and 10. The base of the logarithm has no real impact on the growth rate; log₂ and log₃ are roughly equivalent.

2.8 War Story Pyramids

Cool story bro

2.9 Advanced Analysis

Some advanced stuff

• Inverse Ackerman's Function Union-Find data structure
• log log n Binary search on a sorted array of only log n items
• log n / log log n
• log² n
• \sqrt{,}n

There are also limits and dominance relations

Chapter 3