Chapter 2 notes

DataStructures.org

@@ -1,4 +1,9 @@
-** C
+#+TITLE: Random Data Structures
+#+AUTHOR: Joseph Ferano
+#+OPTIONS: ^:{}
+
+** Stack
+*** C
 
 #+begin_src C :includes stdlib.h stdio.h stdbool.h
 typedef struct Node {

TheAlgorithmDesignManual.org

@@ -4,13 +4,172 @@
 
 * Chapter 1
 
-** Notes
+** 1
-*** 1
 An /algorithm/ is a procedure that takes any of the possible input instances
 and transforms it to the desired output. 
 
-The author provides an example of a sorting algorithm called
+** 1.1 Robots
-~insertion_sort~. Here it is modified so it actually runs
+The Robot arm problem is presented where it is trying to solder contact points
+and visit all points in the shortest path possible.
+
+The first algorithm considered is ~NearestNeighbor~. However this is a naïve, and
+the arm hopscotches around
+
+Next we consider ~ClosestPair~, but that too misses in certain instances.
+
+Next is ~OptimalTSP~, which will always give the correct result because it
+enumerates all possible combinations and return the one with the shortest
+path. For 20 points however, the algorithm grows at a right of O(n!). TSP stands
+for Traveling Salesman Problem.
+
+*** TODO Implement NearestNeighbor
+*** TODO Implement ClosestPair
+*** TODO Implement OptimalTSP for N < 8
+
+** 1.2 Right Jobs
+Here we are introduced to the Movie Scheduling Problem where we try to pick the
+largest amount of mutually non-overlapping movies an actor can pick to maximize
+their time. Algorithms considered are ~EarliestJobFirst~ to start as soon as
+possible, and then ~ShortestJobFirst~ to be done with it the quickest, but both
+fail to find optimal solutions.
+
+~ExhaustiveScheduling~ grows at a rate of O(2^{n}) which is much better than O(n!) as
+in the previous problem. Finally ~OptimalScheduling~ improves efficiency by first
+removing candidates that are overlapping such that it doesn't even compare them.
+
+** 1.3 Correctness
+
+It's important to be clear about the steps in pseudocode when designing
+algorithms on paper. There are important things to consider about algorithm
+correctness;
+
+- Verifiability
+- Simplicity
+- Think small
+- Think exhaustively
+- Hunt for the weakness
+- Go for a tie
+- Seek extremes
+
+Other tecniques include *Induction*, *Recursion*, and *Summations*.
+
+** 1.4 Modeling
+
+Most algorithms are designed to work on rigorously defined abstract
+structures. These fundamental structures include;
+
+- Permutations
+- Subsets
+- Trees
+- Graphs
+- Points
+- Polygons
+- Strings
+
+** 1.5-1.6 War Story about Psychics
+
+
+* Chapter 2
+
+** 2.1 RAM Model of Computation
+
+This is a simpler kind of Big Oh where
+
+- Each simple operation is 1 step
+- Loops and Subroutines are composition of simple operations
+- Each memory access is one time step
+
+Like flat earth theory, in practice we use it when engineering certain
+structures because we don't take into account the curvature of the Earth.
+
+We can already apply the concept of worst, average, and best case to this model.
+
+** 2.2 Big Oh
+
+The previous model often requires concrete implementations to actually measure
+correctly, so instead Big Oh gives us a better, simpler framework for discussing
+the relative performance between algorithms. It ignores factors that don't
+impact how algorithms scale.
+
+** 2.3 Growth Rates and Dominance Relations
+
+These are the functions that occur in algorithm analyses;
+
+- *Constant O(1)*
+    Hashtable look up, array look up, consing a list
+- *Logarithmic O(log n)*
+    Binary Search
+- *Linear O(n)*
+    Iterating over a list
+- *Superlinear O(n log n)*
+    Quicksort and Mergesort
+- *Quadratic* O(n^{2})
+    Insertion Sort and Selection Sort
+- *Cubic* O(n^{3})
+    Some dynamic programming problems
+- *Exponential* O(C^{n}^{}) *c for any constant c > 1*
+    Enumerate all subsets
+- *Factorial O(n!)*
+    Generating all permutations or orderings
+
+*Notes*:
+- O(n!) algorithms become useless for anything n >= 20
+- O(2^{n}) algorithms become impractical for anything n > 40
+- O(n^{2}^{}) algorithms start deteriorating after n > 10,000, a million is hopeless
+- O(n^{2}^{}) and O(n log n) Are fine up to 1 billion
+
+** 2.4 Working with Big Oh
+
+Apparently you can do arithmetic on the Big Oh functions
+
+** 2.5 Efficiency
+
+*** Selection Sort
+**** C
+
+#+begin_src C :includes stdio.h
+void print_nums(int *nums, int length) {
+    for (int i = 0; i < length; i++) {
+        printf("%d,", nums[i]);
+    }
+    printf("\n");
+}
+
+void selection_sort(int *nums, int length) {
+    int i, j;
+    int min_idx;
+    for (i = 0; i < length; i++) {
+        print_nums(nums, length);
+        min_idx = i;
+        for (j = i+1; j < length; j++) {
+            if (nums[j] < nums[min_idx]) {
+                min_idx = j;
+            }
+        }
+        int temp = nums[min_idx];
+        nums[min_idx] = nums[i];
+        nums[i] = temp;
+    }
+}
+
+int nums[9] = { 2, 4, 9, 1, 3, 8, 5, 7, 6 };
+selection_sort(nums, 9);
+#+end_src
+
+#+RESULTS:
+| 2 | 4 | 9 | 1 | 3 | 8 | 5 | 7 | 6 |   |
+| 1 | 4 | 9 | 2 | 3 | 8 | 5 | 7 | 6 |   |
+| 1 | 2 | 9 | 4 | 3 | 8 | 5 | 7 | 6 |   |
+| 1 | 2 | 3 | 4 | 9 | 8 | 5 | 7 | 6 |   |
+| 1 | 2 | 3 | 4 | 9 | 8 | 5 | 7 | 6 |   |
+| 1 | 2 | 3 | 4 | 5 | 8 | 9 | 7 | 6 |   |
+| 1 | 2 | 3 | 4 | 5 | 6 | 9 | 7 | 8 |   |
+| 1 | 2 | 3 | 4 | 5 | 6 | 7 | 9 | 8 |   |
+| 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 |   |
+
+
+*** Insertion Sort
+**** C
 
 #+begin_src C :includes stdio.h
 void insertion_sort(int *nums, int len) {
@@ -36,66 +195,37 @@ for (int i = 0; i < 8; i++) {
 #+RESULTS:
 : 12345789
 
-*** 1.1 Robots
-The Robot arm problem is presented where it is trying to solder contact points
-and visit all points in the shortest path possible.
 
-The first algorithm considered is ~NearestNeighbor~. However this is a naïve, and
+*** TODO String Pattern Matching
-the arm hopscotches around
+*** TODO Matrix Multiplication
 
-Next we consider ~ClosestPair~, but that too misses in certain instances.
+** 2.6 Logarithms
 
-Next is ~OptimalTSP~, which will always give the correct result because it
+Logarithms are the inverse of exponents. Binary search is great for sorted
-enumerates all possible combinations and return the one with the shortest
+lists. There are applications related to fast exponentiation, binary trees,
-path. For 20 points however, the algorithm grows at a right of O(n!). TSP stands
+harmonic numbers, and criminal sentencing.
-for Traveling Salesman Problem.
 
-**** TODO Implement NearestNeighbor
+** 2.7 Properties of Logarithms
-**** TODO Implement ClosestPair
-**** TODO Implement OptimalTSP for N < 8
 
-*** 1.2 Right Jobs
+Common bases for logarithms include 2, /e/, and 10. The base of the logarithm has
-Here we are introduced to the Movie Scheduling Problem where we try to pick the
+no real impact on the growth rate; log_{2} and log_{3} are roughly equivalent.
-largest amount of mutually non-overlapping movies an actor can pick to maximize
-their time. Algorithms considered are ~EarliestJobFirst~ to start as soon as
-possible, and then ~ShortestJobFirst~ to be done with it the quickest, but both
-fail to find optimal solutions.
 
-~ExhaustiveScheduling~ grows at a rate of O(2^{n}) which is much better than O(n!) as
+** 2.8 War Story Pyramids
-in the previous problem. Finally ~OptimalScheduling~ improves efficiency by first
-removing candidates that are overlapping such that it doesn't even compare them.
 
-*** 1.3 Correctness
+Cool story bro
 
-It's important to be clear about the steps in pseudocode when designing
+** 2.9 Advanced Analysis
-algorithms on paper. There are important things to consider about algorithm
-correctness;
 
-- Verifiability
+Some advanced stuff
-- Simplicity
+- *Inverse Ackerman's Function*
-- Think small
+  Union-Find data structure
-- Think exhaustively
+- *log log n*
-- Hunt for the weakness
+  Binary search on a sorted array of only log n items
-- Go for a tie
+- *log n / log log n*
-- Seek extremes
+- log^{2} n
+- \sqrt{,}n
 
-Other tecniques include *Induction*, *Recursion*, and *Summations*.
+There are also limits and dominance relations
 
-*** 1.4 Modeling
+* Chapter 3
-
-Most algorithms are designed to work on rigorously defined abstract
-structures. These fundamental structures include;
-
-- Permutations
-- Subsets
-- Trees
-- Graphs
-- Points
-- Polygons
-- Strings
-
-*** 1.5-1.6 War Story about Psychics
-
-
-* Chapter 2