{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "provenance": []
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    },
    "language_info": {
      "name": "python"
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "source": [
        "# Lecture 5 - Binary Search & Recurrences\n",
        "\n",
        "We will discuss:\n",
        "\n",
        "1. Review of Slice and Dice algorithm\n",
        "2. Analysis - run time, correctness, space usage\n",
        "3. Issues with *slice_dice*\n",
        "4. Binary Search algorithm\n",
        "5. Analysis - run time, correctness, space usage\n",
        "6. Solving recurrences : Guess and check method"
      ],
      "metadata": {
        "id": "TS7qzounFlOt"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "# Slice-and-Dice (review)\n",
        "\n",
        "Main idea : To search through structured data, we can first check the middle position (median) of the sorted list. Depending on the result of the check:\n",
        "\n",
        " focus search on either left half or right half of the list!\n",
        "\n",
        " *   Case 1 -\n",
        "```\n",
        "if(li[n/2] == t): Search successful - nothing more to do!\n",
        "```\n",
        "\n",
        "*  Case 2 -\n",
        "```\n",
        "if(li[n/2] > t): we need to check li[0:(n/2)]  - total {n/2} elements left to check\n",
        "```\n",
        "\n",
        "*  Case 3 -\n",
        "```\n",
        "if(li[n/2] < t): we need to check li[(n/2)+1:] - total {(n/2)-1} elements left to check\n",
        "```\n",
        "\n",
        "Let's write some code!\n"
      ],
      "metadata": {
        "id": "SGjGr-UDH5En"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "li = [1, 4, 11, 18, 24, 36, 39, 41, 47, 59, 60, 62, 65, 71, 77, 88]\n",
        "\n",
        "#function slice_dice that slices search space in half at each step until target is found\n",
        "def slice_dice(l,t):\n",
        "\n",
        "  k = len(l)\n",
        "\n",
        "  #important base case\n",
        "  if(k == 1):\n",
        "    if(l[0] == t):\n",
        "      return \"Element is present in the list\"\n",
        "    else:\n",
        "      return \"Element is not in the list\"\n",
        "\n",
        "  if(l[k//2] == t):\n",
        "     return \"Element is present in the list\"\n",
        "\n",
        "  elif(l[k//2] > t):\n",
        "    return (slice_dice(l[0:k//2],t))\n",
        "\n",
        "  else:\n",
        "    return (slice_dice(l[(k//2)+1:],t))\n",
        "\n",
        "print(slice_dice(li,77))"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "EpjVB3JGIDg4",
        "outputId": "9c32b44c-5502-4cdd-d5b4-d134489ea221"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Element is present in the list\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "# Slice and Dice analysis\n",
        "\n",
        "\n",
        "1.   Run time analysis - How much time does *slice_dice* take on lists of size $n$?\n",
        "\n",
        "2.   Correctness - Does *slice_dice* return the correct answer on all input lists (and targets)?\n",
        "\n"
      ],
      "metadata": {
        "id": "BpQclzlWIVY1"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "# Run time analysis\n",
        "\n",
        "*   Computations performed -\n",
        "\n",
        "Each call of *slice_dice* entails :\n",
        "\n",
        "1.   Computing the length of the list - $O(1)$ time\n",
        "\n",
        "       (python stores list size with each list)\n",
        "\n",
        "2.   Accessing the middle element - $O(1)$ time\n",
        "\n",
        "      (python's lists store references to elements - these references are stored in consecutive memory locations so $O(1)$ access)\n",
        "\n",
        "3.   Comparisons - $O(1) * 4 = O(1)$ time\n",
        "\n",
        "        (comparisons for $k$ vs 1 once, median vs target 3 times)\n",
        "\n",
        "4.   Recursive call on sliced list - $O(n)$ time\n",
        "\n",
        "        (List slicing - elements copied into new instance - so it costs\n",
        "        size_of_slice operations)  \n",
        "\n",
        "\n",
        "\n",
        "> **How many total calls does *slice_dice* make?**\n",
        "\n",
        "(Python time complexities for standard tasks - [link](https://wiki.python.org/moin/TimeComplexity))\n"
      ],
      "metadata": {
        "id": "y7LVRgxRIYX-"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "# Recurrences\n",
        "\n",
        "Let $f(n)$ denote the total number of calls of *slice_dice* on any input of size $n$.\n",
        "\n",
        "We want to understand what $f(n)$ is?\n",
        "\n",
        "What do we know?\n",
        "\n",
        "On inputs of size $n$, *slice_dice* checks if median is equal to target and -\n",
        "\n",
        "*   if (median == target) : *slice_dice* terminates\n",
        "*   else: *slice_dice* calls *slice_dice* on input of length $n/2$.\n",
        "\n",
        "So, $$f(n) \\leq 1 + f(n/2)$$\n",
        "\n",
        "Congratulations, we have written our first recurrence relation!"
      ],
      "metadata": {
        "id": "fU6nkeawIeVg"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "# Solving recurrences\n",
        "\n",
        "1. Warmup -\n",
        "\n",
        "Solve the recurrence : $$T(n) = T(n-1) + 1$$\n",
        "\n",
        "\n",
        "*   What do you think the answer is?\n",
        "*   Do we have all the information we need to solve this recurrence?\n",
        "\n"
      ],
      "metadata": {
        "id": "bD2X4uoDOvNn"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "Solve the recurrence : $$T(n) = T(n-1) + 1$$\n",
        "\n",
        "\n",
        "\n",
        "  $$T(n) = T(n-1) + 1$$\n",
        "\n",
        "Substituting the value of $(n-1)$ in place of $n$:\n",
        "\n",
        "  $$T(n-1) = T(n-2) + 1$$\n",
        "\n",
        "Substituting the value of $(n-2)$ in place of $n$:\n",
        "\n",
        "  $$T(n-2) = T(n-3) + 1$$\n",
        "\n",
        "$\\hspace{13cm}$.\n",
        "\n",
        "$\\hspace{13cm}$.\n",
        "\n",
        "$\\hspace{13cm}$.\n",
        "\n",
        "$\\hspace{13cm}$.\n",
        "\n",
        "$\\hspace{13cm}$.\n",
        "\n",
        "\n",
        "\n",
        "  $$T(2) = T(1) + 1$$\n",
        "\n",
        "   $$T(1) = T(0) + 1$$\n",
        "\n",
        "   $$T(0) = T(-1) + 1$$\n",
        "\n",
        "   $$T(-1) = T(-2) + 1$$\n",
        "\n",
        "$\\hspace{13cm}$What???\n",
        "\n",
        "\n",
        "\n",
        "> Recurrences need **BASE cases**!!!\n",
        "\n",
        "\n",
        "\n"
      ],
      "metadata": {
        "id": "nfFZ0oAAPLlv"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "What is the base case here?\n",
        "\n",
        "Typically, we think of $T(1) = O(1)$ as the base case. Let us use that.\n",
        "\n",
        "Solve the recurrence : $$T(n) = T(n-1) + 1$$\n",
        "\n",
        "\n",
        "\n",
        " $$T(n) = T(n-1) + 1$$\n",
        "\n",
        "Substituting the value of $(n-1)$ in place of $n$:\n",
        "\n",
        "   $$T(n-1) = T(n-2) + 1$$\n",
        "\n",
        "Substituting the value of $(n-2)$ in place of $n$:\n",
        "\n",
        "  $$T(n-2) = T(n-3) + 1$$\n",
        "\n",
        "$\\hspace{13cm}$.\n",
        "\n",
        "$\\hspace{13cm}$.\n",
        "\n",
        "$\\hspace{13cm}$.\n",
        "\n",
        "$\\hspace{13cm}$.\n",
        "\n",
        "$\\hspace{13cm}$.\n",
        "\n",
        "\n",
        "  $$T(2) = T(1) + 1$$\n",
        "\n",
        "Adding up all these equations gives us -\n",
        "\n",
        "$$T(n) = T(1) + (n-1)$$\n",
        "\n",
        "Substituting $T(1) = O(1)$ gives -\n",
        "\n",
        "**$$T(n) = (n-1) + O(1) = O(n)$$**\n",
        "\n",
        "---\n",
        "\n",
        "\n",
        "\n",
        "\n",
        "> Solve the recurrence $$T(n) = T(n-2) + 1$$ given $T(1) = O(1)$.\n",
        "\n"
      ],
      "metadata": {
        "id": "wyZ7BDzeSODQ"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "# Counting *slice_dice* recursive calls:\n",
        "\n",
        "Denote the number of *slice_dice* calls on $n$ sized input lists : $f(n)$\n",
        "\n",
        "Recurrence - $$f(n) \\leq 1 + f(n/2)$$\n",
        "\n",
        "Substituting $n$ with $n/2$ gives us :\n",
        "\n",
        "$$f(n/2) \\leq 1 + f(n/4)$$\n",
        "\n",
        "Going further, we have:\n",
        "\n",
        "$$f(n/4) \\leq 1 + f(n/8)$$\n",
        "\n",
        "\n",
        "$\\hspace{13cm}$.\n",
        "\n",
        "$\\hspace{13cm}$.\n",
        "\n",
        "$\\hspace{13cm}$.\n",
        "\n",
        "$\\hspace{13cm}$.\n",
        "\n",
        "$\\hspace{13cm}$.\n",
        "\n",
        "$$f(2) \\leq 1 + f(1)$$\n",
        "\n",
        "At this point, we know that $f(1) = 1$, since *slice_dice* on inputs of size 1 corresponds to the base case which gets resolved without further calls to *slice_dice*.\n",
        "\n",
        "\n",
        "> Question : In how many steps did we go from $f(n)$ to $f(1)$?\n",
        "\n",
        "\n",
        "\n",
        "*   After $i$ steps, the input size is $\\frac{n}{2^{i}}$. Check for $i = 0,1,2..$\n",
        "\n",
        "*   At the end, we have input size 1, which gives us: $$\\frac{n}{2^{i}} = 1 \\Rightarrow 2^{i} = n \\Rightarrow i = \\log_{2}{n}$$\n",
        "\n",
        "\n",
        "> $n$ has to be halved $\\log_{2}n$ times to reach 1.  \n",
        "\n",
        "\n",
        "\n",
        "> *slice_dice* makes a total of at most $(\\log_{2}{n} + 1)$ recursive calls on inputs of size $n$.\n",
        "\n",
        "\n",
        "\n",
        "\n",
        "\n",
        "\n",
        "\n"
      ],
      "metadata": {
        "id": "bQN2pRria1Bo"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "# Run time analysis for *slice_dice*\n",
        "\n",
        "\n",
        "\n",
        "*   No. of recrusive calls - $(\\log_{2}{n} + 1)$\n",
        "*   Time spent in each call - $O(1) + (\\text{size_of_slice}) = O(n)$\n",
        "\n",
        "\n",
        "*   Let $T(n)$ denote the time taken by *slice_dice* on inputs of length $n$. Then : $$T(n) = O(n) + T(n/2)$$\n",
        "\n",
        "$\\hspace{10cm}$ with $T(1) = O(1)$\n",
        "\n",
        "\n",
        "> We will establish the solution for this recurrence later. What do you expect it to be?"
      ],
      "metadata": {
        "id": "YgJe4pj41So3"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "# Issues with Slice and Dice algorithm\n",
        "\n",
        "  **Run time**\n",
        "\n",
        " The first call to *slice_dice* entails slicing the list in half - which already takes $O(n)$ time. (This is no better than *linear_search* which also took $O(n)$ time)\n",
        "\n",
        "\n",
        " **Index finding**\n",
        "\n",
        " Due to the nature of the recursive calls, *slice_dice* cannot return the index of the element (if it exists). This renders it ineffective in providing access to the element that is found.  "
      ],
      "metadata": {
        "id": "sg5-OJyM_qPr"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "# How to fix these issues?\n",
        "\n",
        "1. In order to fix run time, we must avoid slicing the list in half. So, we can instead just call binary search on the original list itself - but by specifying start and stop positions that simulate *halving*.\n",
        "\n",
        "2. Since we keep the original list as is, all we need to do is monitor the index of the median at each step so that we can return it when the search is successful.\n",
        "\n",
        "Let's code this up!\n",
        "\n"
      ],
      "metadata": {
        "id": "juTMucYzL-RT"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "# Binary Seach\n",
        "\n",
        "Idea - The main idea is essentially the same as before - check the median against the target and based on the outcome of repeat the process on either the left or right half (or terminate if successful). The main difference here is the implementation.  "
      ],
      "metadata": {
        "id": "o70nJBiBBkyO"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "#Use this skeletal structure to code up binary search to practice coding!\n",
        "\n",
        "def binary_search(list,target,start,stop):\n",
        "\n",
        "  #Base cases - what are the base cases? for each one, give the right answer!\n",
        "\n",
        "\n",
        "  #Other cases\n",
        "\n",
        "  #check median against target value - what is the median element?\n",
        "  #if (median > target) what do you do next - be precise\n",
        "  #if (median < target) what do you do next - be precise\n",
        "  #if you find the target, how do you return its index? be precise\n",
        "\n",
        "  #for the recursive calls, list and target don't change - so you must modify start and stop appropriately"
      ],
      "metadata": {
        "id": "mGSP0LS0ZIWM"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "li = [1, 6, 7, 8, 19, 21, 24, 30, 32, 37, 39, 40, 47, 58, 60, 61, 65, 67, 74, 76, 78, 79, 80, 81, 86, 87, 90, 91, 94, 97]\n",
        "\n",
        "target = 77\n",
        "\n",
        "# function binary search that slices search space in half at each step until target is found\n",
        "def binary_Search(l,t,start,stop):\n",
        "\n",
        "  if(start > stop):\n",
        "    return -1\n",
        "\n",
        "  else:\n",
        "    mid = (stop - start)//2\n",
        "\n",
        "    if(l[start+mid] == t):\n",
        "      return (start+mid)\n",
        "\n",
        "    elif(l[start+mid] > t):\n",
        "      return binary_Search(l,t,start,start+mid-1)\n",
        "\n",
        "    else:\n",
        "      return binary_Search(l,t,start+mid+1,stop)\n",
        "\n",
        "print(binary_Search(li,target,0,len(li)-1))"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "t42NgVbWDM0N",
        "outputId": "06265cfd-9cf5-4300-c408-4b89d0a7a3fb"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "-1\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "# Analysis\n",
        "\n",
        "**Run time**\n",
        "\n",
        "Computations performed within each recursive call -\n",
        "\n",
        "*   Base case check + return - $O(1) + O(1) = O(1)$\n",
        "\n",
        "*   Computing median - $O(1)$\n",
        "\n",
        "*   Checking median element against target - $O(1) \\times 2 = O(1)$\n",
        "\n",
        "*   Next recursive call - $O(1)$\n",
        "\n",
        "*   Total time within each call - $O(1) + O(1) + O(1) + O(1) = O(1)$\n",
        "\n",
        "Since the total number of recursive calls is $(\\log_{2}{n} + 1)$, the total time taken is $O(1) \\times (\\log_{2}{n}+1) = O(\\log_{2}{n})$\n",
        "\n",
        "\n",
        "\n",
        "> Binary search searches through any sorted $n$ sized list in $O(\\log n)$ time.\n",
        "\n"
      ],
      "metadata": {
        "id": "RbYykiENNi3D"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "# Correctness (requirements)\n",
        "\n",
        "*   We want to claim : *binary_Search* works correctly on lists of all sizes.\n",
        "*   First, does it work correctly on inputs of size 0?\n",
        "\n",
        "*   What about size 1?\n",
        "   \n",
        "*   What about size 7?\n",
        "\n"
      ],
      "metadata": {
        "id": "j_66QuxVLqam"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "# Correctness (strategy)\n",
        "\n",
        "We will prove the correctness of *binary_Search* using mathematical induction.\n",
        "\n",
        "   Let $P(i)$ denote the statement that *binary_Search* works correctly on all inputs of length $i$. In order to show that $P(i)$ holds for all $i \\in \\mathbb{N}$, we will -\n",
        "\n",
        "1.   Base case(s) - Prove that $P(0)$ holds.\n",
        "2.   Prove that $P(0),P(1),P(2),...,P(k) ⇒ P(k+1)$ for some arbitrary integer $k$."
      ],
      "metadata": {
        "id": "MNKH7pAPobbq"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "# Correctness (proof)\n",
        "\n",
        "Base cases -\n",
        "\n",
        "\n",
        "*   If $i = 0$, the list is an empty list. In this case, *binary_Search* is called with $(stop = -1)$ and since $(start < stop)$, *binary_Search* correctly returns that $target$ is not in the list.\n",
        "\n",
        "The induction hypothesis allows us to assume that $P(i)$ is true for all $0 \\leq i \\leq k$.\n",
        "\n",
        " Now, the induction step where we show that if $P(i)$ is true for all $0 \\leq i \\leq k$, i.e. that *binary_Search* works correctly on all inputs of size up to $k$, then $P(k+1)$ is true.\n",
        "\n",
        " On any input of size $(k+1)$, *binary_Search* compares the median against the target -\n",
        "\n",
        "*  if $(median == target)$, it correctly returns that the element is found.\n",
        "\n",
        "*  if $(median > target)$, *binary_Search* calls itself on $(list, target, median+1, stop)$.\n",
        "\n",
        "The list cannot contain the target in any index preceding the median since it is sorted. The target can only exist in elements succeeding the median - positions $(median + 1)$ through $(stop)$. At this point, *binary_Search* works correctly if and only if the recursive call on $(list, target, median+1, stop)$ works. By our **induction hypothesis**, this recursive call works correctly. So, we can claim that this case is dealt with correctly by *binary_Search*.\n",
        "\n",
        "A similar argument holds for the other case.\n",
        "\n",
        "We have proved that for all possible results of $median$ vs $target$ on lists of size $(k+1)$, *binary_Search* returns the correct answer.\n",
        "\n",
        "So, we have shown that if $P(i)$ holds for all $i \\leq k$, then $P(k+1)$ holds.\n",
        "\n",
        "By the principle of mathematical induction, *binary_Search* works correctly on all input lengths.\n"
      ],
      "metadata": {
        "id": "qKpTDShtIWPE"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "# Binary Search vs Linear Search\n",
        "\n",
        "\n",
        "\n",
        "*   Sorted data - Binary search is extremely efficient : $O(\\log n)$ time.\n",
        "\n",
        "*   Unsorted data - Binary search fails since data isn't sorted. Since data provides no guarantees, we cannot avoid looking at all items in the worst case : $O(n)$ time.\n"
      ],
      "metadata": {
        "id": "z1E1osOnWC5g"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "# Recurrences\n",
        "\n",
        "Assume that $T(1) = O(1)$. Solve:\n",
        "\n",
        "\n",
        "1. $$T(n) = T(n-1) + n$$   \n",
        "\n",
        "\n",
        "\n",
        "2.  $$T(n) = T(n-1) + n^{2}$$\n",
        "\n",
        "\n",
        "3. $$T(n) = 2 T(n/2) + 1$$\n",
        "\n"
      ],
      "metadata": {
        "id": "0aAsrE9claPQ"
      }
    }
  ]
}