{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "provenance": []
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    },
    "language_info": {
      "name": "python"
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "source": [
        "# Lecture 6 - Solving recurrences (Guess & Check method)\n",
        "\n",
        "\n",
        "In this lecture, we will :\n",
        "*   Review Binary Search\n",
        "*   Review a few recurrences we have written\n",
        "*   Solve these recurrences using the Guess & Check method\n",
        "*   Write an induction proof for the correctness of a recurrence solution\n",
        "*   Explore the intuition behind recurrence solutions\n",
        "\n",
        "\n",
        "Reminder : Hw 1 due TONIGHT\n",
        "(Office hours 5pm-8pm @ Siebel Basement,  7pm-10pm @ Siebel 2322 + Zoom)\n",
        "\n",
        "Announcements -\n",
        "\n",
        "Lab question 3.4 - we are working on a resolution - announcement to Ed at some point\n",
        "\n",
        "Today's notebook will be shared after the lecture to the website\n"
      ],
      "metadata": {
        "id": "YCRkdRxkwgf2"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "# Binary Search review\n",
        "\n",
        "*   Binary search is a search algorithm that has performs search on sorted lists.\n",
        "\n",
        "*   Divide & Conquer - The Main idea is to divide the list into two halves and use the median's value to direct search.\n",
        "\n",
        "*   Binary search algorithm makes at most $O(\\log_{2}n)$ recursive calls - as the input size halves after each recursive call.\n",
        "\n",
        "*   The total running time of binary search is $O(log_{2}n)$ - and this is the **best possible run time** for searching through sorted lists.\n",
        "\n",
        "\n",
        "\n"
      ],
      "metadata": {
        "id": "z1h7b_iFxMxb"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "def binary_Search(l,t,start,stop) :\n",
        "\n",
        "  if(start > stop):\n",
        "    return -1\n",
        "\n",
        "  else:\n",
        "    mid = (stop - start)//2\n",
        "\n",
        "    if(l[start+mid] == t):\n",
        "      return (start+mid)\n",
        "\n",
        "    elif(l[start+mid] > t):\n",
        "      return binary_Search(l,t,start,start+mid-1)\n",
        "\n",
        "    else:\n",
        "      return binary_Search(l,t,start+mid+1,stop)\n",
        "\n",
        "li = [1, 6, 7, 8, 19, 21, 24, 30, 32, 37, 39, 40, 47, 58, 60, 61, 65, 67, 74, 77, 78, 79, 80, 81, 86, 87, 90, 91, 94, 97]\n",
        "target = 77\n",
        "print(binary_Search(li,target,0,len(li)-1))"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "Az8hCQcCyhQ9",
        "outputId": "d931a265-eda2-4e54-9119-3e7b76c436dd"
      },
      "execution_count": 4,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "19\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "# Run time analysis\n",
        "\n",
        "\n",
        "Let $T(n)$ denote the running time of *binary_Search* on inputs of size $n$ (that is, lists with $n$ elements).\n",
        "\n",
        "*   We want to find out what $T(n)$ is.\n",
        "*   On any input of size $n$, the function *binary_Search* :\n",
        "\n",
        "1. first checks the if condition in $O(1)$ time\n",
        "\n",
        "2. Then, if needed, computes the value of mid in $O(1)$ time.\n",
        "\n",
        "3. Next, it checks median value against the target (potentially 2 times) before making another recusive call. (time spent is $O(1)$)\n",
        "\n",
        "4. The recursive call to *binary_Searc* is made on a list of size at most $n/2$. How long does this call take? Well, $T(n/2)$ since that's how long *binary_Search* spends on an input list of size $n/2$!\n",
        "\n",
        "What we have so far is -\n",
        "\n",
        "$$T(n) \\leq O(1) + O(1) + O(1) + T(n/2)$$\n",
        "which is\n",
        "$$T(n) \\leq T(n/2) + O(1)$$\n",
        "\n",
        "Now, we will solve the recurrence using the Guess & Check method!\n",
        "\n",
        "\n"
      ],
      "metadata": {
        "id": "iohKtzbrzCXI"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "# Guess and check method for solving recurrences\n",
        "\n",
        "\n",
        "*   As the name suggests, we first guess a solution to the recurrence.\n",
        "*   Then, we plug that value into the recurrence and check if that works.\n",
        "*   If it does, we are done! If not, we guess a different solution and check again.\n",
        "*   We repeat this process until our guess works.\n",
        "\n"
      ],
      "metadata": {
        "id": "4c--nUyP1c8S"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "# Guess 1 : $T(n)$ $= n$\n",
        "\n",
        "In order to solve $$T(n) \\leq T(n/2) + O(1)$$let us first guess $T(n) = n$. Plugging this into the recurrence gives us :   \n",
        "\n",
        "$$n \\leq n/2 + 10$$\n",
        "\n",
        "which is not valid for large values of $n$. Have we learned anything?\n",
        "\n",
        "*   $T(n) = n$ left the inequality invalid since the RHS is lesser than the LHS.\n",
        "*   Gap between LHS and RHS is $n/2$ (roughly speaking).\n",
        "*   This suggests that we should try a smaller value for $T(n)$ to bridge the gap.\n",
        "\n"
      ],
      "metadata": {
        "id": "1YtKbBBC2ALL"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "# Guess 2 : $T(n)$ $= \\sqrt{n}$\n",
        "\n",
        "Plugging this in gives us :  \n",
        "\n",
        "$$\\sqrt{n} \\leq \\sqrt{\\frac{n}{2}} + O(1)$$\n",
        "\n",
        "which is not valid for large values of $n$.\n",
        "\n",
        "\n",
        "*   $T(n) = \\sqrt{n}$ left the inequality invalid since the RHS continues to be lesser than the LHS.\n",
        "*   However, there is progress since the gap between LHS and RHS is now down to $\\sqrt{\\frac{n}{2}}$.\n",
        "*   This suggests that we must continue to decrease $T(n)$.\n",
        "\n",
        "\n"
      ],
      "metadata": {
        "id": "hdRR3X0n20Mq"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "# Guess 3 : $T(n)$ $= log_{2}{n}$\n",
        "\n",
        "Plugging in $T(n) = \\log_{2}{n}$ gives us :     \n",
        "$$\\log_{2}{n} \\leq \\log_{2}{n/2} + O(1) = (\\log_{2}{n} - 1) + O(1) \\sim \\log_{2}{n}$$\n",
        "\n",
        "which works out almost perfectly!\n",
        "\n",
        "\n",
        "*   Our guess $T(n) = \\log_{2}{n}$ has **almost** worked out as the RHS and LHS are now within $O(1)$.\n",
        "\n",
        "\n",
        "(Well, to be fair, we should have tried $n^{1/3}$, $n^{1/4}$ etc. but we jumped ahead to $\\log_{2}{n}$ since we had a \"feeling\" it had something to do with $\\log_{2}{n}$. It worked out so we saved ourselves some time.)\n",
        "\n",
        "All we are left with is the $O(1)$ gap. Typically, this indicates that our asymptotics are in place - and we just need to modify lower order terms.\n"
      ],
      "metadata": {
        "id": "BJT5tarI4Xwx"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "# Binary search recurrence resolution\n",
        "\n",
        "Let us refine our guess to be $$T(n) \\leq a \\log_{2}n + 100$$ for all $n \\geq 1$ and some constant $a$ (to be fixed later)\n",
        "\n",
        "We will now use mathematical induction to prove this statement formally.\n",
        "\n",
        "Let $P(i)$ denote the statement that the run time of *binary_Search* on $i$ sized inputs is at most $a \\log_{2}{i} + 100$.\n",
        "\n",
        "Base case : $i = 1$\n",
        "\n",
        "On inputs of size 1, *binary_Search* executes at most 5 basic steps. Since this is less than $a \\log_{2}{1} + 100 = 0 + 100 = 100$, $P(1)$ holds true.\n",
        "\n",
        "Next, Inductive hypothesis -  $P(i)$ is true for all $1 \\leq i \\leq (k-1)$.\n",
        "\n",
        "Inductive step - We need to now prove that $P(k)$ is true.\n",
        "\n",
        "Case : $i = k$\n",
        "\n",
        "On inputs of size $k$, *binary_Search* spends $O(1)$ time before making a recurisve call to *binary_Search* on input of size $k/2$. So, total time taken is $O(1) + T(k/2) \\leq O(1) + a (\\log_{2}{\\frac{k}{2}}) + 100 = a \\log_{2}{k} - a + 100 + O(1)$.\n",
        "\n",
        "**This is because $P(k/2)$ is true**\n",
        "\n",
        "For the induction hypothesis to be true, we need :      \n",
        "$$a \\log_{2}{k} - a + 100 + O(1) \\leq a \\log_{2}{k} + 100$$\n",
        "\n",
        "In other words, $a$ has to be larger than the $O(1)$ term. If we assume that the $O(1)$ term was (say) 10, then we need $a \\geq 10$. Let's take $a = 10$.\n",
        "\n",
        "This proves that if $P(i)$ is true for all $1 \\leq i \\leq (k-1)$, then $P(k)$ is true.\n",
        "\n",
        "Since $P(1)$ is true and $P(1),P(2),...,P(k-1) ⇒ P(k)$, by the principle of mathemtical induction, $P(i)$ is true for all $i \\geq 1$.\n",
        "\n",
        "\n",
        "\n",
        "> We can conclude that $T(n) \\leq 10 \\log_{2}{n} + 100$.\n",
        "\n",
        "\n"
      ],
      "metadata": {
        "id": "PSCEzUHJ-sUj"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "# Examples\n",
        "\n",
        "1. Solve the recurrence $$T(n) = T(n-1) + n$$\n",
        "\n",
        "---\n",
        "\n",
        "Guess & Check method -\n",
        "\n",
        "Guess 1 : $T(n) = n$\n",
        "\n",
        "Plugging this in gives us :$$n = (n-1) + n = 2n - 1$$\n",
        "\n",
        " which is invalid for $n \\geq 1$. In this case, the RHS is larger than the LHS; so we should increase the LHS to bridge the gap - which is currently $(n-1)$.\n",
        "\n",
        "---\n",
        "Guess 2 : $T(n) = n \\log_{2}{n}$\n",
        "\n",
        "Plugging this in gives us :$$n \\log_{2}{n} = (n-1) \\log_{2}(n-1) + n = n \\log_{2}(n-1) - \\log_{2}(n-1) + n$$\n",
        "\n",
        "which is not valid for large values of $n$ since $n \\log_{2}(n-1) + n - \\log_{2}(n-1) \\sim n \\log_{2}(n) + n - \\log_{2}(n) \\sim n\\log_{2}{n} + n > n \\log_{2}{n}$\n",
        "\n",
        "RHS is still larger than LHS by roughly $n - log_{2}{n}$ - so, we have made progress but the gap is still too large. Let us increase $T(n)$ further!\n",
        "\n",
        "---\n",
        "Guess 3 : $T(n) = n^2$\n",
        "\n",
        "Plugging this in gives us : $$ n^{2} = (n-1)^{2} + n = n^{2} -2n + 1 + n = n^{2} - n + 1 $$\n",
        "\n",
        "which is not valid for $n \\geq 1$.\n",
        "\n",
        "In this case, the LHS is larger than the RHS, so we have somewhat overshot. Observe that the quadratic terms match up alright, so we can modify the lower order terms next.\n",
        "\n",
        "---\n",
        "\n",
        "Guess 4 : $T(n) = an^{2} + bn + c $\n",
        "\n",
        "Plugging this in gives us  : $$an^{2} + bn + c = [a(n-1)^{2} + b(n-1) + c] + n$$\n",
        "\n",
        "simplifying,\n",
        "\n",
        "$$an^{2} + bn + c = an^{2} + (-2a+b+1) n + (a - b + c)  $$\n",
        "\n",
        "which holds when $b = -2a + b + 1$ and $a = b$. Solving this gives us $a = \\frac{1}{2}$, $b = \\frac{1}{2}$ and $c = 0$.\n",
        "\n",
        "  \n",
        "\n",
        "> $T(n) = \\frac{n^{2}}{2} + \\frac{n}{2}$ is the solution to $T(n) = T(n-1) + n$\n",
        "\n",
        "\n",
        "\n"
      ],
      "metadata": {
        "id": "M4eMTYzHCzOe"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "# Guess and Check (example 2)\n",
        "\n",
        "Solve the recurrence  -\n",
        "\n",
        "$$T(n) = 2 T(n/2) + 1$$\n",
        "\n",
        "using the guess and check method.\n",
        "\n",
        "---\n",
        "\n",
        "Guess 1 - $T(n) = n^2$\n",
        "\n",
        "Plugging this in gives us, $$n^{2} = 2 (n/2)^{2} + 1$$\n",
        "simplifying,\n",
        "$$n^{2} = n^{2}/2 + 1$$\n",
        "which is not true for $n \\geq 2$.\n",
        "\n",
        "What next?\n",
        "\n",
        "\n",
        "Note that in this case, the LHS is larger than the RHS. So, we must ??\n",
        "\n",
        "---\n",
        "Guess 2 - $T(n) = \\sqrt{n}$\n",
        "\n",
        "Plugging it in gives us,\n",
        "\n",
        "$$\\sqrt{n} = 2 \\sqrt{n/2} + 1$$\n",
        "simplifying,\n",
        "\n",
        "$$\\sqrt{n} = \\sqrt{2}\\sqrt{n} + 1$$\n",
        "\n",
        "which is not true for $n \\geq 0$. In this case, the LHS is smaller than the RHS. So, we must increase the LHS!\n",
        "\n",
        "---\n",
        "Guess 3 - $T(n) = n$\n",
        "\n",
        "Plugging it in gives us,\n",
        "$$n = 2 (n/2) + 1$$\n",
        "\n",
        "which is almost true! The LHS and RHS differ by just 1. Indeed, if we use the general form\n",
        "\n",
        "$$T(n) = an + b$$\n",
        "\n",
        "we get,\n",
        "\n",
        "$$an + b = 2(a(n/2) + b) + 1$$\n",
        "\n",
        "which is true for $b = -1$ and any constant $a > 0$.\n",
        "\n",
        "That is, $T(n) = O(n)$."
      ],
      "metadata": {
        "id": "nCDIPXH1Ic8K"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "# Guess and Check (example 3)\n",
        "\n",
        "Solve the recurrence -\n",
        "\n",
        "$$T(n) = 2 T(n-1) + 1$$\n",
        "\n",
        "Guess 1 : $T(n) = n$\n",
        "\n",
        "Plugging this gives us:\n",
        "$$n = 2(n-1) + 1$$\n",
        "\n",
        "which is invalid. The RHS is greater than the LHS, and the gap is $(n-1)$. We will guess a higher value of $T(n)$.\n",
        "\n",
        "---\n",
        "\n",
        "Guess 2 : $T(n) = n^{2}$\n",
        "\n",
        "Plugging this gives us:\n",
        "$$n^{2} = 2 [(n-1)^{2}] + 1 = 2n^{2} - 4n + 3$$\n",
        "\n",
        "\n",
        "which is invalid when $n$ is large. The RHS is greater than the LHS, and the gap is now $n^{2} - 4n + 3$ - a lot higher than before!\n",
        "\n",
        "---\n",
        "\n",
        "Guess 3 : $T(n) = n^{3}$\n",
        "\n",
        "Plugging this gives us:\n",
        "$$n^{3} = 2 [(n-1)^{3}] + 1 = 2n^{3} - 6n^{2} + 6n - 2 + 1 = 2n^{3} - 6n^{2} + 6n - 1$$\n",
        "\n",
        "which is again invalid when $n$ is large. The RHS is still greater than the LHS, and the gap is now $n^{3} - 6n^{2} + 6n - 2$!\n",
        "\n",
        "---\n",
        "\n",
        "Guess 4 : $T(n) = 2^{n}$\n",
        "\n",
        "Plugging this in gives us,\n",
        "$$2^{n} = 2[2^{n-1}] + 1 = 2^{n} + 1$$\n",
        "\n",
        "which is almost correct and now the LHS is within $O(1)$ of the RHS. We just need to modify lower order terms.\n",
        "\n",
        "---\n",
        "\n",
        "Guess 5 : $T(n) = a2^{n} + b$\n",
        "\n",
        "Now we have : $$a2^{n} + b = 2[a2^{n-1}+b] + 1 = a2^{n} + 2b + 1 $$\n",
        "\n",
        "This equality always holds for $b = -1$. So, if we pick $a = 1$, we have :$$T(n) = 2^{n} - 1$$\n",
        "\n",
        "Note that any constant $a > 0$ would work in this case."
      ],
      "metadata": {
        "id": "QTe39sCVMmBL"
      }
    },
    {
      "cell_type": "code",
      "source": [],
      "metadata": {
        "id": "mdZZ70Pu2QB6"
      },
      "execution_count": null,
      "outputs": []
    }
  ]
}