{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "provenance": []
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    },
    "language_info": {
      "name": "python"
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "source": [
        "# Lecture 8 - Binary Search\n",
        "\n",
        "We will :\n",
        "\n",
        "1. Review the basic linear search algorithm\n",
        "2. Structured vs Unstructured data\n",
        "3. Implement an idea to search through structured data\n",
        "4. Analyze it and identify scope for improvement\n",
        "5. Implement Binary Search\n",
        "6. Analyze Binary search\n",
        "\n",
        "Learning Objectives -\n",
        "\n",
        "1. Exploiting structure in data to improve efficiency of tasks\n",
        "2. Writing recurrence relations to characterise runtimes of recursive algorithms\n",
        "3. Asmyptotic upper bounds for recurrences\n",
        "4. Identifying bottlenecks in algorithms (efficiency)\n",
        "5. Fixing them to lead to improved efficiency\n",
        "\n",
        "Announcements -\n",
        "\n",
        "1. Homework 2 out on PL (deadline - Sep 30)\n",
        "2. Lab 4 due this Thursday\n",
        "3. REGISTER for exam 1 slots (Prairietest)\n",
        "3. Office hours today 5pm-9pm (HYBRID) https://edstem.org/us/courses/102969/discussion/8226240\n",
        "\n",
        "(My office hours today 6-7pm @ Siebel 2322 + Zoom)"
      ],
      "metadata": {
        "id": "uK4kIBwsN-4R"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "import random\n",
        "\n",
        "# Creating an input list of random numbers\n",
        "# generating a random list of size 30 in the range 1 to 100 (100 excluded)\n",
        "x = 30\n",
        "li = random.sample(range(1,100),x)\n",
        "print(li)\n",
        "\n",
        "#basic search algorithm\n",
        "def basic_search(l,t):\n",
        "    for i in range(len(l)):\n",
        "      if(l[i] == t):\n",
        "        return (\"Yes\")\n",
        "    return (\"No\")\n",
        "\n",
        "basic_search(li,77)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 53
        },
        "collapsed": true,
        "id": "jcwr_WXHG_Vo",
        "outputId": "8c638348-2a67-4b74-affb-8c880a43599e"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "[65, 55, 81, 24, 43, 87, 31, 93, 75, 61, 83, 6, 22, 91, 92, 7, 49, 48, 40, 5, 3, 8, 21, 72, 9, 47, 88, 30, 17, 26]\n"
          ]
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "'No'"
            ],
            "application/vnd.google.colaboratory.intrinsic+json": {
              "type": "string"
            }
          },
          "metadata": {},
          "execution_count": 80
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "> The asymptotic run time of *basic_search* is $O(n)$.\n"
      ],
      "metadata": {
        "id": "CMwI31gpTLth"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "# Structured vs Unstructured data\n",
        "\n",
        "Let us try to search for a target in 2 different contexts -\n",
        "\n",
        "1. Unstructured data\n",
        "2. Structured data"
      ],
      "metadata": {
        "id": "pxDvce2xX-WL"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# Looking for element 77 in lists li, li2\n",
        "import random\n",
        "\n",
        "x = 30\n",
        "li = random.sample(range(1,100),x)\n",
        "print(\"Unstructured list : \")\n",
        "print(li)\n",
        "print(\"\")\n",
        "\n",
        "\n",
        "li.sort()\n",
        "# The above line sorts the list li - we will see more about sorting next week\n",
        "print(\"Structured list : \")\n",
        "print(li)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "KasTpvWlwoMc",
        "outputId": "dd258149-22f6-46a6-918e-ae38dfaf9f40"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Unstructured list : \n",
            "[56, 60, 45, 72, 87, 25, 20, 30, 50, 94, 89, 14, 64, 51, 1, 55, 63, 47, 36, 17, 65, 27, 61, 33, 18, 41, 76, 71, 43, 40]\n",
            "\n",
            "Structured list : \n",
            "[1, 14, 17, 18, 20, 25, 27, 30, 33, 36, 40, 41, 43, 45, 47, 50, 51, 55, 56, 60, 61, 63, 64, 65, 71, 72, 76, 87, 89, 94]\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "# Structured vs Unstructured Data\n",
        "\n",
        "\n",
        "\n",
        "*   If data is structured (in this case sorted), then there are more efficient ways to search for an element.\n",
        "\n",
        "*   Main Idea - Pick some location and check for target. If the check fails, you can exclude one chunk of the list entirely. Now, search for the element in the remaining chunk.\n",
        "\n",
        "\n",
        "\n",
        "> **Which location should we pick to search within the list?**"
      ],
      "metadata": {
        "id": "NgPcxKcy0n8o"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "# Search algorithm for structured data\n",
        "\n",
        "Candidate position : 5\n",
        "\n",
        "\n",
        "*   Case 1 -\n",
        "```\n",
        "if(L[5] == t): Search successful - nothing more to do!\n",
        "```\n",
        "\n",
        "*  Case 2 -\n",
        "```\n",
        "if(L[5] > t): we need to check L[0:5]  - total 5 elements left to check\n",
        "```\n",
        "\n",
        "*  Case 3 -\n",
        "```\n",
        "if(L[5] < t): we need to check L[5:] - total (n-6) elements left to check\n",
        "```\n",
        "\n",
        "Either we find the element or we eliminate the left/right section of the list.\n",
        "\n",
        "Best case - we find the element!\n",
        "\n",
        "Worst case - we have to search through $n-6$ more elements."
      ],
      "metadata": {
        "id": "316KfSuU3oCE"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# Idea to search through sorted data - skip to position 5!\n",
        "\n",
        "# ALL print statements are only to visualize running of algorithm - IGNORE them for time complexity calculations\n",
        "\n",
        "li = [1, 6, 7, 8, 19, 21, 24, 30, 32, 37, 39, 40, 47, 58, 60, 61, 65, 67, 74, 77, 78, 79, 80, 81, 86, 87, 90, 91, 94, 97]\n",
        "li2 = [3, 7, 9, 14, 17, 19, 21, 27, 33, 35, 37, 38, 42, 44, 45, 46, 51, 52, 53, 59, 64, 68, 70, 73, 79, 80, 81, 87, 92, 99]\n",
        "\n",
        "\n",
        "def pos5_search_sorted(L,t):\n",
        "\n",
        "  print(\"Search space : \")\n",
        "  print(L)\n",
        "\n",
        "  if(len(L) <= 5):\n",
        "    for i in range(0,5):\n",
        "      if(L[i] == t):\n",
        "        return \"Yes\"\n",
        "    return \"No\"\n",
        "\n",
        "  print(\"element to compare against target : \", L[5])\n",
        "  print(\"\")\n",
        "\n",
        "  if(L[5] == t):\n",
        "    return(\"Yes\")\n",
        "\n",
        "  elif(L[5] > t):\n",
        "    #search through first 5 elements - L[0:5]\n",
        "    return pos5_search_sorted(L[0:5],t)\n",
        "\n",
        "  elif(L[5] < t):\n",
        "    # search through last n-6 elements\n",
        "    return pos5_search_sorted(L[6:],t)\n",
        "\n",
        "pos5_search_sorted(li,77)\n",
        "\n"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 357
        },
        "id": "uMwK6l6_a6Ob",
        "outputId": "b4276a9e-f9b6-4a72-ef75-ed4e9789e3bb"
      },
      "execution_count": 8,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Search space : \n",
            "[1, 6, 7, 8, 19, 21, 24, 30, 32, 37, 39, 40, 47, 58, 60, 61, 65, 67, 74, 77, 78, 79, 80, 81, 86, 87, 90, 91, 94, 97]\n",
            "element to compare against target :  21\n",
            "\n",
            "Search space : \n",
            "[24, 30, 32, 37, 39, 40, 47, 58, 60, 61, 65, 67, 74, 77, 78, 79, 80, 81, 86, 87, 90, 91, 94, 97]\n",
            "element to compare against target :  40\n",
            "\n",
            "Search space : \n",
            "[47, 58, 60, 61, 65, 67, 74, 77, 78, 79, 80, 81, 86, 87, 90, 91, 94, 97]\n",
            "element to compare against target :  67\n",
            "\n",
            "Search space : \n",
            "[74, 77, 78, 79, 80, 81, 86, 87, 90, 91, 94, 97]\n",
            "element to compare against target :  81\n",
            "\n",
            "Search space : \n",
            "[74, 77, 78, 79, 80]\n"
          ]
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "'Yes'"
            ],
            "application/vnd.google.colaboratory.intrinsic+json": {
              "type": "string"
            }
          },
          "metadata": {},
          "execution_count": 8
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "NOTE - Print statements are not a part of the algorithm - we just added them to visualize the working of it. IGNORE them for time complexity analysis."
      ],
      "metadata": {
        "id": "m1VByxYfUgfy"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "# Analysis of pos5_search_sorted\n",
        "\n",
        "Let $T(n)$ denote the time taken by ```pos5_search_sorted ``` on lists of length $n$.\n",
        "\n",
        "The total time taken depends on -\n",
        "\n",
        "1. Length of list $L$\n",
        "2. The outcome of the comparison $L[5]$ vs $t$\n",
        "\n",
        "If the length of $L$ is less than 6, it runs a simple for loop and within 5 iterations returns the answer. Total time - $O(1)$. We will consider larger list sizes - $n \\to \\infty$ as we care about asymptotic behavior.\n",
        "\n",
        "---\n",
        "\n",
        "\n",
        "**Case 1 - L[5] == t**\n",
        "\n",
        "If $L[5] == t$, then the function returns \"Yes\" and terminates.\n",
        "\n",
        "Time taken :\n",
        "1. Checking if condition for length of $L$ - $O(1)$\n",
        "2. Checking if condition for $L[5] == t$ - $O(1)$\n",
        "3. Return \"Yes\" - $O(1)$\n",
        "\n",
        "> Total time - $O(1) + O(1) + O(1) = O(1)$.\n",
        "---\n",
        "\n",
        "**Case 2 - L[5] > t**\n",
        "\n",
        "If $L[5] > t$, then $L$ is sliced to $L[0:5]$ and the function is called on a 5 sized list.\n",
        "\n",
        "Time taken -\n",
        "\n",
        "1. Checking if condition for length of $L$ - $O(1)$\n",
        "2. Checking if condition for $L[5] == t$ - $O(1)$\n",
        "3. Checking elif condition for $L[5] > t$ - $O(1)$\n",
        "4. Slicing list down to 5 elements - $O(1)$ time\n",
        "5. Implementing function call ```pos5_search_sorted(L[0:5],t)```\n",
        "\n",
        "At this stage, the list has size less than 6, so the function call ```pos5_search_sorted(L[0:5],t)``` resolves in time $O(1)$.\n",
        "\n",
        ">Total time - $O(1) + O(1) + O(1) + O(1) + O(1) = O(1)$\n",
        "\n",
        "---\n",
        "\n",
        "\n",
        "> Python time complexity documentation - https://wiki.python.org/moin/TimeComplexity\n",
        "\n",
        "\n",
        "\n",
        "\n",
        "\n",
        "\n"
      ],
      "metadata": {
        "id": "tHzCmV-reUeU"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "# Case 3 - L[5] < t\n",
        "\n",
        "If $L[5] < t$, then $L$ is sliced to $L[6:]$ in the function call to ```pos5_search_sorted(L[6:],t)```.\n",
        "\n",
        "Time taken -\n",
        "\n",
        "1. Checking if condition for length of $L$ - $O(1)$\n",
        "2. Checking if condition for $L[5] == t$ - $O(1)$\n",
        "3. Checking elif condition for $L[5] > t$ - $O(1)$\n",
        "4. Checking elif condition for $L[5] < t$ - $O(1)$\n",
        "4. Slicing list down to $n-6$ elements - $ (n-6) $ time\n",
        "5. Implementing function call ```pos5_search_sorted(L[6:],t)```\n",
        "\n",
        "The function call ```pos5_search_sorted(L[6:],t)``` runs the same algorithm on a list of size $n-6$. The time complexity of this is $T(n-6)$ in accordance with the definition of $T$.\n",
        "\n",
        ">Total time - $O(1) + O(1) + O(1) + O(1) + n-6 + T(n-6) = T(n-6) + O(n)$ time."
      ],
      "metadata": {
        "id": "QWxNg_gakwYZ"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "# Final time complexity analysis\n",
        "\n",
        "Since we are concerned with the worst-case time complexity of algorithms, we have to consider the worst of all 3 cases for time complexity considerations.\n",
        "\n",
        "That is,\n",
        "\n",
        "$T(n) = max(O(1), O(1), T(n-6) + O(n))$\n",
        "\n",
        "Since the last term is clearly the dominant one,  we have -\n",
        "\n",
        "`$T(n) \\leq T(n-6) + O(n)$`\n",
        "\n",
        "This is a **recurrence relation** that characterizes the running time of ```pos5_search_sorted(L,t)```\n",
        "\n",
        "---\n",
        "\n",
        "1.   On a list of length $n$, we spend $O(n)$ time to process -\n",
        " * identify search space for target by comparing $L[5]$ vs $t$ and\n",
        " * slice the list.\n",
        "2.   Then, we end up with a list of size $n-6$ which we search through by making a recursive call on the smaller list - costing time $T(n-6)$.\n",
        "\n",
        "---\n",
        "\n",
        ">How do we get an asmymptotic bound for $T(n)$?"
      ],
      "metadata": {
        "id": "sVpD8QWll-pN"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "# Resolving recurrences\n",
        "\n",
        "We will consider the following perspective of the above recurrence -\n",
        "\n",
        "1. Understand number of recursive calls\n",
        "2. Time spent during the most expensive recurisve call\n",
        "\n",
        "and simply give an upper bound by multiplying these two quantities.\n",
        "\n",
        "For ```pos5_search_sorted(L,t)```, we have -\n",
        "\n",
        "1. Number of recusive calls = $\\frac{n}{6}$\n",
        "\n",
        "2. Time spent during most expensive recursive call = $n$ (first call)\n",
        "\n",
        "> Asmyptotic upper bound for running time of ```pos5_search_sorted(L,t)``` - $$T(n) = O\\left(\\frac{n}{6} * n\\right) = O\\left(\\frac{n^2}{6}\\right) = O(n^2)$$"
      ],
      "metadata": {
        "id": "Ui1cOFJvZhb-"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "**```pos5_search_sorted(L,t)```  vs ```basic_search(L,t)```**\n",
        "\n",
        "We started out with basic search requiring $O(n)$ time to search through any list (unsorted or sorted). We identified that searching through sorted lists could be better and wrote a recusive algorithm ```pos5_search_sorted(L,t)```  to do this task. But, the time complexity of ```pos5_search_sorted(L,t)```  is WORSE - $O(n^2)$ as opposed to $O(n)$ for ```basic_search(L,t)```.\n",
        "\n",
        ">Why is the time complexity worse despite essentially making only $\\frac{n}{6}$ comparisons?\n",
        "\n",
        "# Sources of inefficiency\n",
        "\n",
        "1. Comparing against 5th element only eliminates 5 elements in the case $L[5] < t$.\n",
        "\n",
        "2. Slicing the list copies a section of list elements at significant cost. Does this work really help us?\n",
        "\n"
      ],
      "metadata": {
        "id": "CLN0UZfMa8Co"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "# Fixing these issues\n",
        "\n",
        "1. If we instead compare with the middle element, we are guaranteed to eliminate at least half of the list irrespective of $L[n/2]$ vs $t$.\n",
        "\n",
        "2. We don't slice the list. Instead, we just use start and end positions of the search space to move down recursive calls."
      ],
      "metadata": {
        "id": "NSnHCd2EcUgJ"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "# Binary Seach\n",
        "\n",
        "Input - A sorted list $L$ containing $n$ elements and a target value $t$\n",
        "\n",
        "Output - Yes if $t$ is in $L$, No otherwise\n",
        "\n",
        "Idea - The main idea is to check the **median** against the target and based on the outcome of it repeat the process on either the left or right half (or terminate if successful) **using start and end positions**.\n"
      ],
      "metadata": {
        "id": "o70nJBiBBkyO"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "#Use this skeletal structure to code up binary search to practice coding!\n",
        "\n",
        "def binary_search(list,target,start,stop):\n",
        "\n",
        "  #Base cases - what are the base cases? for each one, give the right answer!\n",
        "\n",
        "\n",
        "  #Other cases\n",
        "\n",
        "  #check median against target value - what is the median element?\n",
        "  #if you find the target, how do you return its index? be precise\n",
        "  #if (median > target) what do you do next - be precise\n",
        "  #if (median < target) what do you do next - be precise\n",
        "\n",
        "\n",
        "  #for the recursive calls, list and target don't change - so you must modify start and stop appropriately"
      ],
      "metadata": {
        "id": "mGSP0LS0ZIWM"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "li = [1, 6, 7, 8, 19, 21, 24, 30, 32, 37, 39, 40, 47, 58, 60, 61, 65, 67, 74, 77, 78, 79, 80, 81, 86, 87, 90, 91, 94, 97]\n",
        "li2 = [3, 7, 9, 14, 17, 19, 21, 27, 33, 35, 37, 38, 42, 44, 45, 46, 51, 52, 53, 59, 64, 68, 70, 73, 79, 80, 81, 87, 92, 99]\n",
        "\n",
        "target = 77\n",
        "\n",
        "# function binary search that slices search space in half at each step until target is found\n",
        "def binary_Search(L,t,start,stop):\n",
        "\n",
        "  #print(\"Search space : \")\n",
        "  #print(L[start:stop])\n",
        "\n",
        "  if(start > stop):\n",
        "    return (\"No\")\n",
        "\n",
        "  else:\n",
        "\n",
        "    mid = start + (stop - start)//2\n",
        "    #print(\"element to compare against target : \", L[start+mid])\n",
        "    #print(\"\")\n",
        "\n",
        "\n",
        "    if(L[mid] == t):\n",
        "      print(\"Element \" + str(t) + \" found at position \" + str(mid))\n",
        "      return (\"Yes\")\n",
        "\n",
        "    elif(L[mid] > t):\n",
        "      return binary_Search(L,t,start,mid-1)\n",
        "\n",
        "    else:\n",
        "      return binary_Search(L,t,mid+1,stop)\n",
        "\n",
        "print(binary_Search(li,target,0,len(li)-1))\n",
        "#print(binary_Search(li2,target,0,len(li)-1))"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "t42NgVbWDM0N",
        "outputId": "a09f9eba-ff2b-475a-d902-7f23d45aedb4"
      },
      "execution_count": 12,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Element 77 found at position 19\n",
            "Yes\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "# Analysis\n",
        "\n",
        "**Run time**\n",
        "\n",
        "Computations performed within each recursive call -\n",
        "\n",
        "*   Base case check + return - $O(1) + O(1) = O(1)$\n",
        "\n",
        "*   Computing median - $O(1)$\n",
        "\n",
        "*   Checking median element against target - $O(1) \\times 2 = O(1)$\n",
        "\n",
        "*   Next recursive call - $O(1)$\n",
        "\n",
        "*   Total time within each call - $O(1) + O(1) + O(1) + O(1) = O(1)$\n",
        "\n",
        "> What is the total number of recusive calls made?\n",
        "\n",
        "The list is halving in size at each level, so the total number of recurisve calls is at most the number of times the list can be halved $\\sim \\log_{2}n $.\n",
        "\n",
        ">The total number of recursive calls is $(\\log_{2}{n} + 1)$, so the total time taken is at most $O(1) \\times (\\log_{2}{n}+1) = O(\\log_{2}{n})$\n",
        "\n",
        "\n",
        "\n",
        "> Binary search searches through any sorted $n$ sized list in $O(\\log n)$ time.\n",
        "\n"
      ],
      "metadata": {
        "id": "RbYykiENNi3D"
      }
    }
  ]
}