{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# What Shape Is This Sign?\n",
    "\n",
    "road_signs.py\n",
    "\n",
    "AI project on Computer Vision: reading a photo of a road sign and\n",
    "identifying its shape - and so what kind of sign it is.\n",
    "\n",
    "CBSE Class X, Artificial Intelligence (417), Part C project work.\n",
    "SDG 3 (Good Health and Well-being - target 3.6, road safety) and\n",
    "SDG 11 (Sustainable Cities and Communities).\n",
    "\n",
    "In India the shape of a road sign tells you its type before you can read it:\n",
    "  circle    (red)  - MANDATORY: an order you must obey\n",
    "  triangle  (red)  - CAUTIONARY: danger ahead\n",
    "  rectangle (blue) - INFORMATORY: useful information\n",
    "\n",
    "Shape is turned into two numbers (FEATURES):\n",
    "  fill  - how much of the box around the sign the sign fills\n",
    "          (a rectangle fills almost all of it, a circle about 0.79,\n",
    "          a triangle about half)\n",
    "  top   - how wide the top of the sign is, compared with its widest row\n",
    "          (a triangle is narrow at the top)\n",
    "Then two models are compared: a RULE BASED model written from geometry,\n",
    "and a LEARNING BASED model that learns the average of each shape.\n",
    "\n",
    "Libraries: numpy, matplotlib (images and charts), pandas, os."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Stage 1 - Problem scoping"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(\"=\" * 60)\n",
    "print(\"WHAT SHAPE IS THIS SIGN?\")\n",
    "print(\"=\" * 60)\n",
    "print(\"Who   : drivers, and the makers of driver-assistance systems\")\n",
    "print(\"What  : a sign missed at speed can cause an accident\")\n",
    "print(\"Where : Indian roads, where the shape tells the type of sign\")\n",
    "print(\"Why   : a car that reads signs can warn a distracted driver\")\n",
    "print()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Stage 2 - Data acquisition"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import os\n",
    "import numpy as np\n",
    "import pandas as pd\n",
    "import matplotlib.pyplot as plt\n",
    "\n",
    "pd.set_option(\"display.width\", 120)\n",
    "\n",
    "shapes = [\"circle\", \"triangle\", \"rectangle\"]\n",
    "meaning = {\"circle\": \"MANDATORY\", \"triangle\": \"CAUTIONARY\", \"rectangle\": \"INFORMATORY\"}\n",
    "\n",
    "paths = []\n",
    "labels = []\n",
    "for s in shapes:\n",
    "    for name in sorted(os.listdir(os.path.join(\"images\", s))):\n",
    "        paths.append(os.path.join(\"images\", s, name))\n",
    "        labels.append(s)\n",
    "\n",
    "print(\"--- Stage 2: Data acquisition ---\")\n",
    "for s in shapes:\n",
    "    print(s.ljust(10), \"photos :\", labels.count(s))\n",
    "print(\"Total photos      :\", len(paths))\n",
    "print()\n",
    "\n",
    "# Chart 1: four photos of each shape.\n",
    "plt.figure(figsize=(7, 5.4))\n",
    "for r in range(3):\n",
    "    group = []\n",
    "    for i in range(len(paths)):\n",
    "        if labels[i] == shapes[r]:\n",
    "            group.append(paths[i])\n",
    "    for c in range(4):\n",
    "        plt.subplot(3, 4, r * 4 + c + 1)\n",
    "        plt.imshow(plt.imread(group[c * len(group) // 4]))\n",
    "        plt.axis(\"off\")\n",
    "        if c == 0:\n",
    "            plt.title(shapes[r], fontsize=9, loc=\"left\")\n",
    "plt.tight_layout()\n",
    "plt.savefig(\"chart1_sample_signs.png\")\n",
    "plt.close()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Stage 3 - Data exploration"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(\"--- Stage 3: Data exploration ---\")\n",
    "\n",
    "# Step 1 - find the sign. Its border is strongly red or strongly blue, which\n",
    "# the grey and green background never is.\n",
    "# Step 2 - for every row of the photo, measure the sign's width from its\n",
    "# leftmost coloured pixel to its rightmost. Adding those widths up gives the\n",
    "# sign's area, white middle included.\n",
    "rows = []\n",
    "for i in range(len(paths)):\n",
    "    img = plt.imread(paths[i])[:, :, :3].astype(float) * 255\n",
    "    r, g, b = img[:, :, 0], img[:, :, 1], img[:, :, 2]\n",
    "    red = (r > g + 70) & (r > b + 70)\n",
    "    blue = (b > r + 60) & (b > g + 40)\n",
    "    coloured = red | blue\n",
    "    widths = []\n",
    "    for y in range(img.shape[0]):\n",
    "        xs = np.where(coloured[y])[0]\n",
    "        if len(xs) > 0:\n",
    "            widths.append(xs[-1] - xs[0] + 1)\n",
    "    widths = np.array(widths)\n",
    "    height = len(widths)\n",
    "    box = height * widths.max()\n",
    "    fill = widths.sum() / box\n",
    "    top = widths[: max(1, height // 4)].mean() / widths.max()\n",
    "    if red.sum() > blue.sum():\n",
    "        colour = \"red\"\n",
    "    else:\n",
    "        colour = \"blue\"\n",
    "    rows.append([paths[i], labels[i], colour, round(fill, 3), round(top, 3)])\n",
    "\n",
    "data = pd.DataFrame(rows, columns=[\"photo\", \"label\", \"colour\", \"fill\", \"top\"])\n",
    "print(\"Average features of each shape:\")\n",
    "print(data.groupby(\"label\")[[\"fill\", \"top\"]].mean().round(3).reindex(shapes))\n",
    "print()\n",
    "print(\"Colour found for each shape:\")\n",
    "print(pd.crosstab(data[\"label\"], data[\"colour\"]).reindex(shapes))\n",
    "print()\n",
    "\n",
    "# Chart 2: the two features, one dot per photo.\n",
    "plt.figure(figsize=(7, 5))\n",
    "dot_colours = [\"#d62728\", \"#ff7f0e\", \"#1f77b4\"]\n",
    "for k in range(3):\n",
    "    part = data[data[\"label\"] == shapes[k]]\n",
    "    plt.scatter(part[\"fill\"], part[\"top\"], color=dot_colours[k], label=shapes[k], alpha=0.75)\n",
    "plt.title(\"Two numbers that describe a shape\")\n",
    "plt.xlabel(\"fill: share of the box the sign fills\")\n",
    "plt.ylabel(\"top: width of the top / widest width\")\n",
    "plt.legend()\n",
    "plt.grid(True, linestyle=\"--\", alpha=0.4)\n",
    "plt.tight_layout()\n",
    "plt.savefig(\"chart2_shape_features.png\")\n",
    "plt.close()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Stage 4 - Modelling"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(\"--- Stage 4: Modelling ---\")\n",
    "use = [\"fill\", \"top\"]\n",
    "data = data.sample(frac=1, random_state=6)\n",
    "cut = int(len(data) * 0.8)\n",
    "train = data[:cut]\n",
    "test = data[cut:]\n",
    "print(\"Training photos :\", len(train))\n",
    "print(\"Testing photos  :\", len(test))\n",
    "print()\n",
    "\n",
    "# Model A - RULE BASED, from geometry alone, with no training at all:\n",
    "#   a rectangle fills nearly all of its box; of the rest, a triangle has a\n",
    "#   narrow top and a circle does not.\n",
    "rule = []\n",
    "for i in range(len(test)):\n",
    "    row = test.iloc[i]\n",
    "    if row[\"fill\"] > 0.9:\n",
    "        rule.append(\"rectangle\")\n",
    "    elif row[\"top\"] < 0.45:\n",
    "        rule.append(\"triangle\")\n",
    "    else:\n",
    "        rule.append(\"circle\")\n",
    "\n",
    "# Model B - LEARNING BASED: the average fill and top of each shape, learned\n",
    "# from the training photos; a new sign gets the nearest average.\n",
    "averages = train.groupby(\"label\")[use].mean()\n",
    "print(\"What model B learned - the average of each shape:\")\n",
    "print(averages.round(3).reindex(shapes))\n",
    "print()\n",
    "learned = []\n",
    "for i in range(len(test)):\n",
    "    distance = ((averages - test[use].iloc[i]) ** 2).sum(axis=1)\n",
    "    learned.append(distance.idxmin())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Stage 5 - Evaluation"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(\"--- Stage 5: Evaluation ---\")\n",
    "actual = list(test[\"label\"])\n",
    "rule_right = 0\n",
    "learned_right = 0\n",
    "for i in range(len(actual)):\n",
    "    if rule[i] == actual[i]:\n",
    "        rule_right = rule_right + 1\n",
    "    if learned[i] == actual[i]:\n",
    "        learned_right = learned_right + 1\n",
    "print(\"Rule based     :\", rule_right, \"of\", len(actual), \"correct\")\n",
    "print(\"Learning based :\", learned_right, \"of\", len(actual), \"correct\")\n",
    "print(\"Accuracy of the rule based model     :\", round(rule_right / len(actual) * 100, 1), \"%\")\n",
    "print(\"Accuracy of the learning based model :\", round(learned_right / len(actual) * 100, 1), \"%\")\n",
    "print()\n",
    "\n",
    "matrix = pd.crosstab(pd.Series(actual, name=\"Actual\"), pd.Series(learned, name=\"Predicted\"))\n",
    "matrix = matrix.reindex(index=shapes, columns=shapes, fill_value=0)\n",
    "print(\"Confusion matrix of the learning based model:\")\n",
    "print(matrix)\n",
    "print()\n",
    "\n",
    "wrong = []\n",
    "for i in range(len(actual)):\n",
    "    if learned[i] != actual[i] or rule[i] != actual[i]:\n",
    "        wrong.append(i)\n",
    "print(\"Photos at least one model got wrong :\", len(wrong))\n",
    "for i in wrong:\n",
    "    row = test.iloc[i]\n",
    "    print(\"  \" + os.path.basename(row[\"photo\"]).ljust(18), \"fill\", row[\"fill\"], \" top\", row[\"top\"],\n",
    "          \" rule said\", rule[i], \" learned said\", learned[i])\n",
    "print()\n",
    "\n",
    "# Chart 3: confusion matrix.\n",
    "plt.figure(figsize=(5.5, 4.6))\n",
    "plt.imshow(matrix.values, cmap=\"Purples\")\n",
    "for r in range(3):\n",
    "    for c in range(3):\n",
    "        value = matrix.values[r][c]\n",
    "        if value > matrix.values.max() / 2:\n",
    "            ink = \"white\"\n",
    "        else:\n",
    "            ink = \"black\"\n",
    "        plt.text(c, r, value, ha=\"center\", va=\"center\", fontsize=14, color=ink)\n",
    "plt.xticks(range(3), shapes)\n",
    "plt.yticks(range(3), shapes)\n",
    "plt.xlabel(\"Predicted shape\")\n",
    "plt.ylabel(\"Actual shape\")\n",
    "plt.title(\"Learning based model (testing photos)\")\n",
    "plt.tight_layout()\n",
    "plt.savefig(\"chart3_matrix.png\")\n",
    "plt.close()\n",
    "\n",
    "# Chart 4: the photos that fooled a model.\n",
    "if len(wrong) > 0:\n",
    "    plt.figure(figsize=(2 * len(wrong) + 0.5, 2.6))\n",
    "    for n in range(len(wrong)):\n",
    "        row = test.iloc[wrong[n]]\n",
    "        plt.subplot(1, len(wrong), n + 1)\n",
    "        plt.imshow(plt.imread(row[\"photo\"]))\n",
    "        plt.title(\"really \" + row[\"label\"], fontsize=9)\n",
    "        plt.axis(\"off\")\n",
    "    plt.tight_layout()\n",
    "    plt.savefig(\"chart4_mistakes.png\")\n",
    "    plt.close()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Stage 6 - Deployment"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(\"--- Stage 6: Deployment ---\")\n",
    "new_names = sorted(os.listdir(\"new_photos\"))\n",
    "plt.figure(figsize=(7, 2.8))\n",
    "for i in range(len(new_names)):\n",
    "    photo = plt.imread(os.path.join(\"new_photos\", new_names[i]))[:, :, :3]\n",
    "    img = photo.astype(float) * 255\n",
    "    r, g, b = img[:, :, 0], img[:, :, 1], img[:, :, 2]\n",
    "    coloured = ((r > g + 70) & (r > b + 70)) | ((b > r + 60) & (b > g + 40))\n",
    "    widths = []\n",
    "    for y in range(img.shape[0]):\n",
    "        xs = np.where(coloured[y])[0]\n",
    "        if len(xs) > 0:\n",
    "            widths.append(xs[-1] - xs[0] + 1)\n",
    "    widths = np.array(widths)\n",
    "    point = pd.Series({\n",
    "        \"fill\": widths.sum() / (len(widths) * widths.max()),\n",
    "        \"top\": widths[: max(1, len(widths) // 4)].mean() / widths.max(),\n",
    "    })\n",
    "    shape = ((averages - point) ** 2).sum(axis=1).idxmin()\n",
    "    print(new_names[i], \"-> fill\", round(point[\"fill\"], 2), \" top\", round(point[\"top\"], 2),\n",
    "          \"->\", shape, \"->\", meaning[shape], \"sign\")\n",
    "    plt.subplot(1, len(new_names), i + 1)\n",
    "    plt.imshow(photo)\n",
    "    plt.title(shape + \"\\n\" + meaning[shape].lower(), fontsize=9)\n",
    "    plt.axis(\"off\")\n",
    "plt.tight_layout()\n",
    "plt.savefig(\"chart5_new_signs.png\")\n",
    "plt.close()"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "name": "python"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}