{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Reading Handwritten PIN Codes\n",
    "\n",
    "pin_reader.py\n",
    "\n",
    "Capstone project: reading handwritten PIN codes on envelopes, so that\n",
    "letters and parcels can be sorted by machine.\n",
    "\n",
    "CBSE Class XII, Artificial Intelligence (843), Part C - Capstone Project.\n",
    "SDG 9 (Industry, Innovation and Infrastructure).\n",
    "\n",
    "Data Science Methodology, 10 steps. COMPUTER VISION with a NEURAL\n",
    "NETWORK. The data is real: the UCI handwritten digits dataset (8 x 8\n",
    "images). A multi-layer perceptron is compared with Logistic Regression,\n",
    "with stratified 5-fold cross validation; then OpenCV loads, splits and\n",
    "resizes whole PIN-code images for the model to read.\n",
    "\n",
    "Libraries: pandas, numpy, matplotlib, scikit-learn, OpenCV (cv2), joblib."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Step 1 - Business Understanding"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(\"=\" * 64)\n",
    "print(\"READING HANDWRITTEN PIN CODES\")\n",
    "print(\"=\" * 64)\n",
    "print(\"Problem : millions of envelopes carry handwritten PIN codes;\")\n",
    "print(\"          reading them by hand is slow and error-prone\")\n",
    "print(\"Goal    : recognise each handwritten digit, and so the whole PIN\")\n",
    "print()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Step 2 - Analytic Approach"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(\"Analytic approach : predictive - classification into 10 classes (0-9)\")\n",
    "print()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Step 3 - Data Requirements"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import os\n",
    "import pandas as pd\n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "import cv2\n",
    "from sklearn.model_selection import train_test_split, StratifiedKFold, cross_val_score\n",
    "from sklearn.linear_model import LogisticRegression\n",
    "from sklearn.neural_network import MLPClassifier\n",
    "from sklearn.metrics import accuracy_score, confusion_matrix, recall_score\n",
    "import joblib"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Step 4 - Data Collection"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "data = pd.read_csv(\"digits.csv\")\n",
    "pixels = [\"px\" + str(i) for i in range(64)]\n",
    "print(\"Images collected :\", len(data))\n",
    "print(\"Values per image :\", len(pixels), \"(an 8 x 8 grid)\")\n",
    "print(\"Pixel values run from\", data[pixels].values.min(), \"to\", data[pixels].values.max())\n",
    "print()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Step 5 - Data Understanding"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(\"--- Step 5: Data Understanding ---\")\n",
    "print(\"Missing values in the whole table :\", int(data.isnull().sum().sum()))\n",
    "counts = data[\"digit\"].value_counts().sort_index()\n",
    "print(\"Images of each digit:\")\n",
    "print(counts.to_string())\n",
    "print()\n",
    "\n",
    "# Chart 1: sixteen of the images, as the computer stores them.\n",
    "plt.figure(figsize=(8, 4.2))\n",
    "for i in range(16):\n",
    "    plt.subplot(2, 8, i + 1)\n",
    "    plt.imshow(data.loc[i, pixels].values.astype(float).reshape(8, 8), cmap=\"gray_r\")\n",
    "    plt.title(str(data.loc[i, \"digit\"]), fontsize=10)\n",
    "    plt.axis(\"off\")\n",
    "plt.suptitle(\"Sixteen handwritten digits - each is just 64 numbers\")\n",
    "plt.tight_layout()\n",
    "plt.savefig(\"chart1_samples.png\")\n",
    "plt.close()\n",
    "\n",
    "# Chart 2: the average image of each digit.\n",
    "plt.figure(figsize=(9, 2.4))\n",
    "for d in range(10):\n",
    "    plt.subplot(1, 10, d + 1)\n",
    "    plt.imshow(data[data[\"digit\"] == d][pixels].mean().values.reshape(8, 8), cmap=\"magma\")\n",
    "    plt.title(str(d))\n",
    "    plt.axis(\"off\")\n",
    "plt.suptitle(\"The average of every image of each digit\")\n",
    "plt.tight_layout()\n",
    "plt.savefig(\"chart2_average_digits.png\")\n",
    "plt.close()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Step 6 - Data Preparation"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(\"--- Step 6: Data Preparation ---\")\n",
    "before = len(data)\n",
    "data = data.drop_duplicates(subset=pixels + [\"digit\"])\n",
    "print(\"Duplicate images removed :\", before - len(data))\n",
    "# The pixel values run 0-16. Dividing by 16 puts them on a 0-1 scale,\n",
    "# which helps a neural network learn.\n",
    "X = data[pixels] / 16.0\n",
    "y = data[\"digit\"]\n",
    "print(\"Pixels rescaled from 0-16 to 0-1\")\n",
    "print()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Step 7 - Modelling"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(\"--- Step 7: Modelling ---\")\n",
    "X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, stratify=y, random_state=3)\n",
    "print(\"Training images :\", len(X_train), \" Testing images :\", len(X_test))\n",
    "logistic = LogisticRegression(max_iter=2000)\n",
    "network = MLPClassifier(hidden_layer_sizes=(64,), max_iter=1000, random_state=0)\n",
    "logistic.fit(X_train, y_train)\n",
    "network.fit(X_train, y_train)\n",
    "print(\"Neural network : 64 inputs -> 64 hidden neurons -> 10 outputs\")\n",
    "print(\"Weights learned by the network :\", sum(w.size for w in network.coefs_) + sum(b.size for b in network.intercepts_))\n",
    "print()\n",
    "\n",
    "# Chart 3: the network's training - its loss falling with each pass.\n",
    "plt.figure(figsize=(7, 4))\n",
    "plt.plot(network.loss_curve_, color=\"#b07aa1\")\n",
    "plt.title(\"The neural network learning: loss after each pass\")\n",
    "plt.xlabel(\"Pass through the training images (epoch)\")\n",
    "plt.ylabel(\"Loss\")\n",
    "plt.grid(True, linestyle=\"--\", alpha=0.4)\n",
    "plt.tight_layout()\n",
    "plt.savefig(\"chart3_loss_curve.png\")\n",
    "plt.close()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Step 8 - Evaluation"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(\"--- Step 8: Evaluation ---\")\n",
    "pred_log = logistic.predict(X_test)\n",
    "pred_net = network.predict(X_test)\n",
    "print(\"Accuracy, Logistic Regression :\", round(accuracy_score(y_test, pred_log), 4))\n",
    "print(\"Accuracy, neural network      :\", round(accuracy_score(y_test, pred_net), 4))\n",
    "folds = StratifiedKFold(n_splits=5, shuffle=True, random_state=1)\n",
    "cv_log = cross_val_score(LogisticRegression(max_iter=2000), X, y, cv=folds)\n",
    "cv_net = cross_val_score(MLPClassifier(hidden_layer_sizes=(64,), max_iter=1000, random_state=0), X, y, cv=folds)\n",
    "print(\"Accuracy in each fold, Logistic Regression :\", np.round(cv_log, 3))\n",
    "print(\"Accuracy in each fold, neural network      :\", np.round(cv_net, 3))\n",
    "print(\"Mean CV accuracy, Logistic Regression :\", round(cv_log.mean(), 4))\n",
    "print(\"Mean CV accuracy, neural network      :\", round(cv_net.mean(), 4))\n",
    "print()\n",
    "\n",
    "cm = confusion_matrix(y_test, pred_net)\n",
    "recall_each = recall_score(y_test, pred_net, average=None)\n",
    "print(\"Recall for each digit (neural network):\")\n",
    "for d in range(10):\n",
    "    print(\"  digit\", d, \":\", round(recall_each[d], 3))\n",
    "pairs = []\n",
    "for a in range(10):\n",
    "    for b in range(10):\n",
    "        if a != b and cm[a, b] > 0:\n",
    "            pairs.append([cm[a, b], a, b])\n",
    "pairs.sort(reverse=True)\n",
    "print(\"Most common mistakes (true -> predicted, count):\")\n",
    "for count, a, b in pairs[:4]:\n",
    "    print(\"  \", a, \"->\", b, \":\", count)\n",
    "print()\n",
    "\n",
    "# Chart 4: the 10 x 10 confusion matrix.\n",
    "plt.figure(figsize=(6.5, 5.5))\n",
    "plt.imshow(cm, cmap=\"Blues\")\n",
    "for a in range(10):\n",
    "    for b in range(10):\n",
    "        if cm[a, b] > 0:\n",
    "            plt.text(b, a, cm[a, b], ha=\"center\", va=\"center\", fontsize=9,\n",
    "                     color=\"white\" if cm[a, b] > cm.max() / 2 else \"black\")\n",
    "plt.xticks(range(10))\n",
    "plt.yticks(range(10))\n",
    "plt.xlabel(\"Predicted digit\")\n",
    "plt.ylabel(\"True digit\")\n",
    "plt.title(\"Neural network: confusion matrix (test images)\")\n",
    "plt.tight_layout()\n",
    "plt.savefig(\"chart4_confusion.png\")\n",
    "plt.close()\n",
    "\n",
    "# Chart 5: the test images the network got wrong.\n",
    "wrong = np.where(pred_net != y_test.values)[0]\n",
    "plt.figure(figsize=(1.3 * max(len(wrong), 1) + 0.5, 2.0))\n",
    "for k, i in enumerate(wrong):\n",
    "    plt.subplot(1, len(wrong), k + 1)\n",
    "    plt.imshow(X_test.iloc[i].values.reshape(8, 8), cmap=\"gray_r\")\n",
    "    plt.title(str(y_test.iloc[i]) + \" -> \" + str(pred_net[i]), fontsize=9)\n",
    "    plt.axis(\"off\")\n",
    "plt.tight_layout()\n",
    "plt.savefig(\"chart5_mistakes.png\")\n",
    "plt.close()\n",
    "\n",
    "# Chart 6: accuracy in each fold, both models.\n",
    "x = np.arange(5)\n",
    "plt.figure(figsize=(7.5, 4))\n",
    "plt.bar(x - 0.2, cv_log, width=0.4, label=\"Logistic Regression\", color=\"#9c9c9c\")\n",
    "plt.bar(x + 0.2, cv_net, width=0.4, label=\"Neural network\", color=\"#b07aa1\")\n",
    "plt.xticks(x, [\"Fold 1\", \"Fold 2\", \"Fold 3\", \"Fold 4\", \"Fold 5\"])\n",
    "plt.ylim(0.9, 1.0)\n",
    "plt.ylabel(\"Accuracy\")\n",
    "plt.title(\"Stratified 5-fold cross validation\")\n",
    "plt.legend()\n",
    "plt.tight_layout()\n",
    "plt.savefig(\"chart6_kfold.png\")\n",
    "plt.close()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Step 9 - Deployment"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(\"--- Step 9: Deployment ---\")\n",
    "joblib.dump(network, \"digit_model.pkl\")\n",
    "model = joblib.load(\"digit_model.pkl\")\n",
    "truth = pd.read_csv(\"pin_truth.csv\", dtype={\"true_pin\": str})\n",
    "results = []\n",
    "for k in range(len(truth)):\n",
    "    name = truth.loc[k, \"file\"]\n",
    "    strip = cv2.imread(os.path.join(\"pin_codes\", name), cv2.IMREAD_GRAYSCALE)   # load with OpenCV\n",
    "    if strip is None:                         # imread gives None, not an error, if the file is missing\n",
    "        raise FileNotFoundError(\"Could not read pin_codes/\" + name)\n",
    "    strip = 255 - strip                                                          # ink becomes bright\n",
    "    read = \"\"\n",
    "    for j in range(6):\n",
    "        digit_img = strip[:, j * 64:(j + 1) * 64]                                 # cut out one digit\n",
    "        small = cv2.resize(digit_img, (8, 8), interpolation=cv2.INTER_AREA)       # resize to 8 x 8\n",
    "        vector = (small / 255.0 * 16).round() / 16.0                              # same 0-1 scale as training\n",
    "        read = read + str(model.predict(vector.reshape(1, 64))[0])\n",
    "    results.append([name, truth.loc[k, \"true_pin\"], read])\n",
    "    status = \"correct\" if read == truth.loc[k, \"true_pin\"] else \"WRONG\"\n",
    "    print(name, \" true\", truth.loc[k, \"true_pin\"], \" read\", read, \" \", status)\n",
    "print()\n",
    "\n",
    "# Chart 7: four of the PIN images, with what the model read.\n",
    "plt.figure(figsize=(9, 4.5))\n",
    "for n, k in enumerate([0, 1, 7, 9]):\n",
    "    img = cv2.imread(os.path.join(\"pin_codes\", results[k][0]), cv2.IMREAD_GRAYSCALE)\n",
    "    plt.subplot(4, 1, n + 1)\n",
    "    plt.imshow(img, cmap=\"gray\")\n",
    "    plt.title(\"read as \" + results[k][2] + \"   (true \" + results[k][1] + \")\", fontsize=10)\n",
    "    plt.axis(\"off\")\n",
    "plt.tight_layout()\n",
    "plt.savefig(\"chart7_pin_codes.png\")\n",
    "plt.close()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Step 10 - Feedback"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(\"--- Step 10: Feedback ---\")\n",
    "digits_right = 0\n",
    "pins_right = 0\n",
    "clean_right = 0\n",
    "messy_right = 0\n",
    "for k in range(len(results)):\n",
    "    true_pin, read = results[k][1], results[k][2]\n",
    "    digits_right = digits_right + sum(1 for a, b in zip(true_pin, read) if a == b)\n",
    "    if true_pin == read:\n",
    "        pins_right = pins_right + 1\n",
    "        if k < 6:\n",
    "            clean_right = clean_right + 1\n",
    "        else:\n",
    "            messy_right = messy_right + 1\n",
    "print(\"Digits read correctly on envelopes :\", digits_right, \"of\", 6 * len(results))\n",
    "print(\"Whole PIN codes read correctly     :\", pins_right, \"of\", len(results))\n",
    "print(\"Clean envelopes, PINs correct      :\", clean_right, \"of 6\")\n",
    "print(\"Messy envelopes, PINs correct      :\", messy_right, \"of 4\")\n",
    "per_digit = digits_right / (6 * len(results))\n",
    "print(\"If each digit is right with probability\", round(per_digit, 3), \", a 6-digit PIN is all right\")\n",
    "print(\"with probability about\", round(per_digit ** 6, 3))"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "name": "python"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}