{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Which Crop for This Field?\n",
    "\n",
    "crop_recommendation.py\n",
    "\n",
    "AI project on Statistical Data: recommending a crop for a field from its\n",
    "soil test and its weather.\n",
    "\n",
    "CBSE Class X, Artificial Intelligence (417), Part C project work.\n",
    "SDG 2 (Zero Hunger) and SDG 15 (Life on Land).\n",
    "\n",
    "The program follows the six stages of the AI Project Cycle. The model is\n",
    "\"k nearest neighbours\": to recommend a crop for a new field, it finds the\n",
    "k fields in the training data whose soil and weather are most like it, and\n",
    "recommends the crop that most of those neighbours grew well.\n",
    "\n",
    "Libraries: pandas (the data), numpy (the arithmetic), matplotlib (charts)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Stage 1 - Problem scoping"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(\"=\" * 60)\n",
    "print(\"CROP RECOMMENDATION FROM A SOIL TEST\")\n",
    "print(\"=\" * 60)\n",
    "print(\"Who   : small farmers deciding what to sow\")\n",
    "print(\"What  : the wrong crop for the soil means a poor harvest\")\n",
    "print(\"Where : farms that have a soil test but no expert to read it\")\n",
    "print(\"Why   : the right crop gives more food from the same land\")\n",
    "print()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Stage 2 - Data acquisition"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "\n",
    "pd.set_option(\"display.width\", 120)\n",
    "pd.set_option(\"display.max_columns\", 20)\n",
    "\n",
    "data = pd.read_csv(\"soil_samples.csv\")\n",
    "\n",
    "print(\"--- Stage 2: Data acquisition ---\")\n",
    "print(\"Rows read    :\", len(data))\n",
    "print(\"Columns read :\", len(data.columns))\n",
    "print()\n",
    "print(\"The first 10 rows:\")\n",
    "print(data.head(10))\n",
    "print()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Stage 3 - Data exploration"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(\"--- Stage 3: Data exploration ---\")\n",
    "\n",
    "features = [\"nitrogen\", \"phosphorus\", \"potassium\", \"ph\", \"rainfall_mm\", \"temperature_c\"]\n",
    "label = \"crop\"\n",
    "\n",
    "# The same sample typed in twice would count twice, so repeats are removed.\n",
    "before = len(data)\n",
    "data = data.drop_duplicates()\n",
    "print(\"Repeated rows removed        :\", before - len(data))\n",
    "\n",
    "# A soil test with a reading missing cannot be compared with other fields.\n",
    "missing = data[features].isnull().any(axis=1).sum()\n",
    "data = data.dropna(subset=features)\n",
    "print(\"Rows with a reading missing  :\", missing, \"(removed)\")\n",
    "\n",
    "# \"rice\", \" RICE\" and \"Rice\" must all be one crop.\n",
    "data[label] = data[label].str.strip().str.title()\n",
    "print(\"Rows left                    :\", len(data))\n",
    "print()\n",
    "\n",
    "crops = sorted(data[label].unique())\n",
    "print(\"Fields of each crop:\")\n",
    "print(data[label].value_counts().sort_index())\n",
    "print()\n",
    "\n",
    "rain = data[\"rainfall_mm\"]\n",
    "print(\"Rainfall - mean   :\", round(np.mean(rain), 1), \"mm\")\n",
    "print(\"Rainfall - median :\", round(np.median(rain), 1), \"mm\")\n",
    "print()\n",
    "\n",
    "averages_by_crop = data.groupby(label)[features].mean().round(1)\n",
    "print(\"The average field for each crop:\")\n",
    "print(averages_by_crop)\n",
    "print()\n",
    "\n",
    "colours = [\"#1f77b4\", \"#ff7f0e\", \"#2ca02c\", \"#d62728\", \"#9467bd\"]\n",
    "\n",
    "# Chart 1: average rainfall for each crop.\n",
    "plt.figure(figsize=(7, 4))\n",
    "plt.bar(crops, averages_by_crop[\"rainfall_mm\"], color=colours)\n",
    "plt.title(\"Average rainfall where each crop grew well\")\n",
    "plt.xlabel(\"Crop\")\n",
    "plt.ylabel(\"Rainfall (mm a month)\")\n",
    "plt.tight_layout()\n",
    "plt.savefig(\"chart1_rainfall.png\")\n",
    "plt.close()\n",
    "\n",
    "# Chart 2: rainfall against temperature, one colour per crop.\n",
    "plt.figure(figsize=(7, 5))\n",
    "for i in range(len(crops)):\n",
    "    rows = data[data[label] == crops[i]]\n",
    "    plt.scatter(rows[\"rainfall_mm\"], rows[\"temperature_c\"],\n",
    "                color=colours[i], label=crops[i], alpha=0.7, s=18)\n",
    "plt.title(\"Rainfall and temperature of every field\")\n",
    "plt.xlabel(\"Rainfall (mm a month)\")\n",
    "plt.ylabel(\"Temperature (deg C)\")\n",
    "plt.legend()\n",
    "plt.grid(True, linestyle=\"--\", alpha=0.4)\n",
    "plt.tight_layout()\n",
    "plt.savefig(\"chart2_rain_temperature.png\")\n",
    "plt.close()\n",
    "\n",
    "# Chart 3: the three nutrients, side by side for each crop.\n",
    "x = np.arange(len(crops))\n",
    "plt.figure(figsize=(8, 4.5))\n",
    "plt.bar(x - 0.27, averages_by_crop[\"nitrogen\"], width=0.27, label=\"Nitrogen (N)\", color=\"#4e79a7\")\n",
    "plt.bar(x, averages_by_crop[\"phosphorus\"], width=0.27, label=\"Phosphorus (P)\", color=\"#f28e2b\")\n",
    "plt.bar(x + 0.27, averages_by_crop[\"potassium\"], width=0.27, label=\"Potassium (K)\", color=\"#59a14f\")\n",
    "plt.xticks(x, crops)\n",
    "plt.title(\"Average N, P and K in the soil for each crop\")\n",
    "plt.ylabel(\"kg per hectare\")\n",
    "plt.legend()\n",
    "plt.tight_layout()\n",
    "plt.savefig(\"chart3_npk.png\")\n",
    "plt.close()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Stage 4 - Modelling"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(\"--- Stage 4: Modelling ---\")\n",
    "\n",
    "# Train-test split: shuffle (the same way every run), then 80% / 20%.\n",
    "data = data.sample(frac=1, random_state=3)\n",
    "cut = int(len(data) * 0.8)\n",
    "train = data[:cut]\n",
    "test = data[cut:]\n",
    "print(\"Training rows :\", len(train))\n",
    "print(\"Testing rows  :\", len(test))\n",
    "\n",
    "# Rainfall runs into hundreds and pH stays under 10. Each feature is rescaled\n",
    "# to run from 0 to 1 (using the training data only), so that no feature wins\n",
    "# just because its numbers are bigger.\n",
    "low = train[features].min()\n",
    "high = train[features].max()\n",
    "train_scaled = (train[features] - low) / (high - low)\n",
    "test_scaled = (test[features] - low) / (high - low)\n",
    "\n",
    "k = 5\n",
    "print(\"Neighbours asked (k) :\", k)\n",
    "print()\n",
    "\n",
    "predictions = []\n",
    "for i in range(len(test)):\n",
    "    row = test_scaled.iloc[i]\n",
    "    # Distance from this field to EVERY training field.\n",
    "    distance = ((train_scaled - row) ** 2).sum(axis=1)\n",
    "    # The k closest fields, and the crop most of them grew.\n",
    "    nearest = distance.nsmallest(k).index\n",
    "    predictions.append(train.loc[nearest, label].mode()[0])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Stage 5 - Evaluation"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(\"--- Stage 5: Evaluation ---\")\n",
    "actual = list(test[label])\n",
    "\n",
    "correct = 0\n",
    "for i in range(len(actual)):\n",
    "    if predictions[i] == actual[i]:\n",
    "        correct = correct + 1\n",
    "accuracy = correct / len(actual) * 100\n",
    "print(\"Correct recommendations :\", correct, \"of\", len(actual))\n",
    "print(\"Accuracy                :\", round(accuracy, 1), \"%\")\n",
    "print()\n",
    "\n",
    "matrix = pd.crosstab(pd.Series(actual, name=\"Actual\"),\n",
    "                     pd.Series(predictions, name=\"Predicted\"))\n",
    "matrix = matrix.reindex(index=crops, columns=crops, fill_value=0)\n",
    "print(\"Confusion matrix:\")\n",
    "print(matrix)\n",
    "print()\n",
    "\n",
    "print(\"Crop        Precision   Recall\")\n",
    "for c in crops:\n",
    "    tp = matrix.loc[c, c]\n",
    "    if matrix[c].sum() > 0:\n",
    "        precision = tp / matrix[c].sum() * 100\n",
    "    else:\n",
    "        precision = 0\n",
    "    if matrix.loc[c].sum() > 0:\n",
    "        recall = tp / matrix.loc[c].sum() * 100\n",
    "    else:\n",
    "        recall = 0\n",
    "    print(c.ljust(10), str(round(precision, 1)).rjust(9), \"%\", str(round(recall, 1)).rjust(7), \"%\")\n",
    "print()\n",
    "\n",
    "# Chart 4: the confusion matrix.\n",
    "plt.figure(figsize=(6.5, 5.5))\n",
    "plt.imshow(matrix.values, cmap=\"Greens\")\n",
    "for r in range(len(crops)):\n",
    "    for c in range(len(crops)):\n",
    "        value = matrix.values[r][c]\n",
    "        if value > matrix.values.max() / 2:\n",
    "            ink = \"white\"\n",
    "        else:\n",
    "            ink = \"black\"\n",
    "        plt.text(c, r, value, ha=\"center\", va=\"center\", fontsize=13, color=ink)\n",
    "plt.xticks(range(len(crops)), crops)\n",
    "plt.yticks(range(len(crops)), crops)\n",
    "plt.xlabel(\"Recommended crop\")\n",
    "plt.ylabel(\"Crop that grew well\")\n",
    "plt.title(\"Confusion matrix (testing data)\")\n",
    "plt.tight_layout()\n",
    "plt.savefig(\"chart4_confusion_matrix.png\")\n",
    "plt.close()\n",
    "\n",
    "# How much does the choice of k matter? Try every k from 1 to 15.\n",
    "k_values = list(range(1, 16))\n",
    "k_accuracy = []\n",
    "for kk in k_values:\n",
    "    right = 0\n",
    "    for i in range(len(test)):\n",
    "        distance = ((train_scaled - test_scaled.iloc[i]) ** 2).sum(axis=1)\n",
    "        nearest = distance.nsmallest(kk).index\n",
    "        if train.loc[nearest, label].mode()[0] == actual[i]:\n",
    "            right = right + 1\n",
    "    k_accuracy.append(right / len(test) * 100)\n",
    "\n",
    "print(\"Accuracy for k = 1 :\", round(k_accuracy[0], 1), \"%\")\n",
    "print(\"Accuracy for k = 5 :\", round(k_accuracy[4], 1), \"%\")\n",
    "print(\"Accuracy for k = 15 :\", round(k_accuracy[14], 1), \"%\")\n",
    "print()\n",
    "\n",
    "# Chart 5: accuracy for each value of k.\n",
    "plt.figure(figsize=(7, 4))\n",
    "plt.plot(k_values, k_accuracy, marker=\"o\", color=\"#2ca02c\")\n",
    "plt.title(\"How many neighbours should the model ask?\")\n",
    "plt.xlabel(\"k (number of neighbours)\")\n",
    "plt.ylabel(\"Accuracy on testing data (%)\")\n",
    "plt.xticks(k_values)\n",
    "plt.ylim(80, 100)\n",
    "plt.grid(True, linestyle=\"--\", alpha=0.4)\n",
    "plt.tight_layout()\n",
    "plt.savefig(\"chart5_choosing_k.png\")\n",
    "plt.close()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Stage 6 - Deployment"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(\"--- Stage 6: Deployment ---\")\n",
    "print(\"Three new soil test cards:\")\n",
    "\n",
    "new_fields = pd.DataFrame({\n",
    "    \"nitrogen\":      [85, 38, 120],\n",
    "    \"phosphorus\":    [45, 70, 44],\n",
    "    \"potassium\":     [41, 79, 19],\n",
    "    \"ph\":            [6.2, 7.4, 7.1],\n",
    "    \"rainfall_mm\":   [240, 78, 82],\n",
    "    \"temperature_c\": [25, 19, 24],\n",
    "})\n",
    "new_scaled = (new_fields - low) / (high - low)\n",
    "\n",
    "for i in range(len(new_fields)):\n",
    "    distance = ((train_scaled - new_scaled.iloc[i]) ** 2).sum(axis=1)\n",
    "    nearest = distance.nsmallest(k).index\n",
    "    votes = train.loc[nearest, label]\n",
    "    best = votes.mode()[0]\n",
    "    agree = (votes == best).sum()\n",
    "    f = new_fields.iloc[i]\n",
    "    print(\"N\", int(f[\"nitrogen\"]), \" P\", int(f[\"phosphorus\"]), \" K\", int(f[\"potassium\"]),\n",
    "          \" pH\", f[\"ph\"], \" rain\", f[\"rainfall_mm\"], \"mm  ->\", best,\n",
    "          \"(\" + str(agree), \"of\", k, \"neighbours agree)\")"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "name": "python"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}