{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# The Mid-Day Meal Story\n",
    "\n",
    "meal_story.py\n",
    "\n",
    "Capstone project: a data story about the Mid-Day Meal Scheme - how\n",
    "enrolment, attendance and dropout in one state's primary schools changed\n",
    "after the scheme began in 1995 and cooked meals arrived in 2001.\n",
    "\n",
    "CBSE Class XII, Artificial Intelligence (843), Part C - Capstone Project.\n",
    "This is the curriculum's own sample data-storytelling problem.\n",
    "SDG 4 (Quality Education) and SDG 2 (Zero Hunger).\n",
    "\n",
    "Data Science Methodology, 10 steps. The analytics is DESCRIPTIVE (what\n",
    "happened) and DIAGNOSTIC (why it may have happened). Two models of the\n",
    "dropout trend are compared with leave-one-out cross validation:\n",
    "  A. one straight line for all the years\n",
    "  B. a line allowed to change slope in 2001 and again in 2012\n",
    "     (segmented regression)\n",
    "The output of the project is a data story, structured on Freytag's pyramid.\n",
    "\n",
    "Libraries: pandas, numpy, matplotlib, scikit-learn."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Step 1 - Business Understanding"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(\"=\" * 64)\n",
    "print(\"THE MID-DAY MEAL STORY\")\n",
    "print(\"=\" * 64)\n",
    "print(\"Question : did dropout fall after the Mid-Day Meal Scheme began,\")\n",
    "print(\"           and is the change large enough to tell a story about?\")\n",
    "print()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Step 2 - Analytic Approach"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(\"Analytic approach : descriptive (what happened) and diagnostic (why)\")\n",
    "print()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Step 3 - Data Requirements"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "from sklearn.linear_model import LinearRegression\n",
    "from sklearn.model_selection import LeaveOneOut, cross_val_score\n",
    "\n",
    "pd.set_option(\"display.width\", 130)\n",
    "pd.set_option(\"display.max_columns\", 20)\n",
    "required = [\"year\", \"enrolment_lakh\", \"girls_share_pct\", \"attendance_pct\", \"dropout_pct\", \"meal_coverage_pct\"]\n",
    "print(\"Columns required :\", required)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Step 4 - Data Collection"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "data = pd.read_csv(\"school_data.csv\")\n",
    "print(\"Rows collected   :\", len(data))\n",
    "print(\"Years covered    :\", data[\"year\"].min(), \"to\", data[\"year\"].max())\n",
    "print()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Step 5 - Data Understanding"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(\"--- Step 5: Data Understanding ---\")\n",
    "print(\"Missing values:\")\n",
    "print(data.isnull().sum().to_string())\n",
    "print()\n",
    "print(data.describe().round(1))\n",
    "print()\n",
    "\n",
    "events = {1995: \"Scheme begins\\n(dry rations)\", 2001: \"Cooked meals\\n(court order)\",\n",
    "          2010: \"RTE Act\", 2020: \"COVID-19\\nclosures\"}\n",
    "\n",
    "# Chart 1: dropout over the years, with the events marked.\n",
    "plt.figure(figsize=(9, 4.5))\n",
    "plt.plot(data[\"year\"], data[\"dropout_pct\"], marker=\"o\", color=\"#e15759\", markersize=4)\n",
    "for y in events:\n",
    "    plt.axvline(y, color=\"#999999\", linestyle=\"--\", linewidth=0.8)\n",
    "    plt.text(y + 0.3, 37, events[y], fontsize=8, va=\"top\")\n",
    "plt.title(\"Primary school dropout rate, 1990-2024\")\n",
    "plt.xlabel(\"Year\")\n",
    "plt.ylabel(\"Dropout (%)\")\n",
    "plt.grid(True, linestyle=\"--\", alpha=0.3)\n",
    "plt.tight_layout()\n",
    "plt.savefig(\"chart1_dropout.png\")\n",
    "plt.close()\n",
    "\n",
    "# Chart 2: enrolment and the share of girls.\n",
    "plt.figure(figsize=(10, 4))\n",
    "plt.subplot(1, 2, 1)\n",
    "plt.plot(data[\"year\"], data[\"enrolment_lakh\"], color=\"#4e79a7\")\n",
    "plt.title(\"Enrolment (lakh children)\")\n",
    "plt.xlabel(\"Year\")\n",
    "plt.grid(True, linestyle=\"--\", alpha=0.3)\n",
    "plt.subplot(1, 2, 2)\n",
    "plt.plot(data[\"year\"], data[\"girls_share_pct\"], color=\"#b07aa1\")\n",
    "plt.title(\"Girls as % of enrolment\")\n",
    "plt.xlabel(\"Year\")\n",
    "plt.grid(True, linestyle=\"--\", alpha=0.3)\n",
    "plt.tight_layout()\n",
    "plt.savefig(\"chart2_enrolment.png\")\n",
    "plt.close()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Step 6 - Data Preparation"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(\"--- Step 6: Data Preparation ---\")\n",
    "before = len(data)\n",
    "data = data.drop_duplicates()\n",
    "print(\"Duplicate rows removed       :\", before - len(data))\n",
    "\n",
    "# Attendance is missing for 1998 (records lost) and 2020 (schools closed).\n",
    "# 1998 is filled by linear interpolation between 1997 and 1999. 2020 is\n",
    "# left empty on purpose: there was no attendance to record, and inventing\n",
    "# a value would hide the most important fact about that year.\n",
    "data[\"attendance_pct\"] = pd.to_numeric(data[\"attendance_pct\"])\n",
    "data.loc[data[\"year\"] == 1998, \"attendance_pct\"] = np.nan\n",
    "filled = data[\"attendance_pct\"].interpolate()\n",
    "data.loc[data[\"year\"] == 1998, \"attendance_pct\"] = filled[data[\"year\"] == 1998]\n",
    "print(\"Attendance 1998 (interpolated) :\", round(data.loc[data[\"year\"] == 1998, \"attendance_pct\"].values[0], 1))\n",
    "print(\"Attendance 2020                 : left missing - schools were closed\")\n",
    "\n",
    "# Feature engineering: the period each year belongs to.\n",
    "period = []\n",
    "for y in data[\"year\"]:\n",
    "    if y < 1995:\n",
    "        period.append(\"Before the scheme\")\n",
    "    elif y < 2001:\n",
    "        period.append(\"Dry rations\")\n",
    "    else:\n",
    "        period.append(\"Cooked meals\")\n",
    "data[\"period\"] = period\n",
    "# Chart 1 shows the fall steepening in 2001 and flattening around 2012, so\n",
    "# two \"years since\" features let a straight-line model bend at both points.\n",
    "data[\"years_since_2001\"] = np.maximum(0, data[\"year\"] - 2001)\n",
    "data[\"years_since_2012\"] = np.maximum(0, data[\"year\"] - 2012)\n",
    "print()\n",
    "\n",
    "summary = data.groupby(\"period\", sort=False)[[\"dropout_pct\", \"attendance_pct\", \"enrolment_lakh\"]].mean().round(1)\n",
    "print(\"Average of each period:\")\n",
    "print(summary)\n",
    "print()\n",
    "print(\"Average dropout before the scheme :\", summary.loc[\"Before the scheme\", \"dropout_pct\"], \"%\")\n",
    "print(\"Average dropout with cooked meals :\", summary.loc[\"Cooked meals\", \"dropout_pct\"], \"%\")\n",
    "print()\n",
    "\n",
    "corr_cov = data[\"meal_coverage_pct\"].corr(data[\"dropout_pct\"])\n",
    "corr_att = data[\"meal_coverage_pct\"].corr(data[\"attendance_pct\"])\n",
    "print(\"Correlation, meal coverage and dropout    :\", round(corr_cov, 2))\n",
    "print(\"Correlation, meal coverage and attendance :\", round(corr_att, 2))\n",
    "print()\n",
    "\n",
    "# Chart 3: meal coverage against dropout, coloured by period.\n",
    "colours = {\"Before the scheme\": \"#9c9c9c\", \"Dry rations\": \"#f28e2b\", \"Cooked meals\": \"#59a14f\"}\n",
    "plt.figure(figsize=(7, 5))\n",
    "for p in colours:\n",
    "    part = data[data[\"period\"] == p]\n",
    "    plt.scatter(part[\"meal_coverage_pct\"], part[\"dropout_pct\"], color=colours[p], label=p, s=40)\n",
    "plt.title(\"Meal coverage and dropout, one dot per year\")\n",
    "plt.xlabel(\"Schools serving meals (%)\")\n",
    "plt.ylabel(\"Dropout (%)\")\n",
    "plt.legend()\n",
    "plt.grid(True, linestyle=\"--\", alpha=0.3)\n",
    "plt.tight_layout()\n",
    "plt.savefig(\"chart3_coverage_dropout.png\")\n",
    "plt.close()\n",
    "\n",
    "# Chart 4: attendance, with the 2020 gap visible.\n",
    "plt.figure(figsize=(9, 4))\n",
    "plt.plot(data[\"year\"], data[\"attendance_pct\"], marker=\"o\", markersize=4, color=\"#4e79a7\")\n",
    "plt.axvline(2001, color=\"#999999\", linestyle=\"--\", linewidth=0.8)\n",
    "plt.title(\"Average attendance (2020 missing: schools closed)\")\n",
    "plt.xlabel(\"Year\")\n",
    "plt.ylabel(\"Attendance (%)\")\n",
    "plt.grid(True, linestyle=\"--\", alpha=0.3)\n",
    "plt.tight_layout()\n",
    "plt.savefig(\"chart4_attendance.png\")\n",
    "plt.close()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Step 7 - Modelling"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(\"--- Step 7: Modelling ---\")\n",
    "# The trend is modelled on 1990-2019. 2020-2024 are left out of the fit:\n",
    "# COVID-19 was an outside shock that no school-meal model should explain.\n",
    "trend = data[data[\"year\"] <= 2019]\n",
    "y = trend[\"dropout_pct\"]\n",
    "X_a = trend[[\"year\"]]\n",
    "X_b = trend[[\"year\", \"years_since_2001\", \"years_since_2012\"]]\n",
    "model_a = LinearRegression().fit(X_a, y)\n",
    "model_b = LinearRegression().fit(X_b, y)\n",
    "print(\"Model A slope (all years)            :\", round(model_a.coef_[0], 2), \"points a year\")\n",
    "slope_before = model_b.coef_[0]\n",
    "slope_meals = model_b.coef_[0] + model_b.coef_[1]\n",
    "slope_plateau = slope_meals + model_b.coef_[2]\n",
    "print(\"Model B slope before 2001            :\", round(slope_before, 2), \"points a year\")\n",
    "print(\"Model B slope 2001 to 2012           :\", round(slope_meals, 2), \"points a year\")\n",
    "print(\"Model B slope after 2012             :\", round(slope_plateau, 2), \"points a year\")\n",
    "print()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Step 8 - Evaluation"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(\"--- Step 8: Evaluation ---\")\n",
    "# Leave-one-out cross validation: each year in turn is held out, the model\n",
    "# is fitted on the other years, and the held-out year is predicted.\n",
    "loo = LeaveOneOut()\n",
    "err_a = np.sqrt(-cross_val_score(LinearRegression(), X_a, y, cv=loo, scoring=\"neg_mean_squared_error\"))\n",
    "err_b = np.sqrt(-cross_val_score(LinearRegression(), X_b, y, cv=loo, scoring=\"neg_mean_squared_error\"))\n",
    "rmse_a = np.sqrt((err_a ** 2).mean())\n",
    "rmse_b = np.sqrt((err_b ** 2).mean())\n",
    "print(\"Years used for the trend          :\", len(trend))\n",
    "print(\"Leave-one-out RMSE, Model A       :\", round(rmse_a, 2), \"points\")\n",
    "print(\"Leave-one-out RMSE, Model B       :\", round(rmse_b, 2), \"points\")\n",
    "print(\"R squared, Model A                :\", round(model_a.score(X_a, y), 3))\n",
    "print(\"R squared, Model B                :\", round(model_b.score(X_b, y), 3))\n",
    "print()\n",
    "\n",
    "# Chart 5: the two trend models over the data.\n",
    "plt.figure(figsize=(9, 4.5))\n",
    "plt.scatter(trend[\"year\"], y, color=\"#e15759\", s=20, label=\"Actual dropout\")\n",
    "plt.plot(trend[\"year\"], model_a.predict(X_a), color=\"#9c9c9c\", label=\"Model A: one line\")\n",
    "plt.plot(trend[\"year\"], model_b.predict(X_b), color=\"#59a14f\", linewidth=2, label=\"Model B: slope changes in 2001 and 2012\")\n",
    "plt.title(\"Two models of the dropout trend, 1990-2019\")\n",
    "plt.xlabel(\"Year\")\n",
    "plt.ylabel(\"Dropout (%)\")\n",
    "plt.legend()\n",
    "plt.grid(True, linestyle=\"--\", alpha=0.3)\n",
    "plt.tight_layout()\n",
    "plt.savefig(\"chart5_models.png\")\n",
    "plt.close()\n",
    "\n",
    "# Chart 6: leave-one-out error for each year.\n",
    "plt.figure(figsize=(9, 4))\n",
    "plt.bar(trend[\"year\"] - 0.2, err_a, width=0.4, color=\"#9c9c9c\", label=\"Model A\")\n",
    "plt.bar(trend[\"year\"] + 0.2, err_b, width=0.4, color=\"#59a14f\", label=\"Model B\")\n",
    "plt.title(\"Leave-one-out error for each year\")\n",
    "plt.xlabel(\"Year held out\")\n",
    "plt.ylabel(\"Error (percentage points)\")\n",
    "plt.legend()\n",
    "plt.tight_layout()\n",
    "plt.savefig(\"chart6_loo_errors.png\")\n",
    "plt.close()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Step 9 - Deployment"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(\"--- Step 9: Deployment: the data story ---\")\n",
    "d_1990 = data.loc[data[\"year\"] == 1990, \"dropout_pct\"].values[0]\n",
    "d_2001 = data.loc[data[\"year\"] == 2001, \"dropout_pct\"].values[0]\n",
    "d_2012 = data.loc[data[\"year\"] == 2012, \"dropout_pct\"].values[0]\n",
    "d_2021 = data.loc[data[\"year\"] == 2021, \"dropout_pct\"].values[0]\n",
    "d_2024 = data.loc[data[\"year\"] == 2024, \"dropout_pct\"].values[0]\n",
    "story = [\n",
    "    [\"Exposition\", \"1990: \" + str(d_1990) + \"% of children left primary school early.\"],\n",
    "    [\"Rising action\", \"1995 dry rations; 2001 cooked meals in every school.\"],\n",
    "    [\"Climax\", \"Dropout fell from \" + str(d_2001) + \"% (2001) to \" + str(d_2012) + \"% (2012).\"],\n",
    "    [\"Falling action\", \"A plateau after 2012; COVID pushed it to \" + str(d_2021) + \"% in 2021.\"],\n",
    "    [\"Resolution\", \"Back to \" + str(d_2024) + \"% by 2024 - meals helped; the last mile needs more.\"],\n",
    "]\n",
    "for stage, beat in story:\n",
    "    print(stage.ljust(15), \":\", beat)\n",
    "print()\n",
    "\n",
    "# Chart 7: Freytag's pyramid, carrying this story's beats.\n",
    "xs = [0, 1, 2, 3, 4]\n",
    "ys = [0, 1, 2, 1, 0.3]\n",
    "plt.figure(figsize=(10, 4.8))\n",
    "plt.plot(xs, ys, color=\"#4e79a7\", marker=\"o\", linewidth=2)\n",
    "for i in range(5):\n",
    "    plt.text(xs[i], ys[i] + 0.12, story[i][0], ha=\"center\", fontsize=10, fontweight=\"bold\")\n",
    "    plt.text(xs[i], ys[i] - 0.25, story[i][1].replace(\"; \", \";\\n\").replace(\" from\", \"\\nfrom\").replace(\" - \", \"\\n\"),\n",
    "             ha=\"center\", fontsize=7.5, va=\"top\")\n",
    "plt.ylim(-1.1, 2.5)\n",
    "plt.xlim(-0.6, 4.6)\n",
    "plt.axis(\"off\")\n",
    "plt.title(\"The story on Freytag's pyramid\")\n",
    "plt.tight_layout()\n",
    "plt.savefig(\"chart7_freytag.png\")\n",
    "plt.close()\n",
    "\n",
    "# Chart 8: the headline chart of the story.\n",
    "plt.figure(figsize=(9, 4.8))\n",
    "plt.fill_between(data[\"year\"], data[\"dropout_pct\"], color=\"#e15759\", alpha=0.25)\n",
    "plt.plot(data[\"year\"], data[\"dropout_pct\"], color=\"#e15759\", linewidth=2)\n",
    "plt.axvspan(2001, 2012, color=\"#59a14f\", alpha=0.12)\n",
    "plt.text(2006.5, 35, \"Cooked meals:\\nthe steepest fall\", ha=\"center\", fontsize=10, color=\"#2e7d32\")\n",
    "plt.annotate(\"COVID-19\", xy=(2021, d_2021), xytext=(2014, 26), arrowprops=dict(arrowstyle=\"->\"))\n",
    "plt.title(\"Since cooked mid-day meals began, dropout has fallen by more than half\")\n",
    "plt.xlabel(\"Year\")\n",
    "plt.ylabel(\"Dropout (%)\")\n",
    "plt.ylim(0, 42)\n",
    "plt.tight_layout()\n",
    "plt.savefig(\"chart8_headline.png\")\n",
    "plt.close()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Step 10 - Feedback"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(\"--- Step 10: Feedback ---\")\n",
    "# The story makes a claim that new data can test: dropout should stay near\n",
    "# the post-2012 plateau. Model B's plateau-era average is compared with the\n",
    "# years after COVID as they arrive.\n",
    "plateau = data[(data[\"year\"] >= 2012) & (data[\"year\"] <= 2019)][\"dropout_pct\"].mean()\n",
    "latest = data[data[\"year\"] >= 2023][\"dropout_pct\"].mean()\n",
    "print(\"Average dropout 2012-2019 (plateau) :\", round(plateau, 1), \"%\")\n",
    "print(\"Average dropout 2023-2024 (latest)  :\", round(latest, 1), \"%\")\n",
    "if abs(latest - plateau) <= 1.0:\n",
    "    print(\"Feedback: the latest years are back on the plateau - the story's ending holds.\")\n",
    "else:\n",
    "    print(\"Feedback: the latest years differ from the plateau - the story's ending must be revised.\")"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "name": "python"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}