{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "0f73c13a-6a41-41a8-b3cb-bf0aaac86ebc",
   "metadata": {},
   "source": [
    "# Deep Learning - Lab Exercise 5\n",
    "\n",
    "\n",
    "**WARNING:** you must have finished the previous exercise before this one as you will re-use parts of the code.\n",
    "\n",
    "In the first lab exercise, we built a simple linear classifier.\n",
    "Although it can give reasonable results on the MNIST dataset (~92.5% of accuracy), deeper neural networks can achieve more the 99% accuracy.\n",
    "However, it can quickly become really impracical to explicitly code forward and backward passes.\n",
    "Hence, it is useful to rely on an auto-diff library where we specify the forward pass once, and the backward pass is automatically deduced from the computational graph structure.\n",
    "\n",
    "In this lab exercise, we will build a small and simple auto-diff lib that mimics the autograd mechanism from Pytorch (of course, we will simplify a lot!)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "576d2e25-2822-46e1-a6de-80f793898317",
   "metadata": {},
   "outputs": [],
   "source": [
    "# import libs that we will use\n",
    "import os\n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "import math\n",
    "\n",
    "# To load the data we will use the script of Gaetan Marceau Caron\n",
    "# You can download it from the course webiste and move it to the same directory that contains this ipynb file\n",
    "import dataset_loader\n",
    "\n",
    "%matplotlib inline"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "b8ce741f-b2a7-422d-96de-d71cb79cfdbc",
   "metadata": {},
   "outputs": [],
   "source": [
    "if(\"mnist.pkl.gz\" not in os.listdir(\".\")):\n",
    "    # this link doesn't work any more,\n",
    "    # seach on google for the file \"mnist.pkl.gz\"\n",
    "    # and download it\n",
    "    !wget https://github.com/mnielsen/neural-networks-and-deep-learning/raw/master/data/mnist.pkl.gz\n",
    "\n",
    "\n",
    "# if you have it somewhere else, you can comment the lines above\n",
    "# and overwrite the path below\n",
    "mnist_path = \"./mnist.pkl.gz\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "a4204abd-5cce-48f1-a78f-392a8275fb5e",
   "metadata": {},
   "outputs": [],
   "source": [
    "# load the 3 splits\n",
    "train_data, dev_data, test_data = dataset_loader.load_mnist(mnist_path)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f6d35084-f93b-4245-b5cf-55baf1eb5f67",
   "metadata": {},
   "source": [
    "## Computation Graph\n",
    "\n",
    "Instead of directly manipulating numpy arrays, we will manipulate abstraction that contains:\n",
    "- a value (i.e. a numpy array)\n",
    "- a bool indicating if we wish to compute the gradient with respect to the value\n",
    "- the gradient with respect to the value\n",
    "- the operation to call during backpropagation\n",
    "\n",
    "There will be two kind of nodes:\n",
    "- ComputationGraphNode: a generic computation node\n",
    "- Parameter: a computation node that is used to store parameters of the network. Parameters are always leaf nodes, i.e. they cannot be build from other computation nodes.\n",
    "\n",
    "Our implementation of the backward pass will be really simple and incorrect in the general case (i.e. won't work with computation graph with loops).\n",
    "We will just apply the derivative function for a given tensor and then call the ones of its antecedents, recursively.\n",
    "This simple algorithm is good enough for this exercise.\n",
    "\n",
    "Note that a real implementation of backprop will store temporary values during forward that can be used during backward to improve computation speed. We do not do that here.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "a1535415-f6ce-486a-850a-120cd9008642",
   "metadata": {},
   "outputs": [],
   "source": [
    "class ComputationGraphNode(object):\n",
    "    \n",
    "    def __init__(self, data, require_grad=False):\n",
    "        # we initialise the value of the node and the grad\n",
    "        if(not isinstance(data, np.ndarray)):\n",
    "            data = np.array(data)\n",
    "        self.value = data\n",
    "        self.grad = None\n",
    "        \n",
    "        self.require_grad = require_grad\n",
    "        self.func = None\n",
    "        self.input_nodes = None\n",
    "        self.func_parameters = []\n",
    "    \n",
    "    def set_input_nodes(self, *nodes):\n",
    "        self.input_nodes = list(nodes)\n",
    "\n",
    "    def set_func_parameters(self, *func_parameters):\n",
    "        self.func_parameters = list(func_parameters)\n",
    "    \n",
    "    def set_func(self, func):\n",
    "        self.func = func\n",
    "\n",
    "    def zero_grad(self):\n",
    "        if self.grad is not None:\n",
    "            self.grad.fill(0)\n",
    "\n",
    "    def set_gradient(self, gradient):\n",
    "        \"\"\"\n",
    "        Accumulate gradient for this tensor\n",
    "        \"\"\"\n",
    "        if gradient.shape != self.value.shape:\n",
    "            print(gradient.shape, self.value.shape)\n",
    "            raise RuntimeError(\"Invalid gradient dimension\")\n",
    "        if self.grad is None:\n",
    "            self.grad = gradient\n",
    "        else:\n",
    "            self.grad += gradient\n",
    "    \n",
    "    def backward(self, g=None):\n",
    "        if g is None:\n",
    "            g = self.value.copy()\n",
    "            g.fill(1.)\n",
    "        self.set_gradient(g)\n",
    "        if self.func is not None:\n",
    "            grad_list = self.func.backward(*(self.input_nodes + self.func_parameters + [g]))\n",
    "            for input_node, ngrad in zip(self.input_nodes, grad_list):\n",
    "                input_node.backward(ngrad)\n",
    "    \n",
    "    def __add__(self, y):\n",
    "        if not isinstance(y, ComputationGraphNode):\n",
    "            y = ComputationGraphNode(y)\n",
    "        return Addition()(self, y)\n",
    "\n",
    "    def __getitem__(self, slice):\n",
    "        return Selection()(self, slice)\n",
    "\n",
    "    def __str__(self):\n",
    "        return self.value.__str__()\n",
    "\n",
    "    def __repr__(self):\n",
    "        return self.value.__str__()\n",
    "\n",
    "class Parameter(ComputationGraphNode):\n",
    "    def __init__(self, data, name=\"default\"):\n",
    "        super().__init__(data, require_grad=True)\n",
    "        self.name  = name\n",
    "\n",
    "    def backward(self, g=None):\n",
    "        if g is not None:\n",
    "            self.set_gradient(g)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b24dbd86-f592-4f8a-8657-272568a44f02",
   "metadata": {},
   "source": [
    "The class `Operation` is a class that three methods you should reimplement only the forward and the backward methods.\n",
    "* The `forward` method compute the function w.r.t inputs and return a new node that must contains information for backward pass.\n",
    "* The `backward` functions compute the gradient of the function w.r.t gradient of the output and other informations (forward pass input, parameter of the function...).**It should return a tuple**\n",
    "\n",
    "For better understanding below two operation are implemented, the selection and the addition (notice that it should not works yet since we do not defined what is a node)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "ca2caa2d-da91-4765-b709-b31acad0c255",
   "metadata": {},
   "outputs": [],
   "source": [
    "class Operation(object):\n",
    "    @staticmethod\n",
    "    def forward(*args):\n",
    "        raise NotImplementedError(\"It is an abstract method\")\n",
    "    \n",
    "    def __call__(self, *args):\n",
    "        output_node = self.forward(*args)\n",
    "        output_node.set_func(self)\n",
    "        return output_node\n",
    "        \n",
    "    @staticmethod\n",
    "    def backward(*args):\n",
    "        pass\n",
    "class Addition(Operation):\n",
    "    @staticmethod\n",
    "    def forward(x, y):\n",
    "        output_array = x.value + y.value\n",
    "        output_node = ComputationGraphNode(output_array)\n",
    "        output_node.set_input_nodes(x, y)\n",
    "        return output_node\n",
    "\n",
    "    @staticmethod\n",
    "    def backward(x, y, gradient):\n",
    "        return (gradient, gradient)\n",
    "\n",
    "class Selection(Operation):\n",
    "    @staticmethod\n",
    "    def forward(x, slice):\n",
    "        np_x = x.value\n",
    "\n",
    "        output_array = np_x.__getitem__(slice)\n",
    "        \n",
    "        output_node = ComputationGraphNode(output_array)\n",
    "        output_node.set_input_nodes(x)\n",
    "        output_node.set_func_parameters(slice)\n",
    "\n",
    "        return output_node\n",
    "        \n",
    "    @staticmethod\n",
    "    def backward(x, slice, gradient):\n",
    "        np_x = x.value\n",
    "\n",
    "        cgrad = np_x.copy()\n",
    "        cgrad.fill(0)\n",
    "        cgrad.__setitem__(slice, gradient)\n",
    "        \n",
    "        return cgrad,"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9ff8a571-436a-499a-986a-f8a5ae89dab4",
   "metadata": {},
   "source": [
    "**Question 1** Complete the following class "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "3a3f6684-1b56-40ad-8fe4-7dd9373f81d4",
   "metadata": {},
   "outputs": [],
   "source": [
    "class ReLU(Operation):\n",
    "    @staticmethod\n",
    "    def forward(x):\n",
    "        # we copy the value of the input node\n",
    "        np_x = x.value.copy()\n",
    "\n",
    "        # set negative elements to zero\n",
    "        np_x[np_x < 0] = 0 # notice we consider strictly < 0 \n",
    "\n",
    "        # we create the output node needing only the node x\n",
    "        output_node = ComputationGraphNode(np_x)\n",
    "        output_node.set_input_nodes(x)\n",
    "        \n",
    "        return output_node\n",
    "        \n",
    "    @staticmethod\n",
    "    def backward(x, gradient):\n",
    "      raise NotImplementedError"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "82289abf-c8b1-495f-86b7-e8e27cd0cac3",
   "metadata": {},
   "source": [
    "We recall that :  $$tanh(x)= \\frac{e^{z} - e^{-z}}{e^{z} + e^{-z}}$$ \n",
    "\n",
    "However we can have stability issues if $||z||$ is large, e.g. $e^{10000}$ will lead to computation error or infinity. Indeed in python using numpy:\n",
    "\n",
    "\n",
    ">np.exp(10000)\n",
    "\n",
    "\n",
    "will leads to :\n",
    "\n",
    ">/tmp/ipykernel_7784/2473798304.py:1: RuntimeWarning: overflow encountered in exp\n",
    ">np.exp(10000)\n",
    ">\n",
    ">inf\n",
    "\n",
    "We can use the same tricks that the one used in the softmax computation observing the simple following fact: \n",
    "$$\n",
    "\\begin{aligned}\n",
    " tanh(x) &= \\frac{e^{z} - e^{-z}}{e^{z} + e^{-z}} \\\\\n",
    " &= \\left(\\frac{e^{z} - e^{-z}}{e^{z} + e^{-z}}\\right)\\frac{e^{-a}}{e^{-a}} \\\\\n",
    " &= \\frac{e^{z}e^{-a} - e^{-z}e^{-a}}{e^{z}e^{-a} + e^{-z}e^{-a}} \\\\\n",
    "&= \\frac{e^{z-a} - e^{-z-a}}{e^{z-a} + e^{-z-a}}\n",
    "\\end{aligned}\n",
    "$$\n",
    "Thus we want that $z-a$ or $-z-a$ be small, or in our case lower than $0$.  Thus taking $a$ as the absolute value of $z$ ($|z|$) will leads to have \n",
    "$z-a\\leq 0$ and $-z-a\\leq 0$.\n",
    "\n",
    "\n",
    "For the backward notice that $tanh'(x) = 1-\\sigma(x)^2$\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "89adae37-5902-4e50-ac29-5ed332cc0f4a",
   "metadata": {},
   "outputs": [],
   "source": [
    "class TanH(Operation):\n",
    "    @staticmethod\n",
    "    def TanHCompute(x):\n",
    "        raise NotImplementedError\n",
    "        \n",
    "    @staticmethod\n",
    "    def forward(x):\n",
    "        raise NotImplementedError\n",
    "        \n",
    "    @staticmethod\n",
    "    def backward(x, gradient):\n",
    "       raise NotImplementedError\n",
    "        "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f1d873cf-a10f-4e33-ac84-0018933ab4c5",
   "metadata": {},
   "source": [
    "**Question 2:** Next, we implement the affine transform operation.\n",
    "You can reuse the code from the third lab exercise, with one major difference: you have to compute the gradient with respect to x too!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "95cf9932-ddcf-472f-8732-c645b5063cac",
   "metadata": {},
   "outputs": [],
   "source": [
    "class affine_transform(Operation):\n",
    "    @staticmethod\n",
    "    def forward(W, b, x):\n",
    "        raise NotImplementedError\n",
    "        \n",
    "    @staticmethod\n",
    "    def backward(W, b, x, gradient):\n",
    "        raise NotImplementedError"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f372bea6-9480-4c5c-b3c4-0bdc570abecb",
   "metadata": {},
   "source": [
    "**Question 3:** Define the NLL operation\n",
    "\n",
    "We recall that \n",
    "$$nll(x, y)= -log\\left(\\frac{e^{x_{y}}}{ \\sum\\limits_{i=1}^n e^{x_{ j}} }\\right) = -x_{y} + log(\\sum\\limits_{i=1}^n e^{x_{ j} })$$\n",
    "\n",
    "$$\n",
    "    \\begin{align*}\n",
    "        \\frac{\\partial nll(x, y)}{\\partial x_i} &= - \\mathbb{1}_{y = i} + \\frac{\\partial log(\\sum\\limits_{i=1}^n e^{x_{ j} })}{\\partial\\sum\\limits_{i=1}^n e^{x_{ j} }}\\frac{\\sum\\limits_{i=1}^n e^{x_{ j} }}{\\partial x_i} \\\\\n",
    "        &= - \\mathbb{1}_{y = i} + \\frac{e^{x_i}}{\\sum\\limits_{i=1}^n e^{x_{ j} }} \n",
    "    \\end{align*}\n",
    "$$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "b3f742f0-83f3-4983-ae7a-cefd511b96d2",
   "metadata": {},
   "outputs": [],
   "source": [
    "class nll(Operation):\n",
    "    @staticmethod\n",
    "    def forward(x, y):\n",
    "        raise NotImplementedError\n",
    "        \n",
    "    @staticmethod\n",
    "    def backward(x, y, gradient):\n",
    "        raise NotImplementedError"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "016217fb-a9f4-4908-b3e7-5fe372b38946",
   "metadata": {},
   "source": [
    "# Module\n",
    "\n",
    "Neural networks or parts of neural networks will be stored in Modules.\n",
    "They implement method to retrieve all parameters of the network and subnetwork."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "43ea790d-a625-4aa4-95f1-8e4fe5e558f2",
   "metadata": {},
   "outputs": [],
   "source": [
    "class Module:\n",
    "    def __init__(self):\n",
    "        raise NotImplemented(\"\")\n",
    "        \n",
    "    def parameters(self):\n",
    "        ret = []\n",
    "        for name in dir(self):\n",
    "            o = self.__getattribute__(name)\n",
    "\n",
    "            if type(o) is Parameter:\n",
    "                ret.append(o)\n",
    "            if isinstance(o, Module) or isinstance(o, ModuleList):\n",
    "                ret.extend(o.parameters())\n",
    "        return ret\n",
    "\n",
    "# if you want to store a list of Parameters or Module,\n",
    "# you must store them in a ModuleList instead of a python list,\n",
    "# in order to collect the parameters correctly\n",
    "class ModuleList(list):\n",
    "    def parameters(self):\n",
    "        ret = []\n",
    "        for m in self:\n",
    "            if type(m) is Parameter:\n",
    "                ret.append(m)\n",
    "            elif isinstance(m, Module) or isinstance(m, ModuleList):\n",
    "                ret.extend(m.parameters())\n",
    "        return ret"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d9688649-c900-4f00-bd06-38d93c1249cf",
   "metadata": {},
   "source": [
    "# Initialization and optimization\n",
    "\n",
    "**Question 1:** Implement the different initialisation methods"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "337e899b-7551-42f5-ba48-bfef526f86a3",
   "metadata": {},
   "outputs": [],
   "source": [
    "def zero_init(b):\n",
    "    b[:] = 0.\n",
    "\n",
    "def glorot_init(W):\n",
    "    '''inplace initialiase with glorot method'''\n",
    "    #W[:, :] = \n",
    "    raise NotImplementedError\n",
    "# Look at slides for the formula!\n",
    "def kaiming_init(W):\n",
    "    raise NotImplementedError('Implement the initialization')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9f4ee98a-e90a-4ddb-b575-1688371efc05",
   "metadata": {},
   "source": [
    "We will implement the Stochastic gradient descent through an object, in the init function this object will store the different parameters (in a list format). The step function will update the parameters (see slides), notice that the gradient is stored in the nodes (grad attribute). Finally it will be necessary after each update to reset all the gradient to zero (in the method zero_grad) because we do not want to accumumlate gradient of all previous step.\n",
    "\n",
    "**Question 2:** Implement the SGD "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "dbb7d09f-f4ae-472f-8b6a-f39fbaf8dccc",
   "metadata": {},
   "outputs": [],
   "source": [
    "# simple gradient descent optimizer\n",
    "class SGD:\n",
    "    def __init__(self, params, lr=0.1):\n",
    "        raise NotImplementedError\n",
    "        \n",
    "    def step(self):\n",
    "        raise NotImplementedError\n",
    "        \n",
    "    def zero_grad(self):\n",
    "        raise NotImplementedError"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "76263efd-aed5-4a9a-b954-d9df83c9e8f4",
   "metadata": {},
   "source": [
    "# Networks and training loop\n",
    "\n",
    "We first create a simple linear classifier, similar to the first lab exercise."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "50268482-0561-4e42-a85d-d7f592ca9769",
   "metadata": {},
   "outputs": [],
   "source": [
    "class LinearNetwork(Module):\n",
    "    def __init__(self, dim_input, dim_output):\n",
    "        # build the parameters\n",
    "        raise NotImplementedError\n",
    "        \n",
    "    def init_parameters(self):\n",
    "        # init parameters of the network (i.e W and b)\n",
    "        raise NotImplementedError\n",
    "        \n",
    "    def forward(self, x):\n",
    "        raise NotImplementedError"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "49500117-74d1-4012-9096-53e3b8f298fc",
   "metadata": {},
   "outputs": [],
   "source": [
    "np.random.seed(42)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "55d9ac22-7355-4d0f-828c-7a14c34c42dc",
   "metadata": {},
   "outputs": [],
   "source": [
    "# those lines should be executed correctly\n",
    "lin1 = LinearNetwork(784, 10)\n",
    "lin2 = LinearNetwork(10, 5)\n",
    "x = ComputationGraphNode(train_data[0][0])\n",
    "a = lin1.forward(x + x)\n",
    "b = TanH()(a)\n",
    "c = lin2.forward(b)\n",
    "c.backward()\n",
    "x.grad[0:10]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "78b40f7e-5c56-4c49-bcc7-82b27730e3b6",
   "metadata": {},
   "source": [
    "We will train several neural networks.\n",
    "Therefore, we encapsulate the training loop in a function.\n",
    "\n",
    "**warning**: you have to call optimizer.zero_grad() before each backward pass to reinitialize the gradient of the parameters!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "557d5a9a-c5b3-455b-9084-20470d1901db",
   "metadata": {},
   "outputs": [],
   "source": [
    "def training_loop(network, optimizer, train_data, dev_data, n_epochs=10):\n",
    "    raise NotImplementedError(\"Implement the training loop\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "0d3e4305-1736-4987-a5d6-d0902dc78738",
   "metadata": {},
   "outputs": [],
   "source": [
    "dim_input = 28*28\n",
    "dim_output = 10\n",
    "\n",
    "network = LinearNetwork(dim_input, dim_output)\n",
    "optimizer = SGD(network.parameters(), 0.01)\n",
    "\n",
    "training_loop(network, optimizer, train_data, dev_data, n_epochs=5)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c5ab8c45-1492-4be0-a3bd-d257066b5876",
   "metadata": {},
   "source": [
    "After you finished the linear network, you can move to a deep network!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "3180e14e-6490-4008-8fda-a1f7f7f71bd8",
   "metadata": {},
   "outputs": [],
   "source": [
    "class DeepNetwork(Module):\n",
    "    def __init__(self, dim_input, dim_output, hidden_dim, n_layers, tanh=False):\n",
    "        '''Initialize the different modules and call init_parameters !!!'''\n",
    "        raise NotImplementedError('implement')\n",
    "\n",
    "        \n",
    "    def init_parameters(self):\n",
    "        raise NotImplementedError('implement')\n",
    "        \n",
    "    def forward(self, x):\n",
    "        raise NotImplementedError('implement forward pass')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "54b2e256-b2fe-4e7f-8f5d-777ba54cc7be",
   "metadata": {},
   "outputs": [],
   "source": [
    "dim_input = 28*28\n",
    "dim_output = 10\n",
    "\n",
    "network = DeepNetwork(dim_input, dim_output, 100, 2)\n",
    "optimizer = SGD(network.parameters(), 0.01)\n",
    "\n",
    "training_loop(network, optimizer, train_data, dev_data, n_epochs=5)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "44af7721-8e42-4b4f-aa58-fe3b02d3d867",
   "metadata": {},
   "source": [
    "## Better Optimizer\n",
    "Implement the SGD with momentum, notice that you will need to store the cumulated gradient.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "b804a3fc-9136-4d2f-8f18-a394821401de",
   "metadata": {},
   "outputs": [],
   "source": [
    "class SGDWithMomentum:\n",
    "    def __init__(self, params, lr=0.1, momentum=0.5):\n",
    "\n",
    "    def step(self):\n",
    "        \n",
    "    def zero_grad(self):\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d8739e57-a82b-4908-9488-1b83d6e9a51f",
   "metadata": {},
   "source": [
    "## Bonus: Batch SGD\n",
    "Propose a methods to take into account batch of input"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "824876ff-1dca-44ec-a777-620dfdd0d0e1",
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "base",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.12.8"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}