{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "20b14013-9fe1-491a-aafd-0b5f5fb03179",
   "metadata": {
    "tags": []
   },
   "source": [
    "# Écosystème data science Python\n",
    "\n",
    "## Batteries included\n",
    "\n",
    "On dit souvent que Python est livré « avec les piles » tant sa bibliothèque standard, riche de plus de 200 packages et modules, répond aux problèmes courants\n",
    "les plus variés.  \n",
    "Le Python Package Index (https://pypi.org) fournit une grande quantité de modules tierces. Le domaine de la *Data Science* est en pleine extension.\n",
    "\n",
    "On va explorer ces outils de base.\n",
    "\n",
    "## Des outils performants"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "bec9b154-c277-458c-bf09-1e68b7ac2769",
   "metadata": {},
   "source": [
    "### NumPy\n",
    "\n",
    "Le package NumPy est indispensable pour faire du calcul scientifique.  \n",
    "Pour modéliser les vecteurs, matrices et, plus généralement les tableaux, NumPy fournit le type `ndarray`, c'est-à-dire les tableaux à n dimensions.\n",
    "\n",
    "On note deux différences majeures entre les tableaux numpy et les listes :\n",
    "- les `ndarray` sont homogènes, c’est-à-dire constitués d’éléments **du même type**. On trouvera donc des tableaux d’entiers, de flottants, de chaînes de caractères, etc. ;\n",
    "- la taille des tableaux est fixée à la création.\n",
    "\n",
    "***\n",
    "C'est un changement de philosophie : La **souplesse et la simplicité** de Python, tant prônées par GvR, sont ici remplacées par l’**efficacité**.\n",
    "***\n",
    "\n",
    "#### Exemples\n",
    "\n",
    "![Tableau 2D](2D.png)\n",
    "\n",
    "**Exemple de tableau 2D**\n",
    "\n",
    "***\n",
    "\n",
    "![Tableau 3D](3D.png)\n",
    "\n",
    "**Exemple de tableau 3D formé de 3 tabeaux 2D**\n",
    "\n",
    "#### Performances de NumPy"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "91f5e7c4-718f-4bc7-97a4-225bd7b5a231",
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np         # convention de la communauté scientifique"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "042268ea-e1ae-4ef1-b52d-138d4d8a4567",
   "metadata": {},
   "outputs": [],
   "source": [
    "liste = list(range(1_000))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "7c18c4ed-cb91-4ef3-acbd-c54f15f0f983",
   "metadata": {},
   "outputs": [],
   "source": [
    "a = np.array(liste)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5549433d-2b06-413e-afe3-debc67862355",
   "metadata": {},
   "source": [
    "Liste en compréhension classique :"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "c3cf2648-2c8a-4707-8c24-160c5e5d9fd3",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "211 µs ± 516 ns per loop (mean ± std. dev. of 7 runs, 1000 loops each)\n"
     ]
    }
   ],
   "source": [
    "%timeit [x**2 for x in liste]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "ad9726bf-2a4c-493c-9bf2-a56d74af3f60",
   "metadata": {},
   "outputs": [],
   "source": [
    "Utilisation du `ndarray` :"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "e4ed0ec7-4ec8-43ca-b38e-da61b5228854",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "88.5 µs ± 537 ns per loop (mean ± std. dev. of 7 runs, 10000 loops each)\n"
     ]
    }
   ],
   "source": [
    "%timeit [x**2 for x in a]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "e99b16c8-d849-40ec-b891-77ae5add920d",
   "metadata": {},
   "outputs": [],
   "source": [
    "Magie de la vectorisation :"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "18a61277-c274-4f8f-83ff-cb44d1b26f8f",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "797 ns ± 1.39 ns per loop (mean ± std. dev. of 7 runs, 1000000 loops each)\n"
     ]
    }
   ],
   "source": [
    "%timeit a**2"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0e4534f1-cbca-49b2-8ad4-4fa15c6f196e",
   "metadata": {},
   "source": [
    "#### Souplesse de Pandas"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0528ec8c-84a3-4309-a1af-d8e4ccb6b033",
   "metadata": {},
   "source": [
    "### Le tableau NumPy\n",
    "\n",
    "Il est stocké dans une zone de **mémoire continue** qui ne contient que des éléments du même type (tableaux homogènes). Ce qui permet :\n",
    "- d'utiliser des fonctions optimisés pour un type donné\n",
    "- de parcourir un tableau de manière très efficace"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "e45c51db-d6eb-4562-9cd0-0e0dcf033532",
   "metadata": {},
   "outputs": [],
   "source": [
    "np.ones(shape=3)  # création d'un tableau de 1 ligne et de trois colonnes de 1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "9d34939e-42c7-4d4f-a405-c329f63dae58",
   "metadata": {},
   "outputs": [],
   "source": [
    "a = np.ones(shape=(2, 3))  # création d'un tableau de 2 lignes et 3 colonnes"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4f18c5f7-22b7-4fbe-b1df-fe31554223d5",
   "metadata": {},
   "source": [
    "Les tableaux NumPy sont des objets mutables (mais de taille fixe) :"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "9b823985-529a-48f8-ba1e-19959d3e27d9",
   "metadata": {},
   "outputs": [],
   "source": [
    "a[1, 1] = 88\n",
    "a"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8664e047-f806-4f81-aca4-9f6602c77af0",
   "metadata": {},
   "source": [
    "Types de données d'un tableau NumPy (appelé le `dtype`) :"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "a5207833-ffa0-4e93-b342-6af4e7f59aa5",
   "metadata": {},
   "outputs": [],
   "source": [
    "np.sctypes"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "97c95977-78c5-4dfa-b800-94aaffeb3816",
   "metadata": {},
   "source": [
    "Examen d'un tableau"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "fadb5f96-c22b-485c-875e-5cdf30587e9d",
   "metadata": {},
   "outputs": [],
   "source": [
    "a = np.array([31, 84, 101], dtype=np.int8)  # création à partir d'une liste\n",
    "a"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "2fcdb8d2-4f18-4af2-a2bd-6d71b5262b47",
   "metadata": {},
   "outputs": [],
   "source": [
    "a.dtype  # son type"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "0d39b947-4f18-4dfd-8930-756dee04a49e",
   "metadata": {},
   "outputs": [],
   "source": [
    "a.itemsize  # taille d'un élément en octets"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "46f4a5da-fd83-4d8d-bab0-f579c14c3767",
   "metadata": {},
   "outputs": [],
   "source": [
    "a.nbytes  # taille du tableau"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "98d097e7-7806-4971-a5be-8bf19bc9dc3d",
   "metadata": {},
   "source": [
    "Attention au dépassement !"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "id": "0a0ca142-c356-4d27-a0ba-48c44f8db89a",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([  31,   84, -128], dtype=int8)"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "np.array([ 31,  84.7, 128], dtype=np.int8)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "455eaa8a-e61d-48de-ba48-c0dddc6878c5",
   "metadata": {},
   "source": [
    "Valeur invalide `nan` : *Not A Number*.  \n",
    "Elle n'est défini que pour les `float`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "id": "e85571b3-1974-4da5-8b7e-6b9661beed3d",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "dtype('float64')"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "a = np.array([5, 3, np.nan])\n",
    "a.dtype"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "569eb556-8c06-4bef-b76d-e7b071ca80fc",
   "metadata": {},
   "outputs": [],
   "source": [
    "Il est préférable de toujours spécifier le type des données pour éviter les conversions implicites :"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "id": "47f759e4-4b0c-4b5d-a02d-b52937ffcc22",
   "metadata": {},
   "outputs": [
    {
     "ename": "ValueError",
     "evalue": "cannot convert float NaN to integer",
     "output_type": "error",
     "traceback": [
      "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[0;31mValueError\u001b[0m                                Traceback (most recent call last)",
      "\u001b[0;32m/tmp/ipykernel_6841/3868602413.py\u001b[0m in \u001b[0;36m<module>\u001b[0;34m\u001b[0m\n\u001b[0;32m----> 1\u001b[0;31m \u001b[0mb\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mnp\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0marray\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;36m5\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;36m3\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mnp\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mnan\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mdtype\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mnp\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mint16\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m      2\u001b[0m \u001b[0mb\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mdtype\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;31mValueError\u001b[0m: cannot convert float NaN to integer"
     ]
    }
   ],
   "source": [
    "b = np.array([5, 3, np.nan], dtype=np.int16)\n",
    "b.dtype"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4fd7f06e-02be-4435-94cb-222d787b4e7b",
   "metadata": {},
   "source": [
    "### L'indexation (*slicing*)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "aaa3ee05-c84c-4d85-ae57-90cd04ba08da",
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "id": "97b94ec4-4e76-4cac-babc-19195d87dd6f",
   "metadata": {},
   "source": [
    "En Python de base, on est passé de la **boucle** à la **liste en compréhension** puis à l’**expression génératrice**.  \n",
    "Avec NumPy on passe à la **vision globale, vectorielle** : on applique une fonction à un tableau en utilisant une Universal function, souvent appelé ufunc. Toutes les opérations usuelles sont des ufunc NumPy. \n",
    "\n",
    "On créé souvent un ndarray en utilisant la méthode `arange()` quand on connaît le **pas**, ou la méthode `linspace()` quand on connaît le **nombre de valeurs** :"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "ee391e25-082b-4e7b-a66b-b0d516b499a2",
   "metadata": {},
   "outputs": [],
   "source": [
    "a = np.arange(5)  # pas de 1 (par défaut\n",
    "a"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "8a244198-d132-4f14-b649-4be65e84b455",
   "metadata": {},
   "outputs": [],
   "source": [
    "b = np.arange(1.0, 2.0, 0.25)  # pas de 0.25\n",
    "b"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "a50d05c3-2d11-487e-8328-c68d601bed6e",
   "metadata": {},
   "outputs": [],
   "source": [
    "c = np.linspace(0.0, 5.0, 6)  # 6 valeurs\n",
    "c"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "3365f5ea-8a4f-4728-ad7a-2ff025af063f",
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.9"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}
