Llama_3_2_Multimodal,_Stack,_Tool_Calling_and_RAG.ipynb

{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/amitsangani/Llama/blob/main/Llama_3_2_Multimodal%2C_Stack%2C_Tool_Calling_and_RAG.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "RJSnI0Xy-kCm"
      },
      "source": [
        "![Meta---Logo@1x.jpg](data:image/jpeg;base64,/9j/4QAYRXhpZgAASUkqAAgAAAAAAAAAAAAAAP/sABFEdWNreQABAAQAAABkAAD/4QMxaHR0cDovL25zLmFkb2JlLmNvbS94YXAvMS4wLwA8P3hwYWNrZXQgYmVnaW49Iu+7vyIgaWQ9Ilc1TTBNcENlaGlIenJlU3pOVGN6a2M5ZCI/PiA8eDp4bXBtZXRhIHhtbG5zOng9ImFkb2JlOm5zOm1ldGEvIiB4OnhtcHRrPSJBZG9iZSBYTVAgQ29yZSA5LjAtYzAwMCA3OS5kYTRhN2U1ZWYsIDIwMjIvMTEvMjItMTM6NTA6MDcgICAgICAgICI+IDxyZGY6UkRGIHhtbG5zOnJkZj0iaHR0cDovL3d3dy53My5vcmcvMTk5OS8wMi8yMi1yZGYtc3ludGF4LW5zIyI+IDxyZGY6RGVzY3JpcHRpb24gcmRmOmFib3V0PSIiIHhtbG5zOnhtcD0iaHR0cDovL25zLmFkb2JlLmNvbS94YXAvMS4wLyIgeG1sbnM6eG1wTU09Imh0dHA6Ly9ucy5hZG9iZS5jb20veGFwLzEuMC9tbS8iIHhtbG5zOnN0UmVmPSJodHRwOi8vbnMuYWRvYmUuY29tL3hhcC8xLjAvc1R5cGUvUmVzb3VyY2VSZWYjIiB4bXA6Q3JlYXRvclRvb2w9IkFkb2JlIFBob3Rvc2hvcCAyNC4xIChNYWNpbnRvc2gpIiB4bXBNTTpJbnN0YW5jZUlEPSJ4bXAuaWlkOjlDN0Y5QzBDNEIxRDExRUU5MjgwQUNGNjU1QzlDQjREIiB4bXBNTTpEb2N1bWVudElEPSJ4bXAuZGlkOjlDN0Y5QzBENEIxRDExRUU5MjgwQUNGNjU1QzlDQjREIj4gPHhtcE1NOkRlcml2ZWRGcm9tIHN0UmVmOmluc3RhbmNlSUQ9InhtcC5paWQ6OUM3RjlDMEE0QjFEMTFFRTkyODBBQ0Y2NTVDOUNCNEQiIHN0UmVmOmRvY3VtZW50SUQ9InhtcC5kaWQ6OUM3RjlDMEI0QjFEMTFFRTkyODBBQ0Y2NTVDOUNCNEQiLz4gPC9yZGY6RGVzY3JpcHRpb24+IDwvcmRmOlJERj4gPC94OnhtcG1ldGE+IDw/eHBhY2tldCBlbmQ9InIiPz7/7gAOQWRvYmUAZMAAAAAB/9sAhAABAQEBAQEBAQEBAQEBAQEBAQEBAQEBAQEBAQEBAQEBAQEBAQEBAQEBAQEBAgICAgICAgICAgIDAwMDAwMDAwMDAQEBAQEBAQIBAQICAgECAgMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwP/wAARCAA1APADAREAAhEBAxEB/8QAwQAAAgIDAQEBAAAAAAAAAAAACQoACwYHCAUDBAEAAQQDAQEBAAAAAAAAAAAABgAFCAkBAwQCBwoQAAAGAQEGBAMDCAYGCwAAAAECAwQFBgcIABESExQJIRUWFyIYCjEjJEFhMyW3eBkaUTK0djg5lLU2d9dYcYGhQkQ1JrY3RygRAAIBAgMEBAsGBAcAAwAAAAECAxEEABIFIRMGBzFBFAhRYXGBkbEiMnI0FaHB0UJSM/DhIxbxYqIkFxgJU3NU/9oADAMBAAIRAxEAPwB/jZYWNCaj9TWF9J2NZHK2cbi0qVXZqdGwR5aj6ds00oiqs0rtWhGwGezU09KiYSpkAE0kymVXOkgRRUhzy95ccYc0eIo+GOC7R7rUnGZjULHDGCA0s0h9mONaipO1iQiKzsqkU4y424a4B0V9e4ouVt7FTRR7zyPQkRxINruadA2AVZiqgsFTtS31DeerpPqIaZKohhmqslTJM5G1I1S8WSdQAxhK8lYuSrT+Jg3CoDu6ds5dETAP0xx3jtZ9y67g3A2j2IfmPdNrGqOKssBntoYz+lHSZXkA/U6IT+gdGIGca977ivUrsrwTANNsFNA0oinkcfqZWjZEJ/SrMB+o4zvSr9RJfa7JtYLVpRXOQYB84STd3+iBXIWwwCZlClM4JSmkFCRE42KQwioQHzZYALvIJx+AWTmf3AtD1C2a95WXq2F8ikra3O9kilNOjtDSSSRnwHduu3bTpDrwH3wdVs51teP7Vru0cis8G7SSPx7kIiOPCM6nwV6MNP4ZzXizUJjyCyphu6RF7oliTOaOnIhRTcRwgIFdxsmxcpt5GGmY9QeBwzdpIuUDeByF3htWTxfwdxNwFr8/DHF1nLY63bkZ45ANoPuujAlJI2G1JEZkYdBOJ2cN8TaFxfo8WvcOXMd1pUw9l0r0jpVlIDI69DI4DKekDGstVOrzC2j6heuMuTyiK7/qW9TpsMRJ9cLrJNkyHVYwEYos3TBFuChBcPHKiDJqBygoqU6iZDmXKLkvx1zq4h+gcGW4aOPKbi5lJS2tUY0DzSAE1NDkjRXlehyoQrFQ3mpze4L5P6D9c4unIkkqILeMBri5cCpWJCQKCozyOVjSozMCyhlocw98zVDbLctI4haQ2JqemsJWldeR9XvL5w1THhIq+l5qppqpOnBA4lCpBwEMYQKIgACNpnBXcC5TaPoy23Gjz6zrRX2plee1QMekJHFcEFVOwFtpAqaE0xWjxh35eaGraubjhBIdJ0cN7MLJBdMVHQWkkgBDHpIXYCaCo24710f98ah3V9D0DVDCHx3MvFE2TXLDN02fUx47VMQiQ2uNZxUWvUUTqGEvVJEdMybwMuLdMplAjzzp7g3EOhW8/EfKecalYoCzaeyslyqipPZ3aSQXBA27tjHIeiPeMQuPvXJ/vxaDrc8PD/NCA6deuQq36srWzMaU36LGhtwTszqHjHS+7UFsMAtXTZ82bvWThB4zeIIumjtqsm4bOmzhMqqDhuukY6S6C6RwMQ5REpiiAgIgO1cssUtvK0E6sk6MVZWBDKwNCrA7QQdhB2g7Dif8UsU8SzQsrwuoZWUgqykVBBGwgjaCNhG0Y++2vGzE2WFhVLN31UmDsJZny5hmU0m5Ym5LEmTr5jKQmWV+p7ZnLvaHaZWrOpRo2WjlFm7WQXijKppnMY5CHABHeA7OqaU7oHzjaAejw4ZZNZjjkaMo1VJHSOrBpu2z3F8Rdy/AC2b8XRMpTn8DbJalXzHFifsJCx0ueYgk9jercx4JoP4uwwDxu8aOiJkTOJ1UP0rdYC8VzbPbSZG2ilQfDhwtLuO7i3ibCDQjwYIPtz46sTZYWNN6hs7490xYQyhqAytKeUY/xNTpe42NynyjPHKEaj+DholFZVFN5PWGTUQYR7fjKLl85SSAd5w29xxtK4jT3ica5ZEhjMr+6orhWYfq88Abh3aOcwiPjuAci0oAH+jeIRQ7t/5ft3fn2dPpEn6x6Dhm+uxf/G3pGGwcWXpvlHGOOcmNI1zDNci0OoXptDvVkHLyKb26vx9gRjXbhqItl3LFOQBJQ6Y8BjEES+Ahs1MuVivgNMPaNnQP0VAPpxnm3nHrE2WFibLCxNlhY8iwT0TVoGbs888LHwVciJKemn501liMYmIZLSEi8Mi2TWcKlbM25ziVMhzmAu4oCO4NsgEmg6TjBIUFj0DAxcQd7DtkZ6ybRsO4o1PRlsyRkifZ1im1pPHOXotWXnX4HFow6+boEbFMjLCmIAdwukmBtwCYN+3S9lcxqXdaKOnaPxxxx6jZyuI0erk7Nh/DBUduXHbibLCxNlhYmywsTZYWJssLHiWWyQVNrlgt9olGkHWarCStjsU0/U5TGIgoNivJy0o9V3Dy2jBg1UVUNuHcQgjt2adp97q+oQaVpkTzajdTJFFGoq0kkjBERR1szEKB4Tjmvb2106zm1C+kWKygiaSR22KiIpZ2J6gqgk+IYrue4drdu2vDUNM358pJs8dwLp7WcL0RQ6gpVun9WUiDxZgkdREbbbzoJPJVUvMOZYU2xTmbtW5SX7cg+TWjckeAodChEb6/OqzahcilZZ8u1QxodxBUxwqaALmkKiSSQmn7m/zN1PmpxfJq0pddHiZo7ODqjhrsJUVG9loHlO0k0QEoiAG30QfT5Vuw49hciazrFdYiz2eOSkmOG6U7Y19zUWTxMirMl4sLxhKvHFkMgcDLx7RJsVgp92osspxkThvzm7+Wo6fr03D/ACgt7OXTbaQo1/cK0onZTRuzRKyKIqiiyuXMo9pURaM0muWPdGsrzSItY5kTXMd9OgZbOErGYgdo38hVyZKe9GoURnYzMagas1+9g59iSlzWXtINgtmRYSttXMracRWwrOTvDaGap853KUeYh2EcnaTMEimUUi1Wib4yJBFBV0sJUBJ+RXfmh4q1iHhTmxBa6fe3DBIb6DMlsZGNFS5jkZzDmNAJlcxhiM6xpVwxc2e6hLw/psvEPLya4vLWFS0tpLRpwgFS0Doq73KKkxFQ9B7DO1FwMft1dwTI2gnKnn8aWRteIbWok2yji8r3kt5xsmmZJpYoIXHG1jLjBiYDIL8IA5Q42yo8BynTkj3gOQ/D3PHhjsNyY7Xiu1qbO8y1aIk1aKQCjPBJ+ZK1VqSJ7QIb4hyd5t6zyp17tUGe44fuNlza5qLJsosiE7ElQ0o9KFao2wgr17Qa3qA7w+r99MTMspHQzoiUrP2BNNw/qWHMTt3igRUDX2ih0EnDw4LHRYteJJaTklFnLgxQ6twm365rfLXuYck4rbTIlnuKFbeOoSfU75lGeaZgCQuwNLJRlghVIYwSIY2CtL0LmP3tucs0mrO1vGrVuHoWh02zRiFhiUkAttKxJUGeVmmcgGWRWjMYdtTRRi6ltqY0wHQrkBWhW8nZ8jQMbdrbNr7gFd88mZlqudkquoHECTEjRskPgkkQA3bVP8Wd6Tntxbrr65NxFqNj7dY4LKV7W3iHUixRMAwA2ZpTI7fnZjizvhfu1clOF9FXRYtAsL32KPNeRJc3Ep62aSRTlJO3LEI0X8qqMBO7o/agrGHKhKajNMkY/ZUmEOLrJ2MRdO5YlXjnK4F9YVFw8O4kvTzJZUpZBkqosLJI3UJGK2IqRGd3dM74OrcbazDyy5qyxya9OMtjfZVjM7qPlrgKFTfMATDKqrvWG7cGVkLwn70fdQ0vg7SJeY3LKKRNEgOa9sszSCBCfmLcsS+6UkCWNi27U7xSIlYJtPsha45OWWU0cZNmln52ca+msGSsk4FV0mwi0TvbDjbnKGMqs3j2CaklFEHf07ZF2hxAkRqkQR7+nIK0s0HO3hSBY1eVItVjQUUvIQsN7QbAzuVhuD+d2hkpnaV2Ku5Dzxurtzyc4nmMjJG0mmSOasFQFpbOp2kIgM0A/IiypXKsSBkrar3FkmJssLFP5r4SUW14azkUUzqrK6s9QySSSRDKKKqKZetxSJpkKAmOc5hAAAAEREdi+D9hPgHqwC3XzUnxt68EJ7EHcEd9vrXFEwuRZNzAYKz05jsQ5uZSxlWLOpSgSayFGyJJtnAogzcY/sz1VB8osG9tDSMiPAKgEAOe+t+0QVXbIu0fePP66Y6tNuuy3NH2RtsPi8B83qriz62GMGGJssLCNv1UfcR9Q2ipduzGU4Iw9NWhcnajXEe4HgfWx4yK/wAaY3eGSMQToV6GfBPv0D81FVy+jDBwrMjAD5pdvQG4bpOwfefu9OBzWrqrC1ToG1vL1D7/AEYTgfR7+Lcizk2LyOdlSbODNXzZZo5BB62Res1hQcETVBJ2zcJrJG3blEjlMXeUwCLxWvRhhII6cXGGkz/Ctpn/AHfsNfs5rewfN+63xH14PIP2U+EerHQO2vG3Gj8mam9N+FnfQZh1AYUxVICRNQI/I2U6PSX5k1SlOkcjKyTka6OVQhwEogQd4CAh4be1ikf3FY+QE41vNFGaSMqnxkDHv41zhhbM7Vd9h/L2MMrMmpCqOneN79VLw2bEOJSlM4WrEtKJoFMYwAHGIeI7tsMjp74I8opjKSRybY2Vh4iD6sbR284940Rn+zVr2SzawGxQJHoYryQ1M1UmY1Ncjn0hMpigomo5KZNUqngIG3CA/btsjB3i+UY1ykbtto6D6sVdnZpWQbd0jRC4croNm6GdK8qs4crJN0Ek02siY51FljkTIUCh+UfEfD7die9+Vf4cBth85HX9WLWYblTygJjWutgUAEREZ2LAAAPERERdbgAA2FaHwYNcy+EYyFNRNVMiqRyKpKkKomomYp01EzlAxDkOURKchyiAgIDuENsYzj8UtLxUDGvJick4+GiI5AzmQlZZ62jo1i2Ju43Dx88URbNkCb/E5zFKH9O2QCTQdOMEgCp6Mc2sNcGi6VsAVOM1daY5G0GVK3LXmOecWO5o7gxgIDZONQtSjxRwJx3cspBPv/Jts3EwFSjU8hxqFxbk5RImb4h+OOlVZKOQYeaLv2SMZyU3PmKrpBNh06oFFJx1Z1Ab8lUDlEp+LhNvDcPjtqoejrxuqKV6sfiZ2SuyLgrSPnoV86OBjEbM5Ri6cHApTHMJUUFzqGApCiI7g8AAR2zQ4xUHYDgLfftz4+xXoySxhAvTM5/UJb2tMdnROKbktEryRbLcTInKIG4HrlCNjly7hA7WQVKPgO01O4rwBDxbzfbiS+TPYaBZtcCu0dplO5twfGoM0qnqeJT1Yit3u+Nn4X5aJolq+W91m5EJpsO4jG9mI8pEUbeFZGGAK9jjSbH5/wBY7O9W2NTkaHp2iW+S3rR0kVZlIXlV8DDG8c6IYogPSyqbiYIA/Cc8PwG3lMIDODvr8y7jl/ykbRdLkMet8QSmzVgaMtsFzXbqfGhSA9YFxUbRURU7qvBcHGvMZdUv0D6Vo0YuWB2q05bLbKfI4aYdRMNDsNMPUbUlYtVxNlhYr3e6OTA77WPl6w6b40jHHTuwqNJ5Rgo2NW3uSUTrEuc1TkGqZUmlSmpkqhm4FOoiq5Kss3ErZZBJO/zu66XzC0rkxoicyJN5rbW4MYYMJorVgDaxXJY1adYqZqhWUZY5Kyq7NTTze4k4D1zm1rNrwImTT4ptrKQYZ5lqLqS3A2CIS1oASrCssdI2VQSX6f3WRWsbZEtOky6toSJbZllC2bHNuFq3aSTi+xcaKDmjTUqbhO7j5yIbGVhklDlK3kiLIpFOrIlAsZO/byn1TiPh605naTJPMdGiMNzb5maNLaR83aYo+hWSRqXBAq8RR2IW3NZEd0rj/TdD1u64H1COCJ9VdZIZwoWR540yiCWTpZWjH9AE+zIHVatNhv3ap7Fh2PMmoaKscNLV6dj2stBz0Y/hpmKfJFXZScVKNVWMjHvEDgJFmrxoudNQg+BiGEB26rG+vNMvYdS0+R4b+3lSWKRDRkkjYMjqR0MrAMD1EA45r2ytNRs5tPv41lsZ4mjkRhVXjdSrow61ZSQR1g4QiyZCWbQprasEdWl3JZXAeY0JiqLrKnSWlK4wkm1grAPzgG86VhqTtuR0XcYh03ByjxFHx/Q9wtqOld4HkRbXOqKps+ItEMdwAARHM6NDPk8BhuFcxnYQUU7CMUJ8S6fqfIvnZcW+mswutA1kSQEkgvCrrLDm8UsDIHHQQ7DaDh8+o2eLu1UrFyg1RWhbbXoWzw6xgADKxc9GtpWPVMACIAKjR2QR3CIeO356dZ0q70LWLvRL8Zb6zuZYJB4JIXaNx5mU4vl0jU7XWtKtdZsTmsru3jmjPhSVA6HzqwxkOzbhwxT+69znT146zVEznTUJq01CnIoQwkOQ5cv24xTkOUQMU5TBvAQ8QHYvg/YT4B6sAt181J8bes4NN9SNoBd4IzpRtaNHhio4r1axkW6vPl7MjeNq+oRrXWz6zJKFRIVJsTKES2PPIcRjKOJJGXMPCQhA24tOuN4hhb3k6PJ/Lo9GHDVrXdyC4X3H6fi/n0+nDLf07vcPHWnoxYYvv86Mjn3SuhB44uSj5wZWVtuPDNV0cWX5U6xjrvXK8PGKw8ksc6q6sjFHdLCUXiYC2ahb7mbMv7b7R5esYdtKuu0W+Rj/AFU2HxjqP3ebBONf2sak6DNJ2W9TF06V4ekwRmtJrDhcUVLxkmdEYyi09uCZyujJSs6smZ6oiB1GcYi5dCUSIH3c1vC08oiXr6fEOvHZdTrbQNM3UNnjPUMVxfbN0mZH7unccbkyu+lbPXZm3zeoLVXdlTqIKOqp6iTlrDFpu0TJFYSmQrDJowrFNAQOzTeHcJJiizUApHcyraW3sbDSij+PB04FLSB7679vaCczHxfz6Mao7xBSp90DW42SImi2YZ3tMYxbIJJoN2cbFkZx0awaoJFIkg0YMGqaKSZQApEyFKAAAberP5VPhx4v/nJPiOLRPSZ/hW0z/u/Ya/ZzW9hib91viPrwYwfsp8I9WFMe/t33sjY+yNbdDeii5OKVJ0xRWB1AZ0rboE7W2tXCQX+LcbTDc4nrStaA3JnZZAxZIslxsW5motHB3LtYWKsonnFa9A+8/dhk1PUnVzbW5oR7xHTXwDweM4BhpN7HXcm19U1HPFXp8RW6PdTnloTJefbq9rS+QSOBMc9gh2gx1mu05FvB3GSlFWJWbwDcSK6oAYQ7pb62tzuyfaHUB0fdhtg067uV3qiinrY9P34wHU925u5F2j7TT8yW2MsOOG7eZatqdqFwZd3j+tMLIYy7ltCL2qCNFzdalHqTA502ko1ZlfpEOCQLFIqUnqK4trsFBQ+IjHma0u7EiRqjwMD9+HS+wj3eZXuLYqsuJs5uYpLVVhCKjn1hko5u3jW2XMdOXCUUyyS1h24Ebxs/FSqiLGwoNyEZFdOmjlAqRHvStWW/tBbuHT9pvsPg/DBBpl8btCkn7y/aPD+P88LTa2ewX3NrjqP1b55gML1I+MrRmnPGW4aXWzLi5ByvRpm7Wq4sJJSLcWhOTbrrwLkqotlEirJmHlmKAhu2coL+2EaRljmCgdB6aYaLjTLxpnkCjIWY9I6Kk+HABsE4SyJqQy/j/BeJoppOZIydYW1Xp8S+lo6DaP5l0mqqi3Xl5dy0jWBDEQMPMWUIQN27fvENnB3WNC7+6BhsijeaQRptcnZgxQ/TXd3IAEfYWljuD7Aznh7eP5g33EA3jtx/UrT9R9B/DHf9Jvv0D0j8cP6qZZqegPt7U7JOoxYlVidN2mvGcff46PeMpZ0e01ekVuqkpdddJuE4+am5+4FSiY0SqlQdO3CX3hUzCcGDIbi4Kx7SzGnp6fRgmzrbWoeXYEQV8oHR6dmK3HXB3HNafdgze2hptzcZOu2G0+VYX0tYwLOStbieseiSvxbKrxCQusgX1VPgBxLOmyrxwvxcgjVty2qRHBbQ2iVFK02sf42DAnc3dxeyUNaE7FH8bT48dT1D6ajuu2yoNbWvifHtRcvWZXren2/LVSj7eBFCcxJB0yj15WKjXihd29Fy8RUSEeFQCGAQDUdStA1Kk+OmzG9dIvWXNlA8RIrjRl5z13D+35hfUL20tVlQv8di7M1GjoyLxrk+Rdu46iyEHbIGyQGQsIWtstMwr2sjJVwWr1nEu1oR/wAagG5btLmE2LHb3DrcxEZlPSOvxH+K41NLdWsb2k4ORh0Hq21qD+GzG7fpqyFN3Z8GHEB4k6pmnhHeYADjwzfQNvKA8I7wD8oDu216l8o3lHrGNukfOr5D6jg4ff8AckvrhqSpOMxVMZjiivPToIcQimVe9w9JnHCok/qgocrUgb92/cUNraP/AD+4Ti0vlpe8UKv9bVrhQT4rWS5iA8gzH04rd77PFTX3MC04cZv6WmwMQPHcR28hPlIA9GCHfTz44b13TJl7IhkkiyV9zEaDMqUoc08PRarDHjyKH3bxAkna34gH2Bxfn2jn/wCh2vSXfM/SOGwT2ew0YS06t5dTyBiPKkEWPvHcc0lIeXep6+ab681Ux168lvDHl/1zSYYA2r+xNjAe+8BraHTXhUMTUOX6XM2bI2QjWbhmvwP6Zjw3Mj7JbCnSEVmclKiY8bFqfdmBUzhwkcFGW4Zq9yzkOOaPHX948Qw5+B9BlR2DCqXN5seC327GSPZPONoyiKN1yz1EPu+BztPLXgr+09BmycZ63E6KVNHtrTak0+zarybYYDsOYySI2aGmAydqLt31vV85yneszRDxxhiBrs1j+HKkdRqrNZGs0OZIJKLdEHcC+OIp8nIFEwbiyLpiYOMpFibTa75XeMv+UdhpnCnBsyDjO8njupagMIrKCUHK6n/9kqGLZt3Mc49ksjYh93P+Q9tzK1W+4w4ojf8AtWwikt4aVXe3k0ZUlSOkWsTiQg7N7JAfaCuuA5Z6w1k7RrqMteL5948hb9iK5NXletMSK8ed8kxctpylXyuLgcV2yEqxFrINTAbmtzHAh+FVM5Q+58EcXcOc3eX9rxLYok2h6raFZYXo+UsDHcW0o6CUbPE4plYCoqrAlm4q4c1vlzxhPol0zRarp9wDHKtVqFIeGeM9IDLlkXbVSaGjAjD3vbn1kw+trTPUsmmWYt8iwZU6fl+vteBEYm+xLVDrJFuyLuFvCWxoonJsQDjTTScGb8ZlW6u6krvAco7vk3zFuuHArtw/NWexlapz2zk5ULdckDAwydBJUSZQsi1tI5PcxrbmZwXBrdUGsRf0buMbMk6AVYDqSUUkTpADFKlkand+3xLH1PCcnfVp6Vd1rs7AggCQX3D1IsDtUAAOpfxchZKec5t3iYU4+ttSbx/IUA/Jtdl/5/60+pcin06Rq/TtauoVH6UkSG5A87zufPinfvz6Qmn86k1BFp2/R7aVj4XR5revmSFB5sMYdsu0r3DQdpnlnKhlVmmPgrHEcwmMCVJnJimtiCIiI/A1gSAH5gDas3vUaRHoveE4qs4gAj6lv/PdRR3LelpTixTuz6pJrHIjhm7lJLpp+581tLJbr/piGO69o/4+6Yp+9fX+O/Wf+9lqH/a9bti+D9hPgHqwC3XzMnxt6zi0Z1caP6Nrt0N2rTPeitmqd7xpBKVCyLN+etSMiw0Szk6Lc2nAUXAeSWBBEXSaRiHeR53DUxgTXOAi8UzQT71eo+kdYwYzwLc2xhbrGzxHqOK4rt/an8q9oPuNMZnI8RMQKWP7pP4L1P0EnGuvIURWcTh7qg3RQMCcu8rEjGt56HOkcEXrqOb8Kgt1jCYjuIkvLai9Yqp8fV+BwKWsz2N3V6ihow8XX6OkYIT9St3IorVhqMrGmfDtujrNgDTq3bTDyfrMuzmKxkbMFshG7uRsUfIxjlwwlYqj1mRTh2KgDxJPVpXhMZNYg7c+m2xijMrikjfYP5/hjq1e7E8ohjNYk8HQSfw6PThpDsF9u8ug/RTBTN4gxjtQeo4kPlLLfWN+TLVmKWYqGx1jJwByprIDTq/IKOXqCheYjNyj9MTGTIlwtd/cb+ai/trsH3nz+rDzplr2a3BYf1X2n7h5vWThDPvGf5o2uf8AeFu/9pS2fbP5WP4Rgav/AJyT4ziy0p2ST4a7blUy8mmksrivRFA5GSRXDeiurScENLKkiqXeXiTWUjAKIbw3gOw2y57kp4Xp6TguV93aCT9MdfQMVZWmSVw7fNX+K7RrLuj9jhqey80u+oG2OIydsclOQXmy9qtrV0xrTKQn3bq8O0jsFVWyCiqRnwrbtxBECiUOsJEI9ulB/HiwGQmN51Nwf6ZarH7T6cWHsd9RZ2dIiPYxMTqJkIyLjGbaOjY2OwFnBlHx0eyRI2ZsWLNtjZJu0ZtG6RU0kkylImQoFKAAABsPHTrwmpXb5R+OCkarYAUD7Phb8MaG1Zd7vssaqtNWbtPN01CP5WEyvjmzVUib7A+cVSx065j1V6pYWgr47IkhLVe0N2ciyWES8l21TPvDh22RWV7FKsirtB8I/HGqfUdPnhaJm2MP0nzdXUcKK9hfMU5hrur6UnkS8VQj8i22Tw5aGZDmIhMQeSoGSgWzN2UBLzEWVnPHSCZR8OoZJjuHdu2d79A9q9eoV9GGPTZDHepToJp6cWb2oD/4Gzb/ALo8k/8As2Z2GY/3F+IevBhL+23wn1Yq1uzJ/mm6HP8AfxWv7PIbFF78q/w4DdP+dj+LFsLsKYNcKR/VvZnm6vpk0xYMi3qrSMy5lu13SzJIH4BkY/ElcjG8ZGPADxUYnmcipO+AfAXDFI32kDZ20lAZWc9IFPT/AIYY9ckKwpGOhmJPm/xxzR9JRpRpc251F6zLLEspe302YisHYsdu0E1z1I8nAls2SpiPBYpwbS8zES8RHpOkuBZJkd6hxCm7VKO3VpWGWEdB2n7satDgU57g+8Ng8XWfu+3DuezJghwAv6kvBGM8pdrzLuSrdX27q96fpah3fFtoRSQJLwElZciU2hWaNB6KYuT1+xVyxKleMwOCKzls0XMUVGqIl79Ndlugo91qg+gnDZq0aPZs7D2loR6QDhSX6ar/ADZMH/3UzP8Asav2ztqXyjeUesYZNI+dXyH1HBk++HSZeI1pzdseoKJxV2rtXPCrHIIJuArtNqcTIckwhuMCTr4TbvsHa5buG6zZX/I6DSIGBu7G5nEo6131zcSJXyrtGKpu+tpl5Yc45tTmUi1vLeExnqO6t4EenkbYcF+7ClrhJTR9a6i0dJDO07MllUmWG8oOEWdkgq2/hpA6YCJumfi1cpJmHdxHaKAH9XaGf/oRot9Y857PWJkP0++0SERP1FoZZklSv6kzIxHUJFPXiWfcV1myv+Ul1pcTjt9nrE28TrCzRQtG9P0tR1B6yjDqwVvPec8facMU27MGTJUkZWapHqOOSQyYyU9LKFMSIrUE2UOTrZyde8KDdPeBAEwqKGIiRRQkRuXfAHEnM/i+y4L4VhMuq3koWprkijG2SeVgDliiWrudpIGVQzsqmUfHvHPD/LjhS74w4mlEWmWkZNBTPLIdkcMQJGaWVqKg2DbmYqiswRpu9tzT3DdWQyBWgymSM0W9pB1evJOHCkNU4MgCjFQ7dYUjGZVimwDcyztzygHlIOHiwCodUxr+NB0bgXu18nezF9zwvoVk0s8xAEtxKdskhFfanuZmCxpm95o4UIVUAo11vVuNe8NzZ7QE3vEmtXixQRAkxwRDZHGDT2YbeIFpHp7qyTOCxYl4jTfgeo6Z8KUDCtLIB4mlQqTR1JnRIg7sM86Od9YrK/IUx+F5OzLhZwYnEYqJTlSIIJpkAKD+Z/MLWeafHeo8da6aXl/OWWOtVhhUBIYEOz2YolVAaAsQXb2mJN4fLfgPSOWfBOn8FaKK2llAFZ6UaaViWmmcbfalkLORUhQQo9lQMB+76mhc+dcOtNTWO4UXeU8FRDklvaMUBO/tmHSrLSMn8JQEXDzHbxdeURDeX9XryH9c4IE2lP3JudK8FcXty44gmycM63KNwzGiwX9AieRbpQsLdP8AVWD3VznHwfvT8sH4n4aHG+jRZtd0qM74KPals6lm8ptyWlHR/TM3Scgwvd2wNbkjoh1Ex1kmHDxbDmQisajmGFbAsvwQguTmibkyZJcfPmqO9cncpgUh1VmSrtsTcZwByz/7yfJODnRy+k06zVF4vsM09hIaD+pT27dmPRHcqAhqQFkWKRqiOhhlyQ5tS8reNEvbpmPDV5lhvEFTRK+zMFHS8DEsNhLIZEFC9Q/dCTcPZYaJsVelGE3AT0YxmYSZi3SL6MlomTapPY6SjnrY6jd2xfM1yKpKkMYihDAYBEB2okvbK7028l07UIpIL+CRo5I3Uq8ciMVdHU0KsrAqykAggg4t1tLu2v7WO+spEls5o1eN0IZXRwGVlYVBVlIII2EGowo137LZETWrukV2PXTXfUzCVcj50E1CHFnIzFpuE+2YrlKYTpLhDyDZxwmABFNyQweA7XK/+eekXlhyZv8AUrlStvfa9M8VQRmSOC2hZx4RvEdKj8yMOrFSPfy1S1vubllp9uwaey0SFJaEey8k9xKFPgO7dHoepwevB7O1LCO4Ht/acmr0h013les02UhwEB6Sfv1rmY5Qu8AHgXjnySgfmNtXp3vb+HUe8ZxNNAQY0uYIqj9UNpbxOPM6MPNiePdUsZrDkDw5FOCHe3mkof0y3U8iHzoynz4IbtGzEhcU/evr/HfrP/ey1D/tet2xfB+wnwD1YBbr5mT429Zxbs0T/Yem/wB1K7/qhnsJN7x8uDhfdHkwn59SF2gcsZuynQtZGkPEdmyddbui0x7n2iY/hlJewO5KBjeCh5STimZTu3pFoBiaEllg3EblYRhgKIqrqA76beIiGGYgKNoJ+0ff6cMWrWLyOLiBSzHYwH2H7j5sDt7MnY61K3LWvR73rM07ZDxNgzBwtspvmOTqu6gWmTLpByDY1EorJrIFDzSP8+AknLEMkq1Ujo5Rotwi8T39F5fRCArCwLts2dQ6zjl0/TpmuA1whWNdu0dJ6h95/nixA2HsFOKmzvGf5o2uf94W7/2lLYrs/lY/hGAm/wDnJPjOLJVDHkll3tbNsVQqIuJrJWgdrQ4ZAo7jKy9t09pwMYmUd4eJnz9MNhzMEus56BJX7cFmQyWWQdJip6VxVoaT8eYhyNqgwvirUdabNjXEt2yRC0TIVwgDxUdPUptPvBgkJpZayMJCLjWULOOm6kio5bqAgyTXNw8RQ2KJWdYmeMAuBUePAbAkbzKkpIQmhPgw7p/KP6KP+ZHVL/peJv8AhtsyfVp/0p9v44Ivodv+t/s/DE/lH9FH/Mjql/0vE3/DbZfVp/0p9v44X0O3/W/2fhjdOnH6Y/SVpoz5h3UHUc+6jZuz4YyLVckQUNYHONDQcrJ1OWby7OPlgjaEwfjHO1mwEWBFZNQUxECmAfEPEmpyyxmMqtGFOv8AHGyLR4IZVlVnqpB6urzYPrn4pj4JzWQhRMc+JMjlKUobzGManTIFKUA8RERHw24I/wBxfKPXhzl/bb4T6sVZ3Zrct2ndK0NKuVk0Ez5/qLYp1DAUpnD3q2bREBH7VHDpciZA/KYwB+XYovPlX+HAZYbL2P4hi2M2FMG2E9Pq8sbTEphXRxlxo1WVhKXkzJ2P5p0QhzpNn2RaxW5+AKsYoCVIFk8avwAR3AJgAPt3bPGkMA7p1kA+j/HDFrqExxv1Aken/DHlfSKZvqrjFurHTcu/bNrvEX+tZvi4xVUpXk1VbHXY6hzr9gjvE6rasS9Wjk3ZtwAmaXbB48fgtXQ50k/LSn34xoci5Hh/NWvm6P48uHINmfD9gIn1FF1qdS7SOpiNss8xh5C+usUUylsnSnC6stqNlql2jyOKSDeZw9TrdYkX5wDwI1ZLKD4EHbt05SbtSOqpPoOG7VWVbFwTtNAPLUH7sJ2fTVf5smD/AO6mZ/2NX7Z41L5RvKPWMMWkfOr5D6jh1bvB6M7DqhwTDXPG0OpNZVwk9lZ6LgmLcV5a3U2abNUrbXYpFIAVeTaB4tm/Zo/GdbpFW6JDLOCAMqe5Xzv03lPzBn0PiiYQcIa9HHFJKzUjt7mJmNvNITsWI7ySKRtgXeJI7BI2OI6d8Dk5qPM/gSHWeGoTPxVojvKkSislxbyBRPDGBtaQZI5Y12lt28aAvIowqbp61O510lXWQtmGbc9p0y9b+T2WIeMW0lCTrVqsoJGFirssguydLR7gxxRUMQjpqc5+Uonxn4re+ZPKjl/zj0KPRuOLKO9sY23kEiuySxMwFXhmjIZQ4pmUExyALnVsq0ql5e8z+O+U2tyatwbePZ3rru5o2VXjlVSfZmhkBVihrlNA6EtlZatXI9Q2rvUprGn4BPLdzk7iZi7TaVKlQMW3i4BnJyBisyDEVaCbJIvZyQOoCQLqEcPVAMCQH4OEgNnLbkxyu5JadctwbYxWQkQtcXUshkmaNPaO8nlYlYkAzZAUiFM5WtThx5h83eZXOO/t14uvZbwxuFt7aJAkSu/sjdwRABpXrlzENIa5Q1KDDMfaW7dTvTDV1835jiE0c632IKzi4F0Qiq+Lqa8FJypFK+JiI3CwmTTPImAROzQIRoUSGF2ClWHfG7zEPNfVl4C4JmLcv9OmzSTLUC/uVqokHWbaGpEI6JHLTEECErZh3S+7rLyw0tuN+MYQvHV/DlSJtpsbdqExnqFxLQGY9MahYgQTKGNJtBjE0sfNVJJdJRFZNNZFZM6SqSpCqJKpKFEiiaiZwEp0zlEQEBAQEB29KzIwdCQ4NQRsII6CD1EYwyq6lWAKkUIPQR4DhJ3uxdsue0rZEl8yYhrTp7pqvMod8mnEtVHCeH7DIqmO5qUwmiU5mlScujiMK9MAJJkODFUQWSSUdXS91DvJafzT0CHg7i25VOZNlFlJcgG/iQUE8ZPvTquy4jFWJBnUZGdYqp+8lyLvuXesS8U8OQM/Ad3Jm9gE9ilY7YXp7sJP7Eh2AHdMcyqZOdNO3cx1j6ZKF7Z4xyiX0Q2Kp5FA2uvwtub1MzlVddwFXVm2blzFNVXLgyotOM7IFRMcEQMc4m+r8wO7FyZ5m66OJuKNLP1tqb2WCaW3M9AAN+ImUOwUBd5QSZaLnoFp8m4N7xHNfl9o50HhzUR9IWu7jmijnENSSdyZFJQEktkqY81TkqTXEcPY3znr11GtK83kJq7ZHyXYPOLveZgqr5GCiTLIEm7hZHCYJIMYSAYcJUkScog8KLNqTjOgkJtxbxZwD3fuWL6lLHBY8M6Xbbu1tY6IZZKExW0INS0sr1LMcx2vNK2VZHx884c4T44548x00+KSa94h1K43lzcyVYRR1AkuJiKBY4loAoyjYkMQqUTD92PKNA4xoVKxxV0BbVuh1Sv0+CRNwcwkTXIprEMOcKZCEOuZs0KKhgAOI4iP5dvzy8Sa/qHFXEN9xNqzZtT1C8muZTtoZJpGkelakDMxoK7BQYvd4e0Ox4Z0Gy4d0tcunWFrFbxDrCQosa1pTbRRU9ZqcZjsy4eMU/WvkQ+e7WgO8N3zZah/Hf4eGXrfv8fzbF8H7CfAPVgFuvmZPjb1nFu1RP8AYem/3Urv+qGewi3vHy4OF90eTGV7Yx6xNlhYmywsVNneLEB7o2ufcO//APQ14Dw/pB0kAh/1CGxXZ/Kx/CMBN/8AOSfGcWiOksQHStpnEB3gOn3DIgIeICA45re4QHYYm/db4j68GMH7KfCPVhFz6gfsyZDwBmDIWtTTrTZK16bcpTUleMnwtZjnD59gm+TbpaQtT2TjGRFlUMXWWVWUftJBMhGkS4cqMFit0iMjuXzT7xZEEMhpINg8Y/HA5qmntFIbiIViY1PiPX5vV0eDGv8AQr9TZqz0p41ruHsxY9reqak02NZQlQnbJaZSk5SiIJgQG7KGk7q3irSxtbGMZEKk1O9jRflIQCqO1CgUC+p9MhlYuhKMfOPRjzbaxPCgjkAdR0baH07a46Cz19WxqXuVYk4LT9ptxrhCafomboXi2WySzBMw4H4d72GhFq3SKySSS3CBBft5Nr47zIG+zbXHpMQNZGLDwdH442Sa5MwpEgU+Emv4Y6v+mhz33MsoZQzDMZXgr5lnSPlqWsd+tuccqzL5j6bzSLZMDOsWvZVqsa7pWnp0GEvDRxU42KTSQdFWaGRFpIatSjtlRQlBKNlB4PH4PL/A36RLeO7FwWgY1JPh8Xh8Y6vW5TIMGkowexj9AjljItHLB62UDem4aPETt3KCgflIqioYo/mHZm6NuH8iooejFSZrH01Zx7X+uGw0RUs5TbTiLJTPIuB8hJt1E0rFVIizDO4syRWnrhJRpIfDHodQUorFaSbZw0W+9QVKBbDKlzAG6QRQj1jAPPDJZ3BXaGU1B8XUcH2qv1dmoWOpkdGW/SJiSz3ttHJN39uichWyr1+SkE0gIaSGmKQdgdMyuFA4zoJy/CAiIEEhdwA3nSIy1VchfJ9+HNdclC0aNS3hqfV/PDVOoLTtW+6d23mGNspIx1UldQeEMbZJiJiITcSLLG+VZOrwl5rE/DA6OhIPomv2ZwVFZEVEVn8UddsZQnPMYGuOQ2tzmTaFYjyjow9SxC8tMj7C6g+Q9P8AHixWuScRra7QGsVE6pLFgzUJiWUcniJhJt11XulZeGWZHkIlV+1GCyHjK5MSHIPEmogsXiTVIk6RMRIkBgvIepoz9n4EYEiLiwn61lX0H8QcHxrX1depJjTm8datJWF7De0WSaC1rirrdK3XHT0iYEM+VpazSfepkVOHEZJOZIG8RApihuAOA6RHXY7ZfIPX/LDmNcmC0ZFLeGp9X88Ck1OZ37ifeFr+Z9VuXXLVLAGkeqqWB+zh2EpVcI44c2icgICNplKYnNNL2HJtvfS7PjO8dO5EzFHmOHKTVJAm3VFHb2ZWJP3HPnPjPixxTS3d+Gmf9pB5APEPGcbd+mrMUO7Lg0omADGqmaOEoiG827DN+37g+0d2/wAdvGpfKN5R6xjZpHzq+Q+o4sz9hrBdgC/ch/hWes1vfPqPdXqVPVny9e3fuD1/Efi9fdV+I803fb1X4nh4eLw3bWGd2D/t19DX+wMv9oZB2f6x2zseTZ8pl9nd/wD1+xWtNtcQM7yH/Vb603985v7qzHf/AEnsna8235rNtz/H7dKV6sZN20P4YfqFf5deP3X3H8k98/QHuvyOX+N9FdD+N4eV+n6X77l79/3fFs1d6f8A7XfTF/5Mp/Z+ze/Su2fT619ntWf2en3N57OalPaphz7tH/WH6i3/AB1X+69u7+p9l7dSntdmy+10e9k9qla+zXBwtoEYnBibLCxNlhY8ax+nvIJr1b5N6W8rfeovUfQ+QeS9Mp5n515n+rvK+j4+fz/uuXv4/h37dunfUfqEH0jffVN6u53Obe7yoybvJ7efNTLl9qtKbccl/wBh7FN9T3X07dtvd7l3e7oc+8z+zky1zZvZpWuzCpupH+CF7sS2/wB2+Z15ud8uHtj7V83nm5nlvH8HR8e/9H8HD/V8N21r/LT/ALxf2nFT6Rl3ez6x23t1KbM/+by7a9O3FaXML/p7/csub6pXPt+ldk7HWu3JX8vk2eDZg8Wgf5NPaJL5PPRHkn4X1d5P6W9feZ8KnR+5PkH4rzbp9/I6j4OXv5f/AHtoF94D/mn+7z/zL27tvtdn3m/7Jk2Zuxb32d3X3sm3N73ViaHJH/iT+1h/xR2Psns7/d7ntOfbl7Xuvaz093Nsp7vXjunb4Pj7RibLCwvHlb+XS9z8le7HyKe6fuBbvcr1J7U+pPX/AKhfesfPet/Geceoup6vnfe8/j4/i37OKfUcgyZ8lBTp6MNb/Ss5z7rPXb0Vr14YMifLvKozyfp/KfL2XlfScHS+XdMn0PTcv4On6bh4N3hw7t2zcenb04cxSmzox6GyxnE2WFibLCwAbUR/L8+9mW/mK+Sn3z9YzXur639r/WnrT4POvOvNP1l5v1G/m877zncXF47d8f1Ddjd58lNnThsl+mbxt7u95XbWla4OTjz0Z6Ao3tz5T7e+j6z6D8g6fyL0Z5Ky9L+S9J+E8p8k5HTcr7vk8PD4btuFs2Y5vert8uHFMuUZPcps8nVjKXXTdM463kdHyFur6rl9N03LNz+o5v3XI5W/j4vh4d+/w2xj1hNzuffy4vuPIe4HN9d9a59WfIf7IcHnnH+P9T9N+rvPOo4up4fvOdxcz49+zza/Ucvs+7/mrhgvPpOf2ve68lPtxiPbo/lqfcNh5J1/n/VN/JPn59kPTHmnGHQ9L1X6o6vquHl9T91zOHf4bZufqWXb0f5K4xafSc+zp/z5aYc9rfpz0/C+kPJPSvlbH076b6H0/wCS9On5b5L5X+rvK+k4eRyPuuXu4fDdszGtdvTh/FKDLTLj29sYzgbfc2/h3exh/wCIX7P+kN7/ANDev/Q3r7zvko9d7R+rfx/qLpuDndF4crdzvh3bdNr2jef7eubrpWnnxyXnZd3/ALrLl6q0r5sKP4q/llPdBpzPmm4PM/H3V9nPa/dzf/F9N+I8s/6PHg2dn+p5fy+atcMafSM/5/PSmHzMY+hPbbHvtd5N7Z+h6n7denOm9PehPIWHpHyHovwfk3p/p+l5X3XI4eH4d2zE2bMc3vV2+XBKmXIMlMlBTydWOJe5P/D39jHH8Qb2Z9Ebn3o/3K9CesvO+Wj1XtP6v/Hep+RwcfQfFyv0vwbbrbtG8/2+bN4q/bTHPd9l3f8AusuXqrSvmrhPjE38sr7zRvH83fL86/8Atn2Z9md3ON/5l0/4nyX/ALeDds8P9Tyfk81a4Yo/pG8/P56Uw5Faf4dfyMS/XfLZ8gfpyF8w9O+gfYLyL1LCeTb/ACv/ANHb/VvQ7uP7zzDg4/vtmcdo3+zN2ivjrh+bsvZtuTs1PFl/DpxyJoz/AIHvzB1L5Lfk++Yfy+zejvab259c9B6amPVXk/p/9a8r0v1fVcvw6bj4vh37bZu3bs77Pu/HWmNFv9O3o7Pu971UpXx4/9k=)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "OdOfj2oe1U0l"
      },
      "source": [
        "# **Building with Multimodel Llama 3.2**"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "PHhdtzEaeVyC"
      },
      "source": [
        "# Overview\n",
        "\n",
        "Multimodal, Llama Stack and smaller models are the 3 major announcements in [Llama 3.2](https://www.llama.com/). In this workstop, we'll cover multimodal Llama 3.2, Tool calling, Llama Stack and RAG. In this notebook, we will walk through 4 major areas of Llama 3.2:\n",
        "1. Multimodal Llama 3.2\n",
        "2. Tool Calling\n",
        "3. Llama Stack Demo\n",
        "4. RAG using web documents and PDF"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "nri9xAQYe6iw"
      },
      "source": [
        "# Multimodal Llama 3.2\n",
        "\n",
        "This section shows 7 multimodal use cases that can be evolved into full production apps.\n",
        "\n",
        "First we define a helper function llama32 which accepts a list of messages and the model size, default to the 3.2 11B but can also use 90B if passed as 90."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "8TNZwADNcqJa"
      },
      "outputs": [],
      "source": [
        "import requests\n",
        "import json\n",
        "\n",
        "from google.colab import userdata\n",
        "import os\n",
        "os.environ['TOGETHER_API_KEY'] = userdata.get('TOGETHER_API_KEY')\n",
        "\n",
        "def llama32(messages, model_size=11):\n",
        "  model = f\"meta-llama/Llama-3.2-{model_size}B-Vision-Instruct-Turbo\"\n",
        "  url = \"https://api.together.xyz/v1/chat/completions\"\n",
        "  payload = {\n",
        "    \"model\": model,\n",
        "    \"max_tokens\": 4096,\n",
        "    \"temperature\": 0.0,\n",
        "    \"stop\": [\"<|eot_id|>\",\"<|eom_id|>\"],\n",
        "    \"messages\": messages\n",
        "  }\n",
        "\n",
        "  headers = {\n",
        "    \"Accept\": \"application/json\",\n",
        "    \"Content-Type\": \"application/json\",\n",
        "    \"Authorization\": \"Bearer \" + os.environ[\"TOGETHER_API_KEY\"]\n",
        "  }\n",
        "  res = json.loads(requests.request(\"POST\", url, headers=headers, data=json.dumps(payload)).content)\n",
        "\n",
        "  if 'error' in res:\n",
        "    raise Exception(res['error'])\n",
        "\n",
        "  return res['choices'][0]['message']['content']"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "GnQbYwjPFC2h"
      },
      "source": [
        "Next are two helpers that display a local image and remote image."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "pfs6IdayfumM"
      },
      "outputs": [],
      "source": [
        "from PIL import Image\n",
        "import matplotlib.pyplot as plt\n",
        "import requests\n",
        "from io import BytesIO\n",
        "\n",
        "# displays local image given a path\n",
        "def display_local_image(path):\n",
        "  img = Image.open(path)\n",
        "  plt.imshow(img)\n",
        "  plt.axis('off')\n",
        "  plt.show()\n",
        "\n",
        "# displays remote image given a URL\n",
        "def display_image_url(url):\n",
        "    response = requests.get(url)\n",
        "    img = Image.open(BytesIO(response.content))\n",
        "    plt.imshow(img)\n",
        "    plt.axis('off')\n",
        "    plt.show()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "EkkvwNMCFT8-"
      },
      "source": [
        "Finally, another helper llama32pi that accepts a raw prompt and an image url."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "ZyY3H9YqFUXZ"
      },
      "outputs": [],
      "source": [
        "def llama32pi(prompt, image_url, model_size=90):\n",
        "  display_image_url(image_url)\n",
        "  messages = [\n",
        "    {\n",
        "      \"role\": \"user\",\n",
        "      \"content\": [\n",
        "        {\n",
        "          \"type\": \"text\",\n",
        "          \"text\": prompt\n",
        "        },\n",
        "        {\n",
        "          \"type\": \"image_url\",\n",
        "          \"image_url\": {\n",
        "            \"url\": image_url\n",
        "          }\n",
        "        }\n",
        "      ]\n",
        "    },\n",
        "  ]\n",
        "\n",
        "  result = llama32(messages, model_size)\n",
        "  return result\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "SrAI3tUPcpRA"
      },
      "source": [
        "## Image captioning"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "6W1GFMZ_csgO"
      },
      "outputs": [],
      "source": [
        "print(llama32pi(\"describe the image in one sentence\", \"https://images.news18.com/ibnlive/uploads/2023/11/untitled-design-9-2023-11-95acc557353b153d9fd9e074fc345786.png?impolicy=website&width=640&height=480\"))\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "8La2_olFcoZ5"
      },
      "source": [
        "## OCR on nutrition facts"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "ZhBbvs62ctax"
      },
      "outputs": [],
      "source": [
        "print(llama32pi(\"Which drink should one drink? After the answer, also generete nurtrition facts of the two drinks in JSON format for easy comparison.\",\n",
        "                \"https://raw.githubusercontent.com/jeffxtang/llama-stack-apps/refs/heads/main/examples/drinks.png\"))"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Lq9zN_mOhxT1"
      },
      "source": [
        "## Understanding and implementing diagram in Llama 3 paper"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "o4tGUBrmh4Li"
      },
      "outputs": [],
      "source": [
        "print(llama32pi(\"I see this diagram in the Llama 3 paper. Summarize the flow in text and then return a python script that implements the flow.\",\n",
        "                \"https://raw.githubusercontent.com/jeffxtang/llama-stack-apps/refs/heads/main/examples/llama32mm.png\"))"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "v_nInTumiF3_"
      },
      "source": [
        "## Dog breed recognition"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "RpjFxUHViGY0"
      },
      "outputs": [],
      "source": [
        "print(llama32pi(\"What dog breed is this? Tell me in one sentence about the breed.\",\n",
        "                \"https://raw.githubusercontent.com/jeffxtang/llama-stack-apps/refs/heads/main/examples/ww2.png\"))\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "7TiqJPa4ip1K"
      },
      "source": [
        "## Kid's math grader"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "1TkcWPyZ285h"
      },
      "outputs": [],
      "source": [
        "prompt = \"\"\"\n",
        "Check carefully each answer in a kid's math homework.\n",
        "First calculate the correct answer for each problem.\n",
        "Then return the the kid's answers.\n",
        "Finally return a total grade.\"\"\"\n",
        "\n",
        "print(llama32pi(prompt, f\"https://raw.githubusercontent.com/jeffxtang/llama-stack-apps/refs/heads/main/examples/math_hw1.jpg\"))"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "sDxWd_6P1Spp"
      },
      "outputs": [],
      "source": [
        "prompt = \"\"\"\n",
        "Return the kid's answers for all the math problems in the question below.\n",
        "Then calculate the correct answer for each problem.\n",
        "Finally compare kid's answers with the correct answers, and return the grade.\n",
        "\"\"\"\n",
        "\n",
        "print(llama32pi(prompt, f\"https://raw.githubusercontent.com/jeffxtang/llama-stack-apps/refs/heads/main/examples/math_hw1.jpg\"))"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "iHqyTDOEiqHE"
      },
      "outputs": [],
      "source": [
        "prompt = \"\"\"\n",
        "Check carefully each answer in a kid's math homework, first do the calculation, then compare the result with the kid's answer,\n",
        "mark correct or incorrect for each answer, and finally return a total score based on all the problems answered.\"\"\"\n",
        "\n",
        "print(llama32pi(prompt, f\"https://raw.githubusercontent.com/jeffxtang/llama-stack-apps/refs/heads/main/examples/math_hw2.jpg\"))"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "mOXnJfjiiqfN"
      },
      "source": [
        "## Books on the shelf"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "N-R92rhyjho3"
      },
      "outputs": [],
      "source": [
        "prompt = \"\"\"\n",
        "Return all the book titles with author names in the bookshelf, first the top shelf from left to right, then the bottom shelf the same order.\"\"\"\n",
        "\n",
        "print(llama32pi(prompt, f\"https://raw.githubusercontent.com/jeffxtang/llama-stack-apps/refs/heads/main/examples/bookshelf2.png\"))"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Uz5rWEm7YrYn"
      },
      "source": [
        "## Money in the receipts"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "5FMjhXpJYuFQ"
      },
      "outputs": [],
      "source": [
        "prompt = \"What's the total charge of all the recipts below?\"\n",
        "\n",
        "response = llama32pi(prompt, f\"https://raw.githubusercontent.com/jeffxtang/llama-stack-apps/refs/heads/main/examples/receipts.jpg\")\n",
        "print(response)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "eq8Q2hBG13vP"
      },
      "source": [
        "# Tool calling in Llama 3.2\n",
        "\n",
        "Llama 3.2 (and 3.1) supports three built-in tools:\n",
        "\n",
        "1. Brave Web Search: Tool call to perform web searches.\n",
        "2. Wolfram Alpha: Tool call to perform complex mathematical calculations.\n",
        "3. Code Interpreter: Enables the model to output python code."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "UgK0o9vK2SyQ"
      },
      "source": [
        "## The brave_search built-in tool\n",
        "\n",
        "Web search tool is needed when the answer to the user question is beyond the LLM's knowledge cutoff date, e.g. current whether info or recent events. Llama 3.2 has a knowledge cutoff date of December 2023.\n",
        "\n",
        "To see how Llama 3.2 responds to a user question \"What is the current weather in Menlo Park, California?\" with the system prompt above, run the code snippet:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "5t5GFqxA15eV"
      },
      "outputs": [],
      "source": [
        "from datetime import datetime\n",
        "\n",
        "current_date = datetime.now()\n",
        "formatted_date = current_date.strftime(\"%d %B %Y\")\n",
        "\n",
        "messages = [\n",
        "    {\n",
        "      \"role\": \"system\",\n",
        "      \"content\":  f\"\"\"\n",
        "Environment: ipython\n",
        "Tools: brave_search, wolfram_alpha\n",
        "Cutting Knowledge Date: December 2023\n",
        "Today Date: {formatted_date}\n",
        "\"\"\"\n",
        "      },\n",
        "    {\n",
        "      \"role\": \"user\",\n",
        "      \"content\": \"What is the current weather in Menlo Park, California?\"\n",
        "    }\n",
        "  ]\n",
        "\n",
        "response = llama32(messages)\n",
        "print(response)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "zVaic-9i2zmQ"
      },
      "source": [
        "### Calling the search API\n",
        "\n",
        "To ask Llama 3.2 for the final answer to your original question, you'll need to first make the actual search call and then pass the search result back to Llama 3.2. Even though the Llama 3.2 built in search tool name is `brave_search`, you can use any search API; in fact, because you'll need to enter your credit card info at the Brave Search site even to get a trial API key, we'll use Tavily Search, which you can get a free trial API key in seconds using your gmail or github account.\n",
        "\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "PgqftTN-15b2"
      },
      "outputs": [],
      "source": [
        "!pip install -q tavily-python"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "21qwAqIF15ZS"
      },
      "outputs": [],
      "source": [
        "from tavily import TavilyClient\n",
        "\n",
        "TAVILY_API_KEY = userdata.get('TAVILY_API_KEY')\n",
        "tavily_client = TavilyClient(api_key=TAVILY_API_KEY)\n",
        "\n",
        "result = tavily_client.search(\"current weather in Menlo Park, California\")\n",
        "result"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "BDu90NFg15Wx"
      },
      "outputs": [],
      "source": [
        "search_result = result[\"results\"][0][\"content\"]\n",
        "search_result"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "PIOxDEIo3PQT"
      },
      "source": [
        "### Reprompting Llama with search tool response\n",
        "\n",
        "With the tool call result ready, it's time to reprompt Llama 3.2, with all the info below added after the original prompt.\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "0tuctD2N15Jy"
      },
      "outputs": [],
      "source": [
        "messages = [\n",
        "    {\n",
        "      \"role\": \"system\",\n",
        "      \"content\":  f\"\"\"\n",
        "Environment: ipython\n",
        "Tools: brave_search, wolfram_alpha\n",
        "Cutting Knowledge Date: December 2023\n",
        "Today Date: {formatted_date}\n",
        "\"\"\"\n",
        "      },\n",
        "    {\n",
        "      \"role\": \"user\",\n",
        "      \"content\": \"What is the current weather in Menlo Park, California?\"\n",
        "    },\n",
        "    {\n",
        "      \"role\": \"assistant\",\n",
        "      \"content\": response\n",
        "    },\n",
        "    {\n",
        "      \"role\": \"ipython\",\n",
        "      \"content\": search_result\n",
        "    }\n",
        "  ]\n",
        "\n",
        "response = llama32(messages)\n",
        "print(response)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "c0qtbo1v3hNI"
      },
      "source": [
        "## The Wolfram Alpha tool\n",
        "\n",
        "Wolfram Alpha is designed to handle certain types of complex and highly specific mathematical problems that might be challenging for LLMs to answer with complete accuracy. Examples of problems that Wolfram Alpha can answer correctly but an LLM might struggle with include:\n",
        "\n",
        "* Symbolic Computation and Simplification\n",
        "* High-Precision Arithmetic\n",
        "* Complex Integrals and Derivatives\n",
        "* Formal Proofs and Theorems\n",
        "* Advanced Matrix Operations\n",
        "* Multi-step Complex Calculations\n",
        "\n",
        "So if you're building a Llama 3.2 math tutor in one or more of those problem domains, you'll likely need to use the Wolfram Alpha tool. The problem below may not need using the Wolfram Alpha tool by default, but you can hint in the prompt to ask Llama to try using the tool so to a more difficult problem you know how to use the Wolfram tool."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "-NBUck583xVa"
      },
      "outputs": [],
      "source": [
        "math_problem = \"Can you help me solve this equation: x^3 - 2x^2 - x + 2 = 0? try using tool\"\n",
        "messages = [\n",
        "    {\n",
        "      \"role\": \"system\",\n",
        "      \"content\":  f\"\"\"\n",
        "Environment: ipython\n",
        "Tools: brave_search, wolfram_alpha\n",
        "Cutting Knowledge Date: December 2023\n",
        "Today Date: {formatted_date}\n",
        "\"\"\"\n",
        "      },\n",
        "    {\n",
        "      \"role\": \"user\",\n",
        "      \"content\": math_problem\n",
        "    }\n",
        "  ]\n",
        "\n",
        "response = llama32(messages)\n",
        "print(response)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "qmwaoxJ96X99"
      },
      "source": [
        "### Calling the Wolfram Alpha tool"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "jcw70pwV64UV"
      },
      "outputs": [],
      "source": [
        "!pip install -q wolframalpha"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "GyCA5YQu6xgp"
      },
      "outputs": [],
      "source": [
        "WOLFRAM_ALPHA_KEY = userdata.get('WOLFRAM_ALPHA_KEY')\n",
        "\n",
        "from wolframalpha import Client\n",
        "import nest_asyncio\n",
        "\n",
        "nest_asyncio.apply()\n",
        "\n",
        "def wolfram_alpha(query: str) -> str:\n",
        "    client = Client(WOLFRAM_ALPHA_KEY)\n",
        "    result = client.query(query)\n",
        "\n",
        "    results = []\n",
        "    for pod in result.pods:\n",
        "        if pod[\"@title\"] == \"Result\" or pod[\"@title\"] == \"Results\":\n",
        "          for sub in pod.subpods:\n",
        "            results.append(sub.plaintext)\n",
        "\n",
        "    return '\\n'.join(results)\n",
        "\n",
        "tool_result = wolfram_alpha(\"solve x^3 - 2x^2 - x + 2 = 0\")\n",
        "print(tool_result)\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "UcIKfeF77VID"
      },
      "source": [
        "### Reprompting Llama with tool result"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "tWtQInqM7cBM"
      },
      "outputs": [],
      "source": [
        "messages = [\n",
        "    {\n",
        "      \"role\": \"system\",\n",
        "      \"content\":  f\"\"\"\n",
        "Environment: ipython\n",
        "Tools: brave_search, wolfram_alpha\n",
        "Cutting Knowledge Date: December 2023\n",
        "Today Date: {formatted_date}\n",
        "\"\"\"\n",
        "      },\n",
        "    {\n",
        "      \"role\": \"user\",\n",
        "      \"content\": math_problem\n",
        "    },\n",
        "    {\n",
        "      \"role\": \"assistant\",\n",
        "      \"content\": response\n",
        "    },\n",
        "    {\n",
        "      \"role\": \"ipython\",\n",
        "      \"content\": tool_result\n",
        "    }\n",
        "  ]\n",
        "\n",
        "response = llama32(messages)\n",
        "print(response)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "q1vVIG8R7mMd"
      },
      "source": [
        "## The code interpreter built-in tool\n",
        "\n",
        "Just including `Environment: ipython` turns on the code interpreter tool so you don't need to specify the tool on the Tools: line. Let's now ask Llama a million dollar question: \"How much is the monthly payment, total payment, and total interest paid for a 30 year mortgage of $1M at a fixed rate of 6% with a 20% down payment?\" Note that you can replace \"Python\" in \"Generate the code in Python.\" in the System prompt with another language such as \"Java\" to see the Java code to answer the question."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "njJ5aHZG74RC"
      },
      "outputs": [],
      "source": [
        "prompt = \"\"\"How much is the monthly payment, total payment,\n",
        "    and total interest paid for a 30 year mortgage of $1M\n",
        "    at a fixed rate of 6% with a 20% down payment?\n",
        "    \"\"\"\n",
        "\n",
        "messages = [\n",
        "    {\n",
        "      \"role\": \"system\",\n",
        "      \"content\":  f\"\"\"\n",
        "Environment: ipython\n",
        "Tools: brave_search, wolfram_alpha\n",
        "Cutting Knowledge Date: December 2023\n",
        "Today Date: {formatted_date}\n",
        "\n",
        "Generate the code in Python.\n",
        "\"\"\"\n",
        "      },\n",
        "    {\n",
        "      \"role\": \"user\",\n",
        "      \"content\": prompt\n",
        "    },\n",
        "  ]\n",
        "\n",
        "response = llama32(messages)\n",
        "print(response)\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "odmxYrJH9U4o"
      },
      "outputs": [],
      "source": [
        "loan_amount = 1000000  # $1M\n",
        "annual_interest_rate = 0.06  # 6%\n",
        "loan_term_years = 30\n",
        "down_payment = 0.20  # 20%\n",
        "monthly_interest_rate = annual_interest_rate / 12\n",
        "\n",
        "# Calculate the down payment\n",
        "down_payment_amount = loan_amount * down_payment\n",
        "\n",
        "# Calculate the loan amount after down payment\n",
        "loan_amount_after_down_payment = loan_amount - down_payment_amount\n",
        "\n",
        "# Calculate the number of payments\n",
        "number_of_payments = loan_term_years * 12\n",
        "\n",
        "# Calculate the monthly payment\n",
        "monthly_payment = loan_amount_after_down_payment * (monthly_interest_rate * (1 + monthly_interest_rate) ** number_of_payments) / ((1 + monthly_interest_rate) ** number_of_payments - 1)\n",
        "\n",
        "# Calculate the total payment\n",
        "total_payment = monthly_payment * number_of_payments\n",
        "\n",
        "# Calculate the total interest paid\n",
        "total_interest_paid = total_payment - loan_amount_after_down_payment\n",
        "\n",
        "print(f\"Monthly payment: ${monthly_payment:.2f}\")\n",
        "print(f\"Total payment: ${total_payment:.2f}\")\n",
        "print(f\"Total interest paid: ${total_interest_paid:.2f}\")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Q-ySYW0V9y45"
      },
      "source": [
        "### Reprompting Llama with tool result"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "lAlmkZj59YkT"
      },
      "outputs": [],
      "source": [
        "messages = [\n",
        "    {\n",
        "      \"role\": \"system\",\n",
        "      \"content\":  f\"\"\"\n",
        "Environment: ipython\n",
        "Tools: brave_search, wolfram_alpha\n",
        "Cutting Knowledge Date: December 2023\n",
        "Today Date: {formatted_date}\n",
        "\"\"\"\n",
        "      },\n",
        "    {\n",
        "      \"role\": \"user\",\n",
        "      \"content\": prompt\n",
        "    },\n",
        "    {\n",
        "      \"role\": \"assistant\",\n",
        "      \"content\": response\n",
        "    },\n",
        "    {\n",
        "      \"role\": \"ipython\",\n",
        "      \"content\": \"\"\"\n",
        "Monthly payment: $4796.40\n",
        "Total payment: $1726705.51\n",
        "Total interest paid: $926705.51\n",
        "\"\"\"\n",
        "    }\n",
        "  ]\n",
        "\n",
        "response = llama32(messages)\n",
        "print(response)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "CIl9Uwlg964b"
      },
      "source": [
        "## JSON based custom tool calling\n",
        "\n",
        "Llama 3.2's support of custom tools allows you to define your own custom tools, inform Llama of the custom tool descriptions, and expect Llama to return custom tool calls based on the user input.\n",
        "\n",
        "Assume you have a custom function, which returns the top trending songs for one of the three countries (US, France and Spain), defined as follows:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "7E4xH3-1--ER"
      },
      "outputs": [],
      "source": [
        "def trending_songs(country_name, top_number):\n",
        "  top_number = int(top_number)\n",
        "  songs = {\n",
        "        \"US\": [\n",
        "            \"Blinding Lights - The Weeknd\",\n",
        "            \"Levitating - Dua Lipa\",\n",
        "            \"Peaches - Justin Bieber\",\n",
        "            \"Save Your Tears - The Weeknd\",\n",
        "            \"Good 4 U - Olivia Rodrigo\",\n",
        "            \"Montero (Call Me By Your Name) - Lil Nas X\",\n",
        "            \"Kiss Me More - Doja Cat\",\n",
        "            \"Stay - The Kid LAROI, Justin Bieber\",\n",
        "            \"Drivers License - Olivia Rodrigo\",\n",
        "            \"Butter - BTS\"\n",
        "        ],\n",
        "        \"France\": [\n",
        "            \"Dernière danse - Indila\",\n",
        "            \"Je te promets - Johnny Hallyday\",\n",
        "            \"La Vie en rose - Édith Piaf\",\n",
        "            \"Tout oublier - Angèle\",\n",
        "            \"Rien de tout ça - Amel Bent\",\n",
        "            \"J'ai demandé à la lune - Indochine\",\n",
        "            \"Bella - Maître Gims\",\n",
        "            \"À nos souvenirs - Tino Rossi\",\n",
        "            \"Le Sud - Nino Ferrer\",\n",
        "            \"La Nuit je mens - Alain Bashung\"\n",
        "        ],\n",
        "        \"Spain\": [\n",
        "            \"Despacito - Luis Fonsi\",\n",
        "            \"Bailando - Enrique Iglesias\",\n",
        "            \"Con altura - Rosalía, J.Balvin\",\n",
        "            \"Súbeme la Radio - Enrique Iglesias\",\n",
        "            \"Hawái - Maluma\",\n",
        "            \"RITMO (Bad Boys for Life) - Black Eyed Peas, J Balvin\",\n",
        "            \"Dákiti - Bad Bunny, Jhay Cortez\",\n",
        "            \"Vivir mi vida - Marc Anthony\",\n",
        "            \"Una vaina loca - Farruko, Sharlene\",\n",
        "            \"Te boté - Nio García, Casper Mágico, Ozuna\"\n",
        "        ]\n",
        "    }\n",
        "\n",
        "  # Find the list of songs for the given country\n",
        "  if country_name in songs:\n",
        "    return songs[country_name][:top_number]\n",
        "\n",
        "  # If the country is not found, return an empty list\n",
        "  return []\n",
        "\n",
        "# Example usage:\n",
        "country = \"US\"\n",
        "top_num = 5\n",
        "top_songs = trending_songs(country, top_num)\n",
        "print(f\"Top {top_num} trending songs in {country}:\")\n",
        "print(top_songs)\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "sQ5vYK6t_D5g"
      },
      "source": [
        "A mapping of the function name in string and the function name itself is needed:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "CjAB5KKs_Hz1"
      },
      "outputs": [],
      "source": [
        "custom_tools = {\"trending_songs\": trending_songs}"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "qlmQ0QZU_O_2"
      },
      "source": [
        "### Querying Llama for a custom tool call result"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "jrbuIp_O_PKA"
      },
      "outputs": [],
      "source": [
        "messages = [\n",
        "    {\n",
        "      \"role\": \"system\",\n",
        "      \"content\":  \"\"\"\n",
        "\n",
        "Environment: ipython\n",
        "Cutting Knowledge Date: December 2023\n",
        "Today Date: 30 August 2024\n",
        "\"\"\"\n",
        "      },\n",
        "    {\n",
        "      \"role\": \"user\",\n",
        "      \"content\": \"\"\"\n",
        "Answer the user's question by making use of the following functions if needed.\n",
        "If none of the function can be used, please say so.\n",
        "Here is a list of functions in JSON format:\n",
        "{\n",
        "    \"type\": \"function\",\n",
        "    \"function\": {\n",
        "        \"name\": \"get_boiling_point\",\n",
        "        \"description\": \"Get the boiling point of a liquid\",\n",
        "        \"parameters\": {\n",
        "            \"type\": \"object\",\n",
        "            \"properties\": [\n",
        "                {\n",
        "                    \"liquid_name\": {\n",
        "                        \"type\": \"object\",\n",
        "                        \"description\": \"name of the liquid\"\n",
        "                    }\n",
        "                },\n",
        "                {\n",
        "                    \"celsius\": {\n",
        "                        \"type\": \"object\",\n",
        "                        \"description\": \"whether to use celsius\"\n",
        "                    }\n",
        "                }\n",
        "            ],\n",
        "            \"required\": [\"liquid_name\"]\n",
        "        }\n",
        "    }\n",
        "}\n",
        "{\n",
        "    \"type\": \"function\",\n",
        "    \"function\": {\n",
        "        \"name\": \"trending_songs\",\n",
        "        \"description\": \"Returns the trending songs on a Music site\",\n",
        "        \"parameters\": {\n",
        "            \"type\": \"object\",\n",
        "            \"properties\": [\n",
        "                {\n",
        "                    \"country\": {\n",
        "                        \"type\": \"object\",\n",
        "                        \"description\": \"country to return trending songs for\"\n",
        "                    }\n",
        "                },\n",
        "                {\n",
        "                    \"n\": {\n",
        "                        \"type\": \"object\",\n",
        "                        \"description\": \"The number of songs to return\"\n",
        "                    }\n",
        "                }\n",
        "            ],\n",
        "            \"required\": [\"country\"]\n",
        "        }\n",
        "    }\n",
        "}\n",
        "\n",
        "Question: Can you check the top 5 trending songs in US?\n",
        "\"\"\"\n",
        "    },\n",
        "  ]\n",
        "result = llama32(messages)\n",
        "print(result)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Wz5x1QyU_9Ig"
      },
      "source": [
        "### Calling the custom tool\n",
        "\n",
        "Let's convert the string output to a JSON object and get the function name and parameter list."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "BzQwpP4O_9lk"
      },
      "outputs": [],
      "source": [
        "res = json.loads(result.split(\"<|python_tag|>\")[-1])\n",
        "function_name = res['name']\n",
        "parameters = list(res['parameters'].values())\n",
        "function_name, parameters"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "IFVw3Pgr_rdH"
      },
      "outputs": [],
      "source": [
        "tool_result = custom_tools[function_name](*parameters)\n",
        "tool_result"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "nbnnGkboAoDx"
      },
      "source": [
        "### Reprompting Llama with custom tool call result\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "7mcHyzAFAsi_"
      },
      "outputs": [],
      "source": [
        "messages = [\n",
        "    {\n",
        "      \"role\": \"system\",\n",
        "      \"content\":  \"\"\"\n",
        "\n",
        "Environment: ipython\n",
        "Cutting Knowledge Date: December 2023\n",
        "Today Date: 30 August 2024\n",
        "\"\"\"\n",
        "      },\n",
        "    {\n",
        "      \"role\": \"user\",\n",
        "      \"content\": \"\"\"\n",
        "Answer the user's question by making use of the following functions if needed.\n",
        "If none of the function can be used, please say so.\n",
        "Here is a list of functions in JSON format:\n",
        "{\n",
        "    \"type\": \"function\",\n",
        "    \"function\": {\n",
        "        \"name\": \"get_boiling_point\",\n",
        "        \"description\": \"Get the boiling point of a liquid\",\n",
        "        \"parameters\": {\n",
        "            \"type\": \"object\",\n",
        "            \"properties\": [\n",
        "                {\n",
        "                    \"liquid_name\": {\n",
        "                        \"type\": \"object\",\n",
        "                        \"description\": \"name of the liquid\"\n",
        "                    }\n",
        "                },\n",
        "                {\n",
        "                    \"celsius\": {\n",
        "                        \"type\": \"object\",\n",
        "                        \"description\": \"whether to use celsius\"\n",
        "                    }\n",
        "                }\n",
        "            ],\n",
        "            \"required\": [\"liquid_name\"]\n",
        "        }\n",
        "    }\n",
        "}\n",
        "{\n",
        "    \"type\": \"function\",\n",
        "    \"function\": {\n",
        "        \"name\": \"trending_songs\",\n",
        "        \"description\": \"Returns the trending songs on a Music site\",\n",
        "        \"parameters\": {\n",
        "            \"type\": \"object\",\n",
        "            \"properties\": [\n",
        "                {\n",
        "                    \"country\": {\n",
        "                        \"type\": \"object\",\n",
        "                        \"description\": \"country to return trending songs for\"\n",
        "                    }\n",
        "                },\n",
        "                {\n",
        "                    \"n\": {\n",
        "                        \"type\": \"object\",\n",
        "                        \"description\": \"The number of songs to return\"\n",
        "                    }\n",
        "                }\n",
        "            ],\n",
        "            \"required\": [\"country\"]\n",
        "        }\n",
        "    }\n",
        "}\n",
        "\n",
        "Question: Can you check the top 5 trending songs in US?\n",
        "\"\"\"\n",
        "    },\n",
        "    {\n",
        "      \"role\": \"assistant\",\n",
        "      \"content\": result\n",
        "    },\n",
        "    {\n",
        "      \"role\": \"ipython\",\n",
        "      \"content\": ','.join(tool_result)\n",
        "    }\n",
        "\n",
        "  ]\n",
        "response = llama32(messages)\n",
        "print(response)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "zdW_VBZBB3Xc"
      },
      "source": [
        "## Tool calling with image\n",
        "\n",
        "Llama 3.2 vision models don't support combining tool calling with image reasoning, meaning the models only provide a generic non-tool-calling-specified answer. So, you would have to first prompt the model to reason about the image and then prompt it separately to make the tool call.\n",
        "\n",
        "So if the text prompt is \"What is the current weather here?\" with an image of San Francisco Golden Gate Bridge, you need to first prompt the model to reason about the image and then prompt it again for tool calling response.\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "LBZTmGOVCIPS"
      },
      "outputs": [],
      "source": [
        "imgurl = \"https://raw.githubusercontent.com/jeffxtang/llama-stack-apps/refs/heads/main/examples/golden_gate.png\"\n",
        "\n",
        "result = llama32pi(\"What's the weather like here?\", imgurl)\n",
        "print(result)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "9mozp0TSCYLd"
      },
      "outputs": [],
      "source": [
        "messages = [\n",
        "    {\n",
        "      \"role\": \"system\",\n",
        "      \"content\":  f\"\"\"\n",
        "Environment: ipython\n",
        "Tools: brave_search, wolfram_alpha\n",
        "Cutting Knowledge Date: December 2023\n",
        "Today Date: {formatted_date}\n",
        "\"\"\"\n",
        "      },\n",
        "    {\n",
        "      \"role\": \"user\",\n",
        "      \"content\": f\"What is the current weather in the location mentioned in the text below: {result}\"\n",
        "    }\n",
        "  ]\n",
        "\n",
        "print(llama32(messages))"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "PicsOAl4C2iF"
      },
      "source": [
        "Now we can prompt Llama 3.2 for possible tool calling info, as shown in the brave_search built-in tool section above."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ngYc2TrKYrfx"
      },
      "source": [
        "# Llama Stack Demo\n",
        "\n",
        "The quickest way to jump start with Llama Stack."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "LZZY4rIxp7BC"
      },
      "outputs": [],
      "source": [
        "!pip install -q llama-stack-client==0.0.35"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ZVyXGDsbwucX"
      },
      "source": [
        "## Llama Stack Inference"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "Ik-HuGeodkAE"
      },
      "outputs": [],
      "source": [
        "LLAMA_STACK_API_TOGETHER_URL=\"https://llama-stack.together.ai\"\n",
        "LLAMA31_8B_INSTRUCT = \"Llama3.1-8B-Instruct\"\n",
        "\n",
        "from llama_stack_client import LlamaStackClient\n",
        "from llama_stack_client.lib.inference.event_logger import EventLogger\n",
        "from llama_stack_client.types import UserMessage\n",
        "\n",
        "async def run_main():\n",
        "    client = LlamaStackClient(\n",
        "        base_url=LLAMA_STACK_API_TOGETHER_URL,\n",
        "    )\n",
        "\n",
        "    iterator = client.inference.chat_completion(\n",
        "        messages=[\n",
        "            UserMessage(\n",
        "                content=\"Who wrote the book Godfather?\",\n",
        "                role=\"user\",\n",
        "            ),\n",
        "\n",
        "            UserMessage(\n",
        "                content=\"Best 5 quotes in the book.\",\n",
        "                role=\"user\",\n",
        "            ),\n",
        "        ],\n",
        "        model=LLAMA31_8B_INSTRUCT,\n",
        "        stream=True\n",
        "    )\n",
        "\n",
        "    async for log in EventLogger().log(iterator):\n",
        "        log.print()\n",
        "\n",
        "await run_main()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "JSOezWytd0pA"
      },
      "source": [
        "## Llama Stack Agent\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "NGbay0VcWFe4"
      },
      "outputs": [],
      "source": [
        "import asyncio\n",
        "from typing import List, Optional, Dict\n",
        "\n",
        "from llama_stack_client import LlamaStackClient\n",
        "from llama_stack_client.lib.agents.event_logger import EventLogger\n",
        "\n",
        "from llama_stack_client.types import SamplingParams, UserMessage\n",
        "from llama_stack_client.types.agent_create_params import AgentConfig\n",
        "\n",
        "class Agent:\n",
        "    def __init__(self):\n",
        "        self.client = LlamaStackClient(\n",
        "            base_url=LLAMA_STACK_API_TOGETHER_URL,\n",
        "        )\n",
        "\n",
        "    def create_agent(self, agent_config: AgentConfig):\n",
        "        agent = self.client.agents.create(\n",
        "            agent_config=agent_config,\n",
        "        )\n",
        "        self.agent_id = agent.agent_id\n",
        "        session = self.client.agents.sessions.create(\n",
        "            agent_id=agent.agent_id,\n",
        "            session_name=\"example_session\",\n",
        "        )\n",
        "        self.session_id = session.session_id\n",
        "\n",
        "    async def execute_turn(self, content: str):\n",
        "        response = self.client.agents.turns.create(\n",
        "            agent_id=self.agent_id,\n",
        "            session_id=self.session_id,\n",
        "            messages=[\n",
        "                UserMessage(content=content, role=\"user\"),\n",
        "            ],\n",
        "            stream=True,\n",
        "        )\n",
        "\n",
        "        for chunk in response:\n",
        "            if chunk.event.payload.event_type != \"turn_complete\":\n",
        "                yield chunk\n",
        "\n",
        "async def run_main():\n",
        "    agent_config = AgentConfig(\n",
        "        model=LLAMA31_8B_INSTRUCT,\n",
        "        instructions=\"\",\n",
        "        enable_session_persistence=False,\n",
        "    )\n",
        "\n",
        "    agent = Agent()\n",
        "    agent.create_agent(agent_config)\n",
        "\n",
        "    prompts = [\n",
        "        \"Who wrote the book Godfather?\",\n",
        "        \"Best 5 quotes in the book.\",\n",
        "    ]\n",
        "\n",
        "    for prompt in prompts:\n",
        "        print(f\"User> {prompt}\")\n",
        "        response = agent.execute_turn(content=prompt)\n",
        "        async for log in EventLogger().log(response):\n",
        "            if log is not None:\n",
        "                log.print()\n",
        "\n",
        "await run_main()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "IPVNKOxYwvKH"
      },
      "source": [
        "## Llama Stack Multimodel 3.2"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "wxNRgPhg8o9X"
      },
      "outputs": [],
      "source": [
        "import base64\n",
        "\n",
        "def encode_image(image_path):\n",
        "  with open(image_path, \"rb\") as img:\n",
        "    return base64.b64encode(img.read()).decode('utf-8')"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "kbbHZhXH67GD"
      },
      "outputs": [],
      "source": [
        "from llama_stack_client import LlamaStackClient\n",
        "from llama_stack_client.types import agent_create_params\n",
        "\n",
        "LLAMA32_11B_INSTRUCT = \"Llama3.2-11B-Vision-Instruct\"\n",
        "\n",
        "\n",
        "class Agent:\n",
        "    def __init__(self):\n",
        "        self.client = LlamaStackClient(\n",
        "            base_url=LLAMA_STACK_API_TOGETHER_URL,\n",
        "        )\n",
        "\n",
        "    def create_agent(self, agent_config: AgentConfig):\n",
        "        agent = self.client.agents.create(\n",
        "            agent_config=agent_config,\n",
        "        )\n",
        "        self.agent_id = agent.agent_id\n",
        "        session = self.client.agents.sessions.create(\n",
        "            agent_id=agent.agent_id,\n",
        "            session_name=\"example_session\",\n",
        "        )\n",
        "        self.session_id = session.session_id\n",
        "\n",
        "    async def execute_turn(self, prompt: str, image_path: str):\n",
        "        base64_image = encode_image(image_path)\n",
        "\n",
        "        messages = [{\n",
        "            \"role\": \"user\",\n",
        "            \"content\": [\n",
        "              {\n",
        "                \"image\": {\n",
        "                  \"uri\": f\"data:image/jpeg;base64,{base64_image}\"\n",
        "                }\n",
        "              },\n",
        "              prompt,\n",
        "            ]\n",
        "        }]\n",
        "\n",
        "        response = self.client.agents.turns.create(\n",
        "            agent_id=self.agent_id,\n",
        "            session_id=self.session_id,\n",
        "            messages = messages,\n",
        "            stream=True,\n",
        "        )\n",
        "\n",
        "        for chunk in response:\n",
        "            if chunk.event.payload.event_type != \"turn_complete\":\n",
        "                yield chunk\n",
        "\n",
        "async def run_main(image_path, prompt):\n",
        "    agent_config = AgentConfig(\n",
        "        model=LLAMA32_11B_INSTRUCT,\n",
        "        instructions=\"You are a helpful assistant\",\n",
        "        enable_session_persistence=False,\n",
        "    )\n",
        "\n",
        "    agent = Agent()\n",
        "    agent.create_agent(agent_config)\n",
        "\n",
        "    print(f\"User> {prompt}\")\n",
        "    response = agent.execute_turn(prompt=prompt, image_path=image_path)\n",
        "    async for log in EventLogger().log(response):\n",
        "        if log is not None:\n",
        "            log.print()"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "lxJO7kIKHxS7"
      },
      "outputs": [],
      "source": [
        "!wget https://raw.githubusercontent.com/meta-llama/llama-models/refs/heads/main/Llama_Repo.jpeg"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "oeW6VLTd9VhK"
      },
      "outputs": [],
      "source": [
        "display_local_image(\"/content/Llama_Repo.jpeg\")"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "Ekz2w6eixSnO"
      },
      "outputs": [],
      "source": [
        "await run_main(\"/content/Llama_Repo.jpeg\",\n",
        "         \"How many difference colors those llamas are? What're those colors?\")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "fcmY-TudGNLA"
      },
      "source": [
        "# RAG in Llama 3.2\n",
        "\n",
        "Llama 3.2 has the knowledge cutoff date as December 2023. To ask Llama 3.2 to answer questions about internal data or data generated after December 2023, one can integrate RAG (Retrieval Augmented Generation) with Llama 3.2."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "5n0tn6dDbwRw"
      },
      "source": [
        "## Installing LangChain, FAISS and Groq packages"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "UKoWiWWuIdG2"
      },
      "outputs": [],
      "source": [
        "!pip install -q langchain langchain-community sentence-transformers faiss-gpu"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "BS7wc2pSLTGm"
      },
      "source": [
        "We'll use LangChain's Groq integration to create a Llama 3.1 8B instance."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "TbsZpAWmihUY"
      },
      "outputs": [],
      "source": [
        "!pip install -qU langchain-groq"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "zk2PNNovjmvH"
      },
      "outputs": [],
      "source": [
        "from langchain_groq import ChatGroq\n",
        "import os\n",
        "from google.colab import userdata\n",
        "\n",
        "os.environ['GROQ_API_KEY'] = userdata.get('GROQ_API_KEY')\n",
        "llm = ChatGroq(model=\"llama-3.1-8b-instant\")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "oauu4Vowb7Uk"
      },
      "source": [
        "## Load and process web documents"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "x4fTHDMFLMdf"
      },
      "source": [
        "Next we'll load a web document (Huggingface's blog on Llama 3.2) and create a vector store storing the info in it. This may take about 1 min on a CPU and 26s on a T4 GPU."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "fagOEhDVEz4w"
      },
      "outputs": [],
      "source": [
        "from langchain_community.embeddings import HuggingFaceEmbeddings\n",
        "from langchain_community.vectorstores import FAISS\n",
        "from langchain.text_splitter import RecursiveCharacterTextSplitter\n",
        "from langchain_community.document_loaders import WebBaseLoader\n",
        "import bs4\n",
        "\n",
        "# load the document from some web urls - the Huggingface blog on Llama 3.2 and the Sequoiacap's Generative AI Act III\n",
        "loader = WebBaseLoader([\"https://huggingface.co/blog/llama32\", \"https://www.sequoiacap.com/article/generative-ais-act-o1/\"])\n",
        "\n",
        "documents = loader.load()\n",
        "\n",
        "# split the document into chunks with a specified chunk size\n",
        "text_splitter = RecursiveCharacterTextSplitter(chunk_size=500, chunk_overlap=50)\n",
        "all_splits = text_splitter.split_documents(documents)\n",
        "\n",
        "# store the splits into a vector store with a specific embedding model\n",
        "vectorstore = FAISS.from_documents(all_splits, HuggingFaceEmbeddings(model_name=\"sentence-transformers/all-mpnet-base-v2\"))"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "XREhXv6ZcMo8"
      },
      "source": [
        "## Create RAG chain with appropriate prompt"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "y4Vg77DiWJ42"
      },
      "outputs": [],
      "source": [
        "from langchain.chains import create_retrieval_chain\n",
        "from langchain.chains.combine_documents import create_stuff_documents_chain\n",
        "from langchain_core.prompts import ChatPromptTemplate\n",
        "\n",
        "system_prompt = (\n",
        "    \"You are an assistant for question-answering tasks. \"\n",
        "    \"Use the following pieces of retrieved context to answer \"\n",
        "    \"the question. If you don't know the answer, say that you \"\n",
        "    \"don't know. Use three sentences maximum and keep the \"\n",
        "    \"answer concise.\"\n",
        "    \"\\n\\n\"\n",
        "    \"{context}\"\n",
        ")\n",
        "\n",
        "prompt = ChatPromptTemplate.from_messages(\n",
        "    [\n",
        "        (\"system\", system_prompt),\n",
        "        (\"human\", \"{input}\"),\n",
        "    ]\n",
        ")\n",
        "\n",
        "question_answer_chain = create_stuff_documents_chain(llm, prompt)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "Ha7ZOw4qXjDd"
      },
      "outputs": [],
      "source": [
        "retriever = vectorstore.as_retriever()\n",
        "rag_chain = create_retrieval_chain(retriever, question_answer_chain)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "DfwZhozzcUxy"
      },
      "source": [
        "## Ask questions about the documents"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "TEDT8Wf-WjF0"
      },
      "outputs": [],
      "source": [
        "response = rag_chain.invoke({\"input\": \"what can llama 3.2 be used to develop for?\"})\n",
        "response[\"answer\"]"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "5e13pixbWi9q"
      },
      "outputs": [],
      "source": [
        "response = rag_chain.invoke({\"input\": \"tell me about multimodal capabilities?\"})\n",
        "response[\"answer\"]"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "yi-8Kdo4Wi6m"
      },
      "outputs": [],
      "source": [
        "response = rag_chain.invoke({\"input\": \"What should the developer universe worry about?\"})\n",
        "response[\"answer\"]"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "QXyJgcldcagp"
      },
      "source": [
        "## Ask follow up question"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "dRrlgbduZOWJ"
      },
      "source": [
        "To ask follow up questions, we'll use LangChain's create_history_aware_retriever and different system prompts."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "CqHeeciOZNg7"
      },
      "outputs": [],
      "source": [
        "from langchain.chains import create_history_aware_retriever\n",
        "from langchain_core.prompts import ChatPromptTemplate, MessagesPlaceholder\n",
        "\n",
        "contextualize_q_system_prompt = \"\"\"Given a chat history and the latest user question \\\n",
        "which might reference context in the chat history, formulate a standalone question \\\n",
        "which can be understood without the chat history. Do NOT answer the question, \\\n",
        "just reformulate it if needed and otherwise return it as is.\"\"\"\n",
        "contextualize_q_prompt = ChatPromptTemplate.from_messages(\n",
        "    [\n",
        "        (\"system\", contextualize_q_system_prompt),\n",
        "        MessagesPlaceholder(\"chat_history\"),\n",
        "        (\"human\", \"{input}\"),\n",
        "    ]\n",
        ")\n",
        "history_aware_retriever = create_history_aware_retriever(\n",
        "    llm, retriever, contextualize_q_prompt\n",
        ")"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "6VmxbhVTZNjh"
      },
      "outputs": [],
      "source": [
        "from langchain.chains import create_retrieval_chain\n",
        "from langchain.chains.combine_documents import create_stuff_documents_chain\n",
        "\n",
        "qa_system_prompt = \"\"\"You are an assistant for question-answering tasks. \\\n",
        "Use the following pieces of retrieved context to answer the question. \\\n",
        "If you don't know the answer, just say that you don't know. \\\n",
        "Use three sentences maximum and keep the answer concise.\\\n",
        "\n",
        "{context}\"\"\"\n",
        "qa_prompt = ChatPromptTemplate.from_messages(\n",
        "    [\n",
        "        (\"system\", qa_system_prompt),\n",
        "        MessagesPlaceholder(\"chat_history\"),\n",
        "        (\"human\", \"{input}\"),\n",
        "    ]\n",
        ")\n",
        "\n",
        "\n",
        "question_answer_chain = create_stuff_documents_chain(llm, qa_prompt)\n",
        "\n",
        "rag_chain = create_retrieval_chain(history_aware_retriever, question_answer_chain)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "3QhPoa1yc1qT"
      },
      "source": [
        "After a history aware retriever RAG chain is created, we'll populate the chat history and pass it along with a follow up question to Llama."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "mH2FLpP5ZNl_"
      },
      "outputs": [],
      "source": [
        "from langchain_core.messages import HumanMessage\n",
        "\n",
        "chat_history = []\n",
        "chat_history.extend([HumanMessage(content=\"What should the developer universe worry about?\"), response[\"answer\"]])\n",
        "\n",
        "second_question = \"Tell me more.\"\n",
        "response = rag_chain.invoke({\"input\": second_question, \"chat_history\": chat_history})\n",
        "\n",
        "response[\"answer\"]"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "lNorpAFadKkZ"
      },
      "source": [
        "## Load and process PDF"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "b-4eOx5CXsL8"
      },
      "source": [
        "Now let's see how to load a PDF file and ask question about it."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "29ruMXGQWJ7y"
      },
      "outputs": [],
      "source": [
        "!pip install -q pypdf"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "UHAkrNC-Wb0r"
      },
      "outputs": [],
      "source": [
        "# The Llama 3 Herd of Models paper\n",
        "!wget https://arxiv.org/pdf/2407.21783"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "TNbGQcQtNvbl"
      },
      "outputs": [],
      "source": [
        "# loading a 92-page PDF (the Llama 3 paper) and generating a vector store using FAISS CPU may take 10 minutes but only 20s or so on T4 GPU\n",
        "\n",
        "from langchain_community.document_loaders import PyPDFLoader\n",
        "\n",
        "loader = PyPDFLoader(\"/content/2407.21783\")\n",
        "docs = loader.load()\n",
        "print(docs[0], len(docs))\n",
        "text_splitter = RecursiveCharacterTextSplitter(chunk_size=500, chunk_overlap=50)\n",
        "all_pdf_splits = text_splitter.split_documents(docs)\n",
        "vectorstore_pdf = FAISS.from_documents(all_pdf_splits, HuggingFaceEmbeddings(model_name=\"sentence-transformers/all-mpnet-base-v2\"))"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "WVsUFCdydRKK"
      },
      "source": [
        "The same prompt and question answer chain are used for PDF as for web documents, but we create a new RAG chain using the new vector store for the PDF document."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "AIS0NgpuPzWB"
      },
      "outputs": [],
      "source": [
        "from langchain.chains import create_retrieval_chain\n",
        "from langchain.chains.combine_documents import create_stuff_documents_chain\n",
        "from langchain_core.prompts import ChatPromptTemplate\n",
        "\n",
        "system_prompt = (\n",
        "    \"You are an assistant for question-answering tasks. \"\n",
        "    \"Use the following pieces of retrieved context to answer \"\n",
        "    \"the question. If you don't know the answer, say that you \"\n",
        "    \"don't know. Use three sentences maximum and keep the \"\n",
        "    \"answer concise.\"\n",
        "    \"\\n\\n\"\n",
        "    \"{context}\"\n",
        ")\n",
        "\n",
        "prompt = ChatPromptTemplate.from_messages(\n",
        "    [\n",
        "        (\"system\", system_prompt),\n",
        "        (\"human\", \"{input}\"),\n",
        "    ]\n",
        ")\n",
        "\n",
        "question_answer_chain = create_stuff_documents_chain(llm, prompt)\n",
        "\n",
        "retriever = vectorstore_pdf.as_retriever()\n",
        "rag_chain = create_retrieval_chain(retriever, question_answer_chain)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "YVYeIxhBdfPD"
      },
      "source": [
        "## Ask questions about the PDF doc"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "mtklcJaTXmMt"
      },
      "outputs": [],
      "source": [
        "response = rag_chain.invoke({\"input\": \"how long is the context window in Llama 3?\"})\n",
        "response[\"answer\"]"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "4gdCKX3PPzYy"
      },
      "outputs": [],
      "source": [
        "response = rag_chain.invoke({\"input\": \"What's new in Llama 3?\"})\n",
        "response[\"answer\"]"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "c4FkqGV33SVJ"
      },
      "source": [
        "# Resources"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "gSz5dTMxp7xo"
      },
      "source": [
        "* [Meta Llama Site](https://www.llama.com)\n",
        "* [Getting Started with Meta Llama](https://www.llama.com/docs/get-started)\n",
        "* [Llama Recipes](https://github.com/meta-llama/llama-recipes)\n",
        "* [Llama Models](https://github.com/meta-llama/llama-models)\n",
        "* [Llama Stack](https://github.com/meta-llama/llama-stack)\n",
        "* [Responsible Use Guide](https://www.llama.com/responsible-use-guide/)\n",
        "* [Acceptable Use Policy](https://ai.meta.com/llama/use-policy/)\n",
        "\n"
      ]
    }
  ],
  "metadata": {
    "colab": {
      "collapsed_sections": [
        "XREhXv6ZcMo8"
      ],
      "provenance": [],
      "toc_visible": true,
      "gpuType": "T4",
      "include_colab_link": true
    },
    "kernelspec": {
      "display_name": "Python 3",
      "name": "python3"
    },
    "language_info": {
      "name": "python"
    },
    "accelerator": "GPU"
  },
  "nbformat": 4,
  "nbformat_minor": 0
}