Reddit/Reddit_Get_Hot_Posts_From_Subreddit.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "configured-belgium",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2021-08-16T10:37:31.478245Z",
     "iopub.status.busy": "2021-08-16T10:37:31.477978Z",
     "iopub.status.idle": "2021-08-16T10:37:31.481364Z",
     "shell.execute_reply": "2021-08-16T10:37:31.480745Z",
     "shell.execute_reply.started": "2021-08-16T10:37:31.478180Z"
    },
    "papermill": {},
    "tags": []
   },
   "source": [
    "# Reddit - Get Hot Posts From Subreddit\n",
    "<a href=\"https://app.naas.ai/user-redirect/naas/downloader?url=https://raw.githubusercontent.com/jupyter-naas/awesome-notebooks/master/Reddit/Reddit_Get_Hot_Posts_From_Subreddit.ipynb\" target=\"_parent\"><img src=\"https://img.shields.io/badge/-Open%20in%20Naas-success?labelColor=000000&logo=data:image/svg+xml;base64,PD94bWwgdmVyc2lvbj0iMS4wIiBlbmNvZGluZz0iVVRGLTgiPz4KPHN2ZyB3aWR0aD0iMTAyNHB4IiBoZWlnaHQ9IjEwMjRweCIgdmlld0JveD0iMCAwIDEwMjQgMTAyNCIgeG1sbnM9Imh0dHA6Ly93d3cudzMub3JnLzIwMDAvc3ZnIiB4bWxuczp4bGluaz0iaHR0cDovL3d3dy53My5vcmcvMTk5OS94bGluayIgdmVyc2lvbj0iMS4xIj4KIDwhLS0gR2VuZXJhdGVkIGJ5IFBpeGVsbWF0b3IgUHJvIDIuMC41IC0tPgogPGRlZnM+CiAgPHRleHQgaWQ9InN0cmluZyIgdHJhbnNmb3JtPSJtYXRyaXgoMS4wIDAuMCAwLjAgMS4wIDIyOC4wIDU0LjUpIiBmb250LWZhbWlseT0iQ29tZm9ydGFhLVJlZ3VsYXIsIENvbWZvcnRhYSIgZm9udC1zaXplPSI4MDAiIHRleHQtZGVjb3JhdGlvbj0ibm9uZSIgZmlsbD0iI2ZmZmZmZiIgeD0iMS4xOTk5OTk5OTk5OTk5ODg2IiB5PSI3MDUuMCI+bjwvdGV4dD4KIDwvZGVmcz4KIDx1c2UgaWQ9Im4iIHhsaW5rOmhyZWY9IiNzdHJpbmciLz4KPC9zdmc+Cg==\"/></a>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "binding-collaboration",
   "metadata": {
    "papermill": {},
    "tags": []
   },
   "source": [
    "**Tags:** #reddit #subreddit #data #hottopics #rss #information"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "unknown-invite",
   "metadata": {
    "papermill": {},
    "tags": []
   },
   "source": [
    "This notebook explains how to get hot posts from a subreddit. A subreddit is a specific online community, and the posts associated with it, on the social media website Reddit"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "spread-syndicate",
   "metadata": {
    "papermill": {},
    "tags": []
   },
   "source": [
    "**Author:** [Yaswanthkumar GOTHIREDDY](https://www.linkedin.com/in/yaswanthkumargothireddy/)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "postal-restoration",
   "metadata": {
    "papermill": {},
    "tags": []
   },
   "source": [
    "## Input"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "stuffed-prayer",
   "metadata": {
    "papermill": {},
    "tags": []
   },
   "source": [
    "### Install packages"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "executed-tulsa",
   "metadata": {
    "papermill": {},
    "tags": []
   },
   "outputs": [],
   "source": [
    "!pip install praw"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "widespread-moment",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2021-08-16T10:50:25.695106Z",
     "iopub.status.busy": "2021-08-16T10:50:25.694880Z",
     "iopub.status.idle": "2021-08-16T10:50:26.614177Z",
     "shell.execute_reply": "2021-08-16T10:50:26.613505Z",
     "shell.execute_reply.started": "2021-08-16T10:50:25.695074Z"
    },
    "papermill": {},
    "tags": []
   },
   "outputs": [],
   "source": [
    "import praw\n",
    "import pandas as pd\n",
    "import numpy as np\n",
    "from datetime import datetime"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "rational-reason",
   "metadata": {
    "papermill": {},
    "tags": []
   },
   "source": [
    "### Choose Subreddit topic "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "infectious-population",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2021-08-16T10:50:20.318427Z",
     "iopub.status.busy": "2021-08-16T10:50:20.318213Z",
     "iopub.status.idle": "2021-08-16T10:50:20.408912Z",
     "shell.execute_reply": "2021-08-16T10:50:20.408284Z",
     "shell.execute_reply.started": "2021-08-16T10:50:20.318406Z"
    },
    "papermill": {},
    "tags": []
   },
   "outputs": [],
   "source": [
    "SUBREDDIT = \"Python\" #example: \"CryptoCurrency\""
   ]
  },
  {
   "cell_type": "markdown",
   "id": "behavioral-beatles",
   "metadata": {
    "papermill": {},
    "tags": []
   },
   "source": [
    "### Setup App to connect to Reddit API"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "still-drill",
   "metadata": {
    "papermill": {},
    "tags": []
   },
   "source": [
    "* To get data from reddit, you need to [create a reddit app](https://www.reddit.com/prefs/apps) which queries the reddit API.\n",
    "* Select “script” as the type of app.\n",
    "* Name your app and give it a description.\n",
    "* Set-up the redirect uri to be http://localhost:8080.\n",
    "* Once you click on “create app”, you will get a box showing you your \"client_id\" and \"client_secrets\".\n",
    "* \"user_agent\" is the name of your app."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "dedicated-psychiatry",
   "metadata": {
    "papermill": {},
    "tags": []
   },
   "source": [
    "If you need help on setting up and getting your API credentials, please visit ---> [Get Reddit API Credentials](https://www.jcchouinard.com/get-reddit-api-credentials-with-praw/)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "smoking-mount",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2021-08-16T10:50:20.268922Z",
     "iopub.status.busy": "2021-08-16T10:50:20.268382Z",
     "iopub.status.idle": "2021-08-16T10:50:20.276829Z",
     "shell.execute_reply": "2021-08-16T10:50:20.276337Z",
     "shell.execute_reply.started": "2021-08-16T10:50:20.268826Z"
    },
    "papermill": {},
    "tags": []
   },
   "outputs": [],
   "source": [
    "MY_CLIENT_ID = 'EtAr0o-oKbVuEnPOFbrRqQ'\n",
    "MY_CLIENT_SECRET = 'LmNpsZuFM-WXyZULAayVyNsOhMd_ug'\n",
    "MY_USER_AGENT = 'script by u/naas'"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "accurate-bibliography",
   "metadata": {
    "papermill": {},
    "tags": []
   },
   "source": [
    "## Model"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "aquatic-pride",
   "metadata": {
    "papermill": {},
    "tags": []
   },
   "source": [
    "#### Connect with the reddit API"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "supposed-amateur",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2021-08-16T10:50:26.619512Z",
     "iopub.status.busy": "2021-08-16T10:50:26.617908Z",
     "iopub.status.idle": "2021-08-16T10:50:26.624762Z",
     "shell.execute_reply": "2021-08-16T10:50:26.624183Z",
     "shell.execute_reply.started": "2021-08-16T10:50:26.619476Z"
    },
    "papermill": {},
    "tags": []
   },
   "outputs": [],
   "source": [
    "reddit = praw.Reddit(client_id=MY_CLIENT_ID, client_secret=MY_CLIENT_SECRET, user_agent=MY_USER_AGENT)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "possible-opinion",
   "metadata": {
    "papermill": {},
    "tags": []
   },
   "source": [
    "#### Get the subreddit level data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "understood-breed",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2021-08-16T10:50:26.629406Z",
     "iopub.status.busy": "2021-08-16T10:50:26.627722Z",
     "iopub.status.idle": "2021-08-16T10:50:27.760669Z",
     "shell.execute_reply": "2021-08-16T10:50:27.755126Z",
     "shell.execute_reply.started": "2021-08-16T10:50:26.629374Z"
    },
    "papermill": {},
    "tags": []
   },
   "outputs": [],
   "source": [
    "posts =[]\n",
    "for post in reddit.subreddit(SUBREDDIT).hot(limit=50):\n",
    "    \n",
    "    posts.append([post.title, post.score, post.id, post.subreddit, post.url, post.num_comments, post.selftext, post.created])\n",
    "posts = pd.DataFrame(posts,columns=['title', 'score', 'id', 'subreddit', 'url', 'num_comments', 'body', 'created'])"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "eleven-internship",
   "metadata": {
    "papermill": {},
    "tags": []
   },
   "source": [
    "* If you need more variables, check \"vars()\" function``\n",
    "* Usage: 'vars(post)', you'll get post level variables "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "moved-flush",
   "metadata": {
    "papermill": {},
    "tags": []
   },
   "source": [
    "#### Convert unix timestamp to interpretable date-time "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "electronic-breast",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2021-08-16T10:50:27.764769Z",
     "iopub.status.busy": "2021-08-16T10:50:27.764504Z",
     "iopub.status.idle": "2021-08-16T10:50:27.775251Z",
     "shell.execute_reply": "2021-08-16T10:50:27.774729Z",
     "shell.execute_reply.started": "2021-08-16T10:50:27.764736Z"
    },
    "papermill": {},
    "tags": []
   },
   "outputs": [],
   "source": [
    "posts['created']=pd.to_datetime(posts[\"created\"],unit='s')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "expanded-juvenile",
   "metadata": {
    "papermill": {},
    "tags": []
   },
   "source": [
    "## Output"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "limited-savings",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2021-08-16T10:50:27.777295Z",
     "iopub.status.busy": "2021-08-16T10:50:27.776876Z",
     "iopub.status.idle": "2021-08-16T10:50:28.091835Z",
     "shell.execute_reply": "2021-08-16T10:50:28.091148Z",
     "shell.execute_reply.started": "2021-08-16T10:50:27.777262Z"
    },
    "papermill": {},
    "tags": []
   },
   "outputs": [],
   "source": [
    "posts.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "urban-cycle",
   "metadata": {
    "papermill": {},
    "tags": []
   },
   "source": [
    "Hint: Filter data using \"created\" variable for past 24 hours hot posts"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "acquired-outdoors",
   "metadata": {
    "papermill": {},
    "tags": []
   },
   "source": [
    "## Additional Resources\n",
    "- More info on the PRAW package used: https://praw.readthedocs.io/en/stable/"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.8"
  },
  "papermill": {
   "default_parameters": {},
   "environment_variables": {},
   "parameters": {},
   "version": "2.3.3"
  },
  "widgets": {
   "application/vnd.jupyter.widget-state+json": {
    "state": {},
    "version_major": 2,
    "version_minor": 0
   }
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}