diff --git a/README.txt b/README.md similarity index 82% rename from README.txt rename to README.md index a633dd0..99cdf6a 100644 --- a/README.txt +++ b/README.md @@ -1,4 +1,12 @@ -tRECS +# tRECS + +- [![Binder](https://mybinder.org/badge.svg)](https://mybinder.org/v2/gh/4QuantOSS/tRECS/master) [![Binder]( https://img.shields.io/badge/launch-jupyterlab-red.svg)](https://mybinder.org/v2/gh/4QuantOSS/tRECS/master?urlpath=lab) + +## Demos + +- [![Binder](https://img.shields.io/badge/launch-spooky%20demo-green.svg)](https://mybinder.org/v2/gh/4QuantOSS/tRECS/master?urlpath=%2Fapps%2Fdemo_spooky.ipynb) from the [Kaggle Spooky Author Competition](https://www.kaggle.com/c/spooky-author-identification/data) + + Text recommendation system developer built in Python and Dash by Plotly. @@ -6,23 +14,23 @@ Authors: Taylor Olson, Janie Neal, Christiana Prater-Lee, Eshita Nandini This recommendation system developer guides the user through cleaning their data, building models, and ultimately creates a recommendation system (housed within the interface). The user can also visualize some the models and other features of their data. -INSTALLATION +## INSTALLATION The user should be using python 2.7+. Most packages required for the interface can be installed in the command line by navigating to the directory containing the reqs.txt file and running "pip install -r reqs.txt". Some packages require further installation. Links to their installation documentation are provided below. -Spacy: https://spacy.io/docs/usage/ +Spacy: https://spacy.io/docs/usage/ Follow the instructions for downloading the english model. - + NLTK: http://www.nltk.org/data.html# Follow the instructions to open the interactive installer and install the following corpora: wordnet, wordnet_ic, words, and stopwords. -HOW TO START THE DASH INTERFACE +## HOW TO START THE DASH INTERFACE In order to start the interface, the user should navigate to the "interface" folder in the command line and call "python interface.py". The command window will provide a link that the user can then copy into the address bar of a browser of their choice. The interface should open in that window, and the user can begin using the features. -FEATURES OF THE INTERFACE AND HOW TO USE THEM +## FEATURES OF THE INTERFACE AND HOW TO USE THEM Upload your own data: User can copy custom data (see sample data for desired format) into the interface folder or use some of the sets included in the interface folder. @@ -34,28 +42,27 @@ User is required to remove non-ascii characters and punctuation. There are also stemming vs. lemmatizing: https://nlp.stanford.edu/IR-book/html/htmledition/stemming-and-lemmatization-1.html Word cut off: -Numbers on the slider indicate the percentage of documents that the words appear in. Use the slider to remove words that occur too frequently or not frequently enough. +Numbers on the slider indicate the percentage of documents that the words appear in. Use the slider to remove words that occur too frequently or not frequently enough. Choose and Build your models: Options are TF-IDF, LSA, LDA, and spaCy's built in similarity function. -Recommendation: +Recommendation: Uses the models built in the previous step to provide the selected number of most similar objects to the selected object. Visualizations: The following visualizations have been connected to the interface: entity visualization, lda clustering. Please see their descriptions in the description of visualizations.py. -CONTENTS OF MASTER FOLDER +## CONTENTS OF MASTER FOLDER Interface.py: contains the code needed for front end of application, including: HTML layout, Dash components and event handlers, calls to backend. Contains iqss_interface object (below). -Iqss_interface.py: contains all coded needed for back end of application, including: user loaded data frame, cleaned data frame, and built models. Contains two objects, the iqss_interface object (holds the data frames) and the model_object (holds the necessary df, tf-idf, lda, and spacy models). +Iqss_interface.py: contains all coded needed for back end of application, including: user loaded data frame, cleaned data frame, and built models. Contains two objects, the iqss_interface object (holds the data frames) and the model_object (holds the necessary df, tf-idf, lda, and spacy models). Utils.py: contains all methods used for manipulating data frames, including: converting to term and document frequency matrix, cleaning the data frame, extracting entities from the data frame, and determining the number of topics based on the contents of the data frame. - -Visualizations.py: contains all code necessary to build the various visualizations, including: + +Visualizations.py: contains all code necessary to build the various visualizations, including: the LDA cluster graph: http://brandonrose.org/clustering the similarity graph: a graph that plots the similarity score of an object on the x axis, and the group which the object is part of is plotted on the y axis LDA topic distribution: https://pyldavis.readthedocs.io/en/latest/readme.html#installation - diff --git a/__pycache__/iqss_interface.cpython-36.pyc b/__pycache__/iqss_interface.cpython-36.pyc deleted file mode 100644 index 992b59e..0000000 Binary files a/__pycache__/iqss_interface.cpython-36.pyc and /dev/null differ diff --git a/__pycache__/utils.cpython-36.pyc b/__pycache__/utils.cpython-36.pyc deleted file mode 100644 index 63b62b3..0000000 Binary files a/__pycache__/utils.cpython-36.pyc and /dev/null differ diff --git a/__pycache__/visualizations.cpython-36.pyc b/__pycache__/visualizations.cpython-36.pyc deleted file mode 100644 index a2016fc..0000000 Binary files a/__pycache__/visualizations.cpython-36.pyc and /dev/null differ diff --git a/binder/environment.yml b/binder/environment.yml new file mode 100644 index 0000000..4490243 --- /dev/null +++ b/binder/environment.yml @@ -0,0 +1,24 @@ + +name: trecs +channels: + - conda-forge + - defaults +dependencies: + - python=3.6 + - numpy + - matplotlib + - pandas + - scipy + - scikit-learn + - spacy + - nltk + - nbserverproxy + - appmode + - pip: + - ipywidgets==7.0.1 # for interactive plots + - dash==0.21.0 # The core dash backend + - dash-renderer==0.11.3 # The dash front-end + - dash-html-components==0.9.0 # HTML components + - dash-core-components==0.18.1 # Supercharged components + - plotly==2.4.1 # Plotly graphing library used in examples + - lda diff --git a/binder/postBuild b/binder/postBuild new file mode 100755 index 0000000..30d7364 --- /dev/null +++ b/binder/postBuild @@ -0,0 +1,3 @@ +source activate kernel # ensure we are in the current environment +python -m nltk.downloader all +python -m spacy download en diff --git a/demo_spooky.ipynb b/demo_spooky.ipynb new file mode 100644 index 0000000..22f87b9 --- /dev/null +++ b/demo_spooky.ipynb @@ -0,0 +1,377 @@ +{ + "cells": [ + { + "cell_type": "code", + "execution_count": 1, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "--2018-02-26 14:49:13-- https://www.dropbox.com/s/wson7j5ih0126zy/spooky.csv?dl=1\n", + "Resolving www.dropbox.com (www.dropbox.com)... 162.125.3.1, 2620:100:6018:1::a27d:301\n", + "Connecting to www.dropbox.com (www.dropbox.com)|162.125.3.1|:443... connected.\n", + "HTTP request sent, awaiting response... 302 Found\n", + "Location: https://dl.dropboxusercontent.com/content_link/8iAAg1S5JxyVJFxYFWrgXepA6tajtLgCjc7pCrIvHZ1MUTxWN5JAP6IQDTUpJ7FF/file?dl=1 [following]\n", + "--2018-02-26 14:49:13-- https://dl.dropboxusercontent.com/content_link/8iAAg1S5JxyVJFxYFWrgXepA6tajtLgCjc7pCrIvHZ1MUTxWN5JAP6IQDTUpJ7FF/file?dl=1\n", + "Resolving dl.dropboxusercontent.com (dl.dropboxusercontent.com)... 162.125.7.6, 2620:100:601d:6::a27d:506\n", + "Connecting to dl.dropboxusercontent.com (dl.dropboxusercontent.com)|162.125.7.6|:443... connected.\n", + "HTTP request sent, awaiting response... 200 OK\n", + "Length: 3295644 (3.1M) [application/binary]\n", + "Saving to: ‘spooky.csv’\n", + "\n", + "spooky.csv 100%[===================>] 3.14M 11.3MB/s in 0.3s \n", + "\n", + "2018-02-26 14:49:14 (11.3 MB/s) - ‘spooky.csv’ saved [3295644/3295644]\n", + "\n" + ] + } + ], + "source": [ + "!wget https://www.dropbox.com/s/wson7j5ih0126zy/spooky.csv?dl=1 -O spooky.csv" + ] + }, + { + "cell_type": "code", + "execution_count": 2, + "metadata": {}, + "outputs": [], + "source": [ + "import os, sys\n", + "from IPython import display\n", + "import random\n", + "def show_app(app, port = None, \n", + " width = \"100%\", \n", + " height = '800px', \n", + " offline = False,\n", + " in_binder = None):\n", + " if port is None:\n", + " port = random.choice(range(1000, 10000))\n", + " in_binder ='JUPYTERHUB_SERVICE_PREFIX' in os.environ if in_binder is None else in_binder\n", + " if in_binder:\n", + " base_prefix = '{}proxy/{}/'.format(os.environ['JUPYTERHUB_SERVICE_PREFIX'], port)\n", + " url = 'https://hub.mybinder.org{}'.format(base_prefix)\n", + " app.config.requests_pathname_prefix = base_prefix\n", + " else:\n", + " url = 'http://localhost:%d' % port\n", + " \n", + " iframe = 'Open in new window
'.format(url = url, \n", + " width = width, \n", + " height = height)\n", + " \n", + " display.display_html(iframe, raw = True)\n", + " if offline:\n", + " app.css.config.serve_locally = True\n", + " app.scripts.config.serve_locally = True\n", + " else:\n", + " app.css.config.serve_locally = False\n", + " app.scripts.config.serve_locally = False\n", + " return app.run_server(debug=False, # needs to be false in Jupyter\n", + " host = '0.0.0.0',\n", + " port=port)" + ] + }, + { + "cell_type": "code", + "execution_count": 3, + "metadata": {}, + "outputs": [], + "source": [ + "from interface import app" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "scrolled": false + }, + "outputs": [ + { + "data": { + "text/html": [ + "Open in new window
" + ] + }, + "metadata": {}, + "output_type": "display_data" + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + " * Running on http://0.0.0.0:7430/ (Press CTRL+C to quit)\n", + "127.0.0.1 - - [26/Feb/2018 14:49:17] \"\u001b[37mGET / HTTP/1.1\u001b[0m\" 200 -\n", + "127.0.0.1 - - [26/Feb/2018 14:49:18] \"\u001b[37mGET /_dash-dependencies HTTP/1.1\u001b[0m\" 200 -\n", + "127.0.0.1 - - [26/Feb/2018 14:49:18] \"\u001b[37mGET /_dash-layout HTTP/1.1\u001b[0m\" 200 -\n", + "127.0.0.1 - - [26/Feb/2018 14:49:18] \"\u001b[37mPOST /_dash-update-component HTTP/1.1\u001b[0m\" 200 -\n", + "127.0.0.1 - - [26/Feb/2018 14:49:18] \"\u001b[37mPOST /_dash-update-component HTTP/1.1\u001b[0m\" 200 -\n", + "127.0.0.1 - - [26/Feb/2018 14:49:22] \"\u001b[37mPOST /_dash-update-component HTTP/1.1\u001b[0m\" 200 -\n", + "127.0.0.1 - - [26/Feb/2018 14:49:22] \"\u001b[37mPOST /_dash-update-component HTTP/1.1\u001b[0m\" 200 -\n", + "127.0.0.1 - - [26/Feb/2018 14:49:25] \"\u001b[37mPOST /_dash-update-component HTTP/1.1\u001b[0m\" 200 -\n", + "127.0.0.1 - - [26/Feb/2018 14:49:25] \"\u001b[37mPOST /_dash-update-component HTTP/1.1\u001b[0m\" 200 -\n", + "127.0.0.1 - - [26/Feb/2018 14:49:27] \"\u001b[37mPOST /_dash-update-component HTTP/1.1\u001b[0m\" 200 -\n", + "127.0.0.1 - - [26/Feb/2018 14:49:27] \"\u001b[37mPOST /_dash-update-component HTTP/1.1\u001b[0m\" 200 -\n", + "127.0.0.1 - - [26/Feb/2018 14:49:34] \"\u001b[37mPOST /_dash-update-component HTTP/1.1\u001b[0m\" 200 -\n", + "127.0.0.1 - - [26/Feb/2018 14:49:34] \"\u001b[37mPOST /_dash-update-component HTTP/1.1\u001b[0m\" 200 -\n", + "127.0.0.1 - - [26/Feb/2018 14:49:38] \"\u001b[37mPOST /_dash-update-component HTTP/1.1\u001b[0m\" 200 -\n", + "[2018-02-26 14:49:38,845] ERROR in app: Exception on /_dash-update-component [POST]\n", + "Traceback (most recent call last):\n", + " File \"/srv/conda/lib/python3.6/site-packages/flask/app.py\", line 1982, in wsgi_app\n", + " response = self.full_dispatch_request()\n", + " File \"/srv/conda/lib/python3.6/site-packages/flask/app.py\", line 1614, in full_dispatch_request\n", + " rv = self.handle_user_exception(e)\n", + " File \"/srv/conda/lib/python3.6/site-packages/flask/app.py\", line 1517, in handle_user_exception\n", + " reraise(exc_type, exc_value, tb)\n", + " File \"/srv/conda/lib/python3.6/site-packages/flask/_compat.py\", line 33, in reraise\n", + " raise value\n", + " File \"/srv/conda/lib/python3.6/site-packages/flask/app.py\", line 1612, in full_dispatch_request\n", + " rv = self.dispatch_request()\n", + " File \"/srv/conda/lib/python3.6/site-packages/flask/app.py\", line 1598, in dispatch_request\n", + " return self.view_functions[rule.endpoint](**req.view_args)\n", + " File \"/srv/conda/lib/python3.6/site-packages/dash/dash.py\", line 556, in dispatch\n", + " return self.callback_map[target_id]['callback'](*args)\n", + " File \"/srv/conda/lib/python3.6/site-packages/dash/dash.py\", line 513, in add_context\n", + " output_value = func(*args, **kwargs)\n", + " File \"/home/jovyan/interface.py\", line 146, in test\n", + " interface_obj.default_clean()\n", + " File \"/home/jovyan/iqss_interface.py\", line 63, in default_clean\n", + " self.clean_df[self.clean_df.columns[1]] = self.clean_df[self.clean_df.columns[1]].apply(utils.remove_non_ascii).to_frame()\n", + " File \"/srv/conda/lib/python3.6/site-packages/pandas/core/series.py\", line 2551, in apply\n", + " mapped = lib.map_infer(values, f, convert=convert_dtype)\n", + " File \"pandas/_libs/src/inference.pyx\", line 1521, in pandas._libs.lib.map_infer\n", + " File \"/home/jovyan/utils.py\", line 144, in remove_non_ascii\n", + " return ''.join(stripped)\n", + " File \"/home/jovyan/utils.py\", line 143, in \n", + " stripped = (unicode(str(c), 'utf-8') for c in text if 0 < ord(c) < 127)\n", + "NameError: name 'unicode' is not defined\n", + "127.0.0.1 - - [26/Feb/2018 14:49:38] \"\u001b[1m\u001b[35mPOST /_dash-update-component HTTP/1.1\u001b[0m\" 500 -\n", + "127.0.0.1 - - [26/Feb/2018 14:49:48] \"\u001b[37mGET / HTTP/1.1\u001b[0m\" 200 -\n", + "127.0.0.1 - - [26/Feb/2018 14:49:49] \"\u001b[37mGET /_dash-dependencies HTTP/1.1\u001b[0m\" 200 -\n", + "127.0.0.1 - - [26/Feb/2018 14:49:49] \"\u001b[37mGET /_dash-layout HTTP/1.1\u001b[0m\" 200 -\n", + "127.0.0.1 - - [26/Feb/2018 14:49:49] \"\u001b[37mPOST /_dash-update-component HTTP/1.1\u001b[0m\" 200 -\n", + "127.0.0.1 - - [26/Feb/2018 14:49:49] \"\u001b[37mPOST /_dash-update-component HTTP/1.1\u001b[0m\" 200 -\n", + "127.0.0.1 - - [26/Feb/2018 14:49:54] \"\u001b[37mPOST /_dash-update-component HTTP/1.1\u001b[0m\" 200 -\n", + "127.0.0.1 - - [26/Feb/2018 14:49:55] \"\u001b[37mPOST /_dash-update-component HTTP/1.1\u001b[0m\" 200 -\n", + "127.0.0.1 - - [26/Feb/2018 14:49:59] \"\u001b[37mPOST /_dash-update-component HTTP/1.1\u001b[0m\" 200 -\n", + "127.0.0.1 - - [26/Feb/2018 14:49:59] \"\u001b[37mPOST /_dash-update-component HTTP/1.1\u001b[0m\" 200 -\n", + "127.0.0.1 - - [26/Feb/2018 14:50:00] \"\u001b[37mPOST /_dash-update-component HTTP/1.1\u001b[0m\" 200 -\n", + "127.0.0.1 - - [26/Feb/2018 14:50:00] \"\u001b[37mPOST /_dash-update-component HTTP/1.1\u001b[0m\" 200 -\n", + "127.0.0.1 - - [26/Feb/2018 14:50:06] \"\u001b[37mPOST /_dash-update-component HTTP/1.1\u001b[0m\" 200 -\n", + "[2018-02-26 14:50:06,256] ERROR in app: Exception on /_dash-update-component [POST]\n", + "Traceback (most recent call last):\n", + " File \"/srv/conda/lib/python3.6/site-packages/flask/app.py\", line 1982, in wsgi_app\n", + " response = self.full_dispatch_request()\n", + " File \"/srv/conda/lib/python3.6/site-packages/flask/app.py\", line 1614, in full_dispatch_request\n", + " rv = self.handle_user_exception(e)\n", + " File \"/srv/conda/lib/python3.6/site-packages/flask/app.py\", line 1517, in handle_user_exception\n", + " reraise(exc_type, exc_value, tb)\n", + " File \"/srv/conda/lib/python3.6/site-packages/flask/_compat.py\", line 33, in reraise\n", + " raise value\n", + " File \"/srv/conda/lib/python3.6/site-packages/flask/app.py\", line 1612, in full_dispatch_request\n", + " rv = self.dispatch_request()\n", + " File \"/srv/conda/lib/python3.6/site-packages/flask/app.py\", line 1598, in dispatch_request\n", + " return self.view_functions[rule.endpoint](**req.view_args)\n", + " File \"/srv/conda/lib/python3.6/site-packages/dash/dash.py\", line 556, in dispatch\n", + " return self.callback_map[target_id]['callback'](*args)\n", + " File \"/srv/conda/lib/python3.6/site-packages/dash/dash.py\", line 513, in add_context\n", + " output_value = func(*args, **kwargs)\n", + " File \"/home/jovyan/interface.py\", line 202, in test\n", + " interface_obj.build_doc_freq()\n", + " File \"/home/jovyan/iqss_interface.py\", line 53, in build_doc_freq\n", + " self.freq_matrix = utils.build_freq_matrix(self.clean_df)\n", + " File \"/home/jovyan/utils.py\", line 68, in build_freq_matrix\n", + " frequency_matrix, bag_of_words = termdocument_matrix(dataframe)\n", + " File \"/home/jovyan/utils.py\", line 25, in termdocument_matrix\n", + " descriptions = [unicode(str(dscr), 'utf-8').split(\" \") for dscr in descriptions]\n", + " File \"/home/jovyan/utils.py\", line 25, in \n", + " descriptions = [unicode(str(dscr), 'utf-8').split(\" \") for dscr in descriptions]\n", + "NameError: name 'unicode' is not defined\n", + "127.0.0.1 - - [26/Feb/2018 14:50:06] \"\u001b[1m\u001b[35mPOST /_dash-update-component HTTP/1.1\u001b[0m\" 500 -\n", + "127.0.0.1 - - [26/Feb/2018 14:50:11] \"\u001b[37mPOST /_dash-update-component HTTP/1.1\u001b[0m\" 200 -\n", + "127.0.0.1 - - [26/Feb/2018 14:50:11] \"\u001b[37mPOST /_dash-update-component HTTP/1.1\u001b[0m\" 200 -\n", + "[2018-02-26 14:50:19,075] ERROR in app: Exception on /_dash-update-component [POST]\n", + "Traceback (most recent call last):\n", + " File \"/srv/conda/lib/python3.6/site-packages/flask/app.py\", line 1982, in wsgi_app\n", + " response = self.full_dispatch_request()\n", + " File \"/srv/conda/lib/python3.6/site-packages/flask/app.py\", line 1614, in full_dispatch_request\n", + " rv = self.handle_user_exception(e)\n", + " File \"/srv/conda/lib/python3.6/site-packages/flask/app.py\", line 1517, in handle_user_exception\n", + " reraise(exc_type, exc_value, tb)\n", + " File \"/srv/conda/lib/python3.6/site-packages/flask/_compat.py\", line 33, in reraise\n", + " raise value\n", + " File \"/srv/conda/lib/python3.6/site-packages/flask/app.py\", line 1612, in full_dispatch_request\n", + " rv = self.dispatch_request()\n", + " File \"/srv/conda/lib/python3.6/site-packages/flask/app.py\", line 1598, in dispatch_request\n", + " return self.view_functions[rule.endpoint](**req.view_args)\n", + " File \"/srv/conda/lib/python3.6/site-packages/dash/dash.py\", line 556, in dispatch\n", + " return self.callback_map[target_id]['callback'](*args)\n", + " File \"/srv/conda/lib/python3.6/site-packages/dash/dash.py\", line 513, in add_context\n", + " output_value = func(*args, **kwargs)\n", + " File \"/home/jovyan/interface.py\", line 645, in build_models\n", + " lsa_nt = utils.get_num_of_topics(interface_obj.clean_df)\n", + " File \"/home/jovyan/utils.py\", line 44, in get_num_of_topics\n", + " frequency_matrix, bag_of_words = termdocument_matrix(dataframe)\n", + " File \"/home/jovyan/utils.py\", line 25, in termdocument_matrix\n", + " descriptions = [unicode(str(dscr), 'utf-8').split(\" \") for dscr in descriptions]\n", + " File \"/home/jovyan/utils.py\", line 25, in \n", + " descriptions = [unicode(str(dscr), 'utf-8').split(\" \") for dscr in descriptions]\n", + "NameError: name 'unicode' is not defined\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "127.0.0.1 - - [26/Feb/2018 14:50:19] \"\u001b[1m\u001b[35mPOST /_dash-update-component HTTP/1.1\u001b[0m\" 500 -\n", + "127.0.0.1 - - [26/Feb/2018 14:50:24] \"\u001b[37mPOST /_dash-update-component HTTP/1.1\u001b[0m\" 200 -\n", + "127.0.0.1 - - [26/Feb/2018 14:50:24] \"\u001b[37mPOST /_dash-update-component HTTP/1.1\u001b[0m\" 200 -\n", + "[2018-02-26 14:50:26,749] ERROR in app: Exception on /_dash-update-component [POST]\n", + "Traceback (most recent call last):\n", + " File \"/srv/conda/lib/python3.6/site-packages/flask/app.py\", line 1982, in wsgi_app\n", + " response = self.full_dispatch_request()\n", + " File \"/srv/conda/lib/python3.6/site-packages/flask/app.py\", line 1614, in full_dispatch_request\n", + " rv = self.handle_user_exception(e)\n", + " File \"/srv/conda/lib/python3.6/site-packages/flask/app.py\", line 1517, in handle_user_exception\n", + " reraise(exc_type, exc_value, tb)\n", + " File \"/srv/conda/lib/python3.6/site-packages/flask/_compat.py\", line 33, in reraise\n", + " raise value\n", + " File \"/srv/conda/lib/python3.6/site-packages/flask/app.py\", line 1612, in full_dispatch_request\n", + " rv = self.dispatch_request()\n", + " File \"/srv/conda/lib/python3.6/site-packages/flask/app.py\", line 1598, in dispatch_request\n", + " return self.view_functions[rule.endpoint](**req.view_args)\n", + " File \"/srv/conda/lib/python3.6/site-packages/dash/dash.py\", line 556, in dispatch\n", + " return self.callback_map[target_id]['callback'](*args)\n", + " File \"/srv/conda/lib/python3.6/site-packages/dash/dash.py\", line 513, in add_context\n", + " output_value = func(*args, **kwargs)\n", + " File \"/home/jovyan/interface.py\", line 667, in get_similar\n", + " courses = interface_obj.iqss_model.get_similar(course_name, int(amount))\n", + "TypeError: int() argument must be a string, a bytes-like object or a number, not 'NoneType'\n", + "127.0.0.1 - - [26/Feb/2018 14:50:26] \"\u001b[1m\u001b[35mPOST /_dash-update-component HTTP/1.1\u001b[0m\" 500 -\n", + "127.0.0.1 - - [26/Feb/2018 14:50:28] \"\u001b[37mPOST /_dash-update-component HTTP/1.1\u001b[0m\" 200 -\n", + "127.0.0.1 - - [26/Feb/2018 14:50:32] \"\u001b[37mPOST /_dash-update-component HTTP/1.1\u001b[0m\" 200 -\n", + "127.0.0.1 - - [26/Feb/2018 14:50:40] \"\u001b[37mPOST /_dash-update-component HTTP/1.1\u001b[0m\" 200 -\n", + "127.0.0.1 - - [26/Feb/2018 14:50:40] \"\u001b[37mPOST /_dash-update-component HTTP/1.1\u001b[0m\" 200 -\n", + "127.0.0.1 - - [26/Feb/2018 14:50:40] \"\u001b[37mPOST /_dash-update-component HTTP/1.1\u001b[0m\" 200 -\n", + "INFO:lda:n_documents: 19579\n", + "INFO:lda:vocab_size: 3\n", + "INFO:lda:n_words: 19579\n", + "INFO:lda:n_topics: 10\n", + "INFO:lda:n_iter: 1\n", + "/srv/conda/lib/python3.6/site-packages/lda/utils.py:55: FutureWarning:\n", + "\n", + "Conversion of the second argument of issubdtype from `int` to `np.signedinteger` is deprecated. In future, it will be treated as `np.int64 == np.dtype(int).type`.\n", + "\n", + "INFO:lda:<0> log likelihood: -66514\n", + "INFO:lda:<0> log likelihood: -66482\n", + "127.0.0.1 - - [26/Feb/2018 14:50:43] \"\u001b[37mPOST /_dash-update-component HTTP/1.1\u001b[0m\" 200 -\n", + "INFO:werkzeug:127.0.0.1 - - [26/Feb/2018 14:50:43] \"\u001b[37mPOST /_dash-update-component HTTP/1.1\u001b[0m\" 200 -\n", + "[2018-02-26 14:50:43,742] ERROR in app: Exception on /_dash-update-component [POST]\n", + "Traceback (most recent call last):\n", + " File \"/srv/conda/lib/python3.6/site-packages/flask/app.py\", line 1982, in wsgi_app\n", + " response = self.full_dispatch_request()\n", + " File \"/srv/conda/lib/python3.6/site-packages/flask/app.py\", line 1614, in full_dispatch_request\n", + " rv = self.handle_user_exception(e)\n", + " File \"/srv/conda/lib/python3.6/site-packages/flask/app.py\", line 1517, in handle_user_exception\n", + " reraise(exc_type, exc_value, tb)\n", + " File \"/srv/conda/lib/python3.6/site-packages/flask/_compat.py\", line 33, in reraise\n", + " raise value\n", + " File \"/srv/conda/lib/python3.6/site-packages/flask/app.py\", line 1612, in full_dispatch_request\n", + " rv = self.dispatch_request()\n", + " File \"/srv/conda/lib/python3.6/site-packages/flask/app.py\", line 1598, in dispatch_request\n", + " return self.view_functions[rule.endpoint](**req.view_args)\n", + " File \"/srv/conda/lib/python3.6/site-packages/dash/dash.py\", line 556, in dispatch\n", + " return self.callback_map[target_id]['callback'](*args)\n", + " File \"/srv/conda/lib/python3.6/site-packages/dash/dash.py\", line 513, in add_context\n", + " output_value = func(*args, **kwargs)\n", + " File \"/home/jovyan/interface.py\", line 829, in build_cluster_graph\n", + " fig = cluster_obj.set_graph(slider_val2, dim)\n", + " File \"/home/jovyan/visualizations.py\", line 63, in set_graph\n", + " return self.create_plot()\n", + " File \"/home/jovyan/visualizations.py\", line 101, in create_plot\n", + " _lda_keys = self.get_lda_keys()\n", + " File \"/home/jovyan/visualizations.py\", line 152, in get_lda_keys\n", + " for i in xrange(self.X_topics_current.shape[0]):\n", + "NameError: name 'xrange' is not defined\n", + "127.0.0.1 - - [26/Feb/2018 14:50:43] \"\u001b[1m\u001b[35mPOST /_dash-update-component HTTP/1.1\u001b[0m\" 500 -\n", + "INFO:werkzeug:127.0.0.1 - - [26/Feb/2018 14:50:43] \"\u001b[1m\u001b[35mPOST /_dash-update-component HTTP/1.1\u001b[0m\" 500 -\n", + "127.0.0.1 - - [26/Feb/2018 14:50:43] \"\u001b[37mPOST /_dash-update-component HTTP/1.1\u001b[0m\" 200 -\n", + "INFO:werkzeug:127.0.0.1 - - [26/Feb/2018 14:50:43] \"\u001b[37mPOST /_dash-update-component HTTP/1.1\u001b[0m\" 200 -\n", + "INFO:lda:n_documents: 19579\n", + "INFO:lda:vocab_size: 3\n", + "INFO:lda:n_words: 19579\n", + "INFO:lda:n_topics: 10\n", + "INFO:lda:n_iter: 1\n", + "INFO:lda:<0> log likelihood: -66514\n", + "INFO:lda:<0> log likelihood: -66461\n", + "127.0.0.1 - - [26/Feb/2018 14:50:46] \"\u001b[37mPOST /_dash-update-component HTTP/1.1\u001b[0m\" 200 -\n", + "INFO:werkzeug:127.0.0.1 - - [26/Feb/2018 14:50:46] \"\u001b[37mPOST /_dash-update-component HTTP/1.1\u001b[0m\" 200 -\n", + "127.0.0.1 - - [26/Feb/2018 14:50:46] \"\u001b[37mPOST /_dash-update-component HTTP/1.1\u001b[0m\" 200 -\n", + "INFO:werkzeug:127.0.0.1 - - [26/Feb/2018 14:50:46] \"\u001b[37mPOST /_dash-update-component HTTP/1.1\u001b[0m\" 200 -\n", + "[2018-02-26 14:50:46,519] ERROR in app: Exception on /_dash-update-component [POST]\n", + "Traceback (most recent call last):\n", + " File \"/srv/conda/lib/python3.6/site-packages/flask/app.py\", line 1982, in wsgi_app\n", + " response = self.full_dispatch_request()\n", + " File \"/srv/conda/lib/python3.6/site-packages/flask/app.py\", line 1614, in full_dispatch_request\n", + " rv = self.handle_user_exception(e)\n", + " File \"/srv/conda/lib/python3.6/site-packages/flask/app.py\", line 1517, in handle_user_exception\n", + " reraise(exc_type, exc_value, tb)\n", + " File \"/srv/conda/lib/python3.6/site-packages/flask/_compat.py\", line 33, in reraise\n", + " raise value\n", + " File \"/srv/conda/lib/python3.6/site-packages/flask/app.py\", line 1612, in full_dispatch_request\n", + " rv = self.dispatch_request()\n", + " File \"/srv/conda/lib/python3.6/site-packages/flask/app.py\", line 1598, in dispatch_request\n", + " return self.view_functions[rule.endpoint](**req.view_args)\n", + " File \"/srv/conda/lib/python3.6/site-packages/dash/dash.py\", line 556, in dispatch\n", + " return self.callback_map[target_id]['callback'](*args)\n", + " File \"/srv/conda/lib/python3.6/site-packages/dash/dash.py\", line 513, in add_context\n", + " output_value = func(*args, **kwargs)\n", + " File \"/home/jovyan/interface.py\", line 829, in build_cluster_graph\n", + " fig = cluster_obj.set_graph(slider_val2, dim)\n", + " File \"/home/jovyan/visualizations.py\", line 63, in set_graph\n", + " return self.create_plot()\n", + " File \"/home/jovyan/visualizations.py\", line 101, in create_plot\n", + " _lda_keys = self.get_lda_keys()\n", + " File \"/home/jovyan/visualizations.py\", line 152, in get_lda_keys\n", + " for i in xrange(self.X_topics_current.shape[0]):\n", + "NameError: name 'xrange' is not defined\n", + "127.0.0.1 - - [26/Feb/2018 14:50:46] \"\u001b[1m\u001b[35mPOST /_dash-update-component HTTP/1.1\u001b[0m\" 500 -\n", + "INFO:werkzeug:127.0.0.1 - - [26/Feb/2018 14:50:46] \"\u001b[1m\u001b[35mPOST /_dash-update-component HTTP/1.1\u001b[0m\" 500 -\n" + ] + } + ], + "source": [ + "show_app(app)" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.6.4" + } + }, + "nbformat": 4, + "nbformat_minor": 2 +}