BKSCrawler

Please note there are some missing information in this README. I am deeply apologize about such manner.

What is this?

This is a Python library I wrote based on the following 2 sources:

• Packet capturing from Wireshark,
• And PythonTryHard's BKSchedule_Rewrite

The library uses 3 additional libraries: html2json, requests and pyNaCl (all in Python).

When supplied with a username, an encrypted password and an encryption phrase (pyNaCl), the library produces a JSON-formatted output which can be either:

• The schedule;
• The exam timetable;
• Announcements;
• Or the grading as your preferences

The credential information above must be able to access HCMUT MyBK Portal.

Main aims

• Fixes some issues with the original BKSchedule_Rewrite above;
• Adding a logout method;
• Letting others the possiblity to propagates the tables for their tastes.

Installation

Prerequisite:

Assuming you have already installed Python 3.7. It is suggested that the library being installed and used from within Pyhton virtual environment:

python3 -m pip install virtualenv
python3 -m virtualenv venv
source venv/bin/activate

Installation: There are 3 ways to install the library:

Using the wheel release:

This is the easiest way since you don't have to worry about dependencies:

python3 -m pip install BKSCrawler-x.x-py3-none-any.whl

Note: Remember to download the wheel (extension .whl) file from Releases section.

Install from source code:

Clone this repository, change directory to the cloned folder and execute:

python3 setup.py install

Use directly from source code:

Recommended if you don't want to install this package. Still, clone this repository, change directory to the cloned folder and install its dependencies (yes, you still have to install something, at least not this library):

python3 -m pip install -r requirements.txt

Usage:

Initalize:

  import BKSCrawler as bk
  mybk = bk.StInfoCrawl(<Your username here>, <Your NaCl-encrypted password here>)
  mybk.login([Your NaCl encryption phrase here])

Get your information (say, timetable) from HCMUT StInfo Portal:

  mybk.fetch('sched')

Return: A Python dictionary type

Valid commands are:

• 'sched': Get timetable;
• 'grade': Get exam gradings;
• 'exam': Get exam schedule;
• 'msg': Get annoucements available.

Get current week:

  mybk.week

Return: (integer) this week number

Log out of the portal:

  mybk.logout()

Change expiry time of the session:

  mybk.setExpire(<time_in_int>, <Format: min|sec>)

NOTE: The maximum allowed period is 30 minutes

Exceptions

The function will throw BKSCrawler.CrawlerError for most errors (specific exceptions do exist) during the crawling process for most commands.

ADDITIONALLY, for any value errors (missing/invalid command arguments), AssertionError is raised.

License

This library is made available under GNU General Public License v3.0

html2json is available under MIT License

requests is available under Apache 2.0 License

pyNaCl is available under Apache 2.0 License

Future

• Expand Exceptions
• Update other language

FAQ

Q: Can I use non-HCMUT account with this library?

A: No. As stated above, you can only use account that can access HCMUT MyBK Portal.

Q: How does this library works?

A: This library uses requests to mimic the behavior when doing the same thing from the browser. The html2json is used during the request to parse html respond in order to get key component for the success of this crawling proccess.

Q: Why don't you use BeautifulSoup?

A: BeautifulSoup was obsoleted as from Python 3.2! That's why I uses html2json for parsing html output from the request. (proof: Google it)

Q: Can you show me how to create something like this?

A: USE GOOGLE! I can't guide you with that, but you can have my code as a reference.