Account Management Portal

This web site is the backend system for the CSV Manager Web Application

csvbackend

Overview

This back-end web application is designed to be used in conjunction with front-end web site csvmanager.

Anonymous users can view a list of uploaded CSV files, view the contents of a particular file, and view "file statistics". Registered users can upload a new CSV file.

The main features of the application are:

  • Deliver REST API for uploading CSV files
  • Allow user authentication and authorization
  • Provide web-based Account Management

Purpose

Software Design

The software can be run on any computer that has Python version 3.8, and is based on the Django web application framework. In addition, these Python packages provide additional functionality:

  • Django Rest Framework Basis of the Rest API. Includes classes for CRUD Operations, Authentication, and Permissions.

  • Djagno User Unique Email Ensures a given email address can be used for only one username.

  • Django CORS Headers Allows backend website to be accessed by web sites deploy on other specified domains.

  • Django Registration Redux Add methods for creating new accounts via a confirmation email, requesting a an email to reset a password, changing passwords.

  • TOML "Tom's Obvious Markup Language": site-specific settings are stored in a text-based .toml file

Quick Start Guide

Installation

Create a directory to contain both the repo and the virtual environment (parent_dir is the full path of this directory).

mkdir -p {parent_dir}
cd {parent_dir}

Clone the github repo:

git clone repo_URL

Create a Python Virtual Environment and activate it:

python3.8 -m venv backend-env
source backend-env/bin/activate

Install required Python packages

pip install -r requirements.txt

Modify site-specific settings

Copy the example TOML file cp settings.example.toml settings.toml

Edit settings.toml with an SMTP mail account details for host, user name, user password, and user email address.

An existing database (db.sqlite3) is included in the repo, that has been initialized and contains entries for some 3 CSV files. These CSV files are also in the repo in the /media directory. The database contains an entry for a superuser:

name: admin
passsword: keyboardcat (this is not the password on the sample site)

If you wish, you can instead recreate the database from scratch:

rm db.sqlite3
python3.8 manage.py migrate
python3.8 manage.py makemigrations fileupload
python3.8 manage.py migrate

and create a superuser:

python3.8 mange.py createsuperuser
{superuser_name}
{superuser_email}
{superuser_password}

Start the server on port 8000:

python3.8 manage.py runserver 8000

Administrating the database

To use the Django Admin interface, open a web browser to this URL:

http://localhost:8000/admin

Log in using the superuser name and password.

You can use the Django Admin interface to create users, so can operate both the backend and frontend web applications, without the need to configure the email account parameters. In this case, the username and password created in the Admin interface can be used to Log In from the front-end web site. However, the functionality for requesting a reset password email will not be operable.

Sample Deployment Site

The front-end web site is deployed at:

[https://csv-manager.stefanzero.com](CSV Manager)

and the backend is deployed at:

[https://csvbackend.stefanzero.com/](Account Management Portal)

Details

Running the Django Application

The main entry point of a Django application is the manage.py in the root project directory. The manage.py file is a very small Python program that specifies the file that contains all the project settings. The manage.py file accepts command line parameters ,such as the "runserver" parameter described previously.

When a Django application is deployed from a web server such as Apache, the httpd.conf file specifies the location of the wsgi.py (web server gateway interface) file. This Apache directive must be in the httpd.conf file.

WSGIScriptAlias / {path_to_wsgi_file)

Like the manage.py file, the wsgi.py file also specifies the path to the settings.py file. Typically, the httpd.conf file also contains numerous settings particular to the server's Apache configuration. This Django application uses "Token" based authentication, where the HTTP header must contain an AUTHORIZATION Token on each rest to verify that the user is Logged in. The httpd.conf file needs this directive to pass the authorization token to Django:

WSGIPassAuthorization On

User Management Summary

  • New Account: enter username, email, and password. A confirmation email which contains a link to activate your account.
  • Log In: enter username and password
  • Log Out
  • Change Password: enter old password and new password
  • Reset Password: enter email and a email is sent with a link to reset your password

User management is handled directly on this back-end web site. If you are not currently logged in, the navigation bar displays links to New Account, Reset Password, and Log In. If you are currently logged in, the navigation bar displays links to Change Password and Log Out.

Custom Django "Apps"

Adding custom functionality to a Django application is done through creating custom apps. Here, there are 2 custom apps: "config" and "fileupload".

config

The "config" app controls the basic operations of the web application.

  • settings.py
  • admin.py (admin config for user_unique_email)
  • urls.py (custom URL routes)
  • routers.py (interface between rest_framework URLs and viewsets)
  • models.py (custom post_save function to add a row to the TOKEN table when a new user is saved)
  • wsgi.py (Python interface to Apache)
  • views.py (custom functions such as requesting username in an email)
  • context_processors.py (add some settings properties to rendering context)
  • templates (customized rendering templates using Bootstrap CSS framework)
  • template_tags (custom function to return object properties)

fileupload

The "fileupload" app contains the rest_framework implementation for API that allows uploading of CSV files.

Data Model

The models.py contains the class that specifies the properties (columns) stored in the database for each file uploaded:

  • id (primary key)
  • file name
  • date created
  • owner id (user that uploaded the file)

Viewset

The viewset.py contains the viewset classes specifies the operation of the API routes, via these properties:

  • queryset (specification of the database select operation)
  • serializer (conversion between the queryset and values returned by the API)
  • permission_classes (only authenticated users can upload files)
  • authentication_classes (Session authentication is used by the web-browerable API, and Token authentication is used by the REST API)

Serializer

The rest_framework API requires a "serializer" class, that specifies the fields that are returned. Here, the file name is used as a handle to the actual file object, which allows the API to return additional properties than those stored in the database. The serializer returns these properties:

  • file_id
  • file name
  • since_added (file creation date)
  • filetype (file extension)

The serializer also specifies a "validate" method, that is used during file upload to checked if the uploaded file meets these criteria:

  • the filetype must be "csv"
  • the file has a non-zero file size
  • all lines in the file have the same number of fields (same number of commas)

If the validate method fails, the API returns an HTTP status code of 400 and an error message. The front-end application will display the message returned.

If the validate method succeeds, the file is stored to the "media" directory and a row is added to the database. If a file of the same name already exists in the "media" directory, the file stored is given an additional random 6-letter portion, and this renamed file is returned by the API in the HTTP 200 returned response.