SPS Website Architecture Overview

This page will attempt to provide a high level overview of how the SPS website works.

HTTP Serving

Much of the HTTP serving for the SPS website is handled by OCF infrastructure. For our purposes, we can treat requests as arriving to an Apache service and responses leaving from the Apache service, but in reality there is another layer of routing between the Apache service and the client.

Requests to /static or /wiki will match the names of folders stored in public_html, so when Apache receives such a request, it loads the relevant file and sends it. The wiki contains some php scripts. Apache executes these when loading them, but we do not need to concern ourselves with this. For other requests, Apache consults the .htaccess file stored in public_html.

The first section of .htaccess contains instructions for redirecting links to run.fcgi. The second part, containing lines beginning with Redirect, instructs Apache to redirect certain urls. This is done for compatbility with old versions of the website and for link shortening. run.fcgi is a Python script (though not named as such) that sends requests to the Django application installed at ~/sps_web_2020.

Response Generation With Django

Our Django application is responsible for generating responses to HTTP requests for anything that is not a static file in /static or part of the Wiki. Django does a lot of things behind the scenes, but we only need to worry about which of our functions it calls when receiving a request.

When Django receives a response, it consults the list urlpatterns stored in urls.py to decide which function to call to process the request. The function path or re_path takes a path (or a regular expression) as its first argument, the function to call as its second argument, and optionally a name (useful for debugging). For a path, the function to call should take only a single argument, the HTTP request. For a re_path, the function can also take a captured string (the part of the HTTP path in parantheses). In principle, this could also be a regular expression, but this would require more sanitization in functions, so we try to avoid this. If one of the paths or re_paths match, the corresponding function will be called. If there are no matches, handler404 (defined at the bottom of the file) will be called to generate the 404 error page.

Functions that take in HTTP requests and return HTTP responses are referred to in Django terminology as "views". load_static_page is our most commonly used example of one of these functions. These functions combine templates stored in sps_web_2020/templates, static HTML from sps_web_2020/static_html, and data in the database to generate complete HTML pages, which are then returned as the response. The render function is frequently used. This is a Django function that renders a template using a dictionary of information.

Data Loading

Most of the information on the website is added by the committee to the static HTML files, but some frequently updated pages automatically pull data from other sources. For ease of use, data is loaded from various Google services. This also provides authentication and authorization (which are unpleasant to deal with). The sps_web_sync repository is responsible for loading data from Google into the database.

/events is automatically loaded from the ucbsps@gmail.com Google calendar using the download_calendar.py script.

/potw is automatically loaded from the POTW spreadsheet (linked to a Google form) using the download_potw.py script.