#AWS  #Hosting  #DevOps 

Deploy Django app on AWS

Deploy and host a django application on AWS EC2 instance using Nginx as the webserver and Gunicorn as WSGI

Nginx is an open-source web server that, since its success as a web server, is now also used as a reverse proxy , HTTP cache, and load balancer.

WSGI used to forward request from a web server to a python backend.

Please Refer: HTTP request-response lifecycle

1. EC2 Setup

  • Create/Launch EC2 instance: /tags/ec2/
  • Enable ssh to server - to connect to EC2 server
  • EC2 > security group > Edit inbound rules > Allow SSH

Connect to EC2 instance

# Connect
ssh -i "name.pem" ubuntu@host
  • Update OS
sudo apt-get update && sudo apt-get -y upgrade

2. Add SSH key to server

  • Copy ssh from local - id_rsa
  • Create file and paste the id_rsa content
vim ~/.ssh/id_rsa

# Paste the content, save and exit
ctrl + V
ESC + :wq

# Set very restrictive permissions
chmod 400 ~/.ssh/id_rsa

3. Clone Repo and Create Env

  • Clone repo
git clone [email protected]:fds/daadas-be.git
  • Setup server
sudo apt install python3-pip
pip install setuptools
  • Create virtual-env
# Install
sudo apt install python3-virtualenv

# Create
virtualenv .venv

# Activate
source ~/.venv/bin/activate
  • Install Requirements file
pip install -r req.txt

Issue

  • No module named ‘pkg_resources’
pip install --upgrade setuptools

# OR
# sudo apt install python3-setuptools -y

# psycopg2 --> installation issue
pip install psycopg2-binary

sudo apt install libpq-dev

# Python.h issue
sudo apt install libpython3.6-dev

4. Get setup file and .env file

  • Add local_setup.py / server_setup.py to server (For Dev Server)
# Copy content of local_setup.py

vim userman/management/commands/local_setup.py

# Paste the content, save and exit
ctrl + V
ESC + :wq
  • Add .env file
# Copy content of .env

vim bin/.env

# Paste the content, save and exit
ctrl + V
ESC + :wq

5. Setup Database

5.1 Install Postgresql and configure new role and grant all permissions

sudo apt-get install postgresql

sudo /etc/init.d/postgresql start
sudo -u postgres psql

# DROP DATABASE gqrdb;

CREATE DATABASE gqrdb;
CREATE ROLE gqrdbuser WITH LOGIN PASSWORD 'gqrdbpassword';
GRANT ALL PRIVILEGES ON DATABASE gqrdb TO gqrdbuser;
\q

IF ISSUE

  • psycopg2.errors.InsufficientPrivilege: permission denied for schema public
sudo -u postgres psql

\c gqrdb

GRANT ALL ON SCHEMA public TO gqrdbuser;
ALTER SCHEMA public OWNER TO gqrdbuser;

5.2 Install Mysql and add new user with all the privileges (OR)

sudo apt-get update
sudo apt-get install mysql-server   (Remember the password entered for root user)
mysql_secure_installation
mysql -u root -p

mysql> CREATE DATABASE testdb;
mysql> CREATE USER 'testuser'@'localhost' IDENTIFIED BY 'password';
mysql> GRANT ALL PRIVILEGES ON * . * TO 'testuser'@'localhost';
mysql> \q

sudo apt-get install python-mysqldb
mysql -u testuser -p	(Now you are logged in as testuser user)

6. Test your server

python manage.py migrate

Django has inbuilt webserver, it is used in development server but not in production, because

  • Single-threaded server`
  • Can serve only one request at a time

Test - Is the application working properly

  • Update Security group - For test allow all traffic
    • EC2 > security group > Edit inbound rules > Map: All traffic to Anywhere
python manage.py runserver 0.0.0.0:8000
  • Visit
    • http://public_ip:8000/
    • http://public_ip:8000/admin

By using inbuilt webserver there will be no issue to access static file

Issue-1

Getting this error ou’re accessing the development server over HTTPS, but it only supports HTTP.

  • You manually typed https:// in the address bar.
  • Your browser is forcing HTTPS (HSTS) because you previously ran the server with SECURE_SSL_REDIRECT = True (e.g., using production settings), and the browser remembered it. - try opening the link in an Incognito Window.

For security, it’s best to run your application under a non-root user

  • Check users
# check current user
who

# List all users
ls /home/
  • Create a user like ubuntu if not available
# Replace myuser with a username of your choice
sudo adduser ubuntu

# Give the user sudo access (optional but useful)
sudo usermod -aG sudo ubuntu

# Switch to the new user
su - ubuntu

8. Setup Gunicorn

https://www.digitalocean.com/community/tutorials/how-to-set-up-django-with-postgres-nginx-and-gunicorn-on-ubuntu-16-04 

pip install gunicorn

8.1 Testing Gunicorn’s Ability to Serve the Project

gunicorn --bind 0.0.0.0:8000 kharchapani.wsgi
Visit:
  http://public_ip:8000/

Static files are directly not accessible for --bind 0.0.0.0:8000. To Enable add below line to access

# seting.py 
STATIC_ROOT = os.path.join(BASE_DIR, 'static')

# urls.py
from django.conf.urls.static import static
urlpatterns += static(settings.STATIC_URL, document_root=settings.STATIC_ROOT)

8.2 Setup gunicorn to run as a service

  • Create a Gunicorn systemd Service File
# sudo vim /etc/systemd/system/growthqr.service
# Add below line in 'growthqr.service' file

[Unit]
Description=growthqr daemon
After=network.target

[Service]
User=ubuntu
Group=www-data
WorkingDirectory=/home/ubuntu/growthqr
ExecStart=/home/ubuntu/.venv/bin/gunicorn --workers 3 --bind 0.0.0.0:8000 growthqr.wsgi:application

[Install]
WantedBy=multi-user.target
  • Start the server and check the status
sudo systemctl start gunicorn
sudo service gunicorn status
  • Enable the server to automatically start at boot
sudo systemctl enable gunicorn
  • Try
Visit:
  http://public_ip:8000/

9. Setup Nginx

sudo apt-get install nginx

Update ExecStart of gunicorn.service

# sudo vim /etc/systemd/system/gunicorn.service

ExecStart=/home/ubuntu/.virtualenvs/testvenv/bin/gunicorn --access-logfile - --workers 3 --bind unix:/home/ubuntu/Deployment-demo/myproject.sock myproject.wsgi:application

Configure Nginx to Proxy Pass to Gunicorn

# sudo vim /etc/nginx/sites-available/kharchapani

server {
    listen 80;
    server_name public_ip_address;

    location = /favicon.ico { access_log off; log_not_found off; }
    location /static/ {
        root /home/ubuntu/kharchapani;
    }

    location / {
        include proxy_params;
        proxy_pass http://unix:/home/ubuntu/kharchapani/kharchapani.sock;
    }
}

Enable the file by linking it to the sites-enabled directory:

sudo ln -s /etc/nginx/sites-available/kharchapani /etc/nginx/sites-enabled
  • Visit:
    • http://public_ip
    • NOT - https://public_ip

Issue

connect() to unix:/home/sammy/myproject/myproject.sock failed (2: No such file or directory)

Turn DEBUG=True - this may show you actual error

connect() to unix:/home/sammy/myproject/myproject.sock failed (13: Permission denied)

This indicates that - NGINX does not have permission to access your Django/Gunicorn socket file.

  • namei -nom /home/sammy/myproject/myproject.sock
  • chmod 755
  • After fix - restart demon and service

10. Domain Mapping

  • Add in DNS
A    domain_name     ip_address
  • Update nginx setting
# sudo vim /etc/nginx/sites-available/kharchapani
server_name replace_this_with_domain_name;

11. Commands

Nginx

sudo nginx -t
sudo systemctl restart nginx

sudo tail -F /var/log/nginx/access.log
sudo tail -f /var/log/nginx/error.log

Gunicorn / Service

  • Restart If you make changes to the /etc/systemd/system/gunicorn.service file, reload the daemon to reread the service definition and restart the Gunicorn process by typing
sudo systemctl daemon-reload
sudo systemctl restart gunicorn
  • Check gunicorn process logs by typing
# from start
sudo journalctl -u gunicorn

# live logs

sudo journalctl -f -u gunicorn

12. Production Security group

  • Update Security group - Only HTTP, HTTPS and SSH traffic
    • EC2 > security group > Edit inbound rules > Map: All traffic to Anywhere

Reference