Differences

This shows you the differences between two versions of the page.

Link to this comparison view

public:gsoc:amazon_aws_deployment [2016/08/02 20:02] (current)
Line 1: Line 1:
 +===== Amazon AWS deployment ======
  
 +This is a guide for deploying, configuring and administrating CC Repository on Amazon Web Service.
 +
 +===== Server Architecture =====
 +
 +First, let's consider server architecture. I suggest to use the approach based only on EC2, RDS and ELB services provided by Amazon. ​
 +
 +Server architecture includes the following components:
 +  - **MySQL server.** This server is based on Amazon RDS instance and is used for storing MySQL database and accessing it from other servers
 +  - **NFS server.** This EC2 instance is used for sharing file system across other servers and also for running some supplementary programs.
 +  - **Web servers.** These are EC2 instances responsible only for running Apache i.e. only for web access. They  are joined in autoscaling group which can start/​terminate instances depending on chosen policies.
 +  - **Repository servers.** These are EC2 instances responsible only for running repository application (ccr) and receiving traffic from ccextractor (i.e. tuners) clients. They are also joined in autoscaling group.
 +  - **Web severs load balancer.** This is Amazon ELB that receives connections on 80 port and distributing them across web servers. Its DNS name end-clients would use to access web site from their browsers.
 +  - **Repository servers load balancer.** This Amazon ELB is used for handling connections from ccextractor applications and distributing them across repository servers.
 +
 +===== Installation =====
 +==== Creating Security Groups ====
 +
 +We’ll need to create following [[http://​docs.aws.amazon.com/​AWSEC2/​latest/​UserGuide/​using-network-security.html|security groups]] with specified inbound rules:
 +
 +=== cc-tuners-lb ===
 +Security group for repository servers load balancer. For each server with tuners you'll need to create an inbound rule with its IP as a source.
 +
 +^ Type         ^ Ports  ^ Source ​        ^
 +| Custom TCP    | 3030  | <​servers_IPs> ​ |
 +
 +=== cc-web-lb ===
 +Security group for web servers load balancer. This group grants access from any IP to web pages.
 +
 +^ Type         ^ Ports  ^ Source ​       ^
 +| HTTP         | 80     | 0.0.0.0/​0 ​    |
 +
 +=== cc-slave ===
 +Security group for EC2 instances in both autoscaling groups. It opens ports for load balancer and NFS server. ​ SSH is added for administration purposes and can be removed later. To create rules for NFS you will have create cc-master group first.
 +
 +^ Type       ^ Ports   ^ Source ​                  ^
 +| All TCP    | 0-65535 | <​cc-load-balancer sg-id> |
 +| Custom TCP | 111     | <​cc-master sg-id> ​       |
 +| Custom TCP | 2049    | <​cc-master sg-id> ​       |
 +| Custom UDP | 111     | <​cc-master sg-id> ​       |
 +| Custom UDP | 32806   | <​cc-master sg-id> ​       |
 +| SSH        | 22      | "My Ip" ​                 |
 +
 +=== cc-master ===
 +Security group for NFS server.
 +
 +^ Type       ^ Ports     ^ Source ​            ^
 +| Custom TCP | 111       | <​cc-slave sg-id> ​  |
 +| Custom TCP | 2049      | <​cc-slave sg-id> ​  |
 +| Custom UDP | 111       | <​cc-slaver sg-id> ​ |
 +| Custom UDP | 32806     | <​cc-slaver sg-id> ​ |
 +| SSH        | 22        | "My Ip" ​           |
 +
 +=== cc-db ===
 +Security group for RDS MySQL database instance.
 +
 +^ Type         ^ Ports ^ Source ​            ^
 +| MYSQL/​Auror ​ | 3306  | <​cc-slave sg-id> ​  |
 +| MYSQL/​Aurora | 3306  | "My Ip" ​           |
 +
 +==== Creating Database ====
 +  - [[http://​docs.aws.amazon.com/​AmazonRDS/​latest/​UserGuide/​CHAP_GettingStarted.CreatingConnecting.MySQL.html|Launch RDS MySQL instance]] with //cc-db// security group.
 +  - Now you need to create database tables by calling <​code>​
 +mysql --user=root --password=root-password --host=...rds.amazonaws.com cc < misc/​tables.sql
 +</​code>​
 +  - Then edit the following lines in //​misc/​users.sql//​ to set passwords for two users //cc_ro// and //cc_rw// that will be used to connect form PHP scripts and from server application.<​code>​
 +SET @ro_passwd = '​read-only-user-password';​
 +SET @rw_passwd = '​read-write-user-password';​
 +</​code>​
 +  - Execute this SQL file by calling<​code>​
 +mysql --user=root --password=root-password --host=...rds.amazonaws.com cc < misc/​users.sql
 +</​code>​
 +==== Configuring NFS server ====
 +
 +  - [[http://​docs.aws.amazon.com/​AWSEC2/​latest/​UserGuide/​ec2-launch-instance_linux.html|Launch EC2 Instance]] based on Amazon Linux AMI with //​cc-master//​ security group and connect to it.
 +  - [[http://​docs.aws.amazon.com/​AWSEC2/​latest/​UserGuide/​ec2-launch-instance_linux.html|Create empty volume]] in the same region as this instance.
 +  - Attach this volume to the instance. Let's say it's ///​dev/​xvdf//​
 +  - Create file system on this volume: <​code>​sudo mkfs -t ext4 /​dev/​xvdf</​code>​
 +  - Add following line to the end of ///​etc/​fstab//​ file. It make system mount //xvdf// volume to ///srv// directory at server startup. <​code>/​dev/​xvdf /srv ext4 defaults,​nofail 0 0</​code>​
 +  - Mount this volume <​code>​sudo mount -a</​code>​
 +  - Create repository directories for storing logs, temporary files and files with captions: <​code>​sudo mkdir /srv/logs /srv/web /​srv/​archive</​code>​
 +  - Install git and clone source files to your home directory:<​code>​
 +sudo yum install git
 +git clone https://​github.com/​rkuchumov/​ccextractor-server
 +</​code>​
 +  - From //misc// directory install //​ccr-watchdog//​ – a service that closes connections if they are no longer alive. It's not really important, but sometimes magic happens and connections doesn'​t close gracefully.<​code>​
 +cd ccextractor-server/​misc
 +sudo ./​install-watchdog.sh
 +</​code>​
 +  - The command above will place watchdog script to ///​etc/​init.d///​ and configuration files to ///​etc/​ccr///​. You'll need to place your database host, user with read-write access (cc_rw) and its password in ///​etc/​ccr/​db.ini//​
 +  - Now you need to start this service and make it run at each system boot<​code>​
 +sudo service ccr-watchdog start
 +sudo chkconfig ccr-watchdog on
 +</​code> ​
 +  - Next steps will set up NFS server itself. First, you need to install //​nfs-utils//​ and //​rpcbind//<​code>​
 +yum install nfs-utils rpcbind
 +</​code>​
 +  - Now you need to specify which directories to share and options for them. Create ///​etc/​exports//​ file with the following content:<​code>​
 +/​srv/​archive *(rw,​async,​no_root_squash,​no_acl)
 +/​srv/​web ​    ​*(rw,​sync,​no_root_squash,​no_acl)
 +/​srv/​logs ​   *(rw,​sync,​no_root_squash,​no_acl)
 +</​code>​
 +  - Now you need to load the changes by calling<​code>​sudo exportfs -ar</​code>​
 +  - The following commands will start NFS service and related services and make them run on each system startup<​code>​
 +sudo service rpcbind start
 +sudo service nfs start
 +sudo service nfslock start
 +sudo chkconfig rpcbind on
 +sudo chkconfig nfslock on
 +sduo chkconfig nfs on
 +</​code>​
 +  - It's probably a good idea to create an image of this instance in EC2 console and turn on termination protection.
 +
 +==== Configuring repository servers ====
 +  - [[http://​docs.aws.amazon.com/​AWSEC2/​latest/​UserGuide/​ec2-launch-instance_linux.html|Launch EC2 Instance]] based on Amazon Linux AMI with //​cc-slave//​ and //​cc-tuners//​ security groups and connect to it.
 +  - First, we'll set up NFS client. You'll need to install //​nfs-utils//​ and //rpcbind// by calling<​code>​
 +sudo yum install nfs-utils rpcbind
 +</​code>​
 +  - Start and enable NFS services:<​code>​
 +sudo service rpcbind start
 +sudo service nfslock start
 +sudo chkconfig rpcbind on
 +sudo chkconfig nfslock on
 +</​code>​
 +  - Add the following lines to the end of ///​etc/​fstab//​. It'll make the system mount these directories at each system boot. //​nfs_server//​ is public DNS of your NFS server. You can find in EC2 console. It's something like //​ec2-52-26-9-221.us-west-2.compute.amazonaws.com//<​code>​
 +nfs_server:/​srv/​web ​      /​srv/​web ​      ​nfs ​  ​rw,​noac,​noacl,​nocto,​nodiratime ​   0   0
 +nfs_server:/​srv/​logs ​     /​srv/​logs ​     nfs   ​rw,​noac,​noacl,​nocto,​nodiratime ​   0   0
 +nfs_server:/​srv/​archive ​  /​srv/​archive ​  ​nfs ​  ​rw,​async,​noacl,​nocto,​nodiratime ​  ​0 ​  0
 +</​code>​
 +  - To mount directories listed above call<​code>​sudo mount -a</​code>​
 +  - Next, we'll install repository application. First, you'll need to install Git and addition packages for compilation<​code>​
 +sudo yum install git gcc mysql-devel
 +</​code>​
 +  - Download CCExtractor source code, compile and install it. The last command shall place //​ccextractor//​ binary to ///​usr/​bin///​ directory<​code>​
 +https://​github.com/​CCExtractor/​ccextractor ~
 +cd ~/​ccextractor
 +make
 +sudo make install
 +</​code>​
 +  - Download repository application source, compile and install it.<​code>​
 +git clone https://​github.com/​rkuchumov/​ccextractor-server
 +cd ccextractor-server
 +sudo make install
 +</​code>​
 +  - Now you need to specify database host, user with read-write access (cc_rw) and its password in ///​etc/​ccr/​db.ini//​ configuration file.
 +  - Start and enable repository service by calling:<​code>​
 +sudo service ccr start
 +sudo chkconfig ccr on
 +</​code>​
 +  - Finally, create instance image from EC2 console
 +
 +==== Configuring web servers ====
 +  - [[http://​docs.aws.amazon.com/​AWSEC2/​latest/​UserGuide/​ec2-launch-instance_linux.html|Launch EC2 Instance]] based on Amazon Linux AMI with //​cc-slave//​ and //cc-web// security groups and connect to it.
 +  - First, we'll set up NFS client. You'll need to install //​nfs-utils//​ and //rpcbind// by calling<​code>​
 +sudo yum install nfs-utils rpcbind
 +</​code>​
 +  - Start and enable NFS services:<​code>​
 +sudo service rpcbind start
 +sudo service nfslock start
 +sudo chkconfig rpcbind on
 +sudo chkconfig nfslock on
 +</​code>​
 +  - Add the following lines to the end of ///​etc/​fstab//​. It'll make the system mount these directories at each system boot. //​nfs_server//​ is public DNS of your NFS server. You can find in EC2 console. It's something like //​ec2-52-26-9-221.us-west-2.compute.amazonaws.com//<​code>​
 +nfs_server:/​srv/​web ​      /​srv/​web ​      ​nfs ​  ​ro,​noac,​noacl,​nocto,​nodiratime ​   0   0
 +nfs_server:/​srv/​logs ​     /​srv/​logs ​     nfs   ​rw,​noac,​noacl,​nocto,​nodiratime ​   0   0
 +nfs_server:/​srv/​archive ​  /​srv/​archive ​  ​nfs ​  ​ro,​noac,​noacl,​nocto,​nodiratime ​   0   0
 +</​code>​
 +  - To mount directories listed above call<​code>​sudo mount -a</​code>​
 +  - Next, we'll install repository application,​ Apache, PHP and MySQL client. <​code>​
 +sudo yum install git httpd24 php56 php56-mysqlnd
 +git clone https://​github.com/​rkuchumov/​ccextractor-server
 +cd ccextractor-server/​web/​
 +sudo ./​install.sh
 +</​code>​
 +   - The last command will copy web pages to ///​var/​www///,​ configure php.ini, and copy Apache configuration file to ///​etc/​httpd/​conf.d///​. You'll need to edit this file and set database host, read-only user (cc_ro) and its password in environment variables:<​code>​
 +SetEnv CC_DB_HOST localhost
 +SetEnv CC_DB_NAME cc
 +SetEnv CC_DB_USER cc_ro
 +SetEnv CC_DB_PASSWORD read-only-user-password
 +</​code>​
 +   - Uncomment //ErrorLog /​srv/​logs/​apache.log//​ line to redirect Apache error messages in /srv/logs/
 +   - Set //​DocumnetRoot//​ to ///​var/​www/​public///​ in ///​etc/​httpd/​conf/​httpd.conf//​.
 +   - Start and enable Apache server by calling <​code>​sudo service httpd start
 +sudo chkconfig httpd on</​code>​
 +   - Create instance image from EC2 Console.
 +
 +==== Creating Load Balancers ====
 +  - Follow [[http://​docs.aws.amazon.com/​AutoScaling/​latest/​DeveloperGuide/​as-register-lbs-with-asg.html| this guide]] to create launch configuration,​ load balancers and auto scaling groups for web servers.
 +      - Launch configuration should contain web server instance image and //​cc-slave//​ and //cc-web// security groups.
 +      - Load balancer should accept connections on 80 TCP port and have //​cc-web-lb//​ security group.
 +  - Repeat the same procedure for repository servers
 +      - Launch configuration should contain repository server instance image and //​cc-slave//​ and //​cc-tuners//​ security groups.
 +      - Load balancer should accept connections on 3030 TCP port and have //​cc-tuners-lb//​ security group.
 +
 +===== Administration Guidelines =====
 +  * One connected CCExtractor client takes about 250-300 Mb of RAM. Thus, if your repository instance has 1GB of RAM, it can successfully handle 3 connected tuners.
 +  * .....
 +  * ????
 +  * TODO 
  • public/gsoc/amazon_aws_deployment.txt
  • Last modified: 2016/08/02 20:02
  • (external edit)