AWS ElasticBeanstalk: Tips and Tricks

AWS-ElasticBeanstalk — PaaS on the infrastructure of AWS. In my opinion a significant advantage of this service is the ability to directly access elements of the infrastructure (load balancers, instances, queues, etc.) In this article I decided to gather some tricks that solve common problems when using ElasticBeanstalk. Will complement as you acquire more. Questions and suggestions in the comments are welcome.



the

adding application configuration


In my opinion an obvious disadvantage of the platform is vague storage engine configuration. So I used the following methods to add a configuration.

The most obvious and native to ElasticBeanstalk — set via environment variables. Inside these instance variables are not available as usual, but only in the environment of the application. For these settings the most convenient to use the command eb setenv from a batch awsebcli that is used to deploy the application (suitable for small projects) or AWS APIs.

the 
eb setenv RDS_PORT=5432 PYTHONPATH=/opt/python/current/app/myapp:$PYTHONPATH RDS_PASSWORD=12345 DJANGO_SETTINGS_MODULE=myapp.settings RDS_USERNAME=dbuser RDS_DB_NAME=appdb RDS_HOSTNAME=dbcluster.us-east-1.rds.amazonaws.com

The second option is when the config will injected inside created application version. For this we need to explain how to be deployed. Manually or by a script, create a zip archive containing application code is laid out on a special S3 bucket unique to each region ( elasticbeanstalk-<region_name>-<my_account_id> — do not attempt to use the other will not work — tested). You can create this package manually, or edit software. I prefer to use an alternative deployment option, when instead awsebcli used custom code to create the package version.



The third option — uploading configuration remotely to the deployment phase of the external database configuration. IMHO the most correct approach, but beyond the scope of this article. I use a scheme with the storage configuration on S3 and proxy requests to S3 through the API Gateway — this allows the most flexibility to manage configs. S3 also supports versioning.

the

Included tasks in crontab


ElasticBeanstalk supports the creation of jobs for the scheduler using cron.yaml. However, this config only works for worker environment — the configuration used to process a queue of tasks / periodic tasks. For this task, surrounded by a WebServer added to the project directory .ebextensions file with the following content:

the 
files:
"/etc/cron.d/cron_job":
mode: "000644"
owner: root
group: root
content: |
#Add comands below
15 10 * * * root curl www.google.com >/dev/null 2>&1<code>

"/usr/local/bin/cron_job.sh":
mode: "000755"
owner: root
group: root
content: |
#!/bin/bash
/usr/local/bin/test_cron.sh || exit
echo "running Cron at" `date` >> /tmp/cron_job.log
# Now do tasks that should only run on 1 instance ...

"/usr/local/bin/test_cron.sh":
mode: "000755"
owner: root
group: root
content: |
#!/bin/bash

METADATA=/opt/aws/bin/ec2-metadata
INSTANCE_ID=`$METADATA -i | awk '{print $2}"
REGION=`$METADATA-z | awk '{print substr($2, 0, length($2)-1)}"

# Find our Auto Scaling Group name.
ASG=`aws ec2 describe-tags --filters "Name=resource-id,Values=$INSTANCE_ID" \
--region $REGION --output text | awk '/aws:autoscaling:groupName/ {print $5}"

# Find the first instance in the Group
FIRST=`aws autoscaling describe-auto-scaling-groups --auto-scaling-group-names $ASG \
--region $REGION --output text | awk '/InService$/ {print $4}' | sort | head -1`

# Test if they're the same.
[ "$FIRST" = "$INSTANCE_ID" ]

commands:
rm_old_cron:
command: "rm *.bak"
cwd: "/etc/cron.d"
ignoreErrors: true

the

the Automatic application of migrations to Django and build of static when you deploy


Added to the config file to .ebextensions:

the 
container_commands:
01_migrate:
command: "python manage.py migrate --noinput"
leader_only: true
02_collectstatic:
command: "./manage.py collectstatic --noinput"

Similarly apply alembic migration; in order to avoid the use of migrations for each instance autoscaling group, you specify the parameter leader_only
the

Using hooks when deploying applications


Creating scripts in the directory /opt/elasticbeanstalk/hooks/, it is possible to add various control scripts, in particular, to modify the deployment process of the application. Scripts that are executed before deployment, are in the directory /opt/elasticbeanstalk/hooks/appdeploy/pre/*, while in /opt/elasticbeanstalk/hooks/appdeploy/enact/*, and /opt/elasticbeanstalk/hooks/appdeploy/post/*. Scripts are executed in alphabetical order so you can build the correct sequence of deployed applications.

the

adding the Celery daemon in the existing config supervisor


the 
files:
"/opt/elasticbeanstalk/hooks/appdeploy/post/run_supervised_celeryd.sh":
mode: "000755"
owner: root
group: root
content: |
#!/usr/bin/env bash

# Get the django environment variables
celeryenv=`cat /opt/python/current/env | tr '\n' ',' | sed 's/export //g' | sed 's/$PATH/%(ENV_PATH)s/g' | sed 's/$PYTHONPATH//g' | sed 's/$LD_LIBRARY_PATH//g"
celeryenv=${celeryenv%?}

# Create celery configuraiton script
celeryconf="[program:celeryd]
; Set full path to celery program if using virtualenv
command=/opt/python/run/venv/bin/celery worker -A yourapp -B --loglevel=INFO -s /tmp/celerybeat-schedule

directory=/opt/python/current/app
user=nobody
numprocs=1
stdout_logfile=/var/log/celery-worker.log
stderr_logfile=/var/log/celery-worker.log
autostart=true
autorestart=true
startsecs=10

; Need to wait for currently executing tasks to finish at shutdown.
; Increase this if you have very long running tasks.
stopwaitsecs = 600

; When resorting to send SIGKILL to the program to terminate it
; send SIGKILL to its whole process group instead,
; taking care of its children as well.
killasgroup=true

; if rabbitmq is supervised, set its priority higher
; so it starts first
priority=998

environment=$celeryenv"

# Create the celery supervisord conf script
echo "$celeryconf" | tee /opt/python/etc/celery.conf

# Add configuration script to supervisord conf (if not there already)
if ! grep -Fxq "[include]" /opt/python/etc/supervisord.conf
then
echo "[include]" | tee-a /opt/python/etc/supervisord.conf
echo "files: celery.conf" | tee-a /opt/python/etc/supervisord.conf
fi

# Reread the supervisord config
supervisorctl -c /opt/python/etc/supervisord.conf reread

# Update cache in supervisord without restarting all services
supervisorctl -c /opt/python/etc/supervisord.conf update

# Start/Restart celeryd through supervisord
supervisorctl -c /opt/python/etc/supervisord.conf restart celeryd

By the way, I used an experimental option to take as the broker for Celery SQS and it is quite justified; however, flower does not yet have support for such schemes.

the

Automatically redirect HTTP to HTTPS


Here is a Supplement to the ElasticBeanstalk configuration inside Apache

the 
files:
"/etc/httpd/conf.d/ssl_rewrite.conf":
mode: "000644"
owner: root
group: root
content: |
RewriteEngine On
<If "-n '%{HTTP:X-Forwarded-Proto}' && %{HTTP:X-Forwarded-Proto} != 'https'">
RewriteRule (.*) https://%{HTTP_HOST}%{REQUEST_URI} [R,L]
</If>

the

Use of multiple SSL domains


Amazon provides an opportunity for registrants to use free SSL certificates, including wildcard, but only within the AWS. To use multiple domains with SSL on one environment get the certificate using the AWS Certificate Manager to add the ELB balancer and SSL configured on it. Can be used and obtained from another vendor's certificates.



UPDATE Below the comments dear darken99 gave a couple of useful chips will add them here with some explanations

turn Off environment schedule

In this case, depending on the specified time range the number of instances in the autoscaling group is reduced from 1 to 0.

the 

option_settings:
- namespace: aws:autoscaling:scheduledaction
resource_name: Start
option_name: MinSize 
value: 1
- namespace: aws:autoscaling:scheduledaction
resource_name: Start
option_name: MaxSize
value: 1
- namespace: aws:autoscaling:scheduledaction
resource_name: Start
option_name: DesiredCapacity
value: 1
- namespace: aws:autoscaling:scheduledaction
resource_name: Start
option_name: Recurrence
value: "0 9 * * 1-5"
- namespace: aws:autoscaling:scheduledaction
resource_name: Stop
option_name: MinSize
value: 0
- namespace: aws:autoscaling:scheduledaction
resource_name: Stop
option_name: MaxSize
value: 0
- namespace: aws:autoscaling:scheduledaction
resource_name: Stop
option_name: DesiredCapacity
value: 0
- namespace: aws:autoscaling:scheduledaction
resource_name: Stop
option_name: Recurrence
value: "0 18 * * 1-5"


Replacing Apache for Nginx
the 
option_settings:
aws:elasticbeanstalk:environment:proxy:
ProxyServer: nginx
doesn't work for Python
Article based on information from habrahabr.ru

Комментарии

Отправить комментарий

Популярные сообщения из этого блога

Automatically create Liquibase migrations for PostgreSQL

Изображение
Working with Liquibase always brings confidence in the fact that the database objects are in the same condition in which you want it. It is this confidence and makes system migrations so popular databases. Today I want to tell you how to make using liquibase a bit easier. If you write a migration by hand, and your target database is PostgreSQL, then read this article, I am sure, the time spent on reading the article will be repaid with interest. I hope you already know how to work with Liquibase, so only two words describe how the system works. In order to make changes in the database, you need to create a migration file (changeset) that you need to specify in the file change log (changelog), after which the migration can be successfully applied to the target database. Undeniable and obvious benefit of this approach is the ability to rollback the changes. In my opinion, liquibase, there is only one drawback — the automatic rollback of changes only works only if the mi...
Далее...

Looking for books as you want

Изображение
this last week or two, as curious users can watch Russian search service on books from Google . However, the General public before the official announcement from Google itself remains in the dark. Meanwhile, already there is something to see and something to tell. Besides, Google has not gone unnoticed in Yandex, which last week began offering users with Store cards and the output presented there books. What, in turn, did not fail to say publicly. Let's take a closer look at what is offered book lovers the giants of the search. Let's start with Google. The California company offers us their traditional approach. Typing in the familiar search bar the title of the book, we get... well, what we will get, greatly depends on many factors. Mostly while where a book has been published. If it's a popular publication in the United States (as an example from reference service — exposing the social ills of America's best-selling book " Fast Food Nation "), the...
Далее...

Vkontakte sync with address book for iPhone. How it was done

Изображение
the Background application on androidphone when you install the application Vkontakte is possible to use it as a provider of contacts to synchronize the contacts from Facebook with contacts in your address book. In this case stretches the avatar, date of birth, address, phone numbers, etc. When I after prolonged use HTC Wildfire S rated and are so used to this that he wanted to find a similar to the iPhone. Not found, so we took it and made his own — Synchronizer VKontakte :) It was back in November of last year. The application was done fairly quickly — it uses quite a few functions of Vkontakte API and functionality with the address book of the iPhone. But when we started to deal with the automatic matching of contacts, it was fun. Compare head-on by name, last name and phone number was easy to do, easy, and worked through it quickly. But found only 20 percent of coincidences of the contacts from what is provider in Android. Of course, only direct hits do not sui...
Далее...