基于网,分享项目的组网架构和部署。
项目组网架构
架构说明
1.流
项目访问分为两个流,通过nginx分两个端口暴露给外部使用:
数据流:
用户访问网站。控制流:
管理人员通过supervisor监控、管理服务器进程。 图中除了将程序部署在ECS上外,还使用了OSS(对象存储服务,可以理解成一个nosql数据库),主要是为了存放一些静态文件,提高访问速度。阿里的OSS还可以与CDN一起使用,同样可以提高访问速度。
2.nginx
通过nginx对外暴露两个端口,如上所述,80端口供用户访问网站,另一个端口供管理人员使用。
80端口:
根据请求的url配置了方向代理,分别导向client(angular)和server(flask).其中server通过gunicorn部署在[socket]localhost:10000
上配置如下: server { listen 80 default_server; # set client body size to 4M (add by dh) # client_max_body_size 4M; gzip_types text/plain text/css application/json application/javascript text/xml application/xml application/xml+rss text/javascript; # product root /home/raindrop/www/client/dist; # Add index.php to the list if you are using PHP index index.html index.htm index.nginx-debian.html; server_name _; location / { # First attempt to serve request as file, then # as directory, then fall back to displaying a 404. try_files $uri $uri/ =404; } # access flask static folder location /static/ { # product root /home/raindrop/www/server/app; } location /api/ { proxy_pass http://localhost:10000/api/; proxy_redirect off; proxy_set_header Host $host; proxy_set_header X-Real-IP $remote_addr; proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; } # Error pages error_page 413 @413_json; location @413_json { default_type application/json; return 200 '{"msg": "Request Entity Too Large(max=4M)"}'; }}
3.gunicorn
gunicorn作为wsgi容器,用来执行flask server。
gunicorn可以使用异步socket:gevent,其本质是基于greenlet实现协程的第三方库,改善io阻塞问题,通过简单的配置就能使程序获得极高的并发处理能力。注意:使用gunicorn != 使用gevent,如要开启gevent socket,启动gunicorn时需要增加--work-class参数,如下:
gunicorn --workers=4 --worker-class socketio.sgunicorn.GeventSocketIOWorker -b localhost:10000 wsgi:app
除了gunicorn,也可以选用uwsgi,但是有两点限制需要注意:
1.如果使用了Flask-socketio,请不要使用uwsgi,Note regarding uWSGI: While this server has support for gevent and WebSocket, there is no way to use the custom event loop needed by gevent-socketio, so there is no directly available method for hosting Flask-SocketIO applications on it.If you figure out how to do this please let me know!
2.如果使用异步WSGI Server,请勿使用uwsgi,原因:
Flask(Werkzeug)的Local Thread无法与uwsgi基于uGreen的微线程兼容,引起Local Thread工作混乱。
4.celery
在程序运行过程中会有一些比较耗时但并非紧急的工作,这些任务可以使用异步任务来处理,提高server的响应速度。
是一个第三方库,提供异步队列和任务处理功能。使用celery配合实现feed功能。通过flask进行配置,使用redis作为异步队列来存储任务,并将处理结果(feed activity)存储在redis中。5.supervisor
如上所述,我们需要在服务器上运行gunicorn和celery两个进程。
显然,我们需要一个monitor来帮我们管理这两个进程,避免进程崩溃不能及时拉起,阻塞业务。是一个开源monitor程序,并且内置了web管理功能,可以远程监控、重启进程。配置如下:[inet_http_server]# web 管理端口,通过nginx暴露给远端用户port=127.0.0.1:51000[program:raindrop]# 程序启动命令command = gunicorn --workers=4 --worker-class socketio.sgunicorn.GeventSocketIOWorker -b localhost:10000 wsgi:appdirectory = /home/raindrop/www/serveruser = dh stopwaitsecs=60stdout_logfile = /var/log/raindrop/supervisor-raindrop.logredirect_stderr = true[program:celery]command = celery -P gevent -A wsgi.celery workerdirectory = /home/raindrop/www/serveruser = dh stopwaitsecs=60stdout_logfile = /var/log/raindrop/supervisor-celery.logredirect_stderr = true
6.数据库
mysql:
网站主要数据存储在关系型数据库中
redis:
缓存mysql数据 + celery异步队列 服务器
使用阿里云的服务器
ECS:
部署服务端程序OSS:
存储前端静态文件(速度很快,对前端体验改善很大,采用angular框架强烈推荐使用) 组件
nginx:
HTTP Server
angularjs:
client框架flask:
server框架gunicorn:
web 容器celery:
异步任务处理supervisor:
monitorredis:
数据缓存 + 任务队列mysql:
数据库 本地开发环境与工具
ubuntu12.04 64:
操作系统
virtualenv
: python虚拟环境(隔离项目开发环境,不用担心包冲突了)vagrant
: 基于virtualbox的本地虚拟环境(环境可以导入导出,团队开发必备)gulp
: angular工程打包工具pip
: python包管理工具fabric
: 基于python的远程脚本(自动化部署神器) 打包部署
基于Ubuntu 12.0.4,其它系统安装命令(apt-get 等)请自行修改。
繁琐VS自动化
如果是第一次部署,需要初始化ECS服务器,安装基本工具:
nginx, supervisor, redis, mysql, pip, virtualenv
打包项目代码
发布静态文件到OSS服务器
发布项目代码到ECS服务器,安装server依赖的包
修改
mysql, nginx, supervisor
配置文件拉起所需进程
然后各种apt-get install, scp, tar, cp, mv
,不得不说,这是一个烦人且毫无技术含量的工作,干过几次后基本就可以摔键盘了。
fab init_envfab buildfab deploy
这要感谢项目,封装了非常简洁的远程操作命令。
Fabric
使用Fabric,只需要编写一个fabile.py脚本,在启动定义init_env, build, deploy
三个任务:
# -*- coding: utf-8 -*-import os, re, hashlibfrom termcolor import coloredfrom datetime import datetimefrom fabric.api import *from fabric.contrib.files import existsclass FabricException(Exception): passenv.abort_exception = FabricException# 服务器地址,可以有多个,依次部署:env.hosts = [ 'user@120.1.1.1']env.passwords = { 'user@120.1.1.1:22':'123456'}# sudo用户为root:env.sudo_user = 'root'# mysqldb_user = 'root'db_password = '123456'_TAR_FILE = 'raindrop.tar.gz'_REMOTE_TMP_DIR = '/tmp'_REMOTE_BASE_DIR = '/home/raindrop'_ALIYUN_OSS = { 'endpoint' : 'oss-cn-qingdao.aliyuncs.com', 'bucket' : 'yourbucketname', 'accessKeyId' : 'youraccessKeyId' , 'accessKeySecret': 'youraccessKeySecret'}def build(): ''' 必须先打包编译client,再打包整个项目 ''' with lcd(os.path.join(os.path.abspath('.'), 'client')): local('gulp build') # 上传静态文件到oss服务器,并修改index.html中对静态文件的引用 with lcd(os.path.join(os.path.abspath('.'), 'client/dist')): with lcd('scripts'): for file in _list_dir('./'): if oss_put_object_from_file(file, local('pwd', capture=True) + '/' + file): _cdnify('../index.html', file) with lcd('styles'): for file in _list_dir('./'): if oss_put_object_from_file(file, local('pwd', capture=True) + '/' + file): _cdnify('../index.html', file) # 注意在oss上配置跨域规则,否则fonts文件无法加载 # !!修改fonts文件夹请放开此段程序!! # with lcd('fonts'): # for file in _list_dir('./'): # oss_put_object_from_file('fonts/%s' % file, local('pwd', capture=True) + '/' + file) with lcd(os.path.join(os.path.abspath('.'), 'server')): local('pip freeze > requirements/common.txt') excludes = ['oss', 'distribute'] [local('sed -i -r -e \'/^.*' + exclude + '.*$/d\' "requirements/common.txt"') for exclude in excludes] local('python setup.py sdist')# e.g command: fab deploy:'',Fasle# 注意命令两个参数间不要加空格def deploy(archive='', needPut='True'): if archive is '': filename = '%s.tar.gz' % local('python setup.py --fullname', capture=True).strip() archive = 'dist/%s' % filename else: filename = archive.split('/')[-1] tmp_tar = '%s/%s' % (_REMOTE_TMP_DIR, filename) if eval(needPut): # 删除已有的tar文件: run('rm -f %s' % tmp_tar) # 上传新的tar文件: put(archive, _REMOTE_TMP_DIR) # 创建新目录: newdir = 'raindrop-%s' % datetime.now().strftime('%y-%m-%d_%H.%M.%S') with cd(_REMOTE_BASE_DIR): sudo('mkdir %s' % newdir) # 重置项目软链接: with cd(_REMOTE_BASE_DIR): # 解压到新目录: with cd(newdir): sudo('tar -xzvf %s --strip-components=1' % tmp_tar) # 保存上传文件 if exists('www/server/app/static/upload/images/', use_sudo=True): sudo('cp www/server/app/static/upload/images/ %s/server/app/static/upload/ -r' % newdir) sudo('rm -f www') sudo('ln -s %s www' % newdir) with cd(_REMOTE_BASE_DIR): with prefix('source %s/env/local/bin/activate' % _REMOTE_BASE_DIR): sudo('pip install -r www/server/requirements/common.txt') # 启动服务 with cd('www'): # mysql sudo('cp etc/my.cnf /etc/mysql/') sudo('restart mysql') # monitor sudo('cp etc/rd_super.conf /etc/supervisor/conf.d/') sudo('supervisorctl stop celery') sudo('supervisorctl stop raindrop') sudo('supervisorctl reload') sudo('supervisorctl start celery') sudo('supervisorctl start raindrop') # nginx sudo('cp etc/rd_nginx.conf /etc/nginx/sites-available/') # ln -f —-如果要建立的链接名已经存在,则删除之 sudo('ln -sf /etc/nginx/sites-available/rd_nginx.conf /etc/nginx/sites-enabled/default') sudo('nginx -s reload')def init_env(): sudo('aptitude update') sudo('aptitude safe-upgrade') # sudo('apt-get install nginx') sudo('aptitude install python-software-properties') sudo('add-apt-repository ppa:nginx/stable') sudo('aptitude update') sudo('apt-get install nginx') sudo('apt-get install supervisor') sudo('apt-get install redis-server') sudo('apt-get install mysql-server') sudo('apt-get install python-pip python-dev build-essential') sudo('pip install virtualenv') run('mkdir /var/log/raindrop -p') with cd ('/home/raindrop'): sudo('virtualenv env')def oss_put_object_from_file(key, file_path): from oss.oss_api import * oss = OssAPI(_ALIYUN_OSS['endpoint'], _ALIYUN_OSS['accessKeyId'], _ALIYUN_OSS['accessKeySecret']) res = oss.put_object_from_file(_ALIYUN_OSS['bucket'], key, file_path) return res.status == 200 and True or Falsedef _expand_path(path): print path return '"$(echo %s)"' % pathdef sed(filename, before, after, limit='', backup='.bak', flags=''): # Characters to be escaped in both for char in "/'": before = before.replace(char, r'\%s' % char) after = after.replace(char, r'\%s' % char) # Characters to be escaped in replacement only (they're useful in regexen # in the 'before' part) for char in "()": after = after.replace(char, r'\%s' % char) if limit: limit = r'/%s/ ' % limit context = { 'script': r"'%ss/%s/%s/%sg'" % (limit, before, after, flags), 'filename': _expand_path(filename), 'backup': backup } # Test the OS because of differences between sed versions with hide('running', 'stdout'): platform = local("uname") if platform in ('NetBSD', 'OpenBSD', 'QNX'): # Attempt to protect against failures/collisions hasher = hashlib.sha1() hasher.update(env.host_string) hasher.update(filename) context['tmp'] = "/tmp/%s" % hasher.hexdigest() # Use temp file to work around lack of -i expr = r"""cp -p %(filename)s %(tmp)s \&& sed -r -e %(script)s %(filename)s > %(tmp)s \&& cp -p %(filename)s %(filename)s%(backup)s \&& mv %(tmp)s %(filename)s""" else: context['extended_regex'] = '-E' if platform == 'Darwin' else '-r' expr = r"sed -i%(backup)s %(extended_regex)s -e %(script)s %(filename)s" command = expr % context return local(command)def _cdnify(index_file, cdn_file): sed(index_file, '([^<]*(src|href)=")[^<]*' + cdn_file + '"', '\\1http://' + _ALIYUN_OSS['bucket'] + '.' + _ALIYUN_OSS['endpoint'] + '/' + cdn_file + '"')def _list_dir(dir=None, access=lcd, excute=local): """docstring for list_dir""" if dir is None: return [] with access(dir): string = excute("for i in *; do echo $i; done", capture=True) files = string.replace("\r","").split("\n") return files
通过fabric自动化部署有两点需要注意:
1.安装mysql时,设置的密码不能生效,需要登到服务器上手动设置一下:mysql -u rootuse mysql;update user set password=PASSWORD('123456') where User='root';flush privileges;quit;
2.服务器上的用户需要自己手动创建。
gulp
在build任务中,使用gulp build打包client的angular代码。
这里先不多说,有需要请参考github项目setup
是python自己的打包工具。
build任务中,最后使用python setup.py sdist命令把整个工程打包成一个tar.gz文件,上传到服务器。使用这个工具需要编写如下两个文件:setup.py:
打包脚本 #!/usr/bin/env pythonfrom setuptools import setup, find_packagesimport servertry: long_description = open('README.md').read()except: long_description = server.__description__REQUIREMENTS = []exclude_lib = ['oss', 'distribute']for lib in open("server/requirements/common.txt").readlines(): for exclude in exclude_lib: if lib.lower() not in exclude: REQUIREMENTS.append(lib)setup( name='raindrop', url='https://www.yudianer.com', version=server.__version__, author=server.__author__, author_email=server.__email__, description=server.__description__, long_description=long_description, license=server.__license__, packages=find_packages(), zip_safe=False, platforms='any', install_requires=REQUIREMENTS)
MANIFEST.in:
指定打包哪些文件夹
recursive-include etc *recursive-include client/dist *recursive-include server/app *recursive-include server/requirements *recursive-exclude server *.pycprune server/app/static/upload/imagesprune server/env prune server/tests
本文由网码农撰写。欢迎转载,但请注明出处。