註意:WEB伺服器和資料庫需要分離,同時WEB伺服器也需要編譯安裝MySQL。 做集群架構的重要思想就是找到主幹,從主幹區域向外延展。 WEB伺服器: apache nginx 本地做三個產品 dedecms workprocess discuz 將用戶存放圖片,附件的目錄掛載到nfs伺服器的共用目 ...
註意:WEB伺服器和資料庫需要分離,同時WEB伺服器也需要編譯安裝MySQL。
做集群架構的重要思想就是找到主幹,從主幹區域向外延展。
WEB伺服器: apache nginx 本地做三個產品 dedecms workprocess discuz 將用戶存放圖片,附件的目錄掛載到nfs伺服器的共用目錄上
NFS伺服器 本地做三個共用目錄,實現將用戶上傳的圖片及附件分別存放到對應的目錄上,
使用sersync與備份伺服器實現實時同步,批量分發密鑰及腳本,hosts文件(實際生產環境下,在同一區域網下,hosts文件通常保持一致),
MySQL伺服器:用於用戶存放數據的伺服器,
Backup伺服器:用於備份的伺服器,防止其他伺服器宕機、感染病毒、等等數據丟失。同時要將每天備份的內容通過郵件發送給管理員,確保數據備份成功。
我的主幹思想就是先配置LAMP和LNMP伺服器,之後向外延展配置nfs伺服器及MySQL伺服器,然後將所有需要備份的數據打包好,配置backu備份伺服器,最後做nginx負載均衡伺服器,如果有精力又有能力的情況下,繼續延伸一個nginx的高可用(提示ngixn高可用服務使用的是VRRP技術)
1.LAMP(192.168.190.20)
(1)tar xvf /apache-2.2.27 cd apache-2.2.27 編譯安裝 ./configure \ --prefix=/application/appache2.2.27 \ 安裝目錄 --enable-deflate \ 壓縮安裝 --enable-expires \ 過期 緩存時間 --enable-headers \ --enable-modules=most \ 模塊激活 --enable-so \ --with-mpm=worker \ apache的兩種模式:worker,prefork --enable-rewrite && make &&make install (註意編譯環境下換行後邊不能存在空格,上述編譯添加了註釋,如果粘貼請自行刪除,手打忽略) ln -s /application/apache-2.2.27 /application/apache echo “<html> <head><title> a ,s blog. </title><head> <body> Hi,i'm a ,My blog address is <a href="" targe=_parent > </a> </ body> </html>” > /application/apache/htdos/index.html /application/apache/bin/apachectl graceful
瀏覽器輸入192.168.190.20 會出現
Hi,i'm a ,My blog address is 等字樣說明apache服務安裝成功
(2)安裝資料庫msyql
解壓編譯安裝 ,編譯過程略長,安裝結束後進行檢查做軟鏈接
創建MySQL虛擬用戶和用戶組
groupadd mysql cat /etc/group useradd -g mysql -M -s /sbin/nologin mysql id mysql 編譯安裝MySQL ./configure \ --prefix=/application/mysql5.1.72 \ --with-unix-socket-path=/application/mysql5.1.72/tmp/mysql.sock \ --localstatedir=/application/mysql5.1.72/data \ --enable-assembler \ --enable-thread-safe-client \ --with-mysqld-user=mysql \ --with-big-tables \ --without-debug \ --with-pthread \ --enable-assembler \ --with-extra-charsets=complex \ --with-readline \ --with-ssl \ --with-embedded-server \ --enable-local-infile \ --with-plugins=partition,innobase \ --with-mysqld-ldflags=-all-static \ --with-client-ldflags=-all-static #--with-plugin-PLUGIN \ make && make install
echo $? 檢查編譯是否成功
ln -s /application/mysql5.1.72/ /application/mysql 創建軟鏈接
複製配置mysql的配置文件
cd mysql-5.1.72/support-files/ ls cp -p my-small.cnf /etc/my.cnf chown -R mysql.mysql /application/mysql 授權MySQL用戶管理許可權 初始化mysql /application/mysql/bin/mysql_install_db --basedir=/application/mysql --datadir=/application/mysql/data/ --user=mysql ##兩個OK即為初始化成功 /application/mysql/bin/mysqld_safe & 啟動mysql netstat -lntup|grep mysqld ##查看MySQL服務是否啟動成功 mysqladmin -u root password '123456' ##設置MySQL用戶密碼
(3)安裝完apache和mysql之後再安裝PHP(註意php配合apache是以模塊的方式存在)
yum install -y openssl-devel tar -xvf php-5.3.27.tar.gz cd php-5.3.27.tar.gz ./configure \ --prefix=/application/php5.3.27 \ //註意php的安裝目錄 --with-apxs2=/application/apache/bin/apxs \ //註意apache的安裝目錄 --with-mysql=/application/mysql \ --with-xmlrpc \ --with-openssl \ --with-zlib \ --with-freetype-dir \ --with-gd \ --with-jpeg-dir \ --with-png-dir \ --with-iconv=/usr/local/libiconv \ --enable-short-tags \ --enable-sockets \ --enable-zend-multibyte \ --enable-soap \ --enable-mbstring \ --enable-static \ --enable-gd-native-ttf \ --with-curl \ --with-xsl \ --enable-ftp \ --with-libxml-dir && make && make install
ln -s /application/php5.3.27/ /application/php ##做軟鏈接去版本號 cp /application/apache/conf/httpd.conf /application/apache/conf/httpd.conf.bak.1 vim /application/apache/conf/httpd.conf #修改主配置文件 cd /application/apache/conf diff httpd.conf httpd.conf.bak.1 67,68c67,68 < User www < Group www --- > User daemon > Group daemon 149c149 < DirectoryIndex index.php index.html --- > DirectoryIndex index.html 292,294c292 < AddType application/x-httpd-php .php .phtml < AddType application/x-httpd-php-source .phps --- > 401c401 < Include conf/extra/httpd-vhosts.conf --- ># Include conf/extra/httpd-vhosts.conf: 423 <Directory "/data0/www"> 424 Options -Indexes FollowSymLinks 425 AllowOverride None 426 Order allow,deny 427 Allow from all 428 </Directory>
創建對應的apache的虛擬用戶www
useradd www -s /sbin/nologin -M
id www
cd /application/apache/conf/extra
vim httpd-vhosts.conf
NameVirtualHost *:80 # # VirtualHost example: # Almost any Apache directive may go into a VirtualHost container. # The first VirtualHost section is used for all requests that do not # match a ServerName or ServerAlias in any <VirtualHost> block. # <VirtualHost *:80> ServerAdmin [email protected] DocumentRoot "/data0/www/cms" ServerName cms.etiantian.org ServerAlias etiantian.org ErrorLog "logs/dummy-host.example.com-error_log" CustomLog "|/usr/local/sbin/cronolog /app/logs/access_cms_%Y%m%d.log" combined </VirtualHost> <VirtualHost *:800> ServerAdmin [email protected] DocumentRoot "/data0/www/bbs" ServerName bbs.etiantian.org ErrorLog "logs/dummy-host.example.com-error_log" CustomLog "|/usr/local/sbin/cronolog /app/logs/access_bbs_%Y%m%d.log" combined </VirtualHost> <VirtualHost *:8000> ServerAdmin [email protected] DocumentRoot "/data0/www/blog" ServerName blog.etiantian.org ErrorLog "logs/dummy-host.example.com-error_log" CustomLog "|/usr/local/sbin/cronolog /app/logs/access_blog_%Y%m%d.log" combined </VirtualHost>
如果有錯誤則查看錯誤日誌
cd /application/appache/logs/
建立站點目錄
mkdir /data0/{www,blog,bbs}
for n in www blog bbs ;do echo "$n.etiantian.org" > /data0/$n/index.html;done
/application/apache/bin/apachectl -t #檢查配置文件的語法
/application/apache/bin/apachectl graceful #平滑重啟apache
本地做hosts解析 訪問三個網站 ,查看基於功能變數名稱的虛擬主機是否配置成功
Dedecms,Discuz,workprocess三個產品解壓之後自行安裝(安裝完MySQL資料庫再進行安裝)
2.LNMP(192.168.190.10)
(1)安裝nginx服務
編譯nginx前 需安裝
1.pcre pcre-devel
yum install -y pcre pcre-devel
2.openssl
yum install openssl openssl-devel -y
編譯安裝nginx
./configure --user=nginx --group=nginx --prefix=/application/nginx1.6.2 --with-http_stub_status_module --with-http_ssl_module make && make install ln -s /application/nginx1.6.2/ /application/nginx
啟動nginx
/application/nginx/sbin/nginx
lsof -I :80 #查看nginx服務是否啟動成功
(2)安裝MySQL資料庫
編譯安裝MySQL
./configure --prefix=/application/mysql5.1.72 --with-unix-socket-path=/application/mysql5.1.72/tmp/mysql.sock --localstatedir=/application/mysql5.1.72/data --enable-assembler
--enable-thread-safe-client --with-mysqld-user=mysql --with-big-tables --without-debug --with-pthread --enable-assembler --with-extra-charsets=complex --with-readline
--with-ssl --with-embedded-server --enable-local-infile --with-plugins=partition,innobase --with-mysqld-ldflags=-all-static --with-client-ldflags=-all-static make && make install
echo $?
資料庫初始化:
chown -R mysql.mysql /application/mysql 授權MySQL用戶管理許可權
初始化mysql
/application/mysql/bin/mysql_install_db --basedir=/application/mysql --datadir=/application/mysql/data/ --user=mysql
//初始化顯示兩個OK 即為初始化成功
(3)安裝PHP(PHP配合nginx是以守護進程的方式存在工作的)
(安裝php之前需要安裝所需的包 yum install zlib libxml libjpeg freetype libpng gd curl libiconv zlib-devel libxml2-devel libjpeg-devel freetype-devel libpng-devel gd-devel curl-devel -y)
tar zxf libiconv-1.14.tar.gz cd libiconv-1.14 ./configure --prefix=/usr/local/libiconv //然後進行編譯安裝 make && make install
安裝相關依賴
Libmcrypt 、 mhash 、mcrypt
安裝php(編譯之前首先安裝libxslt* ,否則會報錯) 解壓
./configure --prefix=/application/php5.3.27 --with-mysql=/application/mysql --with-iconv-dir=/usr/local/libiconv --with-freetype-dir --with-jpeg-dir --with-png-dir --with-zlib
--with-libxml-dir=/usr --enable-xml --disable-rpath --enable-safe-mode --enable-bcmath --enable-shmop --enable-sysvsem --enable-inline-optimization --with-curl
--with-curlwrappers --enable-mbregex --enable-fpm --enable-mbstring --with-mcrypt --with-gd --enable-gd-native-ttf --with-openssl --with-mhash --enable-sockets --with-xmlrpc
--enable-zip --enable-soap --enable-short-tags --enable-zend-multibyte --enable-static --with-xsl --with-fpm-user=nginx --with-fpm-group=nginx --enable-ftp && make && make install
上傳修改好的啟動文件php-frm.conf(軟體包組裡面有修改完整的) à上傳到/application/php/etc
創建日誌文件
mkdir /app/logs //不創建的話檢查語法的時候會報錯
/application/php/sbin/php-fpm -t //檢查語法
/application/php/sbin/php-fpm //啟動php
在rc.local里設置開機自啟動
Mysql php nginx
##優化配置文件
cat /application/nginx/conf/ cat nginx.conf worker_processes 1; events { worker_connections 1024; } http { include mime.types; default_type application/octet-stream; sendfile on; keepalive_timeout 65; include extra/cms.conf; include extra/bbs.conf; include extra/blog.conf; } mdkir extra touch {bbs,blog,cms}.conf cat cms.conf server { listen 80; server_name cms.etiantian.org; root /data0/www/cms; index index.php index.html index.htm; location ~ \.(php|php5)?$ { fastcgi_pass 127.0.0.1:9000; fastcgi_index index.php; include fastcgi.conf; } } cat bbs.conf server { listen 800; server_name bbs.etiantian.org; root /data0/www/bbs; index index.php index.html index.htm; location ~ \.(php|php5)?$ { fastcgi_pass 127.0.0.1:9000; fastcgi_index index.php; include fastcgi.conf; } } cat blog,conf server { listen 8000; server_name blog.etiantian.org; root /data0/www/blog; index index.php index.html index.htm; location ~ \.(php|php5)?$ { fastcgi_pass 127.0.0.1:9000; fastcgi_index index.php; include fastcgi.conf; } } mkdir /data0/{www,blog,bbs} for n in www blog bbs ;do echo "$n.etiantian.org" > /data0/$n/index.html;done
Discuz,dedecms,workprocess三個產品自行安裝放入站點目錄 bbs,www,blog
3.NFS伺服器(192.168.190.30)
(1)安裝nfs和rpcbind(nfs服務是靠rpcbind轉髮端口的)
yum install nfs-utils rpcbind -y
echo “/etc/init.d/nfs restart” >/etc/rc.local //設置nfs開機自啟動使用chkconfig同樣
vim /etc/exports
#shared storage for LAMP,LNMP /data0 192.168.190.10(rw,async) 192.168.190.20(rw.async)
(2)批量分發
ssh-copy-id -i 是可以實現小環境下的密鑰分發但是如果上百台伺服器,就需要開發腳本進行自動化分發密鑰(附件里有開發好的腳本,僅供參考)
密鑰分發完畢,分發本地的hosts文件(實際生產環境下統一內網下,hosts本地解析一致是很有必要的)
vim /etc/rsync.password
123456
chmod 600 /etc/rsync.password
(3)sersync實現與備份備份伺服器的實時同步
安裝sersync
cd /tools
uzip sersync2.5.4_64bit_binary_stable_final.tar
mv sersync2 /usr/local/sersync
cd /usr/local/sersync/conf
echo ‘export PATH=$PATH:/usr/local/sersync/bin’ >>/etc/profile
vim /usr/local/sersync/conf/www_confxml.xml (需要哪個目錄rsync伺服器同步就寫哪個目錄,這裡以博客workprocess為例name代表rsync服務端的模塊名稱 ip就是目標IP地址)
24 <localpath watch="/data0/www/blog/wp-content/uploads"> 25 <remote ip="192.168.190.50" name="nfs"/> 26 </localpath>
echo ‘sersync -r -d -o /usr/local/sersync/conf/www_confxml.xml’ >> /etc/rc.local
至此sersync與備份伺服器實時同步也完成了。
4.安裝MySQL伺服器(192.168.190.40),創建你所做的產品的資料庫,創建每個產品所使用的資料庫用戶,及授權用戶。
創建MySQL虛擬用戶 useradd -g mysql -M -s /sbin/nologin mysql
(1)編譯安裝MySQL資料庫
tar xvf mysql5.1.72.tar.gz
cd mysql5.1.72
./configure --prefix=/application/mysql5.1.72 --with-unix-socket-path=/application/mysql5.1.72/tmp/mysql.sock --localstatedir=/application/mysql5.1.72/data --enable-assembler
--enable-thread-safe-client --with-mysqld-user=mysql --with-big-tables --without-debug --with-pthread --enable-assembler --with-extra-charsets=complex --with-readline
--with-ssl --with-embedded-server --enable-local-infile --with-plugins=partition,innobase --with-mysqld-ldflags=-all-static --with-client-ldflags=-all-static
ln -s /application/mysql5.1.72/ /application/mysql
/application/mysql/bin/mysql_install_db --basedir=/application/mysql --datadir=/application/mysql/data/ --user=mysql ##初始化資料庫
cd /tools/msyql5.1.71
\cp support-files/mysql.server /etc/init.d/mysqld
/etc/init.d/mysqld start
netstat -lntup |grep mysql #查看MySQL3306埠是否開啟
mysqladmin -u root password '123456' ## 設置資料庫用戶名和密碼
(2)進入資料庫創建用戶併進行授權,創建用戶所用的資料庫cms bbs blog
mysql -uroot -p123456
mysql> create database bbs; mysql> create database cms; mysql> create database blog; mysql> show databases; ##查看資料庫 mysql> grant select,insert,update,delete,alter,create on cms.* to cms@"192.168.190.%" identified by '123456'; mysql> grant select,insert,update,delete,alter,create on bbs.* to bbs@'192.168.190.%' identified by '123456'; mysql> grant select,insert,update,delete,alter,create on blog.* to blog@'192.168.190.%' identified by '123456'; mysql> select user,host from mysql.user; ##查看用戶是否創建成功
(3)MySQL資料庫數據備份,利用周期性計劃任務定時推送(簡單的備份使用mysqldump,高層的備份服務使用replication或者drbd)
[root@mysql scripts]# cat mysqldunmp.sh
!#bin/bash cd /backup echo "You are in backup dir" File = /backup/ mysqldump -uroot -p123456 --quick --databases bbs blog cms --flush-logs --single-transaction > /backup/mysql$(date +%F).bak rsync -az /backup/mysql* [email protected]::mysql --password-file=/etc/rsync.password echo "Your database backup successfully completed"
vim /etc/rsync.password
123456
chmod 600 /etc/rsync.password
crontab -e
#send mysqlbak 00 00 * * * /bin/sh -x /server/script/mysqldump.sh
5.backup伺服器(192.168.190.40 rsync ,每天檢查推送過來的備份內容,定時發送郵件告知系統管理員備份是否成功)
useradd -s /sbin/nologin rsync 創建rsync 用戶
yum install -y rsync
echo "/usr/bin/rsync --daemon" >> /etc/rc.local
vim /etc/rsyncd.conf
##rsyncd.conf start## uid = rsync gid = rsync use chroot = no max connections = 200 timeout = 300 pid file = /var/run/rsyncd.pid lock file = /var/run/rsync.lock log file = /var/log/rsyncd.log ignore errors read only = false list = false hosts allow = 10.0.0.0/24 hosts deny = 0.0.0.0/32 auth users = rsync_backup secrets file = /etc/rsync.password [zhang] path = /zhang/ [cms] path =/data0/www/cms/ [bbs] path =/data0/www/bbs/ [blog] path =/data0/www/blog/ [backup] path =/backup/ [nfs] path =/backup/nfs/ [mysql] path =/mysql/
所有的推送文件夾必須存在 ,否則rsync啟動會報錯
chown -R rsync.rsync /zhang/
chown -R rsync.rsync /data0/www/cms/
chown -R rsync.rsync /data0/www/bbs/
chown -R rsync.rsync /data0/www/blog/
chown -R rsync.rsync /backup/
chown -R rsync.rsync /backup/nfs/
chown -R rsync.rsync /mysql/
vim /etc/rsync.password
rsync_backup:123456
chmod 600 /etc/rsync.password
echo “/usr/bin/rsync --daemon” >> /etc/rc.local
vim /server/script/check.sh
ls /mysql >> /root/check.txt
ls -l /backup/192.168.190.30/|awk '{print $9}' >> /root/check.txt
egrep -v "^$| " /root/check.txt > /root/checkadd.txt
最後推送checkadd.txt文本文檔給系統管理員 管理員就可以看到都備份了什麼文件
mail -s "Hello from linuxde.net by file" [email protected] < checkadd.txt
6.主nginx負載均衡伺服器(192.168.190.23)
(1)安裝配置nginx負載均衡器
編譯nginx前 需安裝
1.pcre pcre-devel
yum install -y pcre pcre-devel
2.openssl
yum install openssl openssl-devel -y
##創建nginx的虛擬用戶
usedadd nginx -s /sbin/nologin -M
編譯安裝nginx
./configure --user=nginx --group=nginx --prefix=/application/nginx1.6.2 --with-http_stub_status_module --with-http_ssl_module make && make install ln -s /application/nginx1.6.2/ /application/nginx
##啟動nginx
/application/nginx/sbin/nginx
vim /application/nginx/conf/nginx.conf
worker_processes 1; events { worker_connections 1024; } http { include mime.types; default_type application/octet-stream; sendfile on; keepalive_timeout 65; upstream backend { ip_hash; server 192.168.190.10:80 max_fails=3 fail_timeout=30s; server 192.168.190.20:80 max_fails=3 fail_timeout=30s; } upstream backendyy { server 192.168.190.10:800 max_fails=3 fail_timeout=30s; server 192.168.190.20:800 max_fails=3 fail_timeout=30s; } upstream backendblog { ip_hash; server 192.168.190.10:8000 max_fails=3 fail_timeout=30s; server 192.168.190.20:8000 max_fails=3 fail_timeout=30s; } server { listen 80; server_name cms.etiantian.org; index index.html index.htm; location / { proxy_pass http://backend; } error_page 500 502 503 504 /50x.html; location = /50x.html { root html; } } server { listen 80; server_name bbs.etiantian.org; index index.html index.htm; location / { proxy_pass http://backendyy; } error_page 500 502 503 504 /50x.html; location = /50x.html { root html; } } server { listen 80; server_name blog.etiantian.org; index index.html index.htm; location / { proxy_pass http://backendblog; } error_page 500 502 503 504 /50x.html; location = /50x.html { root html; } } }
平滑重啟nginx :/application/nginx/bin/nginx -s reload
(2)安裝配置keepalived
yum install -y keepalived
netstat -lntup |grep keepalived
vim /etc/keepalived/keepalived.conf
global_defs { notification_email { [email protected] [email protected] [email protected] } notification_email_from [email protected] smtp_server 192.168.200.1 smtp_connect_timeout 30 router_id lb01 } vrrp_instance VI_1 { state MASTER interface eth1 virtual_router_id 55 priority 150 advert_int 1 authentication { auth_type PASS auth_pass 123456 } virtual_ipaddress { 192.168.190.23/24 dev eth1 label eth1:1 } }
/etc/init.d/keepalived restart ifconfig會發現生成了一個你想要的虛擬IP地址
7.高可用的備份nginx負載均衡伺服器(192.168.190.23)
(1)安裝配置nginx負載均衡器
編譯nginx前 需安裝
1.pcre pcre-devel
yum install -y pcre pcre-devel
2.openssl
yum install openssl openssl-devel -y
編譯安裝nginx
./configure --user=nginx --group=nginx --prefix=/application/nginx1.6.2 --with-http_stub_status_module --with-http_ssl_module make && make install ln -s /application/nginx1.6.2/ /application/nginx
##創建nginx的虛擬用戶
usedadd nginx -s /sbin/nologin -M
##啟動nginx
/application/nginx/sbin/nginx
vim /application/nginx/conf/nginx.conf
worker_processes 1; events { worker_connections 1024; } http { include mime.types; default_type application/octet-stream; sendfile on; keepalive_timeout 65; upstream backend { ip_hash; server 192.168.190.10:80 max_fails=3 fail_timeout=30s; server 192.168.190.20:80 max_fails=3 fail_timeout=30s; } upstream backendyy { server 192.168.190.10:800 max_fails=3 fail_timeout=30s; server 192.168.190.20:800 max_fails=3 fail_timeout=30s; } upstream backendblog { ip_hash; server 192.168.190.10:8000 max_fails=3 fail_timeout=30s; server 192.168.190.20:8000 max_fails=3 fail_timeout=30s; } server { listen 80; server_name cms.etiantian.org; index index.html index.htm; location / { proxy_pass http://backend; } error_page 500 502 503 504 /50x.html; location = /50x.html { root html; } } server { listen 80; server_name bbs.etiantian.org; index index.html index.htm; location / { proxy_pass http://backendyy; } error_page 500 502 503 504 /50x.html; location = /50x.html { root html; } } server { listen 80; server_name blog.etiantian.org; index index.html index.htm; location / { proxy_pass http://backendblog; } error_page 500 502 503 504 /50x.html; location = /50x.html { root html; } } }
平滑重啟nginx :/application/nginx/bin/nginx -s reload
(2)安裝keepalived
yum install -y keepalived
vim /etc/keepalived/keepalived.conf
global_defs { notification_email { [email protected] [email protected] [email protected] } notification_email_from [email protected] smtp_server 192.168.200.1 smtp_connect_timeout 30 router_id lb02 } vrrp_instance VI_1 { state BACKUP interface eth2 virtual_router_id 55 priority 100 ##優先順序 數值越高越優先 advert_int 1 authentication { auth_type PASS auth_pass 123456 } virtual_ipaddress { 192.168.190.23 dev eth2 label eth2:1 } }
/etc/init.d/keepalived restart 啟動keepalived 把主nginx負載均衡伺服器宕掉 ,會發現備keepalived節點伺服器會生成一個虛擬IP
本地做192.168.190.23的host解析
Keepalived高可用故障切換轉移原理
Keepalived高可用服務對之間的故障切換轉移,是通過VRRP來實現的。在keepalived服務工作時,主Master節點會不斷地向備節點發送(多播的方式)心跳消息,用來告訴備Backup節點自己還活著。當主節點發生故障時,就無法發送心跳的消息了,備節點也因此無法繼續檢測到來自主節點的心跳了。於是就會調用自身的接管程式,接管主節點的IP資源和服務。當主節點恢復時,備節點又會釋放主節點故障時自身接管的IP資源和服務,恢復到原來的備用角色