Linux

首页Linux
08
Dec
0

kafka安装 直接复制官网

原始地址:http://kafka.apache.org/quickstart

tar -xzf kafka_2.11-2.1.0.tgz

cd kafka_2.11-2.1.0

bin/zookeeper-server-start.sh config/zookeeper.properties

bin/kafka-server-start.sh config/server.properties

创建主题
bin/kafka-topics.sh --create --zookeeper localhost:2181 --replication-factor 1 --partitions 1 --topic test

查看主题
bin/kafka-topics.sh --list --zookeeper localhost:2181

创建一个producter
bin/kafka-console-producer.sh --broker-list localhost:9092 --topic test

启动一个consumer
bin/kafka-console-consumer.sh --bootstrap-server localhost:9092 --topic test –from-beginning

设置集群
cp config/server.properties config/server-1.properties
    broker.id=1
    listeners=PLAINTEXT://:9093
    log.dirs=/tmp/kafka-logs-1

cp config/server.properties config/server-2.properties

broker.id=2

    listeners=PLAINTEXT://:9094
    log.dirs=/tmp/kafka-logs-2

启动两个新节点
bin/kafka-server-start.sh config/server-1.properties &
bin/kafka-server-start.sh config/server-2.properties &

创建一个分区为3的TOPIC
bin/kafka-topics.sh --create --zookeeper localhost:2181 --replication-factor 3 --partitions 1 --topic my-replicated-topic

查看分区状态
bin/kafka-topics.sh --describe --zookeeper localhost:2181 --topic my-replicated-topic
bin/kafka-topics.sh --describe --zookeeper localhost:2181 --topic test

28
Jun
0

fastdfs安装配置

相关包的下载网址

FastDFS_v5.08.tar.gz

https://sourceforge.net/projects/fastdfs/files/

所有用得上的包下面这个链接都有

https://github.com/happyfish100

安装FastDFS依赖包libfastcommon

wget https://codeload.github.com/happyfish100/libfastcommon/zip/master
unzip master
cd libfastcommon-master/
./make.sh
./make.sh install

安装FastDFS

wget http://jaist.dl.sourceforge.net/project/fastdfs/FastDFS%20Server%20Source%20Code/FastDFS%20Server%20with%20PHP%20Extension%20Source%20Code%20V5.08/FastDFS_v5.08.tar.gz
tar xf FastDFS_v5.08.tar.gz
cd FastDFS
./make.sh && ./make.sh install

服务脚本位置

ll /etc/init.d/ |grep fdfs

样例配置文件

ll /etc/fdfs/

命令行工具

ll /usr/bin|grep fdfs
  1. 虽然FastDFS区分tracker和storage服务器,但是安装的软件及步骤均相同,只是不同的配置文件而已,因此以上安装适用tracker server和storage server

配置跟踪服务器(tracker server)

cp /etc/fdfs/tracker.conf.sample /etc/fdfs/tracker.conf
cp /etc/fdfs/client.conf.sample /etc/fdfs/client.conf

编辑tracker server配置文件tracker.conf

disabled=false(默认为false,表示是否无效)
port=22122(默认为22122)
base_path=/data/fastdfs/tracker

编辑client端的配置文件client.conf

base_path=/data/fastdfs/tracker
tracker_server=172.18.10.232:22122

创建tracker server数据目录

mkdir -p /data/fastdfs/tracker

启动tracker server,启动成功会自动在/data/fastdfs/tracker目录新建data和logs目录

/etc/init.d/fdfs_trackerd start
cd /data/fastdfs/tracker/
ls #自动生成了 data  logs 两个文件夹

编辑storage server配置文件storage.conf

cp /etc/fdfs/storage.conf.sample /etc/fdfs/storage.conf

disabled=false(默认为false,表示是否无效)
port=23000(默认为23000)
base_path=/data/fastdfs/storage
tracker_server=172.18.10.232:22122
store_path0=/data/fastdfs/storage
http.server_port=8888(默认为8888,nginx中配置的监听端口那之一致)

创建storage server数据目录

mkdir -p /data/fastdfs/storage

试启动storage server,启动成功会自动在/data/fastdfs/tracker目录新建data和logs目录(启动storage server的前提是tracker server必须事先已启动)

/etc/init.d/fdfs_storaged start
cd /data/fastdfs/storage/
ls #自动生成了 data  logs 两个文件夹

文件上传测试

/usr/bin/fdfs_upload_file /etc/fdfs/client.conf /download/test.jpg
返回
group1/M00/00/00/rBIK6VcaP0aARXXvAAHrUgHEviQ394.jpg
success ok!!
21
Jun
0

做负载均衡时遇到的多机session不正确(时间同步)

做负载均衡时遇到的多机session不正确

原因:
两个机器时间相差太大,导致session一个过期

处理:
用时间服务器同步 方法如下

同步时间服务器命令

/usr/sbin/ntpdate 10.86.1.76 &> /dev/null
开启定时同步

crontab -e

/5 * /usr/sbin/ntpdate 10.86.1.76 &> /dev/null

21
Jun
0

expect 脚本示例

最近老是在服务器上更新代码并执行 太繁琐 写个脚本玩一下

在凡事都会遇到坑的思想指导下

所以就遇到了坑:

连续执行以下两句语句,第二句就无法执行起来
spawn uwsgi /geelyapp/itil-python/uwsgi/uwsgi.ini
spawn uwsgi /geelyapp/itil-python-test/uwsgi/uwsgi.ini
或者
exec sh -c {uwsgi /geelyapp/itil-python/uwsgi/uwsgi.ini}
exec sh -c {uwsgi /geelyapp/itil-python-test/uwsgi/uwsgi.ini}
原因 应该是有输出
前者
可以用expect {eof}
试试 看看能不屏闭 (还未试过 应该可行)
后者是只要有输出,就一定会停止,暂时没有找到相关处理办法 expect脚本似乎没有异常捕获之类的东西

安装:
yum install expect

可执行:
chmod +x autosu.sh

脚本:

!/bin/expect

set timeout 60

stty -echo

cd /geelyapp/itil-python/itil-python/

spawn git checkout .

spawn git pull

expect {

"*sername*" {
             send "huang.xiao\n" ; exp_continue
            }

"*Password*" {
             send "AA123456\n"; exp_continue
             }
eof
  }

cd /geelyapp/itil-python-test/itil-python/

spawn git checkout .

spawn git pull

expect {

"*sername*" {
             send "huang.xiao\n" ; exp_continue
            }
"*Password*" {
             send "AA123456\n"; exp_continue
             }
eof
  }

spawn cd /geelyapp/itil-python-test/itil-python/Itil/

spawn rm -f /geelyapp/itil-python-test/itil-python/Itil/settings_production.py

spawn mv /geelyapp/itil-python-test/itil-python/Itil/settings_uwsgi_test.py /geelyapp/itil-python-test/itil-python/Itil/settings_production.py

exec sh -c {pgrep uwsgi | xargs kill -9}

spawn uwsgi /geelyapp/itil-python/uwsgi/uwsgi.ini

exec sh -c {nohup uwsgi "/geelyapp/itil-python/uwsgi/uwsgi.ini"}

exec sh -c {uwsgi /geelyapp/itil-python-test/uwsgi/uwsgi.ini}

15
Jun
0

nginx基本配置(测试用)

server {

listen    9007;
server_name  localhost;

location / {
    root   html/app-cloud/dev;
    index  index.html index.htm;
}

location ~ ^/(anon|doc|service|tenant|system) {   #这些开头的都反向代理过去
    proxy_pass http://10.86.130.32:19300;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header Host  $host;
}

}

//////////////////////////////////////////////////////////

upstream geely-sso-passport {

server 10.86.130.32:14150 weight=50;

}

server {

listen 80;
server_name geely-sso-passport.dev.app-cloud.geely.com;
    charset utf-8;
index index.html;

#前端页面负载
location ~ .*\.(html|htm|gif|jpg|jpeg|bmp|png|ico|txt|js|css|swf|ttf|woff2|eot|woff|json)$ {
    root /home/www/geely-sso-passport;
    expires      1d;
}

#首页转发到前端
location = / {
    root /home/www/geely-sso-passport;
    expires 1d;
}

#后端反向代理
location / {
proxy_pass http://geely-sso-passport;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header Host  $host;
}

}

//限制上传文件大小 (在http节点)
client_max_body_size 20M;

微信图片_20180811101707.png