Rsyslog的一些特殊用法

Rsyslog的模板文件按日期存放: $template 10.161.54.11,"/var/log/rsyslog/%fromhost-ip%/netflow_%$YEAR%-%$MONTH%-%$DAY%.log" $template 10.161.50.5,"/var/log/rsyslog/%fromhost-ip%/xdns_webeng_%$YEAR%-%$MONTH%-%$DAY%.log" $template 10.161.50.7,"/var/log/rsyslog/%fromhost-ip%/xdns_webeng_%$YEAR%-%$MONTH%-%$DAY%.log" #从特定ip来的日志发到特定rsyslog服务器上去 #:fromhost-ip, !isequal, "127.0.0.1" ?Remote :fromhost-ip, isequal, "10.161.54.11" ?10.161.54.11 :fromhost-ip, isequal, "10.161.50.5" ?10.161.50.5 :fromhost-ip, isequal, "10.161.50.7" ?10.161.50.7 Rsyslog打出所有调试信息: *.* /var/log/debugfmt;RSYSLOG_DebugFormat 调试信息: FROMHOST: '172.18.18.9', fromhost-ip: '172.18.18.9', HOSTNAME: '172.18.18.9', PRI: 5, syslogtag 'time:', programname: 'time', APP-NAME: 'time', PROCID: '-', MSGID: '-', TIMESTAMP: 'Mar 4 09:04:45', STRUCTURED-DATA: '-', msg: '2021-03-04 09:04:45;danger_degree:1;breaking_sighn:0;event:[50556]MySQL登录认证成功;src_addr:172.18.5.65;src_port:57953;dst_addr:172.18.20.52;dst_port:3306;user:;smt_user:;proto:MYSQL' escaped msg: '2021-03-04 09:04:45;danger_degree:1;breaking_sighn:0;event:[50556]MySQL登录认证成功;src_addr:172.18.5.65;src_port:57953;dst_addr:172.18.20.52;dst_port:3306;user:;smt_user:;proto:MYSQL' inputname: imudp rawmsg: '<5>time:2021-03-04 09:04:45;danger_degree:1;breaking_sighn:0;event:[50556]MySQL登录认证成功;src_addr:172.18.5.65;src_port:57953;dst_addr:172.18.20.52;dst_port:3306;user:;smt_user:;proto:MYSQL' $!: $.: $/: Rsyslog的isequal,不建议,建议用== if $fromhost isequal 172.18.18.9 then /var/log/nips.log if $fromhost-ip == '172.18.18.9' then { action(type="ommysql" server="localhost" db="Syslog" uid="nips" pwd="xxxxxxxx") } Rsyslog的ommysql用法 $ModLoad ommysql *.info;mail.none;authpriv.none;cron.none :ommysql:localhost,Syslog,nips,xxxxxxxx *.info;mail.none;authpriv.none;cron.none action(type="ommysql" server="localhost" db="Syslog" uid="nips" pwd="xxxxxxxx") $template dbFormat,"insert into SystemEvents (Message, Facility,FromHost, Priority, DeviceReportedTime, ReceivedAt, InfoUnitID, SysLogTag) values ('%msg%', %syslogfacility%, '%fromhost-ip%',%syslogpriority%, '%timereported:::date-mysql%', '%timegenerated:::date-mysql%', %iut%, '%syslogtag%')",sql action(type="ommysql" server="localhost" serverport="3306" db="Syslog" uid="nips" pwd="xxxxxxxx" template="dbFormat") #172.18.31.34上的实际用法: if $fromhost-ip == '172.18.18.9' then { if $syslogpriority == 7 then { $template dbFormat1,"insert into SystemEvents (Message, Facility,FromHost, Priority, DeviceReportedTime, ReceivedAt, InfoUnitID, SysLogTag) values ('%msg%', 10, '%fromhost-ip%',%syslogpriority%, '%timereported:::date-mysql%', '%timegenerated:::date-mysql%', %iut%, 'nips')",sql action(type="ommysql" server="localhost" serverport="3306" db="Syslog" uid="nips" pwd="xxxxxxxx" template="dbFormat1") } else { $template dbFormat2,"insert into SystemEvents (Message, Facility,FromHost, Priority, DeviceReportedTime, ReceivedAt, InfoUnitID, SysLogTag) values ('time:%msg%', 10, '%fromhost-ip%',%syslogpriority%, '%timereported:::date-mysql%', '%timegenerated:::date-mysql%', %iut%, 'nips')",sql action(type="ommysql" server="localhost" serverport="3306" db="Syslog" uid="nips" pwd="xxxxxxxx" template="dbFormat2") } } & ~ Rsyslog中& ~的用法 ...

2023年12月11日

网络设备配置的备份

这个其实是网络工程师的工作,有以下两种方法: 一、用ssh备份Cisco设备的脚本 需要事先在/root/.ssh/config配置好直接登录,并且在Cisco设备里设置好权限级别,可以执行show run #!/bin/sh sshcmd="ssh -o LogLevel=quiet" $sshcmd $1 "show run" > /root/backup/$1-$(date '+%Y%m%d').txt 二、备份Cisco设备的Python脚本,这个可控性更高: 首选需要在/export/servers/python363装好python, pip install netmiko 其次,在路由器上可以配置en的密码 #!/export/servers/python363/bin/python3.6 from netmiko import Netmiko import time tw_bgp = { "device_type": "cisco_ios", "host": "tw-bgp", "ip": "192.168.1.10", "username": "noc", "use_keys": True, "secret" : "xxxxxxxx", "key_file": "/root/.ssh/id_jump_rsa_new", } tw_r1_e1 = { "device_type": "cisco_ios", "host": "tw-r1-e1", "ip": "192.168.1.11", "username": "noc", "use_keys": True, "key_file": "/root/.ssh/id_jump_rsa_new", } tw_r1_e2 = { "device_type": "cisco_ios", "host": "tw-r1-e2", "ip": "192.168.1.12", "username": "noc", "use_keys": True, "key_file": "/root/.ssh/id_jump_rsa_new", } tw_r2_e1 = { "device_type": "cisco_ios", "host": "tw-r2-e1", "ip": "192.168.1.13", "username": "noc", "use_keys": True, "key_file": "/root/.ssh/id_jump_rsa_new", } tw_r2_e2 = { "device_type": "cisco_ios", "host": "tw-r2-e2", "ip": "192.168.1.14", "username": "noc", "use_keys": True, "key_file": "/root/.ssh/id_jump_rsa_new", } devices=[tw_bgp, tw_r1_e1, tw_r1_e2, tw_r2_e1, tw_r2_e2] for dev in devices: name = dev["ip"] connection = Netmiko(**dev) connection.enable() out = connection.send_command("show running-config") calender = time.strftime("%Y%m%d") file_name = '{}-{}.txt'.format(dev["host"],calender) file = open(file_name ,"w") file.write(out) file.close() connection.disconnect() print("BACKUP for %s done" %dev["host"])

2023年12月11日

lvm卷的clone方法

用pxe远程启动一个iscsi卷的方法已经会了的话。 如果我们要批量产新虚机,最快的方法应该是把远程的iscsi卷clone一下,供新的虚机用,方法如下: 在172.18.30.18上操作 查看一下,原来有两个LV(逻辑卷) # lvdisplay 那就把pvc-vis-18-31-48的lvm虚机卷,先copy做个mirror,-b参数表示后台运行。 # lvconvert --type mirror --alloc anywhere -m1 /dev/vg-targetd/pvc-vis-18-31-48 -b Logical volume vg-targetd/pvc-vis-18-31-48 converted. 提示一下就运行完毕了,虚假啊,后台正在运行同步信息: lvs -a -o +devices | egrep "LV|48" 看那个Cpy&Sync,那个是进度条,才6.91,务必等走到100,再进行后续操作 然后破开这个mirror,把副本命名为vis-18-31-49 #lvconvert --splitmirrors 1 --name vis-18-31-49 /dev/vg-targetd/pvc-vis-18-31-48 Logical volume vg-targetd/pvc-vis-18-31-48 converted. 再运行 lvdisplay 多出一个逻辑卷pvc-vis-18-31-49,之后我们就可以用这个新卷建立iscsi对象了。 这个卷装的Linux是Centos 7 last,网卡是dhcp,可以当模板复用。

2023年12月8日

开发投诉FTP慢问题的解决

问题说明: 1、研发同事反馈应用程序上传一张图片耗时6分钟,让我们排查一下是什么问题? [INFO] 2022-03-01 17:16:38.295 [vu73wwv64ba4ncfzaiiou2qsem] [DubboServerHandler-172.18.32.254:20881-thread-2400] [FtpFile.java:854:uploadFileToPath()] uploadFileToPath path://c/product/590389/display/,fileName:display.jpg, cost:383667 ms 显示上传一张图片花了383667ms,没道理啊,这事自建的vsftp服务器,也没往oss桶里放东西的。 问题排查: 1、登录到30.18上面查看vsftpd日志信息,过滤display.jpg的信息,发现上传信息是正常的。 2、查看prometheus监控各项指标都很正常。唯有那个点的io比平时高一点,什么原因导致未知. 3、查看vsftpd的全部日志信息,发现ftp每到一个目录就执行一个list操作,由于product目录下面文件很多,所以过了6分钟才相应结果,这就是导致ftp慢的原因。 [图片] 4、找研发部门验证,终于发现ftp公共组件,代码里确实有这个list操作。 总结:开发病得不清,判断目录是否存在居然每次都刷一下所有文件,太弱智了。

2023年12月8日

Lsync 同步软件的运用

公司使用智齿的机器崩了,要做数据迁移。 由三台老机器迁移到三台新kvm机器 我们需要把其中一台34.38的的东西实时同步两份,一份到30.18的glusterfs,一份到34.41的/data 同时也要注意30.18的GFS中已经有两个虚机文件,32.6和34.38的qcow2 所以lsync务必要小心,不能删除已有文件 安装很简单: yum install epel-release yum install lsyncd 172.18.34.38上面的/etc/lsyncd.conf如下,同步到两个目的地: 注意下面的参数: maxProcesses = 2 # 本机用于rsync的进程数 delete = ‘running’ # 只删除lsync启动之后删除的文件,目的文件夹中原有的文件保存 exclude = “upload” # 第二个同步中排除的目录,注意这里是匹配全路径中的部分字串,upload 可以匹配到 /data/new/chatmsg/upload/allajl.jpg,就upload目录下文件大,所以把它排除。**这里的规则是和rsync中exclude的写法不同的!!!**只取路径中的upload字串就可以排除。 ---- -- User configuration file for lsyncd. -- -- Simple example for default rsync, but executing moves through on the target. -- -- For more examples, see /usr/share/doc/lsyncd*/examples/ -- -- sync{default.rsyncssh, source="/var/www/html", host="localhost", targetdir="/tmp/htmlcopy/"} settings { logfile = "/var/log/lsyncd/lsyncd.log", statusFile = "/var/log/lsyncd/lsyncd-status.log", statusInterval = 5, maxProcesses = 2 } sync { default.rsync, source = "/data/new", target = "172.18.30.18::new", delete = 'running', delay = 5, rsync = { binary = "/usr/bin/rsync", archive = true, compress = false, verbose = true } } sync { default.rsync, source = "/data", target = "172.18.34.41::new", delete = 'running', exclude = "upload", delay = 5, rsync = { binary = "/usr/bin/rsync", archive = true, compress = false, verbose = true } } 对端rsyncd的配置如下: ...

2023年12月8日

Linux加密压缩tar包

审计的需求,需要定期备份文件,为了安全起见,这些压缩包需要加密保存,恢复的时候需要密码才能恢复: 把目录 20231224压缩进加密包: tar -zcvf - 20231224|openssl des3 -salt -k <password> | dd of=/backup/cdrom/20231224.tar.gz.des3 解压: dd if=/backup/cdrom/20231224.tar.gz.des3 |openssl des3 -d -k <password>|tar zxf -

2023年12月8日

今年下半年的两张Azure认证

在今年微软的挑战中拿了两张免费考卷,考了AZ-104和AZ-305,捞了一张专家证书,今年的考试到头了。 明年用免费azure考试抵扣税,真是一件好事。

2023年9月21日

Azure认证az-104的考点

在今年微软的挑战中拿了两张免费考卷,一张是115$,怕浪费啊,就分别考了AZ-104和AZ-305,基本上你104能过,305就没有问题,az-104挺多知识点的,最讨厌的是步骤题,不知道死记那些步骤有何意思,把104的知识要点分列如下: ResourceGroup的 Tag不会被resource继承,新建的policy只针对新添加和更新的resource生效,对没有修改的resource不生效,另外需要关注policy的defination是只针对resource还是包括resource groups Adds the specified tag and value when any resource missing this tag is created or updated. Existing resources can be remediated by triggering a remediation task. If the tag exists with a different value it will not be changed. Does not modify tags on resource groups. Resize Availability Set下的VM, 需要停止Availability Set下所有的VM If the VM you wish to resize is part of an availability set, then you must stop all VMs in the availability set before changing the size of any VM in the availability set. ...

2023年9月13日

iscsi卷的释放

上一篇我们用losetup建了一个iscsi卷,现在空间不够了,需要释放掉之前建立的iscsi-volumes的20T空间。 首先去isci卷的宿主机查看一下 targetcli ls / 开始删除,先删除backstores,然后是iscsi,lv,vg,pv: # targetcli /backstores/block delete vg-targetd:pvc-harbor Deleted storage object vg-targetd:pvc-harbor. # targetcli /backstores/block delete vg-targetd:pvc-vis-18-31-48 Deleted storage object vg-targetd:pvc-vis-18-31-48. # targetcli /backstores/block delete vg-targetd:pvc-vis-18-31-49 Deleted storage object vg-targetd:pvc-vis-18-31-49. # targetcli /iscsi delete iqn.2020-07.com.ddky:renhe-18-30-18 Deleted Target iqn.2020-07.com.ddky:renhe-18-30-18. # targetcli /iscsi delete iqn.2020-10.com.ddky:vis-18-31-48 Deleted Target iqn.2020-10.com.ddky:vis-18-31-48. # targetcli /iscsi delete iqn.2020-10.com.ddky:vis-18-31-49 Deleted Target iqn.2020-10.com.ddky:vis-18-31-49. # lvs LV VG Attr LSize Pool Origin Data% Meta% Move Log Cpy%Sync Convert pvc-harbor vg-targetd -wi-a----- 200.00g pvc-vis-18-31-48 vg-targetd -wi-a----- 80.00g pvc-vis-18-31-49 vg-targetd -wi-a----- 80.00g # lvremove /dev/vg-targetd/pvc-harbor Do you really want to remove active logical volume vg-targetd/pvc-harbor? [y/n]: y Logical volume "pvc-harbor" successfully removed # lvremove /dev/vg-targetd/pvc-vis-18-31-48 Do you really want to remove active logical volume vg-targetd/pvc-vis-18-31-48? [y/n]: y Logical volume "pvc-vis-18-31-48" successfully removed # lvremove /dev/vg-targetd/pvc-vis-18-31-49 Do you really want to remove active logical volume vg-targetd/pvc-vis-18-31-49? [y/n]: y Logical volume "pvc-vis-18-31-49" successfully removed # vgremove vg-targetd Volume group "vg-targetd" successfully removed # lvs # vgs # pvs PV VG Fmt Attr PSize PFree /dev/loop0 lvm2 --- 19.53t 19.53t # pvremove /dev/loop0 Labels on physical volume "/dev/loop0" successfully wiped. 一整套下来,基本都干净了。 ...

2023年9月5日

iscsi卷的远程挂载使用

如果机器的磁盘空间不够,可以用iscsi把服务器172.18.30.18上面划出一片空间,远程挂上来用。 注意,服务器用losetup的这种做法是为了将来k8s也可以这样用动态iscsi卷 服务器端安装 登录172.18.30.18 安装: yum install -y targetcli targetd` 用文件来虚拟LVM卷: cd /glusterfs/iscsi-volumes/ 生成20TB文件 dd if=/dev/zero of=k8s-iscsi-volumes.img bs=1G count=20000 export LOOP=`losetup -f` losetup $LOOP k8s-iscsi-volumes.img vgcreate vg-targetd $LOOP 修改targetd.yaml: vi /etc/target/targetd.yaml password: xxxxxxxx # defaults below; uncomment and edit # if using a thin pool, use <volume group name>/<thin pool name> # e.g vg-targetd/pool pool_name: vg-targetd user: admin ssl: false target_name: iqn.2020-04.com.ddky:renhe-18-30-18 注意,这个文件生成后,就不需要改动了,如果以后target_name变了,也不用管,也不需要重启targetd 启动服务: systemctl enable --now target systemctl enable --now targetd 运行一下命令,看看显示结果 pvdisplay vgdisplay lvdisplay targetcli ls / 注意:lvdisplay结果和targetcli ls /结果都是空 ...

2023年8月1日