将设为首页浏览此站
开启辅助访问 天气与日历 收藏本站联系我们切换到窄版

易陆发现论坛

 找回密码
 开始注册
查看: 290|回复: 1
收起左侧

kolla安装openstack otoc

[复制链接]
发表于 2021-4-23 16:31:00 | 显示全部楼层 |阅读模式

马上注册,结交更多好友,享用更多功能,让你轻松玩转社区。

您需要 登录 才可以下载或查看,没有帐号?开始注册

x
Deploy基础环境配置
安装PIP
# yum install epel-release# yum install python-pip# pip install -U pip
安装PIP编译环境
# yum install python-devel libffi-devel gcc openssl-devel
安装ansible
# pip install -U ansible
安装docker
# tee /etc/yum.repos.d/docker.repo <<-'EOF'[dockerrepo]name=Docker Repositorybaseurl=https://yum.dockerproject.org/repo/main/centos/7/enabled=1gpgcheck=1gpgkey=https://yum.dockerproject.org/gpgEOF# yum makecache fast# yum install -y docker-engine-1.12.0
配置docker环境
### 配置镜像加速# mkdir -p /etc/docker# tee /etc/docker/daemon.json <<-'EOF'{"registry-mirrors":  ["https://registry.docker-cn.com"]}EOF# systemctl daemon-reload# systemctl restart docker# systemctl enable docker
安装kolla-ansible
# git clone http://git.trystack.cn/openstack/kolla-ansible# cd kolla-ansible# git checkout stable/ocata# pip install .# cp -r etc/kolla /etc### 拷贝inventory到当前目录,也就是后面要执行kolla-ansible命令的目录# cp ansible/inventory/* ~/
配置kolla-ansible
### 设置网卡信息# vim /etc/kolla/globals.ymlkolla_internal_vip_address: "4.0.0.9" ### 选择一个没被使用的内网地址keepalived_virtual_router_id: "9" ### 与kolla_internal_vip_address末尾相同,防止同内网环境其他openstack部署环境干扰network_interface: "ens3"neutron_external_interface: "ens4"openstack_logging_debug: "True"nova_console: "spice"# kolla-genpwd# vim /etc/kolla/passwords.ymlkeystone_admin_password: admin
关闭防火墙和SELINUX
# systemctl stop firewalld# systemctl disable firewalld# setenforce 0# vim /etc/selinux/configSELINUX=disabled
SSH免密码登入
### Deploy执行# ssh-keygen -t rsa### Sched和Nova执行# scp root@4.0.0.10:~/.ssh/id_rsa.pub ./# cat id_rsa.pub >> ~/.ssh/authorized_keys# chmod 600 ~/.ssh/authorized_keys获取docker镜像
  • 使用kolla官方镜像源. J4 ~3 B7 ?, o; X6 e5 l
# vim /etc/kolla/globals.ymlkolla_install_type: "source"openstack_release: "ocata"docker_namespace: "kolla"
  • 手动构建镜像4 {! P& G; u% r9 p2 J) I
# git clone https://github.com/openstack/kolla# cd kolla# git checkout stable/ocata# pip install tox# tox -e genconfig# cp -r etc/kolla /etc/# vim /etc/kolla/kolla-build.confpush = truenamespace = kollaregistry = 4.0.0.10:4000install_type = sourcetag = ocata# kolla-build --config-file=/etc/kolla/kolla-build.conf### kolla-ansible配置如下# vim /etc/kolla/globals.ymldocker_registry: "4.0.0.10:4000"docker_namespace: "kolla"openstack_release: "ocata"kolla_install_type: "source"
  • 下载镜像包
    - X# ?1 J! \3 [1 S5 g
# wget http://tarballs.openstack.org/ko ... gistry-ocata.tar.gz### kolla-ansible配置如下# vim /etc/kolla/globals.ymldocker_registry: "4.0.0.10:4000"docker_namespace: "lokolla"openstack_release: "4.0.2"kolla_install_type: "source"设置本地镜像源# docker run -d -v /opt/registry:/var/lib/registry -p 4000:5000 --restart=always --name registry registry:2### 此处使用下载镜像包方式# wget http://tarballs.openstack.org/ko ... gistry-ocata.tar.gz# tar zxf centos-source-registry-ocata.tar.gz -C /opt/registry/搭建http代理服务器(可选步骤)
在Deploy节点上搭建http服务器
# yum install squid -y### 修改配置文件,删除默认自带的对IP和端口的限制,然后添加如下规则# vim /etc/squid/squid.confhttp_access allow all### 设置开机自启动# systemctl restart squid# systemctl enable squid### 修改ansible脚本# vim /usr/share/kolla-ansible/ansible/kolla-host.yml  environment:    https_proxy : http://4.0.0.10:3128/    http_proxy : http://4.0.0.10:3128/
修改Sched/Nova的环境的配置文件
### 设置yum代理# vim /etc/yum.confproxy=http://4.0.0.10:3128# yum makecache修改部署文件### Deploy节点上执行# vim ~/multinode[control]4.0.0.11[network]4.0.0.11[compute]4.0.0.12[monitoring]4.0.0.11[storage]4.0.0.12### 配置Sched、Nova主机基础环境,安装必备软件包# kolla-ansible -i multinode bootstrap-servers部署openstack# kolla-ansible prechecks -i multinode# kolla-ansible deploy -i multinode验证openstack安装# kolla-ansible post-deploy -i multinodeOpenStack更新### 修改镜像版本# vim /etc/kolla/globals.ymlopenstack_release: "4.0.3"# kolla-ansible pull -i multinode# kolla-ansible upgrade -i multinode环境还原### 将删除所有容器和卷# kolla-ansible destroy -i multinode可选功能部署Ceph部署
修改配置文件
# vim multinode[storage]4.0.0.12# vim /etc/kolla/globals.ymlenable_ceph: "yes"enable_ceph_rgw: "yes"enable_cinder: "yes"### 也可以同时配置yesglance_backend_file: "no"glance_backend_ceph: "yes"# mkdir -p /etc/kolla/config# tee /etc/kolla/config/ceph.conf <<-'EOF'[global]osd pool default size = 1osd pool default min size = 1osd pool default pg num =  128osd pool default pgp num =  128EOF
修改Nova节点
### 添加一块总线为IDE格式为qcow2的磁盘# fdisk -lDisk /dev/sda: 10.7 GB, 10737418240 bytes, 20971520 sectorsUnits = sectors of 1 * 512 = 512 bytesSector size (logical/physical): 512 bytes / 512 bytesI/O size (minimum/optimal): 512 bytes / 512 bytes# parted /dev/sda -s -- mklabel gpt mkpart KOLLA_CEPH_OSD_BOOTSTRAP 1 -1# parted /dev/sda printModel: ATA QEMU HARDDISK (scsi)Disk /dev/sda: 10.7GBSector size (logical/physical): 512B/512BPartition Table: gptDisk Flags: Number  Start   End     Size    File system  Name                      Flags 1      1049kB  10.7GB  10.7GB               KOLLA_CEPH_OSD_BOOTSTRAP问题处理部署问题
CEPH部署失败
### 问题描述TASK [ceph : Fetching Ceph keyrings] ******************************************************************************************************************************************************** [WARNING]: when statements should not include jinja2 templating delimiters such as {{ }} or {% %}. Found: {{ (ceph_files_json.stdout | from_json).changed }}fatal: [92.0.0.11]: FAILED! => {"failed": true, "msg": "The conditional check '{{ (ceph_files_json.stdout | from_json).changed }}' failed. The error was: No JSON object could be decoded"}### 解决办法### 在OSD节点执行,获取挂载的CEPH磁盘# ds4ft=`mount | grep ceph | awk '{print $1}' | tr -d [:digit:] | sort -u`### 在OSD节点执行,取消挂载# echo $ds4ft | tr ' ' '\n' | xargs -i umount {}### 在OSD节点执行,格式化磁盘# echo $ds4ft | tr ' ' '\n' | xargs -i mkfs.xfs -f {}### 在OSD节点执行,重新设置标记# echo $ds4ft | tr ' ' '\n' | xargs -i  parted {} -s -- mklabel gpt mkpart KOLLA_CEPH_OSD_BOOTSTRAP 1 -1### 在MON和OSD节点执行,删除所有CEPH容器和配置文件# docker ps --filter "label=kolla_version" --format "{{.Names}}" -a | grep -E "ceph|osd" | xargs -i docker rm -f {}# rm -rf /var/lib/kolla/var/lib/ceph /var/lib/kolla/etc/ceph /etc/kolla/ceph-*### 在MON节点执行# docker volume rm ceph_mon_config
Mariadb部署失败
### 问题描述TASK: [mariadb | Creating haproxy mysql user] *********************************......stdout: localhost | FAILED! => {    "changed": false,    "failed": true,    "msg": "unable to connect to database, check login_user and login_password are correct or ~/.my.cnf has the credentials. Exception message: (1045, \"Access denied for user 'root'@'mick-workstation' (using password: YES)\")"}msg: Task failed as maximum retries was encountered### 解决办法# docker rm mariadb# rm -rf /var/lib/docker/volumes/mariadb/_data/*
等待VIP超时
### 问题描述获取不到VIP### 解决办法### 尝试一: 修改kolla_internal_vip_address或keepalived_virtual_router_id防止VIP被其他kolla部署环境占用# vim /etc/kolla/globals.ymlkolla_internal_vip_address: "4.0.0.5"keepalived_virtual_router_id: "5"# kolla-ansible -i multinode deploy --tags="haproxy"### 尝试二: 禁用主机的网络管理器# docker rm -f haproxy keepalived# systemctl stop NetworkManager# systemctl disable NetworkManager# kolla-ansible -i multinode deploy --tags="haproxy"
VIP自动消失
### 问题描述获取到的VIP会自动消失### 解决办法### 尝试一: IP是否被占用# ping 4.0.0.9### 尝试二: 判断keepalived监控的服务是否正常# docker exec -it keepalived bash# ./check_alive.sh### 如果打印的值为0说明keepalived监控的服务正常# echo $?### 尝试三: 参考网上一篇帖子的做法(http://www.cnblogs.com/ayao/p/keepalived-loss-vip.html),将VIP绑定的网卡的BOOTPROTO=none防止因为dhcp导致的VIP消失问题(虽然问题出现时网卡的配置是static的,应该不是dhcp的问题,但是似乎VIP不会消失了)
Bootstrap-servers执行失败
### 问题描述docker-engine安装失败### 解决办法### 在节点上手动安装,下载超时也会导致失败# yum clean all# yum makecache fast# yum install docker-engine-1.12.0 -y
kolla-ansible执行报错
### 问题描述Ansible 2.4.0: No test named 'equalto'### 解决办法# pip install --upgrade Jinja2
kolla-ansible执行报错
### 问题描述ansible "msg": "shade is required for this module"### 解决办法# pip install shade
镜像拉取失败
### 问题描述Error response from daemon: Get https://4.0.0.10:4000/v1/_ping: http: server gave HTTP response to HTTPS clientNetwork timed out while trying to connect to http://4.0.0.10:4000/v1/repositories/lokolla/centos-source-fluentd/images. You may want to check your internet connection or if you are behind a proxy.### 解决办法### 尝试一: 降低docker版本# yum remove docker-engine# yum install docker-engine-1.12.0 -y### 尝试二: 修改docker服务配置文件# vim /etc/systemd/system/docker.service.d/kolla.conf[Service]MountFlags=sharedExecStart=ExecStart=/usr/bin/docker daemon --insecure-registry 92.0.0.10:4000# systemctl daemon-reload# systemctl restart docker# ps -ef|grep dockerroot      1294     1  0 11:36 ?        00:00:01 /usr/bin/dockerd --insecure-registry 92.0.0.10:4000
无法生成/etc/hosts文件
### 问题描述TASK [baremetal : Generate /etc/hosts for all of the nodes] *********************************************************************************************************************************fatal: [4.0.0.11]: FAILED! => {"failed": true, "msg": "the field 'args' has an invalid value, which appears to include a variable that is undefined. The error was: 'dict object' has no attribute u'ansible_ens3'\n\nThe error appears to have been in '/usr/share/kolla-ansible/ansible/roles/baremetal/tasks/pre-install.yml': line 40, column 3, but may\nbe elsewhere in the file depending on the exact syntax problem.\n\nThe offending line appears to be:\n\n\n- name: Generate /etc/hosts for all of the nodes\n  ^ here\n"}    to retry, use: --limit @/usr/share/kolla-ansible/ansible/kolla-host.retry### 解决办法- 查询下是不是有节点安装python失败,导致没有执行Gather facts操作造成失败- 查询下是不是有节点的网卡名不对使用问题
虚拟机创建失败
### 问题描述创建带volume的虚拟机提示"Block device mapping is invalid"- 查看nova-compute日志:VolumeNotCreated: Volume f103b3f3-d0ff-4a2b-9a5e-4b7ea5a9abdc did not finish being created even after we waited 3 seconds or 2 attempts. And its status is error.- 查看cinder-volume日志:Volume group "cinder-volumes" not found### 解决办法### 打印vg列表发现的确没有"cinder-volumes",这个卷是要手动建立的# vgdisplay# dd if=/dev/zero of=./disk.img count=4096 bs=1MB### 查询未被占用的loop设备# losetup -f# losetup /dev/loop2 disk.img# pvcreate /dev/loop2# vgcreate cinder-volumes /dev/loop2### 重启容器,然后查看cinder-volume日志是否成功找到"cinder-volumes"# docker restart cinder-volumes
虚拟机创建失败
### 问题描述创建带volume的虚拟机提示"No valid host was found"- 查看nova-compute日志: 'iscsiadm -m node -T iqn.2010-10.org.openstack:volume-bbbccab7-bdd7-4086-8d0e-e14898439131 -p 127.0.0.1:3260' failed### 解决办法### 从日志看来是登入iscsi服务器失败,IP地址不对# vim /etc/kolla/cinder-volume/cinder.confmy_ip = 4.0.0.12# docker restart cinder-volumes
虚拟机创建失败
### 问题描述- nova-compute日志提示ERROR nova.compute.manager [instance: 3af11e19-b4f8-452a-8f3d-3d659be050bd] libvirtError: Did not receive a reply. Possible causes include: the remote application did not send a reply, the message bus security policy blocked the reply, the reply timeout expired, or the network connection was broken.- libvirtd日志提示2017-11-01 18:46:09.362+0000: 30348: error : virDBusCall:1570 : error from service: CanSuspend: Did not receive a reply. Possible causes include: the remote application did not send a reply, the message bus security policy blocked the reply, the reply timeout expired, or the network connection was broken.### 解决办法检查节点状态,发现selinux开启了,关闭selinux即可
虚拟机创建失败
### 问题描述- nova-compute日志提示ERROR nova.image.glance [req-3d1ff662-1b82-4cf9-8aba-28644c8ba88b ded6f7f63e6743b4ac88213d5d2df5ce dded963ced3946b585c92d0684277eec - - -] Error writing to /var/lib/nova/instances/_base/bc9f28a296e64237d4205a6ea278c3e51c083ba0.part: 'NoneType' object is not iterableERROR nova.compute.manager [instance: 2e6b91e4-96f3-41ae-98d3-4d72f1b2cd37]   File "/usr/lib/python2.7/site-packages/nova/image/glance.py", line 586, in downloadERROR nova.compute.manager [instance: 2e6b91e4-96f3-41ae-98d3-4d72f1b2cd37]     for chunk in image_chunks:ERROR nova.compute.manager [instance: 2e6b91e4-96f3-41ae-98d3-4d72f1b2cd37] TypeError: 'NoneType' object is not iterable- glance_api日志提示WARNING glance.location [req-27feac8f-9cfb-4891-91fd-913bcb7a9f1f ded6f7f63e6743b4ac88213d5d2df5ce dded963ced3946b585c92d0684277eec - default default] Get image 856dbf39-8821-4906-a68c-e78f77d88b27 data failed: Image /var/lib/glance/images/856dbf39-8821-4906-a68c-e78f77d88b27 not found.ERROR glance.location [req-27feac8f-9cfb-4891-91fd-913bcb7a9f1f ded6f7f63e6743b4ac88213d5d2df5ce dded963ced3946b585c92d0684277eec - default default] Glance tried all active locations to get data for image 856dbf39-8821-4906-a68c-e78f77d88b27 but all have failed.### 解决办法到glance节点的/var/lib/glance/images目录下的确没有任何内容,重启horizon容器,然后删除镜像,然后重新上传并且重命名
虚拟机创建失败
### 问题描述rpc超时### 解决办法rabbitmq集群部署的配置文件使用的是主机名,所以HA环境下各个Contoller主机的主机名不能相同
虚拟机创建失败
### 问题描述failed to connect to the hypervisor### 解决办法### 查询虚拟机的嵌套虚拟化是否开启# cat /sys/module/kvm_intel/parameters/nestedN### 如果为N,关闭虚拟机,设置CPU,开启嵌套虚拟化# shutdown now
mariadb容器启动失败
### 问题描述[ERROR] WSREP: failed to open gcomm backend connection: 131: invalid UUID: 00000000 (FATAL)at gcomm/src/pc.cpp:PC():271### 解决办法# rm -rf /var/lib/docker/volumes/mariadb/_data/gvwstate.dat# docker restart mariadb
宕机后mariadb集群恢复
### 问题描述mariadb服务异常### 解决办法########## 停止所有mariadb容器 ########### docker stop mariadb########## 找到最后关闭的mariadb主机,如果不记得就随机选取一台或者根据/var/lib/docker/volumes/mariadb/_data/grastate.dat的seqno进行选取(越大优先级越高),然后修改其grastate.dat文件的safe_to_bootstrap参数 ########### vim /var/lib/docker/volumes/mariadb/_data/grastate.datsafe_to_bootstrap: 1########## 修改mariadb容器启动命令后启动容器,查询日志保证mariadb服务正常启动 ########### vim /etc/kolla/mariadb/config.json"command": "/usr/bin/mysqld_safe --wsrep-new-cluster",# docker start mariadb# tail -200f /var/lib/docker/volumes/kolla_logs/_data/mariadb/mariadb.log########## 启动其他节点的mariadb容器 ########### docker start mariadb# tail -200f /var/lib/docker/volumes/kolla_logs/_data/mariadb/mariadb.log########## 确保集群运行正常后,恢复最初修改的config.json(这样就保证集群中所有的mariadb容器都是平等的)########### vim /etc/kolla/mariadb/config.json"command": "/usr/bin/mysqld_safer",# docker stop mariadb# docker start mariadb# tail -200f /var/lib/docker/volumes/kolla_logs/_data/mariadb/mariadb.log
Horizon无法访问
### 问题描述[Tue May 29 09:58:25.056236 2018] [core:error] [pid 20] [client 192.168.0.1:51277] Script timed out before returning headers: django.wsgi[Tue May 29 09:58:28.017141 2018] [core:error] [pid 81] [client 192.168.0.105:48074] End of script output before headers: django.wsgi, referer: http://47.98.113.179:8011/### 解决办法### 添加 WSGIApplicationGroup 配置项# vim /etc/httpd/conf.d/openstack-dashboard.conf...WSGISocketPrefix run/wsgiWSGIApplicationGroup %{GLOBAL}...
Redis容器异常
### 问题描述5:M 29 May 17:31:11.072 # Short read or OOM loading DB. Unrecoverable error, aborting now.5:M 29 May 17:31:11.072 # Internal error in RDB reading function at rdb.c:1428 -> Unexpected EOF reading RDB file### 解决办法# rm -rf /var/lib/docker/volumes/redis/_data/dump.rdb# docker restart redis
在nova-compute容器执行PIP命令权限不够
### 问题描述容器内无法执行pip命令### 解决办法### 在容器内执行# cat /etc/nova/rootwrap.conf | grep filters_pathfilters_path=/etc/nova/rootwrap.d,/usr/share/nova/rootwrap### 在宿主机上执行# tee pip.filters << EOF[Filters]pip: CommandFilter, pip, rootEOF# docker cp pip.filters nova_compute:/usr/share/nova/rootwrap# sudo nova-rootwrap /etc/nova/rootwrap.conf pip install -U pip
ovs命令无法正常执行
### 问题描述# ovs-appctl ofproto/trace br-tun dl_vlan=32017-06-05T08:37:32Z|00001|daemon_unix|WARN|/var/run/openvswitch/ovs-vswitchd.pid: open: No such file or directoryovs-appctl: cannot read pidfile "/var/run/openvswitch/ovs-vswitchd.pid" (No such file or directory)### 解决办法### 添加启动参数# vim /etc/kolla/openvswitch-vswitchd/config.json--pidfile=/var/run/openvswitch/ovs-vswitchd.pid# docker restart openvswitch_vswitchd
 楼主| 发表于 2021-4-23 16:31:53 | 显示全部楼层
Deploy基础环境配置
安装PIP
# yum install epel-release# yum install python-pip# pip install -U pip
安装PIP编译环境
# yum install python-devel libffi-devel gcc openssl-devel
安装ansible
# pip install -U ansible
安装docker
# tee /etc/yum.repos.d/docker.repo <<-'EOF'[dockerrepo]name=Docker Repositorybaseurl=https://yum.dockerproject.org/repo/main/centos/7/enabled=1gpgcheck=1gpgkey=https://yum.dockerproject.org/gpgEOF# yum makecache fast# yum install -y docker-engine-1.12.0
配置docker环境
### 配置镜像加速# mkdir -p /etc/docker# tee /etc/docker/daemon.json <<-'EOF'{"registry-mirrors":  ["https://registry.docker-cn.com"]}EOF# systemctl daemon-reload# systemctl restart docker# systemctl enable docker
安装kolla-ansible
# git clone http://git.trystack.cn/openstack/kolla-ansible# cd kolla-ansible# git checkout stable/ocata# pip install .# cp -r etc/kolla /etc### 拷贝inventory到当前目录,也就是后面要执行kolla-ansible命令的目录# cp ansible/inventory/* ~/
配置kolla-ansible
### 设置网卡信息# vim /etc/kolla/globals.ymlkolla_internal_vip_address: "4.0.0.9" ### 选择一个没被使用的内网地址keepalived_virtual_router_id: "9" ### 与kolla_internal_vip_address末尾相同,防止同内网环境其他openstack部署环境干扰network_interface: "ens3"neutron_external_interface: "ens4"openstack_logging_debug: "True"nova_console: "spice"# kolla-genpwd# vim /etc/kolla/passwords.ymlkeystone_admin_password: admin
关闭防火墙和SELINUX
# systemctl stop firewalld# systemctl disable firewalld# setenforce 0# vim /etc/selinux/configSELINUX=disabled
SSH免密码登入
### Deploy执行# ssh-keygen -t rsa### Sched和Nova执行# scp root@4.0.0.10:~/.ssh/id_rsa.pub ./# cat id_rsa.pub >> ~/.ssh/authorized_keys# chmod 600 ~/.ssh/authorized_keys
获取docker镜像
  • 使用kolla官方镜像源

    $ A7 p# s4 t& g& \3 W" A# T
# vim /etc/kolla/globals.ymlkolla_install_type: "source"openstack_release: "ocata"docker_namespace: "kolla"
  • 手动构建镜像

    : U/ W+ @3 s: g4 M7 u4 L' o- A
# git clone https://github.com/openstack/kolla# cd kolla# git checkout stable/ocata# pip install tox# tox -e genconfig# cp -r etc/kolla /etc/# vim /etc/kolla/kolla-build.confpush = truenamespace = kollaregistry = 4.0.0.10:4000install_type = sourcetag = ocata# kolla-build --config-file=/etc/kolla/kolla-build.conf### kolla-ansible配置如下# vim /etc/kolla/globals.ymldocker_registry: "4.0.0.10:4000"docker_namespace: "kolla"openstack_release: "ocata"kolla_install_type: "source"
  • 下载镜像包
    : K$ c0 c! s2 s' w( y
# wget http://tarballs.openstack.org/ko ... gistry-ocata.tar.gz### kolla-ansible配置如下# vim /etc/kolla/globals.ymldocker_registry: "4.0.0.10:4000"docker_namespace: "lokolla"openstack_release: "4.0.2"kolla_install_type: "source"
设置本地镜像源
# docker run -d -v /opt/registry:/var/lib/registry -p 4000:5000 --restart=always --name registry registry:2### 此处使用下载镜像包方式# wget http://tarballs.openstack.org/ko ... gistry-ocata.tar.gz# tar zxf centos-source-registry-ocata.tar.gz -C /opt/registry/
搭建http代理服务器(可选步骤)
在Deploy节点上搭建http服务器
# yum install squid -y### 修改配置文件,删除默认自带的对IP和端口的限制,然后添加如下规则# vim /etc/squid/squid.confhttp_access allow all### 设置开机自启动# systemctl restart squid# systemctl enable squid### 修改ansible脚本# vim /usr/share/kolla-ansible/ansible/kolla-host.yml  environment:    https_proxy : http://4.0.0.10:3128/    http_proxy : http://4.0.0.10:3128/
修改Sched/Nova的环境的配置文件
### 设置yum代理# vim /etc/yum.confproxy=http://4.0.0.10:3128# yum makecache
修改部署文件
### Deploy节点上执行# vim ~/multinode[control]4.0.0.11[network]4.0.0.11[compute]4.0.0.12[monitoring]4.0.0.11[storage]4.0.0.12### 配置Sched、Nova主机基础环境,安装必备软件包# kolla-ansible -i multinode bootstrap-servers
部署openstack
# kolla-ansible prechecks -i multinode# kolla-ansible deploy -i multinode
验证openstack安装
# kolla-ansible post-deploy -i multinode
OpenStack更新
### 修改镜像版本# vim /etc/kolla/globals.ymlopenstack_release: "4.0.3"# kolla-ansible pull -i multinode# kolla-ansible upgrade -i multinode
环境还原
### 将删除所有容器和卷# kolla-ansible destroy -i multinode
可选功能部署Ceph部署
修改配置文件
# vim multinode[storage]4.0.0.12# vim /etc/kolla/globals.ymlenable_ceph: "yes"enable_ceph_rgw: "yes"enable_cinder: "yes"### 也可以同时配置yesglance_backend_file: "no"glance_backend_ceph: "yes"# mkdir -p /etc/kolla/config# tee /etc/kolla/config/ceph.conf <<-'EOF'[global]osd pool default size = 1osd pool default min size = 1osd pool default pg num =  128osd pool default pgp num =  128EOF
修改Nova节点
### 添加一块总线为IDE格式为qcow2的磁盘# fdisk -lDisk /dev/sda: 10.7 GB, 10737418240 bytes, 20971520 sectorsUnits = sectors of 1 * 512 = 512 bytesSector size (logical/physical): 512 bytes / 512 bytesI/O size (minimum/optimal): 512 bytes / 512 bytes# parted /dev/sda -s -- mklabel gpt mkpart KOLLA_CEPH_OSD_BOOTSTRAP 1 -1# parted /dev/sda printModel: ATA QEMU HARDDISK (scsi)Disk /dev/sda: 10.7GBSector size (logical/physical): 512B/512BPartition Table: gptDisk Flags: Number  Start   End     Size    File system  Name                      Flags 1      1049kB  10.7GB  10.7GB               KOLLA_CEPH_OSD_BOOTSTRAP
问题处理部署问题
CEPH部署失败
### 问题描述TASK [ceph : Fetching Ceph keyrings] ******************************************************************************************************************************************************** [WARNING]: when statements should not include jinja2 templating delimiters such as {{ }} or {% %}. Found: {{ (ceph_files_json.stdout | from_json).changed }}fatal: [92.0.0.11]: FAILED! => {"failed": true, "msg": "The conditional check '{{ (ceph_files_json.stdout | from_json).changed }}' failed. The error was: No JSON object could be decoded"}### 解决办法### 在OSD节点执行,获取挂载的CEPH磁盘# ds4ft=`mount | grep ceph | awk '{print $1}' | tr -d [:digit:] | sort -u`### 在OSD节点执行,取消挂载# echo $ds4ft | tr ' ' '\n' | xargs -i umount {}### 在OSD节点执行,格式化磁盘# echo $ds4ft | tr ' ' '\n' | xargs -i mkfs.xfs -f {}### 在OSD节点执行,重新设置标记# echo $ds4ft | tr ' ' '\n' | xargs -i  parted {} -s -- mklabel gpt mkpart KOLLA_CEPH_OSD_BOOTSTRAP 1 -1### 在MON和OSD节点执行,删除所有CEPH容器和配置文件# docker ps --filter "label=kolla_version" --format "{{.Names}}" -a | grep -E "ceph|osd" | xargs -i docker rm -f {}# rm -rf /var/lib/kolla/var/lib/ceph /var/lib/kolla/etc/ceph /etc/kolla/ceph-*### 在MON节点执行# docker volume rm ceph_mon_config
Mariadb部署失败
### 问题描述TASK: [mariadb | Creating haproxy mysql user] *********************************......stdout: localhost | FAILED! => {    "changed": false,    "failed": true,    "msg": "unable to connect to database, check login_user and login_password are correct or ~/.my.cnf has the credentials. Exception message: (1045, \"Access denied for user 'root'@'mick-workstation' (using password: YES)\")"}msg: Task failed as maximum retries was encountered### 解决办法# docker rm mariadb# rm -rf /var/lib/docker/volumes/mariadb/_data/*
等待VIP超时
### 问题描述获取不到VIP### 解决办法### 尝试一: 修改kolla_internal_vip_address或keepalived_virtual_router_id防止VIP被其他kolla部署环境占用# vim /etc/kolla/globals.ymlkolla_internal_vip_address: "4.0.0.5"keepalived_virtual_router_id: "5"# kolla-ansible -i multinode deploy --tags="haproxy"### 尝试二: 禁用主机的网络管理器# docker rm -f haproxy keepalived# systemctl stop NetworkManager# systemctl disable NetworkManager# kolla-ansible -i multinode deploy --tags="haproxy"
VIP自动消失
### 问题描述获取到的VIP会自动消失### 解决办法### 尝试一: IP是否被占用# ping 4.0.0.9### 尝试二: 判断keepalived监控的服务是否正常# docker exec -it keepalived bash# ./check_alive.sh### 如果打印的值为0说明keepalived监控的服务正常# echo $?### 尝试三: 参考网上一篇帖子的做法(http://www.cnblogs.com/ayao/p/keepalived-loss-vip.html),将VIP绑定的网卡的BOOTPROTO=none防止因为dhcp导致的VIP消失问题(虽然问题出现时网卡的配置是static的,应该不是dhcp的问题,但是似乎VIP不会消失了)
Bootstrap-servers执行失败
### 问题描述docker-engine安装失败### 解决办法### 在节点上手动安装,下载超时也会导致失败# yum clean all# yum makecache fast# yum install docker-engine-1.12.0 -y
kolla-ansible执行报错
### 问题描述Ansible 2.4.0: No test named 'equalto'### 解决办法# pip install --upgrade Jinja2
kolla-ansible执行报错
### 问题描述ansible "msg": "shade is required for this module"### 解决办法# pip install shade
镜像拉取失败
### 问题描述Error response from daemon: Get https://4.0.0.10:4000/v1/_ping: http: server gave HTTP response to HTTPS clientNetwork timed out while trying to connect to http://4.0.0.10:4000/v1/reposito ... rce-fluentd/images. You may want to check your internet connection or if you are behind a proxy.### 解决办法### 尝试一: 降低docker版本# yum remove docker-engine# yum install docker-engine-1.12.0 -y### 尝试二: 修改docker服务配置文件# vim /etc/systemd/system/docker.service.d/kolla.conf[Service]MountFlags=sharedExecStart=ExecStart=/usr/bin/docker daemon --insecure-registry 92.0.0.10:4000# systemctl daemon-reload# systemctl restart docker# ps -ef|grep dockerroot      1294     1  0 11:36 ?        00:00:01 /usr/bin/dockerd --insecure-registry 92.0.0.10:4000
无法生成/etc/hosts文件
### 问题描述TASK [baremetal : Generate /etc/hosts for all of the nodes] *********************************************************************************************************************************fatal: [4.0.0.11]: FAILED! => {"failed": true, "msg": "the field 'args' has an invalid value, which appears to include a variable that is undefined. The error was: 'dict object' has no attribute u'ansible_ens3'\n\nThe error appears to have been in '/usr/share/kolla-ansible/ansible/roles/baremetal/tasks/pre-install.yml': line 40, column 3, but may\nbe elsewhere in the file depending on the exact syntax problem.\n\nThe offending line appears to be:\n\n\n- name: Generate /etc/hosts for all of the nodes\n  ^ here\n"}    to retry, use: --limit @/usr/share/kolla-ansible/ansible/kolla-host.retry### 解决办法- 查询下是不是有节点安装python失败,导致没有执行Gather facts操作造成失败- 查询下是不是有节点的网卡名不对
使用问题
虚拟机创建失败
### 问题描述创建带volume的虚拟机提示"Block device mapping is invalid"- 查看nova-compute日志:VolumeNotCreated: Volume f103b3f3-d0ff-4a2b-9a5e-4b7ea5a9abdc did not finish being created even after we waited 3 seconds or 2 attempts. And its status is error.- 查看cinder-volume日志:Volume group "cinder-volumes" not found### 解决办法### 打印vg列表发现的确没有"cinder-volumes",这个卷是要手动建立的# vgdisplay# dd if=/dev/zero of=./disk.img count=4096 bs=1MB### 查询未被占用的loop设备# losetup -f# losetup /dev/loop2 disk.img# pvcreate /dev/loop2# vgcreate cinder-volumes /dev/loop2### 重启容器,然后查看cinder-volume日志是否成功找到"cinder-volumes"# docker restart cinder-volumes
虚拟机创建失败
### 问题描述创建带volume的虚拟机提示"No valid host was found"- 查看nova-compute日志: 'iscsiadm -m node -T iqn.2010-10.org.openstack:volume-bbbccab7-bdd7-4086-8d0e-e14898439131 -p 127.0.0.1:3260' failed### 解决办法### 从日志看来是登入iscsi服务器失败,IP地址不对# vim /etc/kolla/cinder-volume/cinder.confmy_ip = 4.0.0.12# docker restart cinder-volumes
虚拟机创建失败
### 问题描述- nova-compute日志提示ERROR nova.compute.manager [instance: 3af11e19-b4f8-452a-8f3d-3d659be050bd] libvirtError: Did not receive a reply. Possible causes include: the remote application did not send a reply, the message bus security policy blocked the reply, the reply timeout expired, or the network connection was broken.- libvirtd日志提示2017-11-01 18:46:09.362+0000: 30348: error : virDBusCall:1570 : error from service: CanSuspend: Did not receive a reply. Possible causes include: the remote application did not send a reply, the message bus security policy blocked the reply, the reply timeout expired, or the network connection was broken.### 解决办法检查节点状态,发现selinux开启了,关闭selinux即可
虚拟机创建失败
### 问题描述- nova-compute日志提示ERROR nova.image.glance [req-3d1ff662-1b82-4cf9-8aba-28644c8ba88b ded6f7f63e6743b4ac88213d5d2df5ce dded963ced3946b585c92d0684277eec - - -] Error writing to /var/lib/nova/instances/_base/bc9f28a296e64237d4205a6ea278c3e51c083ba0.part: 'NoneType' object is not iterableERROR nova.compute.manager [instance: 2e6b91e4-96f3-41ae-98d3-4d72f1b2cd37]   File "/usr/lib/python2.7/site-packages/nova/image/glance.py", line 586, in downloadERROR nova.compute.manager [instance: 2e6b91e4-96f3-41ae-98d3-4d72f1b2cd37]     for chunk in image_chunks:ERROR nova.compute.manager [instance: 2e6b91e4-96f3-41ae-98d3-4d72f1b2cd37] TypeError: 'NoneType' object is not iterable- glance_api日志提示WARNING glance.location [req-27feac8f-9cfb-4891-91fd-913bcb7a9f1f ded6f7f63e6743b4ac88213d5d2df5ce dded963ced3946b585c92d0684277eec - default default] Get image 856dbf39-8821-4906-a68c-e78f77d88b27 data failed: Image /var/lib/glance/images/856dbf39-8821-4906-a68c-e78f77d88b27 not found.ERROR glance.location [req-27feac8f-9cfb-4891-91fd-913bcb7a9f1f ded6f7f63e6743b4ac88213d5d2df5ce dded963ced3946b585c92d0684277eec - default default] Glance tried all active locations to get data for image 856dbf39-8821-4906-a68c-e78f77d88b27 but all have failed.### 解决办法到glance节点的/var/lib/glance/images目录下的确没有任何内容,重启horizon容器,然后删除镜像,然后重新上传并且重命名
虚拟机创建失败
### 问题描述rpc超时### 解决办法rabbitmq集群部署的配置文件使用的是主机名,所以HA环境下各个Contoller主机的主机名不能相同
虚拟机创建失败
### 问题描述failed to connect to the hypervisor### 解决办法### 查询虚拟机的嵌套虚拟化是否开启# cat /sys/module/kvm_intel/parameters/nestedN### 如果为N,关闭虚拟机,设置CPU,开启嵌套虚拟化# shutdown now
mariadb容器启动失败
### 问题描述[ERROR] WSREP: failed to open gcomm backend connection: 131: invalid UUID: 00000000 (FATAL)at gcomm/src/pc.cpp:PC():271### 解决办法# rm -rf /var/lib/docker/volumes/mariadb/_data/gvwstate.dat# docker restart mariadb
宕机后mariadb集群恢复
### 问题描述mariadb服务异常### 解决办法########## 停止所有mariadb容器 ########### docker stop mariadb########## 找到最后关闭的mariadb主机,如果不记得就随机选取一台或者根据/var/lib/docker/volumes/mariadb/_data/grastate.dat的seqno进行选取(越大优先级越高),然后修改其grastate.dat文件的safe_to_bootstrap参数 ########### vim /var/lib/docker/volumes/mariadb/_data/grastate.datsafe_to_bootstrap: 1########## 修改mariadb容器启动命令后启动容器,查询日志保证mariadb服务正常启动 ########### vim /etc/kolla/mariadb/config.json"command": "/usr/bin/mysqld_safe --wsrep-new-cluster",# docker start mariadb# tail -200f /var/lib/docker/volumes/kolla_logs/_data/mariadb/mariadb.log########## 启动其他节点的mariadb容器 ########### docker start mariadb# tail -200f /var/lib/docker/volumes/kolla_logs/_data/mariadb/mariadb.log########## 确保集群运行正常后,恢复最初修改的config.json(这样就保证集群中所有的mariadb容器都是平等的)########### vim /etc/kolla/mariadb/config.json"command": "/usr/bin/mysqld_safer",# docker stop mariadb# docker start mariadb# tail -200f /var/lib/docker/volumes/kolla_logs/_data/mariadb/mariadb.log
Horizon无法访问
### 问题描述[Tue May 29 09:58:25.056236 2018] [core:error] [pid 20] [client 192.168.0.1:51277] Script timed out before returning headers: django.wsgi[Tue May 29 09:58:28.017141 2018] [core:error] [pid 81] [client 192.168.0.105:48074] End of script output before headers: django.wsgi, referer: http://47.98.113.179:8011/### 解决办法### 添加 WSGIApplicationGroup 配置项# vim /etc/httpd/conf.d/openstack-dashboard.conf...WSGISocketPrefix run/wsgiWSGIApplicationGroup %{GLOBAL}...
Redis容器异常
### 问题描述5:M 29 May 17:31:11.072 # Short read or OOM loading DB. Unrecoverable error, aborting now.5:M 29 May 17:31:11.072 # Internal error in RDB reading function at rdb.c:1428 -> Unexpected EOF reading RDB file### 解决办法# rm -rf /var/lib/docker/volumes/redis/_data/dump.rdb# docker restart redis
在nova-compute容器执行PIP命令权限不够
### 问题描述容器内无法执行pip命令### 解决办法### 在容器内执行# cat /etc/nova/rootwrap.conf | grep filters_pathfilters_path=/etc/nova/rootwrap.d,/usr/share/nova/rootwrap### 在宿主机上执行# tee pip.filters << EOF[Filters]pip: CommandFilter, pip, rootEOF# docker cp pip.filters nova_compute:/usr/share/nova/rootwrap# sudo nova-rootwrap /etc/nova/rootwrap.conf pip install -U pip
ovs命令无法正常执行
### 问题描述# ovs-appctl ofproto/trace br-tun dl_vlan=32017-06-05T08:37:32Z|00001|daemon_unix|WARN|/var/run/openvswitch/ovs-vswitchd.pid: open: No such file or directoryovs-appctl: cannot read pidfile "/var/run/openvswitch/ovs-vswitchd.pid" (No such file or directory)### 解决办法### 添加启动参数# vim /etc/kolla/openvswitch-vswitchd/config.json--pidfile=/var/run/openvswitch/ovs-vswitchd.pid# docker restart openvswitch_vswitchd
# s# o6 z5 ~! G9 h
' s9 [! N$ A' q  X
您需要登录后才可以回帖 登录 | 开始注册

本版积分规则

关闭

站长推荐上一条 /4 下一条

如有购买积分卡请联系497906712

QQ|返回首页|Archiver|手机版|小黑屋|易陆发现 点击这里给我发消息

GMT+8, 2021-6-20 20:14 , Processed in 0.103553 second(s), 22 queries .

Powered by 龙睿 bbs168x X3.2

© 2001-2020 Comsenz Inc.

快速回复 返回顶部 返回列表