메뉴 닫기
  • Home
  • 오픈 스택 queens 인스턴스 리사이즈시 Unexpected API Error 에러
    •

오픈 스택 queens 인스턴스 리사이즈시 Unexpected API Error 에러

장영호 0 댓글

최근 오픈 스택 queens 버전에서 인스턴스 리사이즈시 정상적으로 처리 되나 명령어 실행 시 500 에러 출력 되는 현상이 발생 했습니다.

nova api 버전은 초창기 설치 후 업데이트 진행 하지 않은 버전이고 nova-compute가 설치되는 서버의 버전은 최근 패키지가 설치 되어 패키지 버전이 일치 하지 않는 문제로 발생 했습니다.

운영상 문제는 발생하지 않으나 리사이즈 명령어에만 에러를 출력 하여 api 호출시 에러로 인식하는 문제가 있었습니다.

문제 해결은 python-oslo.messaging  패키지와 nova 관련 패키지 모두 업데이트하고 정상적으로 완료 되었습니다.

 

  • 문제 현상

    # openstack server resize –flavor smile_p1 jyh
    Unexpected API Error. Please report this at http://bugs.launchpad.net/nova/ and attach the Nova API log if possible.
    <class ‘oslo_messaging.exceptions.MessagingTimeout’> (HTTP 500) (Request-ID: req-45132656-9bcb-463f-8660-788a6b2e79b5)
  • nova-api 로그
    # tail -f /var/log/nova/nova-api.log
    2023-02-03 08:09:14.309 3879302 ERROR nova.api.openstack.wsgi [req-45132656-9bcb-463f-8660-788a6b2e79b5 50fa7385402749db8c785a2ba5b507c8 809b9880de354679af831dd6b82f7b81 – default default] Unexpected exception in API method: MessagingTimeout: Timed out waiting for a reply to message ID 4e000b04ff22483fa070835af1f0a654
    2023-02-03 08:09:14.309 3879302 ERROR nova.api.openstack.wsgi Traceback (most recent call last):
    2023-02-03 08:09:14.309 3879302 ERROR nova.api.openstack.wsgi File “/usr/lib/python2.7/dist-packages/nova/api/openstack/wsgi.py”, line 788, in wrapped
    2023-02-03 08:09:14.309 3879302 ERROR nova.api.openstack.wsgi return f(*args, **kwargs)
    2023-02-03 08:09:14.309 3879302 ERROR nova.api.openstack.wsgi File “/usr/lib/python2.7/dist-packages/nova/api/validation/__init__.py”, line 108, in wrapper
    2023-02-03 08:09:14.309 3879302 ERROR nova.api.openstack.wsgi return func(*args, **kwargs)
    2023-02-03 08:09:14.309 3879302 ERROR nova.api.openstack.wsgi File “/usr/lib/python2.7/dist-packages/nova/api/openstack/compute/servers.py”, line 882, in _action_resize
    2023-02-03 08:09:14.309 3879302 ERROR nova.api.openstack.wsgi self._resize(req, id, flavor_ref, **kwargs)
    2023-02-03 08:09:14.309 3879302 ERROR nova.api.openstack.wsgi File “/usr/lib/python2.7/dist-packages/nova/api/openstack/compute/servers.py”, line 802, in _resize
    2023-02-03 08:09:14.309 3879302 ERROR nova.api.openstack.wsgi self.compute_api.resize(context, instance, flavor_id, **kwargs)
    2023-02-03 08:09:14.309 3879302 ERROR nova.api.openstack.wsgi File “/usr/lib/python2.7/dist-packages/nova/compute/api.py”, line 205, in inner
    2023-02-03 08:09:14.309 3879302 ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs)
    2023-02-03 08:09:14.309 3879302 ERROR nova.api.openstack.wsgi File “/usr/lib/python2.7/dist-packages/nova/compute/api.py”, line 213, in _wrapped
    2023-02-03 08:09:14.309 3879302 ERROR nova.api.openstack.wsgi return fn(self, context, instance, *args, **kwargs)
    2023-02-03 08:09:14.309 3879302 ERROR nova.api.openstack.wsgi File “/usr/lib/python2.7/dist-packages/nova/compute/api.py”, line 153, in inner
    2023-02-03 08:09:14.309 3879302 ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw)
    2023-02-03 08:09:14.309 3879302 ERROR nova.api.openstack.wsgi File “/usr/lib/python2.7/dist-packages/nova/compute/api.py”, line 3544, in resize
    2023-02-03 08:09:14.309 3879302 ERROR nova.api.openstack.wsgi request_spec=request_spec)
    2023-02-03 08:09:14.309 3879302 ERROR nova.api.openstack.wsgi File “/usr/lib/python2.7/dist-packages/nova/conductor/api.py”, line 98, in resize_instance
    2023-02-03 08:09:14.309 3879302 ERROR nova.api.openstack.wsgi request_spec=request_spec, host_list=host_list)
    2023-02-03 08:09:14.309 3879302 ERROR nova.api.openstack.wsgi File “/usr/lib/python2.7/dist-packages/nova/conductor/rpcapi.py”, line 337, in migrate_server
    2023-02-03 08:09:14.309 3879302 ERROR nova.api.openstack.wsgi return cctxt.call(context, ‘migrate_server’, **kw)
    2023-02-03 08:09:14.309 3879302 ERROR nova.api.openstack.wsgi File “/usr/lib/python2.7/dist-packages/oslo_messaging/rpc/client.py”, line 174, in call
    2023-02-03 08:09:14.309 3879302 ERROR nova.api.openstack.wsgi retry=self.retry)
    2023-02-03 08:09:14.309 3879302 ERROR nova.api.openstack.wsgi File “/usr/lib/python2.7/dist-packages/oslo_messaging/transport.py”, line 131, in _send
    2023-02-03 08:09:14.309 3879302 ERROR nova.api.openstack.wsgi timeout=timeout, retry=retry)
    2023-02-03 08:09:14.309 3879302 ERROR nova.api.openstack.wsgi File “/usr/lib/python2.7/dist-packages/oslo_messaging/_drivers/amqpdriver.py”, line 559, in send
    2023-02-03 08:09:14.309 3879302 ERROR nova.api.openstack.wsgi retry=retry)
    2023-02-03 08:09:14.309 3879302 ERROR nova.api.openstack.wsgi File “/usr/lib/python2.7/dist-packages/oslo_messaging/_drivers/amqpdriver.py”, line 548, in _send
    2023-02-03 08:09:14.309 3879302 ERROR nova.api.openstack.wsgi result = self._waiter.wait(msg_id, timeout)
    2023-02-03 08:09:14.309 3879302 ERROR nova.api.openstack.wsgi File “/usr/lib/python2.7/dist-packages/oslo_messaging/_drivers/amqpdriver.py”, line 440, in wait
    2023-02-03 08:09:14.309 3879302 ERROR nova.api.openstack.wsgi message = self.waiters.get(msg_id, timeout=timeout)
    2023-02-03 08:09:14.309 3879302 ERROR nova.api.openstack.wsgi File “/usr/lib/python2.7/dist-packages/oslo_messaging/_drivers/amqpdriver.py”, line 328, in get
    2023-02-03 08:09:14.309 3879302 ERROR nova.api.openstack.wsgi ‘to message ID %s’ % msg_id)
    2023-02-03 08:09:14.309 3879302 ERROR nova.api.openstack.wsgi MessagingTimeout: Timed out waiting for a reply to message ID 4e000b04ff22483fa070835af1f0a654
    2023-02-03 08:09:14.309 3879302 ERROR nova.api.openstack.wsgi
    2023-02-03 08:09:14.327 3879302 INFO nova.api.openstack.wsgi [req-45132656-9bcb-463f-8660-788a6b2e79b5 50fa7385402749db8c785a2ba5b507c8 809b9880de354679af831dd6b82f7b81 – default default] HTTP exception thrown: Unexpected API Error. Please report this at http://bugs.launchpad.net/nova/ and attach the Nova API log if possible.
    <class ‘oslo_messaging.exceptions.MessagingTimeout’>
  • 인스턴스 액션 로그(에러로 인식하지 않음…)
    # nova instance-action-list 4e394f5d-bb3f-4cae-8048-8f70cea30e4c
    +—————+——————————————+———+—————————-+
    | Action | Request_ID | Message | Start_Time |
    +—————+——————————————+———+—————————-+
    | create | req-49703ea7-2f4c-40be-aca1-b55f4fcb38c0 | – | 2023-02-02T23:07:50.000000 |
    | resize | req-45132656-9bcb-463f-8660-788a6b2e79b5 | – | 2023-02-02T23:08:14.000000 |
    | confirmResize | req-898a0ea9-60e9-44c2-8606-6f8fb45eb10c | – | 2023-02-02T23:10:10.000000 |
    +—————+——————————————+———+—————————-+

     

  • 설치되어 있는 버전 확인

    # dpkg -l |grep oslo.messaging
    ii python-oslo.messaging 5.35.0-0ubuntu3 all oslo messaging library – Python 2.x

    # dpkg -l |grep nova
    ii nova-api 2:17.0.12-0ubuntu1 all OpenStack Compute – API frontend
    ii nova-common 2:17.0.12-0ubuntu1 all OpenStack Compute – common files
    ii nova-conductor 2:17.0.12-0ubuntu1 all OpenStack Compute – conductor service
    ..
    ..

     

  • 패키지 업데이트
    # apt install python-oslo.messaging

    # apt install nova-api nova-scheduer nova-conductor ….기타 nova 서비스들…

  • 업데이트 확인
    # dpkg -l |grep oslo.messaging
    ii python-oslo.messaging 5.35.0-0ubuntu4 all oslo messaging library – Python 2.x

    # dpkg -l |grep nova
    ii nova-api 2:17.0.13-0ubuntu5 all OpenStack Compute – API frontend
    ii nova-common 2:17.0.13-0ubuntu5 all OpenStack Compute – common files

     

Subscribe
Notify of
guest
0 Comments
Inline Feedbacks
View all comments
0
Would love your thoughts, please comment.x
()
x