장영호intel cpu의 경우 socket 당 core, core 당 thread로 cpu 넘버를 가지게 됩니다.
socket이 1개고 socket당 2개의 core, core당 4 thread라고 하면 1x2x4 해서 8개의 cpu를 가지게 됩니다.
intel cpu로 호스팅을 한다고 가정시 같은 core에 가상 서버가 상주할시 다른 서버에 영향 및 상관 관련 테스트를 진행 하겠습니다.
- 장비 스펙 확인
root@intel-node:~# lscpu
Architecture: x86_64
CPU op-mode(s): 32-bit, 64-bit
Byte Order: Little Endian
CPU(s): 8
On-line CPU(s) list: 0-7
Thread(s) per core: 2
Core(s) per socket: 4
Socket(s): 1
NUMA node(s): 1
Vendor ID: GenuineIntel
CPU family: 6
Model: 42
Model name: Intel(R) Xeon(R) CPU E31230 @ 3.20GHz
Stepping: 7
CPU MHz: 1599.875
CPU max MHz: 3600.0000
CPU min MHz: 1600.0000
BogoMIPS: 6385.52
Virtualization: VT-x
L1d cache: 32K
L1i cache: 32K
L2 cache: 256K
L3 cache: 8192K
NUMA node0 CPU(s): 0-7
Flags: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi
mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good
nopl xtopology nonstop_tsc aperfmperf eagerfpu pni pclmulqdq dtes64 monitor ds_cpl vmx smx est
tm2 ssse3 cx16 xtpr pdcm pcid sse4_1 sse4_2 x2apic popcnt tsc_deadline_timer xsave avx lahf_lm
epb tpr_shadow vnmi flexpriority ept vpid xsaveopt dtherm ida arat pln pts
# 1소켓에 소켓당 core가 4개로 총 4개의 core이고 core당 2개의 thread로 총 8 thread 사양이다.
root@intel-node:~# cat /sys/devices/system/cpu/cpu0/topology/core_id
0
root@intel-node:~# cat /sys/devices/system/cpu/cpu1/topology/core_id
1
root@intel-node:~# cat /sys/devices/system/cpu/cpu2/topology/core_id
2
root@intel-node:~# cat /sys/devices/system/cpu/cpu3/topology/core_id
3
root@intel-node:~# cat /sys/devices/system/cpu/cpu4/topology/core_id
0
root@intel-node:~# cat /sys/devices/system/cpu/cpu5/topology/core_id
1
root@intel-node:~# cat /sys/devices/system/cpu/cpu6/topology/core_id
2
root@intel-node:~# cat /sys/devices/system/cpu/cpu7/topology/core_id
3
# core_id로 어느 core에 있는지 확인 된다.
root@intel-node:~# cat /sys/devices/system/cpu/cpu0/topology/thread_siblings_list
0,4
root@intel-node:~# cat /sys/devices/system/cpu/cpu1/topology/thread_siblings_list
1,5
root@intel-node:~# cat /sys/devices/system/cpu/cpu2/topology/thread_siblings_list
2,6
root@intel-node:~# cat /sys/devices/system/cpu/cpu3/topology/thread_siblings_list
3,7
root@intel-node:~# cat /sys/devices/system/cpu/cpu4/topology/thread_siblings_list
0,4
root@intel-node:~# cat /sys/devices/system/cpu/cpu5/topology/thread_siblings_list
1,5
root@intel-node:~# cat /sys/devices/system/cpu/cpu6/topology/thread_siblings_list
2,6
root@intel-node:~# cat /sys/devices/system/cpu/cpu7/topology/thread_siblings_list
3,7
# core에 할당된 thread 번호가 출력된다.
- 동일 core에 서버 생성
root@controller:~# nova list
+--------------------------------------+---------+--------+------------+-------------+------------------------------------+
| ID | Name | Status | Task State | Power State | Networks |
+--------------------------------------+---------+--------+------------+-------------+------------------------------------+
| 91820be1-d01a-45f0-90ad-bb0564dc1d2a | 2core-1 | ACTIVE | - | Running | jyh1=192.168.1.155, xxx.xxx.xxx.233 |
| 5bcee739-fb00-423f-aaf1-399b0d9909ae | 2core-2 | ACTIVE | - | Running | jyh1=192.168.1.159, xxx.xxx.xxx.234 |
+--------------------------------------+---------+--------+------------+-------------+------------------------------------+
root@intel-node:/var/lib/nova/instances# cat 5bcee739-fb00-423f-aaf1-399b0d9909ae/libvirt.xml |grep cpuset
<emulatorpin cpuset="5"/>
<vcpupin vcpu="0" cpuset="5"/>
root@intel-node:/var/lib/nova/instances# cat 91820be1-d01a-45f0-90ad-bb0564dc1d2a/libvirt.xml |grep cpuset
<emulatorpin cpuset="1"/>
<vcpupin vcpu="0" cpuset="1"/>
# cpu가 1,5 할당 확인.
- cpu 부하 없을시 sysbench 측정
root@cpu2-2:~# sysbench --num-threads=1 --test=cpu --cpu-max-prime=20000 --validate run
sysbench 0.4.12: multi-threaded system evaluation benchmark
Running the test with following options:
Number of threads: 1
Additional request validation enabled.
Doing CPU performance benchmark
Threads started!
Done.
Maximum prime number checked in CPU test: 20000
Test execution summary:
total time: 24.7478s
total number of events: 10000
total time taken by event execution: 24.7458
per-request statistics:
min: 2.40ms
avg: 2.47ms
max: 17.49ms
approx. 95 percentile: 2.58ms
Threads fairness:
events (avg/stddev): 10000.0000/0.00
execution time (avg/stddev): 24.7458/0.00
# 24초 확인.
- 동일 코어에 할당된 서버에 cpu 부하 줄때 sysbench 측정
root@cpu2-1:~# stress --cpu 1 --timeout 3600 &
[1] 7435
root@cpu2-1:~# top
top - 15:00:28 up 28 min, 1 user, load average: 0.76, 0.25, 0.09
Tasks: 106 total, 2 running, 104 sleeping, 0 stopped, 0 zombie
%Cpu(s): 98.3 us, 0.3 sy, 0.0 ni, 0.0 id, 0.0 wa, 0.0 hi, 0.0 si, 1.3 st
KiB Mem : 1016064 total, 323020 free, 49804 used, 643240 buff/cache
KiB Swap: 0 total, 0 free, 0 used. 781088 avail Mem
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
7436 root 20 0 7480 92 0 R 99.9 0.0 1:24.61 stress
1 root 20 0 37920 6020 4040 S 0.0 0.6 0:01.27 systemd
2 root 20 0 0 0 0 S 0.0 0.0 0:00.00 kthreadd
3 root 20 0 0 0 0 S 0.0 0.0 0:00.05 ksoftirqd/0
# 가상 서버 두개중 한 서버에서 cpu 부하 발생
root@cpu2-2:~# sysbench --num-threads=1 --test=cpu --cpu-max-prime=20000 --validate run
sysbench 0.4.12: multi-threaded system evaluation benchmark
Running the test with following options:
Number of threads: 1
Additional request validation enabled.
Doing CPU performance benchmark
Threads started!
Done.
Maximum prime number checked in CPU test: 20000
Test execution summary:
total time: 31.2511s
total number of events: 10000
total time taken by event execution: 31.2486
per-request statistics:
min: 3.05ms
avg: 3.12ms
max: 4.52ms
approx. 95 percentile: 3.22ms
Threads fairness:
events (avg/stddev): 10000.0000/0.00
execution time (avg/stddev): 31.2486/0.00
# 7초 가량 delay 발생.
※ 결론
동일 core에 가상 서버를 생성하여 부하를 주고 다른 서버에 영향을 미치는지 테스트 하였는데 dedicated로 cpu를 분리 하였지만 동일 core에 있는 서버로 성능에 영향을 미쳤습니다.
호스팅을 목적으로 한다면 cpu topology를 계획하여 서버간에 영향없이 운영 되도록 해야 합니다.