1. Fix compile warning on 32bit system. Thanks jwh7 for reporting.
2. Reverted code changes which are not essential part of scheduler code. Eg cpufreq and INITIAL_JIFFIES.
3. Extend BMQ for RT tasks. This is the improvement which make bit-map queues(0-99) for RT tasks(FIFO/RR policy) instead of single double-linked queue in previous releases. Thanks whoever comment this in v5.5-r2 release.
Your feedback will be welcome and helpful for BMQ scheduler development. Next release will be come 2 weeks later.
Enjoy BMQ for your linux kernel.
Full kernel tree repository can be found at https://gitlab.com/alfredchen/linux-bmq
And all-in-one patch can be found at gitlab.
Bug report at https://gitlab.com/alfredchen/bmq/issues
Full kernel tree repository can be found at https://gitlab.com/alfredchen/linux-bmq
And all-in-one patch can be found at gitlab.
Bug report at https://gitlab.com/alfredchen/bmq/issues
Compiled on Ryzen@home, boots fine. Thanks!
ReplyDeleteBUT I have noticed the following:
bmq: cpu#0 llc_id = 0, llc_mask idx = 2
bmq: cpu#1 llc_id = 0, llc_mask idx = 2
bmq: cpu#2 llc_id = 0, llc_mask idx = 2
bmq: cpu#3 llc_id = 0, llc_mask idx = 2
bmq: cpu#4 llc_id = 0, llc_mask idx = 2
bmq: cpu#5 llc_id = 0, llc_mask idx = 2
bmq: cpu#6 llc_id = 0, llc_mask idx = 2
bmq: cpu#7 llc_id = 0, llc_mask idx = 2
bmq: cpu#8 llc_id = 0, llc_mask idx = 2
bmq: cpu#9 llc_id = 0, llc_mask idx = 2
bmq: cpu#10 llc_id = 0, llc_mask idx = 2
bmq: cpu#11 llc_id = 0, llc_mask idx = 2
bmq: cpu#12 llc_id = 0, llc_mask idx = 2
bmq: cpu#13 llc_id = 0, llc_mask idx = 2
bmq: cpu#14 llc_id = 0, llc_mask idx = 2
bmq: cpu#15 llc_id = 0, llc_mask idx = 2
Than can not be true. I should have a map with cpu+llcid from times when I added llc support for MuQSS scheduler if You need to know which CPU is related to which LLC.
BR, Eduardo
@Eduardo
DeleteYes, the llc_id(all zero) is not correct.
Would you pls post a full output of "dmesg | grep -i bmq", the one I have is from older BMQ release.
Here it is:
Delete[ 0.103057] bmq: BMQ CPU Scheduler 5.6-r1 by Alfred Chen.
[ 1.981194] bmq: cpu#0 affinity mask - smt 0x00000100
[ 1.981194] bmq: cpu#0 affinity mask - coregroup 0x00000e0e
[ 1.981194] bmq: cpu#0 affinity mask - core 0x0000f0f0
[ 1.981194] bmq: cpu#0 llc_id = 0, llc_mask idx = 2
[ 1.981194] bmq: cpu#1 affinity mask - smt 0x00000200
[ 1.981194] bmq: cpu#1 affinity mask - coregroup 0x00000d0d
[ 1.981194] bmq: cpu#1 affinity mask - core 0x0000f0f0
[ 1.981194] bmq: cpu#1 llc_id = 0, llc_mask idx = 2
[ 1.981194] bmq: cpu#2 affinity mask - smt 0x00000400
[ 1.981194] bmq: cpu#2 affinity mask - coregroup 0x00000b0b
[ 1.981194] bmq: cpu#2 affinity mask - core 0x0000f0f0
[ 1.981194] bmq: cpu#2 llc_id = 0, llc_mask idx = 2
[ 1.981194] bmq: cpu#3 affinity mask - smt 0x00000800
[ 1.981194] bmq: cpu#3 affinity mask - coregroup 0x00000707
[ 1.981194] bmq: cpu#3 affinity mask - core 0x0000f0f0
[ 1.981194] bmq: cpu#3 llc_id = 0, llc_mask idx = 2
[ 1.981194] bmq: cpu#4 affinity mask - smt 0x00001000
[ 1.981194] bmq: cpu#4 affinity mask - coregroup 0x0000e0e0
[ 1.981194] bmq: cpu#4 affinity mask - core 0x00000f0f
[ 1.981194] bmq: cpu#4 llc_id = 0, llc_mask idx = 2
[ 1.981194] bmq: cpu#5 affinity mask - smt 0x00002000
[ 1.981194] bmq: cpu#5 affinity mask - coregroup 0x0000d0d0
[ 1.981194] bmq: cpu#5 affinity mask - core 0x00000f0f
[ 1.981194] bmq: cpu#5 llc_id = 0, llc_mask idx = 2
[ 1.981194] bmq: cpu#6 affinity mask - smt 0x00004000
[ 1.981194] bmq: cpu#6 affinity mask - coregroup 0x0000b0b0
[ 1.981194] bmq: cpu#6 affinity mask - core 0x00000f0f
[ 1.981194] bmq: cpu#6 llc_id = 0, llc_mask idx = 2
[ 1.981194] bmq: cpu#7 affinity mask - smt 0x00008000
[ 1.981194] bmq: cpu#7 affinity mask - coregroup 0x00007070
[ 1.981194] bmq: cpu#7 affinity mask - core 0x00000f0f
[ 1.981194] bmq: cpu#7 llc_id = 0, llc_mask idx = 2
[ 1.981194] bmq: cpu#8 affinity mask - smt 0x00000001
[ 1.981194] bmq: cpu#8 affinity mask - coregroup 0x00000e0e
[ 1.981194] bmq: cpu#8 affinity mask - core 0x0000f0f0
[ 1.981194] bmq: cpu#8 llc_id = 0, llc_mask idx = 2
[ 1.981194] bmq: cpu#9 affinity mask - smt 0x00000002
[ 1.981194] bmq: cpu#9 affinity mask - coregroup 0x00000d0d
[ 1.981194] bmq: cpu#9 affinity mask - core 0x0000f0f0
[ 1.981194] bmq: cpu#9 llc_id = 0, llc_mask idx = 2
[ 1.981194] bmq: cpu#10 affinity mask - smt 0x00000004
[ 1.981194] bmq: cpu#10 affinity mask - coregroup 0x00000b0b
[ 1.981194] bmq: cpu#10 affinity mask - core 0x0000f0f0
[ 1.981194] bmq: cpu#10 llc_id = 0, llc_mask idx = 2
[ 1.981194] bmq: cpu#11 affinity mask - smt 0x00000008
[ 1.981194] bmq: cpu#11 affinity mask - coregroup 0x00000707
[ 1.981194] bmq: cpu#11 affinity mask - core 0x0000f0f0
[ 1.981194] bmq: cpu#11 llc_id = 0, llc_mask idx = 2
[ 1.981194] bmq: cpu#12 affinity mask - smt 0x00000010
[ 1.981194] bmq: cpu#12 affinity mask - coregroup 0x0000e0e0
[ 1.981194] bmq: cpu#12 affinity mask - core 0x00000f0f
[ 1.981194] bmq: cpu#12 llc_id = 0, llc_mask idx = 2
[ 1.981194] bmq: cpu#13 affinity mask - smt 0x00000020
[ 1.981194] bmq: cpu#13 affinity mask - coregroup 0x0000d0d0
[ 1.981194] bmq: cpu#13 affinity mask - core 0x00000f0f
[ 1.981194] bmq: cpu#13 llc_id = 0, llc_mask idx = 2
[ 1.981194] bmq: cpu#14 affinity mask - smt 0x00000040
[ 1.981194] bmq: cpu#14 affinity mask - coregroup 0x0000b0b0
[ 1.981194] bmq: cpu#14 affinity mask - core 0x00000f0f
[ 1.981194] bmq: cpu#14 llc_id = 0, llc_mask idx = 2
[ 1.981194] bmq: cpu#15 affinity mask - smt 0x00000080
[ 1.981194] bmq: cpu#15 affinity mask - coregroup 0x00007070
[ 1.981194] bmq: cpu#15 affinity mask - core 0x00000f0f
[ 1.981194] bmq: cpu#15 llc_id = 0, llc_mask idx = 2
BR, Eduardo
@Edurado
DeletePls also checkout this quick fix at https://gitlab.com/alfredchen/linux-bmq/-/commit/d394787c8c388b4136358b313568a9ac6d9849bb
This should fix the cpu topology setup in ryzen system, and this bug should be existed for some time. I still need to double check it for some rare cpu topology, may add anther commit to address this. But still, I want this quick fix be out for ryzen users.
Now it shows this:
Delete$ dmesg | grep -i "bmq: cpu"
[ 1.752190] bmq: cpu#00 affinity mask: 0x00000100 - smt
[ 1.752190] bmq: cpu#00 affinity mask: 0x00000e0e - coregroup
[ 1.752190] bmq: cpu#00 affinity mask: 0x0000f0f0 - core
[ 1.752190] bmq: cpu#00 llc_id = 0, llc_mask idx = 1
[ 1.752190] bmq: cpu#01 affinity mask: 0x00000200 - smt
[ 1.752190] bmq: cpu#01 affinity mask: 0x00000d0d - coregroup
[ 1.752190] bmq: cpu#01 affinity mask: 0x0000f0f0 - core
[ 1.752190] bmq: cpu#01 llc_id = 0, llc_mask idx = 1
[ 1.752190] bmq: cpu#02 affinity mask: 0x00000400 - smt
[ 1.752190] bmq: cpu#02 affinity mask: 0x00000b0b - coregroup
[ 1.752190] bmq: cpu#02 affinity mask: 0x0000f0f0 - core
[ 1.752190] bmq: cpu#02 llc_id = 0, llc_mask idx = 1
[ 1.752190] bmq: cpu#03 affinity mask: 0x00000800 - smt
[ 1.752190] bmq: cpu#03 affinity mask: 0x00000707 - coregroup
[ 1.752190] bmq: cpu#03 affinity mask: 0x0000f0f0 - core
[ 1.752190] bmq: cpu#03 llc_id = 0, llc_mask idx = 1
[ 1.752190] bmq: cpu#04 affinity mask: 0x00001000 - smt
[ 1.752190] bmq: cpu#04 affinity mask: 0x0000e0e0 - coregroup
[ 1.752190] bmq: cpu#04 affinity mask: 0x00000f0f - core
[ 1.752190] bmq: cpu#04 llc_id = 4, llc_mask idx = 1
[ 1.752190] bmq: cpu#05 affinity mask: 0x00002000 - smt
[ 1.752190] bmq: cpu#05 affinity mask: 0x0000d0d0 - coregroup
[ 1.752190] bmq: cpu#05 affinity mask: 0x00000f0f - core
[ 1.752190] bmq: cpu#05 llc_id = 4, llc_mask idx = 1
[ 1.752190] bmq: cpu#06 affinity mask: 0x00004000 - smt
[ 1.752190] bmq: cpu#06 affinity mask: 0x0000b0b0 - coregroup
[ 1.752190] bmq: cpu#06 affinity mask: 0x00000f0f - core
[ 1.752190] bmq: cpu#06 llc_id = 4, llc_mask idx = 1
[ 1.752190] bmq: cpu#07 affinity mask: 0x00008000 - smt
[ 1.752190] bmq: cpu#07 affinity mask: 0x00007070 - coregroup
[ 1.752190] bmq: cpu#07 affinity mask: 0x00000f0f - core
[ 1.752190] bmq: cpu#07 llc_id = 4, llc_mask idx = 1
[ 1.752190] bmq: cpu#08 affinity mask: 0x00000001 - smt
[ 1.752190] bmq: cpu#08 affinity mask: 0x00000e0e - coregroup
[ 1.752190] bmq: cpu#08 affinity mask: 0x0000f0f0 - core
[ 1.752190] bmq: cpu#08 llc_id = 0, llc_mask idx = 1
[ 1.752190] bmq: cpu#09 affinity mask: 0x00000002 - smt
[ 1.752190] bmq: cpu#09 affinity mask: 0x00000d0d - coregroup
[ 1.752190] bmq: cpu#09 affinity mask: 0x0000f0f0 - core
[ 1.752190] bmq: cpu#09 llc_id = 0, llc_mask idx = 1
[ 1.752190] bmq: cpu#10 affinity mask: 0x00000004 - smt
[ 1.752190] bmq: cpu#10 affinity mask: 0x00000b0b - coregroup
[ 1.752190] bmq: cpu#10 affinity mask: 0x0000f0f0 - core
[ 1.752190] bmq: cpu#10 llc_id = 0, llc_mask idx = 1
[ 1.752190] bmq: cpu#11 affinity mask: 0x00000008 - smt
[ 1.752190] bmq: cpu#11 affinity mask: 0x00000707 - coregroup
[ 1.752190] bmq: cpu#11 affinity mask: 0x0000f0f0 - core
[ 1.752190] bmq: cpu#11 llc_id = 0, llc_mask idx = 1
[ 1.752190] bmq: cpu#12 affinity mask: 0x00000010 - smt
[ 1.752190] bmq: cpu#12 affinity mask: 0x0000e0e0 - coregroup
[ 1.752190] bmq: cpu#12 affinity mask: 0x00000f0f - core
[ 1.752190] bmq: cpu#12 llc_id = 4, llc_mask idx = 1
[ 1.752190] bmq: cpu#13 affinity mask: 0x00000020 - smt
[ 1.752190] bmq: cpu#13 affinity mask: 0x0000d0d0 - coregroup
[ 1.752190] bmq: cpu#13 affinity mask: 0x00000f0f - core
[ 1.752190] bmq: cpu#13 llc_id = 4, llc_mask idx = 1
[ 1.752190] bmq: cpu#14 affinity mask: 0x00000040 - smt
[ 1.752190] bmq: cpu#14 affinity mask: 0x0000b0b0 - coregroup
[ 1.752190] bmq: cpu#14 affinity mask: 0x00000f0f - core
[ 1.752190] bmq: cpu#14 llc_id = 4, llc_mask idx = 1
[ 1.752190] bmq: cpu#15 affinity mask: 0x00000080 - smt
[ 1.752190] bmq: cpu#15 affinity mask: 0x00007070 - coregroup
[ 1.752190] bmq: cpu#15 affinity mask: 0x00000f0f - core
[ 1.752190] bmq: cpu#15 llc_id = 4, llc_mask idx = 1
I have not checked the code, but llc_mask seems to be the same for all CPUs, is that correct?
BR, Eduardo
In addition (I had to split posts, 4K chars max in one post) LLC seems to be correct except the id. I assume 0/1 or 0/4 does not matter much.
DeleteMy info below is based on cpu_llc_shared_map / per_cpu(cpu_llc_id, CPU), but for kernel 5.2:
possible/present/online CPUs: 32/16/16
CPU 0 llc 0
CPU 1 llc 0
CPU 2 llc 0
CPU 3 llc 0
CPU 4 llc 1
CPU 5 llc 1
CPU 6 llc 1
CPU 7 llc 1
CPU 8 llc 0
CPU 9 llc 0
CPU 10 llc 0
CPU 11 llc 0
CPU 12 llc 1
CPU 13 llc 1
CPU 14 llc 1
CPU 15 llc 1
BR, Eduardo
Looks good, llc id in bmq.c is the first cpu index from cpu_coregroup_mask(), it is happened to be 4 in you system.
DeleteOk, gut :) Are llc_mask idx's correct in my pre-previous post above? They are strangely the same for all CPUs...
DeleteBR, Eduardo
Yes, the llc_mask idx is correct.
DeleteRunning on Arch Linux x86_64, the kernel failed to build without BMQ since v5.6-r0(still patched but not enable). There's the compile error(gcc 9.3.0):
ReplyDelete[ 193s] kernel/sched/core.c: In function 'is_cpu_allowed':
[ 193s] kernel/sched/core.c:1467:6: error: implicit declaration of function 'is_per_cpu_kthread'; did you mean 'is_percpu_thread'? [-Werror=implicit-function-declaration]
[ 193s] 1467 | if (is_per_cpu_kthread(p))
[ 193s] | ^~~~~~~~~~~~~~~~~~
[ 193s] | is_percpu_thread
I have compiled BMQ for 5.6 in Ubuntu 16.04.x as well as for ArchLinux, no problems at all.
DeleteMaybe there's a bug, when BMQ patchset is applied but not enabled, I have not tried to compile like that.
If this is the issue, I would say it needs fixing, but there is a workaround - do not patch kernel with BMQ or if You use pf-kernel from gitlab along with other goodies, just revert specifically BMQ using Alfred's all-in-one patch :)
BR, Eduardo
Please try this fix: https://gitlab.com/snippets/1963674
Delete@Eduardo
DeleteThanks for reply and workaround. This can only happen when patchset is applied but not enabled.I have reenabled this patch and built without error. The latest release has some improvement for RT tasks which I would like to test.
@Oleksandr
Thanks for the fix, it works great.
@Alfred
Thank you for continuously improving BMQ.
Thanks again :)
Thanks for reporting and @pf's quick fix patch. It's a cherry-pick mistake during sync-up.
DeleteI have pushed my fix commit at https://gitlab.com/alfredchen/linux-bmq/-/commit/70ef06c844ba112020e548691bffb2b8da70602b