Monday, April 20, 2020

BMQ v5.6-r3 release

BMQ v5.6-r3 is released with the following changes

1. Rework codes in __schedule() and sched_fork(). These are minor changes, but will somehow help to reduce overhead.

Enjoy BMQ for your linux kernel.

Full kernel tree repository can be found at https://gitlab.com/alfredchen/linux-bmq
And all-in-one patch can be found at gitlab.

Bug report at https://gitlab.com/alfredchen/bmq/issues

21 comments:

  1. Thank you very much.

    ReplyDelete
  2. When I used CK1 patch (5.5) all cores would be maxed out at 100% during kernel compiling.
    With this BMQ patch they will be around 50% max.
    Kernel compiling takes much longer of course.

    ReplyDelete
    Replies
    1. Did this happen with previous patch?
      Previous kernel and patch?
      What is the CPU?
      What is the OS and it's version?

      Delete
    2. Yes.
      Yes.
      Xeon E5620.
      Slackware-current.
      It doesn't happen with vanilla or ck/muqss.

      Delete
    3. Pls post "dmesg | grep -i bmq" output on a 5.6 BMQ kernel, I'd check the topology setup.
      During kernel compiling, all 4 cores have cpu load?!

      Delete
    4. [ 0.108934] bmq: BMQ CPU Scheduler 5.6-r3 by Alfred Chen.
      [ 0.353333] bmq: cpu#00 affinity mask: 0x0000000e - coregroup
      [ 0.353334] bmq: cpu#00 llc_id = 0, llc_mask idx = 0
      [ 0.353419] bmq: cpu#01 affinity mask: 0x0000000d - coregroup
      [ 0.353420] bmq: cpu#01 llc_id = 0, llc_mask idx = 0
      [ 0.353500] bmq: cpu#02 affinity mask: 0x0000000b - coregroup
      [ 0.353501] bmq: cpu#02 llc_id = 0, llc_mask idx = 0
      [ 0.353581] bmq: cpu#03 affinity mask: 0x00000007 - coregroup
      [ 0.353582] bmq: cpu#03 llc_id = 0, llc_mask idx = 0

      Yes all cores have load but only 50% on average max.

      Delete
    5. Gcc is only one thread despite putting "-j 4" into MAKEFLAGS.

      Delete
    6. I used the additional parameter -l now all cores are used.
      export MAKEFLAGS="-j -l $(getconf _NPROCESSORS_ONLN)"
      The low priority issue still persists though.

      Delete
    7. I am a little confused here, the last anonymous are the same person?
      And what the low priority issue is?

      Delete
    8. Yes same.
      I think everything is resolved now.

      Delete
    9. Good to know you resolve it. :)

      Delete
  3. Responsiveness is amazing.

    ReplyDelete
  4. Alfred, this has been bothering me for a long time, but finally I've decided to report it to you.

    Given I have 4 CPUs. I run foldingathome (or another CPU-bound app) with idle priority. It maxes out to all CPUs and performs fine. Then I decide to run KVM VM with higher than idle priority, but the qemu process stalls on the RCU synchronization point (a) (please see the stacktrace via link below [1]).

    At the same time, RCU GP kworker eats 100% of CPU time doing nothing. I either see en empty stack, or stack (b), or stack (c).

    Once I send a SIGSTOP to foldingathome aka idle app, things get back to normal, and qemu starts fine.

    Do you think this is some kind of priority inversion issue? Can you reproduce it locally?

    Thanks.

    [1] https://gitlab.com/snippets/1968570

    ReplyDelete
    Replies
    1. Basically, it seems it doesn't even need different priorities. I was able to reproduce it with a couple of parallel dd from /dev/zero to /dev/null, and all dd and qemu had default priorities.

      So I guess the thing that matters is being CPU-bound.

      Delete
    2. I also noticed a priority issue during my kernel compiling^^ htop shows a unusual high amount of "low priority" in the cpu usage bar.
      https://i.imgur.com/QM3ykfH.png

      Delete
    3. @Anonymous
      That's result of boost priority auto adjustment in BMQ has kicked in. When doing kernel compiling, compiler tasks usually used full time slice then they get de-boost to lower priority, while other tasks which use short time slice or sleep for a long time will get boost to higher priority.

      Delete
    4. @pf
      I may need some time to reproduce it locally. Would you pls report a bug so I can trace it easier(and keep the record too).

      Delete
    5. Reported here: https://gitlab.com/alfredchen/bmq/-/issues/16

      Delete
    6. @Oleksandr & @Alfred:
      Besides your discussion in https://gitlab.com/alfredchen/bmq/-/issues/16,
      I find the settings of "threadirqs rcu_nocbs=0-Mx io_delay=none" (with Mx == max CPU No.) quite interesting to test with BMQ. In my subjective experience, it looks like to be beneficial to interactiveness.

      Many thanks to both of you for your thorough work for Linux,
      best regards,

      Manuel

      Delete
    7. Those settings should works for all schedulers, not only BMQ.

      Delete
    8. :-)
      Yes, of course. I just wanted to get it out of the cave (the outsourced bug discussion)...
      In case someone else on here wants to play with one or more of these settings.

      Manuel

      Delete