Wednesday, April 4, 2018

PDS 0.98m release

PDS 0.98m is released with the following changes

1. Unify 32/64bits handing in task_preemptible_rq(). This makes 32/64bits system has the same code base(again). Personally, I'd love to see my raspberry pi running same scheduler code with my laptop.
2. Remove cpu scaling interfaces. Based on recent testing, there is no regression to drop these interfaces. Removing these also cut of overhead.
3. Accurate preempt for RT tasks. Before this release, RT tasks wake up on 32bits system is not fully based on their priority(kind of a bug).
4. Refines and code clean-up in several functions.
5. Fix some wrong likely/unlikely usage in PDS code.

These changes are inspired by the experimental cachehot patch. Aim to reduce the overhead in the existed code path and make the design simple. Another obvious change is in the patch size, -5KB!(No one cares, right?) For the sanity tests, there is slight improvement, no regression was found so far.

Enjoy PDS 0.98m for v4.16 kernel, :)

code are available at
https://bitbucket.org/alfredchen/linux-gc/commits/branch/linux-4.16.y-pds
and also
https://github.com/cchalpha/linux-gc/commits/linux-4.16.y-pds

 All-in-one patch is available too.

PS, PDS-mq patch repository is created, which will be used to category all pds-mq patch since 4. 13. I am thinking about migrate the source code/patch repository from bitbucket to github, because bitbucket used to have limit repository size(2G), but it looks like that they remove such limitation recently, what about your thought?

23 comments:

  1. Building...
    Is the cachehot patch work included in this patch?

    ReplyDelete
    Replies
    1. Up and running... All good...
      Good work.

      Delete
    2. None, cachehot will create another new branch in the code path, but these commits mainly improve existed code path by cutting overhead.

      Delete
  2. @Alfred:
    Also on here, the 0.98m is running well (after I managed to remove the obstacle).
    Fortunately the TOI patch for 4.15 also works for 4.16 without editing (unfortunately all it's limitations remain).

    Good work from yours, thank you! Now I'm curious for your next hot CachHot patch. :-))

    BR, Manuel Krause

    ReplyDelete
  3. Regarding your host migration plans:

    I've also seen the warning messages from your bitbucket about the 2G limitation, but the've gone away some weeks/ months ago. Anyway, unless there is a specific reason to maintain two separate repos, it'll primarily save you time to only have the github. The missing patch folder now is also in place there, so there's no visible reason to me to not go that way.

    BR, Manuel Krause

    ReplyDelete
    Replies
    1. Current status of the cachehot patch is, it is hard to tune the parameters to get benifit as the overhead already been reduced in 098m. I will see how it goes in the incoming weeks.

      Delete
    2. Just feel free to offer experimental CacheHot stuff to test -- even if you don't consider it perfect.

      Your last formula together with your hints really wasn't bad at all for my sytem. With 4.15.xy I also tested a version with SCHED_CACHE_HOT_SWITCHES_TH == 12, where you had suggested 13 for me, and at least one "subsystem" didn't like it, namely TOI, whereas with 13 I only saw improvements (subjectively).

      And, yes, the current PDS for 4.16 is also still running fine. :-)

      BR, thank you for and keep up your good work and brilliant ideas coming,
      Manuel Krause

      Delete
  4. > Another obvious change is in the patch size, -5KB!(No one cares, right?)
    Maybe you should have named this release "PDS 0.98lite" Or "Diet PDS" :-P
    Thanks Alfred; will build x64 and x86-UP tonight.

    ReplyDelete
    Replies
    1. Both built and running; no problems.

      Delete
    2. Or e.g. "PDSlim" ;-)
      OMG, together with a new Cache HOT patch... what naming would be still conforming to conventions ^^
      BR, Manuel Krause

      Delete
  5. @Alfred,

    compiled & running both on i7 and on Ryzen, no problems so far. System seems snappier than before, at least that's the feeling.

    Thanks for Your efforts ;)

    BR, Eduardo

    ReplyDelete
    Replies
    1. My system also looks snappier but I'm very sceptic. I would like to have a benchmark that give me numbers to compare with stock kernel.
      What tools can be used to measure the improvements by PDS? I haven't found a reproducible way of measuring it... Any ideas?

      Delete
    2. @Anonymous:
      Don't be too sceptic -- just trust your feelings... ;-)

      I don't know what you want to compare so that we don't end up with apples and oranges.
      For an overall throughput oriented testing the phoronix test suite may be worth a try.
      Also, Alfred published his sanity test scripts he used some time ago. Don't know if it's up-to-date any more, it's from 2015 [https://bitbucket.org/alfredchen/linux-gc/downloads/sanity AND https://bitbucket.org/alfredchen/linux-gc/downloads/compile_throughput].

      And then, going to interactivity/ low-latency comparisons it'll become more and more difficult. IIRC, Con Kolivas had provided something related, years ago. ATM I'm unable to recall it.

      BR, Manuel Krause

      Delete
    3. @Anonymous:
      Oh, I just remebered it: "interbench". You can search Con's Blog for it, e.g. http://ck-hack.blogspot.de/search?q=benchmark&max-results=20&by-date=true
      or some search like that.

      But I have to add, that I have no experience with it and it's usefulness at all.

      BR, Manuel Krause

      Delete
    4. @Anonymous:
      Also, Pedro shows up on here quite often. Thanks for that!
      His most recent benchmark is: https://docs.google.com/spreadsheets/d/163U3H-gnVeGopMrHiJLeEY1b7XlvND2yoceKbOvQRm4/edit#gid=857299933

      In it's info tab you can see how he composes and describes his benchmarks.

      BR, Manuel Krause

      Delete
  6. Thanks Alfred, I fixed and tested on mainline (4.17). It works well.
    The patch i sent you to the email.

    ReplyDelete
    Replies
    1. It's still too early to port to 4.17. :)

      Delete
    2. There is no choice, I have working on this branch. :)

      Delete
  7. Hi All,

    for my own convenience I've created an AUR package in the same spirit as linux-ck, but with the PDS scheduler instead.

    If anyone is interested, you can check it at http://github.com/pinkfluid/linux-pds

    By default it builds a version optimized for the current CPU. If you want to build a generic kernel (or for a different CPU architecture), use the FLAVOR=generic (or FLAVOR=skylake ...etc).

    Quick build instructions:

    # git clone http://github.com/pinkfluid/linux-pds
    # cd linux-pds
    # FLAVOR=generic makepkg -ci

    As I don't have much free time lately, please feel free to clone and improve it as I will have very little to no time to support it.

    ReplyDelete
  8. Thank you, I will try to use your PKGBUILD (or improve mine), it looks neatly done. I have been making my own, uggly version for Manjaro.

    Dzon

    ReplyDelete
  9. You can use my published to AUR package
    https://aur.archlinux.org/packages/linux-gc/

    ReplyDelete
  10. Hi artafinde,

    If I knew about your package before I wouldn't have bothered with mine. I see you have some changes from -ck, like disabling NUMA.

    I like the target CPU auto detection of my patch because I usually build a new kernel on 3+ machines, each with a different CPU -- is there a way you can incorporate this into your patch? Though, since it renames the package I'm not sure if it breaks AUR managers in some way or not...


    Regards,
    Mitja Horvat


    ReplyDelete