Thursday, November 23, 2017

PDS 0.98g release

PDS 0.98g is released with the following changes

1. Fix rq->online is default false for cpu 0, issue reported by Manuel.
2. Remove update_rq_clock() in hrtick(), it's a minor enhancement.

This is a bug fix/enhancement release. After bug fixes for the new features in the recent releases, I have done a kernel compilation benchmark in 4.14. The results are as expected, but it also indicate that the overhead under high workload is larger than CFS. So, reduce scheduling/balance/migration overhead will the next feature in the todo list. But don't expect such features will be coming soon, as current PDS is stable, I'll tend to slow down and do more research before rolling out new features.

One more thing, current release of PDS is running well on raspberry pi, you can have a try if you own a raspberry pi soc.

Anyway, enjoy PDS 0.98g for v4.14 kernel, :)

code are available at
https://bitbucket.org/alfredchen/linux-gc/commits/branch/linux-4.14.y-pds
and also
https://github.com/cchalpha/linux-gc/commits/linux-4.14.y-pds

 All-in-one patch is available too.

Thursday, November 16, 2017

PDS 0.98f release

PDS 0.98f is released with the following changes

1. Fix set task cpu to offline cpu warning, reported by pf.
2. Fix long existed task schedule runtime accounting bug, reported by Eduardo.
3. Remove yield support, as planned.

It is too good to have no issue in my list. Enjoy PDS 0.98f for v4.14 kernel, :)

code are available at
https://bitbucket.org/alfredchen/linux-gc/commits/branch/linux-4.14.y-pds
and also
https://github.com/cchalpha/linux-gc/commits/linux-4.14.y-pds

 All-in-one patch is available too.

Monday, November 13, 2017

PDS 0.98e release

PDS 0.98e is released with the following changes

1. Sync with 4.14 mainline scheduler changes.

This is sync-up release for 4.14. The remain known issue is the task accounting issue reported by Edurado, hopefully be fixed in this release.

Enjoy PDS 0.98e for v4.14 kernel, :)

code are available at
https://bitbucket.org/alfredchen/linux-gc/commits/branch/linux-4.14.y-pds
and also
https://github.com/cchalpha/linux-gc/commits/linux-4.14.y-pds

 All-in-one patch is available too.

Friday, October 27, 2017

PDS 0.98d release

PDS 0.98d is released with the following changes

1. Rework next_balance for balancing overhead cutting. Which fix task imbalance issue reported by Manuel.

This is a bug fix release, and will be the last one before next kernel release.

Enjoy PDS 0.98d for v4.13 kernel, :)

code are available at
https://bitbucket.org/alfredchen/linux-gc/commits/branch/linux-4.13.y-vrq
and also
https://github.com/cchalpha/linux-gc/commits/linux-4.13.y-vrq

 All-in-one patch is available too.

Sunday, October 22, 2017

PDS 0.98c release

PDS 0.98c is released with the following changes

1.Refine __sched_setscheduler().
2.Task deadline catch-up algorithm V3, which just apply catch-up algorithm for NORMAL policy tasks.
3.Adjust next_balance value and Fix task balance with low HZ system. (Task policy fairness imbalance issue reported by Manuel still under investigation)
4.Set default yield_type to 0. Help with wine running which use yield APIs. Yield support in PDS will be removed if no complains.

This is a bug fix release, hopefully it helps with compatibility and stability.

Enjoy PDS 0.98c for v4.13 kernel, :)

code are available at
https://bitbucket.org/alfredchen/linux-gc/commits/branch/linux-4.13.y-vrq
and also
https://github.com/cchalpha/linux-gc/commits/linux-4.13.y-vrq

 All-in-one patch is available too.

Tuesday, October 10, 2017

PDS 0.98b release

PDS 0.98b is released with the following changes

1. UP compilation fix.
2. Task deadline catch-up algorithm V2. A simple deadline catch-up algorithm, give less cpu usage task earlier deadline when it is woken up. This will help for interactivity.
3. Sync up for 4.13 mainline release. Hopefully help with suspend/resume stability.
4. Fix a long existed hidden sync-up issue of get_user_cpu_mask().
5. Minor improvement.

This is a bug fix release, hopefully it will solve the issues reported in previous release.

Enjoy PDS 0.98b for v4.13 kernel, :)

code are available at
https://bitbucket.org/alfredchen/linux-gc/commits/branch/linux-4.13.y-vrq
and also
https://github.com/cchalpha/linux-gc/commits/linux-4.13.y-vrq

 All-in-one patch is available too.

Saturday, September 30, 2017

PDS 0.98a release

PDS 0.98a is released with the following changes

1. Fix calculation mistake in task_deadline_level() in previous release.
2. Reduce policy fairness balance overhead when task_deadline_level() calcalation is corrected.
3. Refine policy fairness balance.
4. For 32bits kernel, remove a global lock accessing by only preempt lower scheduling level run queue. (32bits Raspberry PI should get some love)
5. Extend one more NORMAL policy deadline level.
6. Fix reverted task policy value.

This is a bug fix release plus some enhancement. Compare to previous release, there is some performance regression in exchange for some interactivity improvment. Now, all design should work as expected.

Enjoy PDS 0.98a for v4.13 kernel, :)

code are available at
https://bitbucket.org/alfredchen/linux-gc/commits/branch/linux-4.13.y-vrq
and also
https://github.com/cchalpha/linux-gc/commits/linux-4.13.y-vrq

 All-in-one patch is available too.

Wednesday, September 20, 2017

PDS 0.98 release

PDS 0.98 is released with the following changes

1. Renamed. As planned, the scheduler is proper described as Priority and Deadline based Skiplist multiple queue Scheduler, in short, PDS-mq or just PDS. Documentation/scheduler/sched-PDS-mq.txt has been added to document the PDS, but it is not finished.
2. Fix UP compilation issue in previous release, reported by jwh7.
3. re-queue task when priority/deadline changed, which fix tasks out of order issue in run queue and cause a WARNING kernel log.
4. Minor code improvements.
5. Skiplist randomization when task burst forking, this help with skiplist level randomization when forking tasks in a very short time frame.

This release is mainly for bug fix and rename. Now there is over 200+ commits for PDS and the scheduler is consider stable in this kernel release, so in next kernel release, it is planned to squash commits into one or just few for better maintenance.

Enjoy the brand new (named) PDS 0.98 for v4.13 kernel, :)

code are available at
https://bitbucket.org/alfredchen/linux-gc/commits/branch/linux-4.13.y-vrq
and also
https://github.com/cchalpha/linux-gc/commits/linux-4.13.y-vrq

(Yes, still use old branch name, will be renamed in 4.14)

 All-in-one patch is available too.


PS:
Found a task deadline level calculation mistaken on a debug load today, please consider to pick up the fix at https://bitbucket.org/alfredchen/linux-gc/commits/543de0b70aed7785c226ad65a39366c80f15711b

Friday, September 8, 2017

VRQ 0.97b release

VRQ 0.97b is released with the following changes

1. Select a random rq when no preemptible rq available. This will help with removing the bottleneck when cpu number increased.
2. Add rr_interval kernel parameter. Now an "rr_interval=" kernel parameter is added, but it's not suggested to change the default rr_interval setting(6ms).
3. Introduce sched_prio_to_deadline[NICE_WIDTH] to simplify deadline calculation.
4. Extend NORMAL policy level to 7 levels, this change help with reducing rq look up cost when cpu number increased. And it also brings deadline fairness for different nice level NORMAL policy tasks. eg, when two nice 19 background tasks runs with 2 nice 0 front-ground tasks in a two cpus system, those two background tasks may both be on same cpu while normal tasks occupied another. In previous release, as a workaround for this use case, the background tasks need to be IDLE policy to trigger the policy fairness balance functionality. With this new changes, there is 7 NORMAL policy levels according to task's deadline, so the nice 19 background tasks(which likely have large virtual deadline) also can be triggered for the policy fairness balance functionality.

The main features for 0.97? release has been all set now for 4.13 kernel, these feature code has been running on my machines for 2week+, so they are consider stable. In next release, the major change will be renaming and documentation.

Enjoy VRQ 0.97a for v4.13 kernel, :)

code are available at
https://bitbucket.org/alfredchen/linux-gc/commits/branch/linux-4.13.y-vrq
and also
https://github.com/cchalpha/linux-gc/commits/linux-4.13.y-vrq

All-in-one patch is available too.

Monday, September 4, 2017

VRQ 0.97a released

VRQ 0.97a is released with the following changes

1. Sync-up 4.13 mainline scheduler code changes.
2. Fix cpu preempt race in task_preemptiable_rq(), reported by Eduardo in D3 wine playing.

This is a sync-up and minor bug fix release for 4.13 kernel. If all goes well, new feature will be in next week.

Enjoy VRQ 0.97a for v4.13 kernel, :)

code are available at
https://bitbucket.org/alfredchen/linux-gc/commits/branch/linux-4.13.y-vrq
and also
https://github.com/cchalpha/linux-gc/commits/linux-4.13.y-vrq

All-in-one patch is available too.

Wednesday, August 16, 2017

VRQ 0.97 released

VRQ 0.97 is released with the following changes

1. Fix UP compilation issue, reported by jwh7.
2. rtmutex deadline adaption for VRQ.
3. Overhead reduction in take_other_rq_task() and SMT code path in task_preemptible_rq().
4. Preparation for Normal policy level expansion feature.

Version bump up to 0.97 because smt sensitive scheduling, the main feature in 0.96 has finally been accomplished. In 0.97, normal policy level expansion and task deadline adjustment are the main features. In this release, just the preparation code of normal policy level expansion is included, the prototype code still under testing.

And, there will be no more release before 4.13 kernel come out.

Enjoy VRQ 0.97 for v4.12 kernel, :)

code are available at
https://bitbucket.org/alfredchen/linux-gc/commits/branch/linux-4.12.y-vrq
and also
https://github.com/cchalpha/linux-gc/commits/linux-4.12.y-vrq

All-in-one patch is available too.

Monday, August 14, 2017

Get ready in "Multi-Core War"

The "Multi-Core War", which was expected to happen soon when first dual core cpu came to pc world decade ago, does not happen due to a lot of reason, unlike the cpu in mobile world, cpu number grows up rapidly in 2~3 years, your mobile phone may has more cpus than what your working pc has. But thanks to AMD, Intel is finally moving his step to the war zone. Likely, customer in mobile platform or desktop platform would get 4cores/8threads and even more core/thread cpu in reasonable price at the end of this year or next year.

Meanwhile, the task scheduler in OS, should be prepared to take advantage of new hardware in this "Multi-Core War". VRQ, in particular, was applied with a lot of cpumask based algorithm, which is scalable with the growing cpus, but there are still some bottlenecks which has to loop over each cpu. In the incoming release of VRQ, I will try to address these bottlenecks and hopefully play a better position in the "War".

PS, as you may noticed that the name of VRQ(/BFS) is now not longer suit for the design of this scheduler code, and it should deserve a proper name, but it is the last thing in my list(feature >> naming), :)

Wednesday, August 2, 2017

VRQ 0.96f release

VRQ 0.96f is released with the following changes

1. Improvement for smt sensitive scheduling.
2. No dequeue/enqueue for task on cpu.
3. Code clean up.


#2 is the major change in this release, unlike the previous version, now when task is going to be run on a cpu, there is no need to dequeue the task from the run queue skip list, enqueue/dequeue will only happen when task become runnable/unrunnable. With this change, the overhead of enqueue/dequeue is significantly reduced, and the logic of scheduler is more simple.

And please also be noticed, to adapt to this new enqueu/dequeue strategy, some important code path has been rewriten. You may experience unstable issues with this release. (So far, none for me at least)

Enjoy VRQ 0.96f for v4.12 kernel, :)

code are available at
https://bitbucket.org/alfredchen/linux-gc/commits/branch/linux-4.12.y-vrq
and also
https://github.com/cchalpha/linux-gc/commits/linux-4.12.y-vrq

All-in-one patch is available too.

Wednesday, July 26, 2017

VRQ 0.96e release

VRQ 0.96e is released with the following changes

1. Remove unused run_list in task_struct.
2. Fix x32 compilation error. Reported and fixed by pf.
3. Sync-up try_to_wake_up_local().
4. cpu affinity fix. Reported by pf.

This is bug fix release.

Enjoy VRQ 0.96e for v4.12 kernel, :)

code are available at
https://bitbucket.org/alfredchen/linux-gc/commits/branch/linux-4.12.y-vrq
and also
https://github.com/cchalpha/linux-gc/commits/linux-4.12.y-vrq

All-in-one patch is available too.

BR Alfred 

Sunday, July 9, 2017

VRQ 0.96d release

VRQ 0.96d is released with the following changes

1. smt sensitive scheduling improvement, which reduce some migration overhead.
2. Fix livepatch compilation issue.

This is bug fix and smt sensitive scheduling improvement release.

Enjoy VRQ 0.96d for v4.12 kernel, :)

code are available at
https://bitbucket.org/alfredchen/linux-gc/commits/branch/linux-4.12.y-vrq
and also
https://github.com/cchalpha/linux-gc/commits/linux-4.12.y-vrq

All-in-one patch is available too.

BR Alfred 

Tuesday, July 4, 2017

VRQ 0.96c release

VRQ 0.96c is released with the following changes

1. Sync up mainline scheduler changes in 4.12
2. Clean up herder files.

This is just a sync-up release to bring VRQ to 4.12 kernel. Not new feature code is added in VRQ itself.

Enjoy VRQ 0.96c for v4.12 kernel, and unlock your SMT cpu ability with VRQ, :)

code are available at
https://bitbucket.org/alfredchen/linux-gc/commits/branch/linux-4.12.y-vrq
and also
https://github.com/cchalpha/linux-gc/commits/linux-4.12.y-vrq

All-in-one patch is available too.

BR Alfred 

Monday, July 3, 2017

VRQ 0.96b release

VRQ 0.96b is released with the following changes

1. smt sensitive scheduling v0.2, which fix issue with NR_CPUS > real cpu cores

Thank all for testing and finally bring SMT sensitive scheduling feature to VRQ. Feel free to test this release comparing to previous and enjoy the pleasure of unlocking the SMT capability, :)

I'm planing further improvement for SMT sensitive scheduling. Currently, I am not happy with implement code, there are duplicated code with migration code, and others. And it's late in this kernel release and time to look at sync-up works in 4.12 mainline.

Enjoy VRQ 0.96b for v4.11 kernel, and unlock your SMT cpu ability with VRQ, :)

code are available at
https://bitbucket.org/alfredchen/linux-gc/commits/branch/linux-4.11.y-vrq
and also
https://github.com/cchalpha/linux-gc/commits/linux-4.1.y-vrq

All-in-one patch is available too.

BR Alfred 

Wednesday, June 7, 2017

VRQ 0.96 release

Abandon this release due to lock-up issues reported by users, the lock-up is caused by "Lock strategy update" commit, which works well on my working machines and continue during my work on SMT sensitive scheduling, that make me believe it was good and stable.

New "Lock strategy update" debug patch will be posted here for testing. Once it is confirmed work well for other users,  the repined 096a will be released.

VRQ 0.96 is released with the following changes

1. Sync up cpufreq util usage.
2. Lock strategy update, which hopefully fix potential lock issue when task migrating.
3. SMT sensitive scheduling v0.1

Main feature in this release is the first version of SMT sensitive scheduling, which reduce 10s kernel compile benchmark on my test machine(original 7m17s) under 50% workload.
Or, you can easy to observe cpu usage changes when any physical cores available, scheduler will not put task to smt core. For example, if two tasks are running on a 2 cores 4 threads cpu, one will be on cpu 0 or 1, another will be on cpu 2 or 3.

Further improvement for SMT sensitive will be in next release. I'd see if any improvement/simplify can be made to current design.

Enjoy VRQ 0.96 for v4.11 kernel, and unlock your SMT cpu ability with VRQ, :)

code are available at
https://bitbucket.org/alfredchen/linux-gc/commits/branch/linux-4.11.y-vrq
and also
https://github.com/cchalpha/linux-gc/commits/linux-4.1.y-vrq

All-in-one patch is available too.

BR Alfred 

Thursday, May 4, 2017

VRQ 0.95b release

VRQ 0.95b is released with the following changes

1. Fix UP compilation issue with hrtick_enabled(), thanks jwh7 for reporting.
2. Sync up 4.11 mainline scheduler code changes.

This is mainly a sync-up release for 4.11 kernel. Please be notices that the frozen some time after suspend/resume issue still remains and it is confirmed that this also happen with vanilla kernel, so it's not caused by VRQ scheduler code. If suspend/resume is important for you, you may need to wait for the fix from mainline.

Enjoy VRQ 0.95b for v4.11 kernel, :)


code are available at
https://bitbucket.org/alfredchen/linux-gc/commits/branch/linux-4.11.y-vrq
and also
https://github.com/cchalpha/linux-gc/commits/linux-4.1.y-vrq

All-in-one patch is available too.

BR Alfred 

Friday, April 14, 2017

VRQ 0.95 release

VRQ 0.95 is released with the following changes

1. Fix compilation issue with CONFIG_CPUSETS(thanks Dzon reporting this)
2. Introduce hrtimer for task time slice expiration.
3. NO_HZ_FULL support
4. cpumask topology output in hex format.
5. use cpu_online_mask in sched_init_topology_cpumask(), this fix wrong topology setup for some system.

The most important feature in this release is introduce hrtimer for task time slice expiration and re-enable NO_HZ_FULL support.

For issues, Dazon also report his issue that unable to achieve full cpu utilization in kernel compilation. And I am still investigating, anyone else has similar issue could let me know.

In next release, SMT improvement is in the plan list.


Enjoy VRQ 0.95 for v4.10 kernel, :)


code are available at
https://bitbucket.org/alfredchen/linux-gc/commits/branch/linux-4.10.y-vrq
and also
https://github.com/cchalpha/linux-gc/commits/linux-4.10.y-vrq

All-in-one patch is available too.

BR Alfred 

Thursday, March 23, 2017

VRQ 0.94 release

VRQ 0.94 is released with the following changes

1. Remove duplicated code
2. Fix compilation issue with CONFIG_CPU_FREQ and CONFIG_IRQ_TIME_ACCOUNTING
3. Remove root_domain and sched_domain, which VRQ doesn't depend on. This reduce about 2k LOC of the scheduler code.

I have done some tests on SMT machine for the >100% regression issue, but for some kind of reason, I can't reproduce it, so now it is in the long-term watch list and let VRQ development going. Enjoy VRQ 0.94 for v4.10 kernel, :)

In next release, hrtimer and full no hz support are in the planning list.

code are available at
https://bitbucket.org/alfredchen/linux-gc/commits/branch/linux-4.10.y-vrq
and also
https://github.com/cchalpha/linux-gc/commits/linux-4.10.y-vrq

All-in-one patch is available too.

BR Alfred 

Thursday, February 23, 2017

VRQ 0.93a release

VRQ 0.93a is released with the following changes

1. UP compilation fix for task policy fairness(reported by jwh7)
2. Sync up mainline 4.10 scheduler changes
3. remove unused stime_pc and utime_pc
4. Fix task cpu runtime accounting (reported by Eduardo)

This is a bug fix release. Enjoy VRQ 0.93a for v4.10 kernel, :)

code are available at
https://bitbucket.org/alfredchen/linux-gc/commits/branch/linux-4.10.y-vrq
and also
https://github.com/cchalpha/linux-gc/commits/linux-4.10.y-vrq

All-in-one patch is available too.

BR Alfred 

Friday, February 10, 2017

VRQ 0.93 release

VRQ 0.93 is released with the following changes

1. remove unused rq->running and refine set_rq_task(), remove unused finish_arch_switch, these two changes are continuous code clean up.
2. Fix 32bit compilation issue in cputime.c(reported by jwh7)
3. task policy fairness

Task policy fairness is the major code change in this release, which address the issue that NORMAL policy(and supper) tasks fail to suppress the background IDLE policy tasks. There are used to be two design options for this issue, but turns out that one of them is better than another with less overhead. So the code changes are in this VRQ release and no more debug patch is needed.

In next release, mainline kernel 4.10 sync-up and code clean-up are in the planning list. And I am investigating new created task issue, hopefully be fixed in next release.

Enjoy VRQ 0.93, :)

code are available at
https://bitbucket.org/alfredchen/linux-gc/commits/branch/linux-4.9.y-vrq
and also
https://github.com/cchalpha/linux-gc/commits/linux-4.9.y-vrq

All-in-one patch is available too.

BR Alfred 

Tuesday, January 17, 2017

VRQ 0.92 release

VRQ 0.92 is released with the following changes

1. remove printk in migrate_tasks()
2. vrq: refine normalize_rt_tasks()
3. vrq: Optimist ffb usage in skiplist_random_level()
4. vrq: introduce cputime.c
5. vrq: remove unused sched_domain_level
6. vrq: remove rq->timekeep_clock

Most are code clean up and little optimist. The major one is introducing mainline cputime.c, which help to reduce vrq scheduler main code size under 7k LOC and reduce the effect syncing up with mainline kernel scheduler code from release to release.

Enjoy VRQ 0.92, :)

code are available at
https://bitbucket.org/alfredchen/linux-gc/commits/branch/linux-4.9.y-vrq
and also
https://github.com/cchalpha/linux-gc/commits/linux-4.9.y-vrq

All-in-one patch is available too.

BR Alfred 

Monday, January 9, 2017

VRQ 0.91 release

VRQ 0.91 is released with the following changes

1. rework yield_to(), make it to be suit in VRQ scheduler.
2. Remove rq_deadline which is only used locally.
3. remove unused sched_interactive, there is policy based interactivity setting in VRQ, so remove this BFS scheduler system parameter.
4. Do irq_enter on scheduler_ipi called when idle to update xtime. Pick up from MuQSS.
5. Remove dup'd vtime_task_switch. Pick up from MuQSS.
6. remove unused double_rq_lock/double_rq_unlock
7. rework take_other_rq_task(). This rework provided more interactivity for RT/ISO/NORMAL tasks.
#7 is considered as a fix for D3 wine playing issue under ondemand cpufreq governor. Is it works, there would be other commits for further tuning.

Enjoy the first release of VRQ in 2017, :)

code are available at
https://bitbucket.org/alfredchen/linux-gc/commits/branch/linux-4.9.y-vrq
and also
https://github.com/cchalpha/linux-gc/commits/linux-4.9.y-vrq

All-in-one patch is available too.

BR Alfred