Thursday, December 19, 2019

BMQ v5.4-r1 release

BMQ v5.4-r1 is released with the following changes

1. Adjust task boost_prio at deactivate&wake_up.

This change makes task which gives up CPU longer than a time slice a priority boost when it wakes up. And now, children tasks fork from high boost priority tasks will have relative higher boost priority than the original lowest boost priority.

Enjoy BMQ for your linux kernel, :)

Full kernel tree repository can be found at https://gitlab.com/alfredchen/linux-bmq
And all-in-one patch can be found at gitlab.

Bug report at https://gitlab.com/alfredchen/bmq/issues

Monday, November 25, 2019

BMQ v5.4-r0 release

BMQ v5.4-r0 is released with the following changes

1. Sync up with 5.4 mainline scheduler code changes.

This is the first BMQ release for 5.4 kernel series. Besides the sync-up changes, there is also fix for BMQ with NUMA kernel config, there were some Ryzen related NUMA in previous kernel releases, hopefully this will help with Ryzen users.

Enjoy BMQ for your linux kernel, :)

Full kernel tree repository can be found at https://gitlab.com/alfredchen/linux-bmq
And all-in-one patch can be found at gitlab.

Bug report at https://gitlab.com/alfredchen/bmq/issues

Monday, October 21, 2019

BMQ v5.3-r2 release

BMQ v5.3-r2 is released with the following changes

1. Several optimization. Remove sched_cpu_llc_start_mask, Optimize migrate_pending_tasks() and Refine bmq_find_first_bit/bmq_find_next_bit macros. 
2. Rename some variables for readable.

Version number pattern has changed per the suggestion from users. Now there should be no confusion.

Enjoy BMQ for your linux kernel, :)

Full kernel tree repository can be found at https://gitlab.com/alfredchen/linux-bmq
And all-in-one patch can be found at gitlab.

Bug report at https://gitlab.com/alfredchen/bmq/issues

Thursday, October 3, 2019

BMQ v5.3.1 release

BMQ v5.3.1 is released with the following changes

1. Rework sched_rq_watermark. This rework shift the workload of sched_rq_watermaek from ttwu code path to enqueue/dequeue code path. Sanity tests show improvement under all kinds of cpu workload.
2. Fix task_sched_prio(). This fixes a long existed stupid plus/minus one mistake in schedule priority.

Yes, the version number is changed. Now it is better reflect the fact that BMQ is related to the kernel release. In current version pattern, the first two digital is the linux kernel version that this BMQ release is applied on, and 3rd digital is the BMQ release in the given(first 2 digital) kernel.

Enjoy BMQ for your linux kernel, :)

Full kernel tree repository can be found at https://gitlab.com/alfredchen/linux-bmq
And all-in-one patch can be found at gitlab.

Bug report at https://gitlab.com/alfredchen/bmq/issues

Monday, September 16, 2019

BMQ 1.00 release

BMQ 1.00 is released with the following changes

1. Syn up with mainline 5.3 scheduler code changes.
2. Enable CONFIG_CGROUP_CPUACCT with BMQ. Requested by user to enable this, not guarantee works 100% as expected, but at least no complain so far.
3. Document update.
4. unbreak systemd-detect-virt. Thanks Oleksandr Natalenko for reporting and providing fix patch. Pls check https://gitlab.com/alfredchen/bmq/issues/8# for detail.

Finally hits 1.00 release, but nothing special, just the first release of 5.3 kernel. Thanks for who contribute to BMQ, by testing,reporting and providing fix patches. All these make BMQ stable and advancing. BMQ development will keep moving forward. And currently there is a rework is under testing, the result is promising, hopefully will be released in next release in one or two weeks.

Enjoy BMQ 1.00 for v5.3 kernel, :)

Full kernel tree repository can be found at https://gitlab.com/alfredchen/linux-bmq
And all-in-one patch can be found at gitlab.

Thursday, August 15, 2019

BMQ 0.99 release

BMQ 0.99 is released with the following changes

1. A lot of refines here and there in BMQ routines and code path.
2. Consider llc(last level cache) in sg balance and ttwu code path. This hopefully can help with cpu topology with multiple llc, like ryzen. But I don't have hw to further test.

Your feedback will be welcome, especially for #2 changes for multiple llc cpus.

Enjoy BMQ 0.99 for v5.2 kernel, :)

Full kernel tree repository can be found at https://gitlab.com/alfredchen/linux-bmq
And all-in-one patch can be found at gitlab.

Sunday, July 14, 2019

BMQ 0.98 release

BMQ 0.98 is released with the following changes

1. Minor comment/code readable changes.
2. BMQ default RLIMIT_NICE, which let normal privilege users can set nice up to -10.
3. Instroduce __bmq_find_first_bit()/__bmq_find_next_bit() for rq->queue.bitmap, to avoid zero checking.

In this release, there is just one minor improvement(#3) for BMQ itself. But there is #2 for user experience. Now, even normal privilege users can promote interactive focused tasks to nice level up to -10, for example, I'd like to use "nice --5 firefox" to run firefox so heavy normal nice level 0 tasks on the terminal won't impact it. Usually, normal privilege users can't set nice level lower than 0. But it should be a nice to have privilege if using BMQ.

Further explanation  about the above use case, consider there is +/-4 auto nice level adjustment in BMQ, the firefox and the normal nice level 0 tasks will still have some overlap in scheduler level for some time, but the result turns out very good. If no overlap is really required, nice level -10 can be used for the firefox.

Enjoy BMQ 0.98 for v5.2 kernel, :)

Full kernel tree repository can be found at https://gitlab.com/alfredchen/linux-bmq
And all-in-one patch can be found at gitlab.

Please report bugs at https://gitlab.com/alfredchen/bmq/issues.

Sunday, July 7, 2019

BMQ 0.97 release

BMQ 0.97 is released with the following changes

1. Sync up with mainline 5.2 scheduler code changes.
2. Fix compilation error when PSI disabled and CGROUP enabled.

This is the first release for 5.2 kernel, just a sync-up release.

Enjoy BMQ 0.97 for v5.2 kernel, :)

Full kernel tree repository can be found at https://gitlab.com/alfredchen/linux-bmq
And all-in-one patch can be found at gitlab.

Please report bugs at https://gitlab.com/alfredchen/bmq/issues.

Thursday, June 6, 2019

BMQ 0.96 release

BMQ 0.96 is released with the following changes

1. Several enhancement in the core path.
2. Redo the sync commit reverted in 0.95.

Enjoy BMQ 0.96 for v5.1 kernel, :)

Full kernel tree repository can be found at https://gitlab.com/alfredchen/linux-bmq
And all-in-one patch can be found at gitlab.

Please report bugs at https://gitlab.com/alfredchen/bmq/issues.

Sunday, May 26, 2019

BMQ 0.95 release

BMQ 0.95 is released with the following changes

1. Fix rq missing on arm arch.
2. Rework/Remove code to reduce overhead.

Enjoy BMQ 0.95 for v5.1 kernel, :)

Full kernel tree repository can be found at https://gitlab.com/alfredchen/linux-bmq
And all-in-one patch can be found at gitlab.

Please report bugs at https://gitlab.com/alfredchen/bmq/issues.

Monday, May 6, 2019

BMQ 0.94 release

BMQ 0.94 is released with the following changes

1. Sync up with mainline kernel scheduler code changes.

This is the first BMQ release for 5.1 kernel, it just contains the sync up changes.

Enjoy BMQ 0.94 for v5.1 kernel, :)

Full kernel tree repository can be found at https://gitlab.com/alfredchen/linux-bmq
And all-in-one patch can be found at gitlab.

Please report bugs at https://gitlab.com/alfredchen/bmq/issues.

UPDATE:
Pls check https://gitlab.com/alfredchen/bmq/issues/5 for detail, in short, pls pick up commit https://gitlab.com/alfredchen/linux-bmq/commit/c98f44724fac5c2b42d831f0ad986008420d13c2 for this release.

Tuesday, April 30, 2019

BMQ 0.93 release

BMQ 0.93 is released with the following changes

1. New tasks has MAX_PRIORITY_ADJ boost_prio. This will help with task fairness while new task boost creating.
2. Rework deboost_task(). Again, this will help to reduce scheduling overhead.
3. Re-implement yield_type 1. The new yield implement now can be set by yield_type 2. Still, default is 1(deboost task and requeue it).

This should be the last BMQ release for v5.0 kernel.

Enjoy BMQ 0.93 for v5.0 kernel, :)

Full kernel tree repository can be found at https://gitlab.com/alfredchen/linux-bmq
And all-in-one patch can be found at gitlab.

Please report bugs at https://gitlab.com/alfredchen/bmq/issues.

Wednesday, April 10, 2019

BMQ 0.92 is release

BMQ 0.92 is released with the following changes

1. Rework boost/deboost task and thresholds.
2. Rework yield(). By introducing skip task in rq structure and skip schedule it in __schedule(). This should fulfilled the response to yield() and the long existed yield problem should be fixed.

This is yet another improvement release of BMQ. There are other bug fixes/improvement on going but ready to be released.

Enjoy BMQ 0.92 for v5.0 kernel, :)

Full kernel tree repository can be found at https://gitlab.com/alfredchen/linux-bmq
And all-in-one patch can be found at gitlab.

Please report bugs at https://gitlab.com/alfredchen/bmq/issues.

NOTICE:
Some untested code are stealth in the yield() rework commit, emergency fix at 

https://gitlab.com/alfredchen/linux-bmq/commit/1b66bf997e19f3fa59e0871cce0de4778e91bce3

Monday, March 25, 2019

BMQ 0.91 release

BMQ 0.91 is released with the following changes

1. Fix BMQ compile fails for x86-UP, again. Thanks pf's code review.
2. treat NORMAL task as dl for lower ttwu latency.
3. Remove uprq. Unified code base for UP and SMP.
4. Rework task switch time thresholds.
5. Rework sg_balance_trigger(). Add one more trigger scenario.

This is improvement release of BMQ. #4 will provide more distributed boost range, users can notice this by watching the cpu monitoring tools.

Enjoy BMQ 0.91 for v5.0 kernel, :)

Full kernel tree repository can be found at https://gitlab.com/alfredchen/linux-bmq
And all-in-one patch can be found at gitlab.

Please report bugs at https://gitlab.com/alfredchen/bmq/issues.

Monday, March 18, 2019

BMQ 0.90 release

BMQ 0.90 is released with the following changes

1. Consider boost_prio in task_prio(). This will allow cpu monitoring tool shows task boost/deboost priority changes.
2. Fix BMQ compile fails for x86-UP.
3. Some code optimization.

This is a bug fix and improvement release of BMQ.

Enjoy BMQ 0.90 for v5.0 kernel, :)

Full kernel tree repository can be found at https://gitlab.com/alfredchen/linux-bmq
And all-in-one patch can be found at gitlab.

Please report bugs at https://gitlab.com/alfredchen/bmq/issues.

Monday, March 11, 2019

BMQ Scheduler call out for testing

BMQ(BitMap Queue) Scheduler is a band new CPU scheduler development from PDS and inspired by the scheduler in zircon project(google). It has been developed for months and it's time for open testing(current version is 0.89, on linux kernel 5.0).

For more design detail of BMQ, pls reference to Documentation/scheduler/sched-BMQ.txt in the repository. The documentation is not yet completed, because the scheduler is still under development and major features are not finalized.

Here is the list of major user visible difference of BMQ with PDS.

1. *NOT* support SCHED_ISO, pls use "nice --20" instead.
2. *NO* rr_interval, but a compile time kernel config CONFIG_SCHED_TIMESLICE(default 4ms) is available for similar usage. Yet, it is *strongly NOT recommended* to change it.
3. "yield_type" is still supported, but only value 0, 1(default) are available, 2 is still accept from interface, but it's same as value 1. (Will be changed when yield implementation is finalized)
4. BATCH and IDLE tasks are treated as the same policy. They compete CPU with NORMAL policy tasks, but they just don't boost. To control the priority of NORMAL/BATCH/IDLE tasks, simply use nice level.
5. BMQ will auto adjust(boost/deboost) task priority within +/- MAX_PRIORITY_ADJ(default 4) ranges. For example, from top/htop and other cpu monitors program, task of nice level 0 may be saw running as nice in cpu time accounting.

BMQ has been running smoothly on 3 machines(NUC Desktop, NAS file server and 7*24 raspberry pi) for ~1 month. Suspend/Resume on NUC Desktop and NAS file server are tested. BMQ shows promising in Desktop activity and kernel compilation sanity comparing to PDS. More benchmark is still on going.

BMQ is simple in design compare to PDS and result in ~20KB less in patch size and ~4KB in compressed kernel binary size.

Full kernel tree repository can be found at https://gitlab.com/alfredchen/linux-bmq
And all-in-one patch can be found at gitlab.

Thanks for testing and your feedback will be welcome.

Sunday, March 3, 2019

PDS 0.99o release

PDS 0.99o is released with the following changes

1. Sync-up changes from mainline scheduler code.

This is the first PDS release for kernel 5.0.

Enjoy PDS 0.99o for v5.0 kernel, :)

Code are available at https://gitlab.com/alfredchen/linux-pds
All-in-one patch is available too. 

Friday, March 1, 2019

PDS 0.99n release

PDS 0.99n is released with the following changes

1. Fix hrtick_start() too short for new tasks.

This is a bug fix release before kernel 5.0.

Enjoy PDS 0.99n for v4.20 kernel, :)

Code are available at https://gitlab.com/alfredchen/linux-pds
All-in-one patch is available too. 

Wednesday, February 13, 2019

PDS 0.99m release

PDS 0.99m is released with the following changes

1. [Sync] f29a8be0e5d2 cpu/hotplug: Fix "SMT disabled by BIOS" detection for KVM

This is a sync-up release for 4.20.8+, and should be the last PDS for 4.20.

Enjoy PDS 0.99m for v4.20 kernel, :)

Code are available at https://gitlab.com/alfredchen/linux-pds
All-in-one patch is available too. 

Saturday, January 19, 2019

PDS 0.99l release

PDS 0.99l is released with the following changes

1. Fix wrong task priority issue. Now, RT(non-normal) tasks has their proper priority, can be verified by 'RT' priority shown in top/htop.

This is a bug fix release.

Enjoy PDS 0.99l for v4.20 kernel, :)

Code are available at https://gitlab.com/alfredchen/linux-pds
All-in-one patch is available too. 

Wednesday, January 9, 2019

PDS 0.99k release

PDS 0.99k is released with the following changes

1. Use exported task_rq_lock/unlock(). These mainline API has been implemented and exported in PDS, so we can use them in other modules to minimized PDS code changes.
2. Optimize pds_skiplist_random_level().
3. Rework SCHED_ISO and non-root user can switch task among ISO/NORMAL/BATCH/IDLE.
 
The most important change in this release is the rework of ISO policy in PDS

"""
    SCHED_ISO tasks now have higher priority to be picked up and run than
    SCHED_NORMAL tasks. When their remaining time slice(default max 4ms in
    PDS) run out, they will degrade to SCHED_NORMAL(and nice level kicks
    in). The degradation will be continued when they still hungry for cpu.

    The above design makes SCHED_ISO good for interactivity but not cpu
    consumption tasks.
"""
And now, normal users can set task policy ISO/NORMAL/BATCH/IDLE freely without root permission. You are encouraged to test any interactive task in PDS new ISO policy(such as gaming, media playback, browsing web ...), and feedback will be welcome.

Enjoy PDS 0.99k for v4.20 kernel, :)

Code are available at https://gitlab.com/alfredchen/linux-pds
All-in-one patch is available too.