mirror of
https://github.com/torvalds/linux.git
synced 2026-10-07 22:37:45 -04:00
Davidlohr Bueso
7a215f89a0
In order to optimize the spinning step, we need to set the lock owner as soon as the lock is acquired; after a successful counter cmpxchg operation, that is. This is particularly useful as rwsems need to set the owner to nil for readers, so there is a greater chance of falling out of the spinning. Currently we only set the owner much later in the game, in the more generic level -- latency can be specially bad when waiting for a node->next pointer when releasing the osq in up_write calls. As such, update the owner inside rwsem_try_write_lock (when the lock is obtained after blocking) and rwsem_try_write_lock_unqueued (when the lock is obtained while spinning). This requires creating a new internal rwsem.h header to share the owner related calls. Also cleanup some headers for mutex and rwsem. Suggested-by: Peter Zijlstra <[email protected]> Signed-off-by: Davidlohr Bueso <[email protected]> Signed-off-by: Peter Zijlstra (Intel) <[email protected]> Cc: Jason Low <[email protected]> Cc: Linus Torvalds <[email protected]> Cc: Michel Lespinasse <[email protected]> Cc: Paul E. McKenney <[email protected]> Cc: Tim Chen <[email protected]> Link: http://lkml.kernel.org/r/[email protected] Signed-off-by: Ingo Molnar <[email protected]>
21 lines
357 B
C
21 lines
357 B
C
#ifdef CONFIG_RWSEM_SPIN_ON_OWNER
|
|
static inline void rwsem_set_owner(struct rw_semaphore *sem)
|
|
{
|
|
sem->owner = current;
|
|
}
|
|
|
|
static inline void rwsem_clear_owner(struct rw_semaphore *sem)
|
|
{
|
|
sem->owner = NULL;
|
|
}
|
|
|
|
#else
|
|
static inline void rwsem_set_owner(struct rw_semaphore *sem)
|
|
{
|
|
}
|
|
|
|
static inline void rwsem_clear_owner(struct rw_semaphore *sem)
|
|
{
|
|
}
|
|
#endif
|