summaryrefslogtreecommitdiff
path: root/include
diff options
context:
space:
mode:
authorEric Dumazet <eric.dumazet@gmail.com>2010-11-11 14:05:08 -0800
committerLinus Torvalds <torvalds@linux-foundation.org>2010-11-12 07:55:30 -0800
commit3f9d35b9514da6757ca98831372518f9eeb71b33 (patch)
tree94dec500b9f6698520488c5e2ce8a24edf7f6c5d /include
parent8705a1baf78287eceeb00bc29401d0ae6a03f213 (diff)
atomic: add atomic_inc_not_zero_hint()
Followup of perf tools session in Netfilter WorkShop 2010 In the network stack we make high usage of atomic_inc_not_zero() in contexts we know the probable value of atomic before increment (2 for udp sockets for example) Using a special version of atomic_inc_not_zero() giving this hint can help processor to use less bus transactions. On x86 (MESI protocol) for example, this avoids entering Shared state, because "lock cmpxchg" issues an RFO (Read For Ownership) akpm: Adds a new include/linux/atomic.h. This means that new code should henceforth include linux/atomic.h and not asm/atomic.h. The presence of include/linux/atomic.h will in fact cause checkpatch.pl to warn about use of asm/atomic.h. The new include/linux/atomic.h becomes the place where arch-neutral atomic_t code should be placed. [akpm@linux-foundation.org: coding-style fixes] Signed-off-by: Eric Dumazet <eric.dumazet@gmail.com> Cc: Christoph Lameter <cl@linux-foundation.org> Cc: Ingo Molnar <mingo@elte.hu> Cc: Andi Kleen <andi@firstfloor.org> Cc: Arnaldo Carvalho de Melo <acme@infradead.org> Cc: David Miller <davem@davemloft.net> Cc: "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> Cc: Nick Piggin <npiggin@kernel.dk> Reviewed-by: "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
Diffstat (limited to 'include')
-rw-r--r--include/linux/atomic.h37
1 files changed, 37 insertions, 0 deletions
diff --git a/include/linux/atomic.h b/include/linux/atomic.h
new file mode 100644
index 000000000000..96c038e43d66
--- /dev/null
+++ b/include/linux/atomic.h
@@ -0,0 +1,37 @@
+#ifndef _LINUX_ATOMIC_H
+#define _LINUX_ATOMIC_H
+#include <asm/atomic.h>
+
+/**
+ * atomic_inc_not_zero_hint - increment if not null
+ * @v: pointer of type atomic_t
+ * @hint: probable value of the atomic before the increment
+ *
+ * This version of atomic_inc_not_zero() gives a hint of probable
+ * value of the atomic. This helps processor to not read the memory
+ * before doing the atomic read/modify/write cycle, lowering
+ * number of bus transactions on some arches.
+ *
+ * Returns: 0 if increment was not done, 1 otherwise.
+ */
+#ifndef atomic_inc_not_zero_hint
+static inline int atomic_inc_not_zero_hint(atomic_t *v, int hint)
+{
+ int val, c = hint;
+
+ /* sanity test, should be removed by compiler if hint is a constant */
+ if (!hint)
+ return atomic_inc_not_zero(v);
+
+ do {
+ val = atomic_cmpxchg(v, c, c + 1);
+ if (val == c)
+ return 1;
+ c = val;
+ } while (c);
+
+ return 0;
+}
+#endif
+
+#endif /* _LINUX_ATOMIC_H */