From a99fc917708665d730332fe20c910e832d8526d3 Mon Sep 17 00:00:00 2001 From: "Matthew Wilcox (Oracle)" Date: Tue, 14 May 2019 16:05:45 -0400 Subject: idr: Fix idr_get_next race with idr_remove commit 5c089fd0c73411f2170ab795c9ffc16718c7d007 upstream. If the entry is deleted from the IDR between the call to radix_tree_iter_find() and rcu_dereference_raw(), idr_get_next() will return NULL, which will end the iteration prematurely. We should instead continue to the next entry in the IDR. This only happens if the iteration is protected by the RCU lock. Most IDR users use a spinlock or semaphore to exclude simultaneous modifications. It was noticed once the PID allocator was converted to use the IDR, as it uses the RCU lock, but there may be other users elsewhere in the kernel. We can't use the normal pattern of calling radix_tree_deref_retry() (which catches both a retry entry in a leaf node and a node entry in the root) as the IDR supports storing entries which are unaligned, which will trigger an infinite loop if they are encountered. Instead, we have to explicitly check whether the entry is a retry entry. Fixes: 0a835c4f090a ("Reimplement IDR and IDA using the radix tree") Reported-by: Brendan Gregg Tested-by: Brendan Gregg Signed-off-by: Matthew Wilcox (Oracle) Signed-off-by: Greg Kroah-Hartman --- lib/idr.c | 20 +++++++++++++++++--- 1 file changed, 17 insertions(+), 3 deletions(-) (limited to 'lib') diff --git a/lib/idr.c b/lib/idr.c index edd9b2be1651..8c1a98d03164 100644 --- a/lib/idr.c +++ b/lib/idr.c @@ -111,13 +111,27 @@ void *idr_get_next(struct idr *idr, int *nextid) { struct radix_tree_iter iter; void __rcu **slot; - - slot = radix_tree_iter_find(&idr->idr_rt, &iter, *nextid); + void *entry = NULL; + + radix_tree_for_each_slot(slot, &idr->idr_rt, &iter, *nextid) { + entry = rcu_dereference_raw(*slot); + if (!entry) + continue; + if (!radix_tree_deref_retry(entry)) + break; + if (slot != (void *)&idr->idr_rt.rnode && + entry != (void *)RADIX_TREE_INTERNAL_NODE) + break; + slot = radix_tree_iter_retry(&iter); + } if (!slot) return NULL; + if (WARN_ON_ONCE(iter.index > INT_MAX)) + return NULL; + *nextid = iter.index; - return rcu_dereference_raw(*slot); + return entry; } EXPORT_SYMBOL(idr_get_next); -- cgit v1.2.3