| Message ID | 20220228110822.491923-1-jakobkoschel@gmail.com |
|---|---|
| Headers | show |
| Series | Remove usage of list iterator past the loop body | expand |
On Mon, Feb 28, 2022 at 12:37:15PM -0800, Linus Torvalds wrote: > On Mon, Feb 28, 2022 at 12:16 PM Matthew Wilcox <willy@infradead.org> wrote: > > > > Then we can never use -Wshadow ;-( I'd love to be able to turn it on; > > it catches real bugs. > > Oh, we already can never use -Wshadow regardless of things like this. > That bridge hasn't just been burned, it never existed in the first > place. > > The whole '-Wshadow' thing simply cannot work with local variables in > macros - something that we've used since day 1. > > Try this (as a "p.c" file): > > #define min(a,b) ({ \ > typeof(a) __a = (a); \ > typeof(b) __b = (b); \ > __a < __b ? __a : __b; }) > > int min3(int a, int b, int c) > { > return min(a,min(b,c)); > } > > and now do "gcc -O2 -S t.c". > > Then try it with -Wshadow. #define ___PASTE(a, b) a##b #define __PASTE(a, b) ___PASTE(a, b) #define _min(a, b, u) ({ \ typeof(a) __PASTE(__a,u) = (a); \ typeof(b) __PASTE(__b,u) = (b); \ __PASTE(__a,u) < __PASTE(__b,u) ? __PASTE(__a,u) : __PASTE(__b,u); }) #define min(a, b) _min(a, b, __COUNTER__) int min3(int a, int b, int c) { return min(a,min(b,c)); } (probably there's a more elegant way to do this)
On Mon, Feb 28, 2022 at 3:26 PM Matthew Wilcox <willy@infradead.org> wrote: > > #define ___PASTE(a, b) a##b > #define __PASTE(a, b) ___PASTE(a, b) > #define _min(a, b, u) ({ \ Yeah, except that's ugly beyond belief, plus it's literally not what we do in the kernel. Really. The "-Wshadow doesn't work on the kernel" is not some new issue, because you have to do completely insane things to the source code to enable it. Just compare your uglier-than-sin version to my straightforward one. One does the usual and obvious "use a private variable to avoid the classic multi-use of a macro argument". And the other one is an abomination. Linus
On Mon, Feb 28, 2022 at 4:45 PM Linus Torvalds <torvalds@linux-foundation.org> wrote: > > Yeah, except that's ugly beyond belief, plus it's literally not what > we do in the kernel. (Of course, I probably shouldn't have used 'min()' as an example, because that is actually one of the few places where we do exactly that, using our __UNIQUE_ID() macros. Exactly because people _have_ tried to do -Wshadow when doing W=2). Linus
From: Matthew Wilcox > Sent: 28 February 2022 20:16 > > On Mon, Feb 28, 2022 at 12:10:24PM -0800, Linus Torvalds wrote: > > We can do > > > > typeof(pos) pos > > > > in the 'for ()' loop, and never use __iter at all. > > > > That means that inside the for-loop, we use a _different_ 'pos' than outside. > > Then we can never use -Wshadow ;-( I'd love to be able to turn it on; > it catches real bugs. > > > +#define list_for_each_entry(pos, head, member) \ > > + for (typeof(pos) pos = list_first_entry(head, typeof(*pos), member); \ > > + !list_entry_is_head(pos, head, member); \ > > pos = list_next_entry(pos, member)) Actually can't you use 'pos' to temporarily hold the address of 'member'. Something like: for (pos = (void *)head; \ pos ? ((pos = (void *)pos - offsetof(member)), 1) : 0; \ pos = (void *)pos->next) So that 'pos' is NULL if the loop terminates. No pointers outside structures are generated. Probably need to kill list_entry_is_head() - or it just checks for NULL. David - Registered Address Lakeside, Bramley Road, Mount Farm, Milton Keynes, MK1 1PT, UK Registration No: 1397386 (Wales)
On Mon, 28 Feb 2022 16:41:04 -0800 Linus Torvalds wrote: > So yes, initially my idea had been to just move the iterator entirely > inside the macro. But specifying the type got so ugly that I think > that > > typeof (pos) pos > > trick inside the macro really ends up giving us the best of all worlds: > > (a) let's us keep the existing syntax and code for all the nice cases > that did everything inside the loop anyway > > (b) gives us a nice warning for any normal use-after-loop case > (unless you explicitly initialized it like that > sgx_mmu_notifier_release() function did for no good reason > > (c) also guarantees that even if you don't get a warning, > non-converted (or newly written) bad code won't actually _work_ > > so you end up getting the new rules without any ambiguity or mistaken I presume the goal is that we can do this without changing existing code? Otherwise actually moving the iterator into the loop body would be an option, by creating a different hidden variable: #define list_iter(head) \ for (struct list head *_l = (head)->next; _l != (head); _l = _l->next) #define list_iter_entry(var, member) \ list_entry(_l, typeof(*var), member) list_iter(&p->a_head) { struct entry *e = list_iter_entry(e, a_member); /* use e->... */ } Or we can slide into soft insanity and exploit one of Kees'es tricks to encode the type of the entries "next to" the head: #define LIST_HEAD_MEM(name, type) \ union { \ struct list_head name; \ type *name ## _entry; \ } struct entry { struct list_head a_member; }; struct parent { LIST_HEAD_MEM(a_head, struct entry); }; #define list_for_each_magic(pos, head, member) \ for (typeof(**(head ## _entry)) *pos = list_first_entry(head, typeof(**(head ## _entry)), member); \ &pos->member != (head); \ pos = list_next_entry(pos, member)) list_for_each_magic(e, &p->a_head, a_member) { /* use e->... */ } I'll show myself out...
> On 1. Mar 2022, at 01:41, Linus Torvalds <torvalds@linux-foundation.org> wrote: > > On Mon, Feb 28, 2022 at 1:47 PM Jakob Koschel <jakobkoschel@gmail.com> wrote: >> >> The goal of this is to get compiler warnings right? This would indeed be great. > > Yes, so I don't mind having a one-time patch that has been gathered > using some automated checker tool, but I don't think that works from a > long-term maintenance perspective. > > So if we have the basic rule being "don't use the loop iterator after > the loop has finished, because it can cause all kinds of subtle > issues", then in _addition_ to fixing the existing code paths that > have this issue, I really would want to (a) get a compiler warning for > future cases and (b) make it not actually _work_ for future cases. > > Because otherwise it will just happen again. > >> Changing the list_for_each_entry() macro first will break all of those cases >> (e.g. the ones using 'list_entry_is_head()). > > So I have no problems with breaking cases that we basically already > have a patch for due to your automated tool. There were certainly > more than a handful, but it didn't look _too_ bad to just make the > rule be "don't use the iterator after the loop". > > Of course, that's just based on that patch of yours. Maybe there are a > ton of other cases that your patch didn't change, because they didn't > match your trigger case, so I may just be overly optimistic here. Based on the coccinelle script there are ~480 cases that need fixing in total. I'll now finish all of them and then split them by submodules as Greg suggested and repost a patch set per submodule. Sounds good? > > But basically to _me_, the important part is that the end result is > maintainable longer-term. I'm more than happy to have a one-time patch > to fix a lot of dubious cases if we can then have clean rules going > forward. > >> I assumed it is better to fix those cases first and then have a simple >> coccinelle script changing the macro + moving the iterator into the scope >> of the macro. > > So that had been another plan of mine, until I actually looked at > changing the macro. In the one case I looked at, it was ugly beyond > belief. > > It turns out that just syntactically, it's really nice to give the > type of the iterator from outside the way we do now. Yeah, it may be a > bit odd, and maybe it's partly because I'm so used to the > "list_for_each_list_entry()" syntax, but moving the type into the loop > construct really made it nasty - either one very complex line, or > having to split it over two lines which was even worse. > > Maybe the place I looked at just happened to have a long typename, but > it's basically always going to be a struct, so it's never a _simple_ > type. And it just looked very odd adn unnatural to have the type as > one of the "arguments" to that list_for_each_entry() macro. > > So yes, initially my idea had been to just move the iterator entirely > inside the macro. But specifying the type got so ugly that I think > that > > typeof (pos) pos > > trick inside the macro really ends up giving us the best of all worlds: > > (a) let's us keep the existing syntax and code for all the nice cases > that did everything inside the loop anyway > > (b) gives us a nice warning for any normal use-after-loop case > (unless you explicitly initialized it like that > sgx_mmu_notifier_release() function did for no good reason > > (c) also guarantees that even if you don't get a warning, > non-converted (or newly written) bad code won't actually _work_ > > so you end up getting the new rules without any ambiguity or mistaken > >> With this you are no longer able to set the 'outer' pos within the list >> iterator loop body or am I missing something? > > Correct. Any assignment inside the loop will be entirely just to the > local loop case. So any "break;" out of the loop will have to set > another variable - like your updated patch did. > >> I fail to see how this will make most of the changes in this >> patch obsolete (if that was the intention). > > I hope my explanation above clarifies my thinking: I do not dislike > your patch, and in fact your patch is indeed required to make the new > semantics work. ok it's all clear now, thanks for clarifying. I've defined all the 'tmp' iterator variables uninitialized so applying your patch on top of that later will just give the nice compiler warning if they are used past the loop body. > > What I disliked was always the maintainability of your patch - making > the rules be something that isn't actually visible in the source code, > and letting the old semantics still work as well as they ever did, and > having to basically run some verification pass to find bad users. Since this patch is not a complete list of cases that need fixing (30%) I haven't included the actual change of moving the iterator variable into the loop and thought that would be a second step coming after this is merged. With these changes alone, yes you still rely on manual verification passes. > > (I also disliked your original patch that mixed up the "CPU > speculation type safety" with the actual non-speculative problems, but > that was another issue). > > Linus - Jakob
On Mon, Feb 28, 2022 at 01:06:57PM +0100, Jakob Koschel wrote: > > > > On 28. Feb 2022, at 12:20, Greg KH <gregkh@linuxfoundation.org> wrote: > > > > On Mon, Feb 28, 2022 at 12:08:18PM +0100, Jakob Koschel wrote: > >> If the list does not contain the expected element, the value of > >> list_for_each_entry() iterator will not point to a valid structure. > >> To avoid type confusion in such case, the list iterator > >> scope will be limited to list_for_each_entry() loop. > >> > >> In preparation to limiting scope of a list iterator to the list traversal > >> loop, use a dedicated pointer to point to the found element. > >> Determining if an element was found is then simply checking if > >> the pointer is != NULL. > >> > >> Signed-off-by: Jakob Koschel <jakobkoschel@gmail.com> > >> --- > >> arch/x86/kernel/cpu/sgx/encl.c | 6 +++-- > >> drivers/scsi/scsi_transport_sas.c | 17 ++++++++----- > >> drivers/thermal/thermal_core.c | 38 ++++++++++++++++++---------- > >> drivers/usb/gadget/configfs.c | 22 ++++++++++------ > >> drivers/usb/gadget/udc/max3420_udc.c | 11 +++++--- > >> drivers/usb/gadget/udc/tegra-xudc.c | 11 +++++--- > >> drivers/usb/mtu3/mtu3_gadget.c | 11 +++++--- > >> drivers/usb/musb/musb_gadget.c | 11 +++++--- > >> drivers/vfio/mdev/mdev_core.c | 11 +++++--- > >> 9 files changed, 88 insertions(+), 50 deletions(-) > > > > The drivers/usb/ portion of this patch should be in patch 1/X, right? > > I kept them separate since it's a slightly different case. > Using 'ptr' instead of '&ptr->other_member'. Regardless, I'll split > this commit up as you mentioned. > > > > > Also, you will have to split these up per-subsystem so that the > > different subsystem maintainers can take these in their trees. > > Thanks for the feedback. > To clarify I understand you correctly: > I should do one patch set per-subsystem with every instance/(file?) > fixed as a separate commit? Yes, each file needs a different commit as they are usually all written or maintained by different people. As I said in my other response, if you need any help with this, just let me know, I'm glad to help. thanks, greg k-h
On Tue, Mar 01, 2022 at 12:28:15PM +0100, Jakob Koschel wrote: > Based on the coccinelle script there are ~480 cases that need fixing > in total. I'll now finish all of them and then split them by > submodules as Greg suggested and repost a patch set per submodule. > Sounds good? To help with this splitting, see: https://github.com/kees/kernel-tools/blob/trunk/split-on-maintainer It's not perfect, but it'll get you really close. For example, if you had a single big tree-wide patch applied to your tree: $ rm 0*.patch $ git format-patch -1 HEAD $ mv 0*.patch treewide.patch $ split-on-maintainer treewide.patch $ ls 0*.patch If you have a build log before the patch that spits out warnings, the --build-log argument can extract those warnings on a per-file basis, too (though this can be fragile).
On Mon, Feb 28, 2022 at 2:29 PM James Bottomley <James.Bottomley@hansenpartnership.com> wrote: > > However, if the desire is really to poison the loop variable then we > can do > > #define list_for_each_entry(pos, head, member) \ > for (pos = list_first_entry(head, typeof(*pos), member); \ > !list_entry_is_head(pos, head, member) && ((pos = NULL) == NULL; \ > pos = list_next_entry(pos, member)) > > Which would at least set pos to NULL when the loop completes. That would actually have been excellent if we had done that originally. It would not only avoid the stale and incorrectly typed head entry left-over turd, it would also have made it very easy to test for "did I find an entry in the loop". But I don't much like it in the situation we are now. Why? Mainly because it basically changes the semantics of the loop _without_ any warnings about it. And we don't actually get the advantage of the nicer semantics, because we can't actually make code do list_for_each_entry(entry, ....) { .. } if (!entry) return -ESRCH; .. use the entry we found .. because that would be a disaster for back-porting, plus it would be a flag-day issue (ie we'd have to change the semantics of the loop at the same time we change every single user). So instead of that simple "if (!entry)", we'd effectively have to continue to use something that still works with the old world order (ie that "if (list_entry_is_head())" model). So we couldn't really take _advantage_ of the nicer semantics, and we'd not even get a warning if somebody does it wrong - the code would just silently do the wrong thing. IOW: I don't think you are wrong about that patch: it would solve the problem that Jakob wants to solve, and it would have absolutely been much better if we had done this from the beginning. But I think that in our current situation, it's actually a really fragile solution to the "don't do that then" problem we have. Linus
On Tue, Mar 1, 2022 at 2:58 PM David Laight <David.Laight@aculab.com> wrote: > > Can it be resolved by making: > #define list_entry_is_head(pos, head, member) ((pos) == NULL) > and double-checking that it isn't used anywhere else (except in > the list macros themselves). Well, yes, except for the fact that then the name is entirely misleading... And somebody possibly uses it together with list_first_entry() etc, so it really is completely broken to mix that change with the list traversal change. Linus Linus
On Tue, Mar 1, 2022 at 3:19 PM David Laight <David.Laight@aculab.com> wrote: > > Having said that there are so few users of list_entry_is_head() > it is reasonable to generate two new names. Well, the problem is that the users of list_entry_is_head() may be few - but there are a number of _other_ ways to check "was that the HEAD pointer", and not all of them are necessarily correct. IOW, different places do different random tests for "did we walk the whole loop without breaking out". And many of them happen to work. In fact, in practice, pretty much *all* of them happen to work, and you have to have the right struct layout and really really bad luck to hit a case of "type confusion ended up causing the test to not work". And *THAT* is the problem here. It's not the "there are 25ish places that current use list_entry_is_head()". It's the "there are ~480 places that use the type-confused HEAD entry that has been cast to the wrong type". And THAT is why I think we'd be better off with that bigger change that simply means that you can't use the iterator variable at all outside the loop, and try to make it something where the compiler can help catch mis-uses. Now, making the list_for_each_entry() thing force the iterator to NULL at the end of the loop does fix the problem. The issue I have with it is really just that you end up getting no warning at all from the compiler if you mix old-style and new-style semantics. Now, you *will* get an oops (if using a new-style iterator with an old-style check), but many of these things will be in odd driver code and may happen only for error cases. And if you use a new-style check with an old-style iterator (ie some backport problem), you will probably end up getting random memory corruption, because you'll decide "it's not a HEAD entry", and then you'll actually *use* the HEAD that has the wrong type cast associated with it. See what my worry is? With the "don't use iterator outside the loop" approach, the exact same code works in both the old world order and the new world order, and you don't have the semantic confusion. And *if* you try to use the iterator outside the loop, you'll _mostly_ (*) get a compiler warning about it not being initialized. Linus (*) Unless somebody initializes the iterator pointer pointlessly. Which clearly does happen. Thus the "mostly". It's not perfect, and that's most definitely not nice - but it should at least hopefully make it that much harder to mess up.
On 02/03/2022 00.55, Linus Torvalds wrote: > On Tue, Mar 1, 2022 at 3:19 PM David Laight <David.Laight@aculab.com> wrote: >> > With the "don't use iterator outside the loop" approach, the exact > same code works in both the old world order and the new world order, > and you don't have the semantic confusion. And *if* you try to use the > iterator outside the loop, you'll _mostly_ (*) get a compiler warning > about it not being initialized. > > Linus > > (*) Unless somebody initializes the iterator pointer pointlessly. > Which clearly does happen. Thus the "mostly". It's not perfect, and > that's most definitely not nice - but it should at least hopefully > make it that much harder to mess up. This won't help the current issue (because it doesn't exist and might never), but just in case some compiler people are listening, I'd like to have some sort of way to tell the compiler "treat this variable as uninitialized from here on". So one could do #define kfree(p) do { __kfree(p); __magic_uninit(p); } while (0) with __magic_uninit being a magic no-op that doesn't affect the semantics of the code, but could be used by the compiler's "[is/may be] used uninitialized" machinery to flag e.g. double frees on some odd error path etc. It would probably only work for local automatic variables, but it should be possible to just ignore the hint if p is some expression like foo->bar or has side effects. If we had that, the end-of-loop test could include that to "uninitialize" the iterator. Maybe sparse/smatch or some other static analyzer could implement such a magic thing? Maybe it's better as a function attribute [__attribute__((uninitializes(1)))] to avoid having to macrofy all functions that release resources. Rasmus
On Wed, Mar 02, 2022 at 10:29:31AM +0100, Rasmus Villemoes wrote: > This won't help the current issue (because it doesn't exist and might > never), but just in case some compiler people are listening, I'd like to > have some sort of way to tell the compiler "treat this variable as > uninitialized from here on". So one could do > > #define kfree(p) do { __kfree(p); __magic_uninit(p); } while (0) > > with __magic_uninit being a magic no-op that doesn't affect the > semantics of the code, but could be used by the compiler's "[is/may be] > used uninitialized" machinery to flag e.g. double frees on some odd > error path etc. It would probably only work for local automatic > variables, but it should be possible to just ignore the hint if p is > some expression like foo->bar or has side effects. If we had that, the > end-of-loop test could include that to "uninitialize" the iterator. I've long wanted to change kfree() to explicitly set pointers to NULL on free. https://github.com/KSPP/linux/issues/87 The thing stopping a trivial transformation of kfree() is: kfree(get_some_pointer()); I would argue, though, that the above is poor form: the thing holding the pointer should be the thing freeing it, so these cases should be refactored and kfree() could do the NULLing by default. Quoting myself in the above issue: Without doing massive tree-wide changes, I think we need compiler support. If we had something like __builtin_is_lvalue(), we could distinguish function returns from lvalues. For example, right now a common case are things like: kfree(get_some_ptr()); But if we could at least gain coverage of the lvalue cases, and detect them statically at compile-time, we could do: #define __kfree_and_null(x) do { __kfree(*x); *x = NULL; } while (0) #define kfree(x) __builtin_choose_expr(__builtin_is_lvalue(x), __kfree_and_null(&(x)), __kfree(x)) Alternatively, we could do a tree-wide change of the former case (findable with Coccinelle) and change them into something like kfree_no_null() and redefine kfree() itself: #define kfree_no_null(x) do { void *__ptr = (x); __kfree(__ptr); } while (0) #define kfree(x) do { __kfree(x); x = NULL; } while (0)
On Wed, Mar 02, 2022 at 12:18:45PM -0800, Linus Torvalds wrote: > On Wed, Mar 2, 2022 at 12:07 PM Kees Cook <keescook@chromium.org> wrote: > > > > I've long wanted to change kfree() to explicitly set pointers to NULL on > > free. https://github.com/KSPP/linux/issues/87 > > We've had this discussion with the gcc people in the past, and gcc > actually has some support for it, but it's sadly tied to the actual > function name (ie gcc has some special-casing for "free()") > > See > > https://gcc.gnu.org/bugzilla/show_bug.cgi?id=94527 > > for some of that discussion. > > Oh, and I see some patch actually got merged since I looked there last > so that you can mark "deallocator" functions, but I think it's only > for the context matching, not for actually killing accesses to the > pointer afterwards. Ah! I missed that getting added in GCC 11. But yes, there it is: https://gcc.gnu.org/onlinedocs/gcc/Common-Function-Attributes.html#index-malloc-function-attribute Hah, now we may need to split __malloc from __alloc_size. ;) I'd still like the NULL assignment behavior, though, since some things can easily avoid static analysis.
On Wed, Mar 02, 2022 at 12:07:04PM -0800, Kees Cook wrote: > On Wed, Mar 02, 2022 at 10:29:31AM +0100, Rasmus Villemoes wrote: > > This won't help the current issue (because it doesn't exist and might > > never), but just in case some compiler people are listening, I'd like to > > have some sort of way to tell the compiler "treat this variable as > > uninitialized from here on". So one could do > > > > #define kfree(p) do { __kfree(p); __magic_uninit(p); } while (0) > > > > with __magic_uninit being a magic no-op that doesn't affect the > > semantics of the code, but could be used by the compiler's "[is/may be] > > used uninitialized" machinery to flag e.g. double frees on some odd > > error path etc. It would probably only work for local automatic > > variables, but it should be possible to just ignore the hint if p is > > some expression like foo->bar or has side effects. If we had that, the > > end-of-loop test could include that to "uninitialize" the iterator. > > I've long wanted to change kfree() to explicitly set pointers to NULL on > free. https://github.com/KSPP/linux/issues/87 You also need to be a bit careful with existing code because there are places which do things like: drivers/usb/host/r8a66597-hcd.c 424 kfree(dev); ^^^ 425 426 for (port = 0; port < r8a66597->max_root_hub; port++) { 427 if (r8a66597->root_hub[port].dev == dev) { ^^^ 428 r8a66597->root_hub[port].dev = NULL; 429 break; 430 } 431 } Printing the freed pointer in debug code is another thing people do. regards, dan carpenter
On Wed, Mar 02, 2022 at 10:29:31AM +0100, Rasmus Villemoes wrote: > This won't help the current issue (because it doesn't exist and might > never), but just in case some compiler people are listening, I'd like to > have some sort of way to tell the compiler "treat this variable as > uninitialized from here on". So one could do > > #define kfree(p) do { __kfree(p); __magic_uninit(p); } while (0) > I think this is a good idea. Smatch can already find all the iterator used outside the loop bugs that Jakob did with a manageably small number of false positives. The problems are that: 1) It would be better to find it in the compile stage instead of later. 2) I hadn't published that check. Will do shortly. 3) A couple weeks back I noticed that the list_for_each_entry() check was no longer working. Fixed now. 4) Smatch was only looking at cases which dereferenced the iterator and not checks for NULL. I will test the fix for that tonight. 5) Smatch is broken on PowerPC. Coccinelle also has checks for iterator used outside the loop. Coccinelle had these checks before Smatch did. I copied Julia's idea. If your annotation was added to GCC it would solve all those problems. But it's kind of awkward that we can't annotate kfree() directly instead of creating the kfree() macro. And there are lots of other functions which free things so you'd have to create a ton of macros like: #define gr_free_dma_desc(a, b) do { __gr_free_dma_desc(a, b); __magic_uninit(b); } while (0) And then there are functions which free a struct member: void free_bar(struct foo *p) { kfree(p->bar); } Or functions which free a container_of(). Smatch is more evolved than designed but what I do these days is use $0, $1, $2 to represent the parameters. So you can say a function frees $0->bar. For container_of() then is "(168<~$0)->bar" which means 168 bytes from $0. Returns are parameter -1 so I guess it would be $(-1), but as I said Smatch evolved so right now places that talk about returned values use a different format. What you could do is just make a parseable table next to the function definition with all the information. Then you would use a Perl script to automatically generate a Coccinelle check to warn about use after frees. diff --git a/mm/slab.c b/mm/slab.c index ddf5737c63d9..c9dffa5c40a2 100644 --- a/mm/slab.c +++ b/mm/slab.c @@ -3771,6 +3771,9 @@ EXPORT_SYMBOL(kmem_cache_free_bulk); * * Don't free memory not originally allocated by kmalloc() * or you will run into trouble. + * + * CHECKER information + * frees: $0 */ void kfree(const void *objp) { regards, dan carpenter
Updating this API is risky because some places rely on the old behavior and not all of them have been updated. Here are some additional places you might want to change. drivers/usb/host/uhci-q.c:466 link_async() warn: iterator used outside loop: 'pqh' drivers/infiniband/core/mad.c:968 ib_get_rmpp_segment() warn: iterator used outside loop: 'mad_send_wr->cur_seg' drivers/opp/debugfs.c:208 opp_migrate_dentry() warn: iterator used outside loop: 'new_dev' drivers/staging/greybus/audio_codec.c:602 gbcodec_mute_stream() warn: iterator used outside loop: 'module' drivers/staging/media/atomisp/pci/atomisp_acc.c:508 atomisp_acc_load_extensions() warn: iterator used outside loop: 'acc_fw' drivers/perf/thunderx2_pmu.c:814 tx2_uncore_pmu_init_dev() warn: iterator used outside loop: 'rentry' drivers/gpu/drm/nouveau/nvkm/engine/device/ctrl.c:111 nvkm_control_mthd_pstate_attr() warn: iterator used outside loop: 'pstate' drivers/gpu/drm/panfrost/panfrost_mmu.c:203 panfrost_mmu_as_get() warn: iterator used outside loop: 'lru_mmu' drivers/media/usb/uvc/uvc_v4l2.c:885 uvc_ioctl_enum_input() warn: iterator used outside loop: 'iterm' drivers/media/usb/uvc/uvc_v4l2.c:896 uvc_ioctl_enum_input() warn: iterator used outside loop: 'iterm' drivers/scsi/dc395x.c:3596 device_alloc() warn: iterator used outside loop: 'p' drivers/net/ethernet/mellanox/mlx4/alloc.c:379 __mlx4_alloc_from_zone() warn: iterator used outside loop: 'curr_node' fs/ocfs2/dlm/dlmdebug.c:573 lockres_seq_start() warn: iterator used outside loop: 'res' This patchset fixes 3 bugs. Initially when it's merged it's probably going to introduce some bugs because there are likely other places which rely on the old behavior. In an ideal world, with the new API the compiler would warn about uninitialized variables, but unfortunately that warning is disabled by default so we still have to rely on kbuild/Clang/Smatch to find the bugs. But hopefully the new API encourages people to write clearer code so it prevents bugs in the long run. regards, dan carpenter
From: Dan Carpenter > Sent: 07 March 2022 15:01 > > Updating this API is risky because some places rely on the old behavior > and not all of them have been updated. Here are some additional places > you might want to change. I really can't help thinking that trying to merge this patch is actually impossible. It affects far too many different parts of the tree. Since (I believe) this is a doubly linked list with forwards and backwards pointers that point to a 'node' (not that there is a nice comment to that effect in the header - and there are lots of ways to do linked lists) the 'head' pretty much has to be a 'node'. I'd write the following new defines (but I might be using the old names here): list_first(head, field) First item, NULL if empty. list_last(head, field) Last item NULL if empty. list_next(head, item, field) Item after 'item', NULL if last. list_prev(head, item. field) Item before 'item', NULL if first. You get (something like): #define list_first(head, field) \ head->next == &head ? NULL : list_item(head->next, field) (probably needs typeof(item) from somewhere). The iterator loop is then just: #define loop_iterate(item, head, field) \ for (item = list_first(head, field); item; \ item = list_next(head, item, field) I'm not sure, but making the 'head' be a structure that contains a single member that is a 'node' might help type checking. Then all the code that uses the current defines can slowly be moved over (probably a couple of releases) before the existing defines are deleted. That should simplify all the open-coded search loops that are just as likely to be buggy (possibly more so). David - Registered Address Lakeside, Bramley Road, Mount Farm, Milton Keynes, MK1 1PT, UK Registration No: 1397386 (Wales)
This is the first patch removing several categories of use cases of the list iterator variable past the loop. This is follow up to the discussion in: https://lore.kernel.org/all/20220217184829.1991035-1-jakobkoschel@gmail.com/ As concluded in: https://lore.kernel.org/all/YhdfEIwI4EdtHdym@kroah.com/ the correct use should be using a separate variable after the loop and using a 'tmp' variable as the list iterator. The list iterator will not point to a valid structure after the loop if no break/goto was hit. Invalid uses of the list iterator variable can be avoided altogether by simply using a separate pointer to iterate the list. Linus and Greg agreed on the following pattern: - struct gr_request *req; + struct gr_request *req = NULL; + struct gr_request *tmp; struct gr_ep *ep; int ret = 0; - list_for_each_entry(req, &ep->queue, queue) { - if (&req->req == _req) + list_for_each_entry(tmp, &ep->queue, queue) { + if (&tmp->req == _req) { + req = tmp; break; + } } - if (&req->req != _req) { + if (!req) { ret = -EINVAL; goto out; } With gnu89 the list iterator variable cannot yet be declared within the for loop of the list iterator. Moving to a more modern version of C would allow defining the list iterator variable within the macro, limiting the scope to the loop. This avoids any incorrect usage past the loop altogether. This are around 30% of the cases where the iterator variable is used past the loop (identified with a slightly modified version of use_after_iter.cocci). I've decided to split it into at a few patches separated by similar use cases. Because the output of get_maintainer.pl was too big, I included all the found lists and everyone from the previous discussion. Jakob Koschel (6): drivers: usb: remove usage of list iterator past the loop body treewide: remove using list iterator after loop body as a ptr treewide: fix incorrect use to determine if list is empty drivers: remove unnecessary use of list iterator variable treewide: remove dereference of list iterator after loop body treewide: remove check of list iterator against head past the loop body arch/arm/mach-mmp/sram.c | 9 ++-- arch/arm/plat-pxa/ssp.c | 28 +++++------- arch/powerpc/sysdev/fsl_gtm.c | 4 +- arch/x86/kernel/cpu/sgx/encl.c | 6 ++- drivers/block/drbd/drbd_req.c | 45 ++++++++++++------- drivers/counter/counter-chrdev.c | 26 ++++++----- drivers/crypto/cavium/nitrox/nitrox_main.c | 11 +++-- drivers/dma/dw-edma/dw-edma-core.c | 4 +- drivers/dma/ppc4xx/adma.c | 11 +++-- drivers/firewire/core-transaction.c | 32 +++++++------ drivers/firewire/sbp2.c | 14 +++--- drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c | 19 +++++--- drivers/gpu/drm/drm_memory.c | 15 ++++--- drivers/gpu/drm/drm_mm.c | 17 ++++--- drivers/gpu/drm/drm_vm.c | 13 +++--- drivers/gpu/drm/gma500/oaktrail_lvds.c | 9 ++-- drivers/gpu/drm/i915/gem/i915_gem_context.c | 14 +++--- .../gpu/drm/i915/gem/i915_gem_execbuffer.c | 15 ++++--- drivers/gpu/drm/i915/gt/intel_ring.c | 15 ++++--- .../gpu/drm/nouveau/nvkm/subdev/clk/base.c | 11 +++-- .../gpu/drm/nouveau/nvkm/subdev/fb/ramgk104.c | 13 +++--- drivers/gpu/drm/scheduler/sched_main.c | 14 +++--- drivers/gpu/drm/ttm/ttm_bo.c | 19 ++++---- drivers/gpu/drm/vmwgfx/vmwgfx_kms.c | 22 +++++---- drivers/infiniband/hw/hfi1/tid_rdma.c | 16 ++++--- drivers/infiniband/hw/mlx4/main.c | 12 ++--- drivers/media/dvb-frontends/mxl5xx.c | 11 +++-- drivers/media/pci/saa7134/saa7134-alsa.c | 4 +- drivers/media/v4l2-core/v4l2-ctrls-api.c | 31 +++++++------ drivers/misc/mei/interrupt.c | 12 ++--- .../net/ethernet/intel/i40e/i40e_ethtool.c | 3 +- .../net/ethernet/qlogic/qede/qede_filter.c | 11 +++-- drivers/net/wireless/ath/ath6kl/htc_mbox.c | 2 +- .../net/wireless/intel/ipw2x00/libipw_rx.c | 15 ++++--- drivers/perf/xgene_pmu.c | 13 +++--- drivers/power/supply/cpcap-battery.c | 11 +++-- drivers/scsi/lpfc/lpfc_bsg.c | 16 ++++--- drivers/scsi/scsi_transport_sas.c | 17 ++++--- drivers/scsi/wd719x.c | 12 +++-- drivers/staging/rtl8192e/rtl819x_TSProc.c | 17 +++---- drivers/staging/rtl8192e/rtllib_rx.c | 17 ++++--- .../staging/rtl8192u/ieee80211/ieee80211_rx.c | 15 ++++--- .../rtl8192u/ieee80211/rtl819x_TSProc.c | 19 ++++---- drivers/thermal/thermal_core.c | 38 ++++++++++------ drivers/usb/gadget/composite.c | 9 ++-- drivers/usb/gadget/configfs.c | 22 +++++---- drivers/usb/gadget/udc/aspeed-vhub/epn.c | 11 +++-- drivers/usb/gadget/udc/at91_udc.c | 26 ++++++----- drivers/usb/gadget/udc/atmel_usba_udc.c | 11 +++-- drivers/usb/gadget/udc/bdc/bdc_ep.c | 11 +++-- drivers/usb/gadget/udc/fsl_qe_udc.c | 11 +++-- drivers/usb/gadget/udc/fsl_udc_core.c | 11 +++-- drivers/usb/gadget/udc/goku_udc.c | 11 +++-- drivers/usb/gadget/udc/gr_udc.c | 11 +++-- drivers/usb/gadget/udc/lpc32xx_udc.c | 11 +++-- drivers/usb/gadget/udc/max3420_udc.c | 11 +++-- drivers/usb/gadget/udc/mv_u3d_core.c | 11 +++-- drivers/usb/gadget/udc/mv_udc_core.c | 11 +++-- drivers/usb/gadget/udc/net2272.c | 12 ++--- drivers/usb/gadget/udc/net2280.c | 11 +++-- drivers/usb/gadget/udc/omap_udc.c | 11 +++-- drivers/usb/gadget/udc/pxa25x_udc.c | 11 +++-- drivers/usb/gadget/udc/s3c-hsudc.c | 11 +++-- drivers/usb/gadget/udc/tegra-xudc.c | 11 +++-- drivers/usb/gadget/udc/udc-xilinx.c | 11 +++-- drivers/usb/mtu3/mtu3_gadget.c | 11 +++-- drivers/usb/musb/musb_gadget.c | 11 +++-- drivers/vfio/mdev/mdev_core.c | 11 +++-- fs/cifs/smb2misc.c | 10 +++-- fs/f2fs/segment.c | 9 ++-- fs/proc/kcore.c | 13 +++--- kernel/debug/kdb/kdb_main.c | 36 +++++++++------ kernel/power/snapshot.c | 10 +++-- kernel/trace/ftrace.c | 22 +++++---- kernel/trace/trace_eprobe.c | 15 ++++--- kernel/trace/trace_events.c | 11 ++--- net/9p/trans_xen.c | 11 +++-- net/ipv4/udp_tunnel_nic.c | 10 +++-- net/tipc/name_table.c | 11 +++-- net/tipc/socket.c | 11 +++-- net/xfrm/xfrm_ipcomp.c | 11 +++-- sound/soc/intel/catpt/pcm.c | 13 +++--- sound/soc/sprd/sprd-mcdt.c | 13 +++--- 83 files changed, 708 insertions(+), 465 deletions(-) base-commit: 7ee022567bf9e2e0b3cd92461a2f4986ecc99673 -- 2.25.1