| Message ID | 1479708496-9828-6-git-send-email-binoy.jayan@linaro.org |
|---|---|
| State | New |
| Headers | show |
> diff --git a/drivers/infiniband/ulp/isert/ib_isert.c b/drivers/infiniband/ulp/isert/ib_isert.c > index 6dd43f6..de80f56 100644 > --- a/drivers/infiniband/ulp/isert/ib_isert.c > +++ b/drivers/infiniband/ulp/isert/ib_isert.c > @@ -619,7 +619,7 @@ > mutex_unlock(&isert_np->mutex); > > isert_info("np %p: Allow accept_np to continue\n", isert_np); > - up(&isert_np->sem); > + complete(&isert_np->comp); > } > > static void > @@ -2311,7 +2311,7 @@ struct rdma_cm_id * > isert_err("Unable to allocate struct isert_np\n"); > return -ENOMEM; > } > - sema_init(&isert_np->sem, 0); > + init_completion(&isert_np->comp); This is still racy, a connect event can complete just before we init the completion and *will* get lost... This code started off with a waitqueue which exposes the same problem, see: 531b7bf4bd79 Target/iser: Fix iscsit_accept_np and rdma_cm racy flow So, still NAK from me...
On Monday, November 21, 2016 9:36:22 AM CET Sagi Grimberg wrote: > > diff --git a/drivers/infiniband/ulp/isert/ib_isert.c b/drivers/infiniband/ulp/isert/ib_isert.c > > index 6dd43f6..de80f56 100644 > > --- a/drivers/infiniband/ulp/isert/ib_isert.c > > +++ b/drivers/infiniband/ulp/isert/ib_isert.c > > @@ -619,7 +619,7 @@ > > mutex_unlock(&isert_np->mutex); > > > > isert_info("np %p: Allow accept_np to continue\n", isert_np); > > - up(&isert_np->sem); > > + complete(&isert_np->comp); > > } > > > > static void > > @@ -2311,7 +2311,7 @@ struct rdma_cm_id * > > isert_err("Unable to allocate struct isert_np\n"); > > return -ENOMEM; > > } > > - sema_init(&isert_np->sem, 0); > > + init_completion(&isert_np->comp); > > This is still racy, a connect event can complete just before we > init the completion and *will* get lost... > > This code started off with a waitqueue which exposes the same > problem, see: > 531b7bf4bd79 Target/iser: Fix iscsit_accept_np and rdma_cm racy flow > > So, still NAK from me... I don't see what you mean here. The code using a waitqueue had no counter but the completion does. I had suggested that Binoy add a comment into the code about this, as it is a rarely used property of completions, but it does seem entirely valid to me. Arnd
>>> diff --git a/drivers/infiniband/ulp/isert/ib_isert.c b/drivers/infiniband/ulp/isert/ib_isert.c >>> index 6dd43f6..de80f56 100644 >>> --- a/drivers/infiniband/ulp/isert/ib_isert.c >>> +++ b/drivers/infiniband/ulp/isert/ib_isert.c >>> @@ -619,7 +619,7 @@ >>> mutex_unlock(&isert_np->mutex); >>> >>> isert_info("np %p: Allow accept_np to continue\n", isert_np); >>> - up(&isert_np->sem); >>> + complete(&isert_np->comp); >>> } >>> >>> static void >>> @@ -2311,7 +2311,7 @@ struct rdma_cm_id * >>> isert_err("Unable to allocate struct isert_np\n"); >>> return -ENOMEM; >>> } >>> - sema_init(&isert_np->sem, 0); >>> + init_completion(&isert_np->comp); >> >> This is still racy, a connect event can complete just before we >> init the completion and *will* get lost... >> >> This code started off with a waitqueue which exposes the same >> problem, see: >> 531b7bf4bd79 Target/iser: Fix iscsit_accept_np and rdma_cm racy flow >> >> So, still NAK from me... > > I don't see what you mean here. The code using a waitqueue had no > counter but the completion does. The problem here is that init_completion sets the counter to zero and between this thread wakes up and until it initializes comp->done we might have another connect event completing it and I fail to see how this is handled correctly.
On Monday, November 21, 2016 2:33:02 PM CET Sagi Grimberg wrote: > >>> diff --git a/drivers/infiniband/ulp/isert/ib_isert.c b/drivers/infiniband/ulp/isert/ib_isert.c > >>> index 6dd43f6..de80f56 100644 > >>> --- a/drivers/infiniband/ulp/isert/ib_isert.c > >>> +++ b/drivers/infiniband/ulp/isert/ib_isert.c > >>> @@ -619,7 +619,7 @@ > >>> mutex_unlock(&isert_np->mutex); > >>> > >>> isert_info("np %p: Allow accept_np to continue\n", isert_np); > >>> - up(&isert_np->sem); > >>> + complete(&isert_np->comp); > >>> } > >>> > >>> static void > >>> @@ -2311,7 +2311,7 @@ struct rdma_cm_id * > >>> isert_err("Unable to allocate struct isert_np\n"); > >>> return -ENOMEM; > >>> } > >>> - sema_init(&isert_np->sem, 0); > >>> + init_completion(&isert_np->comp); > >> > >> This is still racy, a connect event can complete just before we > >> init the completion and *will* get lost... > >> > >> This code started off with a waitqueue which exposes the same > >> problem, see: > >> 531b7bf4bd79 Target/iser: Fix iscsit_accept_np and rdma_cm racy flow > >> > >> So, still NAK from me... > > > > I don't see what you mean here. The code using a waitqueue had no > > counter but the completion does. > > The problem here is that init_completion sets the counter to zero > and between this thread wakes up and until it initializes comp->done > we might have another connect event completing it and I fail to > see how this is handled correctly. But the sema_init()/init_completion() is done right after the structure is allocated, so nothing should have a pointer to it at this time. Later when we do the down()/wait_for_completion(), that will just decrement the counter rather than initialize, and converting from one to the other doesn't change anything here. Arnd
diff --git a/drivers/infiniband/ulp/isert/ib_isert.c b/drivers/infiniband/ulp/isert/ib_isert.c index 6dd43f6..de80f56 100644 --- a/drivers/infiniband/ulp/isert/ib_isert.c +++ b/drivers/infiniband/ulp/isert/ib_isert.c @@ -619,7 +619,7 @@ mutex_unlock(&isert_np->mutex); isert_info("np %p: Allow accept_np to continue\n", isert_np); - up(&isert_np->sem); + complete(&isert_np->comp); } static void @@ -2311,7 +2311,7 @@ struct rdma_cm_id * isert_err("Unable to allocate struct isert_np\n"); return -ENOMEM; } - sema_init(&isert_np->sem, 0); + init_completion(&isert_np->comp); mutex_init(&isert_np->mutex); INIT_LIST_HEAD(&isert_np->accepted); INIT_LIST_HEAD(&isert_np->pending); @@ -2427,7 +2427,7 @@ struct rdma_cm_id * int ret; accept_wait: - ret = down_interruptible(&isert_np->sem); + ret = wait_for_completion_interruptible(&isert_np->comp); if (ret) return -ENODEV; diff --git a/drivers/infiniband/ulp/isert/ib_isert.h b/drivers/infiniband/ulp/isert/ib_isert.h index c02ada5..a1277c0 100644 --- a/drivers/infiniband/ulp/isert/ib_isert.h +++ b/drivers/infiniband/ulp/isert/ib_isert.h @@ -3,6 +3,7 @@ #include <linux/in6.h> #include <rdma/ib_verbs.h> #include <rdma/rdma_cm.h> +#include <linux/completion.h> #include <rdma/rw.h> #include <scsi/iser.h> @@ -190,7 +191,7 @@ struct isert_device { struct isert_np { struct iscsi_np *np; - struct semaphore sem; + struct completion comp; struct rdma_cm_id *cm_id; struct mutex mutex; struct list_head accepted;
The semaphore 'sem' in isert_device is used as completion, but in a counting fashion as isert_connected_handler could be called multiple times during which it allows for that number of waiters (isert_accept_np) to continue without blocking, each consuming one node out from the list isert_np-pending in the same order in which they were enqueued (FIFO). So, convert it to struct completion. Semaphores are going away in the future. Signed-off-by: Binoy Jayan <binoy.jayan@linaro.org> --- drivers/infiniband/ulp/isert/ib_isert.c | 6 +++--- drivers/infiniband/ulp/isert/ib_isert.h | 3 ++- 2 files changed, 5 insertions(+), 4 deletions(-) -- The Qualcomm Innovation Center, Inc. is a member of the Code Aurora Forum, a Linux Foundation Collaborative Project