From patchwork Tue Jul 25 04:35:54 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ilya Dryomov X-Patchwork-Id: 706224 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 68774C00528 for ; Tue, 25 Jul 2023 04:36:21 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231450AbjGYEgU (ORCPT ); Tue, 25 Jul 2023 00:36:20 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:59656 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229877AbjGYEgS (ORCPT ); Tue, 25 Jul 2023 00:36:18 -0400 Received: from mail-wr1-x435.google.com (mail-wr1-x435.google.com [IPv6:2a00:1450:4864:20::435]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 416BAE55 for ; Mon, 24 Jul 2023 21:36:17 -0700 (PDT) Received: by mail-wr1-x435.google.com with SMTP id ffacd0b85a97d-31751d7d96eso1777751f8f.1 for ; Mon, 24 Jul 2023 21:36:17 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1690259776; x=1690864576; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=eL1WFQdHL939xNzGULY0ZSwlbWxB4jBimZZ1LkCYXgA=; b=NEBg3CDBmrbbC6qd8NvUTCJe7SKxOWlB3ZYnn6krfrLhu5tvpw6OVS9TnketJlKrTN Ri7XvyMNt4i4slhZQ9mNYfdsF2zkI4Fe+RK/ctYr8mCMhrVTAx+8dYPGLFLHYhpXUFB4 oK2Utl/NG1OMTqHEp9rJyD/hk36iorEA1uwnV2en6trsqLIJA1Lt4SZwpbZ3vEVx5Mx8 JQDIxJqn8Yn3EghmU1f4HsXLzvXlrC7d+PY9/Z5AJ8YzPbRJcqHhevdKJ9rXpjfseZxM PA7mf/HIiTe1Ml+TsOhthlQBzrQPIS/DT22x2kI+LPs4CJ/q1iar8mMqjmLtlsxe0EpN LCxg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1690259776; x=1690864576; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=eL1WFQdHL939xNzGULY0ZSwlbWxB4jBimZZ1LkCYXgA=; b=D1gt4kAy0srduENjSi4Sm0rRmYQgpxSqfLntWwPFbkZ10jRbJR7XL8vOcq2fVCxMHr +VzWoF80JUGkoe9bbTmfNjsQZEbiQskROKS+8AajNHM82J2O0L12jNy5lTOESvAHIVGw MAvsSiIBExVwTBwBSqgsY8fihlC6p/OefRJbMUA8aUEknueFzbgkbv4R/asL66QlFJkJ tiorkLDaBb31YSrfkbQk+4ITDrVXiOpvBv129bPKj03rW4uXQmu7A1BquZee3BPkHLaE n22LcK/KM7r+CxcgwK9ZIUXUJETdVynLc2P6EsuB/irYV05Un6dtcFbnBwLIjKTx4NZS F1zg== X-Gm-Message-State: ABy/qLaULqG2xYshhNxxVDrZRB8QkgTETg8q6MDhI+lvMn/tcu9L1qcC IOJgWfNlZbo4PHT24tzDyBjPvG8rdys= X-Google-Smtp-Source: APBJJlH55qSmnLDAhAVNG/2kdr6cgtI9Tm+xt6Bw36ChjXVtYfctlxRd7yWgii6mef39SrPWjBId2w== X-Received: by 2002:a5d:4e8f:0:b0:314:2ea7:af4a with SMTP id e15-20020a5d4e8f000000b003142ea7af4amr9175177wru.13.1690259775749; Mon, 24 Jul 2023 21:36:15 -0700 (PDT) Received: from localhost.localdomain (ip-94-112-167-15.bb.vodafone.cz. [94.112.167.15]) by smtp.gmail.com with ESMTPSA id d1-20020a5d6441000000b00317643a93f4sm3500760wrw.96.2023.07.24.21.36.15 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 24 Jul 2023 21:36:15 -0700 (PDT) From: Ilya Dryomov To: ceph-devel@vger.kernel.org Cc: Dongsheng Yang Subject: [PATCH 1/3] rbd: make get_lock_owner_info() return a single locker or NULL Date: Tue, 25 Jul 2023 06:35:54 +0200 Message-ID: <20230725043559.123889-2-idryomov@gmail.com> X-Mailer: git-send-email 2.41.0 In-Reply-To: <20230725043559.123889-1-idryomov@gmail.com> References: <20230725043559.123889-1-idryomov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: ceph-devel@vger.kernel.org Make the "num_lockers can be only 0 or 1" assumption explicit and simplify the API by getting rid of output parameters in preparation for calling get_lock_owner_info() twice before blocklisting. Signed-off-by: Ilya Dryomov --- drivers/block/rbd.c | 84 +++++++++++++++++++++++++++------------------ 1 file changed, 51 insertions(+), 33 deletions(-) diff --git a/drivers/block/rbd.c b/drivers/block/rbd.c index bd0e075a5d89..dca6c1e5f6bc 100644 --- a/drivers/block/rbd.c +++ b/drivers/block/rbd.c @@ -3849,10 +3849,17 @@ static void wake_lock_waiters(struct rbd_device *rbd_dev, int result) list_splice_tail_init(&rbd_dev->acquiring_list, &rbd_dev->running_list); } -static int get_lock_owner_info(struct rbd_device *rbd_dev, - struct ceph_locker **lockers, u32 *num_lockers) +static void free_locker(struct ceph_locker *locker) +{ + if (locker) + ceph_free_lockers(locker, 1); +} + +static struct ceph_locker *get_lock_owner_info(struct rbd_device *rbd_dev) { struct ceph_osd_client *osdc = &rbd_dev->rbd_client->client->osdc; + struct ceph_locker *lockers; + u32 num_lockers; u8 lock_type; char *lock_tag; int ret; @@ -3861,39 +3868,45 @@ static int get_lock_owner_info(struct rbd_device *rbd_dev, ret = ceph_cls_lock_info(osdc, &rbd_dev->header_oid, &rbd_dev->header_oloc, RBD_LOCK_NAME, - &lock_type, &lock_tag, lockers, num_lockers); - if (ret) - return ret; + &lock_type, &lock_tag, &lockers, &num_lockers); + if (ret) { + rbd_warn(rbd_dev, "failed to retrieve lockers: %d", ret); + return ERR_PTR(ret); + } - if (*num_lockers == 0) { + if (num_lockers == 0) { dout("%s rbd_dev %p no lockers detected\n", __func__, rbd_dev); + lockers = NULL; goto out; } if (strcmp(lock_tag, RBD_LOCK_TAG)) { rbd_warn(rbd_dev, "locked by external mechanism, tag %s", lock_tag); - ret = -EBUSY; - goto out; + goto err_busy; } if (lock_type == CEPH_CLS_LOCK_SHARED) { rbd_warn(rbd_dev, "shared lock type detected"); - ret = -EBUSY; - goto out; + goto err_busy; } - if (strncmp((*lockers)[0].id.cookie, RBD_LOCK_COOKIE_PREFIX, + WARN_ON(num_lockers != 1); + if (strncmp(lockers[0].id.cookie, RBD_LOCK_COOKIE_PREFIX, strlen(RBD_LOCK_COOKIE_PREFIX))) { rbd_warn(rbd_dev, "locked by external mechanism, cookie %s", - (*lockers)[0].id.cookie); - ret = -EBUSY; - goto out; + lockers[0].id.cookie); + goto err_busy; } out: kfree(lock_tag); - return ret; + return lockers; + +err_busy: + kfree(lock_tag); + ceph_free_lockers(lockers, num_lockers); + return ERR_PTR(-EBUSY); } static int find_watcher(struct rbd_device *rbd_dev, @@ -3947,51 +3960,56 @@ static int find_watcher(struct rbd_device *rbd_dev, static int rbd_try_lock(struct rbd_device *rbd_dev) { struct ceph_client *client = rbd_dev->rbd_client->client; - struct ceph_locker *lockers; - u32 num_lockers; + struct ceph_locker *locker; int ret; for (;;) { + locker = NULL; + ret = rbd_lock(rbd_dev); if (ret != -EBUSY) - return ret; + goto out; /* determine if the current lock holder is still alive */ - ret = get_lock_owner_info(rbd_dev, &lockers, &num_lockers); - if (ret) - return ret; - - if (num_lockers == 0) + locker = get_lock_owner_info(rbd_dev); + if (IS_ERR(locker)) { + ret = PTR_ERR(locker); + locker = NULL; + goto out; + } + if (!locker) goto again; - ret = find_watcher(rbd_dev, lockers); + ret = find_watcher(rbd_dev, locker); if (ret) goto out; /* request lock or error */ rbd_warn(rbd_dev, "breaking header lock owned by %s%llu", - ENTITY_NAME(lockers[0].id.name)); + ENTITY_NAME(locker->id.name)); ret = ceph_monc_blocklist_add(&client->monc, - &lockers[0].info.addr); + &locker->info.addr); if (ret) { - rbd_warn(rbd_dev, "blocklist of %s%llu failed: %d", - ENTITY_NAME(lockers[0].id.name), ret); + rbd_warn(rbd_dev, "failed to blocklist %s%llu: %d", + ENTITY_NAME(locker->id.name), ret); goto out; } ret = ceph_cls_break_lock(&client->osdc, &rbd_dev->header_oid, &rbd_dev->header_oloc, RBD_LOCK_NAME, - lockers[0].id.cookie, - &lockers[0].id.name); - if (ret && ret != -ENOENT) + locker->id.cookie, &locker->id.name); + if (ret && ret != -ENOENT) { + rbd_warn(rbd_dev, "failed to break header lock: %d", + ret); goto out; + } again: - ceph_free_lockers(lockers, num_lockers); + free_locker(locker); } out: - ceph_free_lockers(lockers, num_lockers); + free_locker(locker); return ret; } From patchwork Tue Jul 25 04:35:55 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ilya Dryomov X-Patchwork-Id: 706673 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id E17ACC0015E for ; Tue, 25 Jul 2023 04:36:22 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231289AbjGYEgV (ORCPT ); Tue, 25 Jul 2023 00:36:21 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:59662 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231234AbjGYEgT (ORCPT ); Tue, 25 Jul 2023 00:36:19 -0400 Received: from mail-wr1-x429.google.com (mail-wr1-x429.google.com [IPv6:2a00:1450:4864:20::429]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 46F9EE64 for ; Mon, 24 Jul 2023 21:36:18 -0700 (PDT) Received: by mail-wr1-x429.google.com with SMTP id ffacd0b85a97d-3142a9ff6d8so4597441f8f.3 for ; Mon, 24 Jul 2023 21:36:18 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1690259777; x=1690864577; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=xY06zrmpkj+XtBzqfRQ39oDihyRHQpBOBuxeg2YX7J0=; b=nlvgW0R/PIHpiAUS/YVwUNUd79mC7eJInoQIea1nW4qD+n9w5H7Zg48UxuS3kIHecu fhlJ9TVmcDL6uE9151lNgBtML6prVr0utGxHk/TeYtuhKH12zqjdKEpqN5Yu1REk9sDP ntiAQRzRvTYPOwhqvMVE3dwwZpfOhnyfoP538E6r48CTNuoQZL29+PTle9BlS8SQ6I/k a5irbpz7s3bi/hzm+SV+z2w5zHGUFJp6tDcwUnbwKdVFQLmSHZByGF1ked7t0X3v9pwv rabHRiVE79vI21qYlV1c4/kFkPyYUz/AcqbCuMoTUQUYBYX7vFbCyjXhg9v9qoGi1VHl smAg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1690259777; x=1690864577; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=xY06zrmpkj+XtBzqfRQ39oDihyRHQpBOBuxeg2YX7J0=; b=RMcWrqR0Q9Wp2cTY1LDFee62wlXSvk0tNi+vN1bo3EseJYrriNPoxB7wiUng4EifNR ltNzteXsbIe9D4gb5l+Bs7GptX+ZMzho92q3D9SrEBXXt/2mj6uVKdKpO4M088FT28rs j1FUXvIOrUr+t89Y8uGPtyc41Bmd+6J7RLnSNrU48yI5531VXxwQSKD7Zj8RWn1LMQks aM3W/HGY5YdfjBL5b3mw8QsnuNYmSdqRconnhSsWxZtnmH2U5X82VmDZWwwxL50qZd2y 9hsny+tF2ncdAL/tK0XuP7LlvtWSzIQm4SOtYRkxSoGNqmnbRVXYtHG6Lwz32FCtgaYR 0Fbw== X-Gm-Message-State: ABy/qLZ9UhvGO50eQxAyf2dtNJTNBN9vTtvp+8R3wXDrzU6Tw3V5yOOO h3Qj3i1puDknel1C4lwORcv7AkZfTW8= X-Google-Smtp-Source: APBJJlFkZu7CYaAS9nv8qm2UMBjes2fyQyNHjuPEMa5OlRYGLwwlINRoXKHf0jt5gaK7iSPMW5nGng== X-Received: by 2002:adf:f184:0:b0:317:58e4:e941 with SMTP id h4-20020adff184000000b0031758e4e941mr4772997wro.33.1690259776553; Mon, 24 Jul 2023 21:36:16 -0700 (PDT) Received: from localhost.localdomain (ip-94-112-167-15.bb.vodafone.cz. [94.112.167.15]) by smtp.gmail.com with ESMTPSA id d1-20020a5d6441000000b00317643a93f4sm3500760wrw.96.2023.07.24.21.36.15 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 24 Jul 2023 21:36:16 -0700 (PDT) From: Ilya Dryomov To: ceph-devel@vger.kernel.org Cc: Dongsheng Yang Subject: [PATCH 2/3] rbd: harden get_lock_owner_info() a bit Date: Tue, 25 Jul 2023 06:35:55 +0200 Message-ID: <20230725043559.123889-3-idryomov@gmail.com> X-Mailer: git-send-email 2.41.0 In-Reply-To: <20230725043559.123889-1-idryomov@gmail.com> References: <20230725043559.123889-1-idryomov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: ceph-devel@vger.kernel.org - we want the exclusive lock type, so test for it directly - use sscanf() to actually parse the lock cookie and avoid admitting invalid handles - bail if locker has a blank address Signed-off-by: Ilya Dryomov --- drivers/block/rbd.c | 21 +++++++++++++++------ 1 file changed, 15 insertions(+), 6 deletions(-) diff --git a/drivers/block/rbd.c b/drivers/block/rbd.c index dca6c1e5f6bc..94629e826369 100644 --- a/drivers/block/rbd.c +++ b/drivers/block/rbd.c @@ -3862,10 +3862,9 @@ static struct ceph_locker *get_lock_owner_info(struct rbd_device *rbd_dev) u32 num_lockers; u8 lock_type; char *lock_tag; + u64 handle; int ret; - dout("%s rbd_dev %p\n", __func__, rbd_dev); - ret = ceph_cls_lock_info(osdc, &rbd_dev->header_oid, &rbd_dev->header_oloc, RBD_LOCK_NAME, &lock_type, &lock_tag, &lockers, &num_lockers); @@ -3886,18 +3885,28 @@ static struct ceph_locker *get_lock_owner_info(struct rbd_device *rbd_dev) goto err_busy; } - if (lock_type == CEPH_CLS_LOCK_SHARED) { - rbd_warn(rbd_dev, "shared lock type detected"); + if (lock_type != CEPH_CLS_LOCK_EXCLUSIVE) { + rbd_warn(rbd_dev, "incompatible lock type detected"); goto err_busy; } WARN_ON(num_lockers != 1); - if (strncmp(lockers[0].id.cookie, RBD_LOCK_COOKIE_PREFIX, - strlen(RBD_LOCK_COOKIE_PREFIX))) { + ret = sscanf(lockers[0].id.cookie, RBD_LOCK_COOKIE_PREFIX " %llu", + &handle); + if (ret != 1) { rbd_warn(rbd_dev, "locked by external mechanism, cookie %s", lockers[0].id.cookie); goto err_busy; } + if (ceph_addr_is_blank(&lockers[0].info.addr)) { + rbd_warn(rbd_dev, "locker has a blank address"); + goto err_busy; + } + + dout("%s rbd_dev %p got locker %s%llu@%pISpc/%u handle %llu\n", + __func__, rbd_dev, ENTITY_NAME(lockers[0].id.name), + &lockers[0].info.addr.in_addr, + le32_to_cpu(lockers[0].info.addr.nonce), handle); out: kfree(lock_tag); From patchwork Tue Jul 25 04:35:56 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ilya Dryomov X-Patchwork-Id: 706223 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id B0E3BEB64DD for ; Tue, 25 Jul 2023 04:36:24 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230314AbjGYEgW (ORCPT ); Tue, 25 Jul 2023 00:36:22 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:59668 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231314AbjGYEgU (ORCPT ); Tue, 25 Jul 2023 00:36:20 -0400 Received: from mail-wm1-x331.google.com (mail-wm1-x331.google.com [IPv6:2a00:1450:4864:20::331]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 061C7E55 for ; Mon, 24 Jul 2023 21:36:19 -0700 (PDT) Received: by mail-wm1-x331.google.com with SMTP id 5b1f17b1804b1-3fbd33a57dcso49753635e9.0 for ; Mon, 24 Jul 2023 21:36:18 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1690259777; x=1690864577; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=sUyjwZ53lab6sfzGfDl2TdRCVDTqp4D/S/ZXJ6DEFB4=; b=grCiKtjrguVb/H1SFWPeG+SAvvOfPIsQTSm9X1INO3J6+aThTbQjqTrVUOOlgJImF1 inNSquZodR/JvYz1k9oLgbYTpgKthT3+j09w4xV94M8uSgLkxJU+9zlNHcql71IaLFRE NZ072ZzGgOelDrOn7LKgL+t7FjkTiS6hXAJnJfR2JmZWpKggl9hWmxsLaTmLBtTGcoEP SnHclX+vnJzvU/MNZMjeNPxWH0a6Mv7rfKKMtgiaq2Li0mB5l+SQ9HAd9QCETD3TLy6/ zah4ajIlBmzvXFe4Ax5uHjGU2oh4/cntekMKm6LDZR9Ic3BB9F2B3JgCgFzGfig5k6Ve +Lyw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1690259777; x=1690864577; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=sUyjwZ53lab6sfzGfDl2TdRCVDTqp4D/S/ZXJ6DEFB4=; b=iqynlS8vLipBKiWwvkhV230Swn+/Kq7kesQt8oETEVd8mGOS+6nTbH5iOLQhaqRQLh IdQjXHGYLNjcQhDTdd3x2+v6mOfvLBOgkIHjw1usyF0m1reEmSCMXPE/bLBv9YJuTl9Y rxEBo10iNpXm3B6OAZAyfmbODssrXLOZRKKyYdkqsZOspyN0TXuNIbhWVSaSIs699K8b NA7k+wP051lv4MX/utCP7nsXlVL9JBoqRMVrpXYvfucpoDcSJ1HMw6qhwRx4AGHFKBVU eUryxIyvscAeu9RwbLcT24uGHr40KiHuZoA7RL5YND54/pk0B6jXZgNieQFN/E1t1h7B Cflw== X-Gm-Message-State: ABy/qLbrfj05P1k3J6cTMWZmxe90TRCJiga9Uky29+9PsJR9a1O5EvPy 0TAfkl8OgoynlCTS8w/PQMTD/DhbJds= X-Google-Smtp-Source: APBJJlH4wsYWUAacKDaVq+3l8Z5d/3ZgPU1nufW3d4Qqvvi0uQ5acnmhq4TiCZ661EC7MBX7km9SFQ== X-Received: by 2002:a05:600c:294b:b0:3f9:b9e7:2f8d with SMTP id n11-20020a05600c294b00b003f9b9e72f8dmr8395834wmd.2.1690259777491; Mon, 24 Jul 2023 21:36:17 -0700 (PDT) Received: from localhost.localdomain (ip-94-112-167-15.bb.vodafone.cz. [94.112.167.15]) by smtp.gmail.com with ESMTPSA id d1-20020a5d6441000000b00317643a93f4sm3500760wrw.96.2023.07.24.21.36.16 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 24 Jul 2023 21:36:16 -0700 (PDT) From: Ilya Dryomov To: ceph-devel@vger.kernel.org Cc: Dongsheng Yang Subject: [PATCH 3/3] rbd: retrieve and check lock owner twice before blocklisting Date: Tue, 25 Jul 2023 06:35:56 +0200 Message-ID: <20230725043559.123889-4-idryomov@gmail.com> X-Mailer: git-send-email 2.41.0 In-Reply-To: <20230725043559.123889-1-idryomov@gmail.com> References: <20230725043559.123889-1-idryomov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: ceph-devel@vger.kernel.org An attempt to acquire exclusive lock can race with the current lock owner closing the image: 1. lock is held by client123, rbd_lock() returns -EBUSY 2. get_lock_owner_info() returns client123 instance details 3. client123 closes the image, lock is released 4. find_watcher() returns 0 as there is no matching watcher anymore 5. client123 instance gets erroneously blocklisted Particularly impacted is mirror snapshot scheduler in snapshot-based mirroring since it happens to open and close images a lot (images are opened only for as long as it takes to take the next mirror snapshot, the same client instance is used for all images). To reduce the potential for erroneous blocklisting, retrieve the lock owner again after find_watcher() returns 0. If it's still there, make sure it matches the previously detected lock owner. Cc: stable@vger.kernel.org # 6d1736a0e432: rbd: make get_lock_owner_info() return a single locker or NULL Cc: stable@vger.kernel.org # 5dc06bec6a5b: rbd: harden get_lock_owner_info() a bit Cc: stable@vger.kernel.org Signed-off-by: Ilya Dryomov --- drivers/block/rbd.c | 16 ++++++++++++++-- include/linux/ceph/cls_lock_client.h | 10 ++++++++++ 2 files changed, 24 insertions(+), 2 deletions(-) diff --git a/drivers/block/rbd.c b/drivers/block/rbd.c index 94629e826369..e4b5829a03b4 100644 --- a/drivers/block/rbd.c +++ b/drivers/block/rbd.c @@ -3969,11 +3969,11 @@ static int find_watcher(struct rbd_device *rbd_dev, static int rbd_try_lock(struct rbd_device *rbd_dev) { struct ceph_client *client = rbd_dev->rbd_client->client; - struct ceph_locker *locker; + struct ceph_locker *locker, *refreshed_locker; int ret; for (;;) { - locker = NULL; + locker = refreshed_locker = NULL; ret = rbd_lock(rbd_dev); if (ret != -EBUSY) @@ -3993,6 +3993,16 @@ static int rbd_try_lock(struct rbd_device *rbd_dev) if (ret) goto out; /* request lock or error */ + refreshed_locker = get_lock_owner_info(rbd_dev); + if (IS_ERR(refreshed_locker)) { + ret = PTR_ERR(refreshed_locker); + refreshed_locker = NULL; + goto out; + } + if (!refreshed_locker || + !ceph_locker_equal(locker, refreshed_locker)) + goto again; + rbd_warn(rbd_dev, "breaking header lock owned by %s%llu", ENTITY_NAME(locker->id.name)); @@ -4014,10 +4024,12 @@ static int rbd_try_lock(struct rbd_device *rbd_dev) } again: + free_locker(refreshed_locker); free_locker(locker); } out: + free_locker(refreshed_locker); free_locker(locker); return ret; } diff --git a/include/linux/ceph/cls_lock_client.h b/include/linux/ceph/cls_lock_client.h index 17bc7584d1fe..b26f44ea38ca 100644 --- a/include/linux/ceph/cls_lock_client.h +++ b/include/linux/ceph/cls_lock_client.h @@ -24,6 +24,16 @@ struct ceph_locker { struct ceph_locker_info info; }; +static inline bool ceph_locker_equal(const struct ceph_locker *lhs, + const struct ceph_locker *rhs) +{ + return lhs->id.name.type == rhs->id.name.type && + lhs->id.name.num == rhs->id.name.num && + !strcmp(lhs->id.cookie, rhs->id.cookie) && + !memcmp(&lhs->info.addr, &rhs->info.addr, + sizeof(rhs->info.addr)); +} + int ceph_cls_lock(struct ceph_osd_client *osdc, struct ceph_object_id *oid, struct ceph_object_locator *oloc,