From patchwork Mon Jun 28 07:55:42 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Venky Shankar X-Patchwork-Id: 468033 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-19.1 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER, INCLUDES_PATCH, MAILING_LIST_MULTI, SPF_HELO_NONE, SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id CF934C2B9F4 for ; Mon, 28 Jun 2021 07:56:01 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id B9D5B60FDA for ; Mon, 28 Jun 2021 07:56:01 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232329AbhF1H60 (ORCPT ); Mon, 28 Jun 2021 03:58:26 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:39533 "EHLO us-smtp-delivery-124.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231725AbhF1H6Z (ORCPT ); Mon, 28 Jun 2021 03:58:25 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1624866959; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=N2IWcSW9pQYT8veexnZPnB6S/6OZGEVQRcr3XsQTZCo=; b=hKWqljIpp5RxT4FklpfZWIDnJIvo7Tdf6Fu8gSeWKwMa1ysbHktFjDMKFO5k1gVBYva4Bs BmvEmG3QFoJksGwYCfWrYj6BTm+2MCZcfCZCSw8Ja12LBkyqCVyXnILYOeblEc74ZHXkLO 5fc6rDCDB7uxRAUlQU8hvtMZP3m3EJg= Received: from mail-pg1-f197.google.com (mail-pg1-f197.google.com [209.85.215.197]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-276-enThur2yM_CvOr4iv_FElg-1; Mon, 28 Jun 2021 03:55:56 -0400 X-MC-Unique: enThur2yM_CvOr4iv_FElg-1 Received: by mail-pg1-f197.google.com with SMTP id p2-20020a63e6420000b02902271082c631so8712151pgj.5 for ; Mon, 28 Jun 2021 00:55:56 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=N2IWcSW9pQYT8veexnZPnB6S/6OZGEVQRcr3XsQTZCo=; b=HYuiBF/6Df3mOE/sY9TOLu2MrkcMBw+D+HvLgmU/93fV64u2iPqP+ERS6g/s10P40V 1KsGlVEObhl4YOSMXgUaoTylUQHOycXbwaSQmPaUB2C3hzi1LkSiTT49oXIWV1cKYZCj g+xs2bb8wsPoeUydlM9biEfTRMTpRKYTSZf45fm61DR9WK4qCj55W9O1Ut0tgoowaim4 4pCJtQKfkUtOLjtIOObvSaR69QJrVCRMacUkxu1ZcTMclGV+ZVpaeWQy0VYhFDORKXFI 8wGU2qWGX3I0YYUcnevBAytONjIZ6f2fWa1xG/Mu3mDPF0GSoYOz8RClg8Xs3BB9cIeA 39lQ== X-Gm-Message-State: AOAM531TsibpRFLkMor33qhhuHy4vNH8+f34FJ/Zm0dssXhzZ5XG71Jo izDsdQdoBzbOkICsu1EFG5dBx3NoChMvgz7KPRYyeKKMjKdVN1NDnMpricDzTASIEsxqU86qEr/ 3rrg8MpFrU/AfVxMy3nvOuA== X-Received: by 2002:aa7:8244:0:b029:2ec:968d:c1b4 with SMTP id e4-20020aa782440000b02902ec968dc1b4mr23520948pfn.32.1624866955280; Mon, 28 Jun 2021 00:55:55 -0700 (PDT) X-Google-Smtp-Source: ABdhPJxiGKZp7mGg7Af0g7LC667k4KHfM5aA/3dvF9h1XmFf1LH9a2f8TvFVCGoAnjj1G1DBQ6yE2A== X-Received: by 2002:aa7:8244:0:b029:2ec:968d:c1b4 with SMTP id e4-20020aa782440000b02902ec968dc1b4mr23520933pfn.32.1624866955036; Mon, 28 Jun 2021 00:55:55 -0700 (PDT) Received: from localhost.localdomain ([49.207.209.6]) by smtp.gmail.com with ESMTPSA id g123sm8304959pfb.187.2021.06.28.00.55.53 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 28 Jun 2021 00:55:54 -0700 (PDT) From: Venky Shankar To: jlayton@redhat.com, idryomov@gmail.com Cc: ceph-devel@vger.kernel.org, Venky Shankar Subject: [PATCH 1/4] ceph: new device mount syntax Date: Mon, 28 Jun 2021 13:25:42 +0530 Message-Id: <20210628075545.702106-2-vshankar@redhat.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210628075545.702106-1-vshankar@redhat.com> References: <20210628075545.702106-1-vshankar@redhat.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: ceph-devel@vger.kernel.org Old mount device syntax (source) has the following problems: - mounts to the same cluster but with different fsnames and/or creds have identical device string which can confuse xfstests. - Userspace mount helper tool resolves monitor addresses and fill in mon addrs automatically, but that means the device shown in /proc/mounts is different than what was used for mounting. New device syntax is as follows: cephuser@fsid.mycephfs2=/path Note, there is no "monitor address" in the device string. That gets passed in as mount option. This keeps the device string same when monitor addresses change (on remounts). Also note that the userspace mount helper tool is backward compatible. I.e., the mount helper will fallback to using old syntax after trying to mount with the new syntax. Signed-off-by: Venky Shankar --- fs/ceph/super.c | 117 +++++++++++++++++++++++++++++++++++++++++++----- fs/ceph/super.h | 3 ++ 2 files changed, 110 insertions(+), 10 deletions(-) diff --git a/fs/ceph/super.c b/fs/ceph/super.c index 9b1b7f4cfdd4..950a28ad9c59 100644 --- a/fs/ceph/super.c +++ b/fs/ceph/super.c @@ -145,6 +145,7 @@ enum { Opt_mds_namespace, Opt_recover_session, Opt_source, + Opt_mon_addr, /* string args above */ Opt_dirstat, Opt_rbytes, @@ -196,6 +197,7 @@ static const struct fs_parameter_spec ceph_mount_parameters[] = { fsparam_u32 ("rsize", Opt_rsize), fsparam_string ("snapdirname", Opt_snapdirname), fsparam_string ("source", Opt_source), + fsparam_string ("mon_addr", Opt_mon_addr), fsparam_u32 ("wsize", Opt_wsize), fsparam_flag_no ("wsync", Opt_wsync), {} @@ -226,10 +228,68 @@ static void canonicalize_path(char *path) path[j] = '\0'; } +static int ceph_parse_old_source(const char *dev_name, const char *dev_name_end, + struct fs_context *fc) +{ + int r; + struct ceph_parse_opts_ctx *pctx = fc->fs_private; + struct ceph_mount_options *fsopt = pctx->opts; + + if (*dev_name_end != ':') + return invalfc(fc, "separator ':' missing in source"); + + r = ceph_parse_mon_ips(dev_name, dev_name_end - dev_name, + pctx->copts, fc->log.log); + if (r) + return r; + + fsopt->new_dev_syntax = false; + return 0; +} + +static int ceph_parse_new_source(const char *dev_name, const char *dev_name_end, + struct fs_context *fc) +{ + struct ceph_parse_opts_ctx *pctx = fc->fs_private; + struct ceph_mount_options *fsopt = pctx->opts; + char *fsid_start, *fs_name_start; + + if (*dev_name_end != '=') + return invalfc(fc, "separator '=' missing in source"); + + fsid_start = strchr(dev_name, '@'); + if (!fsid_start) + return invalfc(fc, "missing cluster fsid"); + ++fsid_start; /* start of cluster fsid */ + + fs_name_start = strchr(fsid_start, '.'); + if (!fs_name_start) + return invalfc(fc, "missing file system name"); + + ++fs_name_start; /* start of file system name */ + fsopt->mds_namespace = kstrndup(fs_name_start, + dev_name_end - fs_name_start, GFP_KERNEL); + if (!fsopt->mds_namespace) + return -ENOMEM; + dout("file system (mds namespace) '%s'\n", fsopt->mds_namespace); + + fsopt->new_dev_syntax = true; + return 0; +} + /* - * Parse the source parameter. Distinguish the server list from the path. + * Parse the source parameter for new device format. Distinguish the device + * spec from the path. Try parsing new device format and fallback to old + * format if needed. * - * The source will look like: + * New device syntax will looks like: + * =/ + * where + * is name@fsid.fsname + * is optional, but if present must begin with '/' + * (monitor addresses are passed via mount option) + * + * Old device syntax is: * [,...]:[] * where * is [:] @@ -262,22 +322,48 @@ static int ceph_parse_source(struct fs_parameter *param, struct fs_context *fc) dev_name_end = dev_name + strlen(dev_name); } - dev_name_end--; /* back up to ':' separator */ - if (dev_name_end < dev_name || *dev_name_end != ':') - return invalfc(fc, "No path or : separator in source"); + dev_name_end--; /* back up to separator */ + if (dev_name_end < dev_name) + return invalfc(fc, "path missing in source"); dout("device name '%.*s'\n", (int)(dev_name_end - dev_name), dev_name); if (fsopt->server_path) dout("server path '%s'\n", fsopt->server_path); - ret = ceph_parse_mon_ips(param->string, dev_name_end - dev_name, - pctx->copts, fc->log.log); + dout("trying new device syntax"); + ret = ceph_parse_new_source(dev_name, dev_name_end, fc); + if (ret == 0) + goto done; + + dout("trying old device syntax"); + ret = ceph_parse_old_source(dev_name, dev_name_end, fc); if (ret) - return ret; + goto out; + done: fc->source = param->string; param->string = NULL; - return 0; + out: + return ret; +} + +static int ceph_parse_mon_addr(struct fs_parameter *param, + struct fs_context *fc) +{ + int r; + struct ceph_parse_opts_ctx *pctx = fc->fs_private; + struct ceph_mount_options *fsopt = pctx->opts; + + kfree(fsopt->mon_addr); + fsopt->mon_addr = kstrdup(param->string, GFP_KERNEL); + if (!fsopt->mon_addr) + return -ENOMEM; + + strreplace(param->string, '/', ','); + r = ceph_parse_mon_ips(param->string, strlen(param->string), + pctx->copts, fc->log.log); + param->string = NULL; + return r; } static int ceph_parse_mount_param(struct fs_context *fc, @@ -322,6 +408,8 @@ static int ceph_parse_mount_param(struct fs_context *fc, if (fc->source) return invalfc(fc, "Multiple sources specified"); return ceph_parse_source(param, fc); + case Opt_mon_addr: + return ceph_parse_mon_addr(param, fc); case Opt_wsize: if (result.uint_32 < PAGE_SIZE || result.uint_32 > CEPH_MAX_WRITE_SIZE) @@ -473,6 +561,7 @@ static void destroy_mount_options(struct ceph_mount_options *args) kfree(args->mds_namespace); kfree(args->server_path); kfree(args->fscache_uniq); + kfree(args->mon_addr); kfree(args); } @@ -516,6 +605,10 @@ static int compare_mount_options(struct ceph_mount_options *new_fsopt, if (ret) return ret; + ret = strcmp_null(fsopt1->mon_addr, fsopt2->mon_addr); + if (ret) + return ret; + return ceph_compare_options(new_opt, fsc->client); } @@ -571,9 +664,13 @@ static int ceph_show_options(struct seq_file *m, struct dentry *root) if ((fsopt->flags & CEPH_MOUNT_OPT_NOCOPYFROM) == 0) seq_puts(m, ",copyfrom"); - if (fsopt->mds_namespace) + /* dump mds_namespace when old device syntax is in use */ + if (fsopt->mds_namespace && !fsopt->new_dev_syntax) seq_show_option(m, "mds_namespace", fsopt->mds_namespace); + if (fsopt->mon_addr) + seq_printf(m, ",mon_addr=%s", fsopt->mon_addr); + if (fsopt->flags & CEPH_MOUNT_OPT_CLEANRECOVER) seq_show_option(m, "recover_session", "clean"); diff --git a/fs/ceph/super.h b/fs/ceph/super.h index 839e6b0239ee..557348ff3203 100644 --- a/fs/ceph/super.h +++ b/fs/ceph/super.h @@ -88,6 +88,8 @@ struct ceph_mount_options { unsigned int max_readdir; /* max readdir result (entries) */ unsigned int max_readdir_bytes; /* max readdir result (bytes) */ + bool new_dev_syntax; + /* * everything above this point can be memcmp'd; everything below * is handled in compare_mount_options() @@ -97,6 +99,7 @@ struct ceph_mount_options { char *mds_namespace; /* default NULL */ char *server_path; /* default NULL (means "/") */ char *fscache_uniq; /* default NULL */ + char *mon_addr; }; struct ceph_fs_client { From patchwork Mon Jun 28 07:55:45 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Venky Shankar X-Patchwork-Id: 468032 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-19.1 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER, INCLUDES_PATCH, MAILING_LIST_MULTI, SPF_HELO_NONE, SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 897D4C49EA7 for ; Mon, 28 Jun 2021 07:56:06 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 7123A60FDA for ; Mon, 28 Jun 2021 07:56:06 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232414AbhF1H6b (ORCPT ); Mon, 28 Jun 2021 03:58:31 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:34964 "EHLO us-smtp-delivery-124.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232317AbhF1H6a (ORCPT ); Mon, 28 Jun 2021 03:58:30 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1624866964; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=ei9np5dyF4bOpwEaKKW/3KrcFjx/uy1Dba5DpeOc0ik=; b=FFa1JcWcsO9M8WWVZL4flguJqV7vkq1MmH+xirJJ1IFDtWw+dLbYAogklAYUz4qhWOs0su GRzbOqxt2zxyMvL4xjmEgGIh16MhgmIOv0WVSX7j71bgKlLHXG+lJGLDi3fWgDJ844hBdP K5TvVfRQCoeoH8kZvH+euH6VReLY3ok= Received: from mail-pl1-f197.google.com (mail-pl1-f197.google.com [209.85.214.197]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-598-rXpBzRy4PumuFZG74PHpjg-1; Mon, 28 Jun 2021 03:56:03 -0400 X-MC-Unique: rXpBzRy4PumuFZG74PHpjg-1 Received: by mail-pl1-f197.google.com with SMTP id a6-20020a1709027d86b02901019f88b046so5433654plm.21 for ; Mon, 28 Jun 2021 00:56:03 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=ei9np5dyF4bOpwEaKKW/3KrcFjx/uy1Dba5DpeOc0ik=; b=cRwryHRngOVo4PkKR12x/lzAAt3iliy4jr4C/TySXOpxogObfHkiitFaMLAUhjPr4K pkezsq0Bn9YpPjgXf951m6ThcTEOIwg7d+flKX2sFPKb5yMZ0EtZiieS6+ciGd+g8gWR k1t89Ig5xoanamkKWbigCKxCGqhElHLj5WgTm4sE4HYtn3aq0jcZQK3C4sCGcyBB8gNT OBgP44cyv1zPfeqzsLhmPaT5CWpr80gJLWDSAppMhrSOmwj/laAI5GjMh6nQdktk7FKM fyDZNpnP3BTXjvTTjspMhIsOHY/N6FBByMjbYvKmPQ4z1/TOR3Z1Ta3Rxh2730oBzhXN /f0g== X-Gm-Message-State: AOAM532oIYMGvlMxsTkGMMYPJzeKwM0RkAAFFQBEACA15oSFZpjfzJhV khpMo6cNtaonFexyaJMAw6iSmerU56sgsS1gWs+PMeHpRhAWKxPXGRNGOZNfNvd9dgaZlGLbSij NwmPoYkTaLWxhKr313G/wuw== X-Received: by 2002:a62:dd8b:0:b029:2e9:731a:e22e with SMTP id w133-20020a62dd8b0000b02902e9731ae22emr23405270pff.69.1624866962250; Mon, 28 Jun 2021 00:56:02 -0700 (PDT) X-Google-Smtp-Source: ABdhPJwkFR5vMnFmW0Mmz3RSrP6kb6axAH8h/cABDxSWcjxLgD6NKorqBdTtmrrPtPYSY1r/51MEZg== X-Received: by 2002:a62:dd8b:0:b029:2e9:731a:e22e with SMTP id w133-20020a62dd8b0000b02902e9731ae22emr23405256pff.69.1624866962075; Mon, 28 Jun 2021 00:56:02 -0700 (PDT) Received: from localhost.localdomain ([49.207.209.6]) by smtp.gmail.com with ESMTPSA id g123sm8304959pfb.187.2021.06.28.00.56.00 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 28 Jun 2021 00:56:01 -0700 (PDT) From: Venky Shankar To: jlayton@redhat.com, idryomov@gmail.com Cc: ceph-devel@vger.kernel.org, Venky Shankar Subject: [PATCH 4/4] doc: document new CephFS mount device syntax Date: Mon, 28 Jun 2021 13:25:45 +0530 Message-Id: <20210628075545.702106-5-vshankar@redhat.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210628075545.702106-1-vshankar@redhat.com> References: <20210628075545.702106-1-vshankar@redhat.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: ceph-devel@vger.kernel.org Signed-off-by: Venky Shankar --- Documentation/filesystems/ceph.rst | 23 ++++++++++++++++++++--- 1 file changed, 20 insertions(+), 3 deletions(-) diff --git a/Documentation/filesystems/ceph.rst b/Documentation/filesystems/ceph.rst index 7d2ef4e27273..e46f9091b851 100644 --- a/Documentation/filesystems/ceph.rst +++ b/Documentation/filesystems/ceph.rst @@ -82,7 +82,7 @@ Mount Syntax The basic mount syntax is:: - # mount -t ceph monip[:port][,monip2[:port]...]:/[subdir] mnt + # mount -t ceph user@fsid.fs_name=/[subdir] mnt -o mon_host=monip1[:port][/monip2[:port]] You only need to specify a single monitor, as the client will get the full list when it connects. (However, if the monitor you specify @@ -90,16 +90,33 @@ happens to be down, the mount won't succeed.) The port can be left off if the monitor is using the default. So if the monitor is at 1.2.3.4:: - # mount -t ceph 1.2.3.4:/ /mnt/ceph + # mount -t ceph cephuser@07fe3187-00d9-42a3-814b-72a4d5e7d5be.cephfs=/ /mnt/ceph -o mon_host=1.2.3.4 is sufficient. If /sbin/mount.ceph is installed, a hostname can be -used instead of an IP address. +used instead of an IP address and the cluster FSID can be left out +(as the mount helper will fill it in by reading the ceph configuration +file):: + # mount -t ceph cephuser@cephfs=/ /mnt/ceph -o mon_host=mon-addr +Multiple monitor addresses can be passed by separating each address with a slash (`/`):: + + # mount -t ceph cephuser@cephfs=/ /mnt/ceph -o mon_host=192.168.1.100/192.168.1.101 + +When using the mount helper, monitor address can be read from ceph +configuration file if available. Note that, the cluster FSID (passed as part +of the device string) is validated by checking it with the FSID reported by +the monitor. Mount Options ============= + mon_host=ip_address[:port][/ip_address[:port]] + Monitor address to the cluster + + fsid=cluster-id + FSID of the cluster + ip=A.B.C.D[:N] Specify the IP and/or port the client should bind to locally. There is normally not much reason to do this. If the IP is not