From patchwork Tue Mar 10 17:47:07 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Lorenz Bauer X-Patchwork-Id: 222750 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-9.9 required=3.0 tests=DKIMWL_WL_HIGH, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY, SPF_HELO_NONE, SPF_PASS, URIBL_BLOCKED, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 83E19C10F27 for ; Tue, 10 Mar 2020 17:47:32 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 44C3920727 for ; Tue, 10 Mar 2020 17:47:32 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=cloudflare.com header.i=@cloudflare.com header.b="v4O5uI+Z" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727282AbgCJRrb (ORCPT ); Tue, 10 Mar 2020 13:47:31 -0400 Received: from mail-wm1-f66.google.com ([209.85.128.66]:52553 "EHLO mail-wm1-f66.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727082AbgCJRrb (ORCPT ); Tue, 10 Mar 2020 13:47:31 -0400 Received: by mail-wm1-f66.google.com with SMTP id 11so2079990wmo.2 for ; Tue, 10 Mar 2020 10:47:29 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cloudflare.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=VOVxpIVTl1mbMOrH4rGvuKP/ssyXIX2mzp/+xoiFMfw=; b=v4O5uI+Zw8nHFm0oKKYzkXlBVbyCW4vvzZMubbOI5hrGUsXM+nWTpQ72uSE4DHuBfs PjWBPJCiEXST44w81hzSCb/sp9DR3DtUWCk5WjnDRjBiref2kWPDkyvbGjOe573KP7Dn f9hWNENyvHoWciaJJ0HFXkOiWkSWj7k5b90OE= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=VOVxpIVTl1mbMOrH4rGvuKP/ssyXIX2mzp/+xoiFMfw=; b=ejBWTpREscPolSJgex1LY7l7ABlfL8/a0raetm9St5W6WOscwSpBVaz0OluzvuOzgA sd8ta9oISUBXo5o6cQjrQu7n6YrLumuzan0TAXAq/x9lW/HDteLhV69cdhLDLnqYLVzD ux2A8egLK1VpJEISkHBn3rMR9zAJAy9bdL2J2vKArUon6qcgEv/9GFdLboifNJcmTPMT JoV3w9AsXJf57VeBz0l/7Z1PcKwgNwVRV0YMV6AdM61bFri9dCqyj6UtJt9ughmpXWni 9oPfLZ7UhhZQIYRIP2/TKOsg5Umqd4m5MoEiRISGOJZcOTD9IvSHXaunGVFB/2coLPoJ AYMg== X-Gm-Message-State: ANhLgQ3yR3ptddIKguwWck8XPXvOzUurIezSevUSOKQ3SKJb4aJq20fX 0+1JEIdifq3atPJDL6VSGy/3+gDGWpE= X-Google-Smtp-Source: ADFU+vuSj7BjZpChlQFTQVFKfYrsLOKUpJKHl7GK7T5wV+os0aT4UwkwkF0NTlFg1QRB0zdW7IW7BQ== X-Received: by 2002:a1c:9c96:: with SMTP id f144mr3131417wme.98.1583862448310; Tue, 10 Mar 2020 10:47:28 -0700 (PDT) Received: from localhost.localdomain ([2a06:98c0:1000:8250:9494:775c:e7b6:e690]) by smtp.gmail.com with ESMTPSA id k4sm9118691wrx.27.2020.03.10.10.47.26 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 10 Mar 2020 10:47:27 -0700 (PDT) From: Lorenz Bauer To: Alexei Starovoitov , Daniel Borkmann Cc: kernel-team@cloudflare.com, Lorenz Bauer , netdev@vger.kernel.org, bpf@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH 1/5] bpf: add map_copy_value hook Date: Tue, 10 Mar 2020 17:47:07 +0000 Message-Id: <20200310174711.7490-2-lmb@cloudflare.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20200310174711.7490-1-lmb@cloudflare.com> References: <20200310174711.7490-1-lmb@cloudflare.com> MIME-Version: 1.0 Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org bpf_map_copy_value has a lot of special cases for different map types that want more control than map_lookup_elem provides. On closer inspection, almost all of them follow the pattern int func(struct bpf_map *, void *, void *) Introduce a new member map_copy_value to struct bpf_map_ops, and convert the current special cases to use it. Signed-off-by: Lorenz Bauer --- include/linux/bpf-cgroup.h | 5 ----- include/linux/bpf.h | 21 +-------------------- include/linux/bpf_types.h | 2 +- kernel/bpf/arraymap.c | 13 ++++++++++--- kernel/bpf/bpf_struct_ops.c | 7 ++++--- kernel/bpf/hashtab.c | 10 +++++++--- kernel/bpf/local_storage.c | 14 +++++++++++++- kernel/bpf/reuseport_array.c | 5 +++-- kernel/bpf/syscall.c | 24 ++++-------------------- 9 files changed, 43 insertions(+), 58 deletions(-) diff --git a/include/linux/bpf-cgroup.h b/include/linux/bpf-cgroup.h index a7cd5c7a2509..6741a6c460f6 100644 --- a/include/linux/bpf-cgroup.h +++ b/include/linux/bpf-cgroup.h @@ -162,7 +162,6 @@ void bpf_cgroup_storage_unlink(struct bpf_cgroup_storage *storage); int bpf_cgroup_storage_assign(struct bpf_prog_aux *aux, struct bpf_map *map); void bpf_cgroup_storage_release(struct bpf_prog_aux *aux, struct bpf_map *map); -int bpf_percpu_cgroup_storage_copy(struct bpf_map *map, void *key, void *value); int bpf_percpu_cgroup_storage_update(struct bpf_map *map, void *key, void *value, u64 flags); @@ -370,10 +369,6 @@ static inline struct bpf_cgroup_storage *bpf_cgroup_storage_alloc( struct bpf_prog *prog, enum bpf_cgroup_storage_type stype) { return NULL; } static inline void bpf_cgroup_storage_free( struct bpf_cgroup_storage *storage) {} -static inline int bpf_percpu_cgroup_storage_copy(struct bpf_map *map, void *key, - void *value) { - return 0; -} static inline int bpf_percpu_cgroup_storage_update(struct bpf_map *map, void *key, void *value, u64 flags) { return 0; diff --git a/include/linux/bpf.h b/include/linux/bpf.h index 94a329b9da81..ad9f3be830f0 100644 --- a/include/linux/bpf.h +++ b/include/linux/bpf.h @@ -44,6 +44,7 @@ struct bpf_map_ops { int (*map_get_next_key)(struct bpf_map *map, void *key, void *next_key); void (*map_release_uref)(struct bpf_map *map); void *(*map_lookup_elem_sys_only)(struct bpf_map *map, void *key); + int (*map_copy_value)(struct bpf_map *map, void *key, void *value); int (*map_lookup_batch)(struct bpf_map *map, const union bpf_attr *attr, union bpf_attr __user *uattr); int (*map_lookup_and_delete_batch)(struct bpf_map *map, @@ -741,8 +742,6 @@ const struct bpf_struct_ops *bpf_struct_ops_find(u32 type_id); void bpf_struct_ops_init(struct btf *btf, struct bpf_verifier_log *log); bool bpf_struct_ops_get(const void *kdata); void bpf_struct_ops_put(const void *kdata); -int bpf_struct_ops_map_sys_lookup_elem(struct bpf_map *map, void *key, - void *value); static inline bool bpf_try_module_get(const void *data, struct module *owner) { if (owner == BPF_MODULE_OWNER) @@ -774,12 +773,6 @@ static inline void bpf_module_put(const void *data, struct module *owner) { module_put(owner); } -static inline int bpf_struct_ops_map_sys_lookup_elem(struct bpf_map *map, - void *key, - void *value) -{ - return -EINVAL; -} #endif struct bpf_array { @@ -1082,8 +1075,6 @@ struct bpf_link *bpf_link_get_from_fd(u32 ufd); int bpf_obj_pin_user(u32 ufd, const char __user *pathname); int bpf_obj_get_user(const char __user *pathname, int flags); -int bpf_percpu_hash_copy(struct bpf_map *map, void *key, void *value); -int bpf_percpu_array_copy(struct bpf_map *map, void *key, void *value); int bpf_percpu_hash_update(struct bpf_map *map, void *key, void *value, u64 flags); int bpf_percpu_array_update(struct bpf_map *map, void *key, void *value, @@ -1093,10 +1084,8 @@ int bpf_stackmap_copy(struct bpf_map *map, void *key, void *value); int bpf_fd_array_map_update_elem(struct bpf_map *map, struct file *map_file, void *key, void *value, u64 map_flags); -int bpf_fd_array_map_lookup_elem(struct bpf_map *map, void *key, u32 *value); int bpf_fd_htab_map_update_elem(struct bpf_map *map, struct file *map_file, void *key, void *value, u64 map_flags); -int bpf_fd_htab_map_lookup_elem(struct bpf_map *map, void *key, u32 *value); int bpf_get_file_flag(int flags); int bpf_check_uarg_tail_zero(void __user *uaddr, size_t expected_size, @@ -1437,8 +1426,6 @@ static inline int sock_map_get_from_fd(const union bpf_attr *attr, #if defined(CONFIG_INET) && defined(CONFIG_BPF_SYSCALL) void bpf_sk_reuseport_detach(struct sock *sk); -int bpf_fd_reuseport_array_lookup_elem(struct bpf_map *map, void *key, - void *value); int bpf_fd_reuseport_array_update_elem(struct bpf_map *map, void *key, void *value, u64 map_flags); #else @@ -1447,12 +1434,6 @@ static inline void bpf_sk_reuseport_detach(struct sock *sk) } #ifdef CONFIG_BPF_SYSCALL -static inline int bpf_fd_reuseport_array_lookup_elem(struct bpf_map *map, - void *key, void *value) -{ - return -EOPNOTSUPP; -} - static inline int bpf_fd_reuseport_array_update_elem(struct bpf_map *map, void *key, void *value, u64 map_flags) diff --git a/include/linux/bpf_types.h b/include/linux/bpf_types.h index c81d4ece79a4..4949638cd049 100644 --- a/include/linux/bpf_types.h +++ b/include/linux/bpf_types.h @@ -81,7 +81,7 @@ BPF_MAP_TYPE(BPF_MAP_TYPE_CGROUP_ARRAY, cgroup_array_map_ops) #endif #ifdef CONFIG_CGROUP_BPF BPF_MAP_TYPE(BPF_MAP_TYPE_CGROUP_STORAGE, cgroup_storage_map_ops) -BPF_MAP_TYPE(BPF_MAP_TYPE_PERCPU_CGROUP_STORAGE, cgroup_storage_map_ops) +BPF_MAP_TYPE(BPF_MAP_TYPE_PERCPU_CGROUP_STORAGE, percpu_cgroup_storage_map_ops) #endif BPF_MAP_TYPE(BPF_MAP_TYPE_HASH, htab_map_ops) BPF_MAP_TYPE(BPF_MAP_TYPE_PERCPU_HASH, htab_percpu_map_ops) diff --git a/kernel/bpf/arraymap.c b/kernel/bpf/arraymap.c index 95d77770353c..58a0a8b3abe3 100644 --- a/kernel/bpf/arraymap.c +++ b/kernel/bpf/arraymap.c @@ -249,7 +249,8 @@ static void *percpu_array_map_lookup_elem(struct bpf_map *map, void *key) return this_cpu_ptr(array->pptrs[index & array->index_mask]); } -int bpf_percpu_array_copy(struct bpf_map *map, void *key, void *value) +static int percpu_array_map_copy_value(struct bpf_map *map, void *key, + void *value) { struct bpf_array *array = container_of(map, struct bpf_array, map); u32 index = *(u32 *)key; @@ -513,6 +514,7 @@ const struct bpf_map_ops percpu_array_map_ops = { .map_free = array_map_free, .map_get_next_key = array_map_get_next_key, .map_lookup_elem = percpu_array_map_lookup_elem, + .map_copy_value = percpu_array_map_copy_value, .map_update_elem = array_map_update_elem, .map_delete_elem = array_map_delete_elem, .map_seq_show_elem = percpu_array_map_seq_show_elem, @@ -550,7 +552,8 @@ static void *fd_array_map_lookup_elem(struct bpf_map *map, void *key) } /* only called from syscall */ -int bpf_fd_array_map_lookup_elem(struct bpf_map *map, void *key, u32 *value) +static int fd_array_map_lookup_elem_sys_copy(struct bpf_map *map, void *key, + void *value) { void **elem, *ptr; int ret = 0; @@ -561,7 +564,7 @@ int bpf_fd_array_map_lookup_elem(struct bpf_map *map, void *key, u32 *value) rcu_read_lock(); elem = array_map_lookup_elem(map, key); if (elem && (ptr = READ_ONCE(*elem))) - *value = map->ops->map_fd_sys_lookup_elem(ptr); + *(u32 *)value = map->ops->map_fd_sys_lookup_elem(ptr); else ret = -ENOENT; rcu_read_unlock(); @@ -872,6 +875,7 @@ const struct bpf_map_ops prog_array_map_ops = { .map_poke_run = prog_array_map_poke_run, .map_get_next_key = array_map_get_next_key, .map_lookup_elem = fd_array_map_lookup_elem, + .map_copy_value = fd_array_map_lookup_elem_sys_copy, .map_delete_elem = fd_array_map_delete_elem, .map_fd_get_ptr = prog_fd_array_get_ptr, .map_fd_put_ptr = prog_fd_array_put_ptr, @@ -962,6 +966,7 @@ const struct bpf_map_ops perf_event_array_map_ops = { .map_free = fd_array_map_free, .map_get_next_key = array_map_get_next_key, .map_lookup_elem = fd_array_map_lookup_elem, + .map_copy_value = fd_array_map_lookup_elem_sys_copy, .map_delete_elem = fd_array_map_delete_elem, .map_fd_get_ptr = perf_event_fd_array_get_ptr, .map_fd_put_ptr = perf_event_fd_array_put_ptr, @@ -995,6 +1000,7 @@ const struct bpf_map_ops cgroup_array_map_ops = { .map_free = cgroup_fd_array_free, .map_get_next_key = array_map_get_next_key, .map_lookup_elem = fd_array_map_lookup_elem, + .map_copy_value = fd_array_map_lookup_elem_sys_copy, .map_delete_elem = fd_array_map_delete_elem, .map_fd_get_ptr = cgroup_fd_array_get_ptr, .map_fd_put_ptr = cgroup_fd_array_put_ptr, @@ -1078,6 +1084,7 @@ const struct bpf_map_ops array_of_maps_map_ops = { .map_free = array_of_map_free, .map_get_next_key = array_map_get_next_key, .map_lookup_elem = array_of_map_lookup_elem, + .map_copy_value = fd_array_map_lookup_elem_sys_copy, .map_delete_elem = fd_array_map_delete_elem, .map_fd_get_ptr = bpf_map_fd_get_ptr, .map_fd_put_ptr = bpf_map_fd_put_ptr, diff --git a/kernel/bpf/bpf_struct_ops.c b/kernel/bpf/bpf_struct_ops.c index ca5cc8cdb6eb..cc1d7d1077c1 100644 --- a/kernel/bpf/bpf_struct_ops.c +++ b/kernel/bpf/bpf_struct_ops.c @@ -238,8 +238,8 @@ static int bpf_struct_ops_map_get_next_key(struct bpf_map *map, void *key, return 0; } -int bpf_struct_ops_map_sys_lookup_elem(struct bpf_map *map, void *key, - void *value) +static int bpf_struct_ops_map_copy_value(struct bpf_map *map, void *key, + void *value) { struct bpf_struct_ops_map *st_map = (struct bpf_struct_ops_map *)map; struct bpf_struct_ops_value *uvalue, *kvalue; @@ -509,7 +509,7 @@ static void bpf_struct_ops_map_seq_show_elem(struct bpf_map *map, void *key, if (!value) return; - err = bpf_struct_ops_map_sys_lookup_elem(map, key, value); + err = bpf_struct_ops_map_copy_value(map, key, value); if (!err) { btf_type_seq_show(btf_vmlinux, map->btf_vmlinux_value_type_id, value, m); @@ -609,6 +609,7 @@ const struct bpf_map_ops bpf_struct_ops_map_ops = { .map_free = bpf_struct_ops_map_free, .map_get_next_key = bpf_struct_ops_map_get_next_key, .map_lookup_elem = bpf_struct_ops_map_lookup_elem, + .map_copy_value = bpf_struct_ops_map_copy_value, .map_delete_elem = bpf_struct_ops_map_delete_elem, .map_update_elem = bpf_struct_ops_map_update_elem, .map_seq_show_elem = bpf_struct_ops_map_seq_show_elem, diff --git a/kernel/bpf/hashtab.c b/kernel/bpf/hashtab.c index d541c8486c95..f5452a8a5177 100644 --- a/kernel/bpf/hashtab.c +++ b/kernel/bpf/hashtab.c @@ -1664,7 +1664,8 @@ static void *htab_lru_percpu_map_lookup_elem(struct bpf_map *map, void *key) return NULL; } -int bpf_percpu_hash_copy(struct bpf_map *map, void *key, void *value) +static int htab_percpu_map_copy_value(struct bpf_map *map, void *key, + void *value) { struct htab_elem *l; void __percpu *pptr; @@ -1749,6 +1750,7 @@ const struct bpf_map_ops htab_percpu_map_ops = { .map_free = htab_map_free, .map_get_next_key = htab_map_get_next_key, .map_lookup_elem = htab_percpu_map_lookup_elem, + .map_copy_value = htab_percpu_map_copy_value, .map_update_elem = htab_percpu_map_update_elem, .map_delete_elem = htab_map_delete_elem, .map_seq_show_elem = htab_percpu_map_seq_show_elem, @@ -1761,6 +1763,7 @@ const struct bpf_map_ops htab_lru_percpu_map_ops = { .map_free = htab_map_free, .map_get_next_key = htab_map_get_next_key, .map_lookup_elem = htab_lru_percpu_map_lookup_elem, + .map_copy_value = htab_percpu_map_copy_value, .map_update_elem = htab_lru_percpu_map_update_elem, .map_delete_elem = htab_lru_map_delete_elem, .map_seq_show_elem = htab_percpu_map_seq_show_elem, @@ -1796,7 +1799,7 @@ static void fd_htab_map_free(struct bpf_map *map) } /* only called from syscall */ -int bpf_fd_htab_map_lookup_elem(struct bpf_map *map, void *key, u32 *value) +static int fd_htab_map_copy_value(struct bpf_map *map, void *key, void *value) { void **ptr; int ret = 0; @@ -1807,7 +1810,7 @@ int bpf_fd_htab_map_lookup_elem(struct bpf_map *map, void *key, u32 *value) rcu_read_lock(); ptr = htab_map_lookup_elem(map, key); if (ptr) - *value = map->ops->map_fd_sys_lookup_elem(READ_ONCE(*ptr)); + *(u32 *)value = map->ops->map_fd_sys_lookup_elem(READ_ONCE(*ptr)); else ret = -ENOENT; rcu_read_unlock(); @@ -1893,6 +1896,7 @@ const struct bpf_map_ops htab_of_maps_map_ops = { .map_free = htab_of_map_free, .map_get_next_key = htab_map_get_next_key, .map_lookup_elem = htab_of_map_lookup_elem, + .map_copy_value = fd_htab_map_copy_value, .map_delete_elem = htab_map_delete_elem, .map_fd_get_ptr = bpf_map_fd_get_ptr, .map_fd_put_ptr = bpf_map_fd_put_ptr, diff --git a/kernel/bpf/local_storage.c b/kernel/bpf/local_storage.c index 33d01866bcc2..fcc0b168dad2 100644 --- a/kernel/bpf/local_storage.c +++ b/kernel/bpf/local_storage.c @@ -167,7 +167,7 @@ static int cgroup_storage_update_elem(struct bpf_map *map, void *_key, return 0; } -int bpf_percpu_cgroup_storage_copy(struct bpf_map *_map, void *_key, +static int percpu_cgroup_storage_copy(struct bpf_map *_map, void *_key, void *value) { struct bpf_cgroup_storage_map *map = map_to_storage(_map); @@ -420,6 +420,18 @@ const struct bpf_map_ops cgroup_storage_map_ops = { .map_seq_show_elem = cgroup_storage_seq_show_elem, }; +const struct bpf_map_ops percpu_cgroup_storage_map_ops = { + .map_alloc = cgroup_storage_map_alloc, + .map_free = cgroup_storage_map_free, + .map_get_next_key = cgroup_storage_get_next_key, + .map_lookup_elem = cgroup_storage_lookup_elem, + .map_copy_value = percpu_cgroup_storage_copy, + .map_update_elem = cgroup_storage_update_elem, + .map_delete_elem = cgroup_storage_delete_elem, + .map_check_btf = cgroup_storage_check_btf, + .map_seq_show_elem = cgroup_storage_seq_show_elem, +}; + int bpf_cgroup_storage_assign(struct bpf_prog_aux *aux, struct bpf_map *_map) { enum bpf_cgroup_storage_type stype = cgroup_storage_type(_map); diff --git a/kernel/bpf/reuseport_array.c b/kernel/bpf/reuseport_array.c index 01badd3eda7a..f36ccbf2612e 100644 --- a/kernel/bpf/reuseport_array.c +++ b/kernel/bpf/reuseport_array.c @@ -178,8 +178,8 @@ static struct bpf_map *reuseport_array_alloc(union bpf_attr *attr) return &array->map; } -int bpf_fd_reuseport_array_lookup_elem(struct bpf_map *map, void *key, - void *value) +static int reuseport_array_copy_value(struct bpf_map *map, void *key, + void *value) { struct sock *sk; int err; @@ -350,6 +350,7 @@ const struct bpf_map_ops reuseport_array_ops = { .map_alloc = reuseport_array_alloc, .map_free = reuseport_array_free, .map_lookup_elem = reuseport_array_lookup_elem, + .map_copy_value = reuseport_array_copy_value, .map_get_next_key = reuseport_array_get_next_key, .map_delete_elem = reuseport_array_delete_elem, }; diff --git a/kernel/bpf/syscall.c b/kernel/bpf/syscall.c index 7ce0815793dd..6503824e81e9 100644 --- a/kernel/bpf/syscall.c +++ b/kernel/bpf/syscall.c @@ -218,27 +218,11 @@ static int bpf_map_copy_value(struct bpf_map *map, void *key, void *value, return bpf_map_offload_lookup_elem(map, key, value); bpf_disable_instrumentation(); - if (map->map_type == BPF_MAP_TYPE_PERCPU_HASH || - map->map_type == BPF_MAP_TYPE_LRU_PERCPU_HASH) { - err = bpf_percpu_hash_copy(map, key, value); - } else if (map->map_type == BPF_MAP_TYPE_PERCPU_ARRAY) { - err = bpf_percpu_array_copy(map, key, value); - } else if (map->map_type == BPF_MAP_TYPE_PERCPU_CGROUP_STORAGE) { - err = bpf_percpu_cgroup_storage_copy(map, key, value); - } else if (map->map_type == BPF_MAP_TYPE_STACK_TRACE) { - err = bpf_stackmap_copy(map, key, value); - } else if (IS_FD_ARRAY(map) || IS_FD_PROG_ARRAY(map)) { - err = bpf_fd_array_map_lookup_elem(map, key, value); - } else if (IS_FD_HASH(map)) { - err = bpf_fd_htab_map_lookup_elem(map, key, value); - } else if (map->map_type == BPF_MAP_TYPE_REUSEPORT_SOCKARRAY) { - err = bpf_fd_reuseport_array_lookup_elem(map, key, value); - } else if (map->map_type == BPF_MAP_TYPE_QUEUE || - map->map_type == BPF_MAP_TYPE_STACK) { + if (map->map_type == BPF_MAP_TYPE_QUEUE || + map->map_type == BPF_MAP_TYPE_STACK) { err = map->ops->map_peek_elem(map, value); - } else if (map->map_type == BPF_MAP_TYPE_STRUCT_OPS) { - /* struct_ops map requires directly updating "value" */ - err = bpf_struct_ops_map_sys_lookup_elem(map, key, value); + } else if (map->ops->map_copy_value) { + err = map->ops->map_copy_value(map, key, value); } else { rcu_read_lock(); if (map->ops->map_lookup_elem_sys_only) From patchwork Tue Mar 10 17:47:08 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Lorenz Bauer X-Patchwork-Id: 222749 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-9.9 required=3.0 tests=DKIMWL_WL_HIGH, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY, SPF_HELO_NONE, SPF_PASS, URIBL_BLOCKED, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 29CE1C10F27 for ; Tue, 10 Mar 2020 17:47:40 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 022AC21D7E for ; Tue, 10 Mar 2020 17:47:40 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=cloudflare.com header.i=@cloudflare.com header.b="witwoPxU" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727329AbgCJRri (ORCPT ); Tue, 10 Mar 2020 13:47:38 -0400 Received: from mail-wm1-f67.google.com ([209.85.128.67]:40607 "EHLO mail-wm1-f67.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727255AbgCJRrb (ORCPT ); Tue, 10 Mar 2020 13:47:31 -0400 Received: by mail-wm1-f67.google.com with SMTP id e26so2374049wme.5 for ; Tue, 10 Mar 2020 10:47:31 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cloudflare.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=npmqyLC4H/hqzxrhW1ZFXspPQ84Dm9XWgo+tS3ZwYuE=; b=witwoPxUMnZ1q4jIFaWCCe9GWQWLXaMvo+Jd0UehTaGGw0ZT+jNrbQDFdHWAVQYfCj ggOpJkXSFnPqHQlIMZS+pKecTPIGzSktjhI3lWajEHuiEE5oRiwJwHCRLflqfXaRE8FM b6pkY6tvS7aBQXslcYSkanLted/u+n9dGm0SU= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=npmqyLC4H/hqzxrhW1ZFXspPQ84Dm9XWgo+tS3ZwYuE=; b=ho2Bdd18rILVd9unUglfPEt7pcCgE9SUfoDQM9OSbfhPzCSnC/+s5I32QZgdg8Nozg PVoBYfBC0+8+cgzbAgiJqvEL0W3hwH9Ks1lNbTfChS5W8/jA8oWGB4I0ey+tE3TXLiLf E8pUsCQalBQh4joP8gIhobbhrpR4FOPc7yTXq5H43fxErtPR3pimi8XpfIPvdU47tktP L2FWbnbOML3GnYUMwRnKx2r9OT6kkQ/bCQ5qd74tMlvKmGw0LpanRhh4stt6ss63eOx1 tOVQ1JxOdwNua2fq+foeRLOxetboOow8QyJlY1a91FyzoxubUImTE9XQ8ZzZCsMC+MMY +Liw== X-Gm-Message-State: ANhLgQ39ZAbSX0awxDonuUBq5woqxP6WM0l3s8eb+uYTUjjy9kOKbeHb w+Lw6qzgjFIed4P1UcvGC7zB4Q== X-Google-Smtp-Source: ADFU+vtai1/9M3ILySaSaKTd96UOxXEAKNdtViy5HhG7OgZRyr5BzXvebagcWd+0u1zYdlwaWP+2Bg== X-Received: by 2002:a1c:4054:: with SMTP id n81mr3074313wma.114.1583862450625; Tue, 10 Mar 2020 10:47:30 -0700 (PDT) Received: from localhost.localdomain ([2a06:98c0:1000:8250:9494:775c:e7b6:e690]) by smtp.gmail.com with ESMTPSA id k4sm9118691wrx.27.2020.03.10.10.47.28 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 10 Mar 2020 10:47:29 -0700 (PDT) From: Lorenz Bauer To: Alexei Starovoitov , Daniel Borkmann Cc: kernel-team@cloudflare.com, Lorenz Bauer , netdev@vger.kernel.org, bpf@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH 2/5] bpf: convert queue and stack map to map_copy_value Date: Tue, 10 Mar 2020 17:47:08 +0000 Message-Id: <20200310174711.7490-3-lmb@cloudflare.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20200310174711.7490-1-lmb@cloudflare.com> References: <20200310174711.7490-1-lmb@cloudflare.com> MIME-Version: 1.0 Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org Migrate BPF_MAP_TYPE_QUEUE and BPF_MAP_TYPE_STACK to map_copy_value, by introducing small wrappers that discard the (unused) key argument. Signed-off-by: Lorenz Bauer --- kernel/bpf/queue_stack_maps.c | 18 ++++++++++++++++++ kernel/bpf/syscall.c | 5 +---- 2 files changed, 19 insertions(+), 4 deletions(-) diff --git a/kernel/bpf/queue_stack_maps.c b/kernel/bpf/queue_stack_maps.c index f697647ceb54..5c89b7583cd2 100644 --- a/kernel/bpf/queue_stack_maps.c +++ b/kernel/bpf/queue_stack_maps.c @@ -262,11 +262,28 @@ static int queue_stack_map_get_next_key(struct bpf_map *map, void *key, return -EINVAL; } +/* Called from syscall */ +static int queue_map_copy_value(struct bpf_map *map, void *key, void *value) +{ + (void)key; + + return queue_map_peek_elem(map, value); +} + +/* Called from syscall */ +static int stack_map_copy_value(struct bpf_map *map, void *key, void *value) +{ + (void)key; + + return stack_map_peek_elem(map, value); +} + const struct bpf_map_ops queue_map_ops = { .map_alloc_check = queue_stack_map_alloc_check, .map_alloc = queue_stack_map_alloc, .map_free = queue_stack_map_free, .map_lookup_elem = queue_stack_map_lookup_elem, + .map_copy_value = queue_map_copy_value, .map_update_elem = queue_stack_map_update_elem, .map_delete_elem = queue_stack_map_delete_elem, .map_push_elem = queue_stack_map_push_elem, @@ -280,6 +297,7 @@ const struct bpf_map_ops stack_map_ops = { .map_alloc = queue_stack_map_alloc, .map_free = queue_stack_map_free, .map_lookup_elem = queue_stack_map_lookup_elem, + .map_copy_value = stack_map_copy_value, .map_update_elem = queue_stack_map_update_elem, .map_delete_elem = queue_stack_map_delete_elem, .map_push_elem = queue_stack_map_push_elem, diff --git a/kernel/bpf/syscall.c b/kernel/bpf/syscall.c index 6503824e81e9..20c6cdace275 100644 --- a/kernel/bpf/syscall.c +++ b/kernel/bpf/syscall.c @@ -218,10 +218,7 @@ static int bpf_map_copy_value(struct bpf_map *map, void *key, void *value, return bpf_map_offload_lookup_elem(map, key, value); bpf_disable_instrumentation(); - if (map->map_type == BPF_MAP_TYPE_QUEUE || - map->map_type == BPF_MAP_TYPE_STACK) { - err = map->ops->map_peek_elem(map, value); - } else if (map->ops->map_copy_value) { + if (map->ops->map_copy_value) { err = map->ops->map_copy_value(map, key, value); } else { rcu_read_lock(); From patchwork Tue Mar 10 17:47:10 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Lorenz Bauer X-Patchwork-Id: 222748 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-9.9 required=3.0 tests=DKIMWL_WL_HIGH, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY, SPF_HELO_NONE, SPF_PASS, URIBL_BLOCKED, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5A635C10F27 for ; Tue, 10 Mar 2020 17:47:54 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 33902222C3 for ; Tue, 10 Mar 2020 17:47:54 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=cloudflare.com header.i=@cloudflare.com header.b="zIJySxHv" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727385AbgCJRrp (ORCPT ); Tue, 10 Mar 2020 13:47:45 -0400 Received: from mail-wr1-f67.google.com ([209.85.221.67]:42726 "EHLO mail-wr1-f67.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727332AbgCJRrj (ORCPT ); Tue, 10 Mar 2020 13:47:39 -0400 Received: by mail-wr1-f67.google.com with SMTP id v11so16999160wrm.9 for ; Tue, 10 Mar 2020 10:47:37 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cloudflare.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=XC8IKQ74vPv+UfMSye3u7t/54ed4ZAgOtHUPCB0KwJI=; b=zIJySxHvPp+R2UTCXnHsTySARcEwLDoOm3+7DSZAUoGamSnvk4Z4m/33Y22UZhZvPi HRjm7o/EH8TQJqeSEacRldso2oXzc0+fB9bwfzz2X23lp9AsuVD2eG/y8ojk15LjQWKQ hd+gxzQvdId2G79HdN+YP70IurA1njyIw1/oQ= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=XC8IKQ74vPv+UfMSye3u7t/54ed4ZAgOtHUPCB0KwJI=; b=JQu0aSEBeoJFQo8ATf2z3E9QXs0eg6hfR0kMUDDMwSQZgEDNvKq1PQjLI87xv1sdTf Uzb+TLr1qcVf6SjT2WjcGmsuHBiB1bsO9xmJ43RJuJYV32pJyzJdp4+bV/h9urRQG1OY 7R7q8hUyxH9Lp2mo/5aLQPJez1vQIofHP981fhnGfzd4WWK9s6UzFAK2WUKebce7/4Og rlTnJLnnT3lFsgPMezZB/K9mRzbpn2ng/UAWNijpZiiQ30scq2gjBKqbUKCS6unTxzRb MgjR+88ns9Ln5JCV70K7ZyOB1ezgYA96b78hoJKTuA+c0Df98yle4vYTXxO3vs9AtzDl QM6g== X-Gm-Message-State: ANhLgQ3vUxOaA3aaIAPkuR1x70krVb04erkhBQLrdbkqBF/09rtXFb3d +UpIL1ho2+L3Wjtnkb7EKJaLqQ== X-Google-Smtp-Source: ADFU+vsjj2OPEOgFyNbWcbNMGg6PA4mx+5cRmNV6PwJWCtb3+fuf9br0wM59YIl8VXByekmeR/p31w== X-Received: by 2002:adf:a3c9:: with SMTP id m9mr26515647wrb.349.1583862456912; Tue, 10 Mar 2020 10:47:36 -0700 (PDT) Received: from localhost.localdomain ([2a06:98c0:1000:8250:9494:775c:e7b6:e690]) by smtp.gmail.com with ESMTPSA id k4sm9118691wrx.27.2020.03.10.10.47.33 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 10 Mar 2020 10:47:34 -0700 (PDT) From: Lorenz Bauer To: John Fastabend , Daniel Borkmann , Jakub Sitnicki , Lorenz Bauer , "David S. Miller" , Jakub Kicinski , Alexei Starovoitov Cc: kernel-team@cloudflare.com, netdev@vger.kernel.org, bpf@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH 4/5] bpf: sockmap, sockhash: return file descriptors from privileged lookup Date: Tue, 10 Mar 2020 17:47:10 +0000 Message-Id: <20200310174711.7490-5-lmb@cloudflare.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20200310174711.7490-1-lmb@cloudflare.com> References: <20200310174711.7490-1-lmb@cloudflare.com> MIME-Version: 1.0 Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org Allow callers with CAP_NET_ADMIN to retrieve file descriptors from a sockmap and sockhash. O_CLOEXEC is enforced on all fds. Without this, it's difficult to resize or otherwise rebuild existing sockmap or sockhashes. Suggested-by: Jakub Sitnicki Signed-off-by: Lorenz Bauer --- net/core/sock_map.c | 19 +++++++++++++++++++ 1 file changed, 19 insertions(+) diff --git a/net/core/sock_map.c b/net/core/sock_map.c index 03e04426cd21..3228936aa31e 100644 --- a/net/core/sock_map.c +++ b/net/core/sock_map.c @@ -347,12 +347,31 @@ static void *sock_map_lookup(struct bpf_map *map, void *key) static int __sock_map_copy_value(struct bpf_map *map, struct sock *sk, void *value) { + struct file *file; + int fd; + switch (map->value_size) { case sizeof(u64): sock_gen_cookie(sk); *(u64 *)value = atomic64_read(&sk->sk_cookie); return 0; + case sizeof(u32): + if (!capable(CAP_NET_ADMIN)) + return -EPERM; + + fd = get_unused_fd_flags(O_CLOEXEC); + if (unlikely(fd < 0)) + return fd; + + read_lock_bh(&sk->sk_callback_lock); + file = get_file(sk->sk_socket->file); + read_unlock_bh(&sk->sk_callback_lock); + + fd_install(fd, file); + *(u32 *)value = fd; + return 0; + default: return -ENOSPC; }