From patchwork Sun Aug 29 18:35:56 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dmitrii Banshchikov X-Patchwork-Id: 504227 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER, INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 20E16C4320E for ; Sun, 29 Aug 2021 18:36:40 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 08BF460E73 for ; Sun, 29 Aug 2021 18:36:40 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235875AbhH2Sh2 (ORCPT ); Sun, 29 Aug 2021 14:37:28 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:34082 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233379AbhH2ShU (ORCPT ); Sun, 29 Aug 2021 14:37:20 -0400 Received: from mail-qt1-x834.google.com (mail-qt1-x834.google.com [IPv6:2607:f8b0:4864:20::834]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 40D22C06175F for ; Sun, 29 Aug 2021 11:36:28 -0700 (PDT) Received: by mail-qt1-x834.google.com with SMTP id r21so9911279qtw.11 for ; Sun, 29 Aug 2021 11:36:28 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ubique-spb-ru.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=JbWKXTLJequ219BGEbUG/AHdIy4kKULzRRS7YJvdafI=; b=Vj55g6BfGD1dbKS21hoVvax++eLV0t1higAS3mMr3h2TmcREHKcnmq5Iwt9FdDdBiY sFAemUSGTahJT0q9Ei61b3kBTNDGH+SPtO6ir13vU9CJeiLSMtXIqYFgS32sHYaLqXnG M0b7Kb1xQmZrHYneHbHMMnwgaJZkcdJY8IgroZEDHblELZA8W+5ekTOBhW2xAi7PEArT jNqFT/1EL7RjKSrieVzw7Hv6GbZBzt1eirOBwo7lAjLZ8wCYwlZwtpPeZ95wMxWfYqvK xwsDdKR94uIIXcc/nZUGSDrakFhNbRR5CExSfwqB/b9KqKP24/tCesH3apn4NJ+VAguk G84w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=JbWKXTLJequ219BGEbUG/AHdIy4kKULzRRS7YJvdafI=; b=Abc0nQCxUHJUg6bh2oP6/j+B9uHK2n+FI8QeGePu/FwUzV87YBpjShSH2dG1b2WcQ1 DmRjtR83kbM6K/kzpqvhjIkaBubAE2pTwufWbLJ1BDF9IW+c0KEAiKVElWDIW9yBQEaR ISGSQRl74eUpfxCXc+vnAbd0RcaA/1v+iszEOK4xxddysUl+FXEE6+KP2XVJB3DQZO4Y Hpb8yFKB9cE3Ch7HvWqi4UQoGQ9wYVwcvM3mPtGQD8dcn/EBgH7Qv+YRP96AemS10ikp RiEZAJs5Q6cfzs9TRFFuDOrfqLAJxRWCrXbzPYqS/hjn6w+GVyKqSrJBwA9sJDAyAvDw yUbg== X-Gm-Message-State: AOAM5328qgxo4WdvOlLErDcyulikVxpCa5rtZMrovVSo6pPdUQv2O0Nu wyitEs2FDGBR5RLyVsF6O0Fu1krOpsa+F75CZZ0= X-Google-Smtp-Source: ABdhPJyq6yQ5lKOw00DPbhbNUkFheTC5HdGS9+DUg5Zth2pcttCgyjyUxKZ7GsOpn8fb+FTOPddoOw== X-Received: by 2002:aed:3147:: with SMTP id 65mr17801912qtg.233.1630262187433; Sun, 29 Aug 2021 11:36:27 -0700 (PDT) Received: from localhost ([154.21.15.43]) by smtp.gmail.com with ESMTPSA id 90sm258372qte.89.2021.08.29.11.36.26 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 29 Aug 2021 11:36:27 -0700 (PDT) From: Dmitrii Banshchikov To: bpf@vger.kernel.org Cc: Dmitrii Banshchikov , ast@kernel.org, davem@davemloft.net, daniel@iogearbox.net, andrii@kernel.org, kafai@fb.com, songliubraving@fb.com, yhs@fb.com, john.fastabend@gmail.com, kpsingh@kernel.org, netdev@vger.kernel.org, rdna@fb.com Subject: [PATCH bpf-next v2 01/13] bpfilter: Add types for usermode helper Date: Sun, 29 Aug 2021 22:35:56 +0400 Message-Id: <20210829183608.2297877-2-me@ubique.spb.ru> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210829183608.2297877-1-me@ubique.spb.ru> References: <20210829183608.2297877-1-me@ubique.spb.ru> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org Add more definitions that mirror existing iptables' ABI. These definitions will be used in bpfilter usermode helper. Signed-off-by: Dmitrii Banshchikov --- include/uapi/linux/bpfilter.h | 154 ++++++++++++++++++++++++++++++++++ 1 file changed, 154 insertions(+) diff --git a/include/uapi/linux/bpfilter.h b/include/uapi/linux/bpfilter.h index cbc1f5813f50..7cd34b1702eb 100644 --- a/include/uapi/linux/bpfilter.h +++ b/include/uapi/linux/bpfilter.h @@ -3,6 +3,10 @@ #define _UAPI_LINUX_BPFILTER_H #include +#include + +#define BPFILTER_STANDARD_TARGET "" +#define BPFILTER_ERROR_TARGET "ERROR" enum { BPFILTER_IPT_SO_SET_REPLACE = 64, @@ -18,4 +22,154 @@ enum { BPFILTER_IPT_GET_MAX, }; +enum { + BPFILTER_XT_TABLE_MAXNAMELEN = 32, + BPFILTER_FUNCTION_MAXNAMELEN = 30, + BPFILTER_EXTENSION_MAXNAMELEN = 29, +}; + +enum { + BPFILTER_NF_DROP = 0, + BPFILTER_NF_ACCEPT = 1, + BPFILTER_NF_STOLEN = 2, + BPFILTER_NF_QUEUE = 3, + BPFILTER_NF_REPEAT = 4, + BPFILTER_NF_STOP = 5, + BPFILTER_NF_MAX_VERDICT = BPFILTER_NF_STOP, + BPFILTER_RETURN = (-BPFILTER_NF_REPEAT - 1), +}; + +enum { + BPFILTER_INET_HOOK_PRE_ROUTING = 0, + BPFILTER_INET_HOOK_LOCAL_IN = 1, + BPFILTER_INET_HOOK_FORWARD = 2, + BPFILTER_INET_HOOK_LOCAL_OUT = 3, + BPFILTER_INET_HOOK_POST_ROUTING = 4, + BPFILTER_INET_HOOK_MAX, +}; + +enum { + BPFILTER_IPT_F_MASK = 0x03, + BPFILTER_IPT_INV_MASK = 0x7f +}; + +struct bpfilter_ipt_match { + union { + struct { + __u16 match_size; + char name[BPFILTER_EXTENSION_MAXNAMELEN]; + __u8 revision; + } user; + struct { + __u16 match_size; + void *match; + } kernel; + __u16 match_size; + } u; + unsigned char data[0]; +}; + +struct bpfilter_ipt_target { + union { + struct { + __u16 target_size; + char name[BPFILTER_EXTENSION_MAXNAMELEN]; + __u8 revision; + } user; + struct { + __u16 target_size; + void *target; + } kernel; + __u16 target_size; + } u; + unsigned char data[0]; +}; + +struct bpfilter_ipt_standard_target { + struct bpfilter_ipt_target target; + int verdict; +}; + +struct bpfilter_ipt_error_target { + struct bpfilter_ipt_target target; + char error_name[BPFILTER_FUNCTION_MAXNAMELEN]; +}; + +struct bpfilter_ipt_get_info { + char name[BPFILTER_XT_TABLE_MAXNAMELEN]; + __u32 valid_hooks; + __u32 hook_entry[BPFILTER_INET_HOOK_MAX]; + __u32 underflow[BPFILTER_INET_HOOK_MAX]; + __u32 num_entries; + __u32 size; +}; + +struct bpfilter_ipt_counters { + __u64 packet_cnt; + __u64 byte_cnt; +}; + +struct bpfilter_ipt_counters_info { + char name[BPFILTER_XT_TABLE_MAXNAMELEN]; + __u32 num_counters; + struct bpfilter_ipt_counters counters[0]; +}; + +struct bpfilter_ipt_get_revision { + char name[BPFILTER_EXTENSION_MAXNAMELEN]; + __u8 revision; +}; + +struct bpfilter_ipt_ip { + __u32 src; + __u32 dst; + __u32 src_mask; + __u32 dst_mask; + char in_iface[IFNAMSIZ]; + char out_iface[IFNAMSIZ]; + __u8 in_iface_mask[IFNAMSIZ]; + __u8 out_iface_mask[IFNAMSIZ]; + __u16 protocol; + __u8 flags; + __u8 invflags; +}; + +struct bpfilter_ipt_entry { + struct bpfilter_ipt_ip ip; + __u32 bfcache; + __u16 target_offset; + __u16 next_offset; + __u32 comefrom; + struct bpfilter_ipt_counters counters; + __u8 elems[0]; +}; + +struct bpfilter_ipt_standard_entry { + struct bpfilter_ipt_entry entry; + struct bpfilter_ipt_standard_target target; +}; + +struct bpfilter_ipt_error_entry { + struct bpfilter_ipt_entry entry; + struct bpfilter_ipt_error_target target; +}; + +struct bpfilter_ipt_get_entries { + char name[BPFILTER_XT_TABLE_MAXNAMELEN]; + __u32 size; + struct bpfilter_ipt_entry entries[0]; +}; + +struct bpfilter_ipt_replace { + char name[BPFILTER_XT_TABLE_MAXNAMELEN]; + __u32 valid_hooks; + __u32 num_entries; + __u32 size; + __u32 hook_entry[BPFILTER_INET_HOOK_MAX]; + __u32 underflow[BPFILTER_INET_HOOK_MAX]; + __u32 num_counters; + struct bpfilter_ipt_counters *cntrs; + struct bpfilter_ipt_entry entries[0]; +}; + #endif /* _UAPI_LINUX_BPFILTER_H */ From patchwork Sun Aug 29 18:35:59 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dmitrii Banshchikov X-Patchwork-Id: 504226 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER, INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0A8E3C43214 for ; Sun, 29 Aug 2021 18:36:44 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id E406A60E73 for ; Sun, 29 Aug 2021 18:36:43 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235924AbhH2She (ORCPT ); Sun, 29 Aug 2021 14:37:34 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:34122 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235844AbhH2Sh1 (ORCPT ); Sun, 29 Aug 2021 14:37:27 -0400 Received: from mail-ej1-x633.google.com (mail-ej1-x633.google.com [IPv6:2a00:1450:4864:20::633]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 88EE3C06175F for ; Sun, 29 Aug 2021 11:36:34 -0700 (PDT) Received: by mail-ej1-x633.google.com with SMTP id t19so26184286ejr.8 for ; Sun, 29 Aug 2021 11:36:34 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ubique-spb-ru.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=yjMwzlzJCsJ2K95uZCEDXsotI3JR8EQAiCrEaLA40kc=; b=vP1uOWqYdK6vBWvlt/QAyX/lNgefEeCuV9l0mIfrScvL51v6Rg8YgGUXIx6SWDcIEL X1/QXBZK26dRj9mCfGJ1O9fqI1++8y7BnA4R4tGpcrfrLp/mznJjJUoQoc+Pr0jQhcGQ ppOmISsWcs3nhQUoXKuaL9JgyII6d+PC1e16Hm5Pvg/bCv1n7WYmmCCZk/SK10k+1Om6 nT1HSRCGfXcjqMr8+n0cl/lademJvI6RO3AjSWUM6mylV5ztP6/aCclMqp9TwR0fAczZ fOJ6aRjJkzDETVV2FGBL/DkpaG9yoD8fSGoWXE5kUPfGDGKBBNEAvTJ8z7Bk5HS4Gag7 XGYw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=yjMwzlzJCsJ2K95uZCEDXsotI3JR8EQAiCrEaLA40kc=; b=fk3QggDQkVwtiXmOVjiaBY06HUzbXZHh9D+7xKEDqAfB3fX8X8bh9GtFgVZu7y9VJf bJ2dbFgbpHyzwsBBsOkZid8ZaZKDsYiAVi+woeiK0lKhWna59iRdByYfDh+MZVxPyOMA mpIndoIomcEFBAQCiN0ytafecG7VSnFpGzE8U3sCtlm3/5BpSPvMSQndi655uJ3T76Pa JvPgkRe61IW4Woy5GOJdLGiug05jGCDg+ccix0oV4h3NtS3710IVs9k69Jr2i+dv4DpE YXKWzopZH3lQFN43yTqAS/1UyngyFJp1IUKQ7TU83G/tNu4iYikJNt0+m1tEBXb3pd7/ uZuw== X-Gm-Message-State: AOAM533wYF93wMtEdf8zsfRKuSVjs+nnIstTPZM70pPyH8rkEJHrSfFu ZQcs93v+rQ65HA3MFtnBiOdMLQ== X-Google-Smtp-Source: ABdhPJwPtLUG1xuQn5zeWY2m07B2NnXNA6nY1TnNsXYMCrIL25NH6tHMnRasLCl6mg0878Eh8xUHBQ== X-Received: by 2002:a17:906:d183:: with SMTP id c3mr21211752ejz.283.1630262193127; Sun, 29 Aug 2021 11:36:33 -0700 (PDT) Received: from localhost ([154.21.15.43]) by smtp.gmail.com with ESMTPSA id lu4sm5440475ejb.103.2021.08.29.11.36.32 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 29 Aug 2021 11:36:32 -0700 (PDT) From: Dmitrii Banshchikov To: bpf@vger.kernel.org Cc: Dmitrii Banshchikov , ast@kernel.org, davem@davemloft.net, daniel@iogearbox.net, andrii@kernel.org, kafai@fb.com, songliubraving@fb.com, yhs@fb.com, john.fastabend@gmail.com, kpsingh@kernel.org, netdev@vger.kernel.org, rdna@fb.com Subject: [PATCH bpf-next v2 04/13] bpfilter: Add map container Date: Sun, 29 Aug 2021 22:35:59 +0400 Message-Id: <20210829183608.2297877-5-me@ubique.spb.ru> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210829183608.2297877-1-me@ubique.spb.ru> References: <20210829183608.2297877-1-me@ubique.spb.ru> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org Introduce common code for an associative container. This common code will be used for maps of matches, targets and tables. Hash search tables from libc are used as an index. The supported set of operations is: find and upsert. Signed-off-by: Dmitrii Banshchikov --- net/bpfilter/Makefile | 2 +- net/bpfilter/map-common.c | 50 +++++++++++++++ net/bpfilter/map-common.h | 18 ++++++ .../testing/selftests/bpf/bpfilter/.gitignore | 2 + tools/testing/selftests/bpf/bpfilter/Makefile | 19 ++++++ .../testing/selftests/bpf/bpfilter/test_map.c | 63 +++++++++++++++++++ 6 files changed, 153 insertions(+), 1 deletion(-) create mode 100644 net/bpfilter/map-common.c create mode 100644 net/bpfilter/map-common.h create mode 100644 tools/testing/selftests/bpf/bpfilter/.gitignore create mode 100644 tools/testing/selftests/bpf/bpfilter/Makefile create mode 100644 tools/testing/selftests/bpf/bpfilter/test_map.c diff --git a/net/bpfilter/Makefile b/net/bpfilter/Makefile index cdac82b8c53a..1809759d08c4 100644 --- a/net/bpfilter/Makefile +++ b/net/bpfilter/Makefile @@ -4,7 +4,7 @@ # userprogs := bpfilter_umh -bpfilter_umh-objs := main.o +bpfilter_umh-objs := main.o map-common.o userccflags += -I $(srctree)/tools/include/ -I $(srctree)/tools/include/uapi ifeq ($(CONFIG_BPFILTER_UMH), y) diff --git a/net/bpfilter/map-common.c b/net/bpfilter/map-common.c new file mode 100644 index 000000000000..f933929a3909 --- /dev/null +++ b/net/bpfilter/map-common.c @@ -0,0 +1,50 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * Copyright (c) 2021 Telegram FZ-LLC + */ + +#include "map-common.h" + +#include + +#include +#include + +int create_map(struct hsearch_data *htab, size_t nelem) +{ + memset(htab, 0, sizeof(*htab)); + if (!hcreate_r(nelem, htab)) + return -errno; + + return 0; +} + +void *map_find(struct hsearch_data *htab, const char *key) +{ + const ENTRY needle = { .key = (char *)key }; + ENTRY *found; + + if (!hsearch_r(needle, FIND, &found, htab)) + return ERR_PTR(-ENOENT); + + return found->data; +} + +int map_upsert(struct hsearch_data *htab, const char *key, void *value) +{ + const ENTRY needle = { .key = (char *)key, .data = value }; + ENTRY *found; + + if (!hsearch_r(needle, ENTER, &found, htab)) + return -errno; + + found->key = (char *)key; + found->data = value; + + return 0; +} + +void free_map(struct hsearch_data *htab) +{ + hdestroy_r(htab); +} diff --git a/net/bpfilter/map-common.h b/net/bpfilter/map-common.h new file mode 100644 index 000000000000..236ba906828e --- /dev/null +++ b/net/bpfilter/map-common.h @@ -0,0 +1,18 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* + * Copyright (c) 2021 Telegram FZ-LLC + */ + +#ifndef NET_BPFILTER_MAP_COMMON_H +#define NET_BPFILTER_MAP_COMMON_H + +#define _GNU_SOURCE + +#include + +int create_map(struct hsearch_data *htab, size_t nelem); +void *map_find(struct hsearch_data *htab, const char *key); +int map_upsert(struct hsearch_data *htab, const char *key, void *value); +void free_map(struct hsearch_data *htab); + +#endif // NET_BPFILTER_MAP_COMMON_H diff --git a/tools/testing/selftests/bpf/bpfilter/.gitignore b/tools/testing/selftests/bpf/bpfilter/.gitignore new file mode 100644 index 000000000000..983fd06cbefa --- /dev/null +++ b/tools/testing/selftests/bpf/bpfilter/.gitignore @@ -0,0 +1,2 @@ +# SPDX-License-Identifier: GPL-2.0-only +test_map diff --git a/tools/testing/selftests/bpf/bpfilter/Makefile b/tools/testing/selftests/bpf/bpfilter/Makefile new file mode 100644 index 000000000000..c262aad8c2a4 --- /dev/null +++ b/tools/testing/selftests/bpf/bpfilter/Makefile @@ -0,0 +1,19 @@ +# SPDX-License-Identifier: GPL-2.0 + +top_srcdir = ../../../../.. +TOOLSDIR := $(abspath ../../../../) +TOOLSINCDIR := $(TOOLSDIR)/include +APIDIR := $(TOOLSINCDIR)/uapi +BPFILTERSRCDIR := $(top_srcdir)/net/bpfilter + +CFLAGS += -Wall -g -pthread -I$(TOOLSINCDIR) -I$(APIDIR) -I$(BPFILTERSRCDIR) + +TEST_GEN_PROGS += test_map + +KSFT_KHDR_INSTALL := 1 + +include ../../lib.mk + +BPFILTER_MAP_SRCS := $(BPFILTERSRCDIR)/map-common.c + +$(OUTPUT)/test_map: test_map.c $(BPFILTER_MAP_SRCS) diff --git a/tools/testing/selftests/bpf/bpfilter/test_map.c b/tools/testing/selftests/bpf/bpfilter/test_map.c new file mode 100644 index 000000000000..7ed737b78816 --- /dev/null +++ b/tools/testing/selftests/bpf/bpfilter/test_map.c @@ -0,0 +1,63 @@ +// SPDX-License-Identifier: GPL-2.0 + +#include "map-common.h" + +#include + +#include "../../kselftest_harness.h" + +FIXTURE(test_map) +{ + struct hsearch_data map; + const char *key; + void *expected; + void *actual; +}; + +FIXTURE_SETUP(test_map) +{ + const int max_nelements = 100; + + create_map(&self->map, max_nelements); + self->key = "key"; + self->expected = "expected"; + self->actual = "actual"; +} + +FIXTURE_TEARDOWN(test_map) +{ + free_map(&self->map); +} + +TEST_F(test_map, upsert_and_find) +{ + void *found; + + found = map_find(&self->map, self->key); + ASSERT_TRUE(IS_ERR(found)) + ASSERT_EQ(-ENOENT, PTR_ERR(found)) + + ASSERT_EQ(0, map_upsert(&self->map, self->key, self->expected)); + ASSERT_EQ(0, map_upsert(&self->map, self->key, self->expected)); + ASSERT_EQ(0, map_upsert(&self->map, self->key, self->actual)); + + found = map_find(&self->map, self->key); + + ASSERT_FALSE(IS_ERR(found)); + ASSERT_STREQ(self->actual, found); +} + +TEST_F(test_map, update) +{ + void *found; + + ASSERT_EQ(0, map_upsert(&self->map, self->key, self->actual)); + ASSERT_EQ(0, map_upsert(&self->map, self->key, self->expected)); + + found = map_find(&self->map, self->key); + + ASSERT_FALSE(IS_ERR(found)); + ASSERT_STREQ(self->expected, found); +} + +TEST_HARNESS_MAIN From patchwork Sun Aug 29 18:36:00 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dmitrii Banshchikov X-Patchwork-Id: 504225 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER, INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 89EE0C432BE for ; Sun, 29 Aug 2021 18:36:45 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 6DFF260E73 for ; Sun, 29 Aug 2021 18:36:45 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235846AbhH2Shg (ORCPT ); Sun, 29 Aug 2021 14:37:36 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:34132 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235882AbhH2Sh3 (ORCPT ); Sun, 29 Aug 2021 14:37:29 -0400 Received: from mail-ej1-x630.google.com (mail-ej1-x630.google.com [IPv6:2a00:1450:4864:20::630]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 80BA8C061575 for ; Sun, 29 Aug 2021 11:36:36 -0700 (PDT) Received: by mail-ej1-x630.google.com with SMTP id u3so26108528ejz.1 for ; Sun, 29 Aug 2021 11:36:36 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ubique-spb-ru.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=1GGWQS2F8TIkjkfRjAwlg/olnvQAdF9KhtCTM+aLY1E=; b=mY9VHMDkf10DOfWWJ9LPFz58W+fOhT0WiNNGEz0oY4yy+QF9rVODBM3wOPrAZUkL/a VfOpjHe4oNSDj/2Dqyuk2SxYIDyZEz8QqzKaGuoj7IIfxJxxN/4hgViFf/V1OFZhgzdZ wnsSZ3Kjlej/9V/FUCzlKd31Tedkl6og7j7QMBtQC8ieNqYBr19rKy+nKMpLZmLst+c4 dA+1Feor3dzyReBq1IVuFjsZTzTwgBtsEQUQrJAoFGfRLa3YslDmTck/PhLKtOswkzlE ycvJxl72ojmhJlIKLrMkczo0l6AEt82gLa/LgmvC83xg9muK61kX80qZRH4QuTZOHsdg jYhg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=1GGWQS2F8TIkjkfRjAwlg/olnvQAdF9KhtCTM+aLY1E=; b=hZPMn/vY2FTviQfFZHNbbtuqlzulNqgtsbBjax3bYBLbdLzSXBGWi3p0qCo5mY1ebS oBYyl3eyvmmyahM8Rqo5plMd/lyF3pX7qH6CeIoMwc8cr8p6noEWfHmxrsocnGhZmh9M qim7NTQ+8SrWL6hQ3avbAmcmsff2Efa8PtlOi0QSYh4r32KorzHhFfQyhKzg061qNaNO 9RP9UwSo2lDiUmVB5PRNTYKWcU4Gx7EpHZLWnc2jRJ2ktgRBwQC5F6qkQtHWQS+chqN7 4WKb9CPzuXyvwZi3SWxBbDIqUU2v/EqMnse5w6Tu7q842s0hU6PHUg6jOKCVuxfKVIcO r17Q== X-Gm-Message-State: AOAM532OmXZq1CmE/KD6r6lOIBQtJ9sgblKEuEqrMJOBAA/vJJ+YD5mn Ft0mTdTPKrOTjR8blgrWHCCa4w== X-Google-Smtp-Source: ABdhPJztWwzHfOlq6NTBL4sR80p+q/yo0f34c2gZbNUrjyvbkWbad3VsG/8KPpe90Rot8ntsglMrAw== X-Received: by 2002:a17:906:7256:: with SMTP id n22mr21093771ejk.173.1630262194958; Sun, 29 Aug 2021 11:36:34 -0700 (PDT) Received: from localhost ([154.21.15.43]) by smtp.gmail.com with ESMTPSA id cr9sm6427992edb.17.2021.08.29.11.36.34 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 29 Aug 2021 11:36:34 -0700 (PDT) From: Dmitrii Banshchikov To: bpf@vger.kernel.org Cc: Dmitrii Banshchikov , ast@kernel.org, davem@davemloft.net, daniel@iogearbox.net, andrii@kernel.org, kafai@fb.com, songliubraving@fb.com, yhs@fb.com, john.fastabend@gmail.com, kpsingh@kernel.org, netdev@vger.kernel.org, rdna@fb.com Subject: [PATCH bpf-next v2 05/13] bpfilter: Add codegen infrastructure Date: Sun, 29 Aug 2021 22:36:00 +0400 Message-Id: <20210829183608.2297877-6-me@ubique.spb.ru> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210829183608.2297877-1-me@ubique.spb.ru> References: <20210829183608.2297877-1-me@ubique.spb.ru> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org Prepare codegen infrastructure to be used by matches, targets, rules and tables. The resulting BPF program is stored as an array of bpf_insn structs. There are multiple flavours of BPF programs needed for bpfilter: TC and XDP. While most of the logic is same for the both flavours there are multiple points where the logic is slightly different. To support such points there is a codegen_ops struct which provides a polymorphic interface to emit a specific set of instructions for each individual flavour. An emitted instruction may need to be fixed up later - as some data isn't known at the moment of emitting. To support such cases there are fixups and relocations. The difference between them is point of time when data becomes known. For fixups such time point is end of code generation and for relocations it is a time just before loading the program. Fixups and relocations are performed during code generation/preparation for program loading when required data becomes known and instructions might be adjusted. Subprogs are required to support user defined chains and helper subprograms. All already generated subprogs are stored in subprogs array. This sorted array acts as an index. All subprogs awaiting generation phase are stored in awaiting_subprogs lists. To support a shared state between multiple BPF programs there is shared_codegen struct. Currently it may be used to have a single counters map both for TC and XDP BPF programs. Beside that there is a runtime_context struct that might be used to store frequently required data such as size of the packet and pointer to L3/L4 headers. This context is stored on the stack and there are macros to access individual fields of this struct. Immediately after runtime_context on stack there is a scratchpad area. The calling convention follows the BPF calling convention with a couple of additions: * CODEGEN_REG_CTX(BPF_REG_9) is a pointer to program context * CODEGEN_REG_RUNTIME_CTX(BPF_REG_8) is a pointer to runtime context Signed-off-by: Dmitrii Banshchikov --- net/bpfilter/Makefile | 14 +- net/bpfilter/codegen.c | 732 ++++++++++++++++++ net/bpfilter/codegen.h | 188 +++++ .../testing/selftests/bpf/bpfilter/.gitignore | 1 + tools/testing/selftests/bpf/bpfilter/Makefile | 22 +- 5 files changed, 954 insertions(+), 3 deletions(-) create mode 100644 net/bpfilter/codegen.c create mode 100644 net/bpfilter/codegen.h diff --git a/net/bpfilter/Makefile b/net/bpfilter/Makefile index 1809759d08c4..f3838368ba08 100644 --- a/net/bpfilter/Makefile +++ b/net/bpfilter/Makefile @@ -3,9 +3,19 @@ # Makefile for the Linux BPFILTER layer. # +LIBBPF_SRCS = $(srctree)/tools/lib/bpf/ +LIBBPF_A = $(obj)/libbpf.a +LIBBPF_OUT = $(abspath $(obj)) + +$(LIBBPF_A): + $(Q)$(MAKE) -C $(LIBBPF_SRCS) O=$(LIBBPF_OUT)/ OUTPUT=$(LIBBPF_OUT)/ $(LIBBPF_OUT)/libbpf.a + userprogs := bpfilter_umh -bpfilter_umh-objs := main.o map-common.o -userccflags += -I $(srctree)/tools/include/ -I $(srctree)/tools/include/uapi +bpfilter_umh-objs := main.o map-common.o codegen.o +bpfilter_umh-userldlibs := $(LIBBPF_A) -lelf -lz +userccflags += -I $(srctree)/tools/include/ -I $(srctree)/tools/include/uapi -I $(srctree)/tools/lib + +$(obj)/bpfilter_umh: $(LIBBPF_A) ifeq ($(CONFIG_BPFILTER_UMH), y) # builtin bpfilter_umh should be linked with -static diff --git a/net/bpfilter/codegen.c b/net/bpfilter/codegen.c new file mode 100644 index 000000000000..0fa84d03af63 --- /dev/null +++ b/net/bpfilter/codegen.c @@ -0,0 +1,732 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * Copyright (c) 2021 Telegram FZ-LLC + */ + +#define _GNU_SOURCE + +#include "codegen.h" + +#include "../../include/uapi/linux/bpfilter.h" + +#include +#include +#include +#include + +#include +#include + +#include +#include +#include + +#include + +#include "context.h" + +enum fixup_insn_type { FIXUP_INSN_OFF, FIXUP_INSN_IMM, __MAX_FIXUP_INSN_TYPE }; + +static int sys_bpf(int cmd, union bpf_attr *attr, unsigned int size) +{ + return syscall(SYS_bpf, cmd, attr, size); +} + +static __u64 bpf_ptr_to_u64(const void *ptr) +{ + return (__u64)(unsigned long)ptr; +} + +static int subprog_desc_comparator(const void *x, const void *y) +{ + const struct codegen_subprog_desc *subprog_x = *(const struct codegen_subprog_desc **)x; + const struct codegen_subprog_desc *subprog_y = *(const struct codegen_subprog_desc **)y; + + if (subprog_x->type != subprog_y->type) + return subprog_x->type - subprog_y->type; + + if (subprog_x->type == CODEGEN_SUBPROG_USER_CHAIN) + return subprog_x->offset - subprog_y->offset; + + BUG_ON(1); + + return -1; +} + +static const struct codegen_subprog_desc * +codegen_find_subprog(struct codegen *codegen, const struct codegen_subprog_desc **subprog) +{ + const struct codegen_subprog_desc **found; + + found = bsearch(subprog, codegen->subprogs, codegen->subprogs_cur, + sizeof(codegen->subprogs[0]), subprog_desc_comparator); + + return found ? *found : NULL; +} + +static const struct codegen_subprog_desc *codegen_find_user_chain_subprog(struct codegen *codegen, + uint32_t offset) +{ + const struct codegen_subprog_desc subprog = { .type = CODEGEN_SUBPROG_USER_CHAIN, + .offset = offset }; + const struct codegen_subprog_desc *subprog_ptr = &subprog; + + return codegen_find_subprog(codegen, &subprog_ptr); +} + +int codegen_push_awaiting_subprog(struct codegen *codegen, struct codegen_subprog_desc *subprog) +{ + struct list_head *t, *n; + + if (codegen_find_subprog(codegen, (const struct codegen_subprog_desc **)&subprog)) { + free(subprog); + return 0; + } + + list_for_each_safe(t, n, &codegen->awaiting_subprogs) { + struct codegen_subprog_desc *awaiting_subprog; + + awaiting_subprog = list_entry(t, struct codegen_subprog_desc, list); + if (!subprog_desc_comparator(&awaiting_subprog, &subprog)) { + free(subprog); + return 0; + } + } + + list_add_tail(&subprog->list, &codegen->awaiting_subprogs); + + return 0; +} + +static int codegen_fixup_insn(struct bpf_insn *insn, enum fixup_insn_type type, __s32 v) +{ + if (type == FIXUP_INSN_OFF) { + if (insn->off) + return -EINVAL; + + insn->off = v; + + return 0; + } + + if (type == FIXUP_INSN_IMM) { + if (insn->imm) + return -EINVAL; + + insn->imm = v; + + return 0; + } + + return -EINVAL; +} + +int codegen_fixup(struct codegen *codegen, enum codegen_fixup_type fixup_type) +{ + struct list_head *t, *n; + + list_for_each_safe(t, n, &codegen->fixup) { + enum fixup_insn_type type = __MAX_FIXUP_INSN_TYPE; + struct codegen_fixup_desc *fixup; + struct bpf_insn *insn; + int err; + __s32 v; + + fixup = list_entry(t, struct codegen_fixup_desc, list); + if (fixup->type != fixup_type) + continue; + + if (fixup->type >= __MAX_CODEGEN_FIXUP_TYPE) + return -EINVAL; + + if (fixup->insn > codegen->len_cur) + return -EINVAL; + + insn = &codegen->img[fixup->insn]; + + if (fixup_type == CODEGEN_FIXUP_NEXT_RULE || + fixup_type == CODEGEN_FIXUP_END_OF_CHAIN) { + type = FIXUP_INSN_OFF; + v = codegen->len_cur - fixup->insn - 1; + } + + if (fixup_type == CODEGEN_FIXUP_JUMP_TO_CHAIN) { + const struct codegen_subprog_desc *subprog; + + subprog = codegen_find_user_chain_subprog(codegen, fixup->offset); + if (!subprog) + return -EINVAL; + + type = FIXUP_INSN_OFF; + v = subprog->insn - fixup->insn - 1; + } + + if (fixup_type == CODEGEN_FIXUP_COUNTERS_INDEX) { + type = FIXUP_INSN_IMM; + v = codegen->rule_index; + } + + err = codegen_fixup_insn(insn, type, v); + if (err) + return err; + + list_del(t); + free(fixup); + } + + return 0; +} + +int emit_fixup(struct codegen *codegen, enum codegen_fixup_type fixup_type, struct bpf_insn insn) +{ + struct codegen_fixup_desc *fixup; + + fixup = malloc(sizeof(*fixup)); + if (!fixup) + return -ENOMEM; + + INIT_LIST_HEAD(&fixup->list); + fixup->type = fixup_type; + fixup->insn = codegen->len_cur; + list_add_tail(&fixup->list, &codegen->fixup); + + EMIT(codegen, insn); + + return 0; +} + +int emit_add_counter(struct codegen *codegen) +{ + struct bpf_insn insns[2] = { BPF_LD_MAP_FD(BPF_REG_ARG1, 0) }; + struct codegen_reloc_desc *reloc; + + reloc = malloc(sizeof(*reloc)); + if (!reloc) + return -ENOMEM; + EMIT(codegen, BPF_LDX_MEM(BPF_W, CODEGEN_REG_SCRATCH3, CODEGEN_REG_RUNTIME_CTX, + STACK_RUNTIME_CONTEXT_OFFSET(data_size))); + + INIT_LIST_HEAD(&reloc->list); + reloc->type = CODEGEN_RELOC_MAP; + reloc->map = CODEGEN_MAP_COUNTERS; + reloc->insn = codegen->len_cur; + list_add_tail(&reloc->list, &codegen->relocs); + + EMIT(codegen, insns[0]); + EMIT(codegen, insns[1]); + EMIT_FIXUP(codegen, CODEGEN_FIXUP_COUNTERS_INDEX, + BPF_ST_MEM(BPF_W, BPF_REG_10, STACK_SCRATCHPAD_OFFSET - 4, 0)); + EMIT(codegen, BPF_MOV64_REG(BPF_REG_ARG2, BPF_REG_10)); + EMIT(codegen, BPF_ALU64_IMM(BPF_ADD, BPF_REG_ARG2, STACK_SCRATCHPAD_OFFSET - 4)); + EMIT(codegen, BPF_EMIT_CALL(BPF_FUNC_map_lookup_elem)); + EMIT(codegen, BPF_JMP_IMM(BPF_JEQ, BPF_REG_0, 0, 14)); + + reloc = malloc(sizeof(*reloc)); + if (!reloc) + return -ENOMEM; + INIT_LIST_HEAD(&reloc->list); + reloc->type = CODEGEN_RELOC_MAP; + reloc->map = CODEGEN_MAP_COUNTERS; + reloc->insn = codegen->len_cur; + list_add_tail(&reloc->list, &codegen->relocs); + + EMIT(codegen, insns[0]); + EMIT(codegen, insns[1]); + EMIT(codegen, BPF_LDX_MEM(BPF_DW, CODEGEN_REG_SCRATCH5, BPF_REG_0, 0)); + EMIT(codegen, BPF_LDX_MEM(BPF_DW, CODEGEN_REG_SCRATCH4, BPF_REG_0, 8)); + EMIT(codegen, BPF_LDX_MEM(BPF_W, CODEGEN_REG_SCRATCH3, CODEGEN_REG_RUNTIME_CTX, + STACK_RUNTIME_CONTEXT_OFFSET(data_size))); + EMIT(codegen, BPF_ALU64_IMM(BPF_ADD, CODEGEN_REG_SCRATCH5, 1)); + EMIT(codegen, BPF_ALU64_REG(BPF_ADD, CODEGEN_REG_SCRATCH4, CODEGEN_REG_SCRATCH3)); + EMIT(codegen, BPF_STX_MEM(BPF_DW, BPF_REG_0, CODEGEN_REG_SCRATCH5, 0)); + EMIT(codegen, BPF_STX_MEM(BPF_DW, BPF_REG_0, CODEGEN_REG_SCRATCH4, 8)); + EMIT(codegen, BPF_MOV64_REG(BPF_REG_ARG2, BPF_REG_10)); + EMIT(codegen, BPF_ALU64_IMM(BPF_ADD, BPF_REG_ARG2, STACK_SCRATCHPAD_OFFSET - 4)); + EMIT(codegen, BPF_MOV64_REG(BPF_REG_ARG3, BPF_REG_0)); + EMIT(codegen, BPF_MOV32_IMM(BPF_REG_ARG4, BPF_EXIST)); + EMIT(codegen, BPF_EMIT_CALL(BPF_FUNC_map_update_elem)); + + return 0; +} + +static int codegen_reloc(struct codegen *codegen) +{ + struct shared_codegen *shared_codegen; + struct list_head *t; + + shared_codegen = codegen->shared_codegen; + + list_for_each(t, &codegen->relocs) { + struct codegen_reloc_desc *reloc; + struct bpf_insn *insn; + + reloc = list_entry(t, struct codegen_reloc_desc, list); + + if (reloc->insn >= codegen->len_cur) + return -EINVAL; + + insn = &codegen->img[reloc->insn]; + + if (reloc->type == CODEGEN_RELOC_MAP) { + enum codegen_map_type map_type; + + if (codegen->len_cur <= reloc->insn + 1) + return -EINVAL; + + if (insn->code != (BPF_LD | BPF_DW | BPF_IMM)) + return -EINVAL; + + map_type = insn->imm; + if (map_type < 0 || map_type >= __MAX_CODEGEN_MAP_TYPE) + return -EINVAL; + + BUG_ON(shared_codegen->maps_fd[map_type] < 0); + insn->imm = shared_codegen->maps_fd[map_type]; + + continue; + } + + return -EINVAL; + } + + return 0; +} + +static int load_maps(struct codegen *codegen) +{ + struct shared_codegen *shared_codegen; + int i; + + shared_codegen = codegen->shared_codegen; + + if (shared_codegen->maps_refcnt++) + return 0; + + for (i = 0; i < __MAX_CODEGEN_MAP_TYPE; ++i) { + int j, fd, saved_errno; + union bpf_attr *map; + + BUG_ON(shared_codegen->maps_fd[i] > -1); + + map = &shared_codegen->maps[i]; + fd = sys_bpf(BPF_MAP_CREATE, map, sizeof(*map)); + + if (fd > -1) { + shared_codegen->maps_fd[i] = fd; + continue; + } + + saved_errno = errno; + + for (j = 0; j < i; ++j) { + close(shared_codegen->maps_fd[j]); + shared_codegen->maps_fd[j] = -1; + } + + return saved_errno; + } + + return 0; +} + +static void unload_maps(struct codegen *codegen) +{ + struct shared_codegen *shared_codegen; + int i; + + shared_codegen = codegen->shared_codegen; + + if (--shared_codegen->maps_refcnt) + return; + + for (i = 0; i < __MAX_CODEGEN_MAP_TYPE; ++i) { + if (shared_codegen->maps_fd[i] > -1) { + close(shared_codegen->maps_fd[i]); + shared_codegen->maps_fd[i] = -1; + } + } +} + +static int xdp_gen_inline_prologue(struct codegen *codegen) +{ + EMIT(codegen, BPF_MOV64_REG(CODEGEN_REG_CTX, BPF_REG_ARG1)); + EMIT(codegen, BPF_MOV64_REG(CODEGEN_REG_RUNTIME_CTX, BPF_REG_FP)); + EMIT(codegen, BPF_MOV32_IMM(CODEGEN_REG_RETVAL, XDP_ABORTED)); + + return 0; +} + +static int xdp_load_packet_data(struct codegen *codegen, int dst_reg) +{ + EMIT(codegen, BPF_LDX_MEM(BPF_W, dst_reg, CODEGEN_REG_CTX, offsetof(struct xdp_md, data))); + + return 0; +} + +static int xdp_load_packet_data_end(struct codegen *codegen, int dst_reg) +{ + EMIT(codegen, + BPF_LDX_MEM(BPF_W, dst_reg, CODEGEN_REG_CTX, offsetof(struct xdp_md, data_end))); + + return 0; +} + +static int xdp_emit_ret_code(struct codegen *codegen, int ret_code) +{ + int xdp_ret_code; + + if (ret_code == BPFILTER_NF_ACCEPT) + xdp_ret_code = XDP_PASS; + else if (ret_code == BPFILTER_NF_DROP) + xdp_ret_code = XDP_DROP; + else + return -EINVAL; + + EMIT(codegen, BPF_MOV32_IMM(BPF_REG_0, xdp_ret_code)); + + return 0; +} + +static int xdp_gen_inline_epilogue(struct codegen *codegen) +{ + EMIT(codegen, BPF_EXIT_INSN()); + + return 0; +} + +struct xdp_img_ctx { + int fd; +}; + +static int xdp_load_img(struct codegen *codegen) +{ + struct xdp_img_ctx *img_ctx; + int fd, err; + + if (codegen->img_ctx) + return -EINVAL; + + img_ctx = malloc(sizeof(*img_ctx)); + if (!img_ctx) + return -ENOMEM; + + fd = load_img(codegen); + if (fd < 0) { + err = fd; + goto err_free; + } + + // TODO: device id + err = bpf_set_link_xdp_fd(2, fd, 0); + if (err) + goto err_free; + + img_ctx->fd = fd; + codegen->img_ctx = img_ctx; + + return fd; + +err_free: + if (fd > -1) + close(fd); + free(img_ctx); + + return err; +} + +static void xdp_unload_img(struct codegen *codegen) +{ + struct xdp_img_ctx *img_ctx; + + BUG_ON(!codegen->img_ctx); + + img_ctx = (struct xdp_img_ctx *)codegen->img_ctx; + + BUG_ON(img_ctx->fd < 0); + + close(img_ctx->fd); + free(img_ctx); + + codegen->img_ctx = NULL; + + unload_img(codegen); +} + +static const struct codegen_ops xdp_codegen_ops = { + .gen_inline_prologue = xdp_gen_inline_prologue, + .load_packet_data = xdp_load_packet_data, + .load_packet_data_end = xdp_load_packet_data_end, + .emit_ret_code = xdp_emit_ret_code, + .gen_inline_epilogue = xdp_gen_inline_epilogue, + .load_img = xdp_load_img, + .unload_img = xdp_unload_img, +}; + +static int tc_gen_inline_prologue(struct codegen *codegen) +{ + EMIT(codegen, BPF_MOV64_REG(CODEGEN_REG_CTX, BPF_REG_ARG1)); + EMIT(codegen, BPF_MOV64_REG(CODEGEN_REG_RUNTIME_CTX, BPF_REG_FP)); + EMIT(codegen, BPF_MOV32_IMM(CODEGEN_REG_RETVAL, TC_ACT_OK)); + + return 0; +} + +static int tc_load_packet_data(struct codegen *codegen, int dst_reg) +{ + EMIT(codegen, + BPF_LDX_MEM(BPF_W, dst_reg, CODEGEN_REG_CTX, offsetof(struct __sk_buff, data))); + + return 0; +} + +static int tc_load_packet_data_end(struct codegen *codegen, int dst_reg) +{ + EMIT(codegen, BPF_LDX_MEM(BPF_W, CODEGEN_REG_DATA_END, CODEGEN_REG_CTX, + offsetof(struct __sk_buff, data_end))); + + return 0; +} + +static int tc_emit_ret_code(struct codegen *codegen, int ret_code) +{ + int tc_ret_code; + + if (ret_code == BPFILTER_NF_ACCEPT) + tc_ret_code = BPF_OK; + else if (ret_code == BPFILTER_NF_DROP) + tc_ret_code = BPF_DROP; + else + return -EINVAL; + + EMIT(codegen, BPF_MOV32_IMM(BPF_REG_0, tc_ret_code)); + + return 0; +} + +static int tc_gen_inline_epilogue(struct codegen *codegen) +{ + EMIT(codegen, BPF_EXIT_INSN()); + + return 0; +} + +struct tc_img_ctx { + int fd; + struct bpf_tc_hook hook; + struct bpf_tc_opts opts; +}; + +static int tc_load_img(struct codegen *codegen) +{ + struct tc_img_ctx *img_ctx; + int fd, err; + + if (codegen->img_ctx) + return -EINVAL; + + img_ctx = calloc(1, sizeof(*img_ctx)); + if (!img_ctx) + return -ENOMEM; + + img_ctx->hook.sz = sizeof(img_ctx->hook); + img_ctx->hook.ifindex = 2; + img_ctx->hook.attach_point = BPF_TC_EGRESS; + + fd = load_img(codegen); + if (fd < 0) { + err = fd; + goto err_free; + } + + err = bpf_tc_hook_create(&img_ctx->hook); + if (err == -EEXIST) + err = 0; + if (err) + goto err_free; + + img_ctx->opts.sz = sizeof(img_ctx->opts); + img_ctx->opts.handle = 1; + img_ctx->opts.priority = 1; + img_ctx->opts.prog_fd = fd; + err = bpf_tc_attach(&img_ctx->hook, &img_ctx->opts); + if (err) + goto err_free; + + img_ctx->fd = fd; + codegen->img_ctx = img_ctx; + + return fd; + +err_free: + if (fd > -1) + close(fd); + free(img_ctx); + return err; +} + +static void tc_unload_img(struct codegen *codegen) +{ + struct tc_img_ctx *img_ctx; + int err; + + BUG_ON(!codegen->img_ctx); + + img_ctx = (struct tc_img_ctx *)codegen->img_ctx; + img_ctx->opts.flags = 0; + img_ctx->opts.prog_fd = 0; + img_ctx->opts.prog_id = 0; + err = bpf_tc_detach(&img_ctx->hook, &img_ctx->opts); + if (err) + BFLOG_EMERG(codegen->ctx, "cannot detach tc program: %s\n", strerror(-err)); + + err = bpf_tc_hook_destroy(&img_ctx->hook); + if (err) + BFLOG_EMERG(codegen->ctx, "cannot destroy tc hook: %s\n", strerror(-err)); + + BUG_ON(img_ctx->fd < 0); + close(img_ctx->fd); + free(img_ctx); + + codegen->img_ctx = NULL; + + unload_img(codegen); +} + +static const struct codegen_ops tc_codegen_ops = { + .gen_inline_prologue = tc_gen_inline_prologue, + .load_packet_data = tc_load_packet_data, + .load_packet_data_end = tc_load_packet_data_end, + .emit_ret_code = tc_emit_ret_code, + .gen_inline_epilogue = tc_gen_inline_epilogue, + .load_img = tc_load_img, + .unload_img = tc_unload_img, +}; + +void create_shared_codegen(struct shared_codegen *shared_codegen) +{ + shared_codegen->maps_refcnt = 0; + + shared_codegen->maps[CODEGEN_MAP_COUNTERS].map_type = BPF_MAP_TYPE_PERCPU_ARRAY; + shared_codegen->maps[CODEGEN_MAP_COUNTERS].key_size = 4; + shared_codegen->maps[CODEGEN_MAP_COUNTERS].value_size = + sizeof(struct bpfilter_ipt_counters); + shared_codegen->maps[CODEGEN_MAP_COUNTERS].max_entries = 0; + snprintf(shared_codegen->maps[CODEGEN_MAP_COUNTERS].map_name, + sizeof(shared_codegen->maps[CODEGEN_MAP_COUNTERS].map_name), "bpfilter_cntrs"); + shared_codegen->maps_fd[CODEGEN_MAP_COUNTERS] = -1; +} + +int create_codegen(struct codegen *codegen, enum bpf_prog_type type) +{ + int err; + + memset(codegen, 0, sizeof(*codegen)); + + if (type == BPF_PROG_TYPE_XDP) + codegen->codegen_ops = &xdp_codegen_ops; + else if (type == BPF_PROG_TYPE_SCHED_CLS) + codegen->codegen_ops = &tc_codegen_ops; + else + return -EINVAL; + codegen->prog_type = type; + + codegen->log_buf_size = 1 << 20; + codegen->log_buf = malloc(codegen->log_buf_size); + if (!codegen->log_buf) { + err = -ENOMEM; + goto err_free; + } + + codegen->len_max = BPF_MAXINSNS; + codegen->img = malloc(codegen->len_max * sizeof(codegen->img[0])); + if (!codegen->img) { + err = -ENOMEM; + goto err_free; + } + + INIT_LIST_HEAD(&codegen->fixup); + INIT_LIST_HEAD(&codegen->relocs); + INIT_LIST_HEAD(&codegen->awaiting_subprogs); + + return 0; + +err_free: + free(codegen->img); + + return err; +} + +int load_img(struct codegen *codegen) +{ + union bpf_attr attr = {}; + int err, fd; + + err = load_maps(codegen); + if (err) + return err; + + err = codegen_reloc(codegen); + if (err) + return err; + + attr.prog_type = codegen->prog_type; + attr.insns = bpf_ptr_to_u64(codegen->img); + attr.insn_cnt = codegen->len_cur; + attr.license = bpf_ptr_to_u64("GPL"); + attr.prog_ifindex = 0; + snprintf(attr.prog_name, sizeof(attr.prog_name), "bpfilter"); + + if (codegen->log_buf && codegen->log_buf_size) { + attr.log_buf = bpf_ptr_to_u64(codegen->log_buf); + attr.log_size = codegen->log_buf_size; + attr.log_level = 1; + } + + fd = sys_bpf(BPF_PROG_LOAD, &attr, sizeof(attr)); + if (fd == -1) { + BFLOG_DEBUG(codegen->ctx, "Cannot load BPF program: %s\n", codegen->log_buf); + return -errno; + } + + return fd; +} + + +void unload_img(struct codegen *codegen) +{ + unload_maps(codegen); +} + +void free_codegen(struct codegen *codegen) +{ + struct list_head *t, *n; + int i; + + list_for_each_safe(t, n, &codegen->fixup) { + struct codegen_fixup_desc *fixup; + + fixup = list_entry(t, struct codegen_fixup_desc, list); + free(fixup); + } + + list_for_each_safe(t, n, &codegen->relocs) { + struct codegen_reloc_desc *reloc; + + reloc = list_entry(t, struct codegen_reloc_desc, list); + free(reloc); + } + + list_for_each_safe(t, n, &codegen->awaiting_subprogs) { + struct codegen_subprog_desc *subprog; + + subprog = list_entry(t, struct codegen_subprog_desc, list); + free(subprog); + } + + for (i = 0; i < codegen->subprogs_cur; ++i) + free(codegen->subprogs[i]); + free(codegen->subprogs); + + free(codegen->log_buf); + free(codegen->img); +} diff --git a/net/bpfilter/codegen.h b/net/bpfilter/codegen.h new file mode 100644 index 000000000000..7953f6938dcc --- /dev/null +++ b/net/bpfilter/codegen.h @@ -0,0 +1,188 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* + * Copyright (c) 2021 Telegram FZ-LLC + */ + +#ifndef NET_BPFILTER_CODEGEN_H +#define NET_BPFILTER_CODEGEN_H + +#include +#include +#include +#include +#include + +#include + +#include +#include +#include +#include +#include + +struct context; +struct table; +struct rule; + +#define CODEGEN_REG_RETVAL BPF_REG_0 +#define CODEGEN_REG_SCRATCH1 BPF_REG_1 +#define CODEGEN_REG_SCRATCH2 BPF_REG_2 +#define CODEGEN_REG_SCRATCH3 BPF_REG_3 +#define CODEGEN_REG_SCRATCH4 BPF_REG_4 +#define CODEGEN_REG_SCRATCH5 BPF_REG_5 +#define CODEGEN_REG_DATA_END CODEGEN_REG_SCRATCH5 +#define CODEGEN_REG_L3 BPF_REG_6 +#define CODEGEN_REG_L4 BPF_REG_7 +#define CODEGEN_REG_RUNTIME_CTX BPF_REG_8 +#define CODEGEN_REG_CTX BPF_REG_9 + +#define EMIT(codegen, x) \ + do { \ + if (codegen->len_cur + 1 > codegen->len_max) \ + return -ENOMEM; \ + codegen->img[codegen->len_cur++] = x; \ + } while (0) + +#define EMIT_FIXUP(codegen, fixup_type, x) \ + do { \ + const int __err = emit_fixup(codegen, fixup_type, x); \ + if (__err) \ + return __err; \ + } while (0) + +#define EMIT_ADD_COUNTER(codegen) \ + do { \ + const int __err = emit_add_counter(codegen); \ + if (__err) \ + return __err; \ + } while (0) + +#if __BYTE_ORDER__ == __ORDER_LITTLE_ENDIAN__ +#define EMIT_LITTLE_ENDIAN(codegen, x) EMIT(codegen, x) +#else +#define EMIT_LITTLE_ENDIAN(codegen, x) +#endif + +#define EMIT_DEBUG(codegen, reg) \ + do { \ + EMIT(codegen, BPF_ST_MEM(BPF_W, BPF_REG_10, STACK_SCRATCHPAD_OFFSET - 44, \ + __bpf_constant_ntohl(0x6c750000))); \ + EMIT(codegen, BPF_ST_MEM(BPF_W, BPF_REG_10, STACK_SCRATCHPAD_OFFSET - 48, \ + __bpf_constant_ntohl(0x4720256c))); \ + EMIT(codegen, BPF_ST_MEM(BPF_W, BPF_REG_10, STACK_SCRATCHPAD_OFFSET - 52, \ + __bpf_constant_ntohl(0x42464442))); \ + EMIT(codegen, BPF_MOV64_REG(BPF_REG_1, BPF_REG_10)); \ + EMIT(codegen, BPF_ALU64_IMM(BPF_ADD, BPF_REG_1, STACK_SCRATCHPAD_OFFSET - 52)); \ + EMIT(codegen, BPF_MOV32_IMM(BPF_REG_2, 12)); \ + EMIT(codegen, BPF_MOV64_REG(BPF_REG_3, reg)); \ + EMIT(codegen, BPF_EMIT_CALL(BPF_FUNC_trace_printk)); \ + } while (0) + +struct runtime_context { + uint32_t data_size; + void *l3; + void *l4; +}; + +#define STACK_RUNTIME_CONTEXT_OFFSET(field) \ + (-(short)(offsetof(struct runtime_context, field) + \ + sizeof(((struct runtime_context *)NULL)->field))) + +#define STACK_SCRATCHPAD_OFFSET (-(short)sizeof(struct runtime_context)) + +enum codegen_map_type { CODEGEN_MAP_COUNTERS, __MAX_CODEGEN_MAP_TYPE }; + +enum codegen_fixup_type { + CODEGEN_FIXUP_NEXT_RULE, + CODEGEN_FIXUP_END_OF_CHAIN, + CODEGEN_FIXUP_JUMP_TO_CHAIN, + CODEGEN_FIXUP_COUNTERS_INDEX, + __MAX_CODEGEN_FIXUP_TYPE +}; + +struct codegen_fixup_desc { + struct list_head list; + enum codegen_fixup_type type; + uint32_t insn; + union { + uint32_t offset; + }; +}; + +enum codegen_reloc_type { CODEGEN_RELOC_MAP, __MAX_CODEGEN_RELOC_TYPE }; + +struct codegen_reloc_desc { + struct list_head list; + enum codegen_reloc_type type; + uint32_t insn; + union { + struct { + enum codegen_map_type map; + // TODO: add BTF + }; + }; +}; + +enum codegen_subprog_type { + CODEGEN_SUBPROG_USER_CHAIN, +}; + +struct codegen_subprog_desc { + struct list_head list; + enum codegen_subprog_type type; + uint32_t insn; + union { + uint32_t offset; + }; +}; + +struct codegen_ops; +struct shared_codegen; + +struct codegen { + struct context *ctx; + struct bpf_insn *img; + char *log_buf; + size_t log_buf_size; + enum bpf_prog_type prog_type; + uint32_t len_cur; + uint32_t len_max; + uint32_t rule_index; + const struct codegen_ops *codegen_ops; + struct shared_codegen *shared_codegen; + struct list_head fixup; + struct list_head relocs; + struct list_head awaiting_subprogs; + uint16_t subprogs_cur; + uint16_t subprogs_max; + struct codegen_subprog_desc **subprogs; + void *img_ctx; +}; + +struct shared_codegen { + int maps_refcnt; + union bpf_attr maps[__MAX_CODEGEN_MAP_TYPE]; + int maps_fd[__MAX_CODEGEN_MAP_TYPE]; +}; + +struct codegen_ops { + int (*gen_inline_prologue)(struct codegen *codegen); + int (*load_packet_data)(struct codegen *codegen, int dst_reg); + int (*load_packet_data_end)(struct codegen *codegen, int dst_reg); + int (*emit_ret_code)(struct codegen *codegen, int ret_code); + int (*gen_inline_epilogue)(struct codegen *codegen); + int (*load_img)(struct codegen *codegen); + void (*unload_img)(struct codegen *codegen); +}; + +void create_shared_codegen(struct shared_codegen *shared_codegen); +int create_codegen(struct codegen *codegen, enum bpf_prog_type type); +int codegen_push_awaiting_subprog(struct codegen *codegen, struct codegen_subprog_desc *subprog); +int codegen_fixup(struct codegen *codegen, enum codegen_fixup_type fixup_type); +int emit_fixup(struct codegen *codegen, enum codegen_fixup_type fixup_type, struct bpf_insn insn); +int emit_add_counter(struct codegen *codegen); +int load_img(struct codegen *codegen); +void unload_img(struct codegen *codegen); +void free_codegen(struct codegen *codegen); + +#endif // NET_BPFILTER_CODEGEN_H diff --git a/tools/testing/selftests/bpf/bpfilter/.gitignore b/tools/testing/selftests/bpf/bpfilter/.gitignore index 983fd06cbefa..39ec0c09dff4 100644 --- a/tools/testing/selftests/bpf/bpfilter/.gitignore +++ b/tools/testing/selftests/bpf/bpfilter/.gitignore @@ -1,2 +1,3 @@ # SPDX-License-Identifier: GPL-2.0-only +tools/** test_map diff --git a/tools/testing/selftests/bpf/bpfilter/Makefile b/tools/testing/selftests/bpf/bpfilter/Makefile index c262aad8c2a4..48dc696e0f09 100644 --- a/tools/testing/selftests/bpf/bpfilter/Makefile +++ b/tools/testing/selftests/bpf/bpfilter/Makefile @@ -4,9 +4,11 @@ top_srcdir = ../../../../.. TOOLSDIR := $(abspath ../../../../) TOOLSINCDIR := $(TOOLSDIR)/include APIDIR := $(TOOLSINCDIR)/uapi +LIBDIR := $(TOOLSDIR)/lib +BPFDIR := $(LIBDIR)/bpf BPFILTERSRCDIR := $(top_srcdir)/net/bpfilter -CFLAGS += -Wall -g -pthread -I$(TOOLSINCDIR) -I$(APIDIR) -I$(BPFILTERSRCDIR) +CFLAGS += -Wall -g -pthread -I$(TOOLSINCDIR) -I$(APIDIR) -I$(BPFILTERSRCDIR) -I$(LIBDIR) TEST_GEN_PROGS += test_map @@ -14,6 +16,24 @@ KSFT_KHDR_INSTALL := 1 include ../../lib.mk +SCRATCH_DIR := $(OUTPUT)/tools +BUILD_DIR := $(SCRATCH_DIR)/build +BPFOBJ_DIR := $(BUILD_DIR)/libbpf +BPFOBJ := $(BPFOBJ_DIR)/libbpf.a + +MAKE_DIRS := $(BPFOBJ_DIR) + +$(MAKE_DIRS): + $(call msg,MKDIR,,$@) + $(Q)mkdir -p $@ + +$(BPFOBJ): $(wildcard $(BPFDIR)/*.[ch] $(BPFDIR)/Makefile) \ + ../../../../include/uapi/linux/bpf.h \ + | $(INCLUDE_DIR) $(BUILD_DIR)/libbpf + $(Q)$(MAKE) $(submake_extras) -C $(BPFDIR) OUTPUT=$(BUILD_DIR)/libbpf/ \ + DESTDIR=$(SCRATCH_DIR) prefix= all install_headers + BPFILTER_MAP_SRCS := $(BPFILTERSRCDIR)/map-common.c +BPFILTER_CODEGEN_SRCS := $(BPFILTERSRCDIR)/codegen.c $(BPFOBJ) -lelf -lz $(OUTPUT)/test_map: test_map.c $(BPFILTER_MAP_SRCS) From patchwork Sun Aug 29 18:36:03 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dmitrii Banshchikov X-Patchwork-Id: 504224 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER, INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 05D74C432BE for ; Sun, 29 Aug 2021 18:36:53 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id BC20860E73 for ; Sun, 29 Aug 2021 18:36:50 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235968AbhH2Shl (ORCPT ); Sun, 29 Aug 2021 14:37:41 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:34190 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235919AbhH2Shg (ORCPT ); Sun, 29 Aug 2021 14:37:36 -0400 Received: from mail-ej1-x636.google.com (mail-ej1-x636.google.com [IPv6:2a00:1450:4864:20::636]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 0B234C061760 for ; Sun, 29 Aug 2021 11:36:43 -0700 (PDT) Received: by mail-ej1-x636.google.com with SMTP id x11so26356475ejv.0 for ; Sun, 29 Aug 2021 11:36:42 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ubique-spb-ru.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=BtgTkv3mSDS8aWw4uQYLtSWS3LezVDlLHx+UPyNzf6Q=; b=FZxNkiiu+tPrRWZCWPWvHCA55MOR2gex7Fla2AClUrGIXRvpATjXzwk8xjpgNhfj4B ZUSqYdDU70IZnmWW0RWLYU/eycmZvKMRqoxIU/CS7PugZAV3qmf2YAix6L6vbRZG13qM CO6NzwwD8rWfFa2WxssM1BZ+jg9pD1JGVkAhWMfnCxoHo9wxwvQFJXaCR5yHjqOXiEOO azSSMmi4XBtIcdDKqkMPnfYleSHOG7cskWCaH/7frOI+7ktB4gh9MrD5bRZo2ljchx9U KmXSbMsZvtZloZNhEDdF+wu50jo7qs8H03jWVn3hgMv/MlOivu53nke7HsOV/liYcepG idzA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=BtgTkv3mSDS8aWw4uQYLtSWS3LezVDlLHx+UPyNzf6Q=; b=RDHxTBl1/mVJgif45+DhL+6NMObrwworOtjYhZOvxZocygPU+ZPolYSI7nyrevegmw Pwrv5tMBAIVUxsMwBUxPgIEiqQFx/7s9DsR2rGPlZ64rSA3asyyhN3S8nzf483moRopi Lohz+ClYbegN6vVRoKj76EXG9gA+OaqBOU6IzPpjHBqyFY1BckeDch7OsTTPyqawdk1S dSIBnbSNTlF3wNWsjntfmFrcQquc5Cu58HcClKxXqZO5Ur4hodhBrRXzWh2P0KQw9m/M z/bokMCs9KX+46GtaQEdMqxCv2bpvZcsOutqB+xd7D8c/f4hobluemNNPYfWKCOq1nuW gexw== X-Gm-Message-State: AOAM530/daDOEtfV0i9tEA3ZqgyEYcvh064IQh/ECM+Dh5eyKrJg56/N kFQFL/bBqGZcV/xwE10hETDFDA== X-Google-Smtp-Source: ABdhPJyNtp5Ml6UutfOPDN9NmiHR5hxX7mEzRu0q/1G6MYGzDTdVeKV0HU29IIQYckeG9BTURyFiSw== X-Received: by 2002:a17:907:2126:: with SMTP id qo6mr21410892ejb.476.1630262201521; Sun, 29 Aug 2021 11:36:41 -0700 (PDT) Received: from localhost ([154.21.15.43]) by smtp.gmail.com with ESMTPSA id l23sm6521425eds.29.2021.08.29.11.36.40 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 29 Aug 2021 11:36:40 -0700 (PDT) From: Dmitrii Banshchikov To: bpf@vger.kernel.org Cc: Dmitrii Banshchikov , ast@kernel.org, davem@davemloft.net, daniel@iogearbox.net, andrii@kernel.org, kafai@fb.com, songliubraving@fb.com, yhs@fb.com, john.fastabend@gmail.com, kpsingh@kernel.org, netdev@vger.kernel.org, rdna@fb.com Subject: [PATCH bpf-next v2 08/13] bpfilter: Add struct rule Date: Sun, 29 Aug 2021 22:36:03 +0400 Message-Id: <20210829183608.2297877-9-me@ubique.spb.ru> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210829183608.2297877-1-me@ubique.spb.ru> References: <20210829183608.2297877-1-me@ubique.spb.ru> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org struct rule is an equivalent of struct ipt_entry. A rule consists of zero or more matches and a target. A rule has a pointer to its ipt_entry in entries blob. struct rule should simplify iteration over a blob and avoid blob's guts in code generation. The inline way of code generation for a rule is performed by gen_inline_rule() and consists of the following: 1) Emit instructions for rule's L3 src/dst addresses and protocol 2) Emit instructions for each rule's match by calling match's interface 3) Emit instructions for rule's target by calling target's interface Signed-off-by: Dmitrii Banshchikov --- net/bpfilter/Makefile | 2 +- net/bpfilter/rule.c | 239 ++++++++++++++++++ net/bpfilter/rule.h | 34 +++ .../testing/selftests/bpf/bpfilter/.gitignore | 1 + tools/testing/selftests/bpf/bpfilter/Makefile | 5 +- .../selftests/bpf/bpfilter/bpfilter_util.h | 8 + .../selftests/bpf/bpfilter/test_rule.c | 55 ++++ 7 files changed, 342 insertions(+), 2 deletions(-) create mode 100644 net/bpfilter/rule.c create mode 100644 net/bpfilter/rule.h create mode 100644 tools/testing/selftests/bpf/bpfilter/test_rule.c diff --git a/net/bpfilter/Makefile b/net/bpfilter/Makefile index a7c643a1b52a..3f7c5c28cca2 100644 --- a/net/bpfilter/Makefile +++ b/net/bpfilter/Makefile @@ -11,7 +11,7 @@ $(LIBBPF_A): $(Q)$(MAKE) -C $(LIBBPF_SRCS) O=$(LIBBPF_OUT)/ OUTPUT=$(LIBBPF_OUT)/ $(LIBBPF_OUT)/libbpf.a userprogs := bpfilter_umh -bpfilter_umh-objs := main.o map-common.o codegen.o context.o match.o target.o +bpfilter_umh-objs := main.o map-common.o codegen.o context.o match.o target.o rule.o bpfilter_umh-objs += xt_udp.o bpfilter_umh-userldlibs := $(LIBBPF_A) -lelf -lz userccflags += -I $(srctree)/tools/include/ -I $(srctree)/tools/include/uapi -I $(srctree)/tools/lib diff --git a/net/bpfilter/rule.c b/net/bpfilter/rule.c new file mode 100644 index 000000000000..4b6a7f10db5b --- /dev/null +++ b/net/bpfilter/rule.c @@ -0,0 +1,239 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * Copyright (c) 2021 Telegram FZ-LLC + */ + +#define _GNU_SOURCE + +#include "rule.h" + +#include "../../include/uapi/linux/bpfilter.h" + +#include +#include +#include +#include + +#include +#include +#include + +#include "codegen.h" +#include "context.h" +#include "match.h" + +static const struct bpfilter_ipt_target * +ipt_entry_target(const struct bpfilter_ipt_entry *ipt_entry) +{ + return (const void *)ipt_entry + ipt_entry->target_offset; +} + +static const struct bpfilter_ipt_match *ipt_entry_match(const struct bpfilter_ipt_entry *entry, + size_t offset) +{ + return (const void *)entry + offset; +} + +static int ipt_entry_num_matches(const struct bpfilter_ipt_entry *ipt_entry) +{ + const struct bpfilter_ipt_match *ipt_match; + uint32_t offset = sizeof(*ipt_entry); + int num_matches = 0; + + while (offset < ipt_entry->target_offset) { + ipt_match = ipt_entry_match(ipt_entry, offset); + + if ((uintptr_t)ipt_match % __alignof__(struct bpfilter_ipt_match)) + return -EINVAL; + + if (ipt_entry->target_offset < offset + sizeof(*ipt_match)) + return -EINVAL; + + if (ipt_match->u.match_size < sizeof(*ipt_match)) + return -EINVAL; + + if (ipt_entry->target_offset < offset + ipt_match->u.match_size) + return -EINVAL; + + ++num_matches; + offset += ipt_match->u.match_size; + } + + if (offset != ipt_entry->target_offset) + return -EINVAL; + + return num_matches; +} + +static int init_rule_matches(struct context *ctx, const struct bpfilter_ipt_entry *ipt_entry, + struct rule *rule) +{ + const struct bpfilter_ipt_match *ipt_match; + uint32_t offset = sizeof(*ipt_entry); + struct match *match; + int err; + + rule->matches = calloc(rule->num_matches, sizeof(rule->matches[0])); + if (!rule->matches) + return -ENOMEM; + + match = rule->matches; + while (offset < ipt_entry->target_offset) { + ipt_match = ipt_entry_match(ipt_entry, offset); + err = init_match(ctx, ipt_match, match); + if (err) { + free(rule->matches); + rule->matches = NULL; + return err; + } + + ++match; + offset += ipt_match->u.match_size; + } + + return 0; +} + +static int check_ipt_entry_ip(const struct bpfilter_ipt_ip *ip) +{ + if (ip->flags & ~BPFILTER_IPT_F_MASK) + return -EINVAL; + + if (ip->invflags & ~BPFILTER_IPT_INV_MASK) + return -EINVAL; + + return 0; +} + +bool rule_has_standard_target(const struct rule *rule) +{ + return rule->target.target_ops == &standard_target_ops; +} + +bool is_rule_unconditional(const struct rule *rule) +{ + static const struct bpfilter_ipt_ip unconditional; + + if (rule->num_matches) + return false; + + return !memcmp(&rule->ipt_entry->ip, &unconditional, sizeof(unconditional)); +} + +int init_rule(struct context *ctx, const struct bpfilter_ipt_entry *ipt_entry, struct rule *rule) +{ + const struct bpfilter_ipt_target *ipt_target; + int err; + + err = check_ipt_entry_ip(&ipt_entry->ip); + if (err) + return err; + + if (ipt_entry->target_offset < sizeof(*ipt_entry)) + return -EINVAL; + + if (ipt_entry->next_offset < ipt_entry->target_offset + sizeof(*ipt_target)) + return -EINVAL; + + ipt_target = ipt_entry_target(ipt_entry); + + if (ipt_target->u.target_size < sizeof(*ipt_target)) + return -EINVAL; + + if (ipt_entry->next_offset < ipt_entry->target_offset + ipt_target->u.target_size) + return -EINVAL; + + rule->ipt_entry = ipt_entry; + + err = init_target(ctx, ipt_target, &rule->target); + if (err) + return err; + + if (rule_has_standard_target(rule)) { + if (XT_ALIGN(ipt_entry->target_offset + + sizeof(struct bpfilter_ipt_standard_target)) != ipt_entry->next_offset) + return -EINVAL; + } + + rule->num_matches = ipt_entry_num_matches(ipt_entry); + if (rule->num_matches < 0) + return rule->num_matches; + + return init_rule_matches(ctx, ipt_entry, rule); +} + +int gen_inline_rule(struct codegen *ctx, const struct rule *rule) +{ + int i, err; + + const struct bpfilter_ipt_ip *ipt_ip = &rule->ipt_entry->ip; + + if (!ipt_ip->src_mask && !ipt_ip->src) { + if (ipt_ip->invflags & IPT_INV_SRCIP) + return 0; + } + + if (!ipt_ip->dst_mask && !ipt_ip->dst) { + if (ipt_ip->invflags & IPT_INV_DSTIP) + return 0; + } + + if (ipt_ip->src_mask || ipt_ip->src) { + const int op = ipt_ip->invflags & IPT_INV_SRCIP ? BPF_JEQ : BPF_JNE; + + EMIT(ctx, BPF_LDX_MEM(BPF_W, CODEGEN_REG_SCRATCH1, CODEGEN_REG_L3, + offsetof(struct iphdr, saddr))); + EMIT(ctx, BPF_ALU32_IMM(BPF_AND, CODEGEN_REG_SCRATCH1, ipt_ip->src_mask)); + EMIT_FIXUP(ctx, CODEGEN_FIXUP_NEXT_RULE, + BPF_JMP_IMM(op, CODEGEN_REG_SCRATCH1, ipt_ip->src, 0)); + } + + if (ipt_ip->dst_mask || ipt_ip->dst) { + const int op = ipt_ip->invflags & IPT_INV_DSTIP ? BPF_JEQ : BPF_JNE; + + EMIT(ctx, BPF_LDX_MEM(BPF_W, CODEGEN_REG_SCRATCH2, CODEGEN_REG_L3, + offsetof(struct iphdr, daddr))); + EMIT(ctx, BPF_ALU32_IMM(BPF_AND, CODEGEN_REG_SCRATCH2, ipt_ip->dst_mask)); + EMIT_FIXUP(ctx, CODEGEN_FIXUP_NEXT_RULE, + BPF_JMP_IMM(op, CODEGEN_REG_SCRATCH2, ipt_ip->dst, 0)); + } + + if (ipt_ip->protocol) { + EMIT(ctx, BPF_LDX_MEM(BPF_B, CODEGEN_REG_SCRATCH4, CODEGEN_REG_L3, + offsetof(struct iphdr, protocol))); + EMIT_FIXUP(ctx, CODEGEN_FIXUP_NEXT_RULE, + BPF_JMP_IMM(BPF_JNE, CODEGEN_REG_SCRATCH4, ipt_ip->protocol, 0)); + + EMIT(ctx, BPF_LDX_MEM(BPF_B, CODEGEN_REG_SCRATCH4, CODEGEN_REG_L3, + offsetof(struct iphdr, protocol))); + EMIT(ctx, BPF_MOV64_REG(CODEGEN_REG_L4, CODEGEN_REG_L3)); + EMIT(ctx, BPF_LDX_MEM(BPF_B, CODEGEN_REG_SCRATCH1, CODEGEN_REG_L3, 0)); + EMIT(ctx, BPF_ALU32_IMM(BPF_AND, CODEGEN_REG_SCRATCH1, 0x0f)); + EMIT(ctx, BPF_ALU32_IMM(BPF_LSH, CODEGEN_REG_SCRATCH1, 2)); + EMIT(ctx, BPF_ALU64_REG(BPF_ADD, CODEGEN_REG_L4, CODEGEN_REG_SCRATCH1)); + } + + for (i = 0; i < rule->num_matches; ++i) { + const struct match *match; + + match = &rule->matches[i]; + err = match->match_ops->gen_inline(ctx, match); + if (err) + return err; + } + + EMIT_ADD_COUNTER(ctx); + + err = rule->target.target_ops->gen_inline(ctx, &rule->target); + if (err) + return err; + + codegen_fixup(ctx, CODEGEN_FIXUP_NEXT_RULE); + + return 0; +} + +void free_rule(struct rule *rule) +{ + free(rule->matches); +} diff --git a/net/bpfilter/rule.h b/net/bpfilter/rule.h new file mode 100644 index 000000000000..a19e698b5976 --- /dev/null +++ b/net/bpfilter/rule.h @@ -0,0 +1,34 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* + * Copyright (c) 2021 Telegram FZ-LLC + */ + +#ifndef NET_BPFILTER_RULE_H +#define NET_BPFILTER_RULE_H + +#include +#include + +#include "target.h" + +struct bpfilter_ipt_entry; +struct codegen; +struct context; +struct match; + +struct rule { + const struct bpfilter_ipt_entry *ipt_entry; + uint32_t came_from; + uint32_t hook_mask; + uint16_t num_matches; + struct match *matches; + struct target target; +}; + +bool rule_has_standard_target(const struct rule *rule); +bool is_rule_unconditional(const struct rule *rule); +int init_rule(struct context *ctx, const struct bpfilter_ipt_entry *ipt_entry, struct rule *rule); +int gen_inline_rule(struct codegen *ctx, const struct rule *rule); +void free_rule(struct rule *rule); + +#endif // NET_BPFILTER_RULE_H diff --git a/tools/testing/selftests/bpf/bpfilter/.gitignore b/tools/testing/selftests/bpf/bpfilter/.gitignore index 89912a44109f..a934ddef58d2 100644 --- a/tools/testing/selftests/bpf/bpfilter/.gitignore +++ b/tools/testing/selftests/bpf/bpfilter/.gitignore @@ -4,3 +4,4 @@ test_map test_match test_xt_udp test_target +test_rule diff --git a/tools/testing/selftests/bpf/bpfilter/Makefile b/tools/testing/selftests/bpf/bpfilter/Makefile index 670f28413e42..779add65fa27 100644 --- a/tools/testing/selftests/bpf/bpfilter/Makefile +++ b/tools/testing/selftests/bpf/bpfilter/Makefile @@ -14,6 +14,7 @@ TEST_GEN_PROGS += test_map TEST_GEN_PROGS += test_match TEST_GEN_PROGS += test_xt_udp TEST_GEN_PROGS += test_target +TEST_GEN_PROGS += test_rule KSFT_KHDR_INSTALL := 1 @@ -40,12 +41,14 @@ BPFILTER_MAP_SRCS := $(BPFILTERSRCDIR)/map-common.c BPFILTER_CODEGEN_SRCS := $(BPFILTERSRCDIR)/codegen.c $(BPFOBJ) -lelf -lz BPFILTER_MATCH_SRCS := $(BPFILTERSRCDIR)/match.c $(BPFILTERSRCDIR)/xt_udp.c BPFILTER_TARGET_SRCS := $(BPFILTERSRCDIR)/target.c +BPFILTER_RULE_SRCS := $(BPFILTERSRCDIR)/rule.c BPFILTER_COMMON_SRCS := $(BPFILTERSRCDIR)/context.c BPFILTER_COMMON_SRCS += $(BPFILTER_MAP_SRCS) $(BPFILTER_CODEGEN_SRCS) $(BPFILTER_MATCH_SRCS) -BPFILTER_COMMON_SRCS += $(BPFILTER_TARGET_SRCS) +BPFILTER_COMMON_SRCS += $(BPFILTER_TARGET_SRCS) $(BPFILTER_RULE_SRCS) $(OUTPUT)/test_map: test_map.c $(BPFILTER_MAP_SRCS) $(OUTPUT)/test_match: test_match.c $(BPFILTER_COMMON_SRCS) $(OUTPUT)/test_xt_udp: test_xt_udp.c $(BPFILTER_COMMON_SRCS) $(OUTPUT)/test_target: test_target.c $(BPFILTER_COMMON_SRCS) +$(OUTPUT)/test_rule: test_rule.c $(BPFILTER_COMMON_SRCS) diff --git a/tools/testing/selftests/bpf/bpfilter/bpfilter_util.h b/tools/testing/selftests/bpf/bpfilter/bpfilter_util.h index 945633c5415e..07cfe24d763d 100644 --- a/tools/testing/selftests/bpf/bpfilter/bpfilter_util.h +++ b/tools/testing/selftests/bpf/bpfilter/bpfilter_util.h @@ -5,6 +5,7 @@ #include #include +#include #include #include @@ -39,4 +40,11 @@ static inline void init_error_target(struct xt_error_target *ipt_target, int rev snprintf(ipt_target->errorname, sizeof(ipt_target->errorname), "%s", error_name); } +static inline void init_standard_entry(struct ipt_entry *entry, __u16 matches_size) +{ + memset(entry, 0, sizeof(*entry)); + entry->target_offset = sizeof(*entry) + matches_size; + entry->next_offset = sizeof(*entry) + matches_size + sizeof(struct xt_standard_target); +} + #endif // BPFILTER_UTIL_H diff --git a/tools/testing/selftests/bpf/bpfilter/test_rule.c b/tools/testing/selftests/bpf/bpfilter/test_rule.c new file mode 100644 index 000000000000..fe12adf32fe5 --- /dev/null +++ b/tools/testing/selftests/bpf/bpfilter/test_rule.c @@ -0,0 +1,55 @@ +// SPDX-License-Identifier: GPL-2.0 + +#define _GNU_SOURCE + +#include "rule.h" + +#include +#include + +#include + +#include +#include + +#include "../../kselftest_harness.h" + +#include "context.h" +#include "rule.h" + +#include "bpfilter_util.h" + +FIXTURE(test_standard_rule) +{ + struct context ctx; + struct { + struct ipt_entry entry; + struct xt_standard_target target; + } entry; + struct rule rule; +}; + +FIXTURE_SETUP(test_standard_rule) +{ + const int verdict = BPFILTER_NF_ACCEPT; + + ASSERT_EQ(create_context(&self->ctx), 0); + self->ctx.log_file = stderr; + + init_standard_entry(&self->entry.entry, 0); + init_standard_target(&self->entry.target, 0, -verdict - 1); +} + +FIXTURE_TEARDOWN(test_standard_rule) +{ + free_rule(&self->rule); + free_context(&self->ctx); +} + +TEST_F(test_standard_rule, init) +{ + ASSERT_EQ(0, init_rule(&self->ctx, (const struct bpfilter_ipt_entry *)&self->entry.entry, + &self->rule)); +} + +TEST_HARNESS_MAIN From patchwork Sun Aug 29 18:36:06 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dmitrii Banshchikov X-Patchwork-Id: 504223 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER, INCLUDES_PATCH, MAILING_LIST_MULTI, SPF_HELO_NONE, SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8DA63C4320A for ; Sun, 29 Aug 2021 18:37:00 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 7900460E73 for ; Sun, 29 Aug 2021 18:37:00 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S236036AbhH2Shu (ORCPT ); Sun, 29 Aug 2021 14:37:50 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:34232 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235975AbhH2Shl (ORCPT ); Sun, 29 Aug 2021 14:37:41 -0400 Received: from mail-ej1-x629.google.com (mail-ej1-x629.google.com [IPv6:2a00:1450:4864:20::629]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id E9B3FC061756 for ; Sun, 29 Aug 2021 11:36:48 -0700 (PDT) Received: by mail-ej1-x629.google.com with SMTP id x11so26356807ejv.0 for ; Sun, 29 Aug 2021 11:36:48 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ubique-spb-ru.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=+8jMT5FQ/RCcRM7BkIWLJpUZ5liVXckU0LjWCk2OYwE=; b=ybwCqmhFps1WWzbkqPb7/mSEbGrzXZcKMBt+2GbRY0NPXwuFFq4ogzHR17CzLyEqXx ngSZHia4jejUZ8w1cvtQxxggim7g9NhPyTnK+U1u5+bvwY7vQCl7OmdBJjledGUElfwq sr2nc97hRZd8j3+k3w9eF1aoJ9Ygr8T6WnEtnKPf7/5dT+Zx0ibMluJCjyu+hP96Qcx2 brw340lu5pOIszPdQT7eB1r7f1UlX0P5WM/oUVcL3c8LL0cdn8d0QtPVCCoCQIwM36HE KNBod2VrcKWgCTcvuhMiFAOnqbd8HW8W2hnzsAVKhM8KDJojMquhtgh+iXC5nhoXDKKc ungA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=+8jMT5FQ/RCcRM7BkIWLJpUZ5liVXckU0LjWCk2OYwE=; b=iVEAZMalT2KA8Iw/RS4V7YIfLTla74gtpz+wO4SD3NfhfyQYwRXqrUiNo0f6iWDRm7 Ea8hxBOB0fQH+AvjpHaPMLyFlTyVAUgvtpkTE8yst6ygmxySTtUphym6nuiSL55ux2B7 FD+3cb9HczZ21T9oprCXsFn71eYj9i8VIHewuHD3o8JqUica3Y+lUOOMjrTawaRH+Gx5 aCSLOBprXLwDLBs3CIDoaNph8mQ55tSXAnzjvnwVnlBt7RgK4Sx+TWuXRIadzPSxIMgN FmXGk+9qjYWR3Yk1fS7rkoqlkJdElzHh7ATRC08SiDZLsaHbnlEWWjea6bDRbRqZH5Bu V6Ww== X-Gm-Message-State: AOAM531F7Pda/I5tifsawt8F7/8z9z061IC6vikdP13Hok2r2O/NnGhW VdUk94otcHMFD2llDOqPqvEoTg== X-Google-Smtp-Source: ABdhPJwvDd42KY2Nvkio7V+okfmxXWskHchXIxnYuyo2Ym04YtyWgEqwcXyMgTcXTJw4GGnt5sGxUg== X-Received: by 2002:a17:907:b08:: with SMTP id h8mr19020552ejl.26.1630262207411; Sun, 29 Aug 2021 11:36:47 -0700 (PDT) Received: from localhost ([154.21.15.43]) by smtp.gmail.com with ESMTPSA id v12sm6480433ede.16.2021.08.29.11.36.46 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 29 Aug 2021 11:36:47 -0700 (PDT) From: Dmitrii Banshchikov To: bpf@vger.kernel.org Cc: Dmitrii Banshchikov , ast@kernel.org, davem@davemloft.net, daniel@iogearbox.net, andrii@kernel.org, kafai@fb.com, songliubraving@fb.com, yhs@fb.com, john.fastabend@gmail.com, kpsingh@kernel.org, netdev@vger.kernel.org, rdna@fb.com Subject: [PATCH bpf-next v2 11/13] bpfilter: Add handling of setsockopt() calls Date: Sun, 29 Aug 2021 22:36:06 +0400 Message-Id: <20210829183608.2297877-12-me@ubique.spb.ru> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210829183608.2297877-1-me@ubique.spb.ru> References: <20210829183608.2297877-1-me@ubique.spb.ru> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org Add support of iptables' setsockopt(2). The parameters of a setsockopt(2) call are passed by struct mbox_request which contains a type of the setsockopt(2) call and its memory buffer description. The supplied memory buffer is read-written by process_vm_readv(2)/process_vm_writev(2). Signed-off-by: Dmitrii Banshchikov --- net/bpfilter/Makefile | 1 + net/bpfilter/sockopt.c | 441 +++++++++++++++++++++++++++++++++++++++++ net/bpfilter/sockopt.h | 14 ++ 3 files changed, 456 insertions(+) create mode 100644 net/bpfilter/sockopt.c create mode 100644 net/bpfilter/sockopt.h diff --git a/net/bpfilter/Makefile b/net/bpfilter/Makefile index cc4a16fbca04..225a8107e372 100644 --- a/net/bpfilter/Makefile +++ b/net/bpfilter/Makefile @@ -12,6 +12,7 @@ $(LIBBPF_A): userprogs := bpfilter_umh bpfilter_umh-objs := main.o map-common.o codegen.o context.o match.o target.o rule.o table.o +bpfilter_umh-objs += sockopt.o bpfilter_umh-objs += xt_udp.o bpfilter_umh-userldlibs := $(LIBBPF_A) -lelf -lz userccflags += -I $(srctree)/tools/include/ -I $(srctree)/tools/include/uapi -I $(srctree)/tools/lib diff --git a/net/bpfilter/sockopt.c b/net/bpfilter/sockopt.c new file mode 100644 index 000000000000..12786c162f30 --- /dev/null +++ b/net/bpfilter/sockopt.c @@ -0,0 +1,441 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * Copyright (c) 2021 Telegram FZ-LLC + */ + +#define _GNU_SOURCE + +#include "sockopt.h" + +#include +#include + +#include +#include +#include +#include +#include + +#include "context.h" +#include "map-common.h" +#include "match.h" +#include "msgfmt.h" + +static int pvm_read(pid_t pid, void *to, const void *from, size_t count) +{ + const struct iovec l_iov = { .iov_base = to, .iov_len = count }; + const struct iovec r_iov = { .iov_base = (void *)from, .iov_len = count }; + ssize_t total_bytes; + + total_bytes = process_vm_readv(pid, &l_iov, 1, &r_iov, 1, 0); + if (total_bytes == -1) + return -errno; + + if (total_bytes != count) + return -EFAULT; + + return 0; +} + +static int pvm_read_from_offset(pid_t pid, void *to, const void *from, size_t offset, size_t count) +{ + return pvm_read(pid, to + offset, from + offset, count); +} + +static int pvm_write(pid_t pid, void *to, const void *from, size_t count) +{ + const struct iovec l_iov = { .iov_base = (void *)from, .iov_len = count }; + const struct iovec r_iov = { .iov_base = to, .iov_len = count }; + ssize_t total_bytes; + + total_bytes = process_vm_writev(pid, &l_iov, 1, &r_iov, 1, 0); + if (total_bytes == -1) + return -errno; + + if (total_bytes != count) + return -EFAULT; + + return 0; +} + +static int read_ipt_get_info(const struct mbox_request *req, struct bpfilter_ipt_get_info *info) +{ + int err; + + if (req->len != sizeof(*info)) + return -EINVAL; + + err = pvm_read(req->pid, info, (const void *)req->addr, sizeof(*info)); + if (err) + return err; + + info->name[sizeof(info->name) - 1] = '\0'; + + return 0; +} + +static int sockopt_get_info(struct context *ctx, const struct mbox_request *req) +{ + struct bpfilter_ipt_get_info info; + struct table *table; + int err; + + BFLOG_DEBUG(ctx, "handling IPT_SO_GET_INFO\n"); + + if (req->len != sizeof(info)) + return -EINVAL; + + err = read_ipt_get_info(req, &info); + if (err) { + BFLOG_DEBUG(ctx, "cannot read ipt_get_info: %s\n", strerror(-err)); + return err; + } + + table = map_find(&ctx->table_index.map, info.name); + if (IS_ERR(table)) { + BFLOG_DEBUG(ctx, "cannot find table: '%s'\n", info.name); + return -ENOENT; + } + + table_get_info(table, &info); + + return pvm_write(req->pid, (void *)req->addr, &info, sizeof(info)); +} + +static int read_ipt_get_entries(const struct mbox_request *req, + struct bpfilter_ipt_get_entries *entries) +{ + int err; + + if (req->len < sizeof(*entries)) + return -EINVAL; + + err = pvm_read(req->pid, entries, (const void *)req->addr, sizeof(*entries)); + if (err) + return err; + + entries->name[sizeof(entries->name) - 1] = '\0'; + + return 0; +} + +static int sockopt_get_entries(struct context *ctx, const struct mbox_request *req) +{ + struct bpfilter_ipt_get_entries get_entries, *entries; + struct table *table; + int err; + + BFLOG_DEBUG(ctx, "handling IPT_SO_GET_ENTRIES\n"); + + err = read_ipt_get_entries(req, &get_entries); + if (err) { + BFLOG_DEBUG(ctx, "cannot read ipt_get_entries: %s\n", strerror(-err)); + return err; + } + + table = map_find(&ctx->table_index.map, get_entries.name); + if (IS_ERR(table)) { + BFLOG_DEBUG(ctx, "cannot find table: '%s'\n", get_entries.name); + return -ENOENT; + } + + if (get_entries.size != table->size) { + BFLOG_DEBUG(ctx, "table '%s' get entries size mismatch\n", get_entries.name); + return -EINVAL; + } + + entries = (struct bpfilter_ipt_get_entries *)req->addr; + + err = pvm_write(req->pid, entries->name, table->table_ops->name, sizeof(entries->name)); + if (err) + return err; + + err = pvm_write(req->pid, &entries->size, &table->size, sizeof(table->size)); + if (err) + return err; + + return pvm_write(req->pid, entries->entries, table->entries, table->size); +} + +static int read_ipt_get_revision(const struct mbox_request *req, + struct bpfilter_ipt_get_revision *revision) +{ + int err; + + if (req->len != sizeof(*revision)) + return -EINVAL; + + err = pvm_read(req->pid, revision, (const void *)req->addr, sizeof(*revision)); + if (err) + return err; + + revision->name[sizeof(revision->name) - 1] = '\0'; + + return 0; +} + +static int sockopt_get_revision_match(struct context *ctx, const struct mbox_request *req) +{ + struct bpfilter_ipt_get_revision get_revision; + const struct match_ops *found; + int err; + + BFLOG_DEBUG(ctx, "handling IPT_SO_GET_REVISION_MATCH\n"); + + err = read_ipt_get_revision(req, &get_revision); + if (err) + return err; + + found = map_find(&ctx->match_ops_map, get_revision.name); + if (IS_ERR(found)) { + BFLOG_DEBUG(ctx, "cannot find match: '%s'\n", get_revision.name); + return PTR_ERR(found); + } + + return found->revision; +} + +static int sockopt_get_revision_target(struct context *ctx, const struct mbox_request *req) +{ + struct bpfilter_ipt_get_revision get_revision; + const struct match_ops *found; + int err; + + BFLOG_DEBUG(ctx, "handling IPT_SO_GET_REVISION_TARGET\n"); + + err = read_ipt_get_revision(req, &get_revision); + if (err) + return err; + + found = map_find(&ctx->target_ops_map, get_revision.name); + if (IS_ERR(found)) { + BFLOG_DEBUG(ctx, "cannot find target: '%s'\n", get_revision.name); + return PTR_ERR(found); + } + + return found->revision; +} + +static struct bpfilter_ipt_replace *read_ipt_replace(const struct mbox_request *req) +{ + struct bpfilter_ipt_replace ipt_header, *ipt_replace; + int err; + + if (req->len < sizeof(ipt_header)) + return ERR_PTR(-EINVAL); + + err = pvm_read(req->pid, &ipt_header, (const void *)req->addr, sizeof(ipt_header)); + if (err) + return ERR_PTR(err); + + if (ipt_header.num_counters == 0) + return ERR_PTR(-EINVAL); + + if (ipt_header.num_counters >= INT_MAX / sizeof(struct bpfilter_ipt_counters)) + return ERR_PTR(-ENOMEM); + + ipt_header.name[sizeof(ipt_header.name) - 1] = '\0'; + + ipt_replace = malloc(sizeof(ipt_header) + ipt_header.size); + if (!ipt_replace) + return ERR_PTR(-ENOMEM); + + memcpy(ipt_replace, &ipt_header, sizeof(ipt_header)); + + err = pvm_read_from_offset(req->pid, ipt_replace, (const void *)req->addr, + sizeof(ipt_header), ipt_header.size); + if (err) { + free(ipt_replace); + return ERR_PTR(err); + } + + return ipt_replace; +} + +static int sockopt_set_replace(struct context *ctx, const struct mbox_request *req) +{ + struct bpfilter_ipt_replace *ipt_replace; + struct table *table, *new_table = NULL; + struct table_ops *table_ops; + int err; + + BFLOG_DEBUG(ctx, "handling IPT_SO_SET_REPLACE\n"); + + ipt_replace = read_ipt_replace(req); + if (IS_ERR(ipt_replace)) { + BFLOG_DEBUG(ctx, "cannot read ipt_replace: %s\n", strerror(-PTR_ERR(ipt_replace))); + return PTR_ERR(ipt_replace); + } + + table_ops = map_find(&ctx->table_ops_map, ipt_replace->name); + if (IS_ERR(table_ops)) { + err = PTR_ERR(table_ops); + BFLOG_DEBUG(ctx, "cannot find table_ops: '%s'\n", ipt_replace->name); + goto cleanup; + } + + new_table = table_ops->create(ctx, ipt_replace); + if (IS_ERR(new_table)) { + err = PTR_ERR(table_ops); + BFLOG_DEBUG(ctx, "cannot create table: '%s'\n", ipt_replace->name); + goto cleanup; + } + + err = new_table->table_ops->codegen(ctx, new_table); + if (err) { + BFLOG_DEBUG(ctx, "cannot codegen table: '%s'\n", ipt_replace->name); + goto cleanup; + } + + table = map_find(&ctx->table_index.map, ipt_replace->name); + if (IS_ERR(table) && PTR_ERR(table) == -ENOENT) + table = NULL; + + if (IS_ERR(table)) { + err = PTR_ERR(table); + BFLOG_DEBUG(ctx, "cannot find table: '%s'\n", ipt_replace->name); + goto cleanup; + } + + if (table) + table->table_ops->uninstall(ctx, table); + + err = new_table->table_ops->install(ctx, new_table); + if (err) { + BFLOG_DEBUG(ctx, "cannot install new table '%s': %s\n", ipt_replace->name, + strerror(-err)); + if (table) { + int err2 = table->table_ops->install(ctx, table); + + if (err2) + BFLOG_EMERG(ctx, "Cannot install old table '%s': %s", + table->table_ops->name, strerror(-err2)); + } + + goto cleanup; + } + + err = map_upsert(&ctx->table_index.map, new_table->table_ops->name, new_table); + if (err) { + BFLOG_DEBUG(ctx, "cannot update table map: %s\n", strerror(-err)); + goto cleanup; + } + + list_add_tail(&new_table->list, &ctx->table_index.list); + + new_table = table; + +cleanup: + if (!IS_ERR_OR_NULL(new_table)) + new_table->table_ops->free(new_table); + + free(ipt_replace); + + return err; +} + +static struct bpfilter_ipt_counters_info *read_ipt_counters_info(const struct mbox_request *req) +{ + struct bpfilter_ipt_counters_info *info; + size_t size; + int err; + + if (req->len < sizeof(*info)) + return ERR_PTR(-EINVAL); + + info = malloc(req->len); + if (!info) + return ERR_PTR(-ENOMEM); + + err = pvm_read(req->pid, info, (const void *)req->addr, sizeof(*info)); + if (err) + goto err_free; + + size = info->num_counters * sizeof(info->counters[0]); + if (req->len != sizeof(*info) + size) { + err = -EINVAL; + goto err_free; + } + + info->name[sizeof(info->name) - 1] = '\0'; + + err = pvm_read_from_offset(req->pid, info, (const void *)req->addr, sizeof(*info), size); + if (err) + goto err_free; + + return info; + +err_free: + free(info); + + return ERR_PTR(err); +} + +static int sockopt_set_add_counters(struct context *ctx, const struct mbox_request *req) +{ + struct bpfilter_ipt_counters_info *info; + struct table *table; + int err = 0; + + BFLOG_DEBUG(ctx, "handling IPT_SO_SET_ADD_COUNTERS\n"); + + info = read_ipt_counters_info(req); + if (IS_ERR(info)) { + err = PTR_ERR(info); + BFLOG_DEBUG(ctx, "cannot read ipt_counters_info: %s\n", strerror(-err)); + goto err_free; + } + + table = map_find(&ctx->table_index.map, info->name); + if (IS_ERR(table)) { + err = PTR_ERR(table); + BFLOG_DEBUG(ctx, "cannot find table: '%s'\n", info->name); + goto err_free; + } + + // TODO handle counters + +err_free: + free(info); + + return err; +} + +static int handle_get_request(struct context *ctx, const struct mbox_request *req) +{ + switch (req->cmd) { + case 0: + return 0; + case BPFILTER_IPT_SO_GET_INFO: + return sockopt_get_info(ctx, req); + case BPFILTER_IPT_SO_GET_ENTRIES: + return sockopt_get_entries(ctx, req); + case BPFILTER_IPT_SO_GET_REVISION_MATCH: + return sockopt_get_revision_match(ctx, req); + case BPFILTER_IPT_SO_GET_REVISION_TARGET: + return sockopt_get_revision_target(ctx, req); + } + + BFLOG_NOTICE(ctx, "Unexpected SO_GET request: %d\n", req->cmd); + + return -ENOPROTOOPT; +} + +static int handle_set_request(struct context *ctx, const struct mbox_request *req) +{ + switch (req->cmd) { + case BPFILTER_IPT_SO_SET_REPLACE: + return sockopt_set_replace(ctx, req); + case BPFILTER_IPT_SO_SET_ADD_COUNTERS: + return sockopt_set_add_counters(ctx, req); + } + + BFLOG_NOTICE(ctx, "Unexpected SO_SET request: %d\n", req->cmd); + + return -ENOPROTOOPT; +} + +int handle_sockopt_request(struct context *ctx, const struct mbox_request *req) +{ + return req->is_set ? handle_set_request(ctx, req) : handle_get_request(ctx, req); +} diff --git a/net/bpfilter/sockopt.h b/net/bpfilter/sockopt.h new file mode 100644 index 000000000000..711c2f295d89 --- /dev/null +++ b/net/bpfilter/sockopt.h @@ -0,0 +1,14 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* + * Copyright (c) 2021 Telegram FZ-LLC + */ + +#ifndef NET_BPFILTER_SOCKOPT_H +#define NET_BPFILTER_SOCKOPT_H + +struct context; +struct mbox_request; + +int handle_sockopt_request(struct context *ctx, const struct mbox_request *req); + +#endif // NET_BPFILTER_SOCKOPT_H From patchwork Sun Aug 29 18:36:08 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dmitrii Banshchikov X-Patchwork-Id: 504222 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER, INCLUDES_PATCH, MAILING_LIST_MULTI, SPF_HELO_NONE, SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 47385C4320A for ; Sun, 29 Aug 2021 18:37:06 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 31F3A60ED8 for ; Sun, 29 Aug 2021 18:37:06 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S236040AbhH2Sh5 (ORCPT ); Sun, 29 Aug 2021 14:37:57 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:34252 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S236011AbhH2Shp (ORCPT ); Sun, 29 Aug 2021 14:37:45 -0400 Received: from mail-ej1-x635.google.com (mail-ej1-x635.google.com [IPv6:2a00:1450:4864:20::635]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B8D03C0613D9 for ; Sun, 29 Aug 2021 11:36:52 -0700 (PDT) Received: by mail-ej1-x635.google.com with SMTP id x11so26357027ejv.0 for ; Sun, 29 Aug 2021 11:36:52 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ubique-spb-ru.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=zlHvhtHeko9hSmaAAfesUNj7/A7YCvEoJ39Eryv7jaU=; b=UTNeA2Vgku4Jwk0ZYLxOhqoCibaeKLC0eDXJDDmzHfxox2bVTlDwyUhGy7GK17IhAC 6zQnZylk+5utdWYcBHMJ89yDE6EiWUNPe6H15dl3Z6NvvLQExo105LyXBZQKIX49wciV bbbryjDzqbp23ZfGVGJkmOENXWjOFA/ET5dmSff/zaMlUJXrSli//KkRxj4aHTpQDd2q pXMzqdjdIOwXLpotgehOHXwwFq98W4kc6Qj2y+bdp+SBuNHHZdvhL3DhJIpvrWwSXFsi EEYBGp4x+0EILKEQwN0R/BE7ZBMoaQjd42GlZIMw3n4q65/HM5Qb4nQpXBpfL/Zex1lR uItA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=zlHvhtHeko9hSmaAAfesUNj7/A7YCvEoJ39Eryv7jaU=; b=GtY24foYvSuBlGP38bZQDLvZ9FA8ZXLVKwqbdOJdRG6p4c9fH1lqh5u9RrQUl+g/CF mZjb4o+PcttHTs0ivj2TPuLmzYTtPfz/zRjORjLDvV+8q4sKvcvJgwv01u5yNIPTLpe5 8S57WyOab5pD+B88IsS8sRE7Twu5loVN1k6/29HrzKgPRa39v26glg++Nq+nGsikgRTg I/+ExWFmlooxps3+BORrLKqbxRedDWPQPY+5bYu9xn3jbk2J4thGyHPB8EdPbwtI9JsB gVsUMaOpPRUDhTVJyat7IkCi7mrENeLeWHuwFBhLhpZm+QNjfEfpy5Z6TxKbHs3SaRAr OHPA== X-Gm-Message-State: AOAM5309jQ5ggmt6cOXxYowgCUuTvAJA46A9E9oaexKiFLXStYW7/DIp b8rX7PXljimMxJseLM4ChurlMQ== X-Google-Smtp-Source: ABdhPJwn+KMqYF+RIIRpSXT1nQz5hcKIizFVfB0g41Fv5hVL0/Aqz9/o0FtovDY14VSHS5tm3ReVew== X-Received: by 2002:a17:906:138d:: with SMTP id f13mr3212896ejc.180.1630262211305; Sun, 29 Aug 2021 11:36:51 -0700 (PDT) Received: from localhost ([154.21.15.43]) by smtp.gmail.com with ESMTPSA id p5sm5639946eju.30.2021.08.29.11.36.50 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 29 Aug 2021 11:36:51 -0700 (PDT) From: Dmitrii Banshchikov To: bpf@vger.kernel.org Cc: Dmitrii Banshchikov , ast@kernel.org, davem@davemloft.net, daniel@iogearbox.net, andrii@kernel.org, kafai@fb.com, songliubraving@fb.com, yhs@fb.com, john.fastabend@gmail.com, kpsingh@kernel.org, netdev@vger.kernel.org, rdna@fb.com Subject: [PATCH bpf-next v2 13/13] bpfilter: Handle setsockopts Date: Sun, 29 Aug 2021 22:36:08 +0400 Message-Id: <20210829183608.2297877-14-me@ubique.spb.ru> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210829183608.2297877-1-me@ubique.spb.ru> References: <20210829183608.2297877-1-me@ubique.spb.ru> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org Use earlier introduced infrastructure and handle setsockopt(2) calls. Signed-off-by: Dmitrii Banshchikov --- net/bpfilter/main.c | 126 +++++++++++++++++++++++++++++--------------- 1 file changed, 84 insertions(+), 42 deletions(-) diff --git a/net/bpfilter/main.c b/net/bpfilter/main.c index 291a92546246..1010e4c49716 100644 --- a/net/bpfilter/main.c +++ b/net/bpfilter/main.c @@ -1,64 +1,106 @@ // SPDX-License-Identifier: GPL-2.0 +/* + * Copyright (c) 2021 Telegram FZ-LLC + */ + #define _GNU_SOURCE -#include + +#include +#include + #include #include -#include -#include +#include #include -#include "../../include/uapi/linux/bpf.h" -#include + +#include "context.h" +#include "filter-table.h" #include "msgfmt.h" +#include "sockopt.h" -FILE *debug_f; +#define do_exact(fd, op, buffer, count) \ + ({ \ + size_t total = 0; \ + int err = 0; \ + \ + do { \ + const ssize_t part = op(fd, (buffer) + total, (count) - total); \ + if (part > 0) { \ + total += part; \ + } else if (part == 0 && (count) > 0) { \ + err = -EIO; \ + break; \ + } else if (part == -1) { \ + if (errno == EINTR) \ + continue; \ + err = -errno; \ + break; \ + } \ + } while (total < (count)); \ + \ + err; \ + }) -static int handle_get_cmd(struct mbox_request *cmd) +static int read_exact(int fd, void *buffer, size_t count) { - switch (cmd->cmd) { - case 0: - return 0; - default: - break; - } - return -ENOPROTOOPT; + return do_exact(fd, read, buffer, count); } -static int handle_set_cmd(struct mbox_request *cmd) +static int write_exact(int fd, const void *buffer, size_t count) { - return -ENOPROTOOPT; + return do_exact(fd, write, buffer, count); } -static void loop(void) +static int setup_context(struct context *ctx) { - while (1) { - struct mbox_request req; - struct mbox_reply reply; - int n; - - n = read(0, &req, sizeof(req)); - if (n != sizeof(req)) { - fprintf(debug_f, "invalid request %d\n", n); - return; - } - - reply.status = req.is_set ? - handle_set_cmd(&req) : - handle_get_cmd(&req); - - n = write(1, &reply, sizeof(reply)); - if (n != sizeof(reply)) { - fprintf(debug_f, "reply failed %d\n", n); - return; - } + ctx->log_file = fopen("/dev/kmsg", "w"); + if (!ctx->log_file) + return -errno; + + errno = 0; + if (setvbuf(ctx->log_file, 0, _IOLBF, 0)) + return errno ? -errno : -EINVAL; + + return create_filter_table(ctx); +} + +static void loop(struct context *ctx) +{ + struct mbox_request req; + struct mbox_reply reply; + int err; + + for (;;) { + err = read_exact(STDIN_FILENO, &req, sizeof(req)); + if (err) + BFLOG_EMERG(ctx, "cannot read request: %s\n", strerror(-err)); + + reply.status = handle_sockopt_request(ctx, &req); + + err = write_exact(STDOUT_FILENO, &reply, sizeof(reply)); + if (err) + BFLOG_EMERG(ctx, "cannot write reply: %s\n", strerror(-err)); } } int main(void) { - debug_f = fopen("/dev/kmsg", "w"); - setvbuf(debug_f, 0, _IOLBF, 0); - fprintf(debug_f, "<5>Started bpfilter\n"); - loop(); - fclose(debug_f); + struct context ctx; + int err; + + err = create_context(&ctx); + if (err) + return err; + + err = setup_context(&ctx); + if (err) { + free_context(&ctx); + return err; + } + + BFLOG_NOTICE(&ctx, "started\n"); + + loop(&ctx); + return 0; }