From patchwork Sun Mar 29 22:53:38 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Joe Stringer X-Patchwork-Id: 221623 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-9.6 required=3.0 tests=DKIM_INVALID,DKIM_SIGNED, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY, SPF_HELO_NONE, SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 875F0C43331 for ; Sun, 29 Mar 2020 22:53:52 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 4581C20748 for ; Sun, 29 Mar 2020 22:53:52 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="mHVmaj++" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729290AbgC2Wxv (ORCPT ); Sun, 29 Mar 2020 18:53:51 -0400 Received: from mail-pj1-f67.google.com ([209.85.216.67]:37961 "EHLO mail-pj1-f67.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727591AbgC2Wxu (ORCPT ); Sun, 29 Mar 2020 18:53:50 -0400 Received: by mail-pj1-f67.google.com with SMTP id m15so6507795pje.3; Sun, 29 Mar 2020 15:53:50 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=7Q9we1QvB6zj/YAzkcMxjsbghvTQXpHH+PS811R2LAY=; b=mHVmaj++gXFIcwTWZwRVJscesHXBCo3LQ+M+UJ8yvab5MFAHShH5x3rj9NZ/8JnDyM AYCj9LI/62D8/9oIvUuyup5Ih4qrMLwzT47vzBKeLChk8Ul8m0kl1hS/ajf3om8+iWmc NUHlb5E0VXTx95thWFOYalzRqyOAr+tsHGUkUKzawC2zGB7xx7LWi/6duC+zf0esuDFQ BuKdXds+5OT/4hScIERhDZH6273U+Nx1nac5nRynch+r3WzzkPap17XwW3fVZ5Jzi0YI VSivM9b46h8aRrdzPTW9XIvMKjZBIoNYWZGuJZ35vl1f7BftKuYcPgkth7y0Y9Qo+PN1 JDHA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:from:to:cc:subject:date:message-id :in-reply-to:references:mime-version:content-transfer-encoding; bh=7Q9we1QvB6zj/YAzkcMxjsbghvTQXpHH+PS811R2LAY=; b=cUlkxzQrLdTg7qCKK88iIw+K+UJiDJTwKyYRAk4pvMvrojwgbjelJmQiesxwGOP/as XV3R0zyNYtZjQKKAWQMeHZwK2s1PZ/X0YEcjp/PQyjkwBHOoV6pVRFiCaY/Wkjj9eEXX I8pyKwSBmrspAXD47WDnlMNNkUZVTRtO2sMCf/Zi0LNx2haPtqFAvD0n5QU2Xcl8ADbF Y+KYn+yQyQPW+OCvS4f0z0qVYSjYLYCvLuLUYX1sJ58RbXHkWF8GqiL/Z2sbMNJz8aoV cGXPk0+lM4OU+/Rn9c71RKP46c72zCKJOKkbFMmuMBbjEokNEodVNap34csxWpK/MPDP SiLw== X-Gm-Message-State: ANhLgQ3Zxce+j2JyRdJ8D5kQX0jI8ZRGrMs5GoEqf1O5e5s/PzcrSY6e bFKVhAUx0nM9Txj59GMHK/TkdyFo X-Google-Smtp-Source: ADFU+vtPSNYivwpoeEhjRald7FBx0SKge4U6AN+eFbiNKPza3MiOmRMXbEOEzi3NZQKzHLhcKq0gUA== X-Received: by 2002:a17:90a:c401:: with SMTP id i1mr12324542pjt.131.1585522429134; Sun, 29 Mar 2020 15:53:49 -0700 (PDT) Received: from localhost.localdomain (c-73-93-5-123.hsd1.ca.comcast.net. [73.93.5.123]) by smtp.gmail.com with ESMTPSA id i187sm8710386pfg.33.2020.03.29.15.53.47 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 29 Mar 2020 15:53:48 -0700 (PDT) From: Joe Stringer To: bpf@vger.kernel.org Cc: netdev@vger.kernel.org, daniel@iogearbox.net, ast@kernel.org, eric.dumazet@gmail.com, lmb@cloudflare.com, kafai@fb.com Subject: [PATCHv5 bpf-next 1/5] bpf: Add socket assign support Date: Sun, 29 Mar 2020 15:53:38 -0700 Message-Id: <20200329225342.16317-2-joe@wand.net.nz> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20200329225342.16317-1-joe@wand.net.nz> References: <20200329225342.16317-1-joe@wand.net.nz> MIME-Version: 1.0 Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org Add support for TPROXY via a new bpf helper, bpf_sk_assign(). This helper requires the BPF program to discover the socket via a call to bpf_sk*_lookup_*(), then pass this socket to the new helper. The helper takes its own reference to the socket in addition to any existing reference that may or may not currently be obtained for the duration of BPF processing. For the destination socket to receive the traffic, the traffic must be routed towards that socket via local route. The simplest example route is below, but in practice you may want to route traffic more narrowly (eg by CIDR): $ ip route add local default dev lo This patch avoids trying to introduce an extra bit into the skb->sk, as that would require more invasive changes to all code interacting with the socket to ensure that the bit is handled correctly, such as all error-handling cases along the path from the helper in BPF through to the orphan path in the input. Instead, we opt to use the destructor variable to switch on the prefetch of the socket. Signed-off-by: Joe Stringer Acked-by: Martin KaFai Lau --- v5: Guard sock_gen_put() with #ifdef for CONFIG_INET v4: Use sock_gen_put() directly in sock_pfree() Rebase Acked v3: Check skb_sk_is_prefetched() in TC level redirect check v2: Use skb->destructor to determine socket prefetch usage instead of introducing a new metadata_dst Restrict socket assign to same netns as TC device Restrict assigning reuseport sockets Adjust commit wording v1: Initial version --- include/net/sock.h | 11 +++++++++++ include/uapi/linux/bpf.h | 25 ++++++++++++++++++++++++- net/core/filter.c | 31 +++++++++++++++++++++++++++++++ net/core/sock.c | 11 +++++++++++ net/ipv4/ip_input.c | 3 ++- net/ipv6/ip6_input.c | 3 ++- net/sched/act_bpf.c | 3 +++ tools/include/uapi/linux/bpf.h | 25 ++++++++++++++++++++++++- 8 files changed, 108 insertions(+), 4 deletions(-) diff --git a/include/net/sock.h b/include/net/sock.h index b5cca7bae69b..dc398cee7873 100644 --- a/include/net/sock.h +++ b/include/net/sock.h @@ -1659,6 +1659,7 @@ void sock_rfree(struct sk_buff *skb); void sock_efree(struct sk_buff *skb); #ifdef CONFIG_INET void sock_edemux(struct sk_buff *skb); +void sock_pfree(struct sk_buff *skb); #else #define sock_edemux sock_efree #endif @@ -2526,6 +2527,16 @@ void sock_net_set(struct sock *sk, struct net *net) write_pnet(&sk->sk_net, net); } +static inline bool +skb_sk_is_prefetched(struct sk_buff *skb) +{ +#ifdef CONFIG_INET + return skb->destructor == sock_pfree; +#else + return false; +#endif /* CONFIG_INET */ +} + static inline struct sock *skb_steal_sock(struct sk_buff *skb) { if (skb->sk) { diff --git a/include/uapi/linux/bpf.h b/include/uapi/linux/bpf.h index 222ba11966e3..2ae402fe272a 100644 --- a/include/uapi/linux/bpf.h +++ b/include/uapi/linux/bpf.h @@ -2981,6 +2981,28 @@ union bpf_attr { * **bpf_get_current_cgroup_id**\ (). * Return * The id is returned or 0 in case the id could not be retrieved. + * + * int bpf_sk_assign(struct sk_buff *skb, struct bpf_sock *sk, u64 flags) + * Description + * Assign the *sk* to the *skb*. When combined with appropriate + * routing configuration to receive the packet towards the socket, + * will cause *skb* to be delivered to the specified socket. + * Subsequent redirection of *skb* via **bpf_redirect**\ (), + * **bpf_clone_redirect**\ () or other methods outside of BPF may + * interfere with successful delivery to the socket. + * + * This operation is only valid from TC ingress path. + * + * The *flags* argument must be zero. + * Return + * 0 on success, or a negative errno in case of failure. + * + * * **-EINVAL** Unsupported flags specified. + * * **-ENOENT** Socket is unavailable for assignment. + * * **-ENETUNREACH** Socket is unreachable (wrong netns). + * * **-EOPNOTSUPP** Unsupported operation, for example a + * call from outside of TC ingress. + * * **-ESOCKTNOSUPPORT** Socket type not supported (reuseport). */ #define __BPF_FUNC_MAPPER(FN) \ FN(unspec), \ @@ -3106,7 +3128,8 @@ union bpf_attr { FN(get_ns_current_pid_tgid), \ FN(xdp_output), \ FN(get_netns_cookie), \ - FN(get_current_ancestor_cgroup_id), + FN(get_current_ancestor_cgroup_id), \ + FN(sk_assign), /* integer value in 'imm' field of BPF_CALL instruction selects which helper * function eBPF program intends to call diff --git a/net/core/filter.c b/net/core/filter.c index bb4a196c8809..ac5c1633f8d2 100644 --- a/net/core/filter.c +++ b/net/core/filter.c @@ -5918,6 +5918,35 @@ static const struct bpf_func_proto bpf_tcp_gen_syncookie_proto = { .arg5_type = ARG_CONST_SIZE, }; +BPF_CALL_3(bpf_sk_assign, struct sk_buff *, skb, struct sock *, sk, u64, flags) +{ + if (flags != 0) + return -EINVAL; + if (!skb_at_tc_ingress(skb)) + return -EOPNOTSUPP; + if (unlikely(dev_net(skb->dev) != sock_net(sk))) + return -ENETUNREACH; + if (unlikely(sk->sk_reuseport)) + return -ESOCKTNOSUPPORT; + if (unlikely(!refcount_inc_not_zero(&sk->sk_refcnt))) + return -ENOENT; + + skb_orphan(skb); + skb->sk = sk; + skb->destructor = sock_pfree; + + return 0; +} + +static const struct bpf_func_proto bpf_sk_assign_proto = { + .func = bpf_sk_assign, + .gpl_only = false, + .ret_type = RET_INTEGER, + .arg1_type = ARG_PTR_TO_CTX, + .arg2_type = ARG_PTR_TO_SOCK_COMMON, + .arg3_type = ARG_ANYTHING, +}; + #endif /* CONFIG_INET */ bool bpf_helper_changes_pkt_data(void *func) @@ -6249,6 +6278,8 @@ tc_cls_act_func_proto(enum bpf_func_id func_id, const struct bpf_prog *prog) return &bpf_skb_ecn_set_ce_proto; case BPF_FUNC_tcp_gen_syncookie: return &bpf_tcp_gen_syncookie_proto; + case BPF_FUNC_sk_assign: + return &bpf_sk_assign_proto; #endif default: return bpf_base_func_proto(func_id); diff --git a/net/core/sock.c b/net/core/sock.c index 0fc8937a7ff4..87e3a03c9056 100644 --- a/net/core/sock.c +++ b/net/core/sock.c @@ -2071,6 +2071,17 @@ void sock_efree(struct sk_buff *skb) } EXPORT_SYMBOL(sock_efree); +/* Buffer destructor for prefetch/receive path where reference count may + * not be held, e.g. for listen sockets. + */ +#ifdef CONFIG_INET +void sock_pfree(struct sk_buff *skb) +{ + sock_gen_put(skb->sk); +} +EXPORT_SYMBOL(sock_pfree); +#endif /* CONFIG_INET */ + kuid_t sock_i_uid(struct sock *sk) { kuid_t uid; diff --git a/net/ipv4/ip_input.c b/net/ipv4/ip_input.c index aa438c6758a7..b0c244af1e4d 100644 --- a/net/ipv4/ip_input.c +++ b/net/ipv4/ip_input.c @@ -509,7 +509,8 @@ static struct sk_buff *ip_rcv_core(struct sk_buff *skb, struct net *net) IPCB(skb)->iif = skb->skb_iif; /* Must drop socket now because of tproxy. */ - skb_orphan(skb); + if (!skb_sk_is_prefetched(skb)) + skb_orphan(skb); return skb; diff --git a/net/ipv6/ip6_input.c b/net/ipv6/ip6_input.c index 7b089d0ac8cd..e96304d8a4a7 100644 --- a/net/ipv6/ip6_input.c +++ b/net/ipv6/ip6_input.c @@ -285,7 +285,8 @@ static struct sk_buff *ip6_rcv_core(struct sk_buff *skb, struct net_device *dev, rcu_read_unlock(); /* Must drop socket now because of tproxy. */ - skb_orphan(skb); + if (!skb_sk_is_prefetched(skb)) + skb_orphan(skb); return skb; err: diff --git a/net/sched/act_bpf.c b/net/sched/act_bpf.c index 46f47e58b3be..54d5652cfe6c 100644 --- a/net/sched/act_bpf.c +++ b/net/sched/act_bpf.c @@ -12,6 +12,7 @@ #include #include +#include #include #include @@ -53,6 +54,8 @@ static int tcf_bpf_act(struct sk_buff *skb, const struct tc_action *act, bpf_compute_data_pointers(skb); filter_res = BPF_PROG_RUN(filter, skb); } + if (skb_sk_is_prefetched(skb) && filter_res != TC_ACT_OK) + skb_orphan(skb); rcu_read_unlock(); /* A BPF program may overwrite the default action opcode. diff --git a/tools/include/uapi/linux/bpf.h b/tools/include/uapi/linux/bpf.h index 222ba11966e3..2ae402fe272a 100644 --- a/tools/include/uapi/linux/bpf.h +++ b/tools/include/uapi/linux/bpf.h @@ -2981,6 +2981,28 @@ union bpf_attr { * **bpf_get_current_cgroup_id**\ (). * Return * The id is returned or 0 in case the id could not be retrieved. + * + * int bpf_sk_assign(struct sk_buff *skb, struct bpf_sock *sk, u64 flags) + * Description + * Assign the *sk* to the *skb*. When combined with appropriate + * routing configuration to receive the packet towards the socket, + * will cause *skb* to be delivered to the specified socket. + * Subsequent redirection of *skb* via **bpf_redirect**\ (), + * **bpf_clone_redirect**\ () or other methods outside of BPF may + * interfere with successful delivery to the socket. + * + * This operation is only valid from TC ingress path. + * + * The *flags* argument must be zero. + * Return + * 0 on success, or a negative errno in case of failure. + * + * * **-EINVAL** Unsupported flags specified. + * * **-ENOENT** Socket is unavailable for assignment. + * * **-ENETUNREACH** Socket is unreachable (wrong netns). + * * **-EOPNOTSUPP** Unsupported operation, for example a + * call from outside of TC ingress. + * * **-ESOCKTNOSUPPORT** Socket type not supported (reuseport). */ #define __BPF_FUNC_MAPPER(FN) \ FN(unspec), \ @@ -3106,7 +3128,8 @@ union bpf_attr { FN(get_ns_current_pid_tgid), \ FN(xdp_output), \ FN(get_netns_cookie), \ - FN(get_current_ancestor_cgroup_id), + FN(get_current_ancestor_cgroup_id), \ + FN(sk_assign), /* integer value in 'imm' field of BPF_CALL instruction selects which helper * function eBPF program intends to call From patchwork Sun Mar 29 22:53:40 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Joe Stringer X-Patchwork-Id: 221622 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-9.6 required=3.0 tests=DKIM_INVALID,DKIM_SIGNED, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY, SPF_HELO_NONE, SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 85F1AC43331 for ; Sun, 29 Mar 2020 22:53:57 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 5972F20748 for ; Sun, 29 Mar 2020 22:53:57 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="e4bMMzJ+" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729305AbgC2Wx4 (ORCPT ); Sun, 29 Mar 2020 18:53:56 -0400 Received: from mail-pj1-f65.google.com ([209.85.216.65]:54778 "EHLO mail-pj1-f65.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727591AbgC2Wxz (ORCPT ); Sun, 29 Mar 2020 18:53:55 -0400 Received: by mail-pj1-f65.google.com with SMTP id np9so6728719pjb.4; Sun, 29 Mar 2020 15:53:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=g6s4kfFeQsm2zTjuoVkgmF3mGhrJ7uY8FHMSRBRtNsA=; b=e4bMMzJ+dvBvnlIYrtr3HiLpftmCdJeT+LVQy6dcgR9HgUXu236Ks/X1L+KgR4M/V4 qNJatBkrZWvAeGipcyfRNE76zaF2/vfcxbjAJ3juwZk/c6Ndps4NuV5gbdjqkD8kVEO6 2mrTfE1xCkh9BzNe/AT/2/L0OZ/EVxJVrPQfLwd6JTVC9jCKnCy7x44Wt89TvdtcyfhJ GEi1H8tJmNfLBsI6qbNwD6i3gxONEdaIV4ZLyg3K7nX6ZbL0Q1lcHgiXGfeCXKrQW4tc mn1mMz686wBVQKQBeIP0S+TQdwUWAcTDLyP7J5dmINYQ0kHfilvXz4CU4zs+yEktItAk VMAQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:from:to:cc:subject:date:message-id :in-reply-to:references:mime-version:content-transfer-encoding; bh=g6s4kfFeQsm2zTjuoVkgmF3mGhrJ7uY8FHMSRBRtNsA=; b=kRb9/m8opJy52N7zzcc/eRKfcLZmzo4XfIwALhy/4OPZ5jDZqsgjK6ptaUZgkb5gwr DK5eCGekXDTSIQU4IhSSMM6Xt1ikjf0kGRwotfROgBt0W6HAMCYIaY1D/kZlLfhEsKIw yiTNv0wWtxHb+iG294eAyPe1MBZSWzN8bd9UpfyUnoqeiO8vQYPwM56MtVMGJohPOsuv Q6zK8DdfD3BWU8X6jCADNFu4tMo+jAn1wDSQuZnfRwnNlMNyNFgXJEqsRAwMHvBk2OEJ hSs28zeEouwstAbPw1adooOKyNW6HHic/PQwPGVSxrbqQqfcQbVt1Zg1llyeVpo0GBjJ vmSA== X-Gm-Message-State: ANhLgQ1IEeZQAr5U20IVqGcLwkDDkt0y7w/Jd/u/3yGSPFzgJwfEXik2 ogzgQi7/H6WX8bA5XOUWxKi3sHKs X-Google-Smtp-Source: ADFU+vto6Sad/DUe2Sh1U/oKdpU4oQp18mim3hqidPeqAt94vil2Tde/9cTvZoh9CZSBcYcV3+tDOQ== X-Received: by 2002:a17:90a:2103:: with SMTP id a3mr12713082pje.181.1585522432426; Sun, 29 Mar 2020 15:53:52 -0700 (PDT) Received: from localhost.localdomain (c-73-93-5-123.hsd1.ca.comcast.net. [73.93.5.123]) by smtp.gmail.com with ESMTPSA id i187sm8710386pfg.33.2020.03.29.15.53.51 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 29 Mar 2020 15:53:51 -0700 (PDT) From: Joe Stringer To: bpf@vger.kernel.org Cc: netdev@vger.kernel.org, daniel@iogearbox.net, ast@kernel.org, eric.dumazet@gmail.com, lmb@cloudflare.com, kafai@fb.com Subject: [PATCHv5 bpf-next 3/5] bpf: Don't refcount LISTEN sockets in sk_assign() Date: Sun, 29 Mar 2020 15:53:40 -0700 Message-Id: <20200329225342.16317-4-joe@wand.net.nz> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20200329225342.16317-1-joe@wand.net.nz> References: <20200329225342.16317-1-joe@wand.net.nz> MIME-Version: 1.0 Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org Avoid taking a reference on listen sockets by checking the socket type in the sk_assign and in the corresponding skb_steal_sock() code in the the transport layer, and by ensuring that the prefetch free (sock_pfree) function uses the same logic to check whether the socket is refcounted. Suggested-by: Martin KaFai Lau Signed-off-by: Joe Stringer Acked-by: Martin KaFai Lau --- v5: Guard sock_gen_put() with #ifdef for CONFIG_INET v4: Directly use sock_gen_put() from sock_pfree() Acked v3: No changes v2: Initial version --- include/net/sock.h | 25 +++++++++++++++++-------- net/core/filter.c | 6 +++--- net/core/sock.c | 3 ++- 3 files changed, 22 insertions(+), 12 deletions(-) diff --git a/include/net/sock.h b/include/net/sock.h index f81d528845f6..6d84784d33fa 100644 --- a/include/net/sock.h +++ b/include/net/sock.h @@ -2537,6 +2537,21 @@ skb_sk_is_prefetched(struct sk_buff *skb) #endif /* CONFIG_INET */ } +/* This helper checks if a socket is a full socket, + * ie _not_ a timewait or request socket. + */ +static inline bool sk_fullsock(const struct sock *sk) +{ + return (1 << sk->sk_state) & ~(TCPF_TIME_WAIT | TCPF_NEW_SYN_RECV); +} + +static inline bool +sk_is_refcounted(struct sock *sk) +{ + /* Only full sockets have sk->sk_flags. */ + return !sk_fullsock(sk) || !sock_flag(sk, SOCK_RCU_FREE); +} + /** * skb_steal_sock * @skb to steal the socket from @@ -2549,6 +2564,8 @@ skb_steal_sock(struct sk_buff *skb, bool *refcounted) struct sock *sk = skb->sk; *refcounted = true; + if (skb_sk_is_prefetched(skb)) + *refcounted = sk_is_refcounted(sk); skb->destructor = NULL; skb->sk = NULL; return sk; @@ -2557,14 +2574,6 @@ skb_steal_sock(struct sk_buff *skb, bool *refcounted) return NULL; } -/* This helper checks if a socket is a full socket, - * ie _not_ a timewait or request socket. - */ -static inline bool sk_fullsock(const struct sock *sk) -{ - return (1 << sk->sk_state) & ~(TCPF_TIME_WAIT | TCPF_NEW_SYN_RECV); -} - /* Checks if this SKB belongs to an HW offloaded socket * and whether any SW fallbacks are required based on dev. * Check decrypted mark in case skb_orphan() cleared socket. diff --git a/net/core/filter.c b/net/core/filter.c index ac5c1633f8d2..7628b947dbc3 100644 --- a/net/core/filter.c +++ b/net/core/filter.c @@ -5401,8 +5401,7 @@ static const struct bpf_func_proto bpf_sk_lookup_udp_proto = { BPF_CALL_1(bpf_sk_release, struct sock *, sk) { - /* Only full sockets have sk->sk_flags. */ - if (!sk_fullsock(sk) || !sock_flag(sk, SOCK_RCU_FREE)) + if (sk_is_refcounted(sk)) sock_gen_put(sk); return 0; } @@ -5928,7 +5927,8 @@ BPF_CALL_3(bpf_sk_assign, struct sk_buff *, skb, struct sock *, sk, u64, flags) return -ENETUNREACH; if (unlikely(sk->sk_reuseport)) return -ESOCKTNOSUPPORT; - if (unlikely(!refcount_inc_not_zero(&sk->sk_refcnt))) + if (sk_is_refcounted(sk) && + unlikely(!refcount_inc_not_zero(&sk->sk_refcnt))) return -ENOENT; skb_orphan(skb); diff --git a/net/core/sock.c b/net/core/sock.c index 87e3a03c9056..da32d9b6d09f 100644 --- a/net/core/sock.c +++ b/net/core/sock.c @@ -2077,7 +2077,8 @@ EXPORT_SYMBOL(sock_efree); #ifdef CONFIG_INET void sock_pfree(struct sk_buff *skb) { - sock_gen_put(skb->sk); + if (sk_is_refcounted(skb->sk)) + sock_gen_put(skb->sk); } EXPORT_SYMBOL(sock_pfree); #endif /* CONFIG_INET */ From patchwork Sun Mar 29 22:53:42 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Joe Stringer X-Patchwork-Id: 221621 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-9.6 required=3.0 tests=DKIM_INVALID,DKIM_SIGNED, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY, SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 96F4CC43331 for ; Sun, 29 Mar 2020 22:54:03 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 6427E20748 for ; Sun, 29 Mar 2020 22:54:03 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="InBocjys" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729310AbgC2WyA (ORCPT ); Sun, 29 Mar 2020 18:54:00 -0400 Received: from mail-pj1-f67.google.com ([209.85.216.67]:36654 "EHLO mail-pj1-f67.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727591AbgC2Wx6 (ORCPT ); Sun, 29 Mar 2020 18:53:58 -0400 Received: by mail-pj1-f67.google.com with SMTP id nu11so6512511pjb.1; Sun, 29 Mar 2020 15:53:56 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=9OfbQfWlVbkzTDO7qVxP147GnOcJkDx6CYQs8tRWEHA=; b=InBocjys5EPbCWuhF4nLLPRlAGQ4i0XyclKh/f5TjqxGO/cU3wXZXSEqQXUIHCfFne LMrythEd5O+nvI3Ae8Z39fyTy5PScapmbk43GCBsMcLWO0NV11o3TJMZZEcnRhWbGNWF tbaV2ClohouN6vgnT6DSWqg0ZmgiWfaFDArc66Zk0PJwtFjyYVrCz3mEPOAr3R80wFK0 vODIWuEkxagtaE6jri2s5ze3QBjDblEJvHU+YvSDRmRQZwheJj3XrGJ1EmDlKtHpwbXi pEEdzma3favJgSC5aMYQrl2fCRmnnCIboIFpRHsbUUFb/7O5wSo76YMcM6YPC7IJ2WzH GOkQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:from:to:cc:subject:date:message-id :in-reply-to:references:mime-version:content-transfer-encoding; bh=9OfbQfWlVbkzTDO7qVxP147GnOcJkDx6CYQs8tRWEHA=; b=uF9LKz3n3VtfLmRMYZKl3hUIZT5IlJBg+ygVPpC1bIHgLL9vK7XihSE9NQVzw4VVUv SSUcsfhtVMBf1bKX4Q2FvA2N1uRkkmjF386UdbgqPwfGjoMVUXZC15SNdRn6o58vRFDd dl6EVYZUs4Xptl9lrWlI/iqnr34zsJ+ULr7z64HohLoB6NjQTowt0rjvTseTLI7grpoM 4jDkpnLkVCXJR6SebU28jNB+wAVPchdAN9hOZu+QLSWY2r/+EKGYTBoNf0oz/CrfKH6x 8PemgvXZwSHopAhEA3TuglXw8LI/hebr0BsIGn3L0cSO30xCjPTCJ5RzkNTGavxS5ZtJ OhLg== X-Gm-Message-State: ANhLgQ0c1IWclf+NaPldkMnWE4sAcegsszG52wgLhmtlnIq9ghz7UjXV wtNbyiBIl9JZQS/L1T5x0iV4zY9M X-Google-Smtp-Source: ADFU+vsAb8FQBBbEqguu4djLUQ+65Y/dQl6/UC2xQrqZNIo5f/GGqwIsM48mgERhdhnFyqmeRE3gcw== X-Received: by 2002:a17:902:598e:: with SMTP id p14mr9305179pli.276.1585522435919; Sun, 29 Mar 2020 15:53:55 -0700 (PDT) Received: from localhost.localdomain (c-73-93-5-123.hsd1.ca.comcast.net. [73.93.5.123]) by smtp.gmail.com with ESMTPSA id i187sm8710386pfg.33.2020.03.29.15.53.54 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 29 Mar 2020 15:53:55 -0700 (PDT) From: Joe Stringer To: bpf@vger.kernel.org Cc: netdev@vger.kernel.org, daniel@iogearbox.net, ast@kernel.org, eric.dumazet@gmail.com, lmb@cloudflare.com, kafai@fb.com Subject: [PATCHv5 bpf-next 5/5] selftests: bpf: Extend sk_assign tests for UDP Date: Sun, 29 Mar 2020 15:53:42 -0700 Message-Id: <20200329225342.16317-6-joe@wand.net.nz> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20200329225342.16317-1-joe@wand.net.nz> References: <20200329225342.16317-1-joe@wand.net.nz> MIME-Version: 1.0 Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org Add support for testing UDP sk_assign to the existing tests. Signed-off-by: Joe Stringer Acked-by: Lorenz Bauer Acked-by: Martin KaFai Lau --- v5: No change v4: Acked v3: Initial post --- .../selftests/bpf/prog_tests/sk_assign.c | 47 +++++++++++-- .../selftests/bpf/progs/test_sk_assign.c | 69 +++++++++++++++++-- 2 files changed, 105 insertions(+), 11 deletions(-) diff --git a/tools/testing/selftests/bpf/prog_tests/sk_assign.c b/tools/testing/selftests/bpf/prog_tests/sk_assign.c index 25f17fe7d678..d572e1a2c297 100644 --- a/tools/testing/selftests/bpf/prog_tests/sk_assign.c +++ b/tools/testing/selftests/bpf/prog_tests/sk_assign.c @@ -69,7 +69,7 @@ start_server(const struct sockaddr *addr, socklen_t len, int type) goto close_out; if (CHECK_FAIL(bind(fd, addr, len) == -1)) goto close_out; - if (CHECK_FAIL(listen(fd, 128) == -1)) + if (type == SOCK_STREAM && CHECK_FAIL(listen(fd, 128) == -1)) goto close_out; goto out; @@ -125,6 +125,20 @@ get_port(int fd) return port; } +static ssize_t +rcv_msg(int srv_client, int type) +{ + struct sockaddr_storage ss; + char buf[BUFSIZ]; + socklen_t slen; + + if (type == SOCK_STREAM) + return read(srv_client, &buf, sizeof(buf)); + else + return recvfrom(srv_client, &buf, sizeof(buf), 0, + (struct sockaddr *)&ss, &slen); +} + static int run_test(int server_fd, const struct sockaddr *addr, socklen_t len, int type) { @@ -139,16 +153,20 @@ run_test(int server_fd, const struct sockaddr *addr, socklen_t len, int type) goto out; } - srv_client = accept(server_fd, NULL, NULL); - if (CHECK_FAIL(srv_client == -1)) { - perror("Can't accept connection"); - goto out; + if (type == SOCK_STREAM) { + srv_client = accept(server_fd, NULL, NULL); + if (CHECK_FAIL(srv_client == -1)) { + perror("Can't accept connection"); + goto out; + } + } else { + srv_client = server_fd; } if (CHECK_FAIL(write(client, buf, sizeof(buf)) != sizeof(buf))) { perror("Can't write on client"); goto out; } - if (CHECK_FAIL(read(srv_client, &buf, sizeof(buf)) != sizeof(buf))) { + if (CHECK_FAIL(rcv_msg(srv_client, type) != sizeof(buf))) { perror("Can't read on server"); goto out; } @@ -156,9 +174,20 @@ run_test(int server_fd, const struct sockaddr *addr, socklen_t len, int type) port = get_port(srv_client); if (CHECK_FAIL(!port)) goto out; - if (CHECK(port != htons(CONNECT_PORT), "Expected", "port %u but got %u", + /* SOCK_STREAM is connected via accept(), so the server's local address + * will be the CONNECT_PORT rather than the BIND port that corresponds + * to the listen socket. SOCK_DGRAM on the other hand is connectionless + * so we can't really do the same check there; the server doesn't ever + * create a socket with CONNECT_PORT. + */ + if (type == SOCK_STREAM && + CHECK(port != htons(CONNECT_PORT), "Expected", "port %u but got %u", CONNECT_PORT, ntohs(port))) goto out; + else if (type == SOCK_DGRAM && + CHECK(port != htons(BIND_PORT), "Expected", + "port %u but got %u", BIND_PORT, ntohs(port))) + goto out; ret = 0; out: @@ -230,6 +259,10 @@ void test_sk_assign(void) TEST("ipv4 tcp addr redir", AF_INET, SOCK_STREAM, true), TEST("ipv6 tcp port redir", AF_INET6, SOCK_STREAM, false), TEST("ipv6 tcp addr redir", AF_INET6, SOCK_STREAM, true), + TEST("ipv4 udp port redir", AF_INET, SOCK_DGRAM, false), + TEST("ipv4 udp addr redir", AF_INET, SOCK_DGRAM, true), + TEST("ipv6 udp port redir", AF_INET6, SOCK_DGRAM, false), + TEST("ipv6 udp addr redir", AF_INET6, SOCK_DGRAM, true), }; int server = -1; int self_net; diff --git a/tools/testing/selftests/bpf/progs/test_sk_assign.c b/tools/testing/selftests/bpf/progs/test_sk_assign.c index bde8748799eb..99547dcaac12 100644 --- a/tools/testing/selftests/bpf/progs/test_sk_assign.c +++ b/tools/testing/selftests/bpf/progs/test_sk_assign.c @@ -21,7 +21,7 @@ char _license[] SEC("license") = "GPL"; /* Fill 'tuple' with L3 info, and attempt to find L4. On fail, return NULL. */ static inline struct bpf_sock_tuple * -get_tuple(struct __sk_buff *skb, bool *ipv4) +get_tuple(struct __sk_buff *skb, bool *ipv4, bool *tcp) { void *data_end = (void *)(long)skb->data_end; void *data = (void *)(long)skb->data; @@ -60,12 +60,64 @@ get_tuple(struct __sk_buff *skb, bool *ipv4) return (struct bpf_sock_tuple *)data; } - if (result + 1 > data_end || proto != IPPROTO_TCP) + if (proto != IPPROTO_TCP && proto != IPPROTO_UDP) return NULL; + *tcp = (proto == IPPROTO_TCP); return result; } +static inline int +handle_udp(struct __sk_buff *skb, struct bpf_sock_tuple *tuple, bool ipv4) +{ + struct bpf_sock_tuple ln = {0}; + struct bpf_sock *sk; + size_t tuple_len; + int ret; + + tuple_len = ipv4 ? sizeof(tuple->ipv4) : sizeof(tuple->ipv6); + if ((void *)tuple + tuple_len > skb->data_end) + return TC_ACT_SHOT; + + sk = bpf_sk_lookup_udp(skb, tuple, tuple_len, BPF_F_CURRENT_NETNS, 0); + if (sk) + goto assign; + + if (ipv4) { + if (tuple->ipv4.dport != bpf_htons(4321)) + return TC_ACT_OK; + + ln.ipv4.daddr = bpf_htonl(0x7f000001); + ln.ipv4.dport = bpf_htons(1234); + + sk = bpf_sk_lookup_udp(skb, &ln, sizeof(ln.ipv4), + BPF_F_CURRENT_NETNS, 0); + } else { + if (tuple->ipv6.dport != bpf_htons(4321)) + return TC_ACT_OK; + + /* Upper parts of daddr are already zero. */ + ln.ipv6.daddr[3] = bpf_htonl(0x1); + ln.ipv6.dport = bpf_htons(1234); + + sk = bpf_sk_lookup_udp(skb, &ln, sizeof(ln.ipv6), + BPF_F_CURRENT_NETNS, 0); + } + + /* workaround: We can't do a single socket lookup here, because then + * the compiler will likely spill tuple_len to the stack. This makes it + * lose all bounds information in the verifier, which then rejects the + * call as unsafe. + */ + if (!sk) + return TC_ACT_SHOT; + +assign: + ret = bpf_sk_assign(skb, sk, 0); + bpf_sk_release(sk); + return ret; +} + static inline int handle_tcp(struct __sk_buff *skb, struct bpf_sock_tuple *tuple, bool ipv4) { @@ -130,14 +182,23 @@ int bpf_sk_assign_test(struct __sk_buff *skb) { struct bpf_sock_tuple *tuple, ln = {0}; bool ipv4 = false; + bool tcp = false; int tuple_len; int ret = 0; - tuple = get_tuple(skb, &ipv4); + tuple = get_tuple(skb, &ipv4, &tcp); if (!tuple) return TC_ACT_SHOT; - ret = handle_tcp(skb, tuple, ipv4); + /* Note that the verifier socket return type for bpf_skc_lookup_tcp() + * differs from bpf_sk_lookup_udp(), so even though the C-level type is + * the same here, if we try to share the implementations they will + * fail to verify because we're crossing pointer types. + */ + if (tcp) + ret = handle_tcp(skb, tuple, ipv4); + else + ret = handle_udp(skb, tuple, ipv4); return ret == 0 ? TC_ACT_OK : TC_ACT_SHOT; }