diff mbox series

[bpf-next,v2,1/3] bpf: Make errors of bpf_tcp_check_syncookie distinguishable

Message ID 20220124151340.376807-2-maximmi@nvidia.com
State New
Headers show
Series New BPF helpers to accelerate synproxy | expand

Commit Message

Maxim Mikityanskiy Jan. 24, 2022, 3:13 p.m. UTC
bpf_tcp_check_syncookie returns ambiguous error codes in some cases. The
list below shows various error conditions and matching error codes:

1. NULL socket: -EINVAL.

2. Invalid packet: -EINVAL, -ENOENT.

3. Bad cookie: -ENOENT.

4. Cookies are not in use: -EINVAL, -ENOENT.

5. Good cookie: 0.

As we see, the same error code may correspond to multiple error
conditions, making them undistinguishable, and at the same time one
error condition may return different codes, although it's typically
handled in the same way.

This patch reassigns error codes of bpf_tcp_check_syncookie and
documents them:

1. Invalid packet or NULL socket: -EINVAL;

2. Bad cookie: -EACCES.

3. Cookies are not in use: -ENOENT.

4. Good cookie: 0.

This change allows XDP programs to make smarter decisions based on error
code, because different error conditions are now easily distinguishable.

Backward compatibility shouldn't suffer because of these reasons:

1. The specific error codes weren't documented. The behavior that used
   to be documented (0 is good cookie, negative values are errors) still
   holds. Anyone who relied on implementation details should have
   understood the risks.

2. Two known usecases (classification of ACKs with cookies that initial
   new connections, SYN flood protection) take decisions which don't
   depend on specific error codes:

     Traffic classification:
       ACK packet is new, error == 0: classify as NEW.
       ACK packet is new, error < 0: classify as INVALID.

     SYN flood protection:
       ACK packet is new, error == 0: good cookie, XDP_PASS.
       ACK packet is new, error < 0: bad cookie, XDP_DROP.

   As Lorenz Bauer confirms, their implementation of traffic classifier
   won't break, as well as the kernel selftests.

3. It's hard to imagine that old error codes could be used for any
   useful decisions.

Signed-off-by: Maxim Mikityanskiy <maximmi@nvidia.com>
Reviewed-by: Tariq Toukan <tariqt@nvidia.com>
---
 include/uapi/linux/bpf.h       | 18 ++++++++++++++++--
 net/core/filter.c              |  6 +++---
 tools/include/uapi/linux/bpf.h | 18 ++++++++++++++++--
 3 files changed, 35 insertions(+), 7 deletions(-)

Comments

John Fastabend Jan. 25, 2022, 7:38 a.m. UTC | #1
Maxim Mikityanskiy wrote:
> bpf_tcp_check_syncookie returns ambiguous error codes in some cases. The
> list below shows various error conditions and matching error codes:
> 
> 1. NULL socket: -EINVAL.
> 
> 2. Invalid packet: -EINVAL, -ENOENT.
> 
> 3. Bad cookie: -ENOENT.
> 
> 4. Cookies are not in use: -EINVAL, -ENOENT.
> 
> 5. Good cookie: 0.
> 
> As we see, the same error code may correspond to multiple error
> conditions, making them undistinguishable, and at the same time one
> error condition may return different codes, although it's typically
> handled in the same way.
> 
> This patch reassigns error codes of bpf_tcp_check_syncookie and
> documents them:
> 
> 1. Invalid packet or NULL socket: -EINVAL;
> 
> 2. Bad cookie: -EACCES.
> 
> 3. Cookies are not in use: -ENOENT.
> 
> 4. Good cookie: 0.
> 
> This change allows XDP programs to make smarter decisions based on error
> code, because different error conditions are now easily distinguishable.

I'm missing the point here it still looks a bit like shuffling
around of code. What do you gain, whats the real bug? Are you
trying to differentiate between an overflow condition and a valid
syncookie? But I don't think you said this anywhere.

At the moment EINVAL tells me somethings wrong with the input or
configuration, although really any app that cares checked the
sysctl flag right?

ENOENT tells me either recent overflow or cookie is invalid. If
there is no '!ack || rst || syn' then I can either learn that
directly from the program (why would a real program through
these at the helper), but it also falls into the incorrect
cookie in some sense.

> 
> Backward compatibility shouldn't suffer because of these reasons:
> 
> 1. The specific error codes weren't documented. The behavior that used
>    to be documented (0 is good cookie, negative values are errors) still
>    holds. Anyone who relied on implementation details should have
>    understood the risks.

I'll disagree, just because a user facing API doesn't document its
behavior very well doesn't mean users should some how understand the
risks. Ideally we would have done better with error codes up front,
but thats old history. If a user complains that this breaks a real
application it would likely be reason to revert it.

At least I would remove this from the commit.

> 
> 2. Two known usecases (classification of ACKs with cookies that initial
>    new connections, SYN flood protection) take decisions which don't
>    depend on specific error codes:
> 
>      Traffic classification:
>        ACK packet is new, error == 0: classify as NEW.
>        ACK packet is new, error < 0: classify as INVALID.
> 
>      SYN flood protection:
>        ACK packet is new, error == 0: good cookie, XDP_PASS.
>        ACK packet is new, error < 0: bad cookie, XDP_DROP.
> 
>    As Lorenz Bauer confirms, their implementation of traffic classifier
>    won't break, as well as the kernel selftests.
> 
> 3. It's hard to imagine that old error codes could be used for any
>    useful decisions.
> 
> Signed-off-by: Maxim Mikityanskiy <maximmi@nvidia.com>
> Reviewed-by: Tariq Toukan <tariqt@nvidia.com>
> ---
>  include/uapi/linux/bpf.h       | 18 ++++++++++++++++--
>  net/core/filter.c              |  6 +++---
>  tools/include/uapi/linux/bpf.h | 18 ++++++++++++++++--
>  3 files changed, 35 insertions(+), 7 deletions(-)
> 
> diff --git a/include/uapi/linux/bpf.h b/include/uapi/linux/bpf.h
> index 16a7574292a5..4d2d4a09bf25 100644
> --- a/include/uapi/linux/bpf.h
> +++ b/include/uapi/linux/bpf.h
> @@ -3575,8 +3575,22 @@ union bpf_attr {
>   * 		*th* points to the start of the TCP header, while *th_len*
>   * 		contains **sizeof**\ (**struct tcphdr**).
>   * 	Return
> - * 		0 if *iph* and *th* are a valid SYN cookie ACK, or a negative
> - * 		error otherwise.
> + *		0 if *iph* and *th* are a valid SYN cookie ACK.
> + *
> + *		On failure, the returned value is one of the following:
> + *
> + *		**-EACCES** if the SYN cookie is not valid.
> + *
> + *		**-EINVAL** if the packet or input arguments are invalid.
> + *
> + *		**-ENOENT** if SYN cookies are not issued (no SYN flood, or SYN
> + *		cookies are disabled in sysctl).
> + *
> + *		**-EOPNOTSUPP** if the kernel configuration does not enable SYN
> + *		cookies (CONFIG_SYN_COOKIES is off).
> + *
> + *		**-EPROTONOSUPPORT** if the IP version is not 4 or 6 (or 6, but
> + *		CONFIG_IPV6 is disabled).
>   *
>   * long bpf_sysctl_get_name(struct bpf_sysctl *ctx, char *buf, size_t buf_len, u64 flags)
>   *	Description
> diff --git a/net/core/filter.c b/net/core/filter.c
> index a06931c27eeb..18559b5828a3 100644
> --- a/net/core/filter.c
> +++ b/net/core/filter.c
> @@ -6998,10 +6998,10 @@ BPF_CALL_5(bpf_tcp_check_syncookie, struct sock *, sk, void *, iph, u32, iph_len
>  		return -EINVAL;
>  
>  	if (!sock_net(sk)->ipv4.sysctl_tcp_syncookies)
> -		return -EINVAL;
> +		return -ENOENT;

I wouldn't change this.

>  
>  	if (!th->ack || th->rst || th->syn)
> -		return -ENOENT;
> +		return -EINVAL;

not sure if this is useful change and it is bpf program visible.

>  
>  	if (tcp_synq_no_recent_overflow(sk))
>  		return -ENOENT;
> @@ -7032,7 +7032,7 @@ BPF_CALL_5(bpf_tcp_check_syncookie, struct sock *, sk, void *, iph, u32, iph_len
>  	if (ret > 0)
>  		return 0;
>  
> -	return -ENOENT;
> +	return -EACCES;

This one might have a valid argument to differentiate between an
overflow condition and an invalid cookie. But, curious what do you
do with this info?

Thanks,
John
Maxim Mikityanskiy Jan. 31, 2022, 1:38 p.m. UTC | #2
On 2022-01-25 09:38, John Fastabend wrote:
> Maxim Mikityanskiy wrote:
>> bpf_tcp_check_syncookie returns ambiguous error codes in some cases. The
>> list below shows various error conditions and matching error codes:
>>
>> 1. NULL socket: -EINVAL.
>>
>> 2. Invalid packet: -EINVAL, -ENOENT.
>>
>> 3. Bad cookie: -ENOENT.
>>
>> 4. Cookies are not in use: -EINVAL, -ENOENT.
>>
>> 5. Good cookie: 0.
>>
>> As we see, the same error code may correspond to multiple error
>> conditions, making them undistinguishable, and at the same time one
>> error condition may return different codes, although it's typically
>> handled in the same way.
>>
>> This patch reassigns error codes of bpf_tcp_check_syncookie and
>> documents them:
>>
>> 1. Invalid packet or NULL socket: -EINVAL;
>>
>> 2. Bad cookie: -EACCES.
>>
>> 3. Cookies are not in use: -ENOENT.
>>
>> 4. Good cookie: 0.
>>
>> This change allows XDP programs to make smarter decisions based on error
>> code, because different error conditions are now easily distinguishable.
> 
> I'm missing the point here it still looks a bit like shuffling
> around of code. What do you gain, whats the real bug?

Current error codes are useless. If the caller gets a negative value, 
all it knows is that some error happened. I'm making different error 
conditions distinguishable (invalid input, bad cookie, not expecting a 
cookie).

Use cases could be: statistic counters, debugging, logging. For example, 
the kernel counts LINUX_MIB_SYNCOOKIESFAILED, which only includes the 
"bad cookie" case.

Another use case is replying with RST when not expecting a cookie (a new 
ACK packet could just mean that the server was rebooted, and it's good 
to tell the client that the connection is broken), but dropping packets 
under the flood on receiving bad cookies (to avoid wasting resources on 
sending replies).

> Are you
> trying to differentiate between an overflow condition and a valid
> syncookie? But I don't think you said this anywhere.

I'm not sure what you mean by the "overflow condition", but a valid 
syncookie is easily distinguishable from other cases, it's 0.

> At the moment EINVAL tells me somethings wrong with the input or
> configuration, although really any app that cares checked the
> sysctl flag right?

There is no way to check sysctl from XDP. A single program could be 
useful both with and without syncookies, it could determine its behavior 
in runtime, not to say that the sysctl option could change in runtime. 
The workaround you suggested will work in some cases, but other cases 
just won't work (there are no notification events on sysctl changes that 
a userspace application could monitor and pass to the XDP program; it 
would be not immediate anyway).

> ENOENT tells me either recent overflow or cookie is invalid.

And these cases should be distinguished, as I said above.

> If
> there is no '!ack || rst || syn' then I can either learn that
> directly from the program (why would a real program through
> these at the helper), but it also falls into the incorrect
> cookie in some sense.
> 
>>
>> Backward compatibility shouldn't suffer because of these reasons:
>>
>> 1. The specific error codes weren't documented. The behavior that used
>>     to be documented (0 is good cookie, negative values are errors) still
>>     holds. Anyone who relied on implementation details should have
>>     understood the risks.
> 
> I'll disagree, just because a user facing API doesn't document its
> behavior very well doesn't mean users should some how understand the
> risks. Ideally we would have done better with error codes up front,
> but thats old history. If a user complains that this breaks a real
> application it would likely be reason to revert it.
> 
> At least I would remove this from the commit.
> 
>>
>> 2. Two known usecases (classification of ACKs with cookies that initial
>>     new connections, SYN flood protection) take decisions which don't
>>     depend on specific error codes:
>>
>>       Traffic classification:
>>         ACK packet is new, error == 0: classify as NEW.
>>         ACK packet is new, error < 0: classify as INVALID.
>>
>>       SYN flood protection:
>>         ACK packet is new, error == 0: good cookie, XDP_PASS.
>>         ACK packet is new, error < 0: bad cookie, XDP_DROP.
>>
>>     As Lorenz Bauer confirms, their implementation of traffic classifier
>>     won't break, as well as the kernel selftests.
>>
>> 3. It's hard to imagine that old error codes could be used for any
>>     useful decisions.
>>
>> Signed-off-by: Maxim Mikityanskiy <maximmi@nvidia.com>
>> Reviewed-by: Tariq Toukan <tariqt@nvidia.com>
>> ---
>>   include/uapi/linux/bpf.h       | 18 ++++++++++++++++--
>>   net/core/filter.c              |  6 +++---
>>   tools/include/uapi/linux/bpf.h | 18 ++++++++++++++++--
>>   3 files changed, 35 insertions(+), 7 deletions(-)
>>
>> diff --git a/include/uapi/linux/bpf.h b/include/uapi/linux/bpf.h
>> index 16a7574292a5..4d2d4a09bf25 100644
>> --- a/include/uapi/linux/bpf.h
>> +++ b/include/uapi/linux/bpf.h
>> @@ -3575,8 +3575,22 @@ union bpf_attr {
>>    * 		*th* points to the start of the TCP header, while *th_len*
>>    * 		contains **sizeof**\ (**struct tcphdr**).
>>    * 	Return
>> - * 		0 if *iph* and *th* are a valid SYN cookie ACK, or a negative
>> - * 		error otherwise.
>> + *		0 if *iph* and *th* are a valid SYN cookie ACK.
>> + *
>> + *		On failure, the returned value is one of the following:
>> + *
>> + *		**-EACCES** if the SYN cookie is not valid.
>> + *
>> + *		**-EINVAL** if the packet or input arguments are invalid.
>> + *
>> + *		**-ENOENT** if SYN cookies are not issued (no SYN flood, or SYN
>> + *		cookies are disabled in sysctl).
>> + *
>> + *		**-EOPNOTSUPP** if the kernel configuration does not enable SYN
>> + *		cookies (CONFIG_SYN_COOKIES is off).
>> + *
>> + *		**-EPROTONOSUPPORT** if the IP version is not 4 or 6 (or 6, but
>> + *		CONFIG_IPV6 is disabled).
>>    *
>>    * long bpf_sysctl_get_name(struct bpf_sysctl *ctx, char *buf, size_t buf_len, u64 flags)
>>    *	Description
>> diff --git a/net/core/filter.c b/net/core/filter.c
>> index a06931c27eeb..18559b5828a3 100644
>> --- a/net/core/filter.c
>> +++ b/net/core/filter.c
>> @@ -6998,10 +6998,10 @@ BPF_CALL_5(bpf_tcp_check_syncookie, struct sock *, sk, void *, iph, u32, iph_len
>>   		return -EINVAL;
>>   
>>   	if (!sock_net(sk)->ipv4.sysctl_tcp_syncookies)
>> -		return -EINVAL;
>> +		return -ENOENT;
> 
> I wouldn't change this.
> 
>>   
>>   	if (!th->ack || th->rst || th->syn)
>> -		return -ENOENT;
>> +		return -EINVAL;
> 
> not sure if this is useful change and it is bpf program visible.
> 
>>   
>>   	if (tcp_synq_no_recent_overflow(sk))
>>   		return -ENOENT;
>> @@ -7032,7 +7032,7 @@ BPF_CALL_5(bpf_tcp_check_syncookie, struct sock *, sk, void *, iph, u32, iph_len
>>   	if (ret > 0)
>>   		return 0;
>>   
>> -	return -ENOENT;
>> +	return -EACCES;
> 
> This one might have a valid argument to differentiate between an
> overflow condition and an invalid cookie. But, curious what do you
> do with this info?
> 
> Thanks,
> John
diff mbox series

Patch

diff --git a/include/uapi/linux/bpf.h b/include/uapi/linux/bpf.h
index 16a7574292a5..4d2d4a09bf25 100644
--- a/include/uapi/linux/bpf.h
+++ b/include/uapi/linux/bpf.h
@@ -3575,8 +3575,22 @@  union bpf_attr {
  * 		*th* points to the start of the TCP header, while *th_len*
  * 		contains **sizeof**\ (**struct tcphdr**).
  * 	Return
- * 		0 if *iph* and *th* are a valid SYN cookie ACK, or a negative
- * 		error otherwise.
+ *		0 if *iph* and *th* are a valid SYN cookie ACK.
+ *
+ *		On failure, the returned value is one of the following:
+ *
+ *		**-EACCES** if the SYN cookie is not valid.
+ *
+ *		**-EINVAL** if the packet or input arguments are invalid.
+ *
+ *		**-ENOENT** if SYN cookies are not issued (no SYN flood, or SYN
+ *		cookies are disabled in sysctl).
+ *
+ *		**-EOPNOTSUPP** if the kernel configuration does not enable SYN
+ *		cookies (CONFIG_SYN_COOKIES is off).
+ *
+ *		**-EPROTONOSUPPORT** if the IP version is not 4 or 6 (or 6, but
+ *		CONFIG_IPV6 is disabled).
  *
  * long bpf_sysctl_get_name(struct bpf_sysctl *ctx, char *buf, size_t buf_len, u64 flags)
  *	Description
diff --git a/net/core/filter.c b/net/core/filter.c
index a06931c27eeb..18559b5828a3 100644
--- a/net/core/filter.c
+++ b/net/core/filter.c
@@ -6998,10 +6998,10 @@  BPF_CALL_5(bpf_tcp_check_syncookie, struct sock *, sk, void *, iph, u32, iph_len
 		return -EINVAL;
 
 	if (!sock_net(sk)->ipv4.sysctl_tcp_syncookies)
-		return -EINVAL;
+		return -ENOENT;
 
 	if (!th->ack || th->rst || th->syn)
-		return -ENOENT;
+		return -EINVAL;
 
 	if (tcp_synq_no_recent_overflow(sk))
 		return -ENOENT;
@@ -7032,7 +7032,7 @@  BPF_CALL_5(bpf_tcp_check_syncookie, struct sock *, sk, void *, iph, u32, iph_len
 	if (ret > 0)
 		return 0;
 
-	return -ENOENT;
+	return -EACCES;
 #else
 	return -ENOTSUPP;
 #endif
diff --git a/tools/include/uapi/linux/bpf.h b/tools/include/uapi/linux/bpf.h
index 16a7574292a5..4d2d4a09bf25 100644
--- a/tools/include/uapi/linux/bpf.h
+++ b/tools/include/uapi/linux/bpf.h
@@ -3575,8 +3575,22 @@  union bpf_attr {
  * 		*th* points to the start of the TCP header, while *th_len*
  * 		contains **sizeof**\ (**struct tcphdr**).
  * 	Return
- * 		0 if *iph* and *th* are a valid SYN cookie ACK, or a negative
- * 		error otherwise.
+ *		0 if *iph* and *th* are a valid SYN cookie ACK.
+ *
+ *		On failure, the returned value is one of the following:
+ *
+ *		**-EACCES** if the SYN cookie is not valid.
+ *
+ *		**-EINVAL** if the packet or input arguments are invalid.
+ *
+ *		**-ENOENT** if SYN cookies are not issued (no SYN flood, or SYN
+ *		cookies are disabled in sysctl).
+ *
+ *		**-EOPNOTSUPP** if the kernel configuration does not enable SYN
+ *		cookies (CONFIG_SYN_COOKIES is off).
+ *
+ *		**-EPROTONOSUPPORT** if the IP version is not 4 or 6 (or 6, but
+ *		CONFIG_IPV6 is disabled).
  *
  * long bpf_sysctl_get_name(struct bpf_sysctl *ctx, char *buf, size_t buf_len, u64 flags)
  *	Description