From patchwork Tue Jul 25 20:45:22 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Amit Pundir X-Patchwork-Id: 108706 Delivered-To: patch@linaro.org Received: by 10.140.101.44 with SMTP id t41csp29076qge; Tue, 25 Jul 2017 13:46:09 -0700 (PDT) X-Received: by 10.99.115.66 with SMTP id d2mr20599078pgn.10.1501015569045; Tue, 25 Jul 2017 13:46:09 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1501015569; cv=none; d=google.com; s=arc-20160816; b=poXkkb5IaXcTgbX2RZRhgiY3rpAV7rZmKImbDM/mTtiDw7QckxKZYZZ2tjUcO9xSOA sCyz/r7IurLBAwQGoWnds9d8Pz88ZoQVryqnQ4XfP/Ly3ub7O8IdZ1fHh1vHl1+e43LO 7xIVCnvanNmjdoWTfwaTciZBEKfe+bmaKRi3CeoV5mlg+CJCGBSwHAadrT0bQ02TeaGn 3IwOykd5jcO6tcwLcfbe5jAeHEJuYR9CpFbKjEuahcOp4BFUJOOr47Os1IxECVvyFxJ+ +0RaINfX+izzywngBTmcPzZlHzwWLIM/1BM/odnf9wA007XeRDx3476t3fb9BSmC5sBk zdDw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:cc:to:from:dkim-signature:arc-authentication-results; bh=qvvNCtzgTl0gZtbAZk6eF2SvtXZf51mivkxbh9xqDhQ=; b=tLlZp7EzkeIlX5BgmzPdc0BwfW0tXlBa2K78oM2ENPYuOSfh+ntTmX3w9krDB4bM3M YGqAthxFqmw0aqtyRtAhJAC4OS6HtVZdjwfXDwKDAsQTqdWhnyijxrmXYSF5YXsivZqD uKKuv2ajARjoY4+0EY0E5iv+iiasMjKR6bsZbRu93ROsLdkhOdD6clKln5elJj6UwWH5 vXWsbmVp/CgSH0rsJj83sG3dJ8QKvWOxdYQIbA75IQZqoVyRvyDbKZD37ihRFn8zKBMS m4NxutCEZAScbs0dLibetIhJOBSryS7nyIjDG/D76gqHYRb9idjsdf32jX/c+cfisA4J 7POg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.b=gpYoYPFP; spf=pass (google.com: best guess record for domain of stable-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=stable-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id k3si387020pgk.256.2017.07.25.13.46.08; Tue, 25 Jul 2017 13:46:09 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of stable-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.b=gpYoYPFP; spf=pass (google.com: best guess record for domain of stable-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=stable-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752748AbdGYUqG (ORCPT + 6 others); Tue, 25 Jul 2017 16:46:06 -0400 Received: from mail-pg0-f48.google.com ([74.125.83.48]:35877 "EHLO mail-pg0-f48.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752469AbdGYUqF (ORCPT ); Tue, 25 Jul 2017 16:46:05 -0400 Received: by mail-pg0-f48.google.com with SMTP id 125so74835343pgi.3 for ; Tue, 25 Jul 2017 13:46:04 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=qvvNCtzgTl0gZtbAZk6eF2SvtXZf51mivkxbh9xqDhQ=; b=gpYoYPFPzgOlEOYRS9SNk9GFv1gN01c5Fm9ZMTYB8M0gxWVfMXQDteNlmydDe1mCZW FlIQODgB2pShUoy9dQcyhgOqHal6lwiT1CRvOnHfHtgkkg9llszuwRKqJUoeXdXoOTSf qFXr7z4U2ESWIprF0WyYaHD6n3SnqcuJN5VF0= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=qvvNCtzgTl0gZtbAZk6eF2SvtXZf51mivkxbh9xqDhQ=; b=tHBUEg7RrJHs9m4KkEJq+0UPblZg/qmP13BGW9xPloZxxr4iDgpz9WfdLC0vTKHVqQ /l1xNjgAqfgq7iKq2QfivvLEE0IypQkp9AphOIE689DgIFaMD21LIvaQSft37bHooGGL m43vqlBnkKHtjsAkqJiEqmS7FUEzm7RQi94ok/8V/z7zcFWbLejd8jkoZo4dYxC+6VF+ mWCF9uGg/gShyVqDdYhz4fIUvNedBJ+4ycRmWf4Dpe6LL3nimmh01/zbawcc6/RvpoaZ hzdYi3l7c9l6cciVwA0kBBkWyA5W45L6iMpJQh15lMpdXYPzCqGCh/aGqdGmv501Cism knvA== X-Gm-Message-State: AIVw111GISyuga5Wnu2+m2jCo6w9i1/Syu5UQ1WIINLrqQfSLLeMz6E2 cxMuui0OdkMod8S9 X-Received: by 10.98.103.11 with SMTP id b11mr13885149pfc.211.1501015564380; Tue, 25 Jul 2017 13:46:04 -0700 (PDT) Received: from localhost.localdomain ([106.51.135.235]) by smtp.gmail.com with ESMTPSA id d4sm532125pfj.59.2017.07.25.13.46.01 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Tue, 25 Jul 2017 13:46:03 -0700 (PDT) From: Amit Pundir To: Greg KH Cc: Stable , Daniel Borkmann , Cong Wang , "David S . Miller" Subject: [PATCH for-3.18 11/15] net, sched: fix soft lockup in tc_classify Date: Wed, 26 Jul 2017 02:15:22 +0530 Message-Id: <1501015526-32178-12-git-send-email-amit.pundir@linaro.org> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1501015526-32178-1-git-send-email-amit.pundir@linaro.org> References: <1501015526-32178-1-git-send-email-amit.pundir@linaro.org> Sender: stable-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: stable@vger.kernel.org From: Daniel Borkmann commit 628185cfddf1dfb701c4efe2cfd72cf5b09f5702 upstream. Shahar reported a soft lockup in tc_classify(), where we run into an endless loop when walking the classifier chain due to tp->next == tp which is a state we should never run into. The issue only seems to trigger under load in the tc control path. What happens is that in tc_ctl_tfilter(), thread A allocates a new tp, initializes it, sets tp_created to 1, and calls into tp->ops->change() with it. In that classifier callback we had to unlock/lock the rtnl mutex and returned with -EAGAIN. One reason why we need to drop there is, for example, that we need to request an action module to be loaded. This happens via tcf_exts_validate() -> tcf_action_init/_1() meaning after we loaded and found the requested action, we need to redo the whole request so we don't race against others. While we had to unlock rtnl in that time, thread B's request was processed next on that CPU. Thread B added a new tp instance successfully to the classifier chain. When thread A returned grabbing the rtnl mutex again, propagating -EAGAIN and destroying its tp instance which never got linked, we goto replay and redo A's request. This time when walking the classifier chain in tc_ctl_tfilter() for checking for existing tp instances we had a priority match and found the tp instance that was created and linked by thread B. Now calling again into tp->ops->change() with that tp was successful and returned without error. tp_created was never cleared in the second round, thus kernel thinks that we need to link it into the classifier chain (once again). tp and *back point to the same object due to the match we had earlier on. Thus for thread B's already public tp, we reset tp->next to tp itself and link it into the chain, which eventually causes the mentioned endless loop in tc_classify() once a packet hits the data path. Fix is to clear tp_created at the beginning of each request, also when we replay it. On the paths that can cause -EAGAIN we already destroy the original tp instance we had and on replay we really need to start from scratch. It seems that this issue was first introduced in commit 12186be7d2e1 ("net_cls: fix unconfigured struct tcf_proto keeps chaining and avoid kernel panic when we use cls_cgroup"). Fixes: 12186be7d2e1 ("net_cls: fix unconfigured struct tcf_proto keeps chaining and avoid kernel panic when we use cls_cgroup") Reported-by: Shahar Klein Signed-off-by: Daniel Borkmann Cc: Cong Wang Acked-by: Eric Dumazet Tested-by: Shahar Klein Signed-off-by: David S. Miller Signed-off-by: Amit Pundir --- net/sched/cls_api.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) -- 2.7.4 diff --git a/net/sched/cls_api.c b/net/sched/cls_api.c index fae88709aaa2..e50272592724 100644 --- a/net/sched/cls_api.c +++ b/net/sched/cls_api.c @@ -137,13 +137,15 @@ static int tc_ctl_tfilter(struct sk_buff *skb, struct nlmsghdr *n) unsigned long cl; unsigned long fh; int err; - int tp_created = 0; + int tp_created; if ((n->nlmsg_type != RTM_GETTFILTER) && !netlink_ns_capable(skb, net->user_ns, CAP_NET_ADMIN)) return -EPERM; replay: + tp_created = 0; + err = nlmsg_parse(n, sizeof(*t), tca, TCA_MAX, NULL); if (err < 0) return err;