From patchwork Thu Jun 11 22:42:15 2015 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: John Stultz X-Patchwork-Id: 49772 Return-Path: X-Original-To: linaro@patches.linaro.org Delivered-To: linaro@patches.linaro.org Received: from mail-lb0-f200.google.com (mail-lb0-f200.google.com [209.85.217.200]) by ip-10-151-82-157.ec2.internal (Postfix) with ESMTPS id 737C320C81 for ; Thu, 11 Jun 2015 22:42:29 +0000 (UTC) Received: by lbcak1 with SMTP id ak1sf5223366lbc.2 for ; Thu, 11 Jun 2015 15:42:28 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:delivered-to:from:to:cc:subject :date:message-id:x-original-sender:x-original-authentication-results :precedence:mailing-list:list-id:list-post:list-help:list-archive :list-unsubscribe; bh=N6suhPX1xWJzkGMYy7XmArjmLNWRbDCr0o3WhGo39qA=; b=AQHcNCt2PTP++9c1ln0lnyK6s9vWnZ0RLk5w9Cg7RJTSVf46OSMlgi7ITkFMENhDSE w1UdAmFCeOtJizrHl12Elk7kMWKpeIzJrlAxJCRlESq4YnLqHsCFKXZO+a14NsShK5a8 zVUmmDI+d9u/TkT/lhoT5e4z/tcI23LeaiiTdbjQuiVotV0BJbPOpl6JUu5n+tqvfgvW zbixt7f6fpBOeXfQErDcHqKvsMyxiRhImW6F18ACI/3m2qkP9idHKeaPnY57pLVKlDM1 EXFhspsg/91fsxm2rt+8XMW5+MXg1RuOBK9LdZBOxpb5XVNAAJbUHrN1EW+E/R/2dlRc rhTQ== X-Gm-Message-State: ALoCoQkM4MRPPn4h6sIptw3VRK4Unc69/wHreGaBV3AtGyRAiL0bT7NL4Lf5fWDCrkLnnQeC9ogo X-Received: by 10.180.187.175 with SMTP id ft15mr638458wic.4.1434062548413; Thu, 11 Jun 2015 15:42:28 -0700 (PDT) MIME-Version: 1.0 X-BeenThere: patchwork-forward@linaro.org Received: by 10.152.36.227 with SMTP id t3ls503228laj.72.gmail; Thu, 11 Jun 2015 15:42:28 -0700 (PDT) X-Received: by 10.152.5.194 with SMTP id u2mr12218286lau.94.1434062548248; Thu, 11 Jun 2015 15:42:28 -0700 (PDT) Received: from mail-la0-f51.google.com (mail-la0-f51.google.com. [209.85.215.51]) by mx.google.com with ESMTPS id g1si1785111laf.139.2015.06.11.15.42.28 for (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 11 Jun 2015 15:42:28 -0700 (PDT) Received-SPF: pass (google.com: domain of patch+caf_=patchwork-forward=linaro.org@linaro.org designates 209.85.215.51 as permitted sender) client-ip=209.85.215.51; Received: by lazv15 with SMTP id v15so11327631laz.3 for ; Thu, 11 Jun 2015 15:42:28 -0700 (PDT) X-Received: by 10.112.234.200 with SMTP id ug8mr12099980lbc.117.1434062548023; Thu, 11 Jun 2015 15:42:28 -0700 (PDT) X-Forwarded-To: patchwork-forward@linaro.org X-Forwarded-For: patch@linaro.org patchwork-forward@linaro.org Delivered-To: patches@linaro.org Received: by 10.112.108.230 with SMTP id hn6csp130067lbb; Thu, 11 Jun 2015 15:42:27 -0700 (PDT) X-Received: by 10.66.150.196 with SMTP id uk4mr18308849pab.54.1434062546313; Thu, 11 Jun 2015 15:42:26 -0700 (PDT) Received: from mail-pd0-f172.google.com (mail-pd0-f172.google.com. [209.85.192.172]) by mx.google.com with ESMTPS id ab8si123847pad.188.2015.06.11.15.42.25 for (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 11 Jun 2015 15:42:26 -0700 (PDT) Received-SPF: pass (google.com: domain of john.stultz@linaro.org designates 209.85.192.172 as permitted sender) client-ip=209.85.192.172; Received: by pdbki1 with SMTP id ki1so11348823pdb.1 for ; Thu, 11 Jun 2015 15:42:25 -0700 (PDT) X-Received: by 10.68.69.65 with SMTP id c1mr16359614pbu.25.1434062545351; Thu, 11 Jun 2015 15:42:25 -0700 (PDT) Received: from localhost.localdomain (c-71-63-136-39.hsd1.or.comcast.net. [71.63.136.39]) by mx.google.com with ESMTPSA id ja1sm1650434pbc.51.2015.06.11.15.42.23 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Thu, 11 Jun 2015 15:42:24 -0700 (PDT) From: John Stultz To: linux-kernel@vger.kernel.org Cc: John Stultz , Ingo Molnar , Peter Zijlstra , Gleb Natapov , Paolo Bonzini , Thomas Gleixner , Marcelo Tosatti Subject: Problems with kvm steal time accounting. Date: Thu, 11 Jun 2015 15:42:15 -0700 Message-Id: <1434062535-28185-1-git-send-email-john.stultz@linaro.org> X-Mailer: git-send-email 1.9.1 X-Removed-Original-Auth: Dkim didn't pass. X-Original-Sender: john.stultz@linaro.org X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of patch+caf_=patchwork-forward=linaro.org@linaro.org designates 209.85.215.51 as permitted sender) smtp.mail=patch+caf_=patchwork-forward=linaro.org@linaro.org Precedence: list Mailing-list: list patchwork-forward@linaro.org; contact patchwork-forward+owners@linaro.org List-ID: X-Google-Group-Id: 836684582541 List-Post: , List-Help: , List-Archive: List-Unsubscribe: , I recently noticed that running the selftest/timers/posix_timers test after a suspend/resume cycle in my VM environment was hanging, because the ITIMER_VIRTUAL signal wasn't firing. More specifically, it seemed like after a suspend, process utime wasn't being updated unless the process was running on cpu 0. I chased this down a bit, and it seems the problem is that on resume, the paravirt_steal_clock() is reset to zero on all cpus other then cpu0. This causes the rq->prev_steal_time value to be set to a negative value, which mucks up the rest of the time accounting. I'm not sure if this is due to a bug in kvm or what. I've not chased back how long this bug has been around, but it seemed to crop up in at least 3.16 when I tried it. The following hack works around this, but I suspect the steal-clock offset needs to be saved on suspend and re-added on resume, or the prev_steal_time values need to be reset, so that we can be robust in the face of this sort of behavior. Thoughts or other ideas? thanks -john Cc: Ingo Molnar Cc: Peter Zijlstra Cc: Gleb Natapov Cc: Paolo Bonzini Cc: Thomas Gleixner Cc: Marcelo Tosatti --- kernel/sched/cputime.c | 11 ++++++++--- 1 file changed, 8 insertions(+), 3 deletions(-) diff --git a/kernel/sched/cputime.c b/kernel/sched/cputime.c index 8394b1e..439d880 100644 --- a/kernel/sched/cputime.c +++ b/kernel/sched/cputime.c @@ -258,12 +258,17 @@ static __always_inline bool steal_account_process_tick(void) { #ifdef CONFIG_PARAVIRT if (static_key_false(¶virt_steal_enabled)) { - u64 steal; + u64 steal, prev; cputime_t steal_ct; steal = paravirt_steal_clock(smp_processor_id()); - steal -= this_rq()->prev_steal_time; - + prev = this_rq()->prev_steal_time; + if (steal >= prev) + steal -= prev; + else { + printk("WARNING: %lld - %lld is negative!!\n", steal, prev); + this_rq()->prev_steal_time = steal; + } /* * cputime_t may be less precise than nsecs (eg: if it's * based on jiffies). Lets cast the result to cputime