From patchwork Tue Mar 31 15:19:00 2015 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Viresh Kumar X-Patchwork-Id: 46595 Return-Path: X-Original-To: linaro@patches.linaro.org Delivered-To: linaro@patches.linaro.org Received: from mail-la0-f71.google.com (mail-la0-f71.google.com [209.85.215.71]) by ip-10-151-82-157.ec2.internal (Postfix) with ESMTPS id EB07A216E5 for ; Tue, 31 Mar 2015 15:20:01 +0000 (UTC) Received: by labtp8 with SMTP id tp8sf4782249lab.2 for ; Tue, 31 Mar 2015 08:20:00 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:delivered-to:from:to:cc:subject :date:message-id:in-reply-to:references:in-reply-to:references :sender:precedence:list-id:x-original-sender :x-original-authentication-results:mailing-list:list-post:list-help :list-archive:list-unsubscribe; bh=r37SQEN+Wa17ecYO/MJP9DkG7qDjKkH8YWRO+URWWQI=; b=gUoOlzPKYvJnmhIoBQfmeos6T8O1rTf3brI+xPxGpL7PQVSXECtpig+iYnmcTRug00 H7ELSINQnEf8xkN5UDIFv/PZKfq0gqzY64271f3dhG7iP77WqLco7w6jFOSyZVDz4Jx9 1h/71vGl/vrkvAag1b8DirvxIEitFuraXQeDY2Kxu6U2ySioV+4K4Upo7F5ss2sAV7bN RBsVwbH4KjRTHapjm0UjWeOTP/HALuSWAp1byf2QAp1GqW26zWShWdgZrfF+Kbhsk/m9 KxD5/ssJmCJW/CcMIvockf3FW8LEdErpdKJhndW4BSpk5YSWwfDODR/8KajeDBTxp1Eg 83gA== X-Gm-Message-State: ALoCoQkBS+YwzQJ9h24bAwZLc5AQgTXoGB7ucmtidRAAlkZrg2nEZr4maqj+GEC6QhSqx34De2jT X-Received: by 10.112.151.232 with SMTP id ut8mr960270lbb.9.1427815200842; Tue, 31 Mar 2015 08:20:00 -0700 (PDT) MIME-Version: 1.0 X-BeenThere: patchwork-forward@linaro.org Received: by 10.153.6.33 with SMTP id cr1ls590978lad.47.gmail; Tue, 31 Mar 2015 08:20:00 -0700 (PDT) X-Received: by 10.112.13.7 with SMTP id d7mr31438161lbc.79.1427815200627; Tue, 31 Mar 2015 08:20:00 -0700 (PDT) Received: from mail-lb0-f177.google.com (mail-lb0-f177.google.com. [209.85.217.177]) by mx.google.com with ESMTPS id n1si9390402lbc.29.2015.03.31.08.20.00 for (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 31 Mar 2015 08:20:00 -0700 (PDT) Received-SPF: pass (google.com: domain of patch+caf_=patchwork-forward=linaro.org@linaro.org designates 209.85.217.177 as permitted sender) client-ip=209.85.217.177; Received: by lbcmq2 with SMTP id mq2so15035689lbc.0 for ; Tue, 31 Mar 2015 08:20:00 -0700 (PDT) X-Received: by 10.152.26.34 with SMTP id i2mr10034050lag.117.1427815200487; Tue, 31 Mar 2015 08:20:00 -0700 (PDT) X-Forwarded-To: patchwork-forward@linaro.org X-Forwarded-For: patch@linaro.org patchwork-forward@linaro.org Delivered-To: patch@linaro.org Received: by 10.112.57.201 with SMTP id k9csp51917lbq; Tue, 31 Mar 2015 08:19:59 -0700 (PDT) X-Received: by 10.66.228.104 with SMTP id sh8mr68182169pac.63.1427815198549; Tue, 31 Mar 2015 08:19:58 -0700 (PDT) Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id g4si19870056pdd.111.2015.03.31.08.19.56; Tue, 31 Mar 2015 08:19:58 -0700 (PDT) Received-SPF: none (google.com: linux-kernel-owner@vger.kernel.org does not designate permitted sender hosts) client-ip=209.132.180.67; Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S932312AbbCaPTq (ORCPT + 27 others); Tue, 31 Mar 2015 11:19:46 -0400 Received: from mail-pd0-f169.google.com ([209.85.192.169]:34304 "EHLO mail-pd0-f169.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752723AbbCaPTl (ORCPT ); Tue, 31 Mar 2015 11:19:41 -0400 Received: by pdbni2 with SMTP id ni2so23874053pdb.1 for ; Tue, 31 Mar 2015 08:19:41 -0700 (PDT) X-Received: by 10.66.161.161 with SMTP id xt1mr69195204pab.35.1427815181031; Tue, 31 Mar 2015 08:19:41 -0700 (PDT) Received: from localhost ([122.167.118.120]) by mx.google.com with ESMTPSA id z6sm7068735pdp.77.2015.03.31.08.19.39 (version=TLSv1.2 cipher=RC4-SHA bits=128/128); Tue, 31 Mar 2015 08:19:40 -0700 (PDT) From: Viresh Kumar To: Ingo Molnar , Peter Zijlstra , Thomas Gleixner Cc: linaro-kernel@lists.linaro.org, linux-kernel@vger.kernel.org, Viresh Kumar Subject: [PATCH V2 1/3] timer: Allocate per-cpu tvec_base's statically Date: Tue, 31 Mar 2015 20:49:00 +0530 Message-Id: <17cdf560f2727f687ab159707d0aa591f8a2f82d.1427814611.git.viresh.kumar@linaro.org> X-Mailer: git-send-email 2.3.0.rc0.44.ga94655d In-Reply-To: References: In-Reply-To: References: Sender: linux-kernel-owner@vger.kernel.org Precedence: list List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Removed-Original-Auth: Dkim didn't pass. X-Original-Sender: viresh.kumar@linaro.org X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of patch+caf_=patchwork-forward=linaro.org@linaro.org designates 209.85.217.177 as permitted sender) smtp.mail=patch+caf_=patchwork-forward=linaro.org@linaro.org Mailing-list: list patchwork-forward@linaro.org; contact patchwork-forward+owners@linaro.org X-Google-Group-Id: 836684582541 List-Post: , List-Help: , List-Archive: List-Unsubscribe: , From: Peter Zijlstra Memory for tvec_base is allocated separately for boot CPU (statically) and non-boot CPUs (dynamically). The reason is because __TIMER_INITIALIZER() needs to set ->base to a valid pointer (because we've made NULL special, hint: lock_timer_base()) and we cannot get a compile time pointer to per-cpu entries because we don't know where we'll map the section, even for the boot cpu. This can be simplified a bit by statically allocating per-cpu memory. The only disadvantage is that memory for one of the structures will stay unused, i.e. for the boot CPU, which uses boot_tvec_bases. This will also guarantee that tvec_base is cacheline aligned. Even though tvec_base has ____cacheline_aligned stuck on, kzalloc_node() does not actually respect that (but guarantees a minimum u64 alignment). Cc: Thomas Gleixner Cc: linaro-kernel@lists.linaro.org Cc: Ingo Molnar Signed-off-by: Viresh Kumar Signed-off-by: Peter Zijlstra (Intel) --- kernel/time/timer.c | 48 +++++++++++++++++++----------------------------- 1 file changed, 19 insertions(+), 29 deletions(-) diff --git a/kernel/time/timer.c b/kernel/time/timer.c index 2d3f5c504939..f3cc653f876c 100644 --- a/kernel/time/timer.c +++ b/kernel/time/timer.c @@ -90,8 +90,19 @@ struct tvec_base { struct tvec tv5; } ____cacheline_aligned; +/* + * __TIMER_INITIALIZER() needs to set ->base to a valid pointer (because we've + * made NULL special, hint: lock_timer_base()) and we cannot get a compile time + * pointer to per-cpu entries because we don't know where we'll map the section, + * even for the boot cpu. + * + * And so we use boot_tvec_bases for boot CPU and per-cpu __tvec_bases for the + * rest of them. + */ struct tvec_base boot_tvec_bases; EXPORT_SYMBOL(boot_tvec_bases); +static DEFINE_PER_CPU(struct tvec_base, __tvec_bases); + static DEFINE_PER_CPU(struct tvec_base *, tvec_bases) = &boot_tvec_bases; /* Functions below help us manage 'deferrable' flag */ @@ -1534,46 +1545,25 @@ EXPORT_SYMBOL(schedule_timeout_uninterruptible); static int init_timers_cpu(int cpu) { - int j; - struct tvec_base *base; + struct tvec_base *base = per_cpu(tvec_bases, cpu); static char tvec_base_done[NR_CPUS]; + int j; if (!tvec_base_done[cpu]) { - static char boot_done; + static char boot_cpu_skipped; - if (boot_done) { - /* - * The APs use this path later in boot - */ - base = kzalloc_node(sizeof(*base), GFP_KERNEL, - cpu_to_node(cpu)); - if (!base) - return -ENOMEM; - - /* Make sure tvec_base has TIMER_FLAG_MASK bits free */ - if (WARN_ON(base != tbase_get_base(base))) { - kfree(base); - return -ENOMEM; - } - per_cpu(tvec_bases, cpu) = base; + if (!boot_cpu_skipped) { + boot_cpu_skipped = 1; /* skip the boot cpu */ } else { - /* - * This is for the boot CPU - we use compile-time - * static initialisation because per-cpu memory isn't - * ready yet and because the memory allocators are not - * initialised either. - */ - boot_done = 1; - base = &boot_tvec_bases; + base = per_cpu_ptr(&__tvec_bases, cpu); + per_cpu(tvec_bases, cpu) = base; } + spin_lock_init(&base->lock); tvec_base_done[cpu] = 1; base->cpu = cpu; - } else { - base = per_cpu(tvec_bases, cpu); } - for (j = 0; j < TVN_SIZE; j++) { INIT_LIST_HEAD(base->tv5.vec + j); INIT_LIST_HEAD(base->tv4.vec + j);