From patchwork Tue Nov 15 04:04:04 2011 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: John Stultz X-Patchwork-Id: 5115 Return-Path: X-Original-To: patchwork@peony.canonical.com Delivered-To: patchwork@peony.canonical.com Received: from fiordland.canonical.com (fiordland.canonical.com [91.189.94.145]) by peony.canonical.com (Postfix) with ESMTP id 89CFA23EF5 for ; Tue, 15 Nov 2011 04:04:31 +0000 (UTC) Received: from mail-fx0-f52.google.com (mail-fx0-f52.google.com [209.85.161.52]) by fiordland.canonical.com (Postfix) with ESMTP id 78DE3A181D1 for ; Tue, 15 Nov 2011 04:04:31 +0000 (UTC) Received: by mail-fx0-f52.google.com with SMTP id a26so64372faa.11 for ; Mon, 14 Nov 2011 20:04:31 -0800 (PST) Received: by 10.152.135.225 with SMTP id pv1mr2520908lab.19.1321329870694; Mon, 14 Nov 2011 20:04:30 -0800 (PST) X-Forwarded-To: linaro-patchwork@canonical.com X-Forwarded-For: patch@linaro.org linaro-patchwork@canonical.com Delivered-To: patches@linaro.org Received: by 10.152.41.198 with SMTP id h6cs24734lal; Mon, 14 Nov 2011 20:04:30 -0800 (PST) Received: by 10.68.38.4 with SMTP id c4mr12309355pbk.54.1321329865395; Mon, 14 Nov 2011 20:04:25 -0800 (PST) Received: from e3.ny.us.ibm.com (e3.ny.us.ibm.com. [32.97.182.143]) by mx.google.com with ESMTPS id 7si4037963pbr.125.2011.11.14.20.04.24 (version=TLSv1/SSLv3 cipher=OTHER); Mon, 14 Nov 2011 20:04:25 -0800 (PST) Received-SPF: pass (google.com: domain of jstultz@us.ibm.com designates 32.97.182.143 as permitted sender) client-ip=32.97.182.143; Authentication-Results: mx.google.com; spf=pass (google.com: domain of jstultz@us.ibm.com designates 32.97.182.143 as permitted sender) smtp.mail=jstultz@us.ibm.com Received: from /spool/local by e3.ny.us.ibm.com with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted for from ; Mon, 14 Nov 2011 23:04:23 -0500 Received: from d01relay05.pok.ibm.com ([9.56.227.237]) by e3.ny.us.ibm.com ([192.168.1.103]) with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted; Mon, 14 Nov 2011 23:04:21 -0500 Received: from d01av03.pok.ibm.com (d01av03.pok.ibm.com [9.56.224.217]) by d01relay05.pok.ibm.com (8.13.8/8.13.8/NCO v10.0) with ESMTP id pAF44KaG249236; Mon, 14 Nov 2011 23:04:20 -0500 Received: from d01av03.pok.ibm.com (loopback [127.0.0.1]) by d01av03.pok.ibm.com (8.14.4/8.13.1/NCO v10.0 AVout) with ESMTP id pAF44Ku9000342; Tue, 15 Nov 2011 02:04:20 -0200 Received: from kernel.beaverton.ibm.com ([9.47.67.96]) by d01av03.pok.ibm.com (8.14.4/8.13.1/NCO v10.0 AVin) with ESMTP id pAF44JMW032757; Tue, 15 Nov 2011 02:04:20 -0200 Received: by kernel.beaverton.ibm.com (Postfix, from userid 1056) id 044761E750A; Mon, 14 Nov 2011 20:04:18 -0800 (PST) From: John Stultz To: LKML Cc: John Stultz , Thomas Gleixner , Eric Dumazet , Richard Cochran Subject: [PATCH 14/16] time: Rework timekeeping functions to take timekeeper ptr as argument Date: Mon, 14 Nov 2011 20:04:04 -0800 Message-Id: <1321329846-14755-15-git-send-email-john.stultz@linaro.org> X-Mailer: git-send-email 1.7.3.2.146.gca209 In-Reply-To: <1321329846-14755-1-git-send-email-john.stultz@linaro.org> References: <1321329846-14755-1-git-send-email-john.stultz@linaro.org> x-cbid: 11111504-8974-0000-0000-00000214575E As part of cleaning up the timekeeping code, this patch converts a number of internal functions to takea timekeeper ptr as an argument, so that the internal functions don't access the global timekeeper structure directly. This allows for further optimizations later. CC: Thomas Gleixner CC: Eric Dumazet CC: Richard Cochran Signed-off-by: John Stultz --- kernel/time/timekeeping.c | 87 +++++++++++++++++++++++---------------------- 1 files changed, 44 insertions(+), 43 deletions(-) diff --git a/kernel/time/timekeeping.c b/kernel/time/timekeeping.c index 6ca50b5..7870a0e 100644 --- a/kernel/time/timekeeping.c +++ b/kernel/time/timekeeping.c @@ -206,16 +206,15 @@ static inline s64 timekeeping_get_ns_raw(void) } /* must hold write on timekeeper.lock */ -static void timekeeping_update(bool clearntp) +static void timekeeping_update(struct timekeeper *tk, bool clearntp) { struct timespec xt; if (clearntp) { - timekeeper.ntp_error = 0; + tk->ntp_error = 0; ntp_clear(); } - xt = timekeeper_xtime(&timekeeper); - update_vsyscall(&xt, &timekeeper.wall_to_monotonic, - timekeeper.clock, timekeeper.mult); + xt = timekeeper_xtime(tk); + update_vsyscall(&xt, &tk->wall_to_monotonic, tk->clock, tk->mult); } @@ -226,7 +225,7 @@ void timekeeping_leap_insert(int leapsecond) write_seqlock_irqsave(&timekeeper.lock, flags); timekeeper.xtime_sec += leapsecond; timekeeper.wall_to_monotonic.tv_sec -= leapsecond; - timekeeping_update(false); + timekeeping_update(&timekeeper, false); write_sequnlock_irqrestore(&timekeeper.lock, flags); } @@ -426,7 +425,7 @@ int do_settimeofday(const struct timespec *tv) timekeeper_set_xtime(&timekeeper, tv); - timekeeping_update(true); + timekeeping_update(&timekeeper, true); write_sequnlock_irqrestore(&timekeeper.lock, flags); @@ -461,7 +460,7 @@ int timekeeping_inject_offset(struct timespec *ts) timekeeper.wall_to_monotonic = timespec_sub(timekeeper.wall_to_monotonic, *ts); - timekeeping_update(true); + timekeeping_update(&timekeeper, true); write_sequnlock_irqrestore(&timekeeper.lock, flags); @@ -698,7 +697,7 @@ void timekeeping_inject_sleeptime(struct timespec *delta) __timekeeping_inject_sleeptime(delta); - timekeeping_update(true); + timekeeping_update(&timekeeper, true); write_sequnlock_irqrestore(&timekeeper.lock, flags); @@ -801,7 +800,8 @@ device_initcall(timekeeping_init_ops); * If the error is already larger, we look ahead even further * to compensate for late or lost adjustments. */ -static __always_inline int timekeeping_bigadjust(s64 error, s64 *interval, +static __always_inline int timekeeping_bigadjust(struct timekeeper *tk, + s64 error, s64 *interval, s64 *offset) { s64 tick_error, i; @@ -817,7 +817,7 @@ static __always_inline int timekeeping_bigadjust(s64 error, s64 *interval, * here. This is tuned so that an error of about 1 msec is adjusted * within about 1 sec (or 2^20 nsec in 2^SHIFT_HZ ticks). */ - error2 = timekeeper.ntp_error >> (NTP_SCALE_SHIFT + 22 - 2 * SHIFT_HZ); + error2 = tk->ntp_error >> (NTP_SCALE_SHIFT + 22 - 2 * SHIFT_HZ); error2 = abs(error2); for (look_ahead = 0; error2 > 0; look_ahead++) error2 >>= 2; @@ -826,8 +826,8 @@ static __always_inline int timekeeping_bigadjust(s64 error, s64 *interval, * Now calculate the error in (1 << look_ahead) ticks, but first * remove the single look ahead already included in the error. */ - tick_error = ntp_tick_length() >> (timekeeper.ntp_error_shift + 1); - tick_error -= timekeeper.xtime_interval >> 1; + tick_error = ntp_tick_length() >> (tk->ntp_error_shift + 1); + tick_error -= tk->xtime_interval >> 1; error = ((error - tick_error) >> look_ahead) + tick_error; /* Finally calculate the adjustment shift value. */ @@ -852,18 +852,19 @@ static __always_inline int timekeeping_bigadjust(s64 error, s64 *interval, * this is optimized for the most common adjustments of -1,0,1, * for other values we can do a bit more work. */ -static void timekeeping_adjust(s64 offset) +static void timekeeping_adjust(struct timekeeper *tk, s64 offset) { - s64 error, interval = timekeeper.cycle_interval; + s64 error, interval = tk->cycle_interval; int adj; - error = timekeeper.ntp_error >> (timekeeper.ntp_error_shift - 1); + error = tk->ntp_error >> (tk->ntp_error_shift - 1); if (error > interval) { error >>= 2; if (likely(error <= interval)) adj = 1; else - adj = timekeeping_bigadjust(error, &interval, &offset); + adj = timekeeping_bigadjust(tk, error, &interval, + &offset); } else if (error < -interval) { error >>= 2; if (likely(error >= -interval)) { @@ -871,15 +872,15 @@ static void timekeeping_adjust(s64 offset) interval = -interval; offset = -offset; } else - adj = timekeeping_bigadjust(error, &interval, &offset); + adj = timekeeping_bigadjust(tk, error, &interval, + &offset); } else return; - timekeeper.mult += adj; - timekeeper.xtime_interval += interval; - timekeeper.xtime_nsec -= offset; - timekeeper.ntp_error -= (interval - offset) << - timekeeper.ntp_error_shift; + tk->mult += adj; + tk->xtime_interval += interval; + tk->xtime_nsec -= offset; + tk->ntp_error -= (interval - offset) << tk->ntp_error_shift; } @@ -892,41 +893,41 @@ static void timekeeping_adjust(s64 offset) * * Returns the unconsumed cycles. */ -static cycle_t logarithmic_accumulation(cycle_t offset, int shift) +static cycle_t logarithmic_accumulation(struct timekeeper *tk, cycle_t offset, + int shift) { - u64 nsecps = (u64)NSEC_PER_SEC << timekeeper.shift; + u64 nsecps = (u64)NSEC_PER_SEC << tk->shift; u64 raw_nsecs; /* If the offset is smaller then a shifted interval, do nothing */ - if (offset < timekeeper.cycle_interval<cycle_interval<cycle_last += timekeeper.cycle_interval << shift; + offset -= tk->cycle_interval << shift; + tk->clock->cycle_last += tk->cycle_interval << shift; - timekeeper.xtime_nsec += timekeeper.xtime_interval << shift; - while (timekeeper.xtime_nsec >= nsecps) { - timekeeper.xtime_nsec -= nsecps; - timekeeper.xtime_sec++; + tk->xtime_nsec += tk->xtime_interval << shift; + while (tk->xtime_nsec >= nsecps) { + tk->xtime_nsec -= nsecps; + tk->xtime_sec++; second_overflow(); } /* Accumulate raw time */ - raw_nsecs = timekeeper.raw_interval << shift; - raw_nsecs += timekeeper.raw_time.tv_nsec; + raw_nsecs = tk->raw_interval << shift; + raw_nsecs += tk->raw_time.tv_nsec; if (raw_nsecs >= NSEC_PER_SEC) { u64 raw_secs = raw_nsecs; raw_nsecs = do_div(raw_secs, NSEC_PER_SEC); - timekeeper.raw_time.tv_sec += raw_secs; + tk->raw_time.tv_sec += raw_secs; } - timekeeper.raw_time.tv_nsec = raw_nsecs; + tk->raw_time.tv_nsec = raw_nsecs; /* Accumulate error between NTP and clock interval */ - timekeeper.ntp_error += ntp_tick_length() << shift; - timekeeper.ntp_error -= - (timekeeper.xtime_interval + timekeeper.xtime_remainder) << - (timekeeper.ntp_error_shift + shift); + tk->ntp_error += ntp_tick_length() << shift; + tk->ntp_error -= (tk->xtime_interval + tk->xtime_remainder) << + (tk->ntp_error_shift + shift); return offset; } @@ -972,13 +973,13 @@ static void update_wall_time(void) maxshift = (64 - (ilog2(ntp_tick_length())+1)) - 1; shift = min(shift, maxshift); while (offset >= timekeeper.cycle_interval) { - offset = logarithmic_accumulation(offset, shift); + offset = logarithmic_accumulation(&timekeeper, offset, shift); if(offset < timekeeper.cycle_interval<