From patchwork Mon May 7 04:51:10 2012 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Spring Zhang X-Patchwork-Id: 8422 Return-Path: X-Original-To: patchwork@peony.canonical.com Delivered-To: patchwork@peony.canonical.com Received: from fiordland.canonical.com (fiordland.canonical.com [91.189.94.145]) by peony.canonical.com (Postfix) with ESMTP id BE0FF23E13 for ; Mon, 7 May 2012 04:51:14 +0000 (UTC) Received: from mail-ob0-f180.google.com (mail-ob0-f180.google.com [209.85.214.180]) by fiordland.canonical.com (Postfix) with ESMTP id 5EC72A180DF for ; Mon, 7 May 2012 04:51:14 +0000 (UTC) Received: by obbup16 with SMTP id up16so10109191obb.11 for ; Sun, 06 May 2012 21:51:13 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=x-forwarded-to:x-forwarded-for:delivered-to:received-spf :content-type:mime-version:x-launchpad-project:x-launchpad-branch :x-launchpad-message-rationale:x-launchpad-branch-revision-number :x-launchpad-notification-type:to:from:subject:message-id:date :reply-to:sender:errors-to:precedence:x-generated-by :x-launchpad-hash:x-gm-message-state; bh=OsyNh/WbXCyTI3R0uOp3cJPV81zuxwRyFkiAyPBHhTk=; b=Z8Ee5ffJyj57AusD1YbqYZZ2eJD4Z0/L5O6G6BpZvGaG1BOVr97Rg9z8oujYJOeEXT wNrDuwjCG0WX9f5ZvQa1uUTtOgObzZPQghI/pr5h7w21BUov4vOagCDcEzjlw85DOBw0 Du3GJNy/zFmovtNHDckv7K0+cawbCVtxCWZK30mxxtfp8KTji/ZAxB383JP1RI7dr8tF p4A4ZHNSGws9rBMtMfA2xQ+b45s8OHaO/JuWIiH/ACchLT/zVQHNuMnnlx0RQuaYMx7L Q2B0gP9eNHyAu/PvyMUqa7qCepAKKRHptOs+PMvdZM2i5RMDnpMu17LBmpFhfJGE+vOM KXTg== Received: by 10.50.47.131 with SMTP id d3mr7505264ign.33.1336366273612; Sun, 06 May 2012 21:51:13 -0700 (PDT) X-Forwarded-To: linaro-patchwork@canonical.com X-Forwarded-For: patch@linaro.org linaro-patchwork@canonical.com Delivered-To: patches@linaro.org Received: by 10.231.73.147 with SMTP id q19csp102083ibj; Sun, 6 May 2012 21:51:12 -0700 (PDT) Received: by 10.216.135.199 with SMTP id u49mr9136439wei.7.1336366271309; Sun, 06 May 2012 21:51:11 -0700 (PDT) Received: from indium.canonical.com (indium.canonical.com. [91.189.90.7]) by mx.google.com with ESMTPS id e18si12950922wea.58.2012.05.06.21.51.10 (version=TLSv1/SSLv3 cipher=OTHER); Sun, 06 May 2012 21:51:11 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of bounces@canonical.com designates 91.189.90.7 as permitted sender) client-ip=91.189.90.7; Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of bounces@canonical.com designates 91.189.90.7 as permitted sender) smtp.mail=bounces@canonical.com Received: from ackee.canonical.com ([91.189.89.26]) by indium.canonical.com with esmtp (Exim 4.71 #1 (Debian)) id 1SRFug-00024F-PG for ; Mon, 07 May 2012 04:51:10 +0000 Received: from ackee.canonical.com (localhost [127.0.0.1]) by ackee.canonical.com (Postfix) with ESMTP id B41FFE002E for ; Mon, 7 May 2012 04:51:10 +0000 (UTC) MIME-Version: 1.0 X-Launchpad-Project: lava-dispatcher X-Launchpad-Branch: ~linaro-validation/lava-dispatcher/trunk X-Launchpad-Message-Rationale: Subscriber X-Launchpad-Branch-Revision-Number: 280 X-Launchpad-Notification-Type: branch-revision To: Linaro Patch Tracker From: noreply@launchpad.net Subject: [Branch ~linaro-validation/lava-dispatcher/trunk] Rev 280: Merge proxy configuration doc Message-Id: <20120507045110.2359.42702.launchpad@ackee.canonical.com> Date: Mon, 07 May 2012 04:51:10 -0000 Reply-To: noreply@launchpad.net Sender: bounces@canonical.com Errors-To: bounces@canonical.com Precedence: bulk X-Generated-By: Launchpad (canonical.com); Revision="15195"; Instance="launchpad-lazr.conf" X-Launchpad-Hash: 23fde02d51f47c2dc7ff6853ba80373cb6615c7d X-Gm-Message-State: ALoCoQlOFwHu4wCb2MzkGkyAxtXFWqhtsQW+eLggDR0nBuj7AE5OfKpv03oW/9x4S7P9Tc2pIjB0 Merge authors: Spring Zhang (qzhang) Related merge proposals: https://code.launchpad.net/~qzhang/lava-dispatcher/doc-squid-proxy/+merge/103598 proposed by: Spring Zhang (qzhang) review: Approve - Spring Zhang (qzhang) ------------------------------------------------------------ revno: 280 [merge] committer: Spring Zhang branch nick: merge-config-proxy timestamp: Mon 2012-05-07 12:48:34 +0800 message: Merge proxy configuration doc added: doc/proxy.rst modified: doc/index.rst --- lp:lava-dispatcher https://code.launchpad.net/~linaro-validation/lava-dispatcher/trunk You are subscribed to branch lp:lava-dispatcher. To unsubscribe from this branch go to https://code.launchpad.net/~linaro-validation/lava-dispatcher/trunk/+edit-subscription === modified file 'doc/index.rst' --- doc/index.rst 2012-04-23 05:51:22 +0000 +++ doc/index.rst 2012-05-07 04:38:42 +0000 @@ -77,6 +77,7 @@ installation.rst configuration.rst + proxy.rst jobfile.rst usage.rst changes.rst === added file 'doc/proxy.rst' --- doc/proxy.rst 1970-01-01 00:00:00 +0000 +++ doc/proxy.rst 2012-05-07 04:43:52 +0000 @@ -0,0 +1,105 @@ +.. _proxy: + +Cache Proxy Setting Up +====================== + +Before, it used to use an internal cache mechanism for downloaded images and +hwpacks to avoid downloading repeatly, which could save time and bandwidth. + +lava-dispatcher switches to use cache proxy for managing cache files +automatically. The recommended proxy is Squid. + +Install Squid +^^^^^^^^^^^^^ + +Squid is easy to install via apt-get:: + + sudo apt-get install squid + +Or if you want a configurable squid, refer to the following link to compile +and install manually: http://wiki.squid-cache.org/SquidFaq/CompilingSquid + +Configure Squid +^^^^^^^^^^^^^^^ + +You will need to customize accroding to your server, like disk layout, space. + +Need to analyse and tune by collecting information when squid running with +real cases, like cache policy, file system. + +Mandatory configuration options +------------------------------- + +Based on original /etc/squid/squid.conf, see below tuning. + +* cache_dir ufs /var/spool/squid 30720 16 256 + + Mandatory option, please modify 30720(MB) to an available size. + + There can be several cache directories on different disk, but it's better not + use RAID on the cache directories, it's recommended by Squid: The Definitive + Guide that it will always degrades fs performance for squid. 30720 is the + cache amount 30GB. 16 and 256 is Level 1 and 2 sub-directories, which is + default. + +* maximum_object_size 1024000 KB + + Mandatory option. + + Setting the value as 1024000KB makes the squid cache large files less than + 1GB, for our images are usually a large one but less than 1G. + +Optional configuration options +------------------------------ + +Some others than mandatory options. + +* acl over_conn_limit maxconn 10 # make max connection limit 10 + +* http_access allow localnet + + Enable localnet, also, we need to define more localnet in server environment + to make sure all boards IP and other permitted clients are included. + + acl localnet src 10.122.0.0/16 + +* http_access deny over_conn_limit + + Make max connection of one client less than 10, it should be enough for + a board, it can be increased. + +* cache_mem 128 MB + + It can be increased if server MEM is enough, it's for squid mem amount for + objects. + +* cache_swap_low 90 + + cache_swap_high 95 + + Cache size will maintain between 90% to 95%. + +* client_lifetime 12 hours + + Make a client continuous accessing time 12hrs, default is 1 days, it can be + increased. + +* high_response_time_warning 2000 + + 2s in 1mins no response will log in cache.log. + +* There is some email configurations to be set, like 'cache_mgr', it will send + mail if cache proxy dies. + +The configuration is only workable, there can be more improvement ways, some +still need to tune on server. + +Other tuning +------------ + +Open files number can be increased for squid will need more than 1024 +limitations sometimes:: + + # ulimit -n + 1024 +