From patchwork Mon May 19 10:51:38 2014 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Ian Campbell X-Patchwork-Id: 30372 Return-Path: X-Original-To: linaro@patches.linaro.org Delivered-To: linaro@patches.linaro.org Received: from mail-ie0-f199.google.com (mail-ie0-f199.google.com [209.85.223.199]) by ip-10-151-82-157.ec2.internal (Postfix) with ESMTPS id 9A89E202FE for ; Mon, 19 May 2014 10:54:31 +0000 (UTC) Received: by mail-ie0-f199.google.com with SMTP id rp18sf16195410iec.2 for ; Mon, 19 May 2014 03:54:31 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:delivered-to:message-id:from:to:date:in-reply-to :references:organization:mime-version:cc:subject:precedence:list-id :list-unsubscribe:list-post:list-help:list-subscribe:sender :errors-to:x-original-sender:x-original-authentication-results :mailing-list:list-archive:content-type:content-transfer-encoding; bh=o4XusW7nf+ekFJ5ba8dV+RcajtY2QdXq06VnuETCEqg=; b=CLW/pzaFLL2qSU6PD4otjC4LMc3h5lQGMsbn6Tot7H/V61yLP2wnIiAmVw7CTyEHyZ uR9WeuZsGHEys6HG31V0abrjsfFHcQXrovYI7a1cmLps9MMQUMC4aDhvfjxpXs5dkrJS KRgCSPUQ9LIrUGzM4eZ7usMnDsBiazPO4RlCLQU9hfKFy/8g3KIyLTFB2Wrf5qDydX6s 0cq89RxhoNiHnJ12UybM183yaMtYlOpVXzRA3ttpF1ihlimEaTBNQ3UhsoZs+gt2D4F6 VsUWV9N2WbZlBpGjsYJRPl5jPhDEf5SPIelWxTuztsirB5g6M2dfBqqTCABqgLYsjTtw pUZQ== X-Gm-Message-State: ALoCoQkzTU8t3ldXV2bsDHRRrXRWVT61VYLRnOq/zUPNSZtEdsfrS7JgOdIkKPzGM1wSj2NYs7jK X-Received: by 10.182.118.162 with SMTP id kn2mr4320418obb.42.1400496871134; Mon, 19 May 2014 03:54:31 -0700 (PDT) X-BeenThere: patchwork-forward@linaro.org Received: by 10.140.107.247 with SMTP id h110ls1665325qgf.44.gmail; Mon, 19 May 2014 03:54:31 -0700 (PDT) X-Received: by 10.140.19.133 with SMTP id 5mr45862542qgh.46.1400496871064; Mon, 19 May 2014 03:54:31 -0700 (PDT) Received: from mail-qc0-f172.google.com (mail-qc0-f172.google.com [209.85.216.172]) by mx.google.com with ESMTPS id t40si8481785qge.104.2014.05.19.03.54.31 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Mon, 19 May 2014 03:54:31 -0700 (PDT) Received-SPF: pass (google.com: domain of patch+caf_=patchwork-forward=linaro.org@linaro.org designates 209.85.216.172 as permitted sender) client-ip=209.85.216.172; Received: by mail-qc0-f172.google.com with SMTP id l6so8750703qcy.31 for ; Mon, 19 May 2014 03:54:31 -0700 (PDT) X-Received: by 10.140.48.1 with SMTP id n1mr10341073qga.107.1400496870937; Mon, 19 May 2014 03:54:30 -0700 (PDT) X-Forwarded-To: patchwork-forward@linaro.org X-Forwarded-For: patch@linaro.org patchwork-forward@linaro.org Delivered-To: patch@linaro.org Received: by 10.96.8.167 with SMTP id s7csp217736qda; Mon, 19 May 2014 03:54:30 -0700 (PDT) X-Received: by 10.50.45.102 with SMTP id l6mr15962374igm.16.1400496870311; Mon, 19 May 2014 03:54:30 -0700 (PDT) Received: from lists.xen.org (lists.xen.org. [50.57.142.19]) by mx.google.com with ESMTPS id o19si11478205igt.38.2014.05.19.03.54.29 for (version=TLSv1 cipher=RC4-SHA bits=128/128); Mon, 19 May 2014 03:54:30 -0700 (PDT) Received-SPF: none (google.com: xen-devel-bounces@lists.xen.org does not designate permitted sender hosts) client-ip=50.57.142.19; Received: from localhost ([127.0.0.1] helo=lists.xen.org) by lists.xen.org with esmtp (Exim 4.72) (envelope-from ) id 1WmLAW-0003iq-RO; Mon, 19 May 2014 10:51:44 +0000 Received: from mail6.bemta4.messagelabs.com ([85.158.143.247]) by lists.xen.org with esmtp (Exim 4.72) (envelope-from ) id 1WmLAV-0003ic-Bv for xen-devel@lists.xen.org; Mon, 19 May 2014 10:51:43 +0000 Received: from [85.158.143.35:17053] by server-1.bemta-4.messagelabs.com id 30/47-09853-E32E9735; Mon, 19 May 2014 10:51:42 +0000 X-Env-Sender: Ian.Campbell@citrix.com X-Msg-Ref: server-9.tower-21.messagelabs.com!1400496700!5787735!1 X-Originating-IP: [66.165.176.63] X-SpamReason: No, hits=0.0 required=7.0 tests=sa_preprocessor: VHJ1c3RlZCBJUDogNjYuMTY1LjE3Ni42MyA9PiAzMDYwNDg=\n X-StarScan-Received: X-StarScan-Version: 6.11.3; banners=-,-,- X-VirusChecked: Checked Received: (qmail 29089 invoked from network); 19 May 2014 10:51:41 -0000 Received: from smtp02.citrix.com (HELO SMTP02.CITRIX.COM) (66.165.176.63) by server-9.tower-21.messagelabs.com with RC4-SHA encrypted SMTP; 19 May 2014 10:51:41 -0000 X-IronPort-AV: E=Sophos;i="4.98,866,1392163200"; d="scan'208";a="132324187" Received: from accessns.citrite.net (HELO FTLPEX01CL02.citrite.net) ([10.9.154.239]) by FTLPIPO02.CITRIX.COM with ESMTP; 19 May 2014 10:51:40 +0000 Received: from kazak.uk.xensource.com (10.80.2.80) by FTLPEX01CL02.citrite.net (10.13.107.79) with Microsoft SMTP Server id 14.3.181.6; Mon, 19 May 2014 06:51:39 -0400 Message-ID: <1400496698.28494.7.camel@kazak.uk.xensource.com> From: Ian Campbell To: Birintxo =?ISO-8859-1?Q?S=E1nchez?= Date: Mon, 19 May 2014 11:51:38 +0100 In-Reply-To: References: <1400489210-15750-1-git-send-email-birin.sanchez@citrix.com> Organization: Citrix Systems, Inc. X-Mailer: Evolution 3.12.1-1 MIME-Version: 1.0 X-Originating-IP: [10.80.2.80] X-DLP: MIA2 Cc: Birin Sanchez , xen-devel@lists.xen.org Subject: Re: [Xen-devel] [PATCH] sqlite_backup.pl: Create backup of DB using locks to avoid problems with queuerunner. X-BeenThere: xen-devel@lists.xen.org X-Mailman-Version: 2.1.13 Precedence: list List-Id: List-Unsubscribe: , List-Post: , List-Help: , List-Subscribe: , Sender: xen-devel-bounces@lists.xen.org Errors-To: xen-devel-bounces@lists.xen.org X-Removed-Original-Auth: Dkim didn't pass. X-Original-Sender: ian.campbell@citrix.com X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of patch+caf_=patchwork-forward=linaro.org@linaro.org designates 209.85.216.172 as permitted sender) smtp.mail=patch+caf_=patchwork-forward=linaro.org@linaro.org Mailing-list: list patchwork-forward@linaro.org; contact patchwork-forward+owners@linaro.org X-Google-Group-Id: 836684582541 List-Archive: On Mon, 2014-05-19 at 11:21 +0200, Birintxo Sánchez wrote: > Hi Ian, > > I forgot to add the runes for the cronjob: > > # SQLite backup for xen-devel-bugs DB. Runs everyday at 2:07 to get DB backup > # ready for offsite backup by Citrix (run at 3:30 am) > 7 2 * * * /usr/bin/env > PERLLIB=/srv/xen-devel-bugs/share/perl/5.14.2/ > /srv/xen-devel-bugs/lib/emesinae/sqlite_backup.pl Thanks, I have applied your patch, modifying it to add the new script to the top-level Makefile and with the following additional changeset on top. I also ran the backups a couple of times by hand and verified they worked, the crontab has been updated. Thanks! Ian. commit 0ade7d795d21ab4729bda6c07fe9f4f169bd7c5f Author: Ian Campbell Date: Mon May 19 11:45:42 2014 +0100 xen-bugs: Configure database backups > > Cheers, > > Birin > > > On 19 May 2014 10:46, Birin Sanchez wrote: > > > > --- > > config/emesinae.conf | 3 +++ > > config/examples/test/emesinae.conf | 3 +++ > > scripts/sqlite_backup.pl | 39 ++++++++++++++++++++++++++++++++++++++ > > 3 files changed, 45 insertions(+) > > create mode 100755 scripts/sqlite_backup.pl > > > > diff --git a/config/emesinae.conf b/config/emesinae.conf > > index 0aa1b3f..dd58c94 100644 > > --- a/config/emesinae.conf > > +++ b/config/emesinae.conf > > @@ -76,3 +76,6 @@ $c{ControlBlacklistPath} = "/etc/emesinae/control.blacklist"; > > # Severity levels, in decending order of criticality > > @{ $c{SeverityLevels} } = qw/blocker critical normal wishlist/; > > $c{DefaultSeverity} = "normal"; > > + > > +# Backup configuration > > +$c{BackupDir} = "/var/backups/"; > > diff --git a/config/examples/test/emesinae.conf b/config/examples/test/emesinae.conf > > index 528dcb6..11e4fad 100644 > > --- a/config/examples/test/emesinae.conf > > +++ b/config/examples/test/emesinae.conf > > @@ -70,3 +70,6 @@ $c{ControlBlacklistPath} = "/srv/test/etc/control.blacklist"; > > # Severity levels, in decending order of criticality > > @{ $c{SeverityLevels} } = qw/blocker critical normal wishlist/; > > $c{DefaultSeverity} = "normal"; > > + > > +# Backup configuration > > +$c{BackupDir} = "/srv/test/var/backups/"; > > diff --git a/scripts/sqlite_backup.pl b/scripts/sqlite_backup.pl > > new file mode 100755 > > index 0000000..557bc59 > > --- /dev/null > > +++ b/scripts/sqlite_backup.pl > > @@ -0,0 +1,39 @@ > > +#!/usr/bin/perl > > +# > > +# Creates a backup of the current DB in $c{BackupDir} > > +# using locks to avoid race conditions with queuerunner > > + > > +use strict; > > +use warnings; > > + > > +use Emesinae::Common; > > +use Emesinae::Paths; > > +use File::Copy; > > +use File::Basename; > > + > > +readconfig(); > > + > > +chdir( $c{BackupDir} ) || die("chdir to dir $c{BackupDir}: $!"); > > + > > + > > +my $lock = subsyslock('queuerunner'); > > + > > +my $BackupDB = $c{BackupDir} . basename($c{DB}); > > +my $BackupDBxz = $c{BackupDir} . basename($c{DB}) . ".xz"; > > +my $OldBackupDBxz = $c{BackupDir} . basename($c{DB}) . ".old.xz"; > > + > > +if ( -f $BackupDBxz ) { > > + move($BackupDBxz, $OldBackupDBxz) or die ("move failed: $!"); > > +} > > + > > +my $dbh = DBI->connect("dbi:SQLite:dbname=$c{DB}","",""); > > +$dbh->sqlite_backup_to_file( $BackupDB ); > > +undef $dbh; > > + > > +my @args = ("xz", "$BackupDB"); > > +system(@args) == 0 > > + or die ("xz compress failed: $?"); > > + > > +subsysunlock($lock); > > + > > +exit(0) > > -- > > 1.8.3.2 > > > > > > _______________________________________________ > > Xen-devel mailing list > > Xen-devel@lists.xen.org > > http://lists.xen.org/xen-devel diff --git a/config/examples/xen-bugs.xenproject.org/README b/config/examples/xen-bugs.xenproject.org/README index f49d934..cf989d0 100644 --- a/config/examples/xen-bugs.xenproject.org/README +++ b/config/examples/xen-bugs.xenproject.org/README @@ -15,10 +15,12 @@ $ make # chmod 1777 /srv/xen-devel-bugs/var/lock # mkdir /srv/xen-devel-bugs/var/raw /srv/xen-devel-bugs/var/run \ - /srv/xen-devel-bugs/var/spool/ /srv/xen-devel-bugs/var/spool/incoming + /srv/xen-devel-bugs/var/spool/ /srv/xen-devel-bugs/var/spool/incoming \ + /srv/xen-devel-bugs/var/backups # chown xen-devel-bugs:xen-devel-bugs \ /srv/xen-devel-bugs/var/raw /srv/xen-devel-bugs/var/run \ - /srv/xen-devel-bugs/var/spool/ /srv/xen-devel-bugs/var/spool/incoming + /srv/xen-devel-bugs/var/spool/ /srv/xen-devel-bugs/var/spool/incoming \ + /srv/xen-devel-bugs/var/backups # cp config/examples/xen-bugs.xenproject.org/emesinae.conf /srv/xen-devel-bugs/etc/ @@ -91,8 +93,8 @@ cron Configuration ================== # mkdir /srv/xen-devel-bugs/var/log -# touch /srv/xen-devel-bugs/var/log/queuerunner.log -# chown xen-devel-bugs:xen-devel-bugs /srv/xen-devel-bugs/var/log/queuerunner.log +# touch /srv/xen-devel-bugs/var/log/{backups,queuerunner}.log +# chown xen-devel-bugs:xen-devel-bugs /srv/xen-devel-bugs/var/log/{backups,queuerunner}.log # crontab -e -u xen-devel-bugs 8<----------------------------- @@ -100,5 +102,9 @@ SHELL=/bin/sh # Process queue every 15 minutes */15 * * * * /usr/bin/env PERLLIB=/srv/xen-devel-bugs/share/perl/5.14.2/ /srv/xen-devel-bugs/lib/emesinae/queuerunner.pl >>/srv/xen-devel-bugs/var/log/queuerunner.log + +# Backup everyday at 2:07 ready for offsite backup by Citrix (run at 3:30 am) +7 2 * * * /usr/bin/env PERLLIB=/srv/xen-devel-bugs/share/perl/5.14.2/ /srv/xen-devel-bugs/lib/emesinae/sqlite_backup.pl >>/srv/xen-devel-bugs/var/log/backups.log + 8<----------------------------- diff --git a/config/examples/xen-bugs.xenproject.org/emesinae.conf b/config/examples/xen-bugs.xenproject.org/emesinae.conf index bf6fcf8..beefa3e 100644 --- a/config/examples/xen-bugs.xenproject.org/emesinae.conf +++ b/config/examples/xen-bugs.xenproject.org/emesinae.conf @@ -72,3 +72,6 @@ $c{ControlBlacklistPath} = "/srv/xen-devel-bugs/etc/control.blacklist"; # Severity levels, in decending order of criticality @{ $c{SeverityLevels} } = qw/blocker critical normal wishlist/; $c{DefaultSeverity} = "normal"; + +# Backup configuration +$c{BackupDir} = "/srv/xen-devel-bugs/var/backups/";