From patchwork Fri Feb 21 08:17:17 2014 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael Roth X-Patchwork-Id: 25099 Return-Path: X-Original-To: linaro@patches.linaro.org Delivered-To: linaro@patches.linaro.org Received: from mail-yk0-f200.google.com (mail-yk0-f200.google.com [209.85.160.200]) by ip-10-151-82-157.ec2.internal (Postfix) with ESMTPS id 89BD720540 for ; Fri, 21 Feb 2014 08:36:59 +0000 (UTC) Received: by mail-yk0-f200.google.com with SMTP id q200sf7521206ykb.3 for ; Fri, 21 Feb 2014 00:36:59 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:delivered-to:from:to:date :message-id:in-reply-to:references:cc:subject:precedence:list-id :list-unsubscribe:list-archive:list-post:list-help:list-subscribe :errors-to:sender:x-original-sender :x-original-authentication-results:mailing-list; bh=H7q8sFQKJWyrp1uqDdP3dKDW4cChkX3T6gfjYNCOYQ0=; b=LX9cF2VkMY8gbpoZzImCXVG7AWNxDNOUdgIm4GjuuLXkgqNPNtyAPORhI/cwsdTGcI XAoFHZh3QCpEAPb922KSe3cyroCg34NoUhKS67UbOFz13qyO5PlHWSOg1qIWk/ZSQpjB Ob/dKzQRl/An0PDCpuKVINpOavb9kvGoIt4Zm39EhqJuGZ7D5tYi68JA3GMvZdILUfOk piRtvoy9wl/Qh28XXy1yDfTHk9qI91yPgq2PYWBeumP/oU20qlpNj6J9liWB4GX2bcQs Nyd/EsR9ItJNfnVklTqfZppvYS4e1mFHKjxGpP9kL2sCdVr7zR66VdnPV6Dhct4jpgVk daxA== X-Gm-Message-State: ALoCoQlOCfhQMqaE7243UzWbc2+3RJgq/h7W2gQe/hUbyeTSaUc5Xc6Z8pPdpgwl6ITpIT4cKGwt X-Received: by 10.236.83.112 with SMTP id p76mr2498980yhe.51.1392971819328; Fri, 21 Feb 2014 00:36:59 -0800 (PST) MIME-Version: 1.0 X-BeenThere: patchwork-forward@linaro.org Received: by 10.140.108.73 with SMTP id i67ls1030912qgf.3.gmail; Fri, 21 Feb 2014 00:36:59 -0800 (PST) X-Received: by 10.53.9.107 with SMTP id dr11mr3266328vdd.1.1392971819187; Fri, 21 Feb 2014 00:36:59 -0800 (PST) Received: from mail-vc0-f178.google.com (mail-vc0-f178.google.com [209.85.220.178]) by mx.google.com with ESMTPS id f5si2707115vej.111.2014.02.21.00.36.59 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Fri, 21 Feb 2014 00:36:59 -0800 (PST) Received-SPF: neutral (google.com: 209.85.220.178 is neither permitted nor denied by best guess record for domain of patch+caf_=patchwork-forward=linaro.org@linaro.org) client-ip=209.85.220.178; Received: by mail-vc0-f178.google.com with SMTP id ik5so2966948vcb.9 for ; Fri, 21 Feb 2014 00:36:59 -0800 (PST) X-Received: by 10.220.225.71 with SMTP id ir7mr3932782vcb.50.1392971819106; Fri, 21 Feb 2014 00:36:59 -0800 (PST) X-Forwarded-To: patchwork-forward@linaro.org X-Forwarded-For: patch@linaro.org patchwork-forward@linaro.org Delivered-To: patch@linaro.org Received: by 10.220.174.196 with SMTP id u4csp17252vcz; Fri, 21 Feb 2014 00:36:58 -0800 (PST) X-Received: by 10.140.41.134 with SMTP id z6mr7625056qgz.112.1392971818595; Fri, 21 Feb 2014 00:36:58 -0800 (PST) Received: from lists.gnu.org (lists.gnu.org. [2001:4830:134:3::11]) by mx.google.com with ESMTPS id 6si3775164qgy.86.2014.02.21.00.36.58 for (version=TLSv1 cipher=RC4-SHA bits=128/128); Fri, 21 Feb 2014 00:36:58 -0800 (PST) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 2001:4830:134:3::11 as permitted sender) client-ip=2001:4830:134:3::11; Received: from localhost ([::1]:42992 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1WGlbO-0005H3-4S for patch@linaro.org; Fri, 21 Feb 2014 03:36:58 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:47185) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1WGlKA-0006QV-C6 for qemu-devel@nongnu.org; Fri, 21 Feb 2014 03:19:22 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1WGlJx-0001qY-Jp for qemu-devel@nongnu.org; Fri, 21 Feb 2014 03:19:10 -0500 Received: from e36.co.us.ibm.com ([32.97.110.154]:43389) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1WGlJx-0001qG-E6 for qemu-devel@nongnu.org; Fri, 21 Feb 2014 03:18:57 -0500 Received: from /spool/local by e36.co.us.ibm.com with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted for from ; Fri, 21 Feb 2014 01:18:57 -0700 Received: from d03dlp02.boulder.ibm.com (9.17.202.178) by e36.co.us.ibm.com (192.168.1.136) with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted; Fri, 21 Feb 2014 01:18:53 -0700 Received: from b03cxnp07029.gho.boulder.ibm.com (b03cxnp07029.gho.boulder.ibm.com [9.17.130.16]) by d03dlp02.boulder.ibm.com (Postfix) with ESMTP id 5D56C3E40045; Fri, 21 Feb 2014 01:18:53 -0700 (MST) Received: from d03av03.boulder.ibm.com (d03av03.boulder.ibm.com [9.17.195.169]) by b03cxnp07029.gho.boulder.ibm.com (8.13.8/8.13.8/NCO v10.0) with ESMTP id s1L6GEu88388866; Fri, 21 Feb 2014 07:16:14 +0100 Received: from d03av03.boulder.ibm.com (localhost [127.0.0.1]) by d03av03.boulder.ibm.com (8.14.4/8.14.4/NCO v10.0 AVout) with ESMTP id s1L8IqVO013181; Fri, 21 Feb 2014 01:18:53 -0700 Received: from localhost ([9.80.9.18]) by d03av03.boulder.ibm.com (8.14.4/8.14.4/NCO v10.0 AVin) with ESMTP id s1L8Iqdm013128; Fri, 21 Feb 2014 01:18:52 -0700 From: Michael Roth To: qemu-devel@nongnu.org Date: Fri, 21 Feb 2014 02:17:17 -0600 Message-Id: <1392970647-21528-42-git-send-email-mdroth@linux.vnet.ibm.com> X-Mailer: git-send-email 1.7.9.5 In-Reply-To: <1392970647-21528-1-git-send-email-mdroth@linux.vnet.ibm.com> References: <1392970647-21528-1-git-send-email-mdroth@linux.vnet.ibm.com> X-TM-AS-MML: disable X-Content-Scanned: Fidelis XPS MAILER x-cbid: 14022108-3532-0000-0000-000005DFD44E X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.4.x-2.6.x [generic] X-Received-From: 32.97.110.154 Cc: lersek@redhat.com, qemu-stable@nongnu.org, Petar.Jovanovic@imgtec.com Subject: [Qemu-devel] [PATCH 41/51] block/curl: Implement the libcurl timer callback interface X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: , List-Help: , List-Subscribe: , Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: qemu-devel-bounces+patch=linaro.org@nongnu.org X-Removed-Original-Auth: Dkim didn't pass. X-Original-Sender: mdroth@linux.vnet.ibm.com X-Original-Authentication-Results: mx.google.com; spf=neutral (google.com: 209.85.220.178 is neither permitted nor denied by best guess record for domain of patch+caf_=patchwork-forward=linaro.org@linaro.org) smtp.mail=patch+caf_=patchwork-forward=linaro.org@linaro.org Mailing-list: list patchwork-forward@linaro.org; contact patchwork-forward+owners@linaro.org X-Google-Group-Id: 836684582541 From: Peter Maydell libcurl versions 7.16.0 and later have a timer callback interface which must be implemented in order for libcurl to make forward progress (it will sometimes rely on being called back on the timeout if there are no file descriptors registered). Implement the callback, and use a QEMU AIO timer to ensure we prod libcurl again when it asks us to. Based on Peter's original patch plus my fix to add curl_multi_timeout_do. Should compile just fine even on older versions of libcurl. I also tried copy-on-read and streaming: $ ./qemu-img create -f qcow2 -o \ backing_file=http://download.fedoraproject.org/pub/fedora/linux/releases/20/Live/x86_64/Fedora-Live-Desktop-x86_64-20-1.iso \ foo.qcow2 1G $ x86_64-softmmu/qemu-system-x86_64 \ -drive if=none,file=foo.qcow2,copy-on-read=on,id=cd \ -device ide-cd,drive=cd --enable-kvm -m 1024 Direct http usage is probably too slow, but with copy-on-read ultimately the image does boot! After some time, streaming gets canceled by an EIO, which needs further investigation. Signed-off-by: Peter Maydell Signed-off-by: Paolo Bonzini Signed-off-by: Kevin Wolf (cherry picked from commit 031fd1be5618c347f9aeb44ec294f14a541e42b2) Signed-off-by: Michael Roth --- block/curl.c | 81 ++++++++++++++++++++++++++++++++++++++++++++++++++-------- 1 file changed, 70 insertions(+), 11 deletions(-) diff --git a/block/curl.c b/block/curl.c index 5a46f97..1c04dcc 100644 --- a/block/curl.c +++ b/block/curl.c @@ -34,6 +34,11 @@ #define DPRINTF(fmt, ...) do { } while (0) #endif +#if LIBCURL_VERSION_NUM >= 0x071000 +/* The multi interface timer callback was introduced in 7.16.0 */ +#define NEED_CURL_TIMER_CALLBACK +#endif + #define PROTOCOLS (CURLPROTO_HTTP | CURLPROTO_HTTPS | \ CURLPROTO_FTP | CURLPROTO_FTPS | \ CURLPROTO_TFTP) @@ -77,6 +82,7 @@ typedef struct CURLState typedef struct BDRVCURLState { CURLM *multi; + QEMUTimer timer; size_t len; CURLState states[CURL_NUM_STATES]; char *url; @@ -87,6 +93,23 @@ typedef struct BDRVCURLState { static void curl_clean_state(CURLState *s); static void curl_multi_do(void *arg); +#ifdef NEED_CURL_TIMER_CALLBACK +static int curl_timer_cb(CURLM *multi, long timeout_ms, void *opaque) +{ + BDRVCURLState *s = opaque; + + DPRINTF("CURL: timer callback timeout_ms %ld\n", timeout_ms); + if (timeout_ms == -1) { + timer_del(&s->timer); + } else { + int64_t timeout_ns = (int64_t)timeout_ms * 1000 * 1000; + timer_mod(&s->timer, + qemu_clock_get_ns(QEMU_CLOCK_REALTIME) + timeout_ns); + } + return 0; +} +#endif + static int curl_sock_cb(CURL *curl, curl_socket_t fd, int action, void *s, void *sp) { @@ -209,20 +232,10 @@ static int curl_find_buf(BDRVCURLState *s, size_t start, size_t len, return FIND_RET_NONE; } -static void curl_multi_do(void *arg) +static void curl_multi_read(BDRVCURLState *s) { - BDRVCURLState *s = (BDRVCURLState *)arg; - int running; - int r; int msgs_in_queue; - if (!s->multi) - return; - - do { - r = curl_multi_socket_all(s->multi, &running); - } while(r == CURLM_CALL_MULTI_PERFORM); - /* Try to find done transfers, so we can free the easy * handle again. */ do { @@ -266,6 +279,41 @@ static void curl_multi_do(void *arg) } while(msgs_in_queue); } +static void curl_multi_do(void *arg) +{ + BDRVCURLState *s = (BDRVCURLState *)arg; + int running; + int r; + + if (!s->multi) { + return; + } + + do { + r = curl_multi_socket_all(s->multi, &running); + } while(r == CURLM_CALL_MULTI_PERFORM); + + curl_multi_read(s); +} + +static void curl_multi_timeout_do(void *arg) +{ +#ifdef NEED_CURL_TIMER_CALLBACK + BDRVCURLState *s = (BDRVCURLState *)arg; + int running; + + if (!s->multi) { + return; + } + + curl_multi_socket_action(s->multi, CURL_SOCKET_TIMEOUT, 0, &running); + + curl_multi_read(s); +#else + abort(); +#endif +} + static CURLState *curl_init_state(BDRVCURLState *s) { CURLState *state = NULL; @@ -473,12 +521,20 @@ static int curl_open(BlockDriverState *bs, QDict *options, int flags, curl_easy_cleanup(state->curl); state->curl = NULL; + aio_timer_init(bdrv_get_aio_context(bs), &s->timer, + QEMU_CLOCK_REALTIME, SCALE_NS, + curl_multi_timeout_do, s); + // Now we know the file exists and its size, so let's // initialize the multi interface! s->multi = curl_multi_init(); curl_multi_setopt(s->multi, CURLMOPT_SOCKETDATA, s); curl_multi_setopt(s->multi, CURLMOPT_SOCKETFUNCTION, curl_sock_cb); +#ifdef NEED_CURL_TIMER_CALLBACK + curl_multi_setopt(s->multi, CURLMOPT_TIMERDATA, s); + curl_multi_setopt(s->multi, CURLMOPT_TIMERFUNCTION, curl_timer_cb); +#endif curl_multi_do(s); qemu_opts_del(opts); @@ -597,6 +653,9 @@ static void curl_close(BlockDriverState *bs) } if (s->multi) curl_multi_cleanup(s->multi); + + timer_del(&s->timer); + g_free(s->url); }