From patchwork Fri Jan 24 13:56:17 2014 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paolo Bonzini X-Patchwork-Id: 23676 Return-Path: X-Original-To: linaro@patches.linaro.org Delivered-To: linaro@patches.linaro.org Received: from mail-ee0-f71.google.com (mail-ee0-f71.google.com [74.125.83.71]) by ip-10-151-82-157.ec2.internal (Postfix) with ESMTPS id 7149E203C5 for ; Fri, 24 Jan 2014 13:57:15 +0000 (UTC) Received: by mail-ee0-f71.google.com with SMTP id t10sf4027102eei.2 for ; Fri, 24 Jan 2014 05:57:14 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:delivered-to:from:to:date :message-id:cc:subject:precedence:list-id:list-unsubscribe :list-archive:list-post:list-help:list-subscribe:errors-to:sender :x-original-sender:x-original-authentication-results:mailing-list; bh=9/GxyRw7DWzWLqU0oWjQXPK0jTgf9x+CzRt+xfvRHCs=; b=OL+34AtTNlPfzcUYdyW1/Te9k6zyCS0ACK3M92n3IanCkTnuUU6rSHIuBAXZilzJQH 2hjcpcBuicLUjipxuGsJua/H3c8ae5ztoC48Y2kTQlxoXPaWxuZly2hOAX4fEV8c8ECU RJHDde5qECBKMz7GO3n1+xk6eVx7euoXVJg530+bGooW5ayI5SxeWioEK4MOcTOO+tD/ RGb2SxL59e2TJbuhkxi9RDKLllpB/cq1LnH7faxAlr52kYS9bK/NlTl3AyfwA4wWpiPa zMUo86dIZT7FkmqdN2TeX/R0qSmbrRo1wYH/6/ytzxQlwGYksrXW1SYsoq53VReQhl+m aMZw== X-Gm-Message-State: ALoCoQlO4RuWu+2SqCUAw4DA0jhpycSaXH38DxRhiM+Tmf90b/ggrPwnFPX8kMEP3FEkcE7eztnQ X-Received: by 10.152.210.197 with SMTP id mw5mr6551123lac.5.1390571833958; Fri, 24 Jan 2014 05:57:13 -0800 (PST) MIME-Version: 1.0 X-BeenThere: patchwork-forward@linaro.org Received: by 10.180.189.174 with SMTP id gj14ls390627wic.6.canary; Fri, 24 Jan 2014 05:57:13 -0800 (PST) X-Received: by 10.204.60.71 with SMTP id o7mr7833379bkh.33.1390571833775; Fri, 24 Jan 2014 05:57:13 -0800 (PST) Received: from mail-vc0-x22c.google.com (mail-vc0-x22c.google.com [2607:f8b0:400c:c03::22c]) by mx.google.com with ESMTPS id kw2si3127684bkb.263.2014.01.24.05.57.13 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Fri, 24 Jan 2014 05:57:13 -0800 (PST) Received-SPF: neutral (google.com: 2607:f8b0:400c:c03::22c is neither permitted nor denied by best guess record for domain of patch+caf_=patchwork-forward=linaro.org@linaro.org) client-ip=2607:f8b0:400c:c03::22c; Received: by mail-vc0-f172.google.com with SMTP id lf12so1902997vcb.3 for ; Fri, 24 Jan 2014 05:57:12 -0800 (PST) X-Received: by 10.220.191.134 with SMTP id dm6mr7795965vcb.16.1390571832560; Fri, 24 Jan 2014 05:57:12 -0800 (PST) X-Forwarded-To: patchwork-forward@linaro.org X-Forwarded-For: patch@linaro.org patchwork-forward@linaro.org Delivered-To: patch@linaro.org Received: by 10.220.174.196 with SMTP id u4csp104048vcz; Fri, 24 Jan 2014 05:57:12 -0800 (PST) X-Received: by 10.205.77.131 with SMTP id zi3mr1305200bkb.101.1390571830615; Fri, 24 Jan 2014 05:57:10 -0800 (PST) Received: from lists.gnu.org (lists.gnu.org. [2001:4830:134:3::11]) by mx.google.com with ESMTPS id h4si3148471bkr.170.2014.01.24.05.57.09 for (version=TLSv1 cipher=RC4-SHA bits=128/128); Fri, 24 Jan 2014 05:57:10 -0800 (PST) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 2001:4830:134:3::11 as permitted sender) client-ip=2001:4830:134:3::11; Received: from localhost ([::1]:46704 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1W6hFt-0006kF-4P for patch@linaro.org; Fri, 24 Jan 2014 08:57:09 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:53370) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1W6hFM-0006NA-7s for qemu-devel@nongnu.org; Fri, 24 Jan 2014 08:56:41 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1W6hFH-0005Lr-Lg for qemu-devel@nongnu.org; Fri, 24 Jan 2014 08:56:36 -0500 Received: from mail-qa0-x231.google.com ([2607:f8b0:400d:c00::231]:51723) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1W6hFH-0005Lk-GY for qemu-devel@nongnu.org; Fri, 24 Jan 2014 08:56:31 -0500 Received: by mail-qa0-f49.google.com with SMTP id w8so3837927qac.22 for ; Fri, 24 Jan 2014 05:56:29 -0800 (PST) X-Received: by 10.224.97.7 with SMTP id j7mr20632992qan.81.1390571789866; Fri, 24 Jan 2014 05:56:29 -0800 (PST) Received: from yakj.usersys.redhat.com (net-2-35-197-229.cust.dsl.vodafone.it. [2.35.197.229]) by mx.google.com with ESMTPSA id z9sm640041qgz.20.2014.01.24.05.56.27 for (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 24 Jan 2014 05:56:28 -0800 (PST) From: Paolo Bonzini To: qemu-devel@nongnu.org Date: Fri, 24 Jan 2014 14:56:17 +0100 Message-Id: <1390571777-19098-1-git-send-email-pbonzini@redhat.com> X-Mailer: git-send-email 1.8.4.2 X-detected-operating-system: by eggs.gnu.org: Error: Malformed IPv6 address (bad octet value). X-Received-From: 2607:f8b0:400d:c00::231 Cc: kwolf@redhat.com, Peter Maydell , stefanha@redhat.com Subject: [Qemu-devel] [PATCH] block/curl: Implement the libcurl timer callback interface X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: , List-Help: , List-Subscribe: , Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: qemu-devel-bounces+patch=linaro.org@nongnu.org X-Original-Sender: pbonzini@redhat.com X-Original-Authentication-Results: mx.google.com; spf=neutral (google.com: 2607:f8b0:400c:c03::22c is neither permitted nor denied by best guess record for domain of patch+caf_=patchwork-forward=linaro.org@linaro.org) smtp.mail=patch+caf_=patchwork-forward=linaro.org@linaro.org; dkim=fail header.i=@gmail.com Mailing-list: list patchwork-forward@linaro.org; contact patchwork-forward+owners@linaro.org X-Google-Group-Id: 836684582541 From: Peter Maydell libcurl versions 7.16.0 and later have a timer callback interface which must be implemented in order for libcurl to make forward progress (it will sometimes rely on being called back on the timeout if there are no file descriptors registered). Implement the callback, and use a QEMU AIO timer to ensure we prod libcurl again when it asks us to. Based on Peter's original patch plus my fix to add curl_multi_timeout_do. Should compile just fine even on older versions of libcurl. I also tried copy-on-read and streaming: $ ./qemu-img create -f qcow2 -o \ backing_file=http://download.fedoraproject.org/pub/fedora/linux/releases/20/Live/x86_64/Fedora-Live-Desktop-x86_64-20-1.iso \ foo.qcow2 1G $ x86_64-softmmu/qemu-system-x86_64 \ -drive if=none,file=foo.qcow2,copy-on-read=on,id=cd \ -device ide-cd,drive=cd --enable-kvm -m 1024 Direct http usage is probably too slow, but with copy-on-read ultimately the image does boot! After some time, streaming gets canceled by an EIO, which needs further investigation. Signed-off-by: Peter Maydell Signed-off-by: Paolo Bonzini --- block/curl.c | 81 +++++++++++++++++++++++++++++++++++++++++++++++++++--------- 1 file changed, 70 insertions(+), 11 deletions(-) diff --git a/block/curl.c b/block/curl.c index a603936..a807584 100644 --- a/block/curl.c +++ b/block/curl.c @@ -34,6 +34,11 @@ #define DPRINTF(fmt, ...) do { } while (0) #endif +#if LIBCURL_VERSION_NUM >= 0x071000 +/* The multi interface timer callback was introduced in 7.16.0 */ +#define NEED_CURL_TIMER_CALLBACK +#endif + #define PROTOCOLS (CURLPROTO_HTTP | CURLPROTO_HTTPS | \ CURLPROTO_FTP | CURLPROTO_FTPS | \ CURLPROTO_TFTP) @@ -77,6 +82,7 @@ typedef struct CURLState typedef struct BDRVCURLState { CURLM *multi; + QEMUTimer timer; size_t len; CURLState states[CURL_NUM_STATES]; char *url; @@ -87,6 +93,23 @@ typedef struct BDRVCURLState { static void curl_clean_state(CURLState *s); static void curl_multi_do(void *arg); +#ifdef NEED_CURL_TIMER_CALLBACK +static int curl_timer_cb(CURLM *multi, long timeout_ms, void *opaque) +{ + BDRVCURLState *s = opaque; + + DPRINTF("CURL: timer callback timeout_ms %ld\n", timeout_ms); + if (timeout_ms == -1) { + timer_del(&s->timer); + } else { + int64_t timeout_ns = (int64_t)timeout_ms * 1000 * 1000; + timer_mod(&s->timer, + qemu_clock_get_ns(QEMU_CLOCK_REALTIME) + timeout_ns); + } + return 0; +} +#endif + static int curl_sock_cb(CURL *curl, curl_socket_t fd, int action, void *s, void *sp) { @@ -209,20 +232,10 @@ static int curl_find_buf(BDRVCURLState *s, size_t start, size_t len, return FIND_RET_NONE; } -static void curl_multi_do(void *arg) +static void curl_multi_read(BDRVCURLState *s) { - BDRVCURLState *s = (BDRVCURLState *)arg; - int running; - int r; int msgs_in_queue; - if (!s->multi) - return; - - do { - r = curl_multi_socket_all(s->multi, &running); - } while(r == CURLM_CALL_MULTI_PERFORM); - /* Try to find done transfers, so we can free the easy * handle again. */ do { @@ -266,6 +279,41 @@ static void curl_multi_do(void *arg) } while(msgs_in_queue); } +static void curl_multi_do(void *arg) +{ + BDRVCURLState *s = (BDRVCURLState *)arg; + int running; + int r; + + if (!s->multi) { + return; + } + + do { + r = curl_multi_socket_all(s->multi, &running); + } while(r == CURLM_CALL_MULTI_PERFORM); + + curl_multi_read(s); +} + +static void curl_multi_timeout_do(void *arg) +{ +#ifdef NEED_CURL_TIMER_CALLBACK + BDRVCURLState *s = (BDRVCURLState *)arg; + int running; + + if (!s->multi) { + return; + } + + curl_multi_socket_action(s->multi, CURL_SOCKET_TIMEOUT, 0, &running); + + curl_multi_read(s); +#else + abort(); +#endif +} + static CURLState *curl_init_state(BDRVCURLState *s) { CURLState *state = NULL; @@ -473,12 +521,20 @@ static int curl_open(BlockDriverState *bs, QDict *options, int flags, curl_easy_cleanup(state->curl); state->curl = NULL; + aio_timer_init(bdrv_get_aio_context(bs), &s->timer, + QEMU_CLOCK_REALTIME, SCALE_NS, + curl_multi_timeout_do, s); + // Now we know the file exists and its size, so let's // initialize the multi interface! s->multi = curl_multi_init(); curl_multi_setopt(s->multi, CURLMOPT_SOCKETDATA, s); curl_multi_setopt(s->multi, CURLMOPT_SOCKETFUNCTION, curl_sock_cb); +#ifdef NEED_CURL_TIMER_CALLBACK + curl_multi_setopt(s->multi, CURLMOPT_TIMERDATA, s); + curl_multi_setopt(s->multi, CURLMOPT_TIMERFUNCTION, curl_timer_cb); +#endif curl_multi_do(s); qemu_opts_del(opts); @@ -597,6 +653,9 @@ static void curl_close(BlockDriverState *bs) } if (s->multi) curl_multi_cleanup(s->multi); + + timer_del(&s->timer); + g_free(s->url); }