From patchwork Fri Jan 24 17:21:43 2014 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kevin Wolf X-Patchwork-Id: 23689 Return-Path: X-Original-To: linaro@patches.linaro.org Delivered-To: linaro@patches.linaro.org Received: from mail-oa0-f71.google.com (mail-oa0-f71.google.com [209.85.219.71]) by ip-10-151-82-157.ec2.internal (Postfix) with ESMTPS id CCDC020307 for ; Fri, 24 Jan 2014 19:40:16 +0000 (UTC) Received: by mail-oa0-f71.google.com with SMTP id g12sf13687619oah.2 for ; Fri, 24 Jan 2014 11:40:16 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:delivered-to:from:to:date :message-id:in-reply-to:references:cc:subject:precedence:list-id :list-unsubscribe:list-archive:list-post:list-help:list-subscribe :errors-to:sender:x-original-sender :x-original-authentication-results:mailing-list; bh=Tdq4C/dZI71bfKGBuHB0Wx2/iQUlZC2OjW5eiL0gRhI=; b=bPPScACDgRuS3ST4FjVBzmNzhAx5EeV956TViMgzCScuN6xHVoFI5Y/KIkkgx6gy24 GRciY7S8lKU7XrcSv4A/8Se4Y3Wl//WlkmRvFQ3HNaCmIW4ukNgh07FsLOhFaiCY+KH5 M3thlH3f2Gb/JZobObQXkI/lxasn6sWC5kWR44hxlPrueruEXkwRSZoGTM+bbLyfhSUi T+psVXfsm5WJSHjTsesO0V1BJdG10Wykhwnrn6Bl7i7uWq0k5r8zFsXICZSJcPe1f4T8 9CFjHX2UK7uoSLAhbIHoVny/wXLUBFmSqUCx0BnPjquEV/SbHKtlCQlgbQHUXjgeju3B MYUQ== X-Gm-Message-State: ALoCoQk326+2Avt1pmoHz6ARzwLe54UKmlZRIKxZ8OPgfyqoNJGptkpqg1RVG4RAT0XRhRoioo8M X-Received: by 10.182.246.39 with SMTP id xt7mr5978747obc.40.1390592415914; Fri, 24 Jan 2014 11:40:15 -0800 (PST) MIME-Version: 1.0 X-BeenThere: patchwork-forward@linaro.org Received: by 10.140.35.208 with SMTP id n74ls713775qgn.65.gmail; Fri, 24 Jan 2014 11:40:15 -0800 (PST) X-Received: by 10.221.37.1 with SMTP id tc1mr8489404vcb.32.1390592385747; Fri, 24 Jan 2014 11:39:45 -0800 (PST) Received: from mail-vb0-f47.google.com (mail-vb0-f47.google.com [209.85.212.47]) by mx.google.com with ESMTPS id gq1si1050199vec.23.2014.01.24.11.39.45 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Fri, 24 Jan 2014 11:39:45 -0800 (PST) Received-SPF: neutral (google.com: 209.85.212.47 is neither permitted nor denied by best guess record for domain of patch+caf_=patchwork-forward=linaro.org@linaro.org) client-ip=209.85.212.47; Received: by mail-vb0-f47.google.com with SMTP id p6so2106702vbe.34 for ; Fri, 24 Jan 2014 11:39:45 -0800 (PST) X-Received: by 10.221.2.138 with SMTP id nu10mr448928vcb.52.1390592385664; Fri, 24 Jan 2014 11:39:45 -0800 (PST) X-Forwarded-To: patchwork-forward@linaro.org X-Forwarded-For: patch@linaro.org patchwork-forward@linaro.org Delivered-To: patch@linaro.org Received: by 10.220.174.196 with SMTP id u4csp1162vcz; Fri, 24 Jan 2014 11:39:45 -0800 (PST) X-Received: by 10.68.162.131 with SMTP id ya3mr16126127pbb.102.1390592354540; Fri, 24 Jan 2014 11:39:14 -0800 (PST) Received: from lists.gnu.org (lists.gnu.org. [2001:4830:134:3::11]) by mx.google.com with ESMTPS id kr4si4163728bkb.137.2014.01.24.11.39.13 for (version=TLSv1 cipher=RC4-SHA bits=128/128); Fri, 24 Jan 2014 11:39:14 -0800 (PST) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 2001:4830:134:3::11 as permitted sender) client-ip=2001:4830:134:3::11; Received: from localhost ([::1]:48660 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1W6mau-0005Dm-Tb for patch@linaro.org; Fri, 24 Jan 2014 14:39:12 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:53245) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1W6kV4-0003oN-Sa for qemu-devel@nongnu.org; Fri, 24 Jan 2014 12:25:07 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1W6kV0-00082T-IZ for qemu-devel@nongnu.org; Fri, 24 Jan 2014 12:25:02 -0500 Received: from mx1.redhat.com ([209.132.183.28]:47351) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1W6kV0-000829-92 for qemu-devel@nongnu.org; Fri, 24 Jan 2014 12:24:58 -0500 Received: from int-mx01.intmail.prod.int.phx2.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11]) by mx1.redhat.com (8.14.4/8.14.4) with ESMTP id s0OHOvgs029949 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=OK); Fri, 24 Jan 2014 12:24:57 -0500 Received: from dhcp-200-207.str.redhat.com (ovpn-116-95.ams2.redhat.com [10.36.116.95]) by int-mx01.intmail.prod.int.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id s0OHMRR0012974; Fri, 24 Jan 2014 12:24:52 -0500 From: Kevin Wolf To: anthony@codemonkey.ws Date: Fri, 24 Jan 2014 18:21:43 +0100 Message-Id: <1390584136-24703-61-git-send-email-kwolf@redhat.com> In-Reply-To: <1390584136-24703-1-git-send-email-kwolf@redhat.com> References: <1390584136-24703-1-git-send-email-kwolf@redhat.com> X-Scanned-By: MIMEDefang 2.67 on 10.5.11.11 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x X-Received-From: 209.132.183.28 Cc: kwolf@redhat.com, qemu-devel@nongnu.org Subject: [Qemu-devel] [PULL 60/93] block/curl: Implement the libcurl timer callback interface X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: , List-Help: , List-Subscribe: , Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: qemu-devel-bounces+patch=linaro.org@nongnu.org X-Removed-Original-Auth: Dkim didn't pass. X-Original-Sender: kwolf@redhat.com X-Original-Authentication-Results: mx.google.com; spf=neutral (google.com: 209.85.212.47 is neither permitted nor denied by best guess record for domain of patch+caf_=patchwork-forward=linaro.org@linaro.org) smtp.mail=patch+caf_=patchwork-forward=linaro.org@linaro.org Mailing-list: list patchwork-forward@linaro.org; contact patchwork-forward+owners@linaro.org X-Google-Group-Id: 836684582541 From: Peter Maydell libcurl versions 7.16.0 and later have a timer callback interface which must be implemented in order for libcurl to make forward progress (it will sometimes rely on being called back on the timeout if there are no file descriptors registered). Implement the callback, and use a QEMU AIO timer to ensure we prod libcurl again when it asks us to. Based on Peter's original patch plus my fix to add curl_multi_timeout_do. Should compile just fine even on older versions of libcurl. I also tried copy-on-read and streaming: $ ./qemu-img create -f qcow2 -o \ backing_file=http://download.fedoraproject.org/pub/fedora/linux/releases/20/Live/x86_64/Fedora-Live-Desktop-x86_64-20-1.iso \ foo.qcow2 1G $ x86_64-softmmu/qemu-system-x86_64 \ -drive if=none,file=foo.qcow2,copy-on-read=on,id=cd \ -device ide-cd,drive=cd --enable-kvm -m 1024 Direct http usage is probably too slow, but with copy-on-read ultimately the image does boot! After some time, streaming gets canceled by an EIO, which needs further investigation. Signed-off-by: Peter Maydell Signed-off-by: Paolo Bonzini Signed-off-by: Kevin Wolf --- block/curl.c | 81 +++++++++++++++++++++++++++++++++++++++++++++++++++--------- 1 file changed, 70 insertions(+), 11 deletions(-) diff --git a/block/curl.c b/block/curl.c index a603936..a807584 100644 --- a/block/curl.c +++ b/block/curl.c @@ -34,6 +34,11 @@ #define DPRINTF(fmt, ...) do { } while (0) #endif +#if LIBCURL_VERSION_NUM >= 0x071000 +/* The multi interface timer callback was introduced in 7.16.0 */ +#define NEED_CURL_TIMER_CALLBACK +#endif + #define PROTOCOLS (CURLPROTO_HTTP | CURLPROTO_HTTPS | \ CURLPROTO_FTP | CURLPROTO_FTPS | \ CURLPROTO_TFTP) @@ -77,6 +82,7 @@ typedef struct CURLState typedef struct BDRVCURLState { CURLM *multi; + QEMUTimer timer; size_t len; CURLState states[CURL_NUM_STATES]; char *url; @@ -87,6 +93,23 @@ typedef struct BDRVCURLState { static void curl_clean_state(CURLState *s); static void curl_multi_do(void *arg); +#ifdef NEED_CURL_TIMER_CALLBACK +static int curl_timer_cb(CURLM *multi, long timeout_ms, void *opaque) +{ + BDRVCURLState *s = opaque; + + DPRINTF("CURL: timer callback timeout_ms %ld\n", timeout_ms); + if (timeout_ms == -1) { + timer_del(&s->timer); + } else { + int64_t timeout_ns = (int64_t)timeout_ms * 1000 * 1000; + timer_mod(&s->timer, + qemu_clock_get_ns(QEMU_CLOCK_REALTIME) + timeout_ns); + } + return 0; +} +#endif + static int curl_sock_cb(CURL *curl, curl_socket_t fd, int action, void *s, void *sp) { @@ -209,20 +232,10 @@ static int curl_find_buf(BDRVCURLState *s, size_t start, size_t len, return FIND_RET_NONE; } -static void curl_multi_do(void *arg) +static void curl_multi_read(BDRVCURLState *s) { - BDRVCURLState *s = (BDRVCURLState *)arg; - int running; - int r; int msgs_in_queue; - if (!s->multi) - return; - - do { - r = curl_multi_socket_all(s->multi, &running); - } while(r == CURLM_CALL_MULTI_PERFORM); - /* Try to find done transfers, so we can free the easy * handle again. */ do { @@ -266,6 +279,41 @@ static void curl_multi_do(void *arg) } while(msgs_in_queue); } +static void curl_multi_do(void *arg) +{ + BDRVCURLState *s = (BDRVCURLState *)arg; + int running; + int r; + + if (!s->multi) { + return; + } + + do { + r = curl_multi_socket_all(s->multi, &running); + } while(r == CURLM_CALL_MULTI_PERFORM); + + curl_multi_read(s); +} + +static void curl_multi_timeout_do(void *arg) +{ +#ifdef NEED_CURL_TIMER_CALLBACK + BDRVCURLState *s = (BDRVCURLState *)arg; + int running; + + if (!s->multi) { + return; + } + + curl_multi_socket_action(s->multi, CURL_SOCKET_TIMEOUT, 0, &running); + + curl_multi_read(s); +#else + abort(); +#endif +} + static CURLState *curl_init_state(BDRVCURLState *s) { CURLState *state = NULL; @@ -473,12 +521,20 @@ static int curl_open(BlockDriverState *bs, QDict *options, int flags, curl_easy_cleanup(state->curl); state->curl = NULL; + aio_timer_init(bdrv_get_aio_context(bs), &s->timer, + QEMU_CLOCK_REALTIME, SCALE_NS, + curl_multi_timeout_do, s); + // Now we know the file exists and its size, so let's // initialize the multi interface! s->multi = curl_multi_init(); curl_multi_setopt(s->multi, CURLMOPT_SOCKETDATA, s); curl_multi_setopt(s->multi, CURLMOPT_SOCKETFUNCTION, curl_sock_cb); +#ifdef NEED_CURL_TIMER_CALLBACK + curl_multi_setopt(s->multi, CURLMOPT_TIMERDATA, s); + curl_multi_setopt(s->multi, CURLMOPT_TIMERFUNCTION, curl_timer_cb); +#endif curl_multi_do(s); qemu_opts_del(opts); @@ -597,6 +653,9 @@ static void curl_close(BlockDriverState *bs) } if (s->multi) curl_multi_cleanup(s->multi); + + timer_del(&s->timer); + g_free(s->url); }