From patchwork Mon Jan 23 15:38:37 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Christophe Milard X-Patchwork-Id: 92224 Delivered-To: patch@linaro.org Received: by 10.140.20.99 with SMTP id 90csp1255507qgi; Mon, 23 Jan 2017 06:40:17 -0800 (PST) X-Received: by 10.107.147.134 with SMTP id v128mr22696501iod.75.1485182417786; Mon, 23 Jan 2017 06:40:17 -0800 (PST) Return-Path: Received: from lists.linaro.org (lists.linaro.org. [54.225.227.206]) by mx.google.com with ESMTP id q15si8917779ite.116.2017.01.23.06.40.17; Mon, 23 Jan 2017 06:40:17 -0800 (PST) Received-SPF: pass (google.com: domain of lng-odp-bounces@lists.linaro.org designates 54.225.227.206 as permitted sender) client-ip=54.225.227.206; Authentication-Results: mx.google.com; spf=pass (google.com: domain of lng-odp-bounces@lists.linaro.org designates 54.225.227.206 as permitted sender) smtp.mailfrom=lng-odp-bounces@lists.linaro.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: by lists.linaro.org (Postfix, from userid 109) id 1945160643; Mon, 23 Jan 2017 14:40:17 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on ip-10-142-244-252 X-Spam-Level: X-Spam-Status: No, score=-2.6 required=5.0 tests=BAYES_00, RCVD_IN_DNSWL_LOW, RCVD_IN_MSPIKE_H3, RCVD_IN_MSPIKE_WL, URIBL_BLOCKED autolearn=disabled version=3.4.0 Received: from [127.0.0.1] (localhost [127.0.0.1]) by lists.linaro.org (Postfix) with ESMTP id 1CBBD60851; Mon, 23 Jan 2017 14:40:08 +0000 (UTC) X-Original-To: lng-odp@lists.linaro.org Delivered-To: lng-odp@lists.linaro.org Received: by lists.linaro.org (Postfix, from userid 109) id DC04660851; Mon, 23 Jan 2017 14:40:03 +0000 (UTC) Received: from mail-lf0-f49.google.com (mail-lf0-f49.google.com [209.85.215.49]) by lists.linaro.org (Postfix) with ESMTPS id AC0DF60F04 for ; Mon, 23 Jan 2017 14:39:30 +0000 (UTC) Received: by mail-lf0-f49.google.com with SMTP id n124so94283380lfd.2 for ; Mon, 23 Jan 2017 06:39:30 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id; bh=JAtrKyNbkdyagELFxASkRxJ87f2MPy//9eaVLYrPK7E=; b=l2wB4kkAsWHkGGWJ9QtXiuWBv//MywiHpZ6eRBGVfdYkknVt81GWW/gr9ihW2AZSWh 2g0AfnmOdrsvnKdprxsdysXlhIPD4nCbBplt+ynghnrynxH7ry6rJ22n7nxVYdbLXRCd t8cJoCZ4niiy6uMippc+b5c476KQcu9J+U7E8O18QNmUtwyEZHeeIdy0ukO4ByTIw45o Iu+zA4E1nz3wn+/CF3NrNBhiI6sdu8XV5CDGptgZjalUYtslK0SbAHBQW7eFPvRUfzys sz1BHtEhckiJOfGO/usHssIMvGBQKfRU/HKinS7cuxF3+5la9kzyt9pnqDCLdexr9a0U x8Kw== X-Gm-Message-State: AIkVDXIipPNh7uGXP2/8wcfZHP4EkBRTeOtk9AqIjRFkcBjuqzBPESCVBltnLT8Ox/yL9iRgr2I= X-Received: by 10.25.15.68 with SMTP id e65mr9554045lfi.117.1485182369293; Mon, 23 Jan 2017 06:39:29 -0800 (PST) Received: from erachmi-ericsson.ki.sw.ericsson.se (c-83-233-76-66.cust.bredband2.com. [83.233.76.66]) by smtp.gmail.com with ESMTPSA id f8sm6503609lji.2.2017.01.23.06.39.28 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Mon, 23 Jan 2017 06:39:28 -0800 (PST) From: Christophe Milard To: mike.holmes@linaro.org, bill.fischofer@linaro.org, lng-odp@lists.linaro.org Date: Mon, 23 Jan 2017 16:38:37 +0100 Message-Id: <1485185917-3677-1-git-send-email-christophe.milard@linaro.org> X-Mailer: git-send-email 2.7.4 Subject: [lng-odp] [PATCH] linux-gen: _ishmphy: fix possible race with malloc X-BeenThere: lng-odp@lists.linaro.org X-Mailman-Version: 2.1.16 Precedence: list List-Id: "The OpenDataPlane \(ODP\) List" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: lng-odp-bounces@lists.linaro.org Sender: "lng-odp" _ishm prereserves address space for _ODP_ISHM_SINGLE_VA allocated memory by reserving unreachable "memory", using PROT_NONE, hence really allocating virtual space (as the memory cannot be hit). The problem came when trying to use some of this preallocated space: strangely, if a new mapping on the preallocated virtual space failed (e.g. due to lack of huge pages), linux would returned MAP_FAILED (OK) but also cancel the previous PROT_NONE mapping, hence making the virtual memory space available for further mmaps. (code Point A) Before this patch, the code simply re-reserved (PROT_NONE) the area (point B): This was NOT OK: yes, _ishm_reserve calls are mutexed, so no other odpthread 2 could do a reserve between point A and B of opdthread1, but if thread2 did its own mmap(), malloc(),..., there was a chance for thread 2 to get virtual space from the preserved area (which should be blocked). This patch does not allow any A-B window by first doing an mmap (non fixed) and then performing a second mapping at the correct address (in the pre-reserved area), which is guaranteed to succeed, and finally removing the non-fixed mapping. The non-fix mapping just acts as a failure guard when the proper maping is done. Fixes https://bugs.linaro.org/show_bug.cgi?id=2834 (but very hard to trigger i.e. to prove) Signed-off-by: Christophe Milard --- platform/linux-generic/_ishmphy.c | 42 +++++++++++++++++++++++++++++---------- 1 file changed, 32 insertions(+), 10 deletions(-) -- 2.7.4 diff --git a/platform/linux-generic/_ishmphy.c b/platform/linux-generic/_ishmphy.c index 2b2d100..d519af6 100644 --- a/platform/linux-generic/_ishmphy.c +++ b/platform/linux-generic/_ishmphy.c @@ -94,7 +94,7 @@ int _odp_ishmphy_unbook_va(void) void *_odp_ishmphy_map(int fd, void *start, uint64_t size, int flags) { - void *mapped_addr; + void *mapped_addr_tmp, *mapped_addr; int mmap_flags = 0; if (flags & _ODP_ISHM_SINGLE_VA) { @@ -103,15 +103,37 @@ void *_odp_ishmphy_map(int fd, void *start, uint64_t size, return NULL; } /* maps over fragment of reserved VA: */ - mapped_addr = mmap(start, size, PROT_READ | PROT_WRITE, - MAP_SHARED | MAP_FIXED | mmap_flags, fd, 0); - /* if mapping fails, re-block the space we tried to take - * as it seems a mapping failure still affect what was there??*/ - if (mapped_addr == MAP_FAILED) { - mmap_flags = MAP_SHARED | MAP_FIXED | - MAP_ANONYMOUS | MAP_NORESERVE; - mmap(start, size, PROT_NONE, mmap_flags, -1, 0); - mprotect(start, size, PROT_NONE); + /* first, try a normal map. If that works, remap it where it + * should (on the prereverved space), and remove the initial + * normal mapping: + * This is because it turned out that if a mapping fails + * on a the prereserved virtual address space, then + * the prereserved address space which was tried to be mapped + * on becomes available to the kernel again! This was not + * according to expectations: the assumption was that if a + * mapping fails, the system should remain unchanged, but this + * is obvioulsy not true (at least for huge pages when + * exhausted). + * So the strategy is to first map at a non reserved place + * (which can then be freed and returned to the kernel on + * failure) and peform a new map to the prereserved space on + * success (which is then guaranteed to work). + * The initial free maping can then be removed. + */ + mapped_addr = MAP_FAILED; + mapped_addr_tmp = mmap(NULL, size, PROT_READ | PROT_WRITE, + MAP_SHARED | mmap_flags, fd, 0); + if (mapped_addr_tmp != MAP_FAILED) { + /* If OK, do new map at right fixed location... */ + mapped_addr = mmap(start, + size, PROT_READ | PROT_WRITE, + MAP_SHARED | MAP_FIXED | mmap_flags, + fd, 0); + if (mapped_addr != start) + ODP_ERR("new map failed:%s\n", strerror(errno)); + /* ... and remove initial mapping: */ + if (munmap(mapped_addr_tmp, size)) + ODP_ERR("munmap failed:%s\n", strerror(errno)); } } else { /* just do a new mapping in the VA space: */