diff mbox

[PULL,5/5] linux-user: Handle brk() attempts with very large sizes

Message ID ef4330c23bb47b97a859dbdbae1c784fd2ca402f.1470319929.git.riku.voipio@linaro.org
State Accepted
Commit ef4330c23bb47b97a859dbdbae1c784fd2ca402f
Headers show

Commit Message

Riku Voipio Aug. 4, 2016, 2:15 p.m. UTC 
From: Peter Maydell <peter.maydell@linaro.org>


In do_brk(), we were inadvertently truncating the size
of a requested brk() from the guest by putting it into an
'int' variable. This meant that we would incorrectly report
success back to the guest rather than a failed allocation,
typically resulting in the guest then segfaulting. Use
abi_ulong instead.

This fixes a crash in the '31370.cc' test in the gcc libstdc++ test
suite (the test case starts by trying to allocate a very large
size and reduces the size until the allocation succeeds).

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

Signed-off-by: Riku Voipio <riku.voipio@linaro.org>

---
 linux-user/syscall.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

-- 
2.1.4
diff mbox

Patch

diff --git a/linux-user/syscall.c b/linux-user/syscall.c
index df6f2a9..833f853 100644
--- a/linux-user/syscall.c
+++ b/linux-user/syscall.c
@@ -839,7 +839,7 @@  void target_set_brk(abi_ulong new_brk)
 abi_long do_brk(abi_ulong new_brk)
 {
     abi_long mapped_addr;
-    int	new_alloc_size;
+    abi_ulong new_alloc_size;
 
     DEBUGF_BRK("do_brk(" TARGET_ABI_FMT_lx ") -> ", new_brk);