| Message ID | 1499703585-17933-1-git-send-email-adhemerval.zanella@linaro.org |
|---|---|
| State | New |
| Headers | show |
* Adhemerval Zanella: > POSIX specifies that the state and size after a fflush on a stream > opened by open_memstream should be updated and current implementation > does not act accordingly on a fflush (NULL) (meant to act on all opened > streams). > > This patch fixes it for open_{w}memstream and also adjust the > tst-memstream3 to use libsupport. Subject is wrong, it should be fflush. Please put the test case into a new file, and change it to call fflush (NULL); it currently calls fflush (fp). As a result, it does not actually test the change. You also need to verify that this does not call malloc: #include <stdio.h> #include <stdlib.h> #include <err.h> int main (void) { char * buffer = NULL; size_t length = 0; FILE *fp = open_memstream (&buffer, &length); if (fp == NULL) err (1, "open_memstream"); abort (); } I think we don't want the malloc call for a call to exit, either. Similarly for __stack_chk_fail, but that's something we need to fix there. > +static int > +_IO_mem_overflow (_IO_FILE *fp, int c) > +{ > + int ret = _IO_str_overflow (fp, c); > + > + struct _IO_FILE_memstream *mp = (struct _IO_FILE_memstream *) fp; > + *mp->sizeloc = fp->_IO_write_ptr - fp->_IO_write_base; > + > + return ret; > +} _IO_str_overflow does not correctly process an EOF argument, but _IO_flush_all_lockp calls the overflow function with EOF. This is the reason why the test with fflush (NULL) fails. The most direct fix appears to be to call _IO_mem_sync for c == EOF. Let me suggest again that libio changes are not appropriate during the freeze.
On 11/07/2017 20:40, Florian Weimer wrote: > * Adhemerval Zanella: > >> POSIX specifies that the state and size after a fflush on a stream >> opened by open_memstream should be updated and current implementation >> does not act accordingly on a fflush (NULL) (meant to act on all opened >> streams). >> >> This patch fixes it for open_{w}memstream and also adjust the >> tst-memstream3 to use libsupport. > > Subject is wrong, it should be fflush. Ack, fixed locally. > > Please put the test case into a new file, and change it to call fflush > (NULL); it currently calls fflush (fp). As a result, it does not > actually test the change. Indeed, I used you suggestion to call _IO_mem_sync for EOF. For tests, used the previous tst-{w}memstream3.c one to avoid all the boilerplate required to add a new tests, but I think adding a new one should be better to reference the bug and backport. > > You also need to verify that this does not call malloc: > > #include <stdio.h> > #include <stdlib.h> > #include <err.h> > > int > main (void) > { > char * buffer = NULL; > size_t length = 0; > FILE *fp = open_memstream (&buffer, &length); > if (fp == NULL) > err (1, "open_memstream"); > abort (); > } > > I think we don't want the malloc call for a call to exit, either. > Similarly for __stack_chk_fail, but that's something we need to fix > there. I am not following, what exactly should not call malloc here? Since the open_memstream is a valid call, it will allocate both the initial internal FILE object (libio/memstream.c:77) and initial buffer (libio/memstream.c:84). Are you saying that we should start with an empty buffer instead? If it is the for I think we can do it, but it would require more intrusive changes I think. > >> +static int >> +_IO_mem_overflow (_IO_FILE *fp, int c) >> +{ >> + int ret = _IO_str_overflow (fp, c); >> + >> + struct _IO_FILE_memstream *mp = (struct _IO_FILE_memstream *) fp; >> + *mp->sizeloc = fp->_IO_write_ptr - fp->_IO_write_base; >> + >> + return ret; >> +} > > _IO_str_overflow does not correctly process an EOF argument, but > _IO_flush_all_lockp calls the overflow function with EOF. This is the > reason why the test with fflush (NULL) fails. > > The most direct fix appears to be to call _IO_mem_sync for c == EOF. > > Let me suggest again that libio changes are not appropriate during the > freeze. > So do you prefer to just add a workaround to disable the single-thread optimization for open_memstream and fix it on 2.27? IMHO this fix touches only the open_memstream itself, so it should not regress any other libio code so I would prefer than carry a workaround for 2.26. In any case, I updated the patch based on your review below. It passed both your threaded case for open_memstream that is failing on master the testcase for BZ#21735. --- [BZ #21735] * libio/memstream.c (_IO_mem_overflow): New function. (_IO_mem_jumps): User _IO_mem_overflow instead of _IO_str_overflow. (__open_memstream): Call _IO_link_in on the internal FILE object. * libio/wmemstream.c (_IO_wmem_overflow): New function. (_IO_wmem_jumps): User _IO_wmem_overflow instead of _IO_wstr_overflow. (__open_wmemstream): Call _IO_link_in on the internal FILE object. * libio/Makefile (test): Add tst-memstream4 and tst-wmemstream4. * libio/tst-memstream4.c: New file. * libio/tst-wmemstream4.c: Likewise. * libio/tst-memstream.h: Likewise. --- -- 2.7.4diff --git a/libio/Makefile b/libio/Makefile index a002a33..6ee809a 100644 --- a/libio/Makefile +++ b/libio/Makefile @@ -57,8 +57,8 @@ tests = tst_swprintf tst_wprintf tst_swscanf tst_wscanf tst_getwc tst_putwc \ tst-mmap-eofsync tst-mmap-fflushsync bug-mmap-fflush \ tst-mmap2-eofsync tst-mmap-offend bug-fopena+ bug-wfflush \ bug-ungetc2 bug-ftell bug-ungetc3 bug-ungetc4 tst-fopenloc2 \ - tst-memstream1 tst-memstream2 tst-memstream3 \ - tst-wmemstream1 tst-wmemstream2 tst-wmemstream3 \ + tst-memstream1 tst-memstream2 tst-memstream3 tst-memstream4 \ + tst-wmemstream1 tst-wmemstream2 tst-wmemstream3 tst-wmemstream4 \ bug-memstream1 bug-wmemstream1 \ tst-setvbuf1 tst-popen1 tst-fgetwc bug-wsetpos tst-fseek \ tst-fwrite-error tst-ftell-partial-wide tst-ftell-active-handler \ diff --git a/libio/memstream.c b/libio/memstream.c index f83d4a5..5fdd071 100644 --- a/libio/memstream.c +++ b/libio/memstream.c @@ -31,13 +31,14 @@ struct _IO_FILE_memstream static int _IO_mem_sync (_IO_FILE* fp) __THROW; static void _IO_mem_finish (_IO_FILE* fp, int) __THROW; +static int _IO_mem_overflow (_IO_FILE *fp, int c) __THROW; static const struct _IO_jump_t _IO_mem_jumps libio_vtable = { JUMP_INIT_DUMMY, JUMP_INIT (finish, _IO_mem_finish), - JUMP_INIT (overflow, _IO_str_overflow), + JUMP_INIT (overflow, _IO_mem_overflow), JUMP_INIT (underflow, _IO_str_underflow), JUMP_INIT (uflow, _IO_default_uflow), JUMP_INIT (pbackfail, _IO_str_pbackfail), @@ -87,6 +88,7 @@ __open_memstream (char **bufloc, _IO_size_t *sizeloc) return NULL; } _IO_init_internal (&new_f->fp._sf._sbf._f, 0); + _IO_link_in ((struct _IO_FILE_plus *) &new_f->fp._sf._sbf); _IO_JUMPS_FILE_plus (&new_f->fp._sf._sbf) = &_IO_mem_jumps; _IO_str_init_static_internal (&new_f->fp._sf, buf, _IO_BUFSIZ, buf); new_f->fp._sf._sbf._f._flags &= ~_IO_USER_BUF; @@ -137,3 +139,12 @@ _IO_mem_finish (_IO_FILE *fp, int dummy) _IO_str_finish (fp, 0); } + +static int +_IO_mem_overflow (_IO_FILE *fp, int c) +{ + /* Updates the returned size location on stream flush. */ + if (c == EOF) + return _IO_mem_sync (fp); + return _IO_str_overflow (fp, c); +} diff --git a/libio/tst-memstream.h b/libio/tst-memstream.h new file mode 100644 index 0000000..249a588 --- /dev/null +++ b/libio/tst-memstream.h @@ -0,0 +1,64 @@ +/* Common definitions for open_memstream tests. + Copyright (C) 2017 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + <http://www.gnu.org/licenses/>. */ + +#include <mcheck.h> +#include <stdio.h> +#include <stdlib.h> +#include <string.h> +#include <stdarg.h> +#include <errno.h> + +#include <support/check.h> + +#ifdef TEST_WCHAR +# include <wchar.h> + +/* Straighforward implementation so tst-memstream3 could use check + fwrite on open_memstream. */ +static size_t __attribute__ ((used)) +fwwrite (const void *ptr, size_t size, size_t nmemb, FILE *arq) +{ + const wchar_t *wcs = (const wchar_t*) (ptr); + for (size_t s = 0; s < size; s++) + { + for (size_t n = 0; n < nmemb; n++) + if (fputwc (wcs[n], arq) == WEOF) + return n; + } + return size * nmemb; +} + +# define CHAR_T wchar_t +# define W(o) L##o +# define OPEN_MEMSTREAM open_wmemstream +# define PRINTF wprintf +# define FWRITE fwwrite +# define FPUTC fputwc +# define STRCMP wcscmp +#else +# define CHAR_T char +# define W(o) o +# define OPEN_MEMSTREAM open_memstream +# define PRINTF printf +# define FWRITE fwrite +# define FPUTC fputc +# define STRCMP strcmp +#endif + +#define S(s) S1 (s) +#define S1(s) #s diff --git a/libio/tst-memstream4.c b/libio/tst-memstream4.c new file mode 100644 index 0000000..d810063 --- /dev/null +++ b/libio/tst-memstream4.c @@ -0,0 +1,58 @@ +/* Test for open_memstream BZ #21735. + Copyright (C) 2017 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + <http://www.gnu.org/licenses/>. */ + +#include "tst-memstream.h" + +static void +mcheck_abort (enum mcheck_status ev) +{ + printf ("mecheck failed with status %d\n", (int) ev); + exit (1); +} + +static int +do_test (void) +{ + mcheck_pedantic (mcheck_abort); + + CHAR_T *buf; + size_t size; + + FILE *fp = OPEN_MEMSTREAM (&buf, &size); + if (fp == NULL) + FAIL_RET ("%s failed", S(OPEN_MEMSTREAM)); + + if (FPUTC (W('a'), fp) != W('a')) + FAIL_RET ("%s failed: %m", S(FPUTC)); + + if (fflush (NULL) != 0) + FAIL_RET ("fflush failed: %m"); + + if (size != 1) + FAIL_RET ("size != 1"); + + if (fileno (fp) != -1) + FAIL_RET ("fileno returned a valid integer descriptor"); + + if (fclose (fp) != 0) + FAIL_RET ("fclose: %m"); + + return 0; +} + +#include <support/test-driver.c> diff --git a/libio/tst-wmemstream4.c b/libio/tst-wmemstream4.c new file mode 100644 index 0000000..9dbf615 --- /dev/null +++ b/libio/tst-wmemstream4.c @@ -0,0 +1,20 @@ +/* Test for open_wmemstream BZ #21735. + Copyright (C) 2017 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + <http://www.gnu.org/licenses/>. */ + +#define TEST_WCHAR +#include <libio/tst-memstream4.c> diff --git a/libio/wmemstream.c b/libio/wmemstream.c index 5bc77f5..3aa5f8a 100644 --- a/libio/wmemstream.c +++ b/libio/wmemstream.c @@ -32,13 +32,14 @@ struct _IO_FILE_wmemstream static int _IO_wmem_sync (_IO_FILE* fp) __THROW; static void _IO_wmem_finish (_IO_FILE* fp, int) __THROW; +static int _IO_wmem_overflow (_IO_FILE *fp, int c) __THROW; static const struct _IO_jump_t _IO_wmem_jumps libio_vtable = { JUMP_INIT_DUMMY, JUMP_INIT (finish, _IO_wmem_finish), - JUMP_INIT (overflow, (_IO_overflow_t) _IO_wstr_overflow), + JUMP_INIT (overflow, (_IO_overflow_t) _IO_wmem_overflow), JUMP_INIT (underflow, (_IO_underflow_t) _IO_wstr_underflow), JUMP_INIT (uflow, (_IO_underflow_t) _IO_wdefault_uflow), JUMP_INIT (pbackfail, (_IO_pbackfail_t) _IO_wstr_pbackfail), @@ -89,6 +90,7 @@ open_wmemstream (wchar_t **bufloc, _IO_size_t *sizeloc) } _IO_no_init (&new_f->fp._sf._sbf._f, 0, 0, &new_f->wd, &_IO_wmem_jumps); _IO_fwide (&new_f->fp._sf._sbf._f, 1); + _IO_link_in ((struct _IO_FILE_plus *) &new_f->fp._sf._sbf); _IO_wstr_init_static (&new_f->fp._sf._sbf._f, buf, _IO_BUFSIZ / sizeof (wchar_t), buf); new_f->fp._sf._sbf._f._flags2 &= ~_IO_FLAGS2_USER_WBUF; @@ -142,3 +144,12 @@ _IO_wmem_finish (_IO_FILE *fp, int dummy) _IO_wstr_finish (fp, 0); } + +static int +_IO_wmem_overflow (_IO_FILE *fp, int c) +{ + /* Updates the returned size location on stream flush. */ + if (c == EOF) + return _IO_wmem_sync (fp); + return _IO_wstr_overflow (fp, c); +}
* Adhemerval Zanella: >> I think we don't want the malloc call for a call to exit, either. >> Similarly for __stack_chk_fail, but that's something we need to fix >> there. > > I am not following, what exactly should not call malloc here? Since > the open_memstream is a valid call, it will allocate both the initial > internal FILE object (libio/memstream.c:77) and initial buffer > (libio/memstream.c:84). Are you saying that we should start with an > empty buffer instead? If it is the for I think we can do it, but it > would require more intrusive changes I think. I think we should make sure that writing the null terminator does not trigger malloc during abort (or __stack_chk_fail). That would be quite wrong IMHO. I wasn't aware that the buffer is pre-allocated. Then we probably need a test which is aware of the size of the preallocated buffer. > So do you prefer to just add a workaround to disable the single-thread > optimization for open_memstream and fix it on 2.27? IMHO this fix touches > only the open_memstream itself, so it should not regress any other libio > code so I would prefer than carry a workaround for 2.26. Which fix? Linking open_memstream into the global list has quite far-reaching consequences. Conservatively setting the flag instead should be fine. I'm also not sure that the intent of POSIX is that open_memstream streams should be affected by fflush (NULL). I expect that open_memstream was added without considering the interaction with fflush (NULL).
On 12/07/2017 12:01, Florian Weimer wrote: > * Adhemerval Zanella: > >>> I think we don't want the malloc call for a call to exit, either. >>> Similarly for __stack_chk_fail, but that's something we need to fix >>> there. >> >> I am not following, what exactly should not call malloc here? Since >> the open_memstream is a valid call, it will allocate both the initial >> internal FILE object (libio/memstream.c:77) and initial buffer >> (libio/memstream.c:84). Are you saying that we should start with an >> empty buffer instead? If it is the for I think we can do it, but it >> would require more intrusive changes I think. > > I think we should make sure that writing the null terminator does not > trigger malloc during abort (or __stack_chk_fail). That would be > quite wrong IMHO. > > I wasn't aware that the buffer is pre-allocated. Then we probably > need a test which is aware of the size of the preallocated buffer. Right, so in this case we can't call _IO_mem_sync for _IO_mem_overflow with EOF to avoid the possible malloc during abort. I updated the patch to check for correct buffer output in such case. > >> So do you prefer to just add a workaround to disable the single-thread >> optimization for open_memstream and fix it on 2.27? IMHO this fix touches >> only the open_memstream itself, so it should not regress any other libio >> code so I would prefer than carry a workaround for 2.26. > > Which fix? Linking open_memstream into the global list has quite > far-reaching consequences. Conservatively setting the flag instead > should be fine. I agree, but I also though adding the single-thread optimization close to freeze was also not the safe move. I do not have a strong preference here and I think we can delay this fix to 2.27. > > I'm also not sure that the intent of POSIX is that open_memstream > streams should be affected by fflush (NULL). I expect that > open_memstream was added without considering the interaction with > fflush (NULL). > I would say to not make the open_memstream stream an exception and have different semantics than other streams. --- -- 2.7.4diff --git a/libio/Makefile b/libio/Makefile index a002a33..6ee809a 100644 --- a/libio/Makefile +++ b/libio/Makefile @@ -57,8 +57,8 @@ tests = tst_swprintf tst_wprintf tst_swscanf tst_wscanf tst_getwc tst_putwc \ tst-mmap-eofsync tst-mmap-fflushsync bug-mmap-fflush \ tst-mmap2-eofsync tst-mmap-offend bug-fopena+ bug-wfflush \ bug-ungetc2 bug-ftell bug-ungetc3 bug-ungetc4 tst-fopenloc2 \ - tst-memstream1 tst-memstream2 tst-memstream3 \ - tst-wmemstream1 tst-wmemstream2 tst-wmemstream3 \ + tst-memstream1 tst-memstream2 tst-memstream3 tst-memstream4 \ + tst-wmemstream1 tst-wmemstream2 tst-wmemstream3 tst-wmemstream4 \ bug-memstream1 bug-wmemstream1 \ tst-setvbuf1 tst-popen1 tst-fgetwc bug-wsetpos tst-fseek \ tst-fwrite-error tst-ftell-partial-wide tst-ftell-active-handler \ diff --git a/libio/memstream.c b/libio/memstream.c index f83d4a5..67ecc09 100644 --- a/libio/memstream.c +++ b/libio/memstream.c @@ -31,13 +31,14 @@ struct _IO_FILE_memstream static int _IO_mem_sync (_IO_FILE* fp) __THROW; static void _IO_mem_finish (_IO_FILE* fp, int) __THROW; +static int _IO_mem_overflow (_IO_FILE *fp, int c) __THROW; static const struct _IO_jump_t _IO_mem_jumps libio_vtable = { JUMP_INIT_DUMMY, JUMP_INIT (finish, _IO_mem_finish), - JUMP_INIT (overflow, _IO_str_overflow), + JUMP_INIT (overflow, _IO_mem_overflow), JUMP_INIT (underflow, _IO_str_underflow), JUMP_INIT (uflow, _IO_default_uflow), JUMP_INIT (pbackfail, _IO_str_pbackfail), @@ -87,6 +88,7 @@ __open_memstream (char **bufloc, _IO_size_t *sizeloc) return NULL; } _IO_init_internal (&new_f->fp._sf._sbf._f, 0); + _IO_link_in ((struct _IO_FILE_plus *) &new_f->fp._sf._sbf); _IO_JUMPS_FILE_plus (&new_f->fp._sf._sbf) = &_IO_mem_jumps; _IO_str_init_static_internal (&new_f->fp._sf, buf, _IO_BUFSIZ, buf); new_f->fp._sf._sbf._f._flags &= ~_IO_USER_BUF; @@ -137,3 +139,17 @@ _IO_mem_finish (_IO_FILE *fp, int dummy) _IO_str_finish (fp, 0); } + +static int +_IO_mem_overflow (_IO_FILE *fp, int c) +{ + if (c == EOF) + { + /* Updates the returned size location on stream flush. */ + struct _IO_FILE_memstream *mp = (struct _IO_FILE_memstream *) fp; + *mp->bufloc = fp->_IO_write_base; + *mp->sizeloc = fp->_IO_write_ptr - fp->_IO_write_base; + return 0; + } + return _IO_str_overflow (fp, c); +} diff --git a/libio/tst-memstream.h b/libio/tst-memstream.h new file mode 100644 index 0000000..ffc2a65 --- /dev/null +++ b/libio/tst-memstream.h @@ -0,0 +1,66 @@ +/* Common definitions for open_memstream tests. + Copyright (C) 2017 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + <http://www.gnu.org/licenses/>. */ + +#include <mcheck.h> +#include <stdio.h> +#include <stdlib.h> +#include <string.h> +#include <stdarg.h> +#include <errno.h> + +#include <support/check.h> + +#ifdef TEST_WCHAR +# include <wchar.h> + +/* Straighforward implementation so tst-memstream3 could use check + fwrite on open_memstream. */ +static size_t __attribute__ ((used)) +fwwrite (const void *ptr, size_t size, size_t nmemb, FILE *arq) +{ + const wchar_t *wcs = (const wchar_t*) (ptr); + for (size_t s = 0; s < size; s++) + { + for (size_t n = 0; n < nmemb; n++) + if (fputwc (wcs[n], arq) == WEOF) + return n; + } + return size * nmemb; +} + +# define CHAR_T wchar_t +# define W(o) L##o +# define OPEN_MEMSTREAM open_wmemstream +# define PRINTF wprintf +# define FWRITE fwwrite +# define FPUTC fputwc +# define STRCMP wcscmp +# define STRLEN wcslen +#else +# define CHAR_T char +# define W(o) o +# define OPEN_MEMSTREAM open_memstream +# define PRINTF printf +# define FWRITE fwrite +# define FPUTC fputc +# define STRCMP strcmp +# define STRLEN strlen +#endif + +#define S(s) S1 (s) +#define S1(s) #s diff --git a/libio/tst-memstream4.c b/libio/tst-memstream4.c new file mode 100644 index 0000000..e98ca51 --- /dev/null +++ b/libio/tst-memstream4.c @@ -0,0 +1,99 @@ +/* Test for open_memstream BZ #21735. + Copyright (C) 2017 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + <http://www.gnu.org/licenses/>. */ + +#include "tst-memstream.h" + +static void +mcheck_abort (enum mcheck_status ev) +{ + printf ("mecheck failed with status %d\n", (int) ev); + exit (1); +} + +static int +do_test (void) +{ + mcheck_pedantic (mcheck_abort); + + CHAR_T *buf; + size_t size; + + FILE *fp = OPEN_MEMSTREAM (&buf, &size); + if (fp == NULL) + FAIL_RET ("%s failed", S(OPEN_MEMSTREAM)); + + /* Basic test to check if 'fflush (NULL)' flushed all stream including + open_memstream ones. */ + { + if (FPUTC (W('a'), fp) != W('a')) + FAIL_RET ("%s failed: %m", S(FPUTC)); + + if (fflush (NULL) != 0) + FAIL_RET ("fflush failed: %m"); + + if (size != 1) + FAIL_RET ("size != 1"); + + TEST_VERIFY (STRCMP (buf, W("a")) == 0); + } + + /* Same as before now with a buf close and equal to BUFSIZ. */ + { + const size_t bufsize = BUFSIZ - 1; + CHAR_T bufcmp[bufsize + 2]; + size_t i; + + bufcmp[0] = W('a'); + for (i=1; i<bufsize; i++) + { + bufcmp[i] = W('a'); + FPUTC (W('a'), fp); + } + bufcmp[i] = W('\0'); + + if (fflush (NULL) != 0) + FAIL_RET ("fflush failed: %m"); + + if (size != bufsize) + FAIL_RET ("size != %zu", bufsize); + + TEST_VERIFY (STRCMP (buf, bufcmp) == 0); + + bufcmp[i] = W('a'); + bufcmp[i + 1] = W('\0'); + FPUTC (W('a'), fp); + + if (fflush (NULL) != 0) + FAIL_RET ("fflush failed: %m"); + + if (size != (bufsize + 1)) + FAIL_RET ("size != %zu", bufsize); + + TEST_VERIFY (STRCMP (buf, bufcmp) == 0); + } + + if (fileno (fp) != -1) + FAIL_RET ("fileno returned a valid integer descriptor"); + + if (fclose (fp) != 0) + FAIL_RET ("fclose: %m"); + + return 0; +} + +#include <support/test-driver.c> diff --git a/libio/tst-wmemstream4.c b/libio/tst-wmemstream4.c new file mode 100644 index 0000000..9dbf615 --- /dev/null +++ b/libio/tst-wmemstream4.c @@ -0,0 +1,20 @@ +/* Test for open_wmemstream BZ #21735. + Copyright (C) 2017 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + <http://www.gnu.org/licenses/>. */ + +#define TEST_WCHAR +#include <libio/tst-memstream4.c> diff --git a/libio/wmemstream.c b/libio/wmemstream.c index 5bc77f5..d68334b 100644 --- a/libio/wmemstream.c +++ b/libio/wmemstream.c @@ -32,13 +32,14 @@ struct _IO_FILE_wmemstream static int _IO_wmem_sync (_IO_FILE* fp) __THROW; static void _IO_wmem_finish (_IO_FILE* fp, int) __THROW; +static int _IO_wmem_overflow (_IO_FILE *fp, int c) __THROW; static const struct _IO_jump_t _IO_wmem_jumps libio_vtable = { JUMP_INIT_DUMMY, JUMP_INIT (finish, _IO_wmem_finish), - JUMP_INIT (overflow, (_IO_overflow_t) _IO_wstr_overflow), + JUMP_INIT (overflow, (_IO_overflow_t) _IO_wmem_overflow), JUMP_INIT (underflow, (_IO_underflow_t) _IO_wstr_underflow), JUMP_INIT (uflow, (_IO_underflow_t) _IO_wdefault_uflow), JUMP_INIT (pbackfail, (_IO_pbackfail_t) _IO_wstr_pbackfail), @@ -89,6 +90,7 @@ open_wmemstream (wchar_t **bufloc, _IO_size_t *sizeloc) } _IO_no_init (&new_f->fp._sf._sbf._f, 0, 0, &new_f->wd, &_IO_wmem_jumps); _IO_fwide (&new_f->fp._sf._sbf._f, 1); + _IO_link_in ((struct _IO_FILE_plus *) &new_f->fp._sf._sbf); _IO_wstr_init_static (&new_f->fp._sf._sbf._f, buf, _IO_BUFSIZ / sizeof (wchar_t), buf); new_f->fp._sf._sbf._f._flags2 &= ~_IO_FLAGS2_USER_WBUF; @@ -142,3 +144,18 @@ _IO_wmem_finish (_IO_FILE *fp, int dummy) _IO_wstr_finish (fp, 0); } + +static int +_IO_wmem_overflow (_IO_FILE *fp, int c) +{ + if (c == EOF) + { + /* Updates the returned size location on stream flush. */ + struct _IO_FILE_wmemstream *mp = (struct _IO_FILE_wmemstream *) fp; + *mp->bufloc = fp->_wide_data->_IO_write_base; + *mp->sizeloc = (fp->_wide_data->_IO_write_ptr + - fp->_wide_data->_IO_write_base); + return 0; + } + return _IO_wstr_overflow (fp, c); +}
On 13/07/17 15:06, Adhemerval Zanella wrote: > On 12/07/2017 12:01, Florian Weimer wrote: >> * Adhemerval Zanella: >>> So do you prefer to just add a workaround to disable the single-thread >>> optimization for open_memstream and fix it on 2.27? IMHO this fix touches >>> only the open_memstream itself, so it should not regress any other libio >>> code so I would prefer than carry a workaround for 2.26. >> >> Which fix? Linking open_memstream into the global list has quite >> far-reaching consequences. Conservatively setting the flag instead >> should be fine. > > I agree, but I also though adding the single-thread optimization close to > freeze was also not the safe move. I do not have a strong preference here > and I think we can delay this fix to 2.27. > i think the single thread optimization is fine, but it's better to just disable it for cases that are problematic. if there is no other complaints then i will commit my patch tomorrow that disables the optimization for open_memstream instead of reverting the entire thing.
On 07/13/2017 01:05 PM, Szabolcs Nagy wrote: > On 13/07/17 15:06, Adhemerval Zanella wrote: >> On 12/07/2017 12:01, Florian Weimer wrote: >>> * Adhemerval Zanella: >>>> So do you prefer to just add a workaround to disable the single-thread >>>> optimization for open_memstream and fix it on 2.27? IMHO this fix touches >>>> only the open_memstream itself, so it should not regress any other libio >>>> code so I would prefer than carry a workaround for 2.26. >>> >>> Which fix? Linking open_memstream into the global list has quite >>> far-reaching consequences. Conservatively setting the flag instead >>> should be fine. >> >> I agree, but I also though adding the single-thread optimization close to >> freeze was also not the safe move. I do not have a strong preference here >> and I think we can delay this fix to 2.27. >> > > i think the single thread optimization is fine, but > it's better to just disable it for cases that are > problematic. > > if there is no other complaints then i will commit > my patch tomorrow that disables the optimization > for open_memstream instead of reverting the entire > thing. This sounds good to me. -- Cheers, Carlos.
On 13/07/2017 14:05, Szabolcs Nagy wrote: > On 13/07/17 15:06, Adhemerval Zanella wrote: >> On 12/07/2017 12:01, Florian Weimer wrote: >>> * Adhemerval Zanella: >>>> So do you prefer to just add a workaround to disable the single-thread >>>> optimization for open_memstream and fix it on 2.27? IMHO this fix touches >>>> only the open_memstream itself, so it should not regress any other libio >>>> code so I would prefer than carry a workaround for 2.26. >>> >>> Which fix? Linking open_memstream into the global list has quite >>> far-reaching consequences. Conservatively setting the flag instead >>> should be fine. >> >> I agree, but I also though adding the single-thread optimization close to >> freeze was also not the safe move. I do not have a strong preference here >> and I think we can delay this fix to 2.27. >> > > i think the single thread optimization is fine, but > it's better to just disable it for cases that are > problematic. > > if there is no other complaints then i will commit > my patch tomorrow that disables the optimization > for open_memstream instead of reverting the entire > thing. LGTM too and I also do not think reverting the single-thread is the best course of action. What I would like we focus on next releases is to try push such patches earlier in release so we have more time to catch such issues.
diff --git a/libio/memstream.c b/libio/memstream.c index f83d4a5..3be5b58 100644 --- a/libio/memstream.c +++ b/libio/memstream.c @@ -31,13 +31,14 @@ struct _IO_FILE_memstream static int _IO_mem_sync (_IO_FILE* fp) __THROW; static void _IO_mem_finish (_IO_FILE* fp, int) __THROW; +static int _IO_mem_overflow (_IO_FILE *fp, int c) __THROW; static const struct _IO_jump_t _IO_mem_jumps libio_vtable = { JUMP_INIT_DUMMY, JUMP_INIT (finish, _IO_mem_finish), - JUMP_INIT (overflow, _IO_str_overflow), + JUMP_INIT (overflow, _IO_mem_overflow), JUMP_INIT (underflow, _IO_str_underflow), JUMP_INIT (uflow, _IO_default_uflow), JUMP_INIT (pbackfail, _IO_str_pbackfail), @@ -87,6 +88,7 @@ __open_memstream (char **bufloc, _IO_size_t *sizeloc) return NULL; } _IO_init_internal (&new_f->fp._sf._sbf._f, 0); + _IO_link_in ((struct _IO_FILE_plus *) &new_f->fp._sf._sbf); _IO_JUMPS_FILE_plus (&new_f->fp._sf._sbf) = &_IO_mem_jumps; _IO_str_init_static_internal (&new_f->fp._sf, buf, _IO_BUFSIZ, buf); new_f->fp._sf._sbf._f._flags &= ~_IO_USER_BUF; @@ -137,3 +139,14 @@ _IO_mem_finish (_IO_FILE *fp, int dummy) _IO_str_finish (fp, 0); } + +static int +_IO_mem_overflow (_IO_FILE *fp, int c) +{ + int ret = _IO_str_overflow (fp, c); + + struct _IO_FILE_memstream *mp = (struct _IO_FILE_memstream *) fp; + *mp->sizeloc = fp->_IO_write_ptr - fp->_IO_write_base; + + return ret; +} diff --git a/libio/tst-memstream3.c b/libio/tst-memstream3.c index ce201d1..395e5c2 100644 --- a/libio/tst-memstream3.c +++ b/libio/tst-memstream3.c @@ -23,6 +23,7 @@ #include <stdarg.h> #include <errno.h> +#include <support/check.h> #ifndef CHAR_T # define CHAR_T char @@ -40,24 +41,9 @@ static void mcheck_abort (enum mcheck_status ev) { - printf ("mecheck failed with status %d\n", (int) ev); - exit (1); + FAIL_EXIT1 ("mecheck failed with status %d", (int) ev); } -static void -error_printf (int line, const char *fmt, ...) -{ - va_list ap; - - printf ("error: %s:%i: ", __FILE__, line); - va_start (ap, fmt); - vprintf (fmt, ap); - va_end (ap); -} - -#define ERROR_RET1(...) \ - { error_printf(__LINE__, __VA_ARGS__); return 1; } - static int do_test_bz18241 (void) { @@ -66,37 +52,37 @@ do_test_bz18241 (void) FILE *fp = OPEN_MEMSTREAM (&buf, &size); if (fp == NULL) - ERROR_RET1 ("%s failed\n", S(OPEN_MEMSTREAM)); + FAIL_RET ("%s failed", S(OPEN_MEMSTREAM)); if (FPUTC (W('a'), fp) != W('a')) - ERROR_RET1 ("%s failed (errno = %d)\n", S(FPUTC), errno); + FAIL_RET ("%s failed: %m", S(FPUTC)); if (fflush (fp) != 0) - ERROR_RET1 ("fflush failed (errno = %d)\n", errno); + FAIL_RET ("fflush failed: %m"); if (fseek (fp, -2, SEEK_SET) != -1) - ERROR_RET1 ("fseek failed (errno = %d)\n", errno); + FAIL_RET ("fseek failed: %m"); if (errno != EINVAL) - ERROR_RET1 ("errno != EINVAL\n"); + FAIL_RET ("errno != EINVAL"); if (ftell (fp) != 1) - ERROR_RET1 ("ftell failed (errno = %d)\n", errno); + FAIL_RET ("ftell failed: %m"); if (ferror (fp) != 0) - ERROR_RET1 ("ferror != 0\n"); + FAIL_RET ("ferror != 0"); if (fseek (fp, -1, SEEK_CUR) == -1) - ERROR_RET1 ("fseek failed (errno = %d)\n", errno); + FAIL_RET ("fseek failed: %m"); if (ftell (fp) != 0) - ERROR_RET1 ("ftell failed (errno = %d)\n", errno); + FAIL_RET ("ftell failed: %m"); if (ferror (fp) != 0) - ERROR_RET1 ("ferror != 0\n"); + FAIL_RET ("ferror != 0"); if (FPUTC (W('b'), fp) != W('b')) - ERROR_RET1 ("%s failed (errno = %d)\n", S(FPUTC), errno); + FAIL_RET ("%s failed: %m", S(FPUTC)); if (fflush (fp) != 0) - ERROR_RET1 ("fflush failed (errno = %d)\n", errno); + FAIL_RET ("fflush failed: %m"); if (fclose (fp) != 0) - ERROR_RET1 ("fclose failed (errno = %d\n", errno); + FAIL_RET ("fclose failed: %m"); if (STRCMP (buf, W("b")) != 0) - ERROR_RET1 ("%s failed\n", S(STRCMP)); + FAIL_RET ("%s failed", S(STRCMP)); free (buf); @@ -112,45 +98,74 @@ do_test_bz20181 (void) FILE *fp = OPEN_MEMSTREAM (&buf, &size); if (fp == NULL) - ERROR_RET1 ("%s failed\n", S(OPEN_MEMSTREAM)); + FAIL_RET ("%s failed\n", S(OPEN_MEMSTREAM)); if ((ret = FWRITE (W("abc"), 1, 3, fp)) != 3) - ERROR_RET1 ("%s failed (errno = %d)\n", S(FWRITE), errno); + FAIL_RET ("%s failed (errno = %d)\n", S(FWRITE), errno); if (fseek (fp, 0, SEEK_SET) != 0) - ERROR_RET1 ("fseek failed (errno = %d)\n", errno); + FAIL_RET ("fseek failed (errno = %d)\n", errno); if (FWRITE (W("z"), 1, 1, fp) != 1) - ERROR_RET1 ("%s failed (errno = %d)\n", S(FWRITE), errno); + FAIL_RET ("%s failed (errno = %d)\n", S(FWRITE), errno); if (fflush (fp) != 0) - ERROR_RET1 ("fflush failed (errno = %d)\n", errno); + FAIL_RET ("fflush failed (errno = %d)\n", errno); /* Avoid truncating the buffer on close. */ if (fseek (fp, 3, SEEK_SET) != 0) - ERROR_RET1 ("fseek failed (errno = %d)\n", errno); + FAIL_RET ("fseek failed (errno = %d)\n", errno); if (fclose (fp) != 0) - ERROR_RET1 ("fclose failed (errno = %d\n", errno); + FAIL_RET ("fclose failed (errno = %d\n", errno); if (size != 3) - ERROR_RET1 ("size != 3\n"); + FAIL_RET ("size != 3\n"); if (buf[0] != W('z') || buf[1] != W('b') || buf[2] != W('c')) { PRINTF (W("error: buf {%c,%c,%c} != {z,b,c}\n"), - buf[0], buf[1], buf[2]); - return 1; + buf[0], buf[1], buf[2]); + FAIL_RET (NULL); } + free (buf); return 0; } static int +do_test_bz21735 (void) +{ + CHAR_T *buf; + size_t size; + + FILE *fp = OPEN_MEMSTREAM (&buf, &size); + if (fp == NULL) + FAIL_RET ("%s failed", S(OPEN_MEMSTREAM)); + + if (FPUTC (W('a'), fp) != W('a')) + FAIL_RET ("%s failed: %m", S(FPUTC)); + + if (fflush (fp) != 0) + FAIL_RET ("fflush failed: %m"); + + if (size != 1) + FAIL_RET ("size != 1"); + + if (fileno (fp) != -1) + FAIL_RET ("fileno returned a valid integer descriptor"); + + if (fclose (fp) != 0) + FAIL_RET ("fclose: %m"); + + return 0; +} + +static int do_test (void) { int ret = 0; @@ -159,9 +174,9 @@ do_test (void) ret += do_test_bz18241 (); ret += do_test_bz20181 (); + ret += do_test_bz21735 (); return ret; } -#define TEST_FUNCTION do_test () -#include "../test-skeleton.c" +#include <support/test-driver.c> diff --git a/libio/wmemstream.c b/libio/wmemstream.c index 5bc77f5..f6d9f14 100644 --- a/libio/wmemstream.c +++ b/libio/wmemstream.c @@ -32,13 +32,14 @@ struct _IO_FILE_wmemstream static int _IO_wmem_sync (_IO_FILE* fp) __THROW; static void _IO_wmem_finish (_IO_FILE* fp, int) __THROW; +static int _IO_wmem_overflow (_IO_FILE *fp, int c) __THROW; static const struct _IO_jump_t _IO_wmem_jumps libio_vtable = { JUMP_INIT_DUMMY, JUMP_INIT (finish, _IO_wmem_finish), - JUMP_INIT (overflow, (_IO_overflow_t) _IO_wstr_overflow), + JUMP_INIT (overflow, (_IO_overflow_t) _IO_wmem_overflow), JUMP_INIT (underflow, (_IO_underflow_t) _IO_wstr_underflow), JUMP_INIT (uflow, (_IO_underflow_t) _IO_wdefault_uflow), JUMP_INIT (pbackfail, (_IO_pbackfail_t) _IO_wstr_pbackfail), @@ -89,6 +90,7 @@ open_wmemstream (wchar_t **bufloc, _IO_size_t *sizeloc) } _IO_no_init (&new_f->fp._sf._sbf._f, 0, 0, &new_f->wd, &_IO_wmem_jumps); _IO_fwide (&new_f->fp._sf._sbf._f, 1); + _IO_link_in ((struct _IO_FILE_plus *) &new_f->fp._sf._sbf); _IO_wstr_init_static (&new_f->fp._sf._sbf._f, buf, _IO_BUFSIZ / sizeof (wchar_t), buf); new_f->fp._sf._sbf._f._flags2 &= ~_IO_FLAGS2_USER_WBUF; @@ -142,3 +144,15 @@ _IO_wmem_finish (_IO_FILE *fp, int dummy) _IO_wstr_finish (fp, 0); } + +static int +_IO_wmem_overflow (_IO_FILE *fp, int c) +{ + int ret = _IO_wstr_overflow (fp, c); + + struct _IO_FILE_wmemstream *mp = (struct _IO_FILE_wmemstream *) fp; + *mp->sizeloc = (fp->_wide_data->_IO_write_ptr + - fp->_wide_data->_IO_write_base); + + return ret; +}
POSIX specifies that the state and size after a fflush on a stream opened by open_memstream should be updated and current implementation does not act accordingly on a fflush (NULL) (meant to act on all opened streams). This patch fixes it for open_{w}memstream and also adjust the tst-memstream3 to use libsupport. Checked on x86_64-linux-gnu. [BZ #21735] * libio/memstream.c (_IO_mem_overflow): New function. (_IO_mem_jumps): User _IO_mem_overflow instead of _IO_str_overflow. (__open_memstream): Call _IO_link_in on the internal FILE object. * libio/wmemstream.c (_IO_wmem_overflow): New function. (_IO_wmem_jumps): User _IO_wmem_overflow instead of _IO_wstr_overflow. (__open_wmemstream): Call _IO_link_in on the internal FILE object. * libio/tst-memstream3.c (mcheck_abort): Use libsupport. (do_test_bz18241): Likewise. (do_test_bz20181): Likewise. (do_test_bz21735): New function. --- ChangeLog | 14 +++++++ libio/memstream.c | 15 +++++++- libio/tst-memstream3.c | 99 +++++++++++++++++++++++++++++--------------------- libio/wmemstream.c | 16 +++++++- 4 files changed, 100 insertions(+), 44 deletions(-) -- 2.7.4