From patchwork Wed Nov 30 11:44:32 2016 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Szabolcs Nagy X-Patchwork-Id: 85008 Delivered-To: patch@linaro.org Received: by 10.182.112.6 with SMTP id im6csp240369obb; Wed, 30 Nov 2016 03:45:11 -0800 (PST) X-Received: by 10.84.178.195 with SMTP id z61mr73615588plb.176.1480506311415; Wed, 30 Nov 2016 03:45:11 -0800 (PST) Return-Path: Received: from sourceware.org (server1.sourceware.org. [209.132.180.131]) by mx.google.com with ESMTPS id b69si35503941pli.222.2016.11.30.03.45.10 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 30 Nov 2016 03:45:11 -0800 (PST) Received-SPF: pass (google.com: domain of libc-alpha-return-75340-patch=linaro.org@sourceware.org designates 209.132.180.131 as permitted sender) client-ip=209.132.180.131; Authentication-Results: mx.google.com; dkim=pass header.i=@sourceware.org; spf=pass (google.com: domain of libc-alpha-return-75340-patch=linaro.org@sourceware.org designates 209.132.180.131 as permitted sender) smtp.mailfrom=libc-alpha-return-75340-patch=linaro.org@sourceware.org DomainKey-Signature: a=rsa-sha1; c=nofws; d=sourceware.org; h=list-id :list-unsubscribe:list-subscribe:list-archive:list-post :list-help:sender:message-id:date:from:mime-version:to:cc :subject:content-type; q=dns; s=default; b=G7GGcXWXASjvHaGZxSIFQ D0izp8ikOI4D3GZH4Tr6l7rvPHooR0xQr+y0HWTE7CYUaLNyssQ7qEDP9fSj7lVP Nim8OucAMUvz+5EvOeOi8Ik7FZx2IuVLBY/ha2kXrd3HavcEZjP8jE7XWBTy8kos KSj9TKLEGWI5CFZA7ichO4= DKIM-Signature: v=1; a=rsa-sha1; c=relaxed; d=sourceware.org; h=list-id :list-unsubscribe:list-subscribe:list-archive:list-post :list-help:sender:message-id:date:from:mime-version:to:cc :subject:content-type; s=default; bh=qj2OBD53g5ueQTAc58nlrcMnTy4 =; b=NKoA2VAZ5lsjreNQI3I8PL2P8IYxfHy5ddH9JJbltAAluDIuRgM2OuBGmqf l8jTDucRDUdfYnAKasuJQgAYI2iR0iy5f+oqvp87aF+1FcuPEcFxs8rSljqAN1Zj 4SqA6WdasYJuFlcmBBlpog8fxMP5PR1Xx3hgGsaNQjkwMTNE= Received: (qmail 40819 invoked by alias); 30 Nov 2016 11:44:49 -0000 Mailing-List: contact libc-alpha-help@sourceware.org; run by ezmlm Precedence: bulk List-Id: List-Unsubscribe: List-Subscribe: List-Archive: List-Post: List-Help: , Sender: libc-alpha-owner@sourceware.org Delivered-To: mailing list libc-alpha@sourceware.org Received: (qmail 40717 invoked by uid 89); 30 Nov 2016 11:44:48 -0000 Authentication-Results: sourceware.org; auth=none X-Virus-Found: No X-Spam-SWARE-Status: No, score=-1.8 required=5.0 tests=AWL, BAYES_00, RCVD_IN_DNSWL_NONE, SPF_HELO_PASS, SPF_PASS autolearn=ham version=3.3.2 spammy=up-to-date, uptodate, HX-Exchange-Antispam-Report-Test:180628864354917, HX-HELO:sk:EUR02-V X-HELO: EUR02-VE1-obe.outbound.protection.outlook.com Authentication-Results: spf=none (sender IP is ) smtp.mailfrom=Szabolcs.Nagy@arm.com; Message-ID: <583EBBA0.9060709@arm.com> Date: Wed, 30 Nov 2016 11:44:32 +0000 From: Szabolcs Nagy User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:31.0) Gecko/20100101 Thunderbird/31.8.0 MIME-Version: 1.0 To: GNU C Library , Torvald Riegel CC: Subject: [RFC PATCH 2/2][BZ 19329] Fix data races between pthread_create and dlopen X-ClientProxiedBy: DB6PR0202CA0040.eurprd02.prod.outlook.com (10.171.70.26) To DB6PR0802MB2152.eurprd08.prod.outlook.com (10.172.227.10) X-Microsoft-Exchange-Diagnostics: 1; DB6PR0802MB2152; 2:fBH/nedR9+bDqXX9a4PQfhoCoFlEvn13zIvm/sHLLZMt64yT3ubsMWFrdn4MmcfNFYFxX/Zk9bu2rXLEV8m/L/xNfyn1gnCEpli+Gr+U6Qm5bpxTh53/YCRSSi2vKPPDjdJZ1K6evPc4mnPo1uzYgpgXskd3lLrAdf+2wmCKlvs=; 3:HAjsYgcYdAPz4JdTNhukI4Q4MowTjvCmn2jG0xbxkc6qcj8kLdW/gUDkNVsbrh74qg+j5epOLzul/BSUZY2vlfC4S3q+a6NT+h1OjORSuU+SL3i8TpxkcK+sPLoDrZR+oityrAjHGjQirK5e/l3/qJmKM8lHi9VU+aA/DmA1JU0= X-MS-Office365-Filtering-Correlation-Id: a01aa825-e21a-4360-2560-08d4191641a9 X-Microsoft-Antispam: UriScan:; BCL:0; PCL:0; RULEID:(22001); SRVR:DB6PR0802MB2152; X-Microsoft-Exchange-Diagnostics: 1; DB6PR0802MB2152; 25:5vVvpjkb12CGyrtBgF1UWYGT+SZGf9wda6Ndkt7hmfO2TQLV+KLg3tDNLlSQ0DTFnQONwgM47OP+PXxrbVfH67Zs3hTpErJbK5dzFUyGZV6LFVFp/QrzOv5xI1/L8EJAig+uyG+wtS7MSPOZW2ZbSO17bOrKfAsukJPcsUdayY0NQFXrdOVe8PGVh1LSWW/q9sNdFhoRZEYV7nc51bQwapMabTKtnkfqL9O4YMMk+9ATpzJr92h6YK/a7KMNmLqBER5ufQjAT01ZgF/mXculiUR4ofMbZVIllWMp7itAHKvjdu31SYry6DW7q9VEAQ8dhod7fLuC45U2NOv7pmwi2uA+T9HdS7dBKzSmNcQe3NlLiUqLl/TX2HWFK/xcfwg23IuW8WSM1WAYzGhF3RQkP4muxmpjoUE1b3bDMOaGoyQrxvSlXYiz/OGt0ybn82O5Pv6CGwLHZlN53yRez4pe7X91e5im7wjcpNnWTisPcsnyVfH9j6HrwfrQrWVe+LzcqS2ko5r6BYkNkrUPiYnYcuPRConlIKpYZO28RrEevQVBYaFo62sKBpHhhb2Bjtd3BsyHu5esyQyG992/KlMw/hf3YYVevFUMxaDXSm3Nbkw0XgALy+j41tuROm/SP4kBRxgAi5XI3ofQg4PCgQFS/aKLNbuN51AwB0UA2UyU9Kqsjj2zLLZVVDdQo/zJZysJI83QNDGjiK3nrUbRczbIu5KSA1cYVo9WKbSbsJy2xAD3jVWFD5wQmWDXH0rXsiITqii9B2tKoy+qWeKnVZWqj/a23PbevStobj/iBQmOAmVCM8cDMbzoKKLa5tI2QWywNrlEyC5FB6Tx+BHpF60XpYl7w7nqmW+AhI4GqpEEGkrg3lGWlo/CEIr1rJKRF7FbJdvOYXaJDKQhCNSWlQzuLtMzzZpvlGqZisAqEmC9rSE= X-Microsoft-Exchange-Diagnostics: 1; DB6PR0802MB2152; 31:huYyz/Z+rYBDE/aqwveaVBZxEAjdMyPJ4gAZes3jRahDK8TfpRR69BdbCVMPW2MsY8N0eZDWHNZ6Og48TfMyt4y7FHddxBY5tdnz6lxeUZ+2JDpZsU9dJt+7QaF1kPXgWwLabRaL2TeNaPLx0BLHeeUwYKM7UIQrFMqubnPo5gL4fVRVmVzXR+mdMi+UtX+LvFGBiVnY6pb/ewe7NwrV2lzmNKFzj8FkM1BzGhTsSm4z+1L28vsSuuwgmxQZwDtAeUBzMspLqFZ/H3xBOQ79IQwLFzPabD9edIKdO3pQjtc=; 20:oTqkbM+/M3H3kuUhvM2xrfYWVZS+8rB9TORHn9tJMAd6ZY4Cq6Yaj/Da+E+mDRN9Tn4PWHqY8TN8q8hH/mwPJN7L9lHYCTDEOCA7WqDFTyU2mmhqFzGvk/3NHbBcjJoFlNq+ttLJDBm9v4EQ2XyTpbrq4sB+F0KBTDhq4jry8eI= NoDisclaimer: True X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-Test: UriScan:(180628864354917); X-Exchange-Antispam-Report-CFA-Test: BCL:0; PCL:0; RULEID:(102415395)(6040375)(601004)(2401047)(5005006)(8121501046)(3002001)(10201501046)(6055026)(6041248)(20161123560025)(20161123562025)(20161123564025)(20161123558021)(20161123555025); SRVR:DB6PR0802MB2152; BCL:0; PCL:0; RULEID:; SRVR:DB6PR0802MB2152; X-Microsoft-Exchange-Diagnostics: 1; DB6PR0802MB2152; 4:uICyi+e0kmJmh+H2ez0Eal08LsTPWaFrMgq6DTYTXfr3pbkQXcs9TxmcHNcsP0RIqijYOh6DXB4mEJdMF9QSyyEyPOpx20QlPkNceAqvhQ+0pCPi0dj+E61g9BOvo4txl4HeQT5FbtFfjGymNzICx4Ko2H+lTIDv61z4Ie9MtUPZR4TPfVkpnQtgM2rosMi2dZ7S4nolDIyVtolmDo3eujOsw1zawv2JNsvZV9ysHQnU2xc43twMYS0tXFi2p2jeYu8CVPn3kJ4WRkRqEoUalI5vjSj/RwgaxLjgqwfbofgeGAL/qDW8tfZuDWiqFn2cHHlP+ILczNFXFQfCn0nELCoLmgUttoZgAPFzGoVCqf5xmyqSYxe4rD8LdRWqHQTLR/12kp1dcPkjeSmvnzhAiOSdLbruTh5h0SiWwOj9Mh4BvXxMo2Or0cUhlvTndd9L4uIDLCMwmpUcgnzbMu7m9i29ERmY6b4jqvO4AviM2WVdpiPenSA+Q6zUha8djxK0XyxWK9ugACKCEEGIhmvK81J6L8kVekWlfQ51onGQYEa9LknULW521SO1CnMZXBUoD3i51dxSGzM70MUaQb6nYJvc4zoLK3TJyKq38cCtuoh71+nqBxYYTuFeY3Hhnlyz4dS4tmONpGRY8eR782Wh3IFpkAiSZv0Ms4sOCx02R3Y= X-Forefront-PRVS: 0142F22657 X-Forefront-Antispam-Report: SFV:NSPM; SFS:(10009020)(4630300001)(6049001)(6009001)(7916002)(189002)(377424004)(199003)(81166006)(50986999)(33656002)(65956001)(77096006)(39450400002)(4610100001)(5660300001)(270700001)(38730400001)(68736007)(568964002)(80316001)(6486002)(512874002)(39410400001)(5890100001)(8676002)(2906002)(81156014)(65806001)(42186005)(4326007)(7736002)(64126003)(66066001)(7846002)(4001350100001)(87266999)(65816999)(5001770100001)(189998001)(3846002)(106356001)(733004)(101416001)(83506001)(97736004)(2476003)(6116002)(36756003)(92566002)(86362001)(105586002)(305945005)(59896002)(4001150100001)(54356999)(84326002)(21490400002)(217873001); DIR:OUT; SFP:1101; SCL:1; SRVR:DB6PR0802MB2152; H:[10.2.206.73]; FPR:; SPF:None; PTR:InfoNoRecords; MX:1; A:1; LANG:en; Received-SPF: None (protection.outlook.com: arm.com does not designate permitted sender hosts) X-Microsoft-Exchange-Diagnostics: =?us-ascii?Q?1; DB6PR0802MB2152; 23:0D9XeA8W1opHV3DfG8sUka4QqD/SYJxgjp72Rz3?= =?us-ascii?Q?3W+FbUbayrtjmv6Y0P6+B3k/5yKdDm488nXc6Gw4es0We0xJ/cH2TK3jg1+g?= =?us-ascii?Q?IfHfkfZq54IumMqB/GIgOHEYMwRkA/r+bGzYV4SOveahkT2gFZjMq2xEZQzT?= =?us-ascii?Q?BrRnV/ErrYokhHXKgSUb+JfAquo8Q7Q432tbPsr8pqNxroQ2XiQMFkNUCz7d?= =?us-ascii?Q?676R99FZHCZqq1N+3vIQoWG/ZvIxwkRhH9BLAolEqrm5RFqpZqP+4ncV24QD?= =?us-ascii?Q?UK9//ST/NaFcfTur1W3sfC+KTxJ3y13IrHKweeM2Vr/gGEITvh6EwJCESjsN?= =?us-ascii?Q?W5V7LSPI2HeEOMPD9I8qyGEkzCRKCNJ+baCJHCCwQAn7RqSgxU2MmYhRBHLG?= =?us-ascii?Q?5Xj78arSSLJB4bPYcgts1T6uwXgHQJz5ojHxVy+gevD7M2xeNkFLVMezZXTx?= =?us-ascii?Q?GchvPN5LBKQryqmA1kKORcNpcy3RaHcWVz3dZbV+CYpsJNSvKuHxD2Bu22k4?= =?us-ascii?Q?AVMCVjsvs/QYI3C8Hz93atbcK37CKnv/4L9uF2eU6QsykYelCre6b0bwbn1Z?= =?us-ascii?Q?+i+eh4ykcEXWkScPRkQIpHT6VBWWhA2TVajYAVqWO8U0HrZ6zzMuzJWX7h9v?= =?us-ascii?Q?37HAvNx312RQERc7F6ZThuU+oCXzhR45KXHhB0Ktoz8y8XGZzLEwynkkZgF2?= =?us-ascii?Q?TSKgH/X6S7Yvf/JGj9UpTVS/934Dr85/y1UGZXps6HdWj8qbSuvpmuesTiz+?= =?us-ascii?Q?Z7IZFbRSuV+CwXoWD/edoHT8N/Rq/ayH4GsN1VNJ9IC6piVavbv1zpjBZ8Tz?= =?us-ascii?Q?T82y6e+T4/crRR9vOWXghdNIIjx++L6xy8D98dRWP6fXT8D2k9zwL5ZvFWUq?= =?us-ascii?Q?83NboEJYJGpWA3y8o0Jy8vQ0cF9lH1LMmiyn/tN4xHmODL7Pf1RqH6NG5OWv?= =?us-ascii?Q?XFFaYVdpEvpWlBQv1UCY2mZCciNpGOuU5sJSTLEiIPXf6MbKZdlO98a/t/sS?= =?us-ascii?Q?aTPp3kAxG9WGEXpkQBaDeUgSrrnlWyFm0YVE24XpW+hEgg5XP3zlJD1Yro75?= =?us-ascii?Q?/bvBHLf3oLYHIAMM2WvCbKPOHgV/k7sruH0QG+F2LrjjhkQyuYSYb1TJr0bj?= =?us-ascii?Q?FhwKdYKwRY6bn6rVy/4f/xXoYACVi0x86nufBytQdwkTrStaxcjFhfiCfhEw?= =?us-ascii?Q?hXJW+E4ZYwbOyeLn6P+ScA8ATpEoiKAXG0s3BKw4lDv7L2prVIJxI4nPFQyr?= =?us-ascii?Q?EbMpxF5m4OafPPZ6XFYSYmBsMx0UHH//pXSVcfEb7+okDEuwfDJYc6ifn75A?= =?us-ascii?Q?EMakMZM039SK9fT7pE/OClygwNU7RV4roDWFHBiiuiFXPy0iA0RR37IsrGRg?= =?us-ascii?Q?h9Btl8v1SnjwpP/8ldHKY78hudyiFsdNvs6j5Lv/4AynTq/2/uB9DjVgEsrL?= =?us-ascii?Q?iiwMZHgWIN0epqfHv5GQ1zQfePXszV7mP3E+cPsCf8UIjlx780YXK?= X-Microsoft-Exchange-Diagnostics: 1; DB6PR0802MB2152; 6:0c5yngIhtv39Hn+VUxKCO477oClv1B2ugEw/wncroGToI3HH4B3N17lh0qUribXsaMR+TmhbSow1cJ3gqXaayYEBWwDIhLkBDwJ1U8tzQwQAb461282bA+n2kvDdaRiJ6IefHDJmTTx8Ps3tFeM5T+pLxxFW6VhtNc7JQC8fWShoJ+juFFxFO/QhtfCV0XiQNMpZlyQ7bEXACAcTd9u9SCRKqWr/cRhRrXUNLGRG8u2Wz2JuQU9BhrJEPlVXkGqGp0rSInmFDVxsnezWVkaeiVk1rC4xMb2q70zWPHZmL1PdK/F9U0EbBtViPeu4XHmR5jaah80KwfGfdQVx7BFaaA==; 5:xqJzunNIRLHgGphkSth16MWDRNe/lKm0TFO7n/ASIHYM2+v58sShwP8Tsuox+B4HzX38Z7Z53Ml0H8rWF2QJytZk1F5u+3miI+kafGohh7mOiXNUAykWCjeKzGYhQ21u625plB8XKQSeLfbPjMVLTg==; 24:v/CGdUDqYtrY/nvhksNo+jGnsvuaSWltE7Z7RMvK5cIiIBSAczDCvqC4j9oIKR62imvR89xd2ZcG+eew4GuovMoBkkOEA39mlN3drBGiKRY= SpamDiagnosticOutput: 1:99 SpamDiagnosticMetadata: NSPM X-Microsoft-Exchange-Diagnostics: 1; DB6PR0802MB2152; 7:h8GSgNRb17K9fiLPk252Bcq/6JMdB3+3Jbq66NJ9BfaALig8nsApPEa+2DALwzlQJFeZ1JIaeitbWPcpKc2JnuSy8IKRRG6wtKyjjCARiOa9F4KobKwcaEucGZD3s9EMmfclNq5QJUiw1yga4ZTiHwUmrlFIcjol5G0o9cq6FUkeaqnCJB8CEbD8wAS/tKTIljM56znGEZnTvD7cJWjoOGxcVKWBiKJIzVkhpVcqGt3XMtS/OVwDNvTTomkcyUdq3IoBwpX5xaeP3GUSSjcX2f0IRT0+Sbw/vHV07aJQUAidVW5kkFNdWvr/coRBnq/QK9ZmmK/Ewai7T6uR9H8uilNPYbmfhIanSWQwpsYAo9E= X-OriginatorOrg: arm.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 30 Nov 2016 11:44:34.6590 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-Transport-CrossTenantHeadersStamped: DB6PR0802MB2152 This fixes a subset of the issues described in https://sourceware.org/ml/libc-alpha/2016-11/msg01026.html without adding locks to pthread_create. Only races between dlopen and pthread_create were considered, and the asserts got removed that tried to check for concurrency issues. The patch is incomplete because dlclose, tls access and dl_iterate_phdr related code paths are not modified. dlclose should be updated in a similar fashion to dlopen to make the patch complete alternatively pthread_create may take the GL(dl_load_write_lock) to sync with dlclose or the GL(dl_load_lock) to sync with dlopen and dlclose (that would simplify the concurrency design, but increase lock contention on the locks). 2016-11-30 Szabolcs Nagy [BZ #19329] * elf/dl-open.c (dl_open_worker): Write GL(dl_tls_generation) atomically. * elf/dl-tls.c (_dl_allocate_tls_init): Read GL(dl_tls_generation), GL(dl_tls_max_dtv_idx), slotinfo entries and listp->next atomically. Remove assertions that cannot be guaranteed. (_dl_add_to_slotinfo): Write the slotinfo entries and listp->next atomically. diff --git a/elf/dl-open.c b/elf/dl-open.c index f5ca261..e6f6e0b 100644 --- a/elf/dl-open.c +++ b/elf/dl-open.c @@ -524,9 +524,17 @@ dl_open_worker (void *a) } /* Bump the generation number if necessary. */ - if (any_tls && __builtin_expect (++GL(dl_tls_generation) == 0, 0)) - _dl_fatal_printf (N_("\ + if (any_tls) + { + /* This cannot be in a data-race so non-atomic load is valid too. */ + size_t newgen = atomic_load_relaxed (&GL(dl_tls_generation)) + 1; + /* Synchronize with _dl_allocate_tls_init (see notes there) and + avoid storing an overflowed counter. */ + if (__builtin_expect (newgen == 0, 0)) + _dl_fatal_printf (N_("\ TLS generation counter wrapped! Please report this.")); + atomic_store_release (&GL(dl_tls_generation), newgen); + } /* We need a second pass for static tls data, because _dl_update_slotinfo must not be run while calls to _dl_add_to_slotinfo are still pending. */ diff --git a/elf/dl-tls.c b/elf/dl-tls.c index 2feee67..7dbee0d 100644 --- a/elf/dl-tls.c +++ b/elf/dl-tls.c @@ -470,6 +470,36 @@ _dl_resize_dtv (dtv_t *dtv) } +/* +CONCURRENCY NOTES + +dlopen (and dlclose) holds the GL(dl_load_lock) while writing shared state, +which may be concurrently read by pthread_create and tls access without taking +the lock, so atomic access should be used. The shared state: + + GL(dl_tls_max_dtv_idx) - max modid assigned, (modid can be reused). + GL(dl_tls_generation) - generation count, incremented by dlopen and dlclose. + GL(dl_tls_dtv_slotinfo_list) - list of entries, contains generation count + and link_map for each module with a modid. + +A module gets a modid assigned if it has tls, a modid identifies a slotinfo +entry and it is the index of the corresponding dtv slot. The generation count +is assigned to slotinfo entries of a newly loaded or unloaded module and its +newly loaded or unloaded dependencies. + +TODO: dlclose may free memory read by a concurrent pthread_create or tls +access. This is broken now, so it is assumed that dlclose does not free +link_map structures while pthread_create or __tls_get_addr is reading them. + +pthread_create calls _dl_allocate_tls_init (before creating the new thread), +which should guarantee that the dtv is in a consistent state at the end: + +All slotinfo updates with generation <= dtv[0].counter are reflected in the +dtv and arbitrary later module unloads may also be reflected as unallocated +entries. (Note: a modid reuse implies a module unload and accessing tls in +an unloaded module is undefined.) +*/ + void * internal_function _dl_allocate_tls_init (void *result) @@ -482,12 +512,24 @@ _dl_allocate_tls_init (void *result) struct dtv_slotinfo_list *listp; size_t total = 0; size_t maxgen = 0; + /* Synchronizes with the increments in dl_{open,close}_worker. + Slotinfo updates of this generation are sequenced before the + write we read from here. */ + size_t gen_count = atomic_load_acquire (&GL(dl_tls_generation)); + /* Either reads from the last write that is sequenced before the + generation counter increment we synchronized with or a write + made by a later dlopen/dlclose. dlclose may decrement this, + but only if related modules are unloaded. So it is an upper + bound on non-unloaded modids up to gen_count generation. */ + size_t dtv_slots = atomic_load_relaxed (&GL(dl_tls_max_dtv_idx)); /* Check if the current dtv is big enough. */ - if (dtv[-1].counter < GL(dl_tls_max_dtv_idx)) + if (dtv[-1].counter < dtv_slots) { /* Resize the dtv. */ dtv = _dl_resize_dtv (dtv); + /* _dl_resize_dtv rereads GL(dl_tls_max_dtv_idx) which may decrease. */ + dtv_slots = dtv[-1].counter; /* Install this new dtv in the thread data structures. */ INSTALL_DTV (result, &dtv[-1]); @@ -504,22 +546,33 @@ _dl_allocate_tls_init (void *result) for (cnt = total == 0 ? 1 : 0; cnt < listp->len; ++cnt) { struct link_map *map; + size_t gen; void *dest; /* Check for the total number of used slots. */ - if (total + cnt > GL(dl_tls_max_dtv_idx)) + if (total + cnt > dtv_slots) break; - map = listp->slotinfo[cnt].map; + /* Synchronize with dl_add_to_slotinfo and remove_slotinfo. */ + map = atomic_load_acquire (&listp->slotinfo[cnt].map); if (map == NULL) /* Unused entry. */ continue; + /* Consistent generation count with the map read above. + Inconsistent gen may be read if the entry is being reused, + in which case it is larger than gen_count and we skip it. */ + gen = atomic_load_relaxed (&listp->slotinfo[cnt].gen); + if (gen > gen_count) + /* New entry. */ + continue; + /* Keep track of the maximum generation number. This might not be the generation counter. */ - assert (listp->slotinfo[cnt].gen <= GL(dl_tls_generation)); - maxgen = MAX (maxgen, listp->slotinfo[cnt].gen); + maxgen = MAX (maxgen, gen); + /* TODO: concurrent dlclose may free map which would break + the rest of the code below. */ dtv[map->l_tls_modid].pointer.val = TLS_DTV_UNALLOCATED; dtv[map->l_tls_modid].pointer.to_free = NULL; @@ -549,11 +602,15 @@ _dl_allocate_tls_init (void *result) } total += cnt; - if (total >= GL(dl_tls_max_dtv_idx)) + if (total >= dtv_slots) break; - listp = listp->next; - assert (listp != NULL); + /* Synchronize with dl_add_to_slotinfo. */ + listp = atomic_load_acquire (&listp->next); + /* dtv_slots is an upper bound on the number of entries we care + about, the list may end sooner. */ + if (listp == NULL) + break; } /* The DTV version is up-to-date now. */ @@ -954,7 +1011,7 @@ _dl_add_to_slotinfo (struct link_map *l) the first slot. */ assert (idx == 0); - listp = prevp->next = (struct dtv_slotinfo_list *) + listp = (struct dtv_slotinfo_list *) malloc (sizeof (struct dtv_slotinfo_list) + TLS_SLOTINFO_SURPLUS * sizeof (struct dtv_slotinfo)); if (listp == NULL) @@ -969,9 +1026,17 @@ _dl_add_to_slotinfo (struct link_map *l) listp->next = NULL; memset (listp->slotinfo, '\0', TLS_SLOTINFO_SURPLUS * sizeof (struct dtv_slotinfo)); + /* Add the new list item and synchronize with _dl_allocate_tls_init. */ + atomic_store_release (&prevp->next, listp); } /* Add the information into the slotinfo data structure. */ - listp->slotinfo[idx].map = l; - listp->slotinfo[idx].gen = GL(dl_tls_generation) + 1; + + /* This cannot be in a data-race so non-atomic load would be valid too. */ + size_t newgen = atomic_load_relaxed (&GL(dl_tls_generation)) + 1; + /* TODO: Concurrent readers may see an overflowed gen, which is bad, + but overflow is guaranteed to crash the dlopen that is executing. */ + atomic_store_relaxed (&listp->slotinfo[idx].gen, newgen); + /* Synchronize with _dl_allocate_tls_init (see notes there). */ + atomic_store_release (&listp->slotinfo[idx].map, l); }