From patchwork Fri Nov 17 15:15:47 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Sandiford X-Patchwork-Id: 119164 Delivered-To: patch@linaro.org Received: by 10.80.225.132 with SMTP id k4csp755805edl; Fri, 17 Nov 2017 07:17:19 -0800 (PST) X-Google-Smtp-Source: AGs4zMYC6Bgarvt9Y1Sp4aoDobJjyjxKKEbBtuyBfOzUnswxCDcS8engc/gnkgP4gohbLayG7OVT X-Received: by 10.98.58.208 with SMTP id v77mr2432744pfj.150.1510931839548; Fri, 17 Nov 2017 07:17:19 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1510931839; cv=none; d=google.com; s=arc-20160816; b=ZUeR4S78/5BRHYuFSf2AQKvxyQktWn0WagAAq6mDya73losnmpVBk8wGx9wuCuQQin jLx4Mb8ZU6vrSglhFS5v4rLatLwcFGjAxmI/LBEKmEZpdAjqT1+kdVun5T69mLNExWRv CrGpZy7AJU2vRsyFaMZVe6WKn8qHuwydu9IHlDT+uQkI55KgxqGAQPen0ZmKnWzJ0NvZ 63v2bxgd7/CR5E4wAuNoJ/+7BLNOVZ1CeexheDrKBbPiGsSyP7mIIE/PFH8BArrYzhhX +060IjzXL/r1b2n11n0yIxD+szX7mV6jg+MadBOzO3/9UqaB2dEYlo5S8Blqv8CR0upG llPw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=mime-version:user-agent:message-id:date:subject:mail-followup-to:to :from:delivered-to:sender:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:mailing-list:dkim-signature :domainkey-signature:arc-authentication-results; bh=ebWbQQ1iB5g02P1acYVi/Rpfxm8Sd5MLNbR398ChcNI=; b=KvFKRKwwNMRCwyc9+lbEr4IiJ/MHr0K/kRRwZb7bn4dS9AwngCpKtZOeDn7rZUVexd hz7Eh132BSI6yqpbznyrTvwDWbvUE5xgZyRJnyVwsYra/EpoZi15gYr9YKq8CYPJwc40 nAvKahdV7Doyb1HCNGPePITmu+ViIjxFp8dkN8AJVv5G4lcdYg4/xF8ITMuJfTKi777o 6x3jcKpyhV4BVtvgZ7uzC7lDaHXWDAKfYovaZkL7VTdvu+G7klgFSEDTlrscsAokwR9o 6CKZhvfovMXyQnY/zAjV7U9HOU8p8SEFjg5vN8hvUgVtFQfQ5hsQT9EpEKLM7LS9xj0p DyRQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@gcc.gnu.org header.s=default header.b=uv3SwLD4; spf=pass (google.com: domain of gcc-patches-return-467150-patch=linaro.org@gcc.gnu.org designates 209.132.180.131 as permitted sender) smtp.mailfrom=gcc-patches-return-467150-patch=linaro.org@gcc.gnu.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from sourceware.org (server1.sourceware.org. [209.132.180.131]) by mx.google.com with ESMTPS id u13si2981711plq.361.2017.11.17.07.17.19 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 17 Nov 2017 07:17:19 -0800 (PST) Received-SPF: pass (google.com: domain of gcc-patches-return-467150-patch=linaro.org@gcc.gnu.org designates 209.132.180.131 as permitted sender) client-ip=209.132.180.131; Authentication-Results: mx.google.com; dkim=pass header.i=@gcc.gnu.org header.s=default header.b=uv3SwLD4; spf=pass (google.com: domain of gcc-patches-return-467150-patch=linaro.org@gcc.gnu.org designates 209.132.180.131 as permitted sender) smtp.mailfrom=gcc-patches-return-467150-patch=linaro.org@gcc.gnu.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org DomainKey-Signature: a=rsa-sha1; c=nofws; d=gcc.gnu.org; h=list-id :list-unsubscribe:list-archive:list-post:list-help:sender:from :to:subject:date:message-id:mime-version:content-type; q=dns; s= default; b=Z0sdel0qqhLjN/Ek13P1zzbL/cc2rudQsaResJIcOHEB8GnC+MvcS Ocfzgqj9zWWU7hXtsePnHjXP5o8U3lGXSOFpjpGRSPkrg6T2RBFqfSoW1IwI6Eno OW/OqYnezRqIUOqCvyc2HW4rvDRRSpMHz0ZAc4JcnwyVBcaRqYDJRw= DKIM-Signature: v=1; a=rsa-sha1; c=relaxed; d=gcc.gnu.org; h=list-id :list-unsubscribe:list-archive:list-post:list-help:sender:from :to:subject:date:message-id:mime-version:content-type; s= default; bh=NMke9ywt5L8uztih9IUSkLwRzhY=; b=uv3SwLD4vf1yLO/Afl87 kyEbW6MS+H1zSwLPPBps+ZIggXsK9eDleA/q59Py78nVLPlJvWAnOS9NPlfrziyK xI1F1LegaVidJXskvJHNt4YR76d3p/noMKcnkckrRYFeYBTaINA2lKROreJ8c1tL xR6kNPgGqKNP8yR+yAa3JP0= Received: (qmail 18803 invoked by alias); 17 Nov 2017 15:15:56 -0000 Mailing-List: contact gcc-patches-help@gcc.gnu.org; run by ezmlm Precedence: bulk List-Id: List-Unsubscribe: List-Archive: List-Post: List-Help: Sender: gcc-patches-owner@gcc.gnu.org Delivered-To: mailing list gcc-patches@gcc.gnu.org Received: (qmail 18698 invoked by uid 89); 17 Nov 2017 15:15:54 -0000 Authentication-Results: sourceware.org; auth=none X-Virus-Found: No X-Spam-SWARE-Status: No, score=-9.5 required=5.0 tests=AWL, BAYES_00, GIT_PATCH_2, GIT_PATCH_3, KAM_ASCII_DIVIDERS, KB_WAM_FROM_NAME_SINGLEWORD, RCVD_IN_DNSWL_NONE, SPF_PASS, URIBL_BLACK autolearn=ham version=3.3.2 spammy=UD:tm.texi.in, tmtexiin, tm.texi.in, targhooksh X-HELO: mail-wm0-f50.google.com Received: from mail-wm0-f50.google.com (HELO mail-wm0-f50.google.com) (74.125.82.50) by sourceware.org (qpsmtpd/0.93/v0.84-503-g423c35a) with ESMTP; Fri, 17 Nov 2017 15:15:52 +0000 Received: by mail-wm0-f50.google.com with SMTP id b189so7148350wmd.0 for ; Fri, 17 Nov 2017 07:15:51 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:mail-followup-to:subject:date:message-id :user-agent:mime-version; bh=ebWbQQ1iB5g02P1acYVi/Rpfxm8Sd5MLNbR398ChcNI=; b=S2b+whikgKQkkDiLz+ZV0+1UI6bhpv2b1hwyRgraRV2V3o3iP/hj+jPY5oz1mODCKT c1hOsgnwj3r6f81Cfky2YsTOo/SoofSa86LxgBTH8I4AsokAnxKdvFnOlQ/SMbWWJU/6 e6pTxQk8Y35gMyODBfOI2S6LXqsGN8ab6sPHuKpHEXPRps8UMfC0nEy1/lrIfw8JKNab PIo3X48dTJUNVJvlv5/YneFGZoRUK8Ku5aWKjo6Xic+C/5hIGt5udR8dAO5iWT5YfrIz 6nN4cRx3jYPA7+EZOnbgRdg/xU7GagsyKZiHMjh5UUfdte3SjiZbPcRK1RLUzATZYjlA h4DA== X-Gm-Message-State: AJaThX5LTayJTXsCquCbht2ELBrISS3iAc/5IGtCKlUMxdUDChaBmabY SsxnmuVAPZUiV2HNkquz/hdOTiPx2hg= X-Received: by 10.28.210.129 with SMTP id j123mr4493577wmg.52.1510931749618; Fri, 17 Nov 2017 07:15:49 -0800 (PST) Received: from localhost ([2.25.234.120]) by smtp.gmail.com with ESMTPSA id n32sm3728068wrb.62.2017.11.17.07.15.48 for (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Fri, 17 Nov 2017 07:15:48 -0800 (PST) From: Richard Sandiford To: gcc-patches@gcc.gnu.org Mail-Followup-To: gcc-patches@gcc.gnu.org, richard.sandiford@linaro.org Subject: Add an empty_mask_is_expensive hook Date: Fri, 17 Nov 2017 15:15:47 +0000 Message-ID: <874lptylzw.fsf@linaro.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/25.3 (gnu/linux) MIME-Version: 1.0 This patch adds a hook to control whether we avoid executing masked (predicated) stores when the mask is all false. We don't want to do that by default for SVE. Tested on aarch64-linux-gnu (with and without SVE), x86_64-linux-gnu and powerpc64le-linux-gnu. OK to install? Richard 2017-11-17 Richard Sandiford Alan Hayward David Sherwood gcc/ * target.def (empty_mask_is_expensive): New hook. * doc/tm.texi.in (TARGET_VECTORIZE_EMPTY_MASK_IS_EXPENSIVE): New hook. * doc/tm.texi: Regenerate. * targhooks.h (default_empty_mask_is_expensive): Declare. * targhooks.c (default_empty_mask_is_expensive): New function. * tree-vectorizer.c (vectorize_loops): Only call optimize_mask_stores if the target says that empty masks are expensive. * config/aarch64/aarch64.c (aarch64_empty_mask_is_expensive): New function. (TARGET_VECTORIZE_EMPTY_MASK_IS_EXPENSIVE): Redefine. Index: gcc/target.def =================================================================== --- gcc/target.def 2017-11-17 15:07:44.219630250 +0000 +++ gcc/target.def 2017-11-17 15:14:34.529678781 +0000 @@ -1907,6 +1907,17 @@ if such a mode exists.", (poly_uint64 nunits, poly_uint64 length), default_get_mask_mode) +/* Function to say whether a masked operation is expensive when the + mask is all zeros. */ +DEFHOOK +(empty_mask_is_expensive, + "This hook returns true if masked internal function @var{ifn} (really of\n\ +type @code{internal_fn}) should be considered expensive when the mask is\n\ +all zeros. GCC can then try to branch around the instruction instead.", + bool, + (unsigned ifn), + default_empty_mask_is_expensive) + /* Target builtin that implements vector gather operation. */ DEFHOOK (builtin_gather, Index: gcc/doc/tm.texi.in =================================================================== --- gcc/doc/tm.texi.in 2017-11-17 15:07:44.217630250 +0000 +++ gcc/doc/tm.texi.in 2017-11-17 15:14:34.529678781 +0000 @@ -4095,6 +4095,8 @@ address; but often a machine-dependent @hook TARGET_VECTORIZE_GET_MASK_MODE +@hook TARGET_VECTORIZE_EMPTY_MASK_IS_EXPENSIVE + @hook TARGET_VECTORIZE_INIT_COST @hook TARGET_VECTORIZE_ADD_STMT_COST Index: gcc/doc/tm.texi =================================================================== --- gcc/doc/tm.texi 2017-11-17 15:07:44.216630250 +0000 +++ gcc/doc/tm.texi 2017-11-17 15:14:34.528767752 +0000 @@ -5884,6 +5884,12 @@ is @var{length} bytes long and that cont if such a mode exists. @end deftypefn +@deftypefn {Target Hook} bool TARGET_VECTORIZE_EMPTY_MASK_IS_EXPENSIVE (unsigned @var{ifn}) +This hook returns true if masked internal function @var{ifn} (really of +type @code{internal_fn}) should be considered expensive when the mask is +all zeros. GCC can then try to branch around the instruction instead. +@end deftypefn + @deftypefn {Target Hook} {void *} TARGET_VECTORIZE_INIT_COST (struct loop *@var{loop_info}) This hook should initialize target-specific data structures in preparation for modeling the costs of vectorizing a loop or basic block. The default allocates three unsigned integers for accumulating costs for the prologue, body, and epilogue of the loop or basic block. If @var{loop_info} is non-NULL, it identifies the loop being vectorized; otherwise a single block is being vectorized. @end deftypefn Index: gcc/targhooks.h =================================================================== --- gcc/targhooks.h 2017-11-17 15:07:43.533630267 +0000 +++ gcc/targhooks.h 2017-11-17 15:14:34.530589811 +0000 @@ -110,6 +110,7 @@ default_builtin_support_vector_misalignm extern machine_mode default_preferred_simd_mode (scalar_mode mode); extern void default_autovectorize_vector_sizes (vector_sizes *); extern opt_machine_mode default_get_mask_mode (poly_uint64, poly_uint64); +extern bool default_empty_mask_is_expensive (unsigned); extern void *default_init_cost (struct loop *); extern unsigned default_add_stmt_cost (void *, int, enum vect_cost_for_stmt, struct _stmt_vec_info *, int, Index: gcc/targhooks.c =================================================================== --- gcc/targhooks.c 2017-11-17 15:07:43.987630256 +0000 +++ gcc/targhooks.c 2017-11-17 15:14:34.530589811 +0000 @@ -1305,6 +1305,14 @@ default_get_mask_mode (poly_uint64 nunit return opt_machine_mode (); } +/* By default consider masked stores to be expensive. */ + +bool +default_empty_mask_is_expensive (unsigned ifn) +{ + return ifn == IFN_MASK_STORE; +} + /* By default, the cost model accumulates three separate costs (prologue, loop body, and epilogue) for a vectorized loop or block. So allocate an array of three unsigned ints, set it to zero, and return its address. */ Index: gcc/tree-vectorizer.c =================================================================== --- gcc/tree-vectorizer.c 2017-11-17 15:07:38.288630400 +0000 +++ gcc/tree-vectorizer.c 2017-11-17 15:14:34.530589811 +0000 @@ -847,7 +847,8 @@ vectorize_loops (void) if (loop_vinfo) has_mask_store = LOOP_VINFO_HAS_MASK_STORE (loop_vinfo); delete loop_vinfo; - if (has_mask_store) + if (has_mask_store + && targetm.vectorize.empty_mask_is_expensive (IFN_MASK_STORE)) optimize_mask_stores (loop); loop->aux = NULL; } Index: gcc/config/aarch64/aarch64.c =================================================================== --- gcc/config/aarch64/aarch64.c 2017-11-17 15:07:44.230630249 +0000 +++ gcc/config/aarch64/aarch64.c 2017-11-17 15:14:34.526945692 +0000 @@ -17016,6 +17016,16 @@ aarch64_gen_adjusted_ldpstp (rtx *operan return true; } +/* Implement TARGET_VECTORIZE_EMPTY_MASK_IS_EXPENSIVE. Assume for now that + it isn't worth branching around empty masked ops (including masked + stores). */ + +static bool +aarch64_empty_mask_is_expensive (unsigned) +{ + return false; +} + /* Return 1 if pseudo register should be created and used to hold GOT address for PIC code. */ @@ -17660,6 +17670,9 @@ #define TARGET_VECTORIZE_VEC_PERM_CONST_ #undef TARGET_VECTORIZE_GET_MASK_MODE #define TARGET_VECTORIZE_GET_MASK_MODE aarch64_get_mask_mode +#undef TARGET_VECTORIZE_EMPTY_MASK_IS_EXPENSIVE +#define TARGET_VECTORIZE_EMPTY_MASK_IS_EXPENSIVE \ + aarch64_empty_mask_is_expensive #undef TARGET_INIT_LIBFUNCS #define TARGET_INIT_LIBFUNCS aarch64_init_libfuncs