From patchwork Mon Oct 23 17:25:02 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Sandiford X-Patchwork-Id: 116805 Delivered-To: patch@linaro.org Received: by 10.140.22.164 with SMTP id 33csp4889757qgn; Mon, 23 Oct 2017 10:25:23 -0700 (PDT) X-Received: by 10.99.44.208 with SMTP id s199mr12252407pgs.80.1508779523534; Mon, 23 Oct 2017 10:25:23 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1508779523; cv=none; d=google.com; s=arc-20160816; b=DBwvovyoUL3znlYsKwOW/4YTjQWWmTEhRP8kQqtnYIQNO7/O5X/0AetBH+ATMyZ/D/ l0qaKSlo0o0YSkfqvbmYba1upz3ha7ItWsLvaYubJULgKumvj7UphfL35Tk0E/6Z+a/4 tVdUnUhD+8X193uIO/HiwKCnp/26JtEM4srwdhUL83p80oXbSBS2A75v2Es029+gWvq8 BKuJJramVecAci9u6fJmVGw618PRKHMbaaGu66Wj3U0c3EDZy2LFxn3+UWFggPrJM4cs gV5MdSRkub1f2KV+xX356T+KDL6POARUBBU1OukKfzzAbs5n77tVNausCiBXbVmVpi6p RVHw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=mime-version:user-agent:message-id:in-reply-to:date:references :subject:mail-followup-to:to:from:delivered-to:sender:list-help :list-post:list-archive:list-unsubscribe:list-id:precedence :mailing-list:dkim-signature:domainkey-signature :arc-authentication-results; bh=xOFEZoYChArpkOGkzC0txOO+Yn4YR2VC27NIfrblWWw=; b=hRiBe7Ovk9MmQD5wqy8k4NFFJc8pObe9cjF1esIPCcw03EUXB69hSpbs+ZauHlPycZ hazVur4eEWE00Jf4OZzyybUUru+YdXSbeCghecfygb1CQGRVy8sfGehFbKFjKWebhdrc S868Jy6wQGU0hoP9gtMNQh4KCo305cf2+UEyhms25AK1eMMVxfedYopkCP12eS8NRHXF n7+8Wyyg+FXX5qtJsB/srBW53ScmD88G87Sr+q+DfLod6ExoNVE4qH9vS5f3bL6u5zaj gEEiAw+wX7KNnc80ATlxfQlmhtCSNDh1O1sGmn25DycmTwN0i6325w52GOdOH/VI7NSd uPqA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@gcc.gnu.org header.s=default header.b=letskDSk; spf=pass (google.com: domain of gcc-patches-return-464836-patch=linaro.org@gcc.gnu.org designates 209.132.180.131 as permitted sender) smtp.mailfrom=gcc-patches-return-464836-patch=linaro.org@gcc.gnu.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from sourceware.org (server1.sourceware.org. [209.132.180.131]) by mx.google.com with ESMTPS id q5si5105547pgp.196.2017.10.23.10.25.23 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 23 Oct 2017 10:25:23 -0700 (PDT) Received-SPF: pass (google.com: domain of gcc-patches-return-464836-patch=linaro.org@gcc.gnu.org designates 209.132.180.131 as permitted sender) client-ip=209.132.180.131; Authentication-Results: mx.google.com; dkim=pass header.i=@gcc.gnu.org header.s=default header.b=letskDSk; spf=pass (google.com: domain of gcc-patches-return-464836-patch=linaro.org@gcc.gnu.org designates 209.132.180.131 as permitted sender) smtp.mailfrom=gcc-patches-return-464836-patch=linaro.org@gcc.gnu.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org DomainKey-Signature: a=rsa-sha1; c=nofws; d=gcc.gnu.org; h=list-id :list-unsubscribe:list-archive:list-post:list-help:sender:from :to:subject:references:date:in-reply-to:message-id:mime-version :content-type; q=dns; s=default; b=I6v151H0eaemZWSabHESKXeI3/pJ0 tte4rDXAbLXcXaccygC/TCfOy7ikDVR4F7voYid4bfrtAMrEV6I+PxaBrtD665C5 mfmwaX9ud5WCbQMjdiAuBskgKaw5aaz1UIncuJrtJNXWroTL9OkUlggzw6pDvixf IaDzj5+hCHjudY= DKIM-Signature: v=1; a=rsa-sha1; c=relaxed; d=gcc.gnu.org; h=list-id :list-unsubscribe:list-archive:list-post:list-help:sender:from :to:subject:references:date:in-reply-to:message-id:mime-version :content-type; s=default; bh=qjoT3FotKpy65bmLG6JPaUHDWeM=; b=let skDSkMgdzZOGHizdigmlq0wsYWcd71tUl8bgQOpoel1yTVZRSTQOiADRueMc6iwb 1IttWQQDsur8U+AL2UM9T1+mbhWrhNTQcZrZPceGgNYnSIjk249RvIp7PlEa0in2 3aQY+XabdcDJ/l15ZAt/TbBTkAZNkuIzgluMFKlU= Received: (qmail 61843 invoked by alias); 23 Oct 2017 17:25:08 -0000 Mailing-List: contact gcc-patches-help@gcc.gnu.org; run by ezmlm Precedence: bulk List-Id: List-Unsubscribe: List-Archive: List-Post: List-Help: Sender: gcc-patches-owner@gcc.gnu.org Delivered-To: mailing list gcc-patches@gcc.gnu.org Received: (qmail 61724 invoked by uid 89); 23 Oct 2017 17:25:07 -0000 Authentication-Results: sourceware.org; auth=none X-Virus-Found: No X-Spam-SWARE-Status: No, score=-11.0 required=5.0 tests=AWL, BAYES_00, GIT_PATCH_2, GIT_PATCH_3, KAM_ASCII_DIVIDERS, RCVD_IN_DNSWL_NONE, SPF_PASS autolearn=ham version=3.3.2 spammy= X-HELO: mail-wr0-f179.google.com Received: from mail-wr0-f179.google.com (HELO mail-wr0-f179.google.com) (209.85.128.179) by sourceware.org (qpsmtpd/0.93/v0.84-503-g423c35a) with ESMTP; Mon, 23 Oct 2017 17:25:05 +0000 Received: by mail-wr0-f179.google.com with SMTP id j15so6027075wre.8 for ; Mon, 23 Oct 2017 10:25:05 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:mail-followup-to:subject:references:date :in-reply-to:message-id:user-agent:mime-version; bh=xOFEZoYChArpkOGkzC0txOO+Yn4YR2VC27NIfrblWWw=; b=Xl16nOi5GfzlAUonDY6e6jUPTtVtQKgiS75pivuKfPUuE+YPXQtx8wg/UDOyRI5w0T WUYK+JP+owQgOwi2Q3KgKw5fmRxMsrF3PLAuKIesHXGkqrQiuwsO6fT/ey10DNHYzvXR 981b5eimyYmh+d1apE48iWzhOEfWd7U8iKF0hAMNxCpZj8SPfp9YAWhXR35qUSbAPoZA HPpT4Q4v1IJOZ73O4e7xePuqz1pB2SG5Yy9FTdfDs8XfP2yGefiFlYjChALYkBsJveVv T8COsplV4SPvvNJedKnFgzcE6w2jNA9bopNpHhzp/Pcfa68t25xqPFFGMPNqZ+TNXmDh UrWg== X-Gm-Message-State: AMCzsaVVcXXT9yadecBgUNy8JCTd8dNnMo+oRfKIAG3BPrXiwEC3jqSR rEVaF95lJpHxpWX0s9Rrby2JBQbhgmg= X-Google-Smtp-Source: ABhQp+QxwDGhRM+IFpZsrH0faTbvJoQNWy2k6G61bsyot2Gbzb1kKAPeaUlheNrobMCdr6c8poEAdg== X-Received: by 10.223.153.234 with SMTP id y97mr5578611wrb.165.1508779503605; Mon, 23 Oct 2017 10:25:03 -0700 (PDT) Received: from localhost ([2.26.27.199]) by smtp.gmail.com with ESMTPSA id m64sm4672116wmb.10.2017.10.23.10.25.02 for (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Mon, 23 Oct 2017 10:25:02 -0700 (PDT) From: Richard Sandiford To: gcc-patches@gcc.gnu.org Mail-Followup-To: gcc-patches@gcc.gnu.org, richard.sandiford@linaro.org Subject: [060/nnn] poly_int: loop versioning threshold References: <871sltvm7r.fsf@linaro.org> Date: Mon, 23 Oct 2017 18:25:02 +0100 In-Reply-To: <871sltvm7r.fsf@linaro.org> (Richard Sandiford's message of "Mon, 23 Oct 2017 17:54:32 +0100") Message-ID: <87o9oxkc9d.fsf@linaro.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/25.2 (gnu/linux) MIME-Version: 1.0 This patch splits the loop versioning threshold out from the cost model threshold so that the former can become a poly_uint64. We still use a single test to enforce both limits where possible. 2017-10-23 Richard Sandiford Alan Hayward David Sherwood gcc/ * tree-vectorizer.h (_loop_vec_info): Add a versioning_threshold field. (LOOP_VINFO_VERSIONING_THRESHOLD): New macro (vect_loop_versioning): Take the loop versioning threshold as a separate parameter. * tree-vect-loop-manip.c (vect_loop_versioning): Likewise. * tree-vect-loop.c (_loop_vec_info::_loop_vec_info): Initialize versioning_threshold. (vect_analyze_loop_2): Compute the loop versioning threshold whenever loop versioning is needed, and store it in the new field rather than combining it with the cost model threshold. (vect_transform_loop): Update call to vect_loop_versioning. Try to combine the loop versioning and cost thresholds here. Index: gcc/tree-vectorizer.h =================================================================== --- gcc/tree-vectorizer.h 2017-10-23 17:11:39.817127625 +0100 +++ gcc/tree-vectorizer.h 2017-10-23 17:22:23.377858186 +0100 @@ -238,6 +238,12 @@ typedef struct _loop_vec_info : public v PARAM_MIN_VECT_LOOP_BOUND. */ unsigned int th; + /* When applying loop versioning, the vector form should only be used + if the number of scalar iterations is >= this value, on top of all + the other requirements. Ignored when loop versioning is not being + used. */ + poly_uint64 versioning_threshold; + /* Unrolling factor */ int vectorization_factor; @@ -357,6 +363,7 @@ #define LOOP_VINFO_NITERS(L) #define LOOP_VINFO_NITERS_UNCHANGED(L) (L)->num_iters_unchanged #define LOOP_VINFO_NITERS_ASSUMPTIONS(L) (L)->num_iters_assumptions #define LOOP_VINFO_COST_MODEL_THRESHOLD(L) (L)->th +#define LOOP_VINFO_VERSIONING_THRESHOLD(L) (L)->versioning_threshold #define LOOP_VINFO_VECTORIZABLE_P(L) (L)->vectorizable #define LOOP_VINFO_VECT_FACTOR(L) (L)->vectorization_factor #define LOOP_VINFO_MAX_VECT_FACTOR(L) (L)->max_vectorization_factor @@ -1143,7 +1150,8 @@ extern void slpeel_make_loop_iterate_nti extern bool slpeel_can_duplicate_loop_p (const struct loop *, const_edge); struct loop *slpeel_tree_duplicate_loop_to_edge_cfg (struct loop *, struct loop *, edge); -extern void vect_loop_versioning (loop_vec_info, unsigned int, bool); +extern void vect_loop_versioning (loop_vec_info, unsigned int, bool, + poly_uint64); extern struct loop *vect_do_peeling (loop_vec_info, tree, tree, tree *, tree *, tree *, int, bool, bool); extern source_location find_loop_location (struct loop *); Index: gcc/tree-vect-loop-manip.c =================================================================== --- gcc/tree-vect-loop-manip.c 2017-10-23 17:11:39.816125711 +0100 +++ gcc/tree-vect-loop-manip.c 2017-10-23 17:22:23.376857985 +0100 @@ -2295,7 +2295,8 @@ vect_create_cond_for_alias_checks (loop_ void vect_loop_versioning (loop_vec_info loop_vinfo, - unsigned int th, bool check_profitability) + unsigned int th, bool check_profitability, + poly_uint64 versioning_threshold) { struct loop *loop = LOOP_VINFO_LOOP (loop_vinfo), *nloop; struct loop *scalar_loop = LOOP_VINFO_SCALAR_LOOP (loop_vinfo); @@ -2320,6 +2321,17 @@ vect_loop_versioning (loop_vec_info loop cond_expr = fold_build2 (GE_EXPR, boolean_type_node, scalar_loop_iters, build_int_cst (TREE_TYPE (scalar_loop_iters), th - 1)); + if (maybe_nonzero (versioning_threshold)) + { + tree expr = fold_build2 (GE_EXPR, boolean_type_node, scalar_loop_iters, + build_int_cst (TREE_TYPE (scalar_loop_iters), + versioning_threshold - 1)); + if (cond_expr) + cond_expr = fold_build2 (BIT_AND_EXPR, boolean_type_node, + expr, cond_expr); + else + cond_expr = expr; + } if (version_niter) vect_create_cond_for_niters_checks (loop_vinfo, &cond_expr); Index: gcc/tree-vect-loop.c =================================================================== --- gcc/tree-vect-loop.c 2017-10-23 17:11:39.816125711 +0100 +++ gcc/tree-vect-loop.c 2017-10-23 17:22:23.377858186 +0100 @@ -1110,6 +1110,7 @@ _loop_vec_info::_loop_vec_info (struct l num_iters_unchanged (NULL_TREE), num_iters_assumptions (NULL_TREE), th (0), + versioning_threshold (0), vectorization_factor (0), max_vectorization_factor (0), unaligned_dr (NULL), @@ -2174,11 +2175,9 @@ vect_analyze_loop_2 (loop_vec_info loop_ enough for both peeled prolog loop and vector loop. This check can be merged along with threshold check of loop versioning, so increase threshold for this case if necessary. */ - if (LOOP_REQUIRES_VERSIONING (loop_vinfo) - && (LOOP_VINFO_PEELING_FOR_GAPS (loop_vinfo) - || LOOP_VINFO_PEELING_FOR_NITER (loop_vinfo))) + if (LOOP_REQUIRES_VERSIONING (loop_vinfo)) { - unsigned niters_th; + poly_uint64 niters_th; /* Niters for peeled prolog loop. */ if (LOOP_VINFO_PEELING_FOR_ALIGNMENT (loop_vinfo) < 0) @@ -2195,9 +2194,8 @@ vect_analyze_loop_2 (loop_vec_info loop_ niters_th += LOOP_VINFO_VECT_FACTOR (loop_vinfo); /* One additional iteration because of peeling for gap. */ if (LOOP_VINFO_PEELING_FOR_GAPS (loop_vinfo)) - niters_th++; - if (LOOP_VINFO_COST_MODEL_THRESHOLD (loop_vinfo) < niters_th) - LOOP_VINFO_COST_MODEL_THRESHOLD (loop_vinfo) = niters_th; + niters_th += 1; + LOOP_VINFO_VERSIONING_THRESHOLD (loop_vinfo) = niters_th; } gcc_assert (vectorization_factor @@ -2300,6 +2298,7 @@ vect_analyze_loop_2 (loop_vec_info loop_ LOOP_VINFO_PEELING_FOR_NITER (loop_vinfo) = false; LOOP_VINFO_PEELING_FOR_GAPS (loop_vinfo) = false; LOOP_VINFO_COST_MODEL_THRESHOLD (loop_vinfo) = 0; + LOOP_VINFO_VERSIONING_THRESHOLD (loop_vinfo) = 0; goto start_over; } @@ -7320,7 +7319,17 @@ vect_transform_loop (loop_vec_info loop_ if (LOOP_REQUIRES_VERSIONING (loop_vinfo)) { - vect_loop_versioning (loop_vinfo, th, check_profitability); + poly_uint64 versioning_threshold + = LOOP_VINFO_VERSIONING_THRESHOLD (loop_vinfo); + if (check_profitability + && ordered_p (poly_uint64 (th), versioning_threshold)) + { + versioning_threshold = ordered_max (poly_uint64 (th), + versioning_threshold); + check_profitability = false; + } + vect_loop_versioning (loop_vinfo, th, check_profitability, + versioning_threshold); check_profitability = false; }