From patchwork Thu Sep 14 11:22:06 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Sandiford X-Patchwork-Id: 112541 Delivered-To: patch@linaro.org Received: by 10.140.106.117 with SMTP id d108csp624262qgf; Thu, 14 Sep 2017 04:22:26 -0700 (PDT) X-Received: by 10.98.158.201 with SMTP id f70mr21682866pfk.162.1505388146587; Thu, 14 Sep 2017 04:22:26 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1505388146; cv=none; d=google.com; s=arc-20160816; b=d+sOjF4NuUwJX1Z2rxClqY6oUx5VKdDfyvsnu43oY3dNrg4+gHhs8qa1rN4wkZpcZP q2r0Na9ha+/jL9+i7AsQphUx013iRolvxlB4fWQFElpzJE5hMstRXTENktYBYawOPx5z +yaP0yWvoIaql8o9mSXbLvLiH2d7DlVrmq1hg4tEDSkyV5WUZ4fcXrgFSQdE15aQ8Q13 ktCXGO53O9o5uGZFkYA59Ofc6DhkdsQmuRX4fu12zrUKp7UBB5nz65cc3WV9fYsvi9kn TrgcU0cofWVEk2JvxWQkwBNxG67SWex3flRbLixfU2Vf8jRvYvacidaWSdu2J+r3lce+ +HIg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=mime-version:user-agent:message-id:date:subject:mail-followup-to:to :from:delivered-to:sender:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:mailing-list:dkim-signature :domainkey-signature:arc-authentication-results; bh=m+rn4IlVUTNPtHqHF6zvxA4oJTfUs6qh0BNgW6zV3ho=; b=cD6Xjf+IBNffl4NmGHi2WX5KRumGd42dsMileiQoWBGydrrFenn3NVoXADNKEqIWCB Rr8dzsX++ER6Dv43vkxoId1uFswxYxJVaspSFl4ZxlvxbNu1QD5cnLQ7CuCqmQ4+RTo8 CXnakJH+7JV8rpB+vpKGs4680/WshIZmB7YaymXw4WKmX9BybaedWvugAbgCRCwrQMwB FRJ4Bo6lC3j1hEVMEQvvXBsMYHlMQJyVcqSNIFE1hA0Wdm0q9zQusnwz61V6cTlnpqCV yY68Co/vzb0HQseMMNeZH79iN5cYEXJ5so4FI2YqL0WPNBNA07WdrkLs+N77xMwhypOu RR3g== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@gcc.gnu.org header.s=default header.b=RXzYF6Iq; spf=pass (google.com: domain of gcc-patches-return-462121-patch=linaro.org@gcc.gnu.org designates 209.132.180.131 as permitted sender) smtp.mailfrom=gcc-patches-return-462121-patch=linaro.org@gcc.gnu.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from sourceware.org (server1.sourceware.org. [209.132.180.131]) by mx.google.com with ESMTPS id m10si896112pgp.211.2017.09.14.04.22.26 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 14 Sep 2017 04:22:26 -0700 (PDT) Received-SPF: pass (google.com: domain of gcc-patches-return-462121-patch=linaro.org@gcc.gnu.org designates 209.132.180.131 as permitted sender) client-ip=209.132.180.131; Authentication-Results: mx.google.com; dkim=pass header.i=@gcc.gnu.org header.s=default header.b=RXzYF6Iq; spf=pass (google.com: domain of gcc-patches-return-462121-patch=linaro.org@gcc.gnu.org designates 209.132.180.131 as permitted sender) smtp.mailfrom=gcc-patches-return-462121-patch=linaro.org@gcc.gnu.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org DomainKey-Signature: a=rsa-sha1; c=nofws; d=gcc.gnu.org; h=list-id :list-unsubscribe:list-archive:list-post:list-help:sender:from :to:subject:date:message-id:mime-version:content-type; q=dns; s= default; b=qJsBSkXZtpoFxSGSqqoZPBsLWZfXs2XEPbShGG8wPR7RNP18YDOEL 04Oy/chC38qQ2f/qKIETAo1UA/iDKnOvtZe6O2y1BffNTO8sQWEqVNGoYU1yq/Sg 52Wyef3MqTSa2QhEFBFR2VIkzSZ4naX1gGug5hgXdRUT+RNsfnJmbo= DKIM-Signature: v=1; a=rsa-sha1; c=relaxed; d=gcc.gnu.org; h=list-id :list-unsubscribe:list-archive:list-post:list-help:sender:from :to:subject:date:message-id:mime-version:content-type; s= default; bh=BTGjJ8EJhjDw8tddmeLVd3pJyes=; b=RXzYF6Iqeof2Y/N5Irfa ORScrwC5Hxr8HwmvH8s9jtzUJPGzrkAlo46K+Ge46hkp5tI7rJMN05lI9Ho7VK6e NRT18n90bKm4wwuP0TtnXW1pt7Y5Vbui8wJn4qDUrK9DNN0m7ylyWsq6mRF5yaAp l419tjrnkORf8gucG/fSiR8= Received: (qmail 122818 invoked by alias); 14 Sep 2017 11:22:14 -0000 Mailing-List: contact gcc-patches-help@gcc.gnu.org; run by ezmlm Precedence: bulk List-Id: List-Unsubscribe: List-Archive: List-Post: List-Help: Sender: gcc-patches-owner@gcc.gnu.org Delivered-To: mailing list gcc-patches@gcc.gnu.org Received: (qmail 122276 invoked by uid 89); 14 Sep 2017 11:22:13 -0000 Authentication-Results: sourceware.org; auth=none X-Virus-Found: No X-Spam-SWARE-Status: No, score=-15.5 required=5.0 tests=AWL, BAYES_00, GIT_PATCH_1, GIT_PATCH_2, GIT_PATCH_3, KAM_ASCII_DIVIDERS, RCVD_IN_DNSWL_NONE, SPF_PASS autolearn=ham version=3.3.2 spammy=67137 X-HELO: mail-wr0-f169.google.com Received: from mail-wr0-f169.google.com (HELO mail-wr0-f169.google.com) (209.85.128.169) by sourceware.org (qpsmtpd/0.93/v0.84-503-g423c35a) with ESMTP; Thu, 14 Sep 2017 11:22:11 +0000 Received: by mail-wr0-f169.google.com with SMTP id 108so5595114wra.5 for ; Thu, 14 Sep 2017 04:22:11 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:mail-followup-to:subject:date:message-id :user-agent:mime-version; bh=m+rn4IlVUTNPtHqHF6zvxA4oJTfUs6qh0BNgW6zV3ho=; b=jl+vW5DRSjUF6jBIoY3PqJSTTr63O5AWdcwhVzFjuCLFokkZri5URGeKeKlQg7NdwU M0cGSvtAr5WVwQeUQwONV3PdankVb3eldbL0yeQHEqfI4ISapW/2fA5dA+y9f3We9Fzk gtGRPuNH8q7GNF+3iRq9Phmlyg58NF+aYBbnzx1B1CCfnsRgjaxDCs+Dmu5OKDpzHPOS c/alhVBQwFJcId5+D/x2smYHYnWmRtHGaPhSoyshTRDN5koFx7sU7VCLqpiv89Ia+a9D yoZqCL+Qzcc6EdDYCP3K5mz63qCm3GA6WFz7dpj2WI5725fTJaD1QU62ShZiX8HZ+4so /oiQ== X-Gm-Message-State: AHPjjUhCrrSJSrWjzJvp0daKoQwb263PEAxlLCBvBADdT/dBRVC1R2VQ r72j4CrwjjtJIX1yfeRK0g== X-Google-Smtp-Source: ADKCNb76JzVtACnqvdle4/91NWHZ2q7h3SijoSbFYIlqki78CTYZGspuW5KJUENTJtcYzRbfGTRiAg== X-Received: by 10.223.179.200 with SMTP id x8mr11999056wrd.274.1505388129471; Thu, 14 Sep 2017 04:22:09 -0700 (PDT) Received: from localhost (92.40.248.70.threembb.co.uk. [92.40.248.70]) by smtp.gmail.com with ESMTPSA id f13sm1356145wre.66.2017.09.14.04.22.08 for (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Thu, 14 Sep 2017 04:22:08 -0700 (PDT) From: Richard Sandiford To: gcc-patches@gcc.gnu.org Mail-Followup-To: gcc-patches@gcc.gnu.org, richard.sandiford@linaro.org Subject: Add a vect_get_num_copies helper routine Date: Thu, 14 Sep 2017 12:22:06 +0100 Message-ID: <87y3phwmn5.fsf@linaro.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/25.2 (gnu/linux) MIME-Version: 1.0 This patch adds a vectoriser helper routine to calculate how many copies of a vector statement we need. At present this is always: LOOP_VINFO_VECT_FACTOR (loop_vinfo) / TYPE_VECTOR_SUBPARTS (vectype) but later patches add other cases. Another benefit of using a helper routine is that it can assert that the division is exact (which it must be). Tested on aarch64-linux-gnu, x86_64-linux-gnu and powerpc64le-linux-gnu. OK to install? Richard 2017-09-14 Richard Sandiford Alan Hayward David Sherwood gcc/ * tree-vectorizer.h (vect_get_num_copies): New function. * tree-vect-data-refs.c (vect_get_data_access_cost): Use it. * tree-vect-loop.c (vectorizable_reduction): Likewise. (vectorizable_induction): Likewise. (vectorizable_live_operation): Likewise. * tree-vect-stmts.c (vectorizable_mask_load_store): Likewise. (vectorizable_bswap): Likewise. (vectorizable_call): Likewise. (vectorizable_conversion): Likewise. (vectorizable_assignment): Likewise. (vectorizable_shift): Likewise. (vectorizable_operation): Likewise. (vectorizable_store): Likewise. (vectorizable_load): Likewise. (vectorizable_condition): Likewise. (vectorizable_comparison): Likewise. (vect_analyze_stmt): Pass the slp node to vectorizable_live_operation. Index: gcc/tree-vectorizer.h =================================================================== --- gcc/tree-vectorizer.h 2017-09-14 11:25:32.166167193 +0100 +++ gcc/tree-vectorizer.h 2017-09-14 11:27:50.352072753 +0100 @@ -1076,6 +1076,20 @@ unlimited_cost_model (loop_p loop) return (flag_vect_cost_model == VECT_COST_MODEL_UNLIMITED); } +/* Return the number of copies needed for loop vectorization when + a statement operates on vectors of type VECTYPE. This is the + vectorization factor divided by the number of elements in + VECTYPE and is always known at compile time. */ + +static inline unsigned int +vect_get_num_copies (loop_vec_info loop_vinfo, tree vectype) +{ + gcc_checking_assert (LOOP_VINFO_VECT_FACTOR (loop_vinfo) + % TYPE_VECTOR_SUBPARTS (vectype) == 0); + return (LOOP_VINFO_VECT_FACTOR (loop_vinfo) + / TYPE_VECTOR_SUBPARTS (vectype)); +} + /* Source location */ extern source_location vect_location; Index: gcc/tree-vect-data-refs.c =================================================================== --- gcc/tree-vect-data-refs.c 2017-09-14 11:25:32.163167193 +0100 +++ gcc/tree-vect-data-refs.c 2017-09-14 11:27:50.350257085 +0100 @@ -1181,10 +1181,13 @@ vect_get_data_access_cost (struct data_r { gimple *stmt = DR_STMT (dr); stmt_vec_info stmt_info = vinfo_for_stmt (stmt); - int nunits = TYPE_VECTOR_SUBPARTS (STMT_VINFO_VECTYPE (stmt_info)); loop_vec_info loop_vinfo = STMT_VINFO_LOOP_VINFO (stmt_info); - int vf = LOOP_VINFO_VECT_FACTOR (loop_vinfo); - int ncopies = MAX (1, vf / nunits); /* TODO: Handle SLP properly */ + int ncopies; + + if (PURE_SLP_STMT (stmt_info)) + ncopies = 1; + else + ncopies = vect_get_num_copies (loop_vinfo, STMT_VINFO_VECTYPE (stmt_info)); if (DR_IS_READ (dr)) vect_get_load_cost (dr, ncopies, true, inside_cost, outside_cost, Index: gcc/tree-vect-loop.c =================================================================== --- gcc/tree-vect-loop.c 2017-09-14 11:27:16.962234838 +0100 +++ gcc/tree-vect-loop.c 2017-09-14 11:27:50.351164919 +0100 @@ -5683,8 +5683,7 @@ vectorizable_reduction (gimple *stmt, gi if (slp_node) ncopies = 1; else - ncopies = (LOOP_VINFO_VECT_FACTOR (loop_vinfo) - / TYPE_VECTOR_SUBPARTS (vectype_in)); + ncopies = vect_get_num_copies (loop_vinfo, vectype_in); use_operand_p use_p; gimple *use_stmt; @@ -5980,8 +5979,7 @@ vectorizable_reduction (gimple *stmt, gi if (slp_node) ncopies = 1; else - ncopies = (LOOP_VINFO_VECT_FACTOR (loop_vinfo) - / TYPE_VECTOR_SUBPARTS (vectype_in)); + ncopies = vect_get_num_copies (loop_vinfo, vectype_in); gcc_assert (ncopies >= 1); @@ -6550,7 +6548,7 @@ vectorizable_induction (gimple *phi, if (slp_node) ncopies = 1; else - ncopies = vf / nunits; + ncopies = vect_get_num_copies (loop_vinfo, vectype); gcc_assert (ncopies >= 1); /* FORNOW. These restrictions should be relaxed. */ @@ -7013,12 +7011,17 @@ vectorizable_live_operation (gimple *stm tree lhs, lhs_type, bitsize, vec_bitsize; tree vectype = STMT_VINFO_VECTYPE (stmt_info); int nunits = TYPE_VECTOR_SUBPARTS (vectype); - int ncopies = LOOP_VINFO_VECT_FACTOR (loop_vinfo) / nunits; + int ncopies; gimple *use_stmt; auto_vec vec_oprnds; gcc_assert (STMT_VINFO_LIVE_P (stmt_info)); + if (slp_node) + ncopies = 1; + else + ncopies = vect_get_num_copies (loop_vinfo, vectype); + if (STMT_VINFO_DEF_TYPE (stmt_info) == vect_reduction_def) return false; Index: gcc/tree-vect-stmts.c =================================================================== --- gcc/tree-vect-stmts.c 2017-09-14 11:25:32.166167193 +0100 +++ gcc/tree-vect-stmts.c 2017-09-14 11:27:50.352072753 +0100 @@ -2038,7 +2038,7 @@ vectorizable_mask_load_store (gimple *st if (slp_node != NULL) return false; - ncopies = LOOP_VINFO_VECT_FACTOR (loop_vinfo) / nunits; + ncopies = vect_get_num_copies (loop_vinfo, vectype); gcc_assert (ncopies >= 1); mask = gimple_call_arg (stmt, 2); @@ -2472,7 +2472,7 @@ vectorizable_bswap (gimple *stmt, gimple if (slp_node) ncopies = 1; else - ncopies = LOOP_VINFO_VECT_FACTOR (loop_vinfo) / nunits; + ncopies = vect_get_num_copies (loop_vinfo, vectype); gcc_assert (ncopies >= 1); @@ -2805,9 +2805,9 @@ vectorizable_call (gimple *gs, gimple_st if (slp_node) ncopies = 1; else if (modifier == NARROW && ifn == IFN_LAST) - ncopies = LOOP_VINFO_VECT_FACTOR (loop_vinfo) / nunits_out; + ncopies = vect_get_num_copies (loop_vinfo, vectype_out); else - ncopies = LOOP_VINFO_VECT_FACTOR (loop_vinfo) / nunits_in; + ncopies = vect_get_num_copies (loop_vinfo, vectype_in); /* Sanity check: make sure that at least one copy of the vectorized stmt needs to be generated. */ @@ -4204,9 +4204,9 @@ vectorizable_conversion (gimple *stmt, g if (slp_node) ncopies = 1; else if (modifier == NARROW) - ncopies = LOOP_VINFO_VECT_FACTOR (loop_vinfo) / nunits_out; + ncopies = vect_get_num_copies (loop_vinfo, vectype_out); else - ncopies = LOOP_VINFO_VECT_FACTOR (loop_vinfo) / nunits_in; + ncopies = vect_get_num_copies (loop_vinfo, vectype_in); /* Sanity check: make sure that at least one copy of the vectorized stmt needs to be generated. */ @@ -4690,7 +4690,7 @@ vectorizable_assignment (gimple *stmt, g if (slp_node) ncopies = 1; else - ncopies = LOOP_VINFO_VECT_FACTOR (loop_vinfo) / nunits; + ncopies = vect_get_num_copies (loop_vinfo, vectype); gcc_assert (ncopies >= 1); @@ -4948,7 +4948,7 @@ vectorizable_shift (gimple *stmt, gimple if (slp_node) ncopies = 1; else - ncopies = LOOP_VINFO_VECT_FACTOR (loop_vinfo) / nunits_in; + ncopies = vect_get_num_copies (loop_vinfo, vectype); gcc_assert (ncopies >= 1); @@ -5379,7 +5379,7 @@ vectorizable_operation (gimple *stmt, gi if (slp_node) ncopies = 1; else - ncopies = LOOP_VINFO_VECT_FACTOR (loop_vinfo) / nunits_in; + ncopies = vect_get_num_copies (loop_vinfo, vectype); gcc_assert (ncopies >= 1); @@ -5718,7 +5718,7 @@ vectorizable_store (gimple *stmt, gimple if (slp) ncopies = 1; else - ncopies = LOOP_VINFO_VECT_FACTOR (loop_vinfo) / nunits; + ncopies = vect_get_num_copies (loop_vinfo, vectype); gcc_assert (ncopies >= 1); @@ -6713,7 +6713,7 @@ vectorizable_load (gimple *stmt, gimple_ if (slp) ncopies = 1; else - ncopies = LOOP_VINFO_VECT_FACTOR (loop_vinfo) / nunits; + ncopies = vect_get_num_copies (loop_vinfo, vectype); gcc_assert (ncopies >= 1); @@ -7926,13 +7926,12 @@ vectorizable_condition (gimple *stmt, gi return false; tree vectype = STMT_VINFO_VECTYPE (stmt_info); - int nunits = TYPE_VECTOR_SUBPARTS (vectype); tree vectype1 = NULL_TREE, vectype2 = NULL_TREE; if (slp_node) ncopies = 1; else - ncopies = LOOP_VINFO_VECT_FACTOR (loop_vinfo) / nunits; + ncopies = vect_get_num_copies (loop_vinfo, vectype); gcc_assert (ncopies >= 1); if (reduc_index && ncopies > 1) @@ -8269,7 +8268,7 @@ vectorizable_comparison (gimple *stmt, g if (slp_node) ncopies = 1; else - ncopies = LOOP_VINFO_VECT_FACTOR (loop_vinfo) / nunits; + ncopies = vect_get_num_copies (loop_vinfo, vectype); gcc_assert (ncopies >= 1); if (STMT_VINFO_DEF_TYPE (stmt_info) != vect_internal_def @@ -8700,7 +8699,7 @@ vect_analyze_stmt (gimple *stmt, bool *n need extra handling, except for vectorizable reductions. */ if (STMT_VINFO_LIVE_P (stmt_info) && STMT_VINFO_TYPE (stmt_info) != reduc_vec_info_type) - ok = vectorizable_live_operation (stmt, NULL, NULL, -1, NULL); + ok = vectorizable_live_operation (stmt, NULL, node, -1, NULL); if (!ok) {