From patchwork Mon Oct 23 17:29:13 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Sandiford X-Patchwork-Id: 116816 Delivered-To: patch@linaro.org Received: by 10.140.22.164 with SMTP id 33csp4893620qgn; Mon, 23 Oct 2017 10:29:46 -0700 (PDT) X-Received: by 10.98.79.24 with SMTP id d24mr13889911pfb.53.1508779786737; Mon, 23 Oct 2017 10:29:46 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1508779786; cv=none; d=google.com; s=arc-20160816; b=JVU2N+y6hQgWWPqxHlK2CFlr+1xpVFqdi4nHAr5pDQ6Y1Ohu4J3yPgKZa8+I8gL9dd bRFlyJ24Gp4hvEkgV1aLShlK9dQZL32q+aI24g31+S94jH9poWcWLATChOnaN0JMRjLB dri8iVMBoocGPHdP9NNf0Mm4qnWqd2nazZxrI/h9k0VZkRM9Sv+nnHurPwOxWV7zxhPy rqVpQGQK/0W7AzP0sh5fh0ll6Ob4vnETauUalInjoVZhyf5nHmUDXCtg+0/PP6xComyq MOqPFZSpbupvhtCPriwWOeJ1EXPeE2FdQCV52r/8VbUz5X83ogJIDo7XiD7DQ3czQ6pI OWXw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=mime-version:user-agent:message-id:in-reply-to:date:references :subject:mail-followup-to:to:from:delivered-to:sender:list-help :list-post:list-archive:list-unsubscribe:list-id:precedence :mailing-list:dkim-signature:domainkey-signature :arc-authentication-results; bh=rhdCGDLdaSjQI8gG/4S7F9JBs71xnryzEi2pvg7xixU=; b=AhJ9j3qdZAklX8jwstpjPXT2W+F83GJRGOvbJaLyBXYIp6jHbDR8GF03CgERBLHMAs Z4MHf/IzqMJXSu/Nf67gE8Xx4HyH/1wNE39/scVLsBV50x4N7EAZrs5Wm9XMysK1PlOF tqVMJQRA1SNbS0J83vMZJ91c96ST3EQ2ZisuBITW6viYmA7iLQHD5jO3qME+JuGHM/4L yUFFcMKRt7K+qaITw3aI9ie4d8vdmMQm8eDcMgkmuKL6WHTrYuwXCGKfBekVdjlinm/y zk+elWApRkrV6EHqNCiqlidUICPidpr3C1ViG54EjikSZpioSAkJWPzi48kSeYczgFrG nZ6Q== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@gcc.gnu.org header.s=default header.b=sdp6+aU6; spf=pass (google.com: domain of gcc-patches-return-464847-patch=linaro.org@gcc.gnu.org designates 209.132.180.131 as permitted sender) smtp.mailfrom=gcc-patches-return-464847-patch=linaro.org@gcc.gnu.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from sourceware.org (server1.sourceware.org. [209.132.180.131]) by mx.google.com with ESMTPS id d1si5667463pfd.28.2017.10.23.10.29.46 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 23 Oct 2017 10:29:46 -0700 (PDT) Received-SPF: pass (google.com: domain of gcc-patches-return-464847-patch=linaro.org@gcc.gnu.org designates 209.132.180.131 as permitted sender) client-ip=209.132.180.131; Authentication-Results: mx.google.com; dkim=pass header.i=@gcc.gnu.org header.s=default header.b=sdp6+aU6; spf=pass (google.com: domain of gcc-patches-return-464847-patch=linaro.org@gcc.gnu.org designates 209.132.180.131 as permitted sender) smtp.mailfrom=gcc-patches-return-464847-patch=linaro.org@gcc.gnu.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org DomainKey-Signature: a=rsa-sha1; c=nofws; d=gcc.gnu.org; h=list-id :list-unsubscribe:list-archive:list-post:list-help:sender:from :to:subject:references:date:in-reply-to:message-id:mime-version :content-type; q=dns; s=default; b=Z40ZFVpbpWnEP+mn4dkK71ux2bmMe hCUY/sALUGBxobwEak54Gw0BZvvsjmYqHesqC6LJTN9hBmHDwIiXgpJ1YSoCxaVg jzw8KpV0IFTlwh11fKY0y42qgHi6Df34PIBLLMWqx7QxcVfgTOd7UxwZAKPMrA+u K+AYQFyBd7PHDs= DKIM-Signature: v=1; a=rsa-sha1; c=relaxed; d=gcc.gnu.org; h=list-id :list-unsubscribe:list-archive:list-post:list-help:sender:from :to:subject:references:date:in-reply-to:message-id:mime-version :content-type; s=default; bh=7hOQnxSJv/xMIME6ofUHRlKzhSI=; b=sdp 6+aU6nJPWwUBpPu1DxjTDF75VKgR3FFsLAJKAp8/NURT3+pd9ajY4j5LfEHugNr5 eQxIkjo3m1awvnOvxNWXerw4MYqRS08So8WU0zb2+b0K4Yj3nhSt9wBvNQ7XpEXb T/d2DUZt97ucwm5eJKQDDl/9UiFIygISLZB8bGrE= Received: (qmail 77446 invoked by alias); 23 Oct 2017 17:29:20 -0000 Mailing-List: contact gcc-patches-help@gcc.gnu.org; run by ezmlm Precedence: bulk List-Id: List-Unsubscribe: List-Archive: List-Post: List-Help: Sender: gcc-patches-owner@gcc.gnu.org Delivered-To: mailing list gcc-patches@gcc.gnu.org Received: (qmail 77308 invoked by uid 89); 23 Oct 2017 17:29:20 -0000 Authentication-Results: sourceware.org; auth=none X-Virus-Found: No X-Spam-SWARE-Status: No, score=-15.8 required=5.0 tests=AWL, BAYES_00, GIT_PATCH_1, GIT_PATCH_2, GIT_PATCH_3, RCVD_IN_DNSWL_NONE, RCVD_IN_SORBS_SPAM, SPF_PASS autolearn=ham version=3.3.2 spammy= X-HELO: mail-wr0-f193.google.com Received: from mail-wr0-f193.google.com (HELO mail-wr0-f193.google.com) (209.85.128.193) by sourceware.org (qpsmtpd/0.93/v0.84-503-g423c35a) with ESMTP; Mon, 23 Oct 2017 17:29:18 +0000 Received: by mail-wr0-f193.google.com with SMTP id o44so18170547wrf.11 for ; Mon, 23 Oct 2017 10:29:17 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:mail-followup-to:subject:references:date :in-reply-to:message-id:user-agent:mime-version; bh=rhdCGDLdaSjQI8gG/4S7F9JBs71xnryzEi2pvg7xixU=; b=LADy3mQXOdz+NIa48XZplljRs0Hq8VLH+aeLaRyz3nAQ37SaJVJonbWDl2NhCgwCjI 6SucqmOR1I52ZGd4xMRPt2BttuzzKTbD5t8WeqqDh9UGyqGMzzg6oHjGPrCuGpK0khWW dApusNI8VdAbCxsAyakBo/Ec+2U3Yb17ZyOxXaQqJD73Ma1N3sc4mLsjcZeNa19tAi2x Lxf8BAuk7bSRBaNhYdxgN504fOop2Wv8RyXAGFoZacymsadtw/5mOHHD7xrhQcBX972P wvCq1Zgtw3OxZklaVvXVX3paL/19tPKWJFv9TBkZL4GDGJ23hTSgVGN7nUBSR9sQtEFV CULg== X-Gm-Message-State: AMCzsaXOAp9/9J+teonONIn0eQzIKaqwUc8jpg2j2clbBLGKax8Ca71g NIHRMP3On2P/F7IbiRFy0DvqarV0m7k= X-Google-Smtp-Source: ABhQp+RX3p4cY1A9Mh0Iq5GnmdffjeYA3M7gI9nC6KVfEMzwd052aFK6LFswvio+jZKBOk7xK9L/ew== X-Received: by 10.223.148.38 with SMTP id 35mr13065075wrq.49.1508779755871; Mon, 23 Oct 2017 10:29:15 -0700 (PDT) Received: from localhost ([2.26.27.199]) by smtp.gmail.com with ESMTPSA id 64sm6594089wrk.46.2017.10.23.10.29.14 for (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Mon, 23 Oct 2017 10:29:14 -0700 (PDT) From: Richard Sandiford To: gcc-patches@gcc.gnu.org Mail-Followup-To: gcc-patches@gcc.gnu.org, richard.sandiford@linaro.org Subject: [071/nnn] poly_int: vectorizable_induction References: <871sltvm7r.fsf@linaro.org> Date: Mon, 23 Oct 2017 18:29:13 +0100 In-Reply-To: <871sltvm7r.fsf@linaro.org> (Richard Sandiford's message of "Mon, 23 Oct 2017 17:54:32 +0100") Message-ID: <87d15dixhy.fsf@linaro.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/25.2 (gnu/linux) MIME-Version: 1.0 This patch makes vectorizable_induction cope with variable-length vectors. For now we punt on SLP inductions, but patchees after the main SVE submission add support for those too. 2017-10-23 Richard Sandiford Alan Hayward David Sherwood gcc/ * tree-vect-loop.c (vectorizable_induction): Treat the number of units as polynomial. Punt on SLP inductions. Use an integer VEC_SERIES_EXPR for variable-length integer reductions. Use a cast of such a series for variable-length floating-point reductions. Index: gcc/tree-vect-loop.c =================================================================== --- gcc/tree-vect-loop.c 2017-10-23 17:22:35.829905285 +0100 +++ gcc/tree-vect-loop.c 2017-10-23 17:22:36.904793787 +0100 @@ -6624,7 +6624,7 @@ vectorizable_induction (gimple *phi, return false; tree vectype = STMT_VINFO_VECTYPE (stmt_info); - unsigned nunits = TYPE_VECTOR_SUBPARTS (vectype); + poly_uint64 nunits = TYPE_VECTOR_SUBPARTS (vectype); if (slp_node) ncopies = 1; @@ -6689,6 +6689,16 @@ vectorizable_induction (gimple *phi, iv_loop = loop; gcc_assert (iv_loop == (gimple_bb (phi))->loop_father); + if (slp_node && !nunits.is_constant ()) + { + /* The current SLP code creates the initial value element-by-element. */ + if (dump_enabled_p ()) + dump_printf_loc (MSG_MISSED_OPTIMIZATION, vect_location, + "SLP induction not supported for variable-length" + " vectors.\n"); + return false; + } + if (!vec_stmt) /* transformation not required. */ { STMT_VINFO_TYPE (stmt_info) = induc_vec_info_type; @@ -6737,6 +6747,9 @@ vectorizable_induction (gimple *phi, [VF*S, VF*S, VF*S, VF*S] for all. */ if (slp_node) { + /* Enforced above. */ + unsigned int const_nunits = nunits.to_constant (); + /* Convert the init to the desired type. */ stmts = NULL; init_expr = gimple_convert (&stmts, TREE_TYPE (vectype), init_expr); @@ -6765,19 +6778,20 @@ vectorizable_induction (gimple *phi, /* Now generate the IVs. */ unsigned group_size = SLP_TREE_SCALAR_STMTS (slp_node).length (); unsigned nvects = SLP_TREE_NUMBER_OF_VEC_STMTS (slp_node); - unsigned elts = nunits * nvects; - unsigned nivs = least_common_multiple (group_size, nunits) / nunits; + unsigned elts = const_nunits * nvects; + unsigned nivs = least_common_multiple (group_size, + const_nunits) / const_nunits; gcc_assert (elts % group_size == 0); tree elt = init_expr; unsigned ivn; for (ivn = 0; ivn < nivs; ++ivn) { - auto_vec elts (nunits); + auto_vec elts (const_nunits); stmts = NULL; - for (unsigned eltn = 0; eltn < nunits; ++eltn) + for (unsigned eltn = 0; eltn < const_nunits; ++eltn) { - if (ivn*nunits + eltn >= group_size - && (ivn*nunits + eltn) % group_size == 0) + if (ivn*const_nunits + eltn >= group_size + && (ivn * const_nunits + eltn) % group_size == 0) elt = gimple_build (&stmts, PLUS_EXPR, TREE_TYPE (elt), elt, step_expr); elts.quick_push (elt); @@ -6814,7 +6828,7 @@ vectorizable_induction (gimple *phi, if (ivn < nvects) { unsigned vfp - = least_common_multiple (group_size, nunits) / group_size; + = least_common_multiple (group_size, const_nunits) / group_size; /* Generate [VF'*S, VF'*S, ... ]. */ if (SCALAR_FLOAT_TYPE_P (TREE_TYPE (step_expr))) { @@ -6889,18 +6903,45 @@ vectorizable_induction (gimple *phi, stmts = NULL; new_name = gimple_convert (&stmts, TREE_TYPE (vectype), init_expr); - auto_vec elts (nunits); - elts.quick_push (new_name); - for (i = 1; i < nunits; i++) - { - /* Create: new_name_i = new_name + step_expr */ - new_name = gimple_build (&stmts, PLUS_EXPR, TREE_TYPE (new_name), - new_name, step_expr); + unsigned HOST_WIDE_INT const_nunits; + if (nunits.is_constant (&const_nunits)) + { + auto_vec elts (const_nunits); elts.quick_push (new_name); + for (i = 1; i < const_nunits; i++) + { + /* Create: new_name_i = new_name + step_expr */ + new_name = gimple_build (&stmts, PLUS_EXPR, TREE_TYPE (new_name), + new_name, step_expr); + elts.quick_push (new_name); + } + /* Create a vector from [new_name_0, new_name_1, ..., + new_name_nunits-1] */ + vec_init = gimple_build_vector (&stmts, vectype, elts); + } + else if (INTEGRAL_TYPE_P (TREE_TYPE (step_expr))) + /* Build the initial value directly from a VEC_SERIES_EXPR. */ + vec_init = gimple_build (&stmts, VEC_SERIES_EXPR, vectype, + new_name, step_expr); + else + { + /* Build: + [base, base, base, ...] + + (vectype) [0, 1, 2, ...] * [step, step, step, ...]. */ + gcc_assert (SCALAR_FLOAT_TYPE_P (TREE_TYPE (step_expr))); + gcc_assert (flag_associative_math); + tree index = build_index_vector (vectype, 0, 1); + tree base_vec = gimple_build_vector_from_val (&stmts, vectype, + new_name); + tree step_vec = gimple_build_vector_from_val (&stmts, vectype, + step_expr); + vec_init = gimple_build (&stmts, FLOAT_EXPR, vectype, index); + vec_init = gimple_build (&stmts, MULT_EXPR, vectype, + vec_init, step_vec); + vec_init = gimple_build (&stmts, PLUS_EXPR, vectype, + vec_init, base_vec); } - /* Create a vector from [new_name_0, new_name_1, ..., - new_name_nunits-1] */ - vec_init = gimple_build_vector (&stmts, vectype, elts); + if (stmts) { new_bb = gsi_insert_seq_on_edge_immediate (pe, stmts);