From patchwork Thu Jul 28 13:35:59 2016 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Prathamesh Kulkarni X-Patchwork-Id: 72948 Delivered-To: patch@linaro.org Received: by 10.140.29.52 with SMTP id a49csp856429qga; Thu, 28 Jul 2016 06:36:32 -0700 (PDT) X-Received: by 10.66.218.70 with SMTP id pe6mr59918842pac.29.1469712992187; Thu, 28 Jul 2016 06:36:32 -0700 (PDT) Return-Path: Received: from sourceware.org (server1.sourceware.org. [209.132.180.131]) by mx.google.com with ESMTPS id m6si12564006pfj.88.2016.07.28.06.36.31 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 28 Jul 2016 06:36:32 -0700 (PDT) Received-SPF: pass (google.com: domain of gcc-patches-return-432695-patch=linaro.org@gcc.gnu.org designates 209.132.180.131 as permitted sender) client-ip=209.132.180.131; Authentication-Results: mx.google.com; dkim=pass header.i=@gcc.gnu.org; spf=pass (google.com: domain of gcc-patches-return-432695-patch=linaro.org@gcc.gnu.org designates 209.132.180.131 as permitted sender) smtp.mailfrom=gcc-patches-return-432695-patch=linaro.org@gcc.gnu.org; dmarc=fail (p=NONE dis=NONE) header.from=linaro.org DomainKey-Signature: a=rsa-sha1; c=nofws; d=gcc.gnu.org; h=list-id :list-unsubscribe:list-archive:list-post:list-help:sender :mime-version:in-reply-to:references:from:date:message-id :subject:to:cc:content-type; q=dns; s=default; b=sHyDlZtng7hVFyV mKkpviCWbiv7jA43kZ9dvnUEMBSnzGiqgCemoGPo3jTVcoUDunw4Vmrl+XjGigt0 3KMjYaXX6gqQrS+uyDBFzYSJ85h8vJnMNiIwV4Xj0uIZ2vMM6DcpUZquWljzHnDX deJ7ZkD83hruVJ69JVTLcpsyF8o8= DKIM-Signature: v=1; a=rsa-sha1; c=relaxed; d=gcc.gnu.org; h=list-id :list-unsubscribe:list-archive:list-post:list-help:sender :mime-version:in-reply-to:references:from:date:message-id :subject:to:cc:content-type; s=default; bh=GOTWLD3ibZqrrQInhML0U rgb7Tc=; b=Fb5ueJ5FNLVyats29yO+csdlp/Cr9pNnvltXWfMuicWO35n5URMK5 fKAHFVtWWUcyJboGv5FjA7EWj61/kS49oxG5elC7xryJKCFCOWqmeeUFnCZGkr1N 8v637cHAZZMf4j5cJGHOz6GyDONP0J/hdCS3PgaE/XLumOOLVap56Y= Received: (qmail 7108 invoked by alias); 28 Jul 2016 13:36:14 -0000 Mailing-List: contact gcc-patches-help@gcc.gnu.org; run by ezmlm Precedence: bulk List-Id: List-Unsubscribe: List-Archive: List-Post: List-Help: Sender: gcc-patches-owner@gcc.gnu.org Delivered-To: mailing list gcc-patches@gcc.gnu.org Received: (qmail 7073 invoked by uid 89); 28 Jul 2016 13:36:13 -0000 Authentication-Results: sourceware.org; auth=none X-Virus-Found: No X-Spam-SWARE-Status: No, score=-1.0 required=5.0 tests=AWL, BAYES_50, RCVD_IN_DNSWL_LOW, SPF_PASS, URIBL_RED autolearn=ham version=3.3.2 spammy=mitchell, Mitchell, sk:make_te, sk:gsi_ins X-HELO: mail-io0-f180.google.com Received: from mail-io0-f180.google.com (HELO mail-io0-f180.google.com) (209.85.223.180) by sourceware.org (qpsmtpd/0.93/v0.84-503-g423c35a) with (AES128-GCM-SHA256 encrypted) ESMTPS; Thu, 28 Jul 2016 13:36:02 +0000 Received: by mail-io0-f180.google.com with SMTP id b62so99465641iod.3 for ; Thu, 28 Jul 2016 06:36:02 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=u+sFYaDI1LuQamrh7q1Y0SAzezsCdqBE0RdeS8/fHRw=; b=PSkIPCVmPzzjfQfpTRENLxPu084AQQdpWIB3lX4Rj9u5iILqUmqwKsAQajzZ9YSNLO SfxxRqpP/hm8w6HdwV0EDwmv9sqs2SO7CssY8Ttv7c6UxtYt1zgDm16BCLj0YmxzDhwM RJSlXJAn3ls9OAjTyySqqgMMsLPSzcHEgxGc/frV5ahkBy7Hj8nxShR+jq9NZij4GV5r c3dk3RuLIF6fO8ElXd2jhPopiWjC7afJefXkCRi6EPbbeE/CPHBYgjF4uJeUNxx0aP9x d/YbFW0yHSyxJJpMPxxhL/X/Grr/t22dXDh9qxYoAS/rjm87LWCaDp+escNQVoHo9lBd Ac3Q== X-Gm-Message-State: AEkooutBsiAzD+bNDcnFmgeS9UubygaHYquVTno5VT7Mvi7is02T0IgjfXfuJHStjyK1Pebih9mnolDVm7tEYwl3 X-Received: by 10.107.175.27 with SMTP id y27mr43166981ioe.137.1469712960347; Thu, 28 Jul 2016 06:36:00 -0700 (PDT) MIME-Version: 1.0 Received: by 10.36.48.197 with HTTP; Thu, 28 Jul 2016 06:35:59 -0700 (PDT) In-Reply-To: References: From: Prathamesh Kulkarni Date: Thu, 28 Jul 2016 19:05:59 +0530 Message-ID: Subject: Re: RFC [1/2] divmod transform To: Richard Biener Cc: Jim Wilson , Richard Biener , gcc Patches , Ramana Radhakrishnan , Kugan Vivekanandarajah , "Joseph S. Myers" X-IsSubscribed: yes On 8 June 2016 at 19:53, Richard Biener wrote: > On Fri, 3 Jun 2016, Jim Wilson wrote: > >> On Mon, May 30, 2016 at 12:45 AM, Richard Biener wrote: >> > Joseph - do you know sth about why there's not a full set of divmod >> > libfuncs in libgcc? >> >> Because udivmoddi4 isn't a libfunc, it is a helper function for the >> div and mov libfuncs. Since we can compute the signed div and mod >> results from udivmoddi4, there was no need to also add a signed >> version of it. It was given a libfunc style name so that we had the >> option of making it a libfunc in the future, but that never happened. >> There was no support for calling any divmod libfunc until it was added >> as a special case to call an ARM library (not libgcc) function. This >> happened here >> >> 2004-08-09 Mark Mitchell >> >> * config.gcc (arm*-*-eabi*): New target. >> * defaults.h (TARGET_LIBGCC_FUNCS): New macro. >> (TARGET_LIB_INT_CMP_BIASED): Likewise. >> * expmed.c (expand_divmod): Try a two-valued divmod function as a >> last resort. >> ... >> * config/arm/arm.c (arm_init_libfuncs): New function. >> (arm_compute_initial_eliminatino_offset): Return HOST_WIDE_INT. >> (TARGET_INIT_LIBFUNCS): Define it. >> ... >> >> Later, two ports added their own divmod libfuncs, but I don't see any >> evidence that they were ever used, since there is no support for >> calling divmod other than the expand_divmod last resort code that only >> triggers for ARM. >> >> It is only now that Prathamesh is adding gimple support for divmod >> operations that we need to worry about getting this right, without >> breaking the existing ARM library support or the existing udivmoddi4 >> support. > > Ok, so as he is primarily targeting the special arm divmod libcall > I suppose we can live with special-casing libcall handling to > udivmoddi3. It would be nice to not lie about divmod availablilty > as libcall though... - it looks like the libcall is also guarded > on TARGET_HAS_NO_HW_DIVIDE (unless it was available historically > like on x86). > > So not sure where to go from here. Hi, I have attached patch, which is rebased on trunk. Needed to update divmod-7.c, which now gets transformed to divmod thanks to your code-hoisting patch -;) We still have the issue of optab_libfunc() returning non-existent libcalls. As in previous patch, I am checking explicitly for "__udivmoddi4", with a FIXME note. I hope that's okay for now ? Bootstrapped and tested on x86_64-unknown-linux-gnu, armv8l-unknown-linux-gnueabihf. Bootstrap+test in progress on i686-linux-gnu. Cross-tested on arm*-*-*. Thanks, Prathamesh > > Richard. diff --git a/gcc/doc/tm.texi b/gcc/doc/tm.texi index 83bd9ab..e4815cf 100644 --- a/gcc/doc/tm.texi +++ b/gcc/doc/tm.texi @@ -7010,6 +7010,12 @@ This is firstly introduced on ARM/AArch64 targets, please refer to the hook implementation for how different fusion types are supported. @end deftypefn +@deftypefn {Target Hook} void TARGET_EXPAND_DIVMOD_LIBFUNC (bool @var{unsignedp}, machine_mode @var{mode}, @var{rtx}, @var{rtx}, rtx *@var{quot}, rtx *@var{rem}) +Define this hook if the port does not have hardware div and divmod insn for +the given mode but has divmod libfunc, which is incompatible +with libgcc2.c:__udivmoddi4 +@end deftypefn + @node Sections @section Dividing the Output into Sections (Texts, Data, @dots{}) @c the above section title is WAY too long. maybe cut the part between diff --git a/gcc/doc/tm.texi.in b/gcc/doc/tm.texi.in index a72c3d8..3efaf4d 100644 --- a/gcc/doc/tm.texi.in +++ b/gcc/doc/tm.texi.in @@ -4864,6 +4864,8 @@ them: try the first ones in this list first. @hook TARGET_SCHED_FUSION_PRIORITY +@hook TARGET_EXPAND_DIVMOD_LIBFUNC + @node Sections @section Dividing the Output into Sections (Texts, Data, @dots{}) @c the above section title is WAY too long. maybe cut the part between diff --git a/gcc/internal-fn.c b/gcc/internal-fn.c index 49f3495..18876ce 100644 --- a/gcc/internal-fn.c +++ b/gcc/internal-fn.c @@ -2326,6 +2326,48 @@ multi_vector_optab_supported_p (convert_optab optab, tree_pair types, #define direct_mask_store_optab_supported_p direct_optab_supported_p #define direct_store_lanes_optab_supported_p multi_vector_optab_supported_p +/* Expand DIVMOD() using: + a) optab handler for udivmod/sdivmod if it is available. + b) If optab_handler doesn't exist, Generate call to + optab_libfunc for udivmod/sdivmod. */ + +static void +expand_DIVMOD (internal_fn, gcall *stmt) +{ + tree lhs = gimple_call_lhs (stmt); + tree arg0 = gimple_call_arg (stmt, 0); + tree arg1 = gimple_call_arg (stmt, 1); + + gcc_assert (TREE_CODE (TREE_TYPE (lhs)) == COMPLEX_TYPE); + tree type = TREE_TYPE (TREE_TYPE (lhs)); + machine_mode mode = TYPE_MODE (type); + bool unsignedp = TYPE_UNSIGNED (type); + optab tab = (unsignedp) ? udivmod_optab : sdivmod_optab; + + rtx op0 = expand_normal (arg0); + rtx op1 = expand_normal (arg1); + rtx target = expand_expr (lhs, NULL_RTX, VOIDmode, EXPAND_WRITE); + + rtx quotient, remainder; + + /* Check if optab handler exists for [u]divmod. */ + if (optab_handler (tab, mode) != CODE_FOR_nothing) + { + quotient = gen_reg_rtx (mode); + remainder = gen_reg_rtx (mode); + expand_twoval_binop (tab, op0, op1, quotient, remainder, unsignedp); + } + else + targetm.expand_divmod_libfunc (unsignedp, mode, op0, op1, + "ient, &remainder); + + /* Wrap the return value (quotient, remainder) within COMPLEX_EXPR. */ + expand_expr (build2 (COMPLEX_EXPR, TREE_TYPE (lhs), + make_tree (TREE_TYPE (arg0), quotient), + make_tree (TREE_TYPE (arg1), remainder)), + target, VOIDmode, EXPAND_NORMAL); +} + /* Return true if FN is supported for the types in TYPES when the optimization type is OPT_TYPE. The types are those associated with the "type0" and "type1" fields of FN's direct_internal_fn_info diff --git a/gcc/internal-fn.def b/gcc/internal-fn.def index 6701cd9..b221ef9 100644 --- a/gcc/internal-fn.def +++ b/gcc/internal-fn.def @@ -195,6 +195,9 @@ DEF_INTERNAL_FN (ATOMIC_BIT_TEST_AND_COMPLEMENT, ECF_LEAF | ECF_NOTHROW, NULL) DEF_INTERNAL_FN (ATOMIC_BIT_TEST_AND_RESET, ECF_LEAF | ECF_NOTHROW, NULL) DEF_INTERNAL_FN (ATOMIC_COMPARE_EXCHANGE, ECF_LEAF | ECF_NOTHROW, NULL) +/* Divmod function */ +DEF_INTERNAL_FN (DIVMOD, ECF_CONST | ECF_LEAF, NULL) + #undef DEF_INTERNAL_INT_FN #undef DEF_INTERNAL_FLT_FN #undef DEF_INTERNAL_OPTAB_FN diff --git a/gcc/target.def b/gcc/target.def index 27f9ac2..120a653 100644 --- a/gcc/target.def +++ b/gcc/target.def @@ -5001,6 +5001,16 @@ Normally, this is not needed.", bool, (const_tree field, machine_mode mode), default_member_type_forces_blk) +/* See tree-ssa-math-opts.c:divmod_candidate_p for conditions that gate + the divmod transform. */ +DEFHOOK +(expand_divmod_libfunc, + "Define this hook if the port does not have hardware div and divmod insn for\n\ +the given mode but has divmod libfunc, which is incompatible\n\ +with libgcc2.c:__udivmoddi4", + void, (bool unsignedp, machine_mode mode, rtx, rtx, rtx *quot, rtx *rem), + default_expand_divmod_libfunc) + /* Return the class for a secondary reload, and fill in extra information. */ DEFHOOK (secondary_reload, diff --git a/gcc/targhooks.c b/gcc/targhooks.c index 69037c1..f506a83 100644 --- a/gcc/targhooks.c +++ b/gcc/targhooks.c @@ -2008,4 +2008,33 @@ default_max_noce_ifcvt_seq_cost (edge e) return BRANCH_COST (true, predictable_p) * COSTS_N_INSNS (3); } +/* Generate call to + DImode __udivmoddi4 (DImode op0, DImode op1, DImode *rem). */ + +void +default_expand_divmod_libfunc (bool unsignedp, machine_mode mode, + rtx op0, rtx op1, + rtx *quot_p, rtx *rem_p) +{ + gcc_assert (mode == DImode); + gcc_assert (unsignedp); + + rtx libfunc = optab_libfunc (udivmod_optab, DImode); + gcc_assert (libfunc); + gcc_assert (!strcmp (XSTR (libfunc, 0), "__udivmoddi4")); + + rtx remainder = assign_stack_temp (DImode, GET_MODE_SIZE (DImode)); + rtx address = XEXP (remainder, 0); + + rtx quotient = emit_library_call_value (libfunc, NULL_RTX, LCT_CONST, + DImode, 3, + op0, GET_MODE (op0), + op1, GET_MODE (op1), + address, GET_MODE (address)); + + *quot_p = quotient; + *rem_p = remainder; +} + + #include "gt-targhooks.h" diff --git a/gcc/targhooks.h b/gcc/targhooks.h index 2e7ca72..0c0dbe2 100644 --- a/gcc/targhooks.h +++ b/gcc/targhooks.h @@ -258,4 +258,7 @@ extern bool default_optab_supported_p (int, machine_mode, machine_mode, extern unsigned int default_max_noce_ifcvt_seq_cost (edge); +extern void default_expand_divmod_libfunc (bool, machine_mode, rtx, rtx, + rtx *, rtx *); + #endif /* GCC_TARGHOOKS_H */ diff --git a/gcc/tree-ssa-math-opts.c b/gcc/tree-ssa-math-opts.c index b93bcf3..ad32744 100644 --- a/gcc/tree-ssa-math-opts.c +++ b/gcc/tree-ssa-math-opts.c @@ -112,6 +112,9 @@ along with GCC; see the file COPYING3. If not see #include "params.h" #include "internal-fn.h" #include "case-cfn-macros.h" +#include "optabs-libfuncs.h" +#include "tree-eh.h" +#include "targhooks.h" /* This structure represents one basic block that either computes a division, or is a common dominator for basic block that compute a @@ -184,6 +187,9 @@ static struct /* Number of fp fused multiply-add ops inserted. */ int fmas_inserted; + + /* Number of divmod calls inserted. */ + int divmod_calls_inserted; } widen_mul_stats; /* The instance of "struct occurrence" representing the highest @@ -3793,6 +3799,228 @@ match_uaddsub_overflow (gimple_stmt_iterator *gsi, gimple *stmt, return true; } +/* Return true if target has support for divmod. */ + +static bool +target_supports_divmod_p (optab divmod_optab, optab div_optab, machine_mode mode) +{ + /* If target supports hardware divmod insn, use it for divmod. */ + if (optab_handler (divmod_optab, mode) != CODE_FOR_nothing) + return true; + + /* Check if libfunc for divmod is available. */ + rtx libfunc = optab_libfunc (divmod_optab, mode); + if (libfunc != NULL_RTX) + { + /* If optab_handler exists for div_optab, perhaps in a wider mode, + we don't want to use the libfunc even if it exists for given mode. */ + for (machine_mode div_mode = mode; + div_mode != VOIDmode; + div_mode = GET_MODE_WIDER_MODE (div_mode)) + if (optab_handler (div_optab, div_mode) != CODE_FOR_nothing) + return false; + + /* FIXME: This is a hack to workaround an issue with optab_libfunc(). + optab_libfunc (sdivmod_optab, DImode) returns libfunc "__divmoddi4", + although __divmoddi4() does not exist in libgcc. For now, enable the + transform only if libfunc is guaranteed to be __udivmoddi4. */ + return (targetm.expand_divmod_libfunc != default_expand_divmod_libfunc + || !strcmp (XSTR (libfunc, 0), "__udivmoddi4")); + } + + return false; +} + +/* Check if stmt is candidate for divmod transform. */ + +static bool +divmod_candidate_p (gassign *stmt) +{ + tree type = TREE_TYPE (gimple_assign_lhs (stmt)); + enum machine_mode mode = TYPE_MODE (type); + optab divmod_optab, div_optab; + + if (TYPE_UNSIGNED (type)) + { + divmod_optab = udivmod_optab; + div_optab = udiv_optab; + } + else + { + divmod_optab = sdivmod_optab; + div_optab = sdiv_optab; + } + + tree op1 = gimple_assign_rhs1 (stmt); + tree op2 = gimple_assign_rhs2 (stmt); + + /* Disable the transform if either is a constant, since division-by-constant + may have specialized expansion. */ + if (CONSTANT_CLASS_P (op1) || CONSTANT_CLASS_P (op2)) + return false; + + /* Exclude the case where TYPE_OVERFLOW_TRAPS (type) as that should + expand using the [su]divv optabs. */ + if (TYPE_OVERFLOW_TRAPS (type)) + return false; + + if (!target_supports_divmod_p (divmod_optab, div_optab, mode)) + return false; + + return true; +} + +/* This function looks for: + t1 = a TRUNC_DIV_EXPR b; + t2 = a TRUNC_MOD_EXPR b; + and transforms it to the following sequence: + complex_tmp = DIVMOD (a, b); + t1 = REALPART_EXPR(a); + t2 = IMAGPART_EXPR(b); + For conditions enabling the transform see divmod_candidate_p(). + + The pass works in two phases: + 1) Walk through all immediate uses of stmt's operand and find a + TRUNC_DIV_EXPR with matching operands and if such a stmt is found add + it to stmts vector. + 2) Insert DIVMOD call before first div/mod stmt in top_bb (basic block that + dominates other div/mod stmts with same operands) and update entries in + stmts vector to use return value of DIMOVD (REALEXPR_PART for div, + IMAGPART_EXPR for mod). */ + +static bool +convert_to_divmod (gassign *stmt) +{ + if (!divmod_candidate_p (stmt)) + return false; + + tree op1 = gimple_assign_rhs1 (stmt); + tree op2 = gimple_assign_rhs2 (stmt); + + imm_use_iterator use_iter; + gimple *use_stmt; + auto_vec stmts; + + gimple *top_stmt = stmt; + basic_block top_bb = gimple_bb (stmt); + + /* Try to set top_stmt to "topmost" stmt + with code TRUNC_DIV_EXPR/TRUNC_MOD_EXPR having same operands as stmt. */ + + FOR_EACH_IMM_USE_STMT (use_stmt, use_iter, op1) + { + if (is_gimple_assign (use_stmt) + && (gimple_assign_rhs_code (use_stmt) == TRUNC_DIV_EXPR + || gimple_assign_rhs_code (use_stmt) == TRUNC_MOD_EXPR) + && operand_equal_p (op1, gimple_assign_rhs1 (use_stmt), 0) + && operand_equal_p (op2, gimple_assign_rhs2 (use_stmt), 0)) + { + if (stmt_can_throw_internal (use_stmt)) + continue; + + basic_block bb = gimple_bb (use_stmt); + + if (bb == top_bb) + { + if (gimple_uid (use_stmt) < gimple_uid (top_stmt)) + top_stmt = use_stmt; + } + else if (dominated_by_p (CDI_DOMINATORS, top_bb, bb)) + { + top_bb = bb; + top_stmt = use_stmt; + } + } + } + + if (top_stmt == stmt && stmt_can_throw_internal (top_stmt)) + return false; + + tree top_op1 = gimple_assign_rhs1 (top_stmt); + tree top_op2 = gimple_assign_rhs2 (top_stmt); + + stmts.safe_push (top_stmt); + bool div_seen = (gimple_assign_rhs_code (top_stmt) == TRUNC_DIV_EXPR); + + /* Ensure that gimple_bb (use_stmt) is dominated by top_bb. */ + + FOR_EACH_IMM_USE_STMT (use_stmt, use_iter, top_op1) + { + if (is_gimple_assign (use_stmt) + && (gimple_assign_rhs_code (use_stmt) == TRUNC_DIV_EXPR + || gimple_assign_rhs_code (use_stmt) == TRUNC_MOD_EXPR) + && operand_equal_p (top_op1, gimple_assign_rhs1 (use_stmt), 0) + && operand_equal_p (top_op2, gimple_assign_rhs2 (use_stmt), 0)) + { + if (use_stmt == top_stmt) + continue; + + if (stmt_can_throw_internal (use_stmt)) + continue; + + if (!dominated_by_p (CDI_DOMINATORS, gimple_bb (use_stmt), top_bb)) + { + end_imm_use_stmt_traverse (&use_iter); + return false; + } + + stmts.safe_push (use_stmt); + if (gimple_assign_rhs_code (use_stmt) == TRUNC_DIV_EXPR) + div_seen = true; + } + } + + if (!div_seen) + return false; + + /* Create libcall to internal fn DIVMOD: + divmod_tmp = DIVMOD (op1, op2). */ + + gcall *call_stmt = gimple_build_call_internal (IFN_DIVMOD, 2, op1, op2); + tree res = make_temp_ssa_name ( + build_complex_type (TREE_TYPE (op1)), + call_stmt, "divmod_tmp"); + gimple_call_set_lhs (call_stmt, res); + + /* Insert the call before top_stmt. */ + gimple_stmt_iterator top_stmt_gsi = gsi_for_stmt (top_stmt); + gsi_insert_before (&top_stmt_gsi, call_stmt, GSI_SAME_STMT); + + widen_mul_stats.divmod_calls_inserted++; + + /* Update all statements in stmts. + if stmt is lhs = op1 TRUNC_DIV_EXPR op2, change to lhs = REALPART_EXPR + if stmt is lhs = op1 TRUNC_MOD_EXPR op2, change to lhs = IMAGPART_EXPR. */ + + bool cfg_changed = false; + for (unsigned i = 0; stmts.iterate (i, &use_stmt); ++i) + { + tree new_rhs; + + switch (gimple_assign_rhs_code (use_stmt)) + { + case TRUNC_DIV_EXPR: + new_rhs = fold_build1 (REALPART_EXPR, TREE_TYPE (op1), res); + break; + + case TRUNC_MOD_EXPR: + new_rhs = fold_build1 (IMAGPART_EXPR, TREE_TYPE (op2), res); + break; + + default: + gcc_unreachable (); + } + + gimple_stmt_iterator gsi = gsi_for_stmt (use_stmt); + gimple_assign_set_rhs_from_tree (&gsi, new_rhs); + update_stmt (use_stmt); + + if (maybe_clean_or_replace_eh_stmt (use_stmt, use_stmt)) + cfg_changed = true; + } + + return cfg_changed; +} /* Find integer multiplications where the operands are extended from smaller types, and replace the MULT_EXPR with a WIDEN_MULT_EXPR @@ -3837,6 +4065,8 @@ pass_optimize_widening_mul::execute (function *fun) bool cfg_changed = false; memset (&widen_mul_stats, 0, sizeof (widen_mul_stats)); + calculate_dominance_info (CDI_DOMINATORS); + renumber_gimple_stmt_uids (); FOR_EACH_BB_FN (bb, fun) { @@ -3870,6 +4100,10 @@ pass_optimize_widening_mul::execute (function *fun) match_uaddsub_overflow (&gsi, stmt, code); break; + case TRUNC_MOD_EXPR: + cfg_changed = convert_to_divmod (as_a (stmt)); + break; + default:; } } @@ -3916,6 +4150,8 @@ pass_optimize_widening_mul::execute (function *fun) widen_mul_stats.maccs_inserted); statistics_counter_event (fun, "fused multiply-adds inserted", widen_mul_stats.fmas_inserted); + statistics_counter_event (fun, "divmod calls inserted", + widen_mul_stats.divmod_calls_inserted); return cfg_changed ? TODO_cleanup_cfg : 0; }