| Message ID | 20230418111459.811553-4-vladimir.oltean@nxp.com |
|---|---|
| State | Accepted |
| Commit | 827145392a4aad635b93e5235b7d7fecc2fa31c7 |
| Headers | show |
| Series | ethtool mm API consolidation | expand |
On Tue, Apr 18, 2023 at 02:14:53PM +0300, Vladimir Oltean wrote: > This was left as TODO in commit 01e23b2b3bad ("net: enetc: add support > for preemptible traffic classes") since it's relatively complicated. > > Where this makes a difference is with a configuration as follows: > > ethtool --set-mm eno0 pmac-enabled on tx-enabled on verify-enabled on > > Preemptible packets should only be sent when the MAC Merge TX direction > becomes active (i.o.w. when the verification process succeeds, aka when > the link partner confirms it can process preemptible traffic). But the > tc qdisc with the preemptible traffic classes is offloaded completely > asynchronously w.r.t. the MM becoming active. > > The ENETC manual does suggest that this should be handled in the driver: > "On startup, software should wait for the verification process to > complete (MMCSR[VSTS]=011) before initiating traffic". > > Adding the necessary logic allows future selftests to uphold the claim > that an inactive or disabled MAC Merge layer should never send data > packets through the pMAC. > > This change moves enetc_set_ptcfpr() from enetc.c to enetc_ethtool.c, > where its only caller is now - enetc_mm_commit_preemptible_tcs(). > > Signed-off-by: Vladimir Oltean <vladimir.oltean@nxp.com> Reviewed-by: Simon Horman <simon.horman@corigine.com> > int enetc_qos_query_caps(struct net_device *ndev, void *type_data); > diff --git a/drivers/net/ethernet/freescale/enetc/enetc_ethtool.c b/drivers/net/ethernet/freescale/enetc/enetc_ethtool.c > index deb674752851..838a92131963 100644 > --- a/drivers/net/ethernet/freescale/enetc/enetc_ethtool.c > +++ b/drivers/net/ethernet/freescale/enetc/enetc_ethtool.c > @@ -991,6 +991,64 @@ static int enetc_get_mm(struct net_device *ndev, struct ethtool_mm_state *state) > return 0; > } > > +static int enetc_mm_wait_tx_active(struct enetc_hw *hw, int verify_time) > +{ > + int timeout = verify_time * USEC_PER_MSEC * ENETC_MM_VERIFY_RETRIES; > + u32 val; > + > + /* This will time out after the standard value of 3 verification > + * attempts. To not sleep forever, it relies on a non-zero verify_time, > + * guarantee which is provided by the ethtool nlattr policy. > + */ > + return read_poll_timeout(enetc_port_rd, val, > + ENETC_MMCSR_GET_VSTS(val) == 3, nit: 3 is doing a lot of work here. As a follow-up, perhaps it could become part of an enum? > + ENETC_MM_VERIFY_SLEEP_US, timeout, > + true, hw, ENETC_MMCSR); > +} ...
On Thu, Apr 20, 2023 at 04:42:52PM +0200, Simon Horman wrote: > > + /* This will time out after the standard value of 3 verification > > + * attempts. To not sleep forever, it relies on a non-zero verify_time, > > + * guarantee which is provided by the ethtool nlattr policy. > > + */ > > + return read_poll_timeout(enetc_port_rd, val, > > + ENETC_MMCSR_GET_VSTS(val) == 3, > > nit: 3 is doing a lot of work here. > As a follow-up, perhaps it could become part of an enum? IMHO it's easy to abuse enums, when numbers could do just fine. I think that in context (seeing the entire enetc_ethtool.c), this is not as bad as just this patch makes it to be, because the other occurrence of ENETC_MMCSR_GET_VSTS() is: switch (ENETC_MMCSR_GET_VSTS(val)) { case 0: state->verify_status = ETHTOOL_MM_VERIFY_STATUS_DISABLED; break; case 2: state->verify_status = ETHTOOL_MM_VERIFY_STATUS_VERIFYING; break; case 3: state->verify_status = ETHTOOL_MM_VERIFY_STATUS_SUCCEEDED; break; case 4: state->verify_status = ETHTOOL_MM_VERIFY_STATUS_FAILED; break; case 5: default: state->verify_status = ETHTOOL_MM_VERIFY_STATUS_UNKNOWN; break; } so it's immediately clear what the 3 represents (in vim I just press '*' to see the other occurrences of ENETC_MMCSR_GET_VSTS). I considered it, but I don't feel an urgent necessity to add an enum here. Doing that would essentially transform the code into: return read_poll_timeout(enetc_port_rd, val, ENETC_MMCSR_GET_VSTS(val) == ENETC_MM_VSTS_SUCCEEDED, switch (ENETC_MMCSR_GET_VSTS(val)) { case ENETC_MMCSR_VSTS_DISABLED: state->verify_status = ETHTOOL_MM_VERIFY_STATUS_DISABLED; break; case ENETC_MMCSR_VSTS_VERIFYING: state->verify_status = ETHTOOL_MM_VERIFY_STATUS_VERIFYING; break; case ENETC_MMCSR_VSTS_SUCCEEDED: state->verify_status = ETHTOOL_MM_VERIFY_STATUS_SUCCEEDED; break; case ENETC_MMCSR_VSTS_FAILED: state->verify_status = ETHTOOL_MM_VERIFY_STATUS_FAILED; break; case ENETC_MMCSR_VSTS_UNKNOWN: default: state->verify_status = ETHTOOL_MM_VERIFY_STATUS_UNKNOWN; break; } which to my eye is more bloated.
On Thu, Apr 20, 2023 at 07:34:53PM +0300, Vladimir Oltean wrote: > On Thu, Apr 20, 2023 at 04:42:52PM +0200, Simon Horman wrote: > > > + /* This will time out after the standard value of 3 verification > > > + * attempts. To not sleep forever, it relies on a non-zero verify_time, > > > + * guarantee which is provided by the ethtool nlattr policy. > > > + */ > > > + return read_poll_timeout(enetc_port_rd, val, > > > + ENETC_MMCSR_GET_VSTS(val) == 3, > > > > nit: 3 is doing a lot of work here. > > As a follow-up, perhaps it could become part of an enum? > > IMHO it's easy to abuse enums, when numbers could do just fine. I think > that in context (seeing the entire enetc_ethtool.c), this is not as bad > as just this patch makes it to be, because the other occurrence of > ENETC_MMCSR_GET_VSTS() is: > > switch (ENETC_MMCSR_GET_VSTS(val)) { > case 0: > state->verify_status = ETHTOOL_MM_VERIFY_STATUS_DISABLED; > break; > case 2: > state->verify_status = ETHTOOL_MM_VERIFY_STATUS_VERIFYING; > break; > case 3: > state->verify_status = ETHTOOL_MM_VERIFY_STATUS_SUCCEEDED; > break; > case 4: > state->verify_status = ETHTOOL_MM_VERIFY_STATUS_FAILED; > break; > case 5: > default: > state->verify_status = ETHTOOL_MM_VERIFY_STATUS_UNKNOWN; > break; > } > > so it's immediately clear what the 3 represents (in vim I just press '*' > to see the other occurrences of ENETC_MMCSR_GET_VSTS). Thanks. I did see the code above, and I do agree it is informational wrt the meaning of the values. > I considered it, but I don't feel an urgent necessity to add an enum here. > Doing that would essentially transform the code into: > > return read_poll_timeout(enetc_port_rd, val, > ENETC_MMCSR_GET_VSTS(val) == ENETC_MM_VSTS_SUCCEEDED, > > switch (ENETC_MMCSR_GET_VSTS(val)) { > case ENETC_MMCSR_VSTS_DISABLED: > state->verify_status = ETHTOOL_MM_VERIFY_STATUS_DISABLED; > break; > case ENETC_MMCSR_VSTS_VERIFYING: > state->verify_status = ETHTOOL_MM_VERIFY_STATUS_VERIFYING; > break; > case ENETC_MMCSR_VSTS_SUCCEEDED: > state->verify_status = ETHTOOL_MM_VERIFY_STATUS_SUCCEEDED; > break; > case ENETC_MMCSR_VSTS_FAILED: > state->verify_status = ETHTOOL_MM_VERIFY_STATUS_FAILED; > break; > case ENETC_MMCSR_VSTS_UNKNOWN: > default: > state->verify_status = ETHTOOL_MM_VERIFY_STATUS_UNKNOWN; > break; > } > > which to my eye is more bloated. I guess it's subjective. I certainly don't feel strongly about this. And I appreciate you taking the time to respond to my idea. I have no objections to leaving this patch as is (with '3').
diff --git a/drivers/net/ethernet/freescale/enetc/enetc.c b/drivers/net/ethernet/freescale/enetc/enetc.c index 41c194c1672d..3c4fa26f0f9b 100644 --- a/drivers/net/ethernet/freescale/enetc/enetc.c +++ b/drivers/net/ethernet/freescale/enetc/enetc.c @@ -25,23 +25,12 @@ void enetc_port_mac_wr(struct enetc_si *si, u32 reg, u32 val) } EXPORT_SYMBOL_GPL(enetc_port_mac_wr); -void enetc_set_ptcfpr(struct enetc_hw *hw, unsigned long preemptible_tcs) +static void enetc_change_preemptible_tcs(struct enetc_ndev_priv *priv, + u8 preemptible_tcs) { - u32 val; - int tc; - - for (tc = 0; tc < 8; tc++) { - val = enetc_port_rd(hw, ENETC_PTCFPR(tc)); - - if (preemptible_tcs & BIT(tc)) - val |= ENETC_PTCFPR_FPE; - else - val &= ~ENETC_PTCFPR_FPE; - - enetc_port_wr(hw, ENETC_PTCFPR(tc), val); - } + priv->preemptible_tcs = preemptible_tcs; + enetc_mm_commit_preemptible_tcs(priv); } -EXPORT_SYMBOL_GPL(enetc_set_ptcfpr); static int enetc_num_stack_tx_queues(struct enetc_ndev_priv *priv) { @@ -2659,7 +2648,7 @@ static void enetc_reset_tc_mqprio(struct net_device *ndev) enetc_debug_tx_ring_prios(priv); - enetc_set_ptcfpr(hw, 0); + enetc_change_preemptible_tcs(priv, 0); } int enetc_setup_tc_mqprio(struct net_device *ndev, void *type_data) @@ -2714,7 +2703,7 @@ int enetc_setup_tc_mqprio(struct net_device *ndev, void *type_data) enetc_debug_tx_ring_prios(priv); - enetc_set_ptcfpr(hw, mqprio->preemptible_tcs); + enetc_change_preemptible_tcs(priv, mqprio->preemptible_tcs); return 0; diff --git a/drivers/net/ethernet/freescale/enetc/enetc.h b/drivers/net/ethernet/freescale/enetc/enetc.h index 143078a9ef16..c97a8e3d7a7f 100644 --- a/drivers/net/ethernet/freescale/enetc/enetc.h +++ b/drivers/net/ethernet/freescale/enetc/enetc.h @@ -355,6 +355,9 @@ struct enetc_ndev_priv { u16 rx_bd_count, tx_bd_count; u16 msg_enable; + + u8 preemptible_tcs; + enum enetc_active_offloads active_offloads; u32 speed; /* store speed for compare update pspeed */ @@ -433,6 +436,7 @@ int enetc_xdp_xmit(struct net_device *ndev, int num_frames, /* ethtool */ void enetc_set_ethtool_ops(struct net_device *ndev); void enetc_mm_link_state_update(struct enetc_ndev_priv *priv, bool link); +void enetc_mm_commit_preemptible_tcs(struct enetc_ndev_priv *priv); /* control buffer descriptor ring (CBDR) */ int enetc_setup_cbdr(struct device *dev, struct enetc_hw *hw, int bd_count, @@ -486,7 +490,6 @@ static inline void enetc_cbd_free_data_mem(struct enetc_si *si, int size, void enetc_reset_ptcmsdur(struct enetc_hw *hw); void enetc_set_ptcmsdur(struct enetc_hw *hw, u32 *queue_max_sdu); -void enetc_set_ptcfpr(struct enetc_hw *hw, unsigned long preemptible_tcs); #ifdef CONFIG_FSL_ENETC_QOS int enetc_qos_query_caps(struct net_device *ndev, void *type_data); diff --git a/drivers/net/ethernet/freescale/enetc/enetc_ethtool.c b/drivers/net/ethernet/freescale/enetc/enetc_ethtool.c index deb674752851..838a92131963 100644 --- a/drivers/net/ethernet/freescale/enetc/enetc_ethtool.c +++ b/drivers/net/ethernet/freescale/enetc/enetc_ethtool.c @@ -991,6 +991,64 @@ static int enetc_get_mm(struct net_device *ndev, struct ethtool_mm_state *state) return 0; } +static int enetc_mm_wait_tx_active(struct enetc_hw *hw, int verify_time) +{ + int timeout = verify_time * USEC_PER_MSEC * ENETC_MM_VERIFY_RETRIES; + u32 val; + + /* This will time out after the standard value of 3 verification + * attempts. To not sleep forever, it relies on a non-zero verify_time, + * guarantee which is provided by the ethtool nlattr policy. + */ + return read_poll_timeout(enetc_port_rd, val, + ENETC_MMCSR_GET_VSTS(val) == 3, + ENETC_MM_VERIFY_SLEEP_US, timeout, + true, hw, ENETC_MMCSR); +} + +static void enetc_set_ptcfpr(struct enetc_hw *hw, u8 preemptible_tcs) +{ + u32 val; + int tc; + + for (tc = 0; tc < 8; tc++) { + val = enetc_port_rd(hw, ENETC_PTCFPR(tc)); + + if (preemptible_tcs & BIT(tc)) + val |= ENETC_PTCFPR_FPE; + else + val &= ~ENETC_PTCFPR_FPE; + + enetc_port_wr(hw, ENETC_PTCFPR(tc), val); + } +} + +/* ENETC does not have an IRQ to notify changes to the MAC Merge TX status + * (active/inactive), but the preemptible traffic classes should only be + * committed to hardware once TX is active. Resort to polling. + */ +void enetc_mm_commit_preemptible_tcs(struct enetc_ndev_priv *priv) +{ + struct enetc_hw *hw = &priv->si->hw; + u8 preemptible_tcs = 0; + u32 val; + int err; + + val = enetc_port_rd(hw, ENETC_MMCSR); + if (!(val & ENETC_MMCSR_ME)) + goto out; + + if (!(val & ENETC_MMCSR_VDIS)) { + err = enetc_mm_wait_tx_active(hw, ENETC_MMCSR_GET_VT(val)); + if (err) + goto out; + } + + preemptible_tcs = priv->preemptible_tcs; +out: + enetc_set_ptcfpr(hw, preemptible_tcs); +} + /* FIXME: Workaround for the link partner's verification failing if ENETC * priorly received too much express traffic. The documentation doesn't * suggest this is needed. @@ -1061,6 +1119,8 @@ static int enetc_set_mm(struct net_device *ndev, struct ethtool_mm_cfg *cfg, enetc_restart_emac_rx(priv->si); + enetc_mm_commit_preemptible_tcs(priv); + mutex_unlock(&priv->mm_lock); return 0; @@ -1094,6 +1154,8 @@ void enetc_mm_link_state_update(struct enetc_ndev_priv *priv, bool link) enetc_port_wr(hw, ENETC_MMCSR, val); + enetc_mm_commit_preemptible_tcs(priv); + mutex_unlock(&priv->mm_lock); } EXPORT_SYMBOL_GPL(enetc_mm_link_state_update); diff --git a/drivers/net/ethernet/freescale/enetc/enetc_hw.h b/drivers/net/ethernet/freescale/enetc/enetc_hw.h index 36bb2d6d5658..1619943fb263 100644 --- a/drivers/net/ethernet/freescale/enetc/enetc_hw.h +++ b/drivers/net/ethernet/freescale/enetc/enetc_hw.h @@ -3,6 +3,9 @@ #include <linux/bitops.h> +#define ENETC_MM_VERIFY_SLEEP_US USEC_PER_MSEC +#define ENETC_MM_VERIFY_RETRIES 3 + /* ENETC device IDs */ #define ENETC_DEV_ID_PF 0xe100 #define ENETC_DEV_ID_VF 0xef00
This was left as TODO in commit 01e23b2b3bad ("net: enetc: add support for preemptible traffic classes") since it's relatively complicated. Where this makes a difference is with a configuration as follows: ethtool --set-mm eno0 pmac-enabled on tx-enabled on verify-enabled on Preemptible packets should only be sent when the MAC Merge TX direction becomes active (i.o.w. when the verification process succeeds, aka when the link partner confirms it can process preemptible traffic). But the tc qdisc with the preemptible traffic classes is offloaded completely asynchronously w.r.t. the MM becoming active. The ENETC manual does suggest that this should be handled in the driver: "On startup, software should wait for the verification process to complete (MMCSR[VSTS]=011) before initiating traffic". Adding the necessary logic allows future selftests to uphold the claim that an inactive or disabled MAC Merge layer should never send data packets through the pMAC. This change moves enetc_set_ptcfpr() from enetc.c to enetc_ethtool.c, where its only caller is now - enetc_mm_commit_preemptible_tcs(). Signed-off-by: Vladimir Oltean <vladimir.oltean@nxp.com> --- v1->v2: patch is new drivers/net/ethernet/freescale/enetc/enetc.c | 23 ++----- drivers/net/ethernet/freescale/enetc/enetc.h | 5 +- .../ethernet/freescale/enetc/enetc_ethtool.c | 62 +++++++++++++++++++ .../net/ethernet/freescale/enetc/enetc_hw.h | 3 + 4 files changed, 75 insertions(+), 18 deletions(-)