[v2,0/1] Add add-maintainer.py script

When pushing patches to upstream, the `get_maintainer.pl` script is used to
determine whom to send the patches to. Instead of having to manually process
the output of the script, add a wrapper script to do that for you.

The add-maintainer.py script adds maintainers (and mailing lists) to a patch,
editing it in-place.

Thanks to Bjorn for being a sounding board to this idea and for his valuable
suggestions.

Please try out this script with `--verbosity debug` for verifying that it's
doing "the right thing". I've tested this with a patch series from various
subsystems to ensure variety of maintainers and lists output and found it to be
doing what it is supposed to do.

I referred to the following links during development of this script:
- https://stackoverflow.com/questions/4427542/how-to-do-sed-like-text-replace-with-python
- https://stackoverflow.com/questions/4146009/python-get-list-indexes-using-regular-expression
- https://stackoverflow.com/questions/10507230/insert-line-at-middle-of-file-with-python

v1 -> v2:
- Added set-union logic based on Pavan's comments [1] and Bjorn's early suggestion
- Expanded audience and added more mailing lists to get more review comments and feedback

[1] https://lore.kernel.org/lkml/63764b84-3ebd-4081-836f-4863af196228@quicinc.com/

Guru Das Srinagesh (1):
  scripts: Add add-maintainer.py

 scripts/add-maintainer.py | 113 ++++++++++++++++++++++++++++++++++++++
 1 file changed, 113 insertions(+)
 create mode 100755 scripts/add-maintainer.py

On Aug 03 2023 11:16, Neil Armstrong wrote:
> Hi,
> 
> On 03/08/2023 10:23, Guru Das Srinagesh wrote:
> >When pushing patches to upstream, the `get_maintainer.pl` script is used to
> >determine whom to send the patches to. Instead of having to manually process
> >the output of the script, add a wrapper script to do that for you.
> >
> >The add-maintainer.py script adds maintainers (and mailing lists) to a patch,
> >editing it in-place.
> 
> FYI the b4 prep command does this:
> https://github.com/mricon/b4/blob/e8045d1353165cc065b2f1b180bf1b0846af510e/b4/ez.py#L2055
> 
> Perhaps it could be useful to make sure the output is similar ?
> 
> So far I've been very satisfied by the output of b4 auto_to_cc.

Thank you - let me check this tool out.

Guru Das.

On Aug 03 2023 01:23, Guru Das Srinagesh wrote:
> When pushing patches to upstream, the `get_maintainer.pl` script is used to
> determine whom to send the patches to. Instead of having to manually process
> the output of the script, add a wrapper script to do that for you.
> 
> The add-maintainer.py script adds maintainers (and mailing lists) to a patch,
> editing it in-place.

Could I request reviews from the other maintainers as well, please? Just to see
if I should continue working on this script or if the `b4` tool obviates the
need for such a script.

Thank you.

Guru Das.

On 10/08/2023 20:55, Guru Das Srinagesh wrote:
> On Aug 03 2023 01:23, Guru Das Srinagesh wrote:
>> When pushing patches to upstream, the `get_maintainer.pl` script is used to
>> determine whom to send the patches to. Instead of having to manually process
>> the output of the script, add a wrapper script to do that for you.
>>
>> The add-maintainer.py script adds maintainers (and mailing lists) to a patch,
>> editing it in-place.
> 
> Could I request reviews from the other maintainers as well, please? Just to see
> if I should continue working on this script or if the `b4` tool obviates the
> need for such a script.

I send a bit of patches but I use very simple workflow. It is really
simple, so simple, that I was always surprised how people can make their
life difficult with some complicated process to send patches... and then
obviously skip some maintainers, because of that process.

I almost always feed git send-email with addresses from
scripts/get_maintainers.pl. This tool would not bring any benefits to my
simple workflow.

For newcomers, OTOH, I would either recommend simple workflow or just
use b4. Why? Because if you cannot use git-send-email, then it means
your email setup will make your life difficult and adding maintainers to
existing patch won't help you.

This tool depends on the command line and shell interface of
scripts/get_maintainers.pl which is another reason why it might not be a
good idea.

Best regards,
Krzysztof

Thanks for the comments, Krzysztof.

On Aug 15 2023 23:06, Krzysztof Kozlowski wrote:
> On 10/08/2023 20:55, Guru Das Srinagesh wrote:
> > On Aug 03 2023 01:23, Guru Das Srinagesh wrote:
> >> When pushing patches to upstream, the `get_maintainer.pl` script is used to
> >> determine whom to send the patches to. Instead of having to manually process
> >> the output of the script, add a wrapper script to do that for you.
> >>
> >> The add-maintainer.py script adds maintainers (and mailing lists) to a patch,
> >> editing it in-place.
> > 
> > Could I request reviews from the other maintainers as well, please? Just to see
> > if I should continue working on this script or if the `b4` tool obviates the
> > need for such a script.
> 
> I send a bit of patches but I use very simple workflow. It is really
> simple, so simple, that I was always surprised how people can make their
> life difficult with some complicated process to send patches... and then
> obviously skip some maintainers, because of that process.

Exactly - this script aims to solve precisely that problem. It fills the gap
between running `get_maintainers.pl` and having to manually edit its output to
add "To: " and "Cc: " and somehow incorporate it in the body of the patch(es).

With this script, the workflow would be as simple as:

  1. Generate patches using `git format-patch`
  2. Run `add-maintainer.py` on the above patches
  3. `git send-email` the patches.

That's it - no need to manually work with email addresses.
  
> I almost always feed git send-email with addresses from
> scripts/get_maintainers.pl. This tool would not bring any benefits to my
> simple workflow.

In the light of the 3-step workflow I've envisioned above, could you please
elaborate why not? If anything, it will only save a developer's time.

> For newcomers, OTOH, I would either recommend simple workflow or just
> use b4. Why? Because if you cannot use git-send-email, then it means
> your email setup will make your life difficult and adding maintainers to
> existing patch won't help you.

You've mentioned a "simple workflow" many times - could you please share more
details on the steps you follow in your workflow for sending patches?

> This tool depends on the command line and shell interface of
> scripts/get_maintainers.pl which is another reason why it might not be a
> good idea.

Could you please elaborate on why depending on the output of
`get_maintainer.pl` is a bad idea? It's what everyone uses, no?

Thank you.

Guru Das.

On 16/08/2023 19:15, Guru Das Srinagesh wrote:
> Thanks for the comments, Krzysztof.
> 
> On Aug 15 2023 23:06, Krzysztof Kozlowski wrote:
>> On 10/08/2023 20:55, Guru Das Srinagesh wrote:
>>> On Aug 03 2023 01:23, Guru Das Srinagesh wrote:
>>>> When pushing patches to upstream, the `get_maintainer.pl` script is used to
>>>> determine whom to send the patches to. Instead of having to manually process
>>>> the output of the script, add a wrapper script to do that for you.
>>>>
>>>> The add-maintainer.py script adds maintainers (and mailing lists) to a patch,
>>>> editing it in-place.
>>>
>>> Could I request reviews from the other maintainers as well, please? Just to see
>>> if I should continue working on this script or if the `b4` tool obviates the
>>> need for such a script.
>>
>> I send a bit of patches but I use very simple workflow. It is really
>> simple, so simple, that I was always surprised how people can make their
>> life difficult with some complicated process to send patches... and then
>> obviously skip some maintainers, because of that process.
> 
> Exactly - this script aims to solve precisely that problem. It fills the gap
> between running `get_maintainers.pl` and having to manually edit its output to
> add "To: " and "Cc: " and somehow incorporate it in the body of the patch(es).
> 
> With this script, the workflow would be as simple as:
> 
>    1. Generate patches using `git format-patch`
>    2. Run `add-maintainer.py` on the above patches
>    3. `git send-email` the patches.
> 
> That's it - no need to manually work with email addresses.
>    
>> I almost always feed git send-email with addresses from
>> scripts/get_maintainers.pl. This tool would not bring any benefits to my
>> simple workflow.
> 
> In the light of the 3-step workflow I've envisioned above, could you please
> elaborate why not? If anything, it will only save a developer's time.
> 
>> For newcomers, OTOH, I would either recommend simple workflow or just
>> use b4. Why? Because if you cannot use git-send-email, then it means
>> your email setup will make your life difficult and adding maintainers to
>> existing patch won't help you.
> 
> You've mentioned a "simple workflow" many times - could you please share more
> details on the steps you follow in your workflow for sending patches?
> 
>> This tool depends on the command line and shell interface of
>> scripts/get_maintainers.pl which is another reason why it might not be a
>> good idea.
> 
> Could you please elaborate on why depending on the output of
> `get_maintainer.pl` is a bad idea? It's what everyone uses, no?

My opinion is that it would be a better idea to add a new output mode
to scripts/get_maintainer.pl than adding another script on top of it.

Or document somewhere how to use get_maintainer.pl with git-format-patch
without any additional scripts.

Neil

> 
> Thank you.
> 
> Guru Das.

On 16/08/2023 19:15, Guru Das Srinagesh wrote:
> Thanks for the comments, Krzysztof.
> 
> On Aug 15 2023 23:06, Krzysztof Kozlowski wrote:
>> On 10/08/2023 20:55, Guru Das Srinagesh wrote:
>>> On Aug 03 2023 01:23, Guru Das Srinagesh wrote:
>>>> When pushing patches to upstream, the `get_maintainer.pl` script is used to
>>>> determine whom to send the patches to. Instead of having to manually process
>>>> the output of the script, add a wrapper script to do that for you.
>>>>
>>>> The add-maintainer.py script adds maintainers (and mailing lists) to a patch,
>>>> editing it in-place.
>>>
>>> Could I request reviews from the other maintainers as well, please? Just to see
>>> if I should continue working on this script or if the `b4` tool obviates the
>>> need for such a script.
>>
>> I send a bit of patches but I use very simple workflow. It is really
>> simple, so simple, that I was always surprised how people can make their
>> life difficult with some complicated process to send patches... and then
>> obviously skip some maintainers, because of that process.
> 
> Exactly - this script aims to solve precisely that problem. It fills the gap
> between running `get_maintainers.pl` and having to manually edit its output to
> add "To: " and "Cc: " and somehow incorporate it in the body of the patch(es).

Why would anyone need to manually update it? Just some simple bash
function or git send-email identity.

> 
> With this script, the workflow would be as simple as:
> 
>   1. Generate patches using `git format-patch`
>   2. Run `add-maintainer.py` on the above patches
>   3. `git send-email` the patches.

So one more unnecessary step (2). I don't think it is easier than my
workflow.

I just do only 1 and 3 and that's it. The simplest way ever.

> 
> That's it - no need to manually work with email addresses.

No one suggested it...

>   
>> I almost always feed git send-email with addresses from
>> scripts/get_maintainers.pl. This tool would not bring any benefits to my
>> simple workflow.
> 
> In the light of the 3-step workflow I've envisioned above, could you please
> elaborate why not? If anything, it will only save a developer's time.

Because of unnecessary step 2? One more tool to remember to run?

> 
>> For newcomers, OTOH, I would either recommend simple workflow or just
>> use b4. Why? Because if you cannot use git-send-email, then it means
>> your email setup will make your life difficult and adding maintainers to
>> existing patch won't help you.
> 
> You've mentioned a "simple workflow" many times - could you please share more
> details on the steps you follow in your workflow for sending patches?

I shared it on LKML few times already (and Rob's git send-email identity
is also on LKML), so one more time:

https://github.com/krzk/tools/blob/master/linux/.bash_aliases_linux#L91


> 
>> This tool depends on the command line and shell interface of
>> scripts/get_maintainers.pl which is another reason why it might not be a
>> good idea.
> 
> Could you please elaborate on why depending on the output of
> `get_maintainer.pl` is a bad idea? It's what everyone uses, no?

No, because if interface changes you need to update two tools.

Best regards,
Krzysztof

On Fri, Aug 18, 2023 at 10:43:31AM +0200, Krzysztof Kozlowski wrote:
> On 16/08/2023 19:15, Guru Das Srinagesh wrote:
> > Thanks for the comments, Krzysztof.
> > 
> > On Aug 15 2023 23:06, Krzysztof Kozlowski wrote:
> >> On 10/08/2023 20:55, Guru Das Srinagesh wrote:
> >>> On Aug 03 2023 01:23, Guru Das Srinagesh wrote:
> >>>> When pushing patches to upstream, the `get_maintainer.pl` script is used to
> >>>> determine whom to send the patches to. Instead of having to manually process
> >>>> the output of the script, add a wrapper script to do that for you.
> >>>>
> >>>> The add-maintainer.py script adds maintainers (and mailing lists) to a patch,
> >>>> editing it in-place.
> >>>
> >>> Could I request reviews from the other maintainers as well, please? Just to see
> >>> if I should continue working on this script or if the `b4` tool obviates the
> >>> need for such a script.
> >>
> >> I send a bit of patches but I use very simple workflow. It is really
> >> simple, so simple, that I was always surprised how people can make their
> >> life difficult with some complicated process to send patches... and then
> >> obviously skip some maintainers, because of that process.
> > 
> > Exactly - this script aims to solve precisely that problem. It fills the gap
> > between running `get_maintainers.pl` and having to manually edit its output to
> > add "To: " and "Cc: " and somehow incorporate it in the body of the patch(es).
> 
> Why would anyone need to manually update it? Just some simple bash
> function or git send-email identity.
> 

I do this all the time, either to add additional, or remove unnecessary,
recipients from what's provided by get_maintainers.pl.

> > 
> > With this script, the workflow would be as simple as:
> > 
> >   1. Generate patches using `git format-patch`
> >   2. Run `add-maintainer.py` on the above patches
> >   3. `git send-email` the patches.
> 
> So one more unnecessary step (2). I don't think it is easier than my
> workflow.
> 
> I just do only 1 and 3 and that's it. The simplest way ever.
> 

There's no get_maintainer.pl in either 1, or 3, so obviously this isn't
the only thing you do.

Thanks for the link to your alias below, it's now clear that you don't
need an extra step in the procedure, if you only have your extra wrapper
around step 3.


I now also understand why you never ever have a cover-letter, something
Guru's proposed flow handles quite nicely.


That said, b4 prep and b4 send seems like a better suggestion to those
who doesn't already have a workflow in place.

[..]
> > 
> >> This tool depends on the command line and shell interface of
> >> scripts/get_maintainers.pl which is another reason why it might not be a
> >> good idea.
> > 
> > Could you please elaborate on why depending on the output of
> > `get_maintainer.pl` is a bad idea? It's what everyone uses, no?
> 
> No, because if interface changes you need to update two tools.
> 

This is a valid objection, but I've heard that "the simplest way ever"
also depends on exactly this output...

Regards,
Bjorn

On Aug 18 2023 10:43, Krzysztof Kozlowski wrote:
> >> For newcomers, OTOH, I would either recommend simple workflow or just
> >> use b4. Why? Because if you cannot use git-send-email, then it means
> >> your email setup will make your life difficult and adding maintainers to
> >> existing patch won't help you.
> > 
> > You've mentioned a "simple workflow" many times - could you please share more
> > details on the steps you follow in your workflow for sending patches?
> 
> I shared it on LKML few times already (and Rob's git send-email identity
> is also on LKML), so one more time:
> 
> https://github.com/krzk/tools/blob/master/linux/.bash_aliases_linux#L91

Thank you for sharing this - it is really neat indeed and you certainly don't
need a step #2 with this method.

However, I see areas for improvement in the alias:
- Subsystem-specific mailing lists, maintainers, reviewers, and other roles are
  all marked as "To: " sans distinction via the alias whereas
  `add-maintainer.py` and `b4` both provide marking of lists as "Cc: " which
  seems aesthetically and semantically more pleasing.
- Only `add-maintainer.py` allows for maintainers to be selectively in "To: "
  and "Cc: " for patches in a series depending on whether they are the
  maintainers for that particular patch or not.

> >> This tool depends on the command line and shell interface of
> >> scripts/get_maintainers.pl which is another reason why it might not be a
> >> good idea.
> > 
> > Could you please elaborate on why depending on the output of
> > `get_maintainer.pl` is a bad idea? It's what everyone uses, no?
> 
> No, because if interface changes you need to update two tools.

But `b4 prep --auto-to-cc` also uses `get_maintainer.pl`!

Also, in your previous email you said to "just use b4", which also depends on
the command line and shell interface of `get_maintainers.pl`. Depending on
`get_maintainer.pl` is therefore perfectly okay - there is no need to reinvent
it or disregard it for whatever reasons.

Thank you.

Guru Das.

On Aug 18 2023 10:33, Neil Armstrong wrote:
> My opinion is that it would be a better idea to add a new output mode
> to scripts/get_maintainer.pl than adding another script on top of it.

Sorry, I don't follow. The problem that this script is solving is getting the
output of `get_maintainer.pl` neatly into a patch according to this scheme:

  1. Generate patches using `git format-patch`
  2. Run `add-maintainer.py` on the above patches
  3. `git send-email` the patches.

Not sure how adding a new output mode to `get_maintainer.pl` fits in this
problem space.

Unless you mean to add a switch to it so that it automatically
edits the patch in-place (like `add-maintainer.py` does) and adds all the
addresses directly to the patch - in which case, that would be feature creep.

> Or document somewhere how to use get_maintainer.pl with git-format-patch
> without any additional scripts.

IMHO, `Documentation/process/submitting-patches.rst` should be updated with at
least one or two options addressing how to get from the aforementioned
step #1 -> step #3 addressing the problem that is being solved by step #2. In
this vacuum, every developer and maintainer appears to have their own solution
that works for them.

Thank you.

Guru Das.

On Aug 10 2023 11:49, Guru Das Srinagesh wrote:
> On Aug 03 2023 11:16, Neil Armstrong wrote:
> > Hi,
> > 
> > On 03/08/2023 10:23, Guru Das Srinagesh wrote:
> > >When pushing patches to upstream, the `get_maintainer.pl` script is used to
> > >determine whom to send the patches to. Instead of having to manually process
> > >the output of the script, add a wrapper script to do that for you.
> > >
> > >The add-maintainer.py script adds maintainers (and mailing lists) to a patch,
> > >editing it in-place.
> > 
> > FYI the b4 prep command does this:
> > https://github.com/mricon/b4/blob/e8045d1353165cc065b2f1b180bf1b0846af510e/b4/ez.py#L2055
> > 
> > Perhaps it could be useful to make sure the output is similar ?
> > 
> > So far I've been very satisfied by the output of b4 auto_to_cc.
> 
> Thank you - let me check this tool out.

I gave `b4` a spin and specifically compared the results of `b4 prep
--auto-to-cc` vs this script. I notice from the code that b4 calls
`get_maintainer.pl` with the following flags:

    --nogit --nogit-fallback --nogit-chief-penguins

I can add these to this script too.

And it collects maintainers and lists by passing, respectively, `--nol` and
`--nom` in two separate calls whereas this script parses the actual roles by
making only one call. b4's approach seems cleaner.

The perl script also can provide just the reviewers by passing `--nol --nom`.

b4 and this script both do:

    - Create set-union of all maintainers and all lists from all patches in a
      series.
    - Apply the same addresses to all patches in a series.

The main thing b4 doesn't do (which this script does) is:

    - add only that specific patch's maintainers as "To: ", and
    - the other maintainers from the other patches as "Cc: " 

Overall, b4 seems like a fantastic feature-rich tool for managing and sending
patches.

Thank you for sharing the link to the actual code - it was very instructive.

Guru Das.

On 18/08/2023 21:46, Bjorn Andersson wrote:
>>>
>>> With this script, the workflow would be as simple as:
>>>
>>>   1. Generate patches using `git format-patch`
>>>   2. Run `add-maintainer.py` on the above patches
>>>   3. `git send-email` the patches.
>>
>> So one more unnecessary step (2). I don't think it is easier than my
>> workflow.
>>
>> I just do only 1 and 3 and that's it. The simplest way ever.
>>
> 
> There's no get_maintainer.pl in either 1, or 3, so obviously this isn't
> the only thing you do.
> 
> Thanks for the link to your alias below, it's now clear that you don't
> need an extra step in the procedure, if you only have your extra wrapper
> around step 3.
> 
> 
> I now also understand why you never ever have a cover-letter, something
> Guru's proposed flow handles quite nicely.

It's not related. I usually don't create cover letter from laziness, but
pretty often I create them as well and my script/alias works there
perfectly. Cover letter is just one more step:
1. git branch --edit-description
2. git format-patch --cover-letter (with format.coverFromDescription =
subject in gitconfig)
3. git_send_email 0*

No need to run any other tool, no need to add any maintainer entries
(unless touching defconfig and specific soc@ stuff, but this is always
the case regardless of tools).

Really, that script proposed here is the unnecessary step.

Rob's approach with git send-email identity required some work for
cover-letter, but it was also running get_maintainer.pl per each patch,
so recipients did not receive everything. Unless patchset is big, I
prefer to send everything to everyone.

> 
> 
> That said, b4 prep and b4 send seems like a better suggestion to those
> who doesn't already have a workflow in place.

Yes.


Best regards,
Krzysztof

On 19/08/2023 03:33, Guru Das Srinagesh wrote:
> On Aug 18 2023 10:43, Krzysztof Kozlowski wrote:
>>>> For newcomers, OTOH, I would either recommend simple workflow or just
>>>> use b4. Why? Because if you cannot use git-send-email, then it means
>>>> your email setup will make your life difficult and adding maintainers to
>>>> existing patch won't help you.
>>>
>>> You've mentioned a "simple workflow" many times - could you please share more
>>> details on the steps you follow in your workflow for sending patches?
>>
>> I shared it on LKML few times already (and Rob's git send-email identity
>> is also on LKML), so one more time:
>>
>> https://github.com/krzk/tools/blob/master/linux/.bash_aliases_linux#L91
> 
> Thank you for sharing this - it is really neat indeed and you certainly don't
> need a step #2 with this method.
> 
> However, I see areas for improvement in the alias:
> - Subsystem-specific mailing lists, maintainers, reviewers, and other roles are
>   all marked as "To: " sans distinction via the alias whereas
>   `add-maintainer.py` and `b4` both provide marking of lists as "Cc: " which
>   seems aesthetically and semantically more pleasing.

To or Cc does not matter.

> - Only `add-maintainer.py` allows for maintainers to be selectively in "To: "
>   and "Cc: " for patches in a series depending on whether they are the
>   maintainers for that particular patch or not.

It's intentional to CC everyone. If I wanted to Cc/To
maintainer-per-patch, I would use Rob's send-email identity.

> 
>>>> This tool depends on the command line and shell interface of
>>>> scripts/get_maintainers.pl which is another reason why it might not be a
>>>> good idea.
>>>
>>> Could you please elaborate on why depending on the output of
>>> `get_maintainer.pl` is a bad idea? It's what everyone uses, no?
>>
>> No, because if interface changes you need to update two tools.
> 
> But `b4 prep --auto-to-cc` also uses `get_maintainer.pl`!

Yep, and it's Konstantin's headache to keep it updated. :)

> 
> Also, in your previous email you said to "just use b4", which also depends on
> the command line and shell interface of `get_maintainers.pl`. Depending on
> `get_maintainer.pl` is therefore perfectly okay - there is no need to reinvent
> it or disregard it for whatever reasons.

True, it is okay, but adding more tools to depend on it is more work. b4
is awesome tool, thus I feel it is justified to depend on that
interface. I don't see the need for more tools doing exactly the same.

Best regards,
Krzysztof