| Message ID | 20241113-refactor-blk-affinity-helpers-v4-0-dd3baa1e267f@kernel.org |
|---|---|
| Headers | show |
| Series | blk: refactor queue affinity helpers | expand |
On Thu, Nov 14, 2024 at 09:58:25AM +0800, Ming Lei wrote: > > +void blk_mq_hctx_map_queues(struct blk_mq_queue_map *qmap, > > Some drivers may not know hctx at all, maybe blk_mq_map_hw_queues()? I am not really attach to the name, I am fine with renaming it to blk_mq_map_hw_queues. > > + if (dev->driver->irq_get_affinity) > > + irq_get_affinity = dev->driver->irq_get_affinity; > > + else if (dev->bus->irq_get_affinity) > > + irq_get_affinity = dev->bus->irq_get_affinity; > > It is one generic API, I think both 'dev->driver' and > 'dev->bus' should be validated here. What do you have in mind here if we get two masks? What should the operation be: AND, OR? This brings up another topic I left out in this series. blk_mq_map_queues does almost the same thing except it starts with the mask returned by group_cpus_evenely. If we figure out how this could be combined in a sane way it's possible to cleanup even a bit more. A bunch of drivers do if (i != HCTX_TYPE_POLL && offset) blk_mq_hctx_map_queues(map, dev->dev, offset); else blk_mq_map_queues(map); IMO it would be nice just to have one blk_mq_map_queues() which handles this correctly for both cases.
On Thu, Nov 14, 2024 at 05:12:22PM +0800, Ming Lei wrote: > I feel driver should get higher priority, but in the probe() example, > call_driver_probe() actually tries bus->probe() first. > > But looks not an issue for this patchset since only hisi_sas_v2_driver(platform_driver) > defines ->irq_get_affinity(), but the platform_bus_type doesn't have > the callback. Oh, I was not aware of this ordering. And after digging this up here: https://lore.kernel.org/all/20060105142951.13.01@flint.arm.linux.org.uk/ I don't think we it's worthwhile to add the callback to device_driver just for hisi_sas_v2. So I am going to drop this part again. > > This brings up another topic I left out in this series. > > blk_mq_map_queues does almost the same thing except it starts with the > > mask returned by group_cpus_evenely. If we figure out how this could be > > combined in a sane way it's possible to cleanup even a bit more. A bunch > > of drivers do > > > > if (i != HCTX_TYPE_POLL && offset) > > blk_mq_hctx_map_queues(map, dev->dev, offset); > > else > > blk_mq_map_queues(map); > > > > IMO it would be nice just to have one blk_mq_map_queues() which handles > > this correctly for both cases. > > I guess it is doable, and the driver just setup the tag_set->map[], then call > one generic map_queues API to do everything? Yes, that is my idea. Just having one function which handles what blk_mq_map_queues and blk_mq_hctx_map_queues/blk_mq_map_hw_queues currently do.
On Thu, Nov 14, 2024 at 01:06:49PM +0100, Daniel Wagner wrote: > Oh, I was not aware of this ordering. And after digging this up here: > > https://lore.kernel.org/all/20060105142951.13.01@flint.arm.linux.org.uk/ > > I don't think we it's worthwhile to add the callback to device_driver > just for hisi_sas_v2. So I am going to drop this part again. Yes, I don't really see how querying driver specific information like this from code called by the driver make much sense.
On 14/11/2024 12:06, Daniel Wagner wrote: > On Thu, Nov 14, 2024 at 05:12:22PM +0800, Ming Lei wrote: >> I feel driver should get higher priority, but in the probe() example, >> call_driver_probe() actually tries bus->probe() first. >> >> But looks not an issue for this patchset since only hisi_sas_v2_driver(platform_driver) >> defines ->irq_get_affinity(), but the platform_bus_type doesn't have >> the callback. > Oh, I was not aware of this ordering. And after digging this up here: > > https://urldefense.com/v3/__https://lore.kernel.org/ > all/20060105142951.13.01@flint.arm.linux.org.uk/__;!!ACWV5N9M2RV99hQ! > IWDpRwKfvo0y1LzokS_0YDXzX7eZLeVUcaTOFCpn0yEcV2e5gknO2Q3igMwPhA1Mhq6IOKBOkiLxHYe-0sM$ > > I don't think we it's worthwhile to add the callback to device_driver > just for hisi_sas_v2. So I am going to drop this part again. I agree. I would not add anything new to core code just for that driver.
Baes on Johns' idea, I added another irq_get_affinity callback to struct device_driver and made blk_mq_hctx_map_queues to try that one first. With this I could also replace the open coded queue mapping in hisi_sas v2. Besides this change I fixed couple of typos and also addressed John feedback in hisi_sas v3. Original cover letter: These patches were part of 'honor isolcpus configuration' [1] series. To simplify the review process I decided to send this as separate series because I think it's a nice cleanup independent of the isolcpus feature. Signed-off-by: Daniel Wagner <wagi@kernel.org> --- Changes in v4: - added irq_get_affinity callback to struct device_driver - hisi_sas use dev pointer directly from hisi_hba. - blk_mq_hctx_map_queues: ty irq_get_affinity callback from device_driver first then from bus_type - collected tags - fixed typos - Link to v3: https://lore.kernel.org/r/20241112-refactor-blk-affinity-helpers-v3-0-573bfca0cbd8@kernel.org Changes in v3: - dropped the additinal argument in blk_mq_hctx_map_queues. leave open coded version in hisi_sas_v2. - splitted "blk-mp: introduce blk_mq_hctx_map_queues" patch into three patches. - dropped local variable in pci_device_irq_get_affinity - Link to v2: https://lore.kernel.org/r/20241111-refactor-blk-affinity-helpers-v2-0-f360ddad231a@kernel.org Changes in v2: - added new callback to struct bus_type and call directly the affinity helpers from there. - Link to v1: https://lore.kernel.org/r/20240913-refactor-blk-affinity-helpers-v1-0-8e058f77af12@suse.de Changes in v1: - renamed blk_mq_dev_map_queues to blk_mq_hctx_map_queues - squased 'virito: add APIs for retrieving vq affinity' into 'blk-mq: introduce blk_mq_hctx_map_queues' - moved hisi_sas changed into a new patch - hisi_sas use define instead of hard coded value - moved helpers into their matching subsystem, removed blk-mq-pci and blk-mq-virtio files - fix spelling/typos - fixed long lines in docu (yep new lines in brief descriptions are supported, tested ti) - based on the first part of [1] https://lore.kernel.org/all/20240806-isolcpus-io-queues-v3-0-da0eecfeaf8b@suse.de --- Daniel Wagner (10): driver core: bus: add irq_get_affinity callback to bus_type driver core: add irq_get_affinity callback device_driver PCI: hookup irq_get_affinity callback virtio: hookup irq_get_affinity callback blk-mq: introduce blk_mq_hctx_map_queues scsi: replace blk_mq_pci_map_queues with blk_mq_hctx_map_queues scsi: hisi_sas: use blk_mq_hctx_map_queues to map queues nvme: replace blk_mq_pci_map_queues with blk_mq_hctx_map_queues virtio: blk/scsi: replace blk_mq_virtio_map_queues with blk_mq_hctx_map_queues blk-mq: remove unused queue mapping helpers block/Makefile | 2 -- block/blk-mq-cpumap.c | 43 +++++++++++++++++++++++++++++ block/blk-mq-pci.c | 46 ------------------------------- block/blk-mq-virtio.c | 46 ------------------------------- drivers/block/virtio_blk.c | 4 +-- drivers/nvme/host/fc.c | 1 - drivers/nvme/host/pci.c | 3 +- drivers/pci/pci-driver.c | 14 ++++++++++ drivers/scsi/fnic/fnic_main.c | 3 +- drivers/scsi/hisi_sas/hisi_sas.h | 1 - drivers/scsi/hisi_sas/hisi_sas_v2_hw.c | 22 +++++++-------- drivers/scsi/hisi_sas/hisi_sas_v3_hw.c | 4 +-- drivers/scsi/megaraid/megaraid_sas_base.c | 3 +- drivers/scsi/mpi3mr/mpi3mr.h | 1 - drivers/scsi/mpi3mr/mpi3mr_os.c | 2 +- drivers/scsi/mpt3sas/mpt3sas_scsih.c | 3 +- drivers/scsi/pm8001/pm8001_init.c | 2 +- drivers/scsi/pm8001/pm8001_sas.h | 1 - drivers/scsi/qla2xxx/qla_nvme.c | 3 +- drivers/scsi/qla2xxx/qla_os.c | 4 +-- drivers/scsi/smartpqi/smartpqi_init.c | 7 ++--- drivers/scsi/virtio_scsi.c | 3 +- drivers/virtio/virtio.c | 19 +++++++++++++ include/linux/blk-mq-pci.h | 11 -------- include/linux/blk-mq-virtio.h | 11 -------- include/linux/blk-mq.h | 2 ++ include/linux/device/bus.h | 3 ++ include/linux/device/driver.h | 3 ++ 28 files changed, 112 insertions(+), 155 deletions(-) --- base-commit: c9af98a7e8af266bae73e9d662b8341da1ec5824 change-id: 20240912-refactor-blk-affinity-helpers-7089b95b4b10 Best regards,