| Message ID | 20220120001621.705352-3-jsd@semihalf.com |
|---|---|
| State | New |
| Headers | show |
| Series | i2c-designware: Add support for AMD PSP semaphore | expand |
Hi Mario, Tom, +Nimesh (PSP) pt., 21 sty 2022 o 20:18 Tom Lendacky <thomas.lendacky@amd.com> napisał(a): > > On 1/21/22 11:38 AM, Limonciello, Mario wrote: > > [Public] > > > > +Thomas (ccp driver) > > +Alex (amdgpu driver) > > > >> > >> On 1/21/22 10:59, Jan Dąbroś wrote: > > >>>> > >>>> Through here seems to all be generic code for accessing > >>>> the AMD PSP. To me this seems like something which belongs > >>>> in a separate AMD-PSP-mbox driver/lib, which can then be > >>>> shared between other kernel drivers which may also want > >>>> to access PSP. > >>> > >>> I see your point clearly and actually it is not an accident that I've > >>> put all PSP-mailbox methods in one "block". They are logically > >>> different than the rest of i2c-adapter specific methods. > >>> > >>> That being said, above PSP mailbox was created by AMD solely for the > >>> purpose of i2c_arbitration. It has its own set of commands and > >>> specific format of the command-response buffer. Thus it is not and it > >>> won't be generic in the future. There are already upstreamed drivers > >>> from AMD (under drivers/crypto/ccp/) which made use of PSP, however > >>> their channel of communication looks completely different than the > >>> very simple i2c_arbitration model implemented above. > >>> > > > > Can you please double confirm no other users for this mailbox and it's for > > you only? And if so is it only specific to this platform that no other users? > > At least on some UEFI AMD platforms that mailbox is defined for > > communication with something else. We might need some way to attest > > from the firmware that it's safe to use. > > > > The only mailbox that is defined for OS use across different silicon AFAIK is > > the GPU driver mailbox. It may be safer to use that, but I'm not sure if > > GPU driver has come up early enough for your use. > > The CCP/PSP driver will load as a PCIe device driver on this platform and > will ioremap the MMIO space. Today, that driver doesn't reference those > mailbox registers, and I don't know that there will be a need in the > future. But if there is a need in the future, you could end up with a > conflict between these two drivers. Right, so I have confirmed this with AMD PSP firmware developers, that this particular x86-PSP mailbox is created and will be reserved _solely_ for the purpose of i2c arbitration (and thus this driver). There is no intend to use it elsewhere or share with another users. > Thanks, > Tom > > > > >>> Because of this I'm treating this as an i2c_semaphore-related code and > >>> putting this in this module. In my opinion moving this into some > >>> separate driver (which will be actually used only here) makes code > >>> less clear. But let's also hear some voice from AMD. > >> > >> Since as you say this mailbox is special and only for i2c-arbitration, > >> keeping it inside this patch / .c file is fine. > >> > >>> > >>>> > >>>> Sorta like the generic iosf_mbi_read() and > >>>> iosf_mbi_write() functions from: > >>>> > >>>> arch/x86/platform/intel/iosf_mbi.c > >>>> > >>>> used on the Intel chips, which are also used outside of > >>>> the I2C bus-locking code. > >>>> > >>>> This is also one of the reasons why I think it would be > >>>> good to get some AMD folks involved in this, since they > >>>> may be aware of other drivers which also need to access > >>>> the PSP mbox. > >>>> > >>> > >>> Right, I'm adding mario.limonciello@amd.com to the CC, so that he can > >> comment. > >>> > >>> (...) > >>> > >>>>> +/* > >>>>> + * Locking methods are based on the default implementation from > >>>>> + * drivers/i2c/i2c-core.base.c, but with psp acquire and release operations > >>>>> + * added. With this in place we can ensure that i2c clients on the bus shared > >>>>> + * with psp are able to lock HW access to the bus for arbitrary number of > >>>>> + * operations - that is e.g. write-wait-read. > >>>>> + */ > >>>>> +static void i2c_adapter_dw_psp_lock_bus(struct i2c_adapter *adapter, > >>>>> + unsigned int flags) > >>>>> +{ > >>>>> + psp_acquire_i2c_bus(); > >>>>> + rt_mutex_lock_nested(&adapter->bus_lock, > >> i2c_adapter_depth(adapter)); > >>>> > >>>> This does not do what you think it does and you will still deadlock > >>>> when things nest because of someone taking the bus-lock and then > >>>> the main i2c-designware transfer function calling the acquire_lock > >>>> callback. > >>> > >>> I haven't used rt_mutex_lock_nested() with the intent to prevent me > >>> from deadlock when i2c-designware calls acquire_lock with bus-lock > >>> already taken. This is a method copied from > >>> drivers/i2c/i2c-core-base.c (BTW, I have a typo in above comment). > >>> This is the default implementation applied by i2c-core when particular > >>> adapter doesn't register its own locking callbacks - thus it is called > >>> for i2c-designware for all platforms. > >>> > >>> In case of this driver internal i2c-designware acquire_lock() is equal > >>> to psp_acquire_i2c_bus(). In other words, bus-level lock > >>> i2c_adapter_dw_psp_lock_bus() is a superset of internal adapter's > >>> acquire_lock(). > >> > >> Ah I missed that this is just mimicking the core functions + > >> an extra call to psp_acquire_i2c_bus(). > >> > >> I assumed that the dwc->acquire callback path was also taking > >> the mutex and I thought you had fallen for the _nested meaning > >> something different then it does, my bad. > >> > >>> In order to prevent deadlock which you are talking about, I'm using > >>> reference lock counter inside psp_acquire_i2c_bus() thus it is safe to > >>> invoke acquire_lock() when bus-lock is already taken. > >> > >> Ah good, that is pretty much is the same as what the Bay Trail code > >> is doing. > >> > >>> > >>>> > >>>> The _nested postfix is only for the lockdep lock-debugger, this > >>>> actually turns into a regular mutex_lock when lockdep is not enabled: > >>>> > >>>> #ifdef CONFIG_DEBUG_LOCK_ALLOC > >>>> extern void rt_mutex_lock_nested(struct rt_mutex *lock, unsigned int > >> subclass); > >>>> #define rt_mutex_lock(lock) rt_mutex_lock_nested(lock, 0) > >>>> #else > >>>> extern void rt_mutex_lock(struct rt_mutex *lock); > >>>> #define rt_mutex_lock_nested(lock, subclass) rt_mutex_lock(lock) > >>>> #endif > >>>> > >>>> The _nested postfix as such is only to tell the lockdep code that > >>>> even though it seems we are trying to take the same mutex twice > >>>> since in both cases it is of i2c_adapter.rt_mutex "lock class" > >>>> that we are sure it is never the same i2c_adapter (but rather > >>>> one which always gets called in a nested fashion from another > >>>> i2c_adapter). > >>>> > >>>> IOW this only disables a false-positive lockdep warning, it does > >>>> not allow taking the same mutex twice, you will still hang on > >>>> the second mutex_lock call on the same lock. > >>> > >>> Thanks for the technical background about rt_mutex_lock_nested. I > >>> think we should keep using it as is, since as I wrote above I don't > >>> have any reasoning to modify it here. > >> > >> Ack, now that my misreading of the code has been cleared up > >> I agree. > >> > >>>> Also I don't think you are allowed to use the bus_locking code > >>>> like this. The i2c bus-locking code is intended to deal with > >>>> busses which have muxes in them, where the mux must be set > >>>> to the right branch of the bus to reach the client and then > >>>> not be changed during the transfer to that client. > >>>> > >>>> So i2c-client drivers are never supposed to directly call > >>>> the bus-locking functions. > >>> > >>> I think you are not correct here. There are examples of i2c-clients > >>> which are using i2c bus_locking for the purpose of locking adapter for > >>> the bunch of i2c transactions. > >>> > >>> As an example let's take drivers/char/tpm/tpm_tis_i2c_cr50.c. It > >>> operates in write-wait-read model and there is i2c_lock_bus() call > >>> used to ensure that bus won't be released - > >>> > >> https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.c > >> om%2Ftorvalds%2Flinux%2Fblob%2Fmaster%2Fdrivers%2Fchar%2Ftpm%2Ftpm_ > >> tis_i2c_cr50.c%23L202&data=04%7C01%7Cmario.limonciello%40amd.com > >> %7C1bdc742bc2a24f59b7d908d9dcc95438%7C3dd8961fe4884e608e11a82d994 > >> e183d%7C0%7C0%7C637783579554955840%7CUnknown%7CTWFpbGZsb3d8ey > >> JWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C > >> 3000&sdata=fr0UEka%2BxYyPxqUG6oOZo%2Bc6pWgto2mD7hWA20YulVQ > >> %3D&reserved=0. > >>> > >>> Similar model is followed in drivers/char/tpm/tpm_i2c_infineon.c and > >>> couple of other i2c-client drivers. > >> > >> Ah I see, interesting (live and learn). > >> > >> But this is then combined with using the special __i2c_transfer() > >> function for the actual i2c reads/writes, since using the regular > >> i2c_transfer() function after already taking the lock would deadlock. > >> > >> There is a similar unlocked raw __i2c_smbus_xfer(), but as the > >> comment in include/linux/i2c.h above the locked i2c_smbus_xfer() says: > >> > >> /* This is the very generalized SMBus access routine. You probably do not > >> want to use this, though; one of the functions below may be much easier, > >> and probably just as fast. > >> Note that we use i2c_adapter here, because you do not need a specific > >> smbus adapter to call this function. */ > >> s32 i2c_smbus_xfer(...); > >> > >> So in this case a driver cannot use the usual > >> i2c_smbus_read_byte/word/byte_data/word_data() helpers and > >> the same for writes. Also using an i2c_regmap (which is used > >> in a ton of places like PMIC drivers) will not work this way. > >> > >> So yes you can use i2c_bus_lock() for this; but only if all the > >> drivers where you want to do that limit themselves to > >> __i2c_transfer() and __i2c_smbus_xfer() calls and/or are > >> rewritten to only use those. > >>>> This is why in the Bay Trail case we have i2c-drivers > >>>> directly calling iosf_mbi_block_punit_i2c_access() and > >>>> iosf_mbi_unblock_punit_i2c_access() to lock the bus > >>>> for multiple i2c-transfers. We can get away with this there > >>>> because the bus in question is only used to access the > >>>> PMIC and that PMIC is only used on Bay Trail (and CHT) > >>>> boards, so the PMIC drivers can just hard-code these > >>>> calls. > >>>> > >>>> If you need to take the PSP I2C semaphore for multiple > >>>> transfers in some generic drivers, then I guess that the > >>>> i2c-subsys will need to get some new i2c_adapter callbacks > >>>> to acquire / release the bus for i2c-controllers where > >>>> the bus/controller is shared with some co-processor like > >>>> in the PSP case. > >>> > >>> This is exactly my intention to support generic i2c-clients drivers > >>> without them being aware that i2c-adapter above is using some > >>> semaphore/arbitration. Hopefully you can agree with me that currently > >>> available bus_locking can be used and is enough for this purpose. > >> > >> It can be used, but with limitations, see above. > >> > >>> > >>>> Also note that iosf_mbi_block_punit_i2c_access() and > >>>> iosf_mbi_unblock_punit_i2c_access() do their own > >>>> ref/lock-counting to allow calling them multiple times and > >>>> the first block call takes the bus and the last unblock > >>>> call releases it. > >>> > >>> This is exactly what I was talking about above and also implemented > >>> within psp_acquire_i2c_bus() and psp_release_i2c_bus(). > >> > >> Right, I was to quick in skimming over your code when > >> I wrote down my concerns about there being a deadlock > >> there, sorry. > >> > >> Regards, > >> > >> Hans
[Public] > > > > >>>> > > >>>> Through here seems to all be generic code for accessing > > >>>> the AMD PSP. To me this seems like something which belongs > > >>>> in a separate AMD-PSP-mbox driver/lib, which can then be > > >>>> shared between other kernel drivers which may also want > > >>>> to access PSP. > > >>> > > >>> I see your point clearly and actually it is not an accident that I've > > >>> put all PSP-mailbox methods in one "block". They are logically > > >>> different than the rest of i2c-adapter specific methods. > > >>> > > >>> That being said, above PSP mailbox was created by AMD solely for the > > >>> purpose of i2c_arbitration. It has its own set of commands and > > >>> specific format of the command-response buffer. Thus it is not and it > > >>> won't be generic in the future. There are already upstreamed drivers > > >>> from AMD (under drivers/crypto/ccp/) which made use of PSP, however > > >>> their channel of communication looks completely different than the > > >>> very simple i2c_arbitration model implemented above. > > >>> > > > > > > Can you please double confirm no other users for this mailbox and it's for > > > you only? And if so is it only specific to this platform that no other users? > > > At least on some UEFI AMD platforms that mailbox is defined for > > > communication with something else. We might need some way to attest > > > from the firmware that it's safe to use. > > > > > > The only mailbox that is defined for OS use across different silicon AFAIK is > > > the GPU driver mailbox. It may be safer to use that, but I'm not sure if > > > GPU driver has come up early enough for your use. > > > > The CCP/PSP driver will load as a PCIe device driver on this platform and > > will ioremap the MMIO space. Today, that driver doesn't reference those > > mailbox registers, and I don't know that there will be a need in the > > future. But if there is a need in the future, you could end up with a > > conflict between these two drivers. > > Right, so I have confirmed this with AMD PSP firmware developers, that > this particular x86-PSP mailbox is created and will be reserved > _solely_ for the purpose of i2c arbitration (and thus this driver). > There is no intend to use it elsewhere or share with another users. I've learned never to say never. People move on to different roles and history gets lost. As it's exclusive to this use case of I2C arbitration it will probably still be useful to leave in the comments nearby for our future selves what model/family SOC this was introduced in case the number of mailboxes are extinguished some day and this ends up being needed for a secondary purpose for some future SOC. > > > Thanks, > > Tom > > > > > > > >>> Because of this I'm treating this as an i2c_semaphore-related code and > > >>> putting this in this module. In my opinion moving this into some > > >>> separate driver (which will be actually used only here) makes code > > >>> less clear. But let's also hear some voice from AMD. > > >> > > >> Since as you say this mailbox is special and only for i2c-arbitration, > > >> keeping it inside this patch / .c file is fine. > > >> > > >>> > > >>>> > > >>>> Sorta like the generic iosf_mbi_read() and > > >>>> iosf_mbi_write() functions from: > > >>>> > > >>>> arch/x86/platform/intel/iosf_mbi.c > > >>>> > > >>>> used on the Intel chips, which are also used outside of > > >>>> the I2C bus-locking code. > > >>>> > > >>>> This is also one of the reasons why I think it would be > > >>>> good to get some AMD folks involved in this, since they > > >>>> may be aware of other drivers which also need to access > > >>>> the PSP mbox. > > >>>> > > >>> > > >>> Right, I'm adding mario.limonciello@amd.com to the CC, so that he can > > >> comment. > > >>> > > >>> (...) > > >>> > > >>>>> +/* > > >>>>> + * Locking methods are based on the default implementation from > > >>>>> + * drivers/i2c/i2c-core.base.c, but with psp acquire and release > operations > > >>>>> + * added. With this in place we can ensure that i2c clients on the bus > shared > > >>>>> + * with psp are able to lock HW access to the bus for arbitrary number > of > > >>>>> + * operations - that is e.g. write-wait-read. > > >>>>> + */ > > >>>>> +static void i2c_adapter_dw_psp_lock_bus(struct i2c_adapter *adapter, > > >>>>> + unsigned int flags) > > >>>>> +{ > > >>>>> + psp_acquire_i2c_bus(); > > >>>>> + rt_mutex_lock_nested(&adapter->bus_lock, > > >> i2c_adapter_depth(adapter)); > > >>>> > > >>>> This does not do what you think it does and you will still deadlock > > >>>> when things nest because of someone taking the bus-lock and then > > >>>> the main i2c-designware transfer function calling the acquire_lock > > >>>> callback. > > >>> > > >>> I haven't used rt_mutex_lock_nested() with the intent to prevent me > > >>> from deadlock when i2c-designware calls acquire_lock with bus-lock > > >>> already taken. This is a method copied from > > >>> drivers/i2c/i2c-core-base.c (BTW, I have a typo in above comment). > > >>> This is the default implementation applied by i2c-core when particular > > >>> adapter doesn't register its own locking callbacks - thus it is called > > >>> for i2c-designware for all platforms. > > >>> > > >>> In case of this driver internal i2c-designware acquire_lock() is equal > > >>> to psp_acquire_i2c_bus(). In other words, bus-level lock > > >>> i2c_adapter_dw_psp_lock_bus() is a superset of internal adapter's > > >>> acquire_lock(). > > >> > > >> Ah I missed that this is just mimicking the core functions + > > >> an extra call to psp_acquire_i2c_bus(). > > >> > > >> I assumed that the dwc->acquire callback path was also taking > > >> the mutex and I thought you had fallen for the _nested meaning > > >> something different then it does, my bad. > > >> > > >>> In order to prevent deadlock which you are talking about, I'm using > > >>> reference lock counter inside psp_acquire_i2c_bus() thus it is safe to > > >>> invoke acquire_lock() when bus-lock is already taken. > > >> > > >> Ah good, that is pretty much is the same as what the Bay Trail code > > >> is doing. > > >> > > >>> > > >>>> > > >>>> The _nested postfix is only for the lockdep lock-debugger, this > > >>>> actually turns into a regular mutex_lock when lockdep is not enabled: > > >>>> > > >>>> #ifdef CONFIG_DEBUG_LOCK_ALLOC > > >>>> extern void rt_mutex_lock_nested(struct rt_mutex *lock, unsigned int > > >> subclass); > > >>>> #define rt_mutex_lock(lock) rt_mutex_lock_nested(lock, 0) > > >>>> #else > > >>>> extern void rt_mutex_lock(struct rt_mutex *lock); > > >>>> #define rt_mutex_lock_nested(lock, subclass) rt_mutex_lock(lock) > > >>>> #endif > > >>>> > > >>>> The _nested postfix as such is only to tell the lockdep code that > > >>>> even though it seems we are trying to take the same mutex twice > > >>>> since in both cases it is of i2c_adapter.rt_mutex "lock class" > > >>>> that we are sure it is never the same i2c_adapter (but rather > > >>>> one which always gets called in a nested fashion from another > > >>>> i2c_adapter). > > >>>> > > >>>> IOW this only disables a false-positive lockdep warning, it does > > >>>> not allow taking the same mutex twice, you will still hang on > > >>>> the second mutex_lock call on the same lock. > > >>> > > >>> Thanks for the technical background about rt_mutex_lock_nested. I > > >>> think we should keep using it as is, since as I wrote above I don't > > >>> have any reasoning to modify it here. > > >> > > >> Ack, now that my misreading of the code has been cleared up > > >> I agree. > > >> > > >>>> Also I don't think you are allowed to use the bus_locking code > > >>>> like this. The i2c bus-locking code is intended to deal with > > >>>> busses which have muxes in them, where the mux must be set > > >>>> to the right branch of the bus to reach the client and then > > >>>> not be changed during the transfer to that client. > > >>>> > > >>>> So i2c-client drivers are never supposed to directly call > > >>>> the bus-locking functions. > > >>> > > >>> I think you are not correct here. There are examples of i2c-clients > > >>> which are using i2c bus_locking for the purpose of locking adapter for > > >>> the bunch of i2c transactions. > > >>> > > >>> As an example let's take drivers/char/tpm/tpm_tis_i2c_cr50.c. It > > >>> operates in write-wait-read model and there is i2c_lock_bus() call > > >>> used to ensure that bus won't be released - > > >>> > > >> > https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.c > %2F&data=04%7C01%7CMario.Limonciello%40amd.com%7Cf8cd037a72b > 749fb003408d9df3f5ced%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0 > %7C637786285501955886%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAw > MDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata > =s4N9vTWbuRUww0qvsRz16fV%2Fnj41SxyLOB%2BjGFjyOaA%3D&reserved > =0 > > >> > om%2Ftorvalds%2Flinux%2Fblob%2Fmaster%2Fdrivers%2Fchar%2Ftpm%2Ftpm_ > > >> > tis_i2c_cr50.c%23L202&data=04%7C01%7Cmario.limonciello%40amd.com > > >> > %7C1bdc742bc2a24f59b7d908d9dcc95438%7C3dd8961fe4884e608e11a82d994 > > >> > e183d%7C0%7C0%7C637783579554955840%7CUnknown%7CTWFpbGZsb3d8ey > > >> > JWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C > > >> > 3000&sdata=fr0UEka%2BxYyPxqUG6oOZo%2Bc6pWgto2mD7hWA20YulVQ > > >> %3D&reserved=0. > > >>> > > >>> Similar model is followed in drivers/char/tpm/tpm_i2c_infineon.c and > > >>> couple of other i2c-client drivers. > > >> > > >> Ah I see, interesting (live and learn). > > >> > > >> But this is then combined with using the special __i2c_transfer() > > >> function for the actual i2c reads/writes, since using the regular > > >> i2c_transfer() function after already taking the lock would deadlock. > > >> > > >> There is a similar unlocked raw __i2c_smbus_xfer(), but as the > > >> comment in include/linux/i2c.h above the locked i2c_smbus_xfer() says: > > >> > > >> /* This is the very generalized SMBus access routine. You probably do not > > >> want to use this, though; one of the functions below may be much easier, > > >> and probably just as fast. > > >> Note that we use i2c_adapter here, because you do not need a specific > > >> smbus adapter to call this function. */ > > >> s32 i2c_smbus_xfer(...); > > >> > > >> So in this case a driver cannot use the usual > > >> i2c_smbus_read_byte/word/byte_data/word_data() helpers and > > >> the same for writes. Also using an i2c_regmap (which is used > > >> in a ton of places like PMIC drivers) will not work this way. > > >> > > >> So yes you can use i2c_bus_lock() for this; but only if all the > > >> drivers where you want to do that limit themselves to > > >> __i2c_transfer() and __i2c_smbus_xfer() calls and/or are > > >> rewritten to only use those. > > >>>> This is why in the Bay Trail case we have i2c-drivers > > >>>> directly calling iosf_mbi_block_punit_i2c_access() and > > >>>> iosf_mbi_unblock_punit_i2c_access() to lock the bus > > >>>> for multiple i2c-transfers. We can get away with this there > > >>>> because the bus in question is only used to access the > > >>>> PMIC and that PMIC is only used on Bay Trail (and CHT) > > >>>> boards, so the PMIC drivers can just hard-code these > > >>>> calls. > > >>>> > > >>>> If you need to take the PSP I2C semaphore for multiple > > >>>> transfers in some generic drivers, then I guess that the > > >>>> i2c-subsys will need to get some new i2c_adapter callbacks > > >>>> to acquire / release the bus for i2c-controllers where > > >>>> the bus/controller is shared with some co-processor like > > >>>> in the PSP case. > > >>> > > >>> This is exactly my intention to support generic i2c-clients drivers > > >>> without them being aware that i2c-adapter above is using some > > >>> semaphore/arbitration. Hopefully you can agree with me that currently > > >>> available bus_locking can be used and is enough for this purpose. > > >> > > >> It can be used, but with limitations, see above. > > >> > > >>> > > >>>> Also note that iosf_mbi_block_punit_i2c_access() and > > >>>> iosf_mbi_unblock_punit_i2c_access() do their own > > >>>> ref/lock-counting to allow calling them multiple times and > > >>>> the first block call takes the bus and the last unblock > > >>>> call releases it. > > >>> > > >>> This is exactly what I was talking about above and also implemented > > >>> within psp_acquire_i2c_bus() and psp_release_i2c_bus(). > > >> > > >> Right, I was to quick in skimming over your code when > > >> I wrote down my concerns about there being a deadlock > > >> there, sorry. > > >> > > >> Regards, > > >> > > >> Hans
czw., 20 sty 2022 o 18:12 Andy Shevchenko <andriy.shevchenko@linux.intel.com> napisał(a): > > On Thu, Jan 20, 2022 at 11:33:05PM +0800, kernel test robot wrote: > > > If you fix the issue, kindly add following tag as appropriate > > Reported-by: kernel test robot <lkp@intel.com> > > > > All errors (new ones prefixed by >>): > > > > drivers/i2c/busses/i2c-designware-amdpsp.c: In function 'psp_send_cmd': > > >> drivers/i2c/busses/i2c-designware-amdpsp.c:130:2: error: implicit declaration of function 'writeq'; did you mean 'writel'? [-Werror=implicit-function-declaration] > > 130 | writeq((uintptr_t)__psp_pa((void *)req), &mbox->i2c_req_addr); > > | ^~~~~~ > > | writel > > cc1: some warnings being treated as errors > > Adding io-64-nonatomic-lo-hi.h after io.h should fix this. Correct, thanks! Actually io.h is directly included from io-64-nonatomic-lo-hi.h. Best Regards, Jan > > > -- > With Best Regards, > Andy Shevchenko > >
pt., 21 sty 2022 o 18:52 Andy Shevchenko <andriy.shevchenko@linux.intel.com> napisał(a): > > On Fri, Jan 21, 2022 at 11:20:19AM +0100, Jan Dąbroś wrote: > > czw., 20 sty 2022 o 15:21 Andy Shevchenko > > <andriy.shevchenko@linux.intel.com> napisał(a): > > > On Thu, Jan 20, 2022 at 01:16:21AM +0100, Jan Dabros wrote: > > ... > > > > > Add new entry in MAINTAINERS file to cover new module. > > > > > > It's confusing. You added yourself as a reviewer for I2C DesignWare driver, > > > which is great, but not described in the commit message. > > > > Should I rephrase this sentence (to be more specific that I may be > > helpful for reviewing amdpsp.c module) or rather you mean to exclude > > drivers/i2c/busses/i2c-designware-amdpsp.c from the rest of > > designware-* modules and create separate entry? > > > > Actually initially I wasn't planning to modify MAINTAINERS (after all > > I'm not an I2C DesignWare expert) until run checkpatch.pl which > > recommended to do so when adding new file. Eventually for me it made > > some sense since I have a platform equipped with AMD Cezanne SoC and I > > will be able to review and test potential changes in > > i2c-designware-amdpsp.c or in general around semaphore areas. > > > > This may also work with different model, similar to how you pointed me > > to Hans as an owner of Bay Trail platform who is acquinted with how > > its i2c semaphore is working. I will simply remove myself from the > > MAINTAINERS file and you can add me to the threads if there is > > anything requiring my help. > > > > Let me know which way is working for you. I just thought it is not > > good to leave you alone with a new module which you cannot actually > > test and don't have deep insight about how PSP-x86 communications > > works. > > You have a few options: > - leave it for us (but it probably won't go well in long-term as you noticed) > - add yourself as a reviewer (it doesn't require to review everything, but > you will get all i2c DesignWare driver changes) I think this is the best option. So I will leave initial change to the MAINTAINERS as is, but will put few more words of description into commit message. > - add a new MAINTAINERS database entry where you can put yourself for that > file even as a maintainaer > > ... > > > > > +#include <asm/msr.h> > > > > > > Usually linux/* followed by asm/*. > > > > > > > +#include <linux/i2c.h> > > > > +#include <linux/psp-sev.h> > > > > > > types.h? > > > > I need to take a deeper look at the headers included here, especially > > considering errors pointed by kernel test robot. Not sure why I > > haven't seen any issues on my setup. > > The problem here is not so visible. Headers should be a compromise between > what is really used and what we may include for that. There are headers > that guaranteed to be included by others, otherwise it will be an implicit > dependency which is not good in cases of generic headers, such as types.h. ACK. I included couple of new headers in v2. > > ... > > > > > +union psp_req_buffer_hdr { > > > > + struct { > > > > + u32 total_size; > > > > + u32 status; > > > > + } __packed; > > > > > > What does packet bring you here? > > > > In this particular case binary-wise nothing, I can as well drop this. > > It may be necessary if there are some changes in this structs fields > > in future (e.g changing total_size to u8), since PSP expects every > > field to be byte-aligned. > > _packed will make another thing which you probably won't need, it brings > the entire structure to be possible on unaligned addresses and then some > warnings from compiler may be issued even if there is no problem in the > flow. > > Read this discussion: https://lore.kernel.org/linux-media/20220110224656.266536-1-sakari.ailus@linux.intel.com/ OK, I see your point - thanks for this pointer, something new for me. > > > > > + u64 hdr_val; > > > > > > And why does this not have the same alignment since it's also part of > > > the union? > > > > __packed is not about alignment of the whole struct/union > > It's. See above. ACK. > > > but about > > lack of padding between its fields. As above - in this particular case > > with two u32 it doesn't matter. > > ... > > > > > +struct psp_i2c_req { > > > > + union psp_req_buffer_hdr hdr; > > > > + enum psp_i2c_req_type type; > > > > > > > +} __packed __aligned(32); > > > > > > Can you explain, what this means and how it's supposed to work? > > > > This means that each instance of the struct should be aligned (32) > > while at the same time no padding within members - thus this may > > result in non-aligned addresses of members. > > 32 bytes? And on unaligned address at the same time. Yes, it was initial recommendation. I left this as is in v2, will work with AMD to see if we can lower (get rid of) this requirement. > > ... > > > > > +union psp_mbox_cmd_reg { > > > > + struct psp_mbox_cmd_fields { > > > > + u16 mbox_status; > > > > + u8 mbox_cmd; > > > > + u8 reserved:6; > > > > + u8 recovery:1; > > > > + u8 ready:1; > > > > > > > + } __packed fields; > > > > > > So, what is the __packed purpose here? > > > > As in all above cases - considering current layout of members and > > their sizes dropping `__packed` will not results in any errors. > > > > However PSP expects all members os structs within shared buffers to be > > byte-aligned, that's why I've added this attributes to be on the safe > > side. If you think this doesn't make sense, I can remove them - in > > (very unlikely) case of changes, one will need to add this specifier. > > I guess you don't need them at all in this case in any of the data structure > you created here. ACK, I removed unnecessary __packed attributes in v2. > ... > > > > > + struct psp_mbox *mbox = (struct psp_mbox *)mbox_iomem; > > > > > > Heck, no! > > > > I need to get acquinted to the kernel-reviewer language:) Pardon my > > ignorance, but just to be sure I get what you mean here: > > I'm using global mbox_iomem to keep address of PSP mailbox in IO > > memory. Casting this to struct psp_mbox layout here, to make access > > more convenient. > > Your point here is that: > > 1. I should move the assignment out from the variable declaration part > > of this function; > > 2. I should use ioremap/iounmap each time in psp_send_cmd instead of > > using it once in `probe` and unmap in `remove`? > > I thought about this option as to be less effective performance-wise > > (even though I can get rid of one global variable). > > 3. Something else? > > Casting an __iomem pointer to the some struct without keeping it. I believe > sparse should blow because of this. Oh right.. pretty obvious. Fixed. > > ... > > > > > + /* Fill address of command-response buffer */ > > > > + writeq((uintptr_t)__psp_pa((void *)req), &mbox->i2c_req_addr); > > > > > > What does this voodoo mean?! > > > > Basically I need to take physical address (__psp_pa) of request buffer > > req and write this down into mailbox IO memory. > > This should be spread into more lines with some comments, is this your point? > > It needs much better comment explaining what is this address and its meaning > for the hardware and why you need physical address here (DMA?). For me it looks > like a voodoo. Ah, and not using phys_addr_t / dma_addr_t / etc type here, but > uintptr_t just adds a confusion. ACK. > ... > > > > > + start = jiffies; > > > > + do { > > > > + if (psp_send_cmd(req)) { > > > > + ret = -EIO; > > > > + goto cleanup; > > > > + } > > > > + > > > > + status = check_i2c_req_sts(req); > > > > + if (!status) { > > > > + dev_dbg(psp_i2c_dev, "Request accepted by PSP after %ums\n", > > > > + jiffies_to_msecs(jiffies - start)); > > > > + ret = 0; > > > > + goto cleanup; > > > > + } else if (status == -EBUSY) { > > > > + retry_cnt--; > > > > + } else { > > > > + ret = -EIO; > > > > + goto cleanup; > > > > + }; > > > > + > > > > + /* IF EBUSY, give PSP time to finish its i2c activities */ > > > > + mdelay(PSP_I2C_REQ_RETRY_DELAY_MSEC); > > > > + } while (retry_cnt); > > > > > > NIH iopoll.h API(s). > > > > I don't think macros avaialble in iopoll.h are suitable here. > > Procedure above is not about simply reading some IO and waiting for > > particular condition to be met with this particular value. Eventually > > `psp_send_cmd()` invokes `psp_wait_cmd()` where I'm using > > `readl_poll_timeout()`, so on lower level I'm making use of this API. > > However here I don't see any obvious method how to incorporate > > iopoll.h API to reach my goal. > > You do not go to clean up if and only if status == -EBUSY, so here we have > a condition. the rest can be moved to a function that you wrap by > read_poll_timeout_atomic() (pay attention to the macro name). > Here is the question, btw, why _atomic()? I.o.w. why mdelay() and not msleep()? Right, this was another error - above code shouldn't be executed in atomic context. I used read_poll_timeout() in v2. > > > > + ret = -ETIMEDOUT; > > ... > > > > Handle errors first. > > > Addressing above two comments - what do you think about below: > > if (status) { > > if (status == -ETIMEDOUT) > > dev_err(psp_i2c_dev, "Timed out waiting for PSP to > > release I2C bus\n"); > > else > > dev_err(psp_i2c_dev, "PSP communication error\n"); > > > dev_err(psp_i2c_dev, "PSP communication error\n"); > > This dup message is not needed, otherwise fine to me. ACK. > > > psp_i2c_mbox_fail = true; > > goto cleanup; > > } > > > > psp_i2c_sem_acquired = jiffies; > > psp_i2c_access_count++; > > (...) > > ... > > > > > +static int i2c_dw_probe_lock_support(struct dw_i2c_dev *dev) > > > > +{ > > > > + int ret; > > > > + int i; > > > > + > > > > + dev->semaphore_idx = -1; > > > > + > > > > + for (i = 0; i < ARRAY_SIZE(i2c_dw_semaphore_cb_table); i++) { > > > > > > > + if (!i2c_dw_semaphore_cb_table[i].probe) > > > > + continue; > > > > > > Huh? > > > > Just to be sure I get your point. > > Once I found terminating entry, I will get out of the loop and return > > 0 as there are no semaphores to be "applied". Actually I should > > probably use `break;` here as there shouldn't be a case when certain > > type of semaphore installs without `probe` being implemented. > > Yes, that's what I though, and rather using ARRAY_SIZE(), use a terminator you > provided. > > Originally you have used two approaches which seems competing with each other: > - ARRAY_SIZE > - terminator entry > > And on top of that so confusing continue on top of not-found ->probe() that > makes me think what the meaning of the entry that has set ->remove(), but no > ->probe(). > > That said, I would rather see something like > > struct ... *p = &...; > > while (p->probe) { > ... > p++; > } ACK. > -- > With Best Regards, > Andy Shevchenko > >
śr., 26 sty 2022 o 20:36 Limonciello, Mario <Mario.Limonciello@amd.com> napisał(a): > > [Public] > > > > > > > >>>> > > > >>>> Through here seems to all be generic code for accessing > > > >>>> the AMD PSP. To me this seems like something which belongs > > > >>>> in a separate AMD-PSP-mbox driver/lib, which can then be > > > >>>> shared between other kernel drivers which may also want > > > >>>> to access PSP. > > > >>> > > > >>> I see your point clearly and actually it is not an accident that I've > > > >>> put all PSP-mailbox methods in one "block". They are logically > > > >>> different than the rest of i2c-adapter specific methods. > > > >>> > > > >>> That being said, above PSP mailbox was created by AMD solely for the > > > >>> purpose of i2c_arbitration. It has its own set of commands and > > > >>> specific format of the command-response buffer. Thus it is not and it > > > >>> won't be generic in the future. There are already upstreamed drivers > > > >>> from AMD (under drivers/crypto/ccp/) which made use of PSP, however > > > >>> their channel of communication looks completely different than the > > > >>> very simple i2c_arbitration model implemented above. > > > >>> > > > > > > > > Can you please double confirm no other users for this mailbox and it's for > > > > you only? And if so is it only specific to this platform that no other users? > > > > At least on some UEFI AMD platforms that mailbox is defined for > > > > communication with something else. We might need some way to attest > > > > from the firmware that it's safe to use. > > > > > > > > The only mailbox that is defined for OS use across different silicon AFAIK is > > > > the GPU driver mailbox. It may be safer to use that, but I'm not sure if > > > > GPU driver has come up early enough for your use. > > > > > > The CCP/PSP driver will load as a PCIe device driver on this platform and > > > will ioremap the MMIO space. Today, that driver doesn't reference those > > > mailbox registers, and I don't know that there will be a need in the > > > future. But if there is a need in the future, you could end up with a > > > conflict between these two drivers. > > > > Right, so I have confirmed this with AMD PSP firmware developers, that > > this particular x86-PSP mailbox is created and will be reserved > > _solely_ for the purpose of i2c arbitration (and thus this driver). > > There is no intend to use it elsewhere or share with another users. > > I've learned never to say never. People move on to different roles and history > gets lost. As it's exclusive to this use case of I2C arbitration it will probably still > be useful to leave in the comments nearby for our future selves what model/family > SOC this was introduced in case the number of mailboxes are extinguished some > day and this ends up being needed for a secondary purpose for some future SOC. OK. I put a comment in v2. Please take a look. > > > > > > Thanks, > > > Tom > > > > > > > > > > >>> Because of this I'm treating this as an i2c_semaphore-related code and > > > >>> putting this in this module. In my opinion moving this into some > > > >>> separate driver (which will be actually used only here) makes code > > > >>> less clear. But let's also hear some voice from AMD. > > > >> > > > >> Since as you say this mailbox is special and only for i2c-arbitration, > > > >> keeping it inside this patch / .c file is fine. > > > >> > > > >>> > > > >>>> > > > >>>> Sorta like the generic iosf_mbi_read() and > > > >>>> iosf_mbi_write() functions from: > > > >>>> > > > >>>> arch/x86/platform/intel/iosf_mbi.c > > > >>>> > > > >>>> used on the Intel chips, which are also used outside of > > > >>>> the I2C bus-locking code. > > > >>>> > > > >>>> This is also one of the reasons why I think it would be > > > >>>> good to get some AMD folks involved in this, since they > > > >>>> may be aware of other drivers which also need to access > > > >>>> the PSP mbox. > > > >>>> > > > >>> > > > >>> Right, I'm adding mario.limonciello@amd.com to the CC, so that he can > > > >> comment. > > > >>> > > > >>> (...) > > > >>> > > > >>>>> +/* > > > >>>>> + * Locking methods are based on the default implementation from > > > >>>>> + * drivers/i2c/i2c-core.base.c, but with psp acquire and release > > operations > > > >>>>> + * added. With this in place we can ensure that i2c clients on the bus > > shared > > > >>>>> + * with psp are able to lock HW access to the bus for arbitrary number > > of > > > >>>>> + * operations - that is e.g. write-wait-read. > > > >>>>> + */ > > > >>>>> +static void i2c_adapter_dw_psp_lock_bus(struct i2c_adapter *adapter, > > > >>>>> + unsigned int flags) > > > >>>>> +{ > > > >>>>> + psp_acquire_i2c_bus(); > > > >>>>> + rt_mutex_lock_nested(&adapter->bus_lock, > > > >> i2c_adapter_depth(adapter)); > > > >>>> > > > >>>> This does not do what you think it does and you will still deadlock > > > >>>> when things nest because of someone taking the bus-lock and then > > > >>>> the main i2c-designware transfer function calling the acquire_lock > > > >>>> callback. > > > >>> > > > >>> I haven't used rt_mutex_lock_nested() with the intent to prevent me > > > >>> from deadlock when i2c-designware calls acquire_lock with bus-lock > > > >>> already taken. This is a method copied from > > > >>> drivers/i2c/i2c-core-base.c (BTW, I have a typo in above comment). > > > >>> This is the default implementation applied by i2c-core when particular > > > >>> adapter doesn't register its own locking callbacks - thus it is called > > > >>> for i2c-designware for all platforms. > > > >>> > > > >>> In case of this driver internal i2c-designware acquire_lock() is equal > > > >>> to psp_acquire_i2c_bus(). In other words, bus-level lock > > > >>> i2c_adapter_dw_psp_lock_bus() is a superset of internal adapter's > > > >>> acquire_lock(). > > > >> > > > >> Ah I missed that this is just mimicking the core functions + > > > >> an extra call to psp_acquire_i2c_bus(). > > > >> > > > >> I assumed that the dwc->acquire callback path was also taking > > > >> the mutex and I thought you had fallen for the _nested meaning > > > >> something different then it does, my bad. > > > >> > > > >>> In order to prevent deadlock which you are talking about, I'm using > > > >>> reference lock counter inside psp_acquire_i2c_bus() thus it is safe to > > > >>> invoke acquire_lock() when bus-lock is already taken. > > > >> > > > >> Ah good, that is pretty much is the same as what the Bay Trail code > > > >> is doing. > > > >> > > > >>> > > > >>>> > > > >>>> The _nested postfix is only for the lockdep lock-debugger, this > > > >>>> actually turns into a regular mutex_lock when lockdep is not enabled: > > > >>>> > > > >>>> #ifdef CONFIG_DEBUG_LOCK_ALLOC > > > >>>> extern void rt_mutex_lock_nested(struct rt_mutex *lock, unsigned int > > > >> subclass); > > > >>>> #define rt_mutex_lock(lock) rt_mutex_lock_nested(lock, 0) > > > >>>> #else > > > >>>> extern void rt_mutex_lock(struct rt_mutex *lock); > > > >>>> #define rt_mutex_lock_nested(lock, subclass) rt_mutex_lock(lock) > > > >>>> #endif > > > >>>> > > > >>>> The _nested postfix as such is only to tell the lockdep code that > > > >>>> even though it seems we are trying to take the same mutex twice > > > >>>> since in both cases it is of i2c_adapter.rt_mutex "lock class" > > > >>>> that we are sure it is never the same i2c_adapter (but rather > > > >>>> one which always gets called in a nested fashion from another > > > >>>> i2c_adapter). > > > >>>> > > > >>>> IOW this only disables a false-positive lockdep warning, it does > > > >>>> not allow taking the same mutex twice, you will still hang on > > > >>>> the second mutex_lock call on the same lock. > > > >>> > > > >>> Thanks for the technical background about rt_mutex_lock_nested. I > > > >>> think we should keep using it as is, since as I wrote above I don't > > > >>> have any reasoning to modify it here. > > > >> > > > >> Ack, now that my misreading of the code has been cleared up > > > >> I agree. > > > >> > > > >>>> Also I don't think you are allowed to use the bus_locking code > > > >>>> like this. The i2c bus-locking code is intended to deal with > > > >>>> busses which have muxes in them, where the mux must be set > > > >>>> to the right branch of the bus to reach the client and then > > > >>>> not be changed during the transfer to that client. > > > >>>> > > > >>>> So i2c-client drivers are never supposed to directly call > > > >>>> the bus-locking functions. > > > >>> > > > >>> I think you are not correct here. There are examples of i2c-clients > > > >>> which are using i2c bus_locking for the purpose of locking adapter for > > > >>> the bunch of i2c transactions. > > > >>> > > > >>> As an example let's take drivers/char/tpm/tpm_tis_i2c_cr50.c. It > > > >>> operates in write-wait-read model and there is i2c_lock_bus() call > > > >>> used to ensure that bus won't be released - > > > >>> > > > >> > > https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.c > > %2F&data=04%7C01%7CMario.Limonciello%40amd.com%7Cf8cd037a72b > > 749fb003408d9df3f5ced%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0 > > %7C637786285501955886%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAw > > MDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata > > =s4N9vTWbuRUww0qvsRz16fV%2Fnj41SxyLOB%2BjGFjyOaA%3D&reserved > > =0 > > > >> > > om%2Ftorvalds%2Flinux%2Fblob%2Fmaster%2Fdrivers%2Fchar%2Ftpm%2Ftpm_ > > > >> > > tis_i2c_cr50.c%23L202&data=04%7C01%7Cmario.limonciello%40amd.com > > > >> > > %7C1bdc742bc2a24f59b7d908d9dcc95438%7C3dd8961fe4884e608e11a82d994 > > > >> > > e183d%7C0%7C0%7C637783579554955840%7CUnknown%7CTWFpbGZsb3d8ey > > > >> > > JWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C > > > >> > > 3000&sdata=fr0UEka%2BxYyPxqUG6oOZo%2Bc6pWgto2mD7hWA20YulVQ > > > >> %3D&reserved=0. > > > >>> > > > >>> Similar model is followed in drivers/char/tpm/tpm_i2c_infineon.c and > > > >>> couple of other i2c-client drivers. > > > >> > > > >> Ah I see, interesting (live and learn). > > > >> > > > >> But this is then combined with using the special __i2c_transfer() > > > >> function for the actual i2c reads/writes, since using the regular > > > >> i2c_transfer() function after already taking the lock would deadlock. > > > >> > > > >> There is a similar unlocked raw __i2c_smbus_xfer(), but as the > > > >> comment in include/linux/i2c.h above the locked i2c_smbus_xfer() says: > > > >> > > > >> /* This is the very generalized SMBus access routine. You probably do not > > > >> want to use this, though; one of the functions below may be much easier, > > > >> and probably just as fast. > > > >> Note that we use i2c_adapter here, because you do not need a specific > > > >> smbus adapter to call this function. */ > > > >> s32 i2c_smbus_xfer(...); > > > >> > > > >> So in this case a driver cannot use the usual > > > >> i2c_smbus_read_byte/word/byte_data/word_data() helpers and > > > >> the same for writes. Also using an i2c_regmap (which is used > > > >> in a ton of places like PMIC drivers) will not work this way. > > > >> > > > >> So yes you can use i2c_bus_lock() for this; but only if all the > > > >> drivers where you want to do that limit themselves to > > > >> __i2c_transfer() and __i2c_smbus_xfer() calls and/or are > > > >> rewritten to only use those. > > > >>>> This is why in the Bay Trail case we have i2c-drivers > > > >>>> directly calling iosf_mbi_block_punit_i2c_access() and > > > >>>> iosf_mbi_unblock_punit_i2c_access() to lock the bus > > > >>>> for multiple i2c-transfers. We can get away with this there > > > >>>> because the bus in question is only used to access the > > > >>>> PMIC and that PMIC is only used on Bay Trail (and CHT) > > > >>>> boards, so the PMIC drivers can just hard-code these > > > >>>> calls. > > > >>>> > > > >>>> If you need to take the PSP I2C semaphore for multiple > > > >>>> transfers in some generic drivers, then I guess that the > > > >>>> i2c-subsys will need to get some new i2c_adapter callbacks > > > >>>> to acquire / release the bus for i2c-controllers where > > > >>>> the bus/controller is shared with some co-processor like > > > >>>> in the PSP case. > > > >>> > > > >>> This is exactly my intention to support generic i2c-clients drivers > > > >>> without them being aware that i2c-adapter above is using some > > > >>> semaphore/arbitration. Hopefully you can agree with me that currently > > > >>> available bus_locking can be used and is enough for this purpose. > > > >> > > > >> It can be used, but with limitations, see above. > > > >> > > > >>> > > > >>>> Also note that iosf_mbi_block_punit_i2c_access() and > > > >>>> iosf_mbi_unblock_punit_i2c_access() do their own > > > >>>> ref/lock-counting to allow calling them multiple times and > > > >>>> the first block call takes the bus and the last unblock > > > >>>> call releases it. > > > >>> > > > >>> This is exactly what I was talking about above and also implemented > > > >>> within psp_acquire_i2c_bus() and psp_release_i2c_bus(). > > > >> > > > >> Right, I was to quick in skimming over your code when > > > >> I wrote down my concerns about there being a deadlock > > > >> there, sorry. > > > >> > > > >> Regards, > > > >> > > > >> Hans
diff --git a/MAINTAINERS b/MAINTAINERS index b84e2d5642bc..3c81084bc6e6 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -18659,6 +18659,7 @@ SYNOPSYS DESIGNWARE I2C DRIVER M: Jarkko Nikula <jarkko.nikula@linux.intel.com> R: Andy Shevchenko <andriy.shevchenko@linux.intel.com> R: Mika Westerberg <mika.westerberg@linux.intel.com> +R: Jan Dabros <jsd@semihalf.com> L: linux-i2c@vger.kernel.org S: Maintained F: drivers/i2c/busses/i2c-designware-* diff --git a/drivers/acpi/acpi_apd.c b/drivers/acpi/acpi_apd.c index e7934ba79b02..4a812476a418 100644 --- a/drivers/acpi/acpi_apd.c +++ b/drivers/acpi/acpi_apd.c @@ -236,6 +236,7 @@ static const struct acpi_device_id acpi_apd_device_ids[] = { { "AMD0020", APD_ADDR(cz_uart_desc) }, { "AMDI0020", APD_ADDR(cz_uart_desc) }, { "AMDI0022", APD_ADDR(cz_uart_desc) }, + { "AMDI0019", APD_ADDR(wt_i2c_desc) }, { "AMD0030", }, { "AMD0040", APD_ADDR(fch_misc_desc)}, { "HYGO0010", APD_ADDR(wt_i2c_desc) }, diff --git a/drivers/i2c/busses/Kconfig b/drivers/i2c/busses/Kconfig index 42da31c1ab70..9177a56d2e94 100644 --- a/drivers/i2c/busses/Kconfig +++ b/drivers/i2c/busses/Kconfig @@ -553,6 +553,16 @@ config I2C_DESIGNWARE_PLATFORM This driver can also be built as a module. If so, the module will be called i2c-designware-platform. +config I2C_DESIGNWARE_AMDPSP + bool "AMD PSP I2C semaphore support" + depends on ACPI + depends on I2C_DESIGNWARE_PLATFORM + help + This driver enables managed host access to the selected I2C bus shared + between AMD CPU and AMD PSP. + + You should say Y if running on an AMD system equipped with the PSP. + config I2C_DESIGNWARE_BAYTRAIL bool "Intel Baytrail I2C semaphore support" depends on ACPI diff --git a/drivers/i2c/busses/Makefile b/drivers/i2c/busses/Makefile index 1d00dce77098..752f47be3fc1 100644 --- a/drivers/i2c/busses/Makefile +++ b/drivers/i2c/busses/Makefile @@ -54,6 +54,7 @@ i2c-designware-core-y += i2c-designware-master.o i2c-designware-core-$(CONFIG_I2C_DESIGNWARE_SLAVE) += i2c-designware-slave.o obj-$(CONFIG_I2C_DESIGNWARE_PLATFORM) += i2c-designware-platform.o i2c-designware-platform-y := i2c-designware-platdrv.o +i2c-designware-platform-$(CONFIG_I2C_DESIGNWARE_AMDPSP) += i2c-designware-amdpsp.o i2c-designware-platform-$(CONFIG_I2C_DESIGNWARE_BAYTRAIL) += i2c-designware-baytrail.o obj-$(CONFIG_I2C_DESIGNWARE_PCI) += i2c-designware-pci.o i2c-designware-pci-y := i2c-designware-pcidrv.o diff --git a/drivers/i2c/busses/i2c-designware-amdpsp.c b/drivers/i2c/busses/i2c-designware-amdpsp.c new file mode 100644 index 000000000000..e86cc56df921 --- /dev/null +++ b/drivers/i2c/busses/i2c-designware-amdpsp.c @@ -0,0 +1,357 @@ +// SPDX-License-Identifier: GPL-2.0 + +#include <asm/msr.h> +#include <linux/i2c.h> +#include <linux/psp-sev.h> + +#include "i2c-designware-core.h" + +#define MSR_AMD_PSP_ADDR 0xc00110a2 +#define PSP_MBOX_OFFSET 0x10570 +#define PSP_CMD_TIMEOUT_MS 500 + +#define PSP_I2C_REQ_BUS_CMD 0x64 +#define PSP_I2C_REQ_RETRY_CNT 10 +#define PSP_I2C_REQ_RETRY_DELAY_MSEC 50 +#define PSP_I2C_REQ_STS_OK 0x0 +#define PSP_I2C_REQ_STS_BUS_BUSY 0x1 +#define PSP_I2C_REQ_STS_INV_PARAM 0x3 + +union psp_req_buffer_hdr { + struct { + u32 total_size; + u32 status; + } __packed; + u64 hdr_val; +}; + +enum psp_i2c_req_type { + PSP_I2C_REQ_ACQUIRE, + PSP_I2C_REQ_RELEASE, + PSP_I2C_REQ_MAX, +}; + +struct psp_i2c_req { + union psp_req_buffer_hdr hdr; + enum psp_i2c_req_type type; +} __packed __aligned(32); + +union psp_mbox_cmd_reg { + struct psp_mbox_cmd_fields { + u16 mbox_status; + u8 mbox_cmd; + u8 reserved:6; + u8 recovery:1; + u8 ready:1; + } __packed fields; + u32 val; +}; + +struct psp_mbox { + union psp_mbox_cmd_reg fields; + uintptr_t i2c_req_addr; +} __packed; + +static DEFINE_MUTEX(psp_i2c_access_mutex); +static unsigned long psp_i2c_sem_acquired; +static void __iomem *mbox_iomem; +static u32 psp_i2c_access_count; +static bool psp_i2c_mbox_fail; +static struct device *psp_i2c_dev; + +static int psp_get_mbox_addr(unsigned long *mbox_addr) +{ + unsigned long long psp_mmio; + + if (rdmsrl_safe(MSR_AMD_PSP_ADDR, &psp_mmio)) + return -EIO; + + *mbox_addr = (unsigned long)(psp_mmio + PSP_MBOX_OFFSET); + + return 0; +} + +static int psp_mbox_probe(void) +{ + unsigned long mbox_addr; + + if (psp_get_mbox_addr(&mbox_addr)) + return -1; + + mbox_iomem = ioremap(mbox_addr, sizeof(struct psp_mbox)); + if (!mbox_iomem) + return -ENOMEM; + + return 0; +} + +/* Recovery field should be equal 0 to start sending commands */ +static int psp_check_mbox_recovery(struct psp_mbox *mbox) +{ + union psp_mbox_cmd_reg tmp = {0}; + + tmp.val = readl(&mbox->fields.val); + return !!tmp.fields.recovery; +} + +static int psp_wait_cmd(struct psp_mbox *mbox) +{ + union psp_mbox_cmd_reg expected = { .val = 0 }; + u32 tmp; + + /* Expect mbox_cmd to be cleared and ready bit to be set by PSP */ + expected.fields.ready = 1; + + return readl_poll_timeout(&mbox->fields.val, tmp, (tmp == expected.val), + 0, 1000 * PSP_CMD_TIMEOUT_MS); +} + +/* Status equal to 0 means that PSP succeed processing command */ +static int psp_check_mbox_sts(struct psp_mbox *mbox) +{ + union psp_mbox_cmd_reg cmd_reg = {0}; + + cmd_reg.val = readl(&mbox->fields.val); + return cmd_reg.fields.mbox_status; +} + +static int psp_send_cmd(struct psp_i2c_req *req) +{ + struct psp_mbox *mbox = (struct psp_mbox *)mbox_iomem; + union psp_mbox_cmd_reg cmd_reg = {0}; + + if (psp_check_mbox_recovery(mbox)) + return -EIO; + + if (psp_wait_cmd(mbox)) + return -EBUSY; + + /* Fill address of command-response buffer */ + writeq((uintptr_t)__psp_pa((void *)req), &mbox->i2c_req_addr); + + /* Write command register to trigger processing */ + cmd_reg.fields.mbox_cmd = PSP_I2C_REQ_BUS_CMD; + writel(cmd_reg.val, &mbox->fields.val); + + if (psp_wait_cmd(mbox)) + return -ETIMEDOUT; + + if (psp_check_mbox_sts(mbox)) + return -EIO; + + return 0; +} + +/* Helper to verify status returned by PSP */ +static int check_i2c_req_sts(struct psp_i2c_req *req) +{ + int status; + + status = readl(&req->hdr.status); + + switch (status) { + case PSP_I2C_REQ_STS_OK: + return 0; + case PSP_I2C_REQ_STS_BUS_BUSY: + return -EBUSY; + case PSP_I2C_REQ_STS_INV_PARAM: + default: + return -EIO; + }; +} + +static int psp_send_i2c_req(enum psp_i2c_req_type i2c_req_type) +{ + int status, ret, retry_cnt = PSP_I2C_REQ_RETRY_CNT; + struct psp_i2c_req *req; + unsigned long start; + + /* Allocate command-response buffer */ + req = kzalloc(sizeof(*req), GFP_KERNEL); + if (!req) + return -ENOMEM; + + req->hdr.total_size = sizeof(*req); + req->type = i2c_req_type; + + start = jiffies; + do { + if (psp_send_cmd(req)) { + ret = -EIO; + goto cleanup; + } + + status = check_i2c_req_sts(req); + if (!status) { + dev_dbg(psp_i2c_dev, "Request accepted by PSP after %ums\n", + jiffies_to_msecs(jiffies - start)); + ret = 0; + goto cleanup; + } else if (status == -EBUSY) { + retry_cnt--; + } else { + ret = -EIO; + goto cleanup; + }; + + /* IF EBUSY, give PSP time to finish its i2c activities */ + mdelay(PSP_I2C_REQ_RETRY_DELAY_MSEC); + } while (retry_cnt); + + + ret = -ETIMEDOUT; + +cleanup: + kfree(req); + return ret; +} + +static int psp_acquire_i2c_bus(void) +{ + int status; + + mutex_lock(&psp_i2c_access_mutex); + + /* Return early if mailbox malfunctioned */ + if (psp_i2c_mbox_fail) + goto cleanup; + + /* + * Simply increment usage counter and return if PSP semaphore was + * already taken by kernel + */ + if (psp_i2c_access_count > 0) { + psp_i2c_access_count++; + goto cleanup; + }; + + status = psp_send_i2c_req(PSP_I2C_REQ_ACQUIRE); + if (!status) { + psp_i2c_sem_acquired = jiffies; + psp_i2c_access_count++; + goto cleanup; + } else if (status == -ETIMEDOUT) { + dev_err(psp_i2c_dev, "Timed out waiting for PSP to release I2C bus\n"); + } else { + dev_err(psp_i2c_dev, "PSP communication error\n"); + }; + + dev_err(psp_i2c_dev, "Assume i2c bus is for exclusive host usage\n"); + psp_i2c_mbox_fail = true; + +cleanup: + mutex_unlock(&psp_i2c_access_mutex); + return 0; +} + +static void psp_release_i2c_bus(void) +{ + int status; + + mutex_lock(&psp_i2c_access_mutex); + + /* Return early if mailbox was malfunctional */ + if (psp_i2c_mbox_fail) + goto cleanup; + + /* + * If we are last owner of PSP semaphore, need to release aribtration + * via mailbox + */ + psp_i2c_access_count--; + if (psp_i2c_access_count > 0) + goto cleanup; + + /* Send a release command to PSP */ + status = psp_send_i2c_req(PSP_I2C_REQ_RELEASE); + if (!status) { + dev_dbg(psp_i2c_dev, "PSP semaphore held for %ums\n", + jiffies_to_msecs(jiffies - psp_i2c_sem_acquired)); + goto cleanup; + } else if (status == -ETIMEDOUT) { + dev_err(psp_i2c_dev, "Timed out waiting for PSP to acquire I2C bus\n"); + } else { + dev_err(psp_i2c_dev, "PSP communication error\n"); + } + + dev_err(psp_i2c_dev, "Assume i2c bus is for exclusive host usage\n"); + psp_i2c_mbox_fail = true; + +cleanup: + mutex_unlock(&psp_i2c_access_mutex); +} + +/* + * Locking methods are based on the default implementation from + * drivers/i2c/i2c-core.base.c, but with psp acquire and release operations + * added. With this in place we can ensure that i2c clients on the bus shared + * with psp are able to lock HW access to the bus for arbitrary number of + * operations - that is e.g. write-wait-read. + */ +static void i2c_adapter_dw_psp_lock_bus(struct i2c_adapter *adapter, + unsigned int flags) +{ + psp_acquire_i2c_bus(); + rt_mutex_lock_nested(&adapter->bus_lock, i2c_adapter_depth(adapter)); +} + +static int i2c_adapter_dw_psp_trylock_bus(struct i2c_adapter *adapter, + unsigned int flags) +{ + int ret; + + ret = rt_mutex_trylock(&adapter->bus_lock); + if (!ret) + psp_acquire_i2c_bus(); + + return ret; +} + +static void i2c_adapter_dw_psp_unlock_bus(struct i2c_adapter *adapter, + unsigned int flags) +{ + psp_release_i2c_bus(); + rt_mutex_unlock(&adapter->bus_lock); +} + +static const struct i2c_lock_operations i2c_dw_psp_lock_ops = { + .lock_bus = i2c_adapter_dw_psp_lock_bus, + .trylock_bus = i2c_adapter_dw_psp_trylock_bus, + .unlock_bus = i2c_adapter_dw_psp_unlock_bus, +}; + +int i2c_dw_amdpsp_probe_lock_support(struct dw_i2c_dev *dev) +{ + if (!dev || !dev->dev) + return -ENODEV; + + if (!(dev->flags & ARBITRATION_SEMAPHORE)) + return -ENODEV; + + /* Allow to bind only one instance of a driver */ + if (!psp_i2c_dev) + psp_i2c_dev = dev->dev; + else + return -EEXIST; + + if (psp_mbox_probe()) + return -EIO; + + dev_info(psp_i2c_dev, "I2C bus managed by AMD PSP\n"); + + /* + * Install global locking callbacks for adapter as well as internal i2c + * controller locks + */ + dev->adapter.lock_ops = &i2c_dw_psp_lock_ops; + dev->acquire_lock = psp_acquire_i2c_bus; + dev->release_lock = psp_release_i2c_bus; + + return 0; +} + +/* Unmap area used as a mailbox with PSP */ +void i2c_dw_amdpsp_remove_lock_support(struct dw_i2c_dev *dev) +{ + iounmap(mbox_iomem); +} diff --git a/drivers/i2c/busses/i2c-designware-baytrail.c b/drivers/i2c/busses/i2c-designware-baytrail.c index c6a7a00e1d52..0c674542dd99 100644 --- a/drivers/i2c/busses/i2c-designware-baytrail.c +++ b/drivers/i2c/busses/i2c-designware-baytrail.c @@ -12,25 +12,25 @@ #include "i2c-designware-core.h" -int i2c_dw_probe_lock_support(struct dw_i2c_dev *dev) +int i2c_dw_baytrail_probe_lock_support(struct dw_i2c_dev *dev) { acpi_status status; unsigned long long shared_host = 0; acpi_handle handle; if (!dev || !dev->dev) - return 0; + return -ENODEV; handle = ACPI_HANDLE(dev->dev); if (!handle) - return 0; + return -ENODEV; status = acpi_evaluate_integer(handle, "_SEM", NULL, &shared_host); if (ACPI_FAILURE(status)) - return 0; + return -ENODEV; if (!shared_host) - return 0; + return -ENODEV; if (!iosf_mbi_available()) return -EPROBE_DEFER; diff --git a/drivers/i2c/busses/i2c-designware-core.h b/drivers/i2c/busses/i2c-designware-core.h index 4b26cba40139..1d65212fddbd 100644 --- a/drivers/i2c/busses/i2c-designware-core.h +++ b/drivers/i2c/busses/i2c-designware-core.h @@ -227,6 +227,8 @@ struct reset_control; * @hs_lcnt: high speed LCNT value * @acquire_lock: function to acquire a hardware lock on the bus * @release_lock: function to release a hardware lock on the bus + * @semaphore_idx: Index of table with semaphore type attached to the bus. It's + * -1 if there is no semaphore. * @shared_with_punit: true if this bus is shared with the SoCs PUNIT * @disable: function to disable the controller * @disable_int: function to disable all interrupts @@ -285,6 +287,7 @@ struct dw_i2c_dev { u16 hs_lcnt; int (*acquire_lock)(void); void (*release_lock)(void); + int semaphore_idx; bool shared_with_punit; void (*disable)(struct dw_i2c_dev *dev); void (*disable_int)(struct dw_i2c_dev *dev); @@ -297,6 +300,7 @@ struct dw_i2c_dev { #define ACCESS_INTR_MASK BIT(0) #define ACCESS_NO_IRQ_SUSPEND BIT(1) +#define ARBITRATION_SEMAPHORE BIT(2) #define MODEL_MSCC_OCELOT BIT(8) #define MODEL_BAIKAL_BT1 BIT(9) @@ -310,6 +314,11 @@ struct dw_i2c_dev { #define AMD_UCSI_INTR_REG 0x474 #define AMD_UCSI_INTR_EN 0xd +struct i2c_dw_semaphore_callbacks { + int (*probe)(struct dw_i2c_dev *dev); + void (*remove)(struct dw_i2c_dev *dev); +}; + int i2c_dw_init_regmap(struct dw_i2c_dev *dev); u32 i2c_dw_scl_hcnt(u32 ic_clk, u32 tSYMBOL, u32 tf, int cond, int offset); u32 i2c_dw_scl_lcnt(u32 ic_clk, u32 tLOW, u32 tf, int offset); @@ -370,9 +379,12 @@ static inline void i2c_dw_configure(struct dw_i2c_dev *dev) } #if IS_ENABLED(CONFIG_I2C_DESIGNWARE_BAYTRAIL) -extern int i2c_dw_probe_lock_support(struct dw_i2c_dev *dev); -#else -static inline int i2c_dw_probe_lock_support(struct dw_i2c_dev *dev) { return 0; } +int i2c_dw_baytrail_probe_lock_support(struct dw_i2c_dev *dev); +#endif + +#if IS_ENABLED(CONFIG_I2C_DESIGNWARE_AMDPSP) +int i2c_dw_amdpsp_probe_lock_support(struct dw_i2c_dev *dev); +void i2c_dw_amdpsp_remove_lock_support(struct dw_i2c_dev *dev); #endif int i2c_dw_validate_speed(struct dw_i2c_dev *dev); diff --git a/drivers/i2c/busses/i2c-designware-platdrv.c b/drivers/i2c/busses/i2c-designware-platdrv.c index 2bd81abc86f6..5844a4df4141 100644 --- a/drivers/i2c/busses/i2c-designware-platdrv.c +++ b/drivers/i2c/busses/i2c-designware-platdrv.c @@ -51,6 +51,7 @@ static const struct acpi_device_id dw_i2c_acpi_match[] = { { "AMD0010", ACCESS_INTR_MASK }, { "AMDI0010", ACCESS_INTR_MASK }, { "AMDI0510", 0 }, + { "AMDI0019", ACCESS_INTR_MASK | ARBITRATION_SEMAPHORE }, { "APMC0D0F", 0 }, { "HISI02A1", 0 }, { "HISI02A2", 0 }, @@ -204,6 +205,64 @@ static const struct dmi_system_id dw_i2c_hwmon_class_dmi[] = { { } /* terminate list */ }; +static const struct i2c_dw_semaphore_callbacks i2c_dw_semaphore_cb_table[] = { +#ifdef CONFIG_I2C_DESIGNWARE_BAYTRAIL + { + .probe = i2c_dw_baytrail_probe_lock_support, + .remove = NULL, + }, +#endif +#ifdef CONFIG_I2C_DESIGNWARE_AMDPSP + { + .probe = i2c_dw_amdpsp_probe_lock_support, + .remove = i2c_dw_amdpsp_remove_lock_support, + }, +#endif + { + .probe = NULL, + .remove = NULL, + }, +}; + +static int i2c_dw_probe_lock_support(struct dw_i2c_dev *dev) +{ + int ret; + int i; + + dev->semaphore_idx = -1; + + for (i = 0; i < ARRAY_SIZE(i2c_dw_semaphore_cb_table); i++) { + if (!i2c_dw_semaphore_cb_table[i].probe) + continue; + + ret = i2c_dw_semaphore_cb_table[i].probe(dev); + if (!ret) { + dev->semaphore_idx = i; + break; + } else if (ret == -ENODEV) { + /* + * If there is no semaphore device attached to this + * controller, we shouldn't abort general i2c_controller + * probe. + */ + continue; + } else { + return ret; + } + } + + return 0; +} + +static void i2c_dw_remove_lock_support(struct dw_i2c_dev *dev) +{ + if (dev->semaphore_idx < 0) + return; + + if (i2c_dw_semaphore_cb_table[dev->semaphore_idx].remove) + i2c_dw_semaphore_cb_table[dev->semaphore_idx].remove(dev); +} + static int dw_i2c_plat_probe(struct platform_device *pdev) { struct i2c_adapter *adap; @@ -334,6 +393,8 @@ static int dw_i2c_plat_remove(struct platform_device *pdev) pm_runtime_put_sync(&pdev->dev); dw_i2c_plat_pm_cleanup(dev); + i2c_dw_remove_lock_support(dev); + reset_control_assert(dev->rst); return 0;
Implement an I2C controller sharing mechanism between the host (kernel) and PSP co-processor on some platforms equipped with AMD Cezanne SoC. On these platforms we need to implement "software" i2c arbitration. Default arbitration owner is PSP and kernel asks for acquire as well as inform about release of the i2c bus via mailbox mechanism. +---------+ <- ACQUIRE | | +---------| CPU |\ | | | \ +----------+ SDA | +---------+ \ | |------- MAILBOX +--> | I2C-DW | SCL | +---------+ | |------- | | | +----------+ +---------| PSP | <- ACK | | +---------+ +---------+ <- RELEASE | | +---------| CPU | | | | +----------+ SDA | +---------+ | |------- MAILBOX +--> | I2C-DW | SCL | +---------+ / | |------- | | | / +----------+ +---------| PSP |/ <- ACK | | +---------+ The solution is similar to i2c-designware-baytrail.c implementation, where we are using a generic i2c-designware-* driver with a small "wrapper". In contrary to baytrail semaphore implementation, beside internal acquire_lock() and release_lock() methods we are also applying quirks to lock_bus() and unlock_bus() global adapter methods. With this in place all i2c clients drivers may lock i2c bus for a desired number of i2c transactions (e.g. write-wait-read) without being aware of that such bus is shared with another entity. Modify i2c_dw_probe_lock_support() to select correct semaphore implementation at runtime, since now we have more than one available. Configure new matching ACPI ID "AMDI0019" and register ARBITRATION_SEMAPHORE flag in order to distinguish setup with PSP arbitration. Add new entry in MAINTAINERS file to cover new module. Signed-off-by: Jan Dabros <jsd@semihalf.com> --- MAINTAINERS | 1 + drivers/acpi/acpi_apd.c | 1 + drivers/i2c/busses/Kconfig | 10 + drivers/i2c/busses/Makefile | 1 + drivers/i2c/busses/i2c-designware-amdpsp.c | 357 +++++++++++++++++++ drivers/i2c/busses/i2c-designware-baytrail.c | 10 +- drivers/i2c/busses/i2c-designware-core.h | 18 +- drivers/i2c/busses/i2c-designware-platdrv.c | 61 ++++ 8 files changed, 451 insertions(+), 8 deletions(-) create mode 100644 drivers/i2c/busses/i2c-designware-amdpsp.c