| Message ID | 1476319476-18500-1-git-send-email-bill.fischofer@linaro.org |
|---|---|
| State | Superseded |
| Headers | show |
On 13 October 2016 at 02:44, Bill Fischofer <bill.fischofer@linaro.org> wrote: > Add the odp_dev_id() API used for NUMA support > I am a bit confused here: what is a device? a numa_id or other things as well? In this patch series everything that relates to numa is called "device". Shouldn't be called numa_dev when it is a numa device? If devices are numa dev only, they should be called numa_dev. If a device can be anything else (which you general approach seems to imply), how are they different from handles? Not sure I understand where these patches lead to... Christophe. > Signed-off-by: Bill Fischofer <bill.fischofer@linaro.org> > --- > include/odp/api/spec/dev.h | 89 ++++++++++++++++++++++++++++++++++++++++++++++ > 1 file changed, 89 insertions(+) > create mode 100644 include/odp/api/spec/dev.h > > diff --git a/include/odp/api/spec/dev.h b/include/odp/api/spec/dev.h > new file mode 100644 > index 0000000..1f7ed8b > --- /dev/null > +++ b/include/odp/api/spec/dev.h > @@ -0,0 +1,89 @@ > +/* Copyright (c) 2016, Linaro Limited > + * All rights reserved. > + * > + * SPDX-License-Identifier: BSD-3-Clause > + */ > + > +/** > + * @file > + * > + * ODP device > + */ > + > +#ifndef ODP_API_DEV_H_ > +#define ODP_API_DEV_H_ > +#include <odp/visibility_begin.h> > + > +#ifdef __cplusplus > +extern "C" { > +#endif > + > +#include <odp/api/std_types.h> > + > +/** @defgroup odp_dev ODP DEVICE > + * Operations on devices > + * @{ > + */ > + > +/** > + * @typedef odp_dev_t > + * ODP Device > + */ > + > +/** > + * @def ODP_DEV_NAME_LEN > + * Maximum device name length in chars > + */ > + > +/** > + * @def ODP_DEV_ANY > + * Any device > + */ > + > +/** > + * @def ODP_DEV_INVALID > + * Invalid device > + */ > + > +/** > + * Get Device ID by Name > + * > + * Get an implementation-defined device identifier from a device name. Device > + * names are supplied as parameter info (command line, file, etc.) to the > + * application. This routine translates this symbolic name into an internal > + * identifier that can be used to define a device connection hierarchy for > + * NUMA or other purposes. > + * > + * The reserved id ODP_DEV_ANY may be used as a "don't care" placeholder > + * wherever a device id is required. > + * > + * @param name Name of the device > + * > + * @return Device ID > + * @retval ODP_DEV_INVALID Device is unknown > + */ > +odp_dev_t odp_dev_id(const char *name); > + > +/** > + * Get printable value for an odp_dev_t > + * > + * @param hdl odp_dev_t handle to be printed > + * @return uint64_t value that can be used to print/display this > + * handle > + * > + * @note This routine is intended to be used for diagnostic purposes > + * to enable applications to generate a printable value that represents > + * an odp_dev_t handle. > + */ > +uint64_t odp_dev_to_u64(odp_dev_t hdl); > + > +/** > + * @} > + */ > + > +#ifdef __cplusplus > +} > +#endif > + > +#include <odp/visibility_end.h> > +#endif > -- > 2.7.4 >
On Thu, Oct 13, 2016 at 2:03 AM, Christophe Milard < christophe.milard@linaro.org> wrote: > On 13 October 2016 at 02:44, Bill Fischofer <bill.fischofer@linaro.org> > wrote: > > Add the odp_dev_id() API used for NUMA support > > > > I am a bit confused here: what is a device? a numa_id or other things > as well? In this patch series everything that relates to numa is > called "device". Shouldn't be called numa_dev when it is a numa > device? > If devices are numa dev only, they should be called numa_dev. If a > device can be anything else (which you general approach seems to > imply), how are they different from handles? > > Not sure I understand where these patches lead to... > These patches are just implementing the APIs proposed by Petri during the ODP Design Summit at LAS16. We can consider them RFCs for now if that's preferable. A dev_id in ODP is supposed to be the same as a socket_id in DPDK, but not necessarily tied to the CPU socket config. The intent is simply to have a placeholder where hierarchical NUMA-type identifiers can be obtained and then used as part of resource (pools, etc.) creation. This is inherently system dependent, which is why the odp-linux versions are mostly placeholders, and why I put the implementations under the arch directory. > > Christophe. > > > Signed-off-by: Bill Fischofer <bill.fischofer@linaro.org> > > --- > > include/odp/api/spec/dev.h | 89 ++++++++++++++++++++++++++++++ > ++++++++++++++++ > > 1 file changed, 89 insertions(+) > > create mode 100644 include/odp/api/spec/dev.h > > > > diff --git a/include/odp/api/spec/dev.h b/include/odp/api/spec/dev.h > > new file mode 100644 > > index 0000000..1f7ed8b > > --- /dev/null > > +++ b/include/odp/api/spec/dev.h > > @@ -0,0 +1,89 @@ > > +/* Copyright (c) 2016, Linaro Limited > > + * All rights reserved. > > + * > > + * SPDX-License-Identifier: BSD-3-Clause > > + */ > > + > > +/** > > + * @file > > + * > > + * ODP device > > + */ > > + > > +#ifndef ODP_API_DEV_H_ > > +#define ODP_API_DEV_H_ > > +#include <odp/visibility_begin.h> > > + > > +#ifdef __cplusplus > > +extern "C" { > > +#endif > > + > > +#include <odp/api/std_types.h> > > + > > +/** @defgroup odp_dev ODP DEVICE > > + * Operations on devices > > + * @{ > > + */ > > + > > +/** > > + * @typedef odp_dev_t > > + * ODP Device > > + */ > > + > > +/** > > + * @def ODP_DEV_NAME_LEN > > + * Maximum device name length in chars > > + */ > > + > > +/** > > + * @def ODP_DEV_ANY > > + * Any device > > + */ > > + > > +/** > > + * @def ODP_DEV_INVALID > > + * Invalid device > > + */ > > + > > +/** > > + * Get Device ID by Name > > + * > > + * Get an implementation-defined device identifier from a device name. > Device > > + * names are supplied as parameter info (command line, file, etc.) to > the > > + * application. This routine translates this symbolic name into an > internal > > + * identifier that can be used to define a device connection hierarchy > for > > + * NUMA or other purposes. > > + * > > + * The reserved id ODP_DEV_ANY may be used as a "don't care" placeholder > > + * wherever a device id is required. > > + * > > + * @param name Name of the device > > + * > > + * @return Device ID > > + * @retval ODP_DEV_INVALID Device is unknown > > + */ > > +odp_dev_t odp_dev_id(const char *name); > > + > > +/** > > + * Get printable value for an odp_dev_t > > + * > > + * @param hdl odp_dev_t handle to be printed > > + * @return uint64_t value that can be used to print/display this > > + * handle > > + * > > + * @note This routine is intended to be used for diagnostic purposes > > + * to enable applications to generate a printable value that represents > > + * an odp_dev_t handle. > > + */ > > +uint64_t odp_dev_to_u64(odp_dev_t hdl); > > + > > +/** > > + * @} > > + */ > > + > > +#ifdef __cplusplus > > +} > > +#endif > > + > > +#include <odp/visibility_end.h> > > +#endif > > -- > > 2.7.4 > > >
On 13 October 2016 at 13:20, Bill Fischofer <bill.fischofer@linaro.org> wrote: > > > On Thu, Oct 13, 2016 at 2:03 AM, Christophe Milard > <christophe.milard@linaro.org> wrote: >> >> On 13 October 2016 at 02:44, Bill Fischofer <bill.fischofer@linaro.org> >> wrote: >> > Add the odp_dev_id() API used for NUMA support >> > >> >> I am a bit confused here: what is a device? a numa_id or other things >> as well? In this patch series everything that relates to numa is >> called "device". Shouldn't be called numa_dev when it is a numa >> device? >> If devices are numa dev only, they should be called numa_dev. If a >> device can be anything else (which you general approach seems to >> imply), how are they different from handles? >> >> Not sure I understand where these patches lead to... > > > These patches are just implementing the APIs proposed by Petri during the > ODP Design Summit at LAS16. We can consider them RFCs for now if that's > preferable. A dev_id in ODP is supposed to be the same as a socket_id in > DPDK, but not necessarily tied to the CPU socket config. The intent is > simply to have a placeholder where hierarchical NUMA-type identifiers can be > obtained and then used as part of resource (pools, etc.) creation. This is > inherently system dependent, which is why the odp-linux versions are mostly > placeholders, and why I put the implementations under the arch directory. > But still: is this for numa only (in which case I would expect a clearer name) or are these devices meant to be used by other things (which ones?). And how does this differ from the handles we already have for other objects? Christophe >> >> >> Christophe. >> >> > Signed-off-by: Bill Fischofer <bill.fischofer@linaro.org> >> > --- >> > include/odp/api/spec/dev.h | 89 >> > ++++++++++++++++++++++++++++++++++++++++++++++ >> > 1 file changed, 89 insertions(+) >> > create mode 100644 include/odp/api/spec/dev.h >> > >> > diff --git a/include/odp/api/spec/dev.h b/include/odp/api/spec/dev.h >> > new file mode 100644 >> > index 0000000..1f7ed8b >> > --- /dev/null >> > +++ b/include/odp/api/spec/dev.h >> > @@ -0,0 +1,89 @@ >> > +/* Copyright (c) 2016, Linaro Limited >> > + * All rights reserved. >> > + * >> > + * SPDX-License-Identifier: BSD-3-Clause >> > + */ >> > + >> > +/** >> > + * @file >> > + * >> > + * ODP device >> > + */ >> > + >> > +#ifndef ODP_API_DEV_H_ >> > +#define ODP_API_DEV_H_ >> > +#include <odp/visibility_begin.h> >> > + >> > +#ifdef __cplusplus >> > +extern "C" { >> > +#endif >> > + >> > +#include <odp/api/std_types.h> >> > + >> > +/** @defgroup odp_dev ODP DEVICE >> > + * Operations on devices >> > + * @{ >> > + */ >> > + >> > +/** >> > + * @typedef odp_dev_t >> > + * ODP Device >> > + */ >> > + >> > +/** >> > + * @def ODP_DEV_NAME_LEN >> > + * Maximum device name length in chars >> > + */ >> > + >> > +/** >> > + * @def ODP_DEV_ANY >> > + * Any device >> > + */ >> > + >> > +/** >> > + * @def ODP_DEV_INVALID >> > + * Invalid device >> > + */ >> > + >> > +/** >> > + * Get Device ID by Name >> > + * >> > + * Get an implementation-defined device identifier from a device name. >> > Device >> > + * names are supplied as parameter info (command line, file, etc.) to >> > the >> > + * application. This routine translates this symbolic name into an >> > internal >> > + * identifier that can be used to define a device connection hierarchy >> > for >> > + * NUMA or other purposes. >> > + * >> > + * The reserved id ODP_DEV_ANY may be used as a "don't care" >> > placeholder >> > + * wherever a device id is required. >> > + * >> > + * @param name Name of the device >> > + * >> > + * @return Device ID >> > + * @retval ODP_DEV_INVALID Device is unknown >> > + */ >> > +odp_dev_t odp_dev_id(const char *name); >> > + >> > +/** >> > + * Get printable value for an odp_dev_t >> > + * >> > + * @param hdl odp_dev_t handle to be printed >> > + * @return uint64_t value that can be used to print/display this >> > + * handle >> > + * >> > + * @note This routine is intended to be used for diagnostic purposes >> > + * to enable applications to generate a printable value that represents >> > + * an odp_dev_t handle. >> > + */ >> > +uint64_t odp_dev_to_u64(odp_dev_t hdl); >> > + >> > +/** >> > + * @} >> > + */ >> > + >> > +#ifdef __cplusplus >> > +} >> > +#endif >> > + >> > +#include <odp/visibility_end.h> >> > +#endif >> > -- >> > 2.7.4 >> > > >
On Thu, Oct 13, 2016 at 6:33 AM, Christophe Milard < christophe.milard@linaro.org> wrote: > On 13 October 2016 at 13:20, Bill Fischofer <bill.fischofer@linaro.org> > wrote: > > > > > > On Thu, Oct 13, 2016 at 2:03 AM, Christophe Milard > > <christophe.milard@linaro.org> wrote: > >> > >> On 13 October 2016 at 02:44, Bill Fischofer <bill.fischofer@linaro.org> > >> wrote: > >> > Add the odp_dev_id() API used for NUMA support > >> > > >> > >> I am a bit confused here: what is a device? a numa_id or other things > >> as well? In this patch series everything that relates to numa is > >> called "device". Shouldn't be called numa_dev when it is a numa > >> device? > >> If devices are numa dev only, they should be called numa_dev. If a > >> device can be anything else (which you general approach seems to > >> imply), how are they different from handles? > >> > >> Not sure I understand where these patches lead to... > > > > > > These patches are just implementing the APIs proposed by Petri during the > > ODP Design Summit at LAS16. We can consider them RFCs for now if that's > > preferable. A dev_id in ODP is supposed to be the same as a socket_id in > > DPDK, but not necessarily tied to the CPU socket config. The intent is > > simply to have a placeholder where hierarchical NUMA-type identifiers > can be > > obtained and then used as part of resource (pools, etc.) creation. This > is > > inherently system dependent, which is why the odp-linux versions are > mostly > > placeholders, and why I put the implementations under the arch directory. > > > > But still: is this for numa only (in which case I would expect a > clearer name) or are these devices meant to be used by other things > (which ones?). And how does this differ from the handles we already > have for other objects? > It's not necessarily NUMA since the intent is to be able to cover multi-SoC configurations as well. A dev_id differs from a handle because it's a qualifier. So, for example, an odp_pool_t is the ODP handle for a pool, however the pool may have a couple of dev_id qualifiers that are used as part of it's creation (Petri identified pool_id and dram_id as two). Similarly a crypto_session uses a dev_id to identify a specific crypto resource bound to that session. For example a system might have four crypto engines that have different "distance" depending on where the thread is running and the dev_id would distinguish those. > > Christophe > >> > >> > >> Christophe. > >> > >> > Signed-off-by: Bill Fischofer <bill.fischofer@linaro.org> > >> > --- > >> > include/odp/api/spec/dev.h | 89 > >> > ++++++++++++++++++++++++++++++++++++++++++++++ > >> > 1 file changed, 89 insertions(+) > >> > create mode 100644 include/odp/api/spec/dev.h > >> > > >> > diff --git a/include/odp/api/spec/dev.h b/include/odp/api/spec/dev.h > >> > new file mode 100644 > >> > index 0000000..1f7ed8b > >> > --- /dev/null > >> > +++ b/include/odp/api/spec/dev.h > >> > @@ -0,0 +1,89 @@ > >> > +/* Copyright (c) 2016, Linaro Limited > >> > + * All rights reserved. > >> > + * > >> > + * SPDX-License-Identifier: BSD-3-Clause > >> > + */ > >> > + > >> > +/** > >> > + * @file > >> > + * > >> > + * ODP device > >> > + */ > >> > + > >> > +#ifndef ODP_API_DEV_H_ > >> > +#define ODP_API_DEV_H_ > >> > +#include <odp/visibility_begin.h> > >> > + > >> > +#ifdef __cplusplus > >> > +extern "C" { > >> > +#endif > >> > + > >> > +#include <odp/api/std_types.h> > >> > + > >> > +/** @defgroup odp_dev ODP DEVICE > >> > + * Operations on devices > >> > + * @{ > >> > + */ > >> > + > >> > +/** > >> > + * @typedef odp_dev_t > >> > + * ODP Device > >> > + */ > >> > + > >> > +/** > >> > + * @def ODP_DEV_NAME_LEN > >> > + * Maximum device name length in chars > >> > + */ > >> > + > >> > +/** > >> > + * @def ODP_DEV_ANY > >> > + * Any device > >> > + */ > >> > + > >> > +/** > >> > + * @def ODP_DEV_INVALID > >> > + * Invalid device > >> > + */ > >> > + > >> > +/** > >> > + * Get Device ID by Name > >> > + * > >> > + * Get an implementation-defined device identifier from a device > name. > >> > Device > >> > + * names are supplied as parameter info (command line, file, etc.) to > >> > the > >> > + * application. This routine translates this symbolic name into an > >> > internal > >> > + * identifier that can be used to define a device connection > hierarchy > >> > for > >> > + * NUMA or other purposes. > >> > + * > >> > + * The reserved id ODP_DEV_ANY may be used as a "don't care" > >> > placeholder > >> > + * wherever a device id is required. > >> > + * > >> > + * @param name Name of the device > >> > + * > >> > + * @return Device ID > >> > + * @retval ODP_DEV_INVALID Device is unknown > >> > + */ > >> > +odp_dev_t odp_dev_id(const char *name); > >> > + > >> > +/** > >> > + * Get printable value for an odp_dev_t > >> > + * > >> > + * @param hdl odp_dev_t handle to be printed > >> > + * @return uint64_t value that can be used to print/display this > >> > + * handle > >> > + * > >> > + * @note This routine is intended to be used for diagnostic purposes > >> > + * to enable applications to generate a printable value that > represents > >> > + * an odp_dev_t handle. > >> > + */ > >> > +uint64_t odp_dev_to_u64(odp_dev_t hdl); > >> > + > >> > +/** > >> > + * @} > >> > + */ > >> > + > >> > +#ifdef __cplusplus > >> > +} > >> > +#endif > >> > + > >> > +#include <odp/visibility_end.h> > >> > +#endif > >> > -- > >> > 2.7.4 > >> > > > > > >
On 13 October 2016 at 13:44, Bill Fischofer <bill.fischofer@linaro.org> wrote: > > > On Thu, Oct 13, 2016 at 6:33 AM, Christophe Milard > <christophe.milard@linaro.org> wrote: >> >> On 13 October 2016 at 13:20, Bill Fischofer <bill.fischofer@linaro.org> >> wrote: >> > >> > >> > On Thu, Oct 13, 2016 at 2:03 AM, Christophe Milard >> > <christophe.milard@linaro.org> wrote: >> >> >> >> On 13 October 2016 at 02:44, Bill Fischofer <bill.fischofer@linaro.org> >> >> wrote: >> >> > Add the odp_dev_id() API used for NUMA support >> >> > >> >> >> >> I am a bit confused here: what is a device? a numa_id or other things >> >> as well? In this patch series everything that relates to numa is >> >> called "device". Shouldn't be called numa_dev when it is a numa >> >> device? >> >> If devices are numa dev only, they should be called numa_dev. If a >> >> device can be anything else (which you general approach seems to >> >> imply), how are they different from handles? >> >> >> >> Not sure I understand where these patches lead to... >> > >> > >> > These patches are just implementing the APIs proposed by Petri during >> > the >> > ODP Design Summit at LAS16. We can consider them RFCs for now if that's >> > preferable. A dev_id in ODP is supposed to be the same as a socket_id in >> > DPDK, but not necessarily tied to the CPU socket config. The intent is >> > simply to have a placeholder where hierarchical NUMA-type identifiers >> > can be >> > obtained and then used as part of resource (pools, etc.) creation. This >> > is >> > inherently system dependent, which is why the odp-linux versions are >> > mostly >> > placeholders, and why I put the implementations under the arch >> > directory. >> > >> >> But still: is this for numa only (in which case I would expect a >> clearer name) or are these devices meant to be used by other things >> (which ones?). And how does this differ from the handles we already >> have for other objects? > > > It's not necessarily NUMA since the intent is to be able to cover multi-SoC > configurations as well. A dev_id differs from a handle because it's a > qualifier. So, for example, an odp_pool_t is the ODP handle for a pool, > however the pool may have a couple of dev_id qualifiers that are used as > part of it's creation (Petri identified pool_id and dram_id as two). > Similarly a crypto_session uses a dev_id to identify a specific crypto > resource bound to that session. For example a system might have four crypto > engines that have different "distance" depending on where the thread is > running and the dev_id would distinguish those. hmmm interesting... not very clear to me what would be the difference between a handle and a dev. If handles can be references to anything, I don't really see why we wouldn't keep using handles for these things. And I am not sure either having a name as "dev" to cover everything makes it clear. Thanks for answering anyway :-) maybe it will become clearer in the future. > >> >> >> Christophe >> >> >> >> >> >> Christophe. >> >> >> >> > Signed-off-by: Bill Fischofer <bill.fischofer@linaro.org> >> >> > --- >> >> > include/odp/api/spec/dev.h | 89 >> >> > ++++++++++++++++++++++++++++++++++++++++++++++ >> >> > 1 file changed, 89 insertions(+) >> >> > create mode 100644 include/odp/api/spec/dev.h >> >> > >> >> > diff --git a/include/odp/api/spec/dev.h b/include/odp/api/spec/dev.h >> >> > new file mode 100644 >> >> > index 0000000..1f7ed8b >> >> > --- /dev/null >> >> > +++ b/include/odp/api/spec/dev.h >> >> > @@ -0,0 +1,89 @@ >> >> > +/* Copyright (c) 2016, Linaro Limited >> >> > + * All rights reserved. >> >> > + * >> >> > + * SPDX-License-Identifier: BSD-3-Clause >> >> > + */ >> >> > + >> >> > +/** >> >> > + * @file >> >> > + * >> >> > + * ODP device >> >> > + */ >> >> > + >> >> > +#ifndef ODP_API_DEV_H_ >> >> > +#define ODP_API_DEV_H_ >> >> > +#include <odp/visibility_begin.h> >> >> > + >> >> > +#ifdef __cplusplus >> >> > +extern "C" { >> >> > +#endif >> >> > + >> >> > +#include <odp/api/std_types.h> >> >> > + >> >> > +/** @defgroup odp_dev ODP DEVICE >> >> > + * Operations on devices >> >> > + * @{ >> >> > + */ >> >> > + >> >> > +/** >> >> > + * @typedef odp_dev_t >> >> > + * ODP Device >> >> > + */ >> >> > + >> >> > +/** >> >> > + * @def ODP_DEV_NAME_LEN >> >> > + * Maximum device name length in chars >> >> > + */ >> >> > + >> >> > +/** >> >> > + * @def ODP_DEV_ANY >> >> > + * Any device >> >> > + */ >> >> > + >> >> > +/** >> >> > + * @def ODP_DEV_INVALID >> >> > + * Invalid device >> >> > + */ >> >> > + >> >> > +/** >> >> > + * Get Device ID by Name >> >> > + * >> >> > + * Get an implementation-defined device identifier from a device >> >> > name. >> >> > Device >> >> > + * names are supplied as parameter info (command line, file, etc.) >> >> > to >> >> > the >> >> > + * application. This routine translates this symbolic name into an >> >> > internal >> >> > + * identifier that can be used to define a device connection >> >> > hierarchy >> >> > for >> >> > + * NUMA or other purposes. >> >> > + * >> >> > + * The reserved id ODP_DEV_ANY may be used as a "don't care" >> >> > placeholder >> >> > + * wherever a device id is required. >> >> > + * >> >> > + * @param name Name of the device >> >> > + * >> >> > + * @return Device ID >> >> > + * @retval ODP_DEV_INVALID Device is unknown >> >> > + */ >> >> > +odp_dev_t odp_dev_id(const char *name); >> >> > + >> >> > +/** >> >> > + * Get printable value for an odp_dev_t >> >> > + * >> >> > + * @param hdl odp_dev_t handle to be printed >> >> > + * @return uint64_t value that can be used to print/display this >> >> > + * handle >> >> > + * >> >> > + * @note This routine is intended to be used for diagnostic purposes >> >> > + * to enable applications to generate a printable value that >> >> > represents >> >> > + * an odp_dev_t handle. >> >> > + */ >> >> > +uint64_t odp_dev_to_u64(odp_dev_t hdl); >> >> > + >> >> > +/** >> >> > + * @} >> >> > + */ >> >> > + >> >> > +#ifdef __cplusplus >> >> > +} >> >> > +#endif >> >> > + >> >> > +#include <odp/visibility_end.h> >> >> > +#endif >> >> > -- >> >> > 2.7.4 >> >> > >> > >> > > >
On Thu, Oct 13, 2016 at 6:53 AM, Christophe Milard < christophe.milard@linaro.org> wrote: > On 13 October 2016 at 13:44, Bill Fischofer <bill.fischofer@linaro.org> > wrote: > > > > > > On Thu, Oct 13, 2016 at 6:33 AM, Christophe Milard > > <christophe.milard@linaro.org> wrote: > >> > >> On 13 October 2016 at 13:20, Bill Fischofer <bill.fischofer@linaro.org> > >> wrote: > >> > > >> > > >> > On Thu, Oct 13, 2016 at 2:03 AM, Christophe Milard > >> > <christophe.milard@linaro.org> wrote: > >> >> > >> >> On 13 October 2016 at 02:44, Bill Fischofer < > bill.fischofer@linaro.org> > >> >> wrote: > >> >> > Add the odp_dev_id() API used for NUMA support > >> >> > > >> >> > >> >> I am a bit confused here: what is a device? a numa_id or other things > >> >> as well? In this patch series everything that relates to numa is > >> >> called "device". Shouldn't be called numa_dev when it is a numa > >> >> device? > >> >> If devices are numa dev only, they should be called numa_dev. If a > >> >> device can be anything else (which you general approach seems to > >> >> imply), how are they different from handles? > >> >> > >> >> Not sure I understand where these patches lead to... > >> > > >> > > >> > These patches are just implementing the APIs proposed by Petri during > >> > the > >> > ODP Design Summit at LAS16. We can consider them RFCs for now if > that's > >> > preferable. A dev_id in ODP is supposed to be the same as a socket_id > in > >> > DPDK, but not necessarily tied to the CPU socket config. The intent is > >> > simply to have a placeholder where hierarchical NUMA-type identifiers > >> > can be > >> > obtained and then used as part of resource (pools, etc.) creation. > This > >> > is > >> > inherently system dependent, which is why the odp-linux versions are > >> > mostly > >> > placeholders, and why I put the implementations under the arch > >> > directory. > >> > > >> > >> But still: is this for numa only (in which case I would expect a > >> clearer name) or are these devices meant to be used by other things > >> (which ones?). And how does this differ from the handles we already > >> have for other objects? > > > > > > It's not necessarily NUMA since the intent is to be able to cover > multi-SoC > > configurations as well. A dev_id differs from a handle because it's a > > qualifier. So, for example, an odp_pool_t is the ODP handle for a pool, > > however the pool may have a couple of dev_id qualifiers that are used as > > part of it's creation (Petri identified pool_id and dram_id as two). > > Similarly a crypto_session uses a dev_id to identify a specific crypto > > resource bound to that session. For example a system might have four > crypto > > engines that have different "distance" depending on where the thread is > > running and the dev_id would distinguish those. > > hmmm interesting... not very clear to me what would be the difference > between a handle and a dev. If handles can be references to anything, > I don't really see why we wouldn't keep using handles for these > things. And I am not sure either having a name as "dev" to cover > everything makes it clear. > Thanks for answering anyway :-) maybe it will become clearer in the future. > > I'll add this to Monday's ARCH call so Petri can explain it in more detail :) > > > >> > >> > >> Christophe > >> >> > >> >> > >> >> Christophe. > >> >> > >> >> > Signed-off-by: Bill Fischofer <bill.fischofer@linaro.org> > >> >> > --- > >> >> > include/odp/api/spec/dev.h | 89 > >> >> > ++++++++++++++++++++++++++++++++++++++++++++++ > >> >> > 1 file changed, 89 insertions(+) > >> >> > create mode 100644 include/odp/api/spec/dev.h > >> >> > > >> >> > diff --git a/include/odp/api/spec/dev.h > b/include/odp/api/spec/dev.h > >> >> > new file mode 100644 > >> >> > index 0000000..1f7ed8b > >> >> > --- /dev/null > >> >> > +++ b/include/odp/api/spec/dev.h > >> >> > @@ -0,0 +1,89 @@ > >> >> > +/* Copyright (c) 2016, Linaro Limited > >> >> > + * All rights reserved. > >> >> > + * > >> >> > + * SPDX-License-Identifier: BSD-3-Clause > >> >> > + */ > >> >> > + > >> >> > +/** > >> >> > + * @file > >> >> > + * > >> >> > + * ODP device > >> >> > + */ > >> >> > + > >> >> > +#ifndef ODP_API_DEV_H_ > >> >> > +#define ODP_API_DEV_H_ > >> >> > +#include <odp/visibility_begin.h> > >> >> > + > >> >> > +#ifdef __cplusplus > >> >> > +extern "C" { > >> >> > +#endif > >> >> > + > >> >> > +#include <odp/api/std_types.h> > >> >> > + > >> >> > +/** @defgroup odp_dev ODP DEVICE > >> >> > + * Operations on devices > >> >> > + * @{ > >> >> > + */ > >> >> > + > >> >> > +/** > >> >> > + * @typedef odp_dev_t > >> >> > + * ODP Device > >> >> > + */ > >> >> > + > >> >> > +/** > >> >> > + * @def ODP_DEV_NAME_LEN > >> >> > + * Maximum device name length in chars > >> >> > + */ > >> >> > + > >> >> > +/** > >> >> > + * @def ODP_DEV_ANY > >> >> > + * Any device > >> >> > + */ > >> >> > + > >> >> > +/** > >> >> > + * @def ODP_DEV_INVALID > >> >> > + * Invalid device > >> >> > + */ > >> >> > + > >> >> > +/** > >> >> > + * Get Device ID by Name > >> >> > + * > >> >> > + * Get an implementation-defined device identifier from a device > >> >> > name. > >> >> > Device > >> >> > + * names are supplied as parameter info (command line, file, etc.) > >> >> > to > >> >> > the > >> >> > + * application. This routine translates this symbolic name into an > >> >> > internal > >> >> > + * identifier that can be used to define a device connection > >> >> > hierarchy > >> >> > for > >> >> > + * NUMA or other purposes. > >> >> > + * > >> >> > + * The reserved id ODP_DEV_ANY may be used as a "don't care" > >> >> > placeholder > >> >> > + * wherever a device id is required. > >> >> > + * > >> >> > + * @param name Name of the device > >> >> > + * > >> >> > + * @return Device ID > >> >> > + * @retval ODP_DEV_INVALID Device is unknown > >> >> > + */ > >> >> > +odp_dev_t odp_dev_id(const char *name); > >> >> > + > >> >> > +/** > >> >> > + * Get printable value for an odp_dev_t > >> >> > + * > >> >> > + * @param hdl odp_dev_t handle to be printed > >> >> > + * @return uint64_t value that can be used to print/display > this > >> >> > + * handle > >> >> > + * > >> >> > + * @note This routine is intended to be used for diagnostic > purposes > >> >> > + * to enable applications to generate a printable value that > >> >> > represents > >> >> > + * an odp_dev_t handle. > >> >> > + */ > >> >> > +uint64_t odp_dev_to_u64(odp_dev_t hdl); > >> >> > + > >> >> > +/** > >> >> > + * @} > >> >> > + */ > >> >> > + > >> >> > +#ifdef __cplusplus > >> >> > +} > >> >> > +#endif > >> >> > + > >> >> > +#include <odp/visibility_end.h> > >> >> > +#endif > >> >> > -- > >> >> > 2.7.4 > >> >> > > >> > > >> > > > > > >
On 13 October 2016 at 14:06, Bill Fischofer <bill.fischofer@linaro.org> wrote: > > > On Thu, Oct 13, 2016 at 6:53 AM, Christophe Milard > <christophe.milard@linaro.org> wrote: >> >> On 13 October 2016 at 13:44, Bill Fischofer <bill.fischofer@linaro.org> >> wrote: >> > >> > >> > On Thu, Oct 13, 2016 at 6:33 AM, Christophe Milard >> > <christophe.milard@linaro.org> wrote: >> >> >> >> On 13 October 2016 at 13:20, Bill Fischofer <bill.fischofer@linaro.org> >> >> wrote: >> >> > >> >> > >> >> > On Thu, Oct 13, 2016 at 2:03 AM, Christophe Milard >> >> > <christophe.milard@linaro.org> wrote: >> >> >> >> >> >> On 13 October 2016 at 02:44, Bill Fischofer >> >> >> <bill.fischofer@linaro.org> >> >> >> wrote: >> >> >> > Add the odp_dev_id() API used for NUMA support >> >> >> > >> >> >> >> >> >> I am a bit confused here: what is a device? a numa_id or other >> >> >> things >> >> >> as well? In this patch series everything that relates to numa is >> >> >> called "device". Shouldn't be called numa_dev when it is a numa >> >> >> device? >> >> >> If devices are numa dev only, they should be called numa_dev. If a >> >> >> device can be anything else (which you general approach seems to >> >> >> imply), how are they different from handles? >> >> >> >> >> >> Not sure I understand where these patches lead to... >> >> > >> >> > >> >> > These patches are just implementing the APIs proposed by Petri during >> >> > the >> >> > ODP Design Summit at LAS16. We can consider them RFCs for now if >> >> > that's >> >> > preferable. A dev_id in ODP is supposed to be the same as a socket_id >> >> > in >> >> > DPDK, but not necessarily tied to the CPU socket config. The intent >> >> > is >> >> > simply to have a placeholder where hierarchical NUMA-type identifiers >> >> > can be >> >> > obtained and then used as part of resource (pools, etc.) creation. >> >> > This >> >> > is >> >> > inherently system dependent, which is why the odp-linux versions are >> >> > mostly >> >> > placeholders, and why I put the implementations under the arch >> >> > directory. >> >> > >> >> >> >> But still: is this for numa only (in which case I would expect a >> >> clearer name) or are these devices meant to be used by other things >> >> (which ones?). And how does this differ from the handles we already >> >> have for other objects? >> > >> > >> > It's not necessarily NUMA since the intent is to be able to cover >> > multi-SoC >> > configurations as well. A dev_id differs from a handle because it's a >> > qualifier. So, for example, an odp_pool_t is the ODP handle for a pool, >> > however the pool may have a couple of dev_id qualifiers that are used as >> > part of it's creation (Petri identified pool_id and dram_id as two). >> > Similarly a crypto_session uses a dev_id to identify a specific crypto >> > resource bound to that session. For example a system might have four >> > crypto >> > engines that have different "distance" depending on where the thread is >> > running and the dev_id would distinguish those. >> >> hmmm interesting... not very clear to me what would be the difference >> between a handle and a dev. If handles can be references to anything, >> I don't really see why we wouldn't keep using handles for these >> things. And I am not sure either having a name as "dev" to cover >> everything makes it clear. >> Thanks for answering anyway :-) maybe it will become clearer in the >> future. >> > I'll add this to Monday's ARCH call so Petri can explain it in more detail > :) Thanks! Christophe > > >> >> > >> >> >> >> >> >> Christophe >> >> >> >> >> >> >> >> >> Christophe. >> >> >> >> >> >> > Signed-off-by: Bill Fischofer <bill.fischofer@linaro.org> >> >> >> > --- >> >> >> > include/odp/api/spec/dev.h | 89 >> >> >> > ++++++++++++++++++++++++++++++++++++++++++++++ >> >> >> > 1 file changed, 89 insertions(+) >> >> >> > create mode 100644 include/odp/api/spec/dev.h >> >> >> > >> >> >> > diff --git a/include/odp/api/spec/dev.h >> >> >> > b/include/odp/api/spec/dev.h >> >> >> > new file mode 100644 >> >> >> > index 0000000..1f7ed8b >> >> >> > --- /dev/null >> >> >> > +++ b/include/odp/api/spec/dev.h >> >> >> > @@ -0,0 +1,89 @@ >> >> >> > +/* Copyright (c) 2016, Linaro Limited >> >> >> > + * All rights reserved. >> >> >> > + * >> >> >> > + * SPDX-License-Identifier: BSD-3-Clause >> >> >> > + */ >> >> >> > + >> >> >> > +/** >> >> >> > + * @file >> >> >> > + * >> >> >> > + * ODP device >> >> >> > + */ >> >> >> > + >> >> >> > +#ifndef ODP_API_DEV_H_ >> >> >> > +#define ODP_API_DEV_H_ >> >> >> > +#include <odp/visibility_begin.h> >> >> >> > + >> >> >> > +#ifdef __cplusplus >> >> >> > +extern "C" { >> >> >> > +#endif >> >> >> > + >> >> >> > +#include <odp/api/std_types.h> >> >> >> > + >> >> >> > +/** @defgroup odp_dev ODP DEVICE >> >> >> > + * Operations on devices >> >> >> > + * @{ >> >> >> > + */ >> >> >> > + >> >> >> > +/** >> >> >> > + * @typedef odp_dev_t >> >> >> > + * ODP Device >> >> >> > + */ >> >> >> > + >> >> >> > +/** >> >> >> > + * @def ODP_DEV_NAME_LEN >> >> >> > + * Maximum device name length in chars >> >> >> > + */ >> >> >> > + >> >> >> > +/** >> >> >> > + * @def ODP_DEV_ANY >> >> >> > + * Any device >> >> >> > + */ >> >> >> > + >> >> >> > +/** >> >> >> > + * @def ODP_DEV_INVALID >> >> >> > + * Invalid device >> >> >> > + */ >> >> >> > + >> >> >> > +/** >> >> >> > + * Get Device ID by Name >> >> >> > + * >> >> >> > + * Get an implementation-defined device identifier from a device >> >> >> > name. >> >> >> > Device >> >> >> > + * names are supplied as parameter info (command line, file, >> >> >> > etc.) >> >> >> > to >> >> >> > the >> >> >> > + * application. This routine translates this symbolic name into >> >> >> > an >> >> >> > internal >> >> >> > + * identifier that can be used to define a device connection >> >> >> > hierarchy >> >> >> > for >> >> >> > + * NUMA or other purposes. >> >> >> > + * >> >> >> > + * The reserved id ODP_DEV_ANY may be used as a "don't care" >> >> >> > placeholder >> >> >> > + * wherever a device id is required. >> >> >> > + * >> >> >> > + * @param name Name of the device >> >> >> > + * >> >> >> > + * @return Device ID >> >> >> > + * @retval ODP_DEV_INVALID Device is unknown >> >> >> > + */ >> >> >> > +odp_dev_t odp_dev_id(const char *name); >> >> >> > + >> >> >> > +/** >> >> >> > + * Get printable value for an odp_dev_t >> >> >> > + * >> >> >> > + * @param hdl odp_dev_t handle to be printed >> >> >> > + * @return uint64_t value that can be used to print/display >> >> >> > this >> >> >> > + * handle >> >> >> > + * >> >> >> > + * @note This routine is intended to be used for diagnostic >> >> >> > purposes >> >> >> > + * to enable applications to generate a printable value that >> >> >> > represents >> >> >> > + * an odp_dev_t handle. >> >> >> > + */ >> >> >> > +uint64_t odp_dev_to_u64(odp_dev_t hdl); >> >> >> > + >> >> >> > +/** >> >> >> > + * @} >> >> >> > + */ >> >> >> > + >> >> >> > +#ifdef __cplusplus >> >> >> > +} >> >> >> > +#endif >> >> >> > + >> >> >> > +#include <odp/visibility_end.h> >> >> >> > +#endif >> >> >> > -- >> >> >> > 2.7.4 >> >> >> > >> >> > >> >> > >> > >> > > >
On 10/13/16 03:44, Bill Fischofer wrote: > + > +#ifndef ODP_API_DEV_H_ > +#define ODP_API_DEV_H_ > +#include <odp/visibility_begin.h> > + > +#ifdef __cplusplus > +extern "C" { > +#endif > + > +#include <odp/api/std_types.h> > + > +/** @defgroup odp_dev ODP DEVICE > + * Operations on devices > + * @{ > + */ > + > +/** > + * @typedef odp_dev_t > + * ODP Device > + */ > + > +/** > + * @def ODP_DEV_NAME_LEN > + * Maximum device name length in chars > + */ > + > +/** > + * @def ODP_DEV_ANY > + * Any device > + */ > + > +/** > + * @def ODP_DEV_INVALID > + * Invalid device > + */ By defining INVALID and ANY will we miss SOME? Maybe on memory allocation it makes sense to provide mask of possible numa nodes and implementation will choose what is the best? Maxim.
On Mon, Oct 17, 2016 at 8:59 AM, Maxim Uvarov <maxim.uvarov@linaro.org> wrote: > On 10/13/16 03:44, Bill Fischofer wrote: > >> + >> +#ifndef ODP_API_DEV_H_ >> +#define ODP_API_DEV_H_ >> +#include <odp/visibility_begin.h> >> + >> +#ifdef __cplusplus >> +extern "C" { >> +#endif >> + >> +#include <odp/api/std_types.h> >> + >> +/** @defgroup odp_dev ODP DEVICE >> + * Operations on devices >> + * @{ >> + */ >> + >> +/** >> + * @typedef odp_dev_t >> + * ODP Device >> + */ >> + >> +/** >> + * @def ODP_DEV_NAME_LEN >> + * Maximum device name length in chars >> + */ >> + >> +/** >> + * @def ODP_DEV_ANY >> + * Any device >> + */ >> + >> +/** >> + * @def ODP_DEV_INVALID >> + * Invalid device >> + */ >> > > By defining INVALID and ANY will we miss SOME? Maybe on memory allocation > it makes sense to provide mask of possible numa nodes and > implementation will choose what is the best? These two special symbols seem needed. What is the use case for ODP_DEV_SOME and how would that have a platform-independent meaning? > > > Maxim. >
On Mon, Oct 17, 2016 at 9:21 AM, Maxim Uvarov <maxim.uvarov@linaro.org> wrote: > by 'some' I mean mask of numa nodes: > > odp_numa_mask_t mask; > > odp_pool_create(.., param.mask = 0x5); > > where 0x5 will say that numa node 0 and numa node 2 available for > allocation, but not numa node 1. > Given that an odp_dev_t is abstract, That could be encoded in the string that is supplied to odp_dev_id(), especially since this is intentionally platform-dependent. Currently there is no application-visible structure in this. I'm not sure we want to introduce one, as masks would imply. > > > Maxim. > > > On 17 October 2016 at 17:18, Bill Fischofer <bill.fischofer@linaro.org> > wrote: > >> >> >> On Mon, Oct 17, 2016 at 8:59 AM, Maxim Uvarov <maxim.uvarov@linaro.org> >> wrote: >> >>> On 10/13/16 03:44, Bill Fischofer wrote: >>> >>>> + >>>> +#ifndef ODP_API_DEV_H_ >>>> +#define ODP_API_DEV_H_ >>>> +#include <odp/visibility_begin.h> >>>> + >>>> +#ifdef __cplusplus >>>> +extern "C" { >>>> +#endif >>>> + >>>> +#include <odp/api/std_types.h> >>>> + >>>> +/** @defgroup odp_dev ODP DEVICE >>>> + * Operations on devices >>>> + * @{ >>>> + */ >>>> + >>>> +/** >>>> + * @typedef odp_dev_t >>>> + * ODP Device >>>> + */ >>>> + >>>> +/** >>>> + * @def ODP_DEV_NAME_LEN >>>> + * Maximum device name length in chars >>>> + */ >>>> + >>>> +/** >>>> + * @def ODP_DEV_ANY >>>> + * Any device >>>> + */ >>>> + >>>> +/** >>>> + * @def ODP_DEV_INVALID >>>> + * Invalid device >>>> + */ >>>> >>> >>> By defining INVALID and ANY will we miss SOME? Maybe on memory >>> allocation it makes sense to provide mask of possible numa nodes and >>> implementation will choose what is the best? >> >> >> These two special symbols seem needed. What is the use case for >> ODP_DEV_SOME and how would that have a platform-independent meaning? >> >> >>> >>> >>> Maxim. >>> >> >> >
> -----Original Message----- > From: lng-odp [mailto:lng-odp-bounces@lists.linaro.org] On Behalf Of Bill > Fischofer > Sent: Thursday, October 13, 2016 3:45 AM > To: lng-odp@lists.linaro.org > Subject: [lng-odp] [API-NEXT PATCHv2 1/2] api: dev: add device apis for > numa support > > Add the odp_dev_id() API used for NUMA support > > Signed-off-by: Bill Fischofer <bill.fischofer@linaro.org> > --- > include/odp/api/spec/dev.h | 89 > ++++++++++++++++++++++++++++++++++++++++++++++ > 1 file changed, 89 insertions(+) > create mode 100644 include/odp/api/spec/dev.h > > diff --git a/include/odp/api/spec/dev.h b/include/odp/api/spec/dev.h > new file mode 100644 > index 0000000..1f7ed8b > --- /dev/null > +++ b/include/odp/api/spec/dev.h > @@ -0,0 +1,89 @@ > +/* Copyright (c) 2016, Linaro Limited > + * All rights reserved. > + * > + * SPDX-License-Identifier: BSD-3-Clause > + */ > + > +/** > + * @file > + * > + * ODP device > + */ > + > +#ifndef ODP_API_DEV_H_ > +#define ODP_API_DEV_H_ > +#include <odp/visibility_begin.h> > + > +#ifdef __cplusplus > +extern "C" { > +#endif > + > +#include <odp/api/std_types.h> > + > +/** @defgroup odp_dev ODP DEVICE > + * Operations on devices > + * @{ > + */ > + > +/** > + * @typedef odp_dev_t > + * ODP Device > + */ > + > +/** > + * @def ODP_DEV_NAME_LEN > + * Maximum device name length in chars ... (including null char). We'll harmonize all name length defines with that. > + */ > + > +/** > + * @def ODP_DEV_ANY > + * Any device > + */ Change to ODP_DEV_DEFAULT, since it's more descriptive. Also _DEFAULT is already used by API and _ANY is not. Also move specification text from below: "The reserved id ODP_DEV_DEFAULT may be used as a default value ("don't care") wherever a device id is required." > + > +/** > + * @def ODP_DEV_INVALID > + * Invalid device > + */ > + > +/** > + * Get Device ID by Name No CamelCase please. "Get device identifier by name" > + * > + * Get an implementation-defined device identifier from a device name. > Device > + * names are supplied as parameter info (command line, file, etc.) to the > + * application. This routine translates this symbolic name into an > internal > + * identifier that can be used to define a device connection hierarchy ... can be used to define a device for NUMA or other purposes. No need to mention "connection hierarchy". > for > + * NUMA or other purposes. > + * > + * The reserved id ODP_DEV_ANY may be used as a "don't care" placeholder > + * wherever a device id is required. This should be moved from here to ODP_DEV_DEFAULT documentation. The default value is not needed for using the function, but where the dev ID is used. > + * > + * @param name Name of the device > + * > + * @return Device ID > + * @retval ODP_DEV_INVALID Device is unknown > + */ > +odp_dev_t odp_dev_id(const char *name); > + > +/** > + * Get printable value for an odp_dev_t > + * > + * @param hdl odp_dev_t handle to be printed Better to use "dev" as param name than "hdl". That will be the common param name ("odp_dev_t dev") for other APIs. -Petri > + * @return uint64_t value that can be used to print/display this > + * handle > + * > + * @note This routine is intended to be used for diagnostic purposes > + * to enable applications to generate a printable value that represents > + * an odp_dev_t handle. > + */ > +uint64_t odp_dev_to_u64(odp_dev_t hdl); > + > +/** > + * @} > + */ > + > +#ifdef __cplusplus > +} > +#endif > + > +#include <odp/visibility_end.h> > +#endif > -- > 2.7.4
diff --git a/include/odp/api/spec/dev.h b/include/odp/api/spec/dev.h new file mode 100644 index 0000000..1f7ed8b --- /dev/null +++ b/include/odp/api/spec/dev.h @@ -0,0 +1,89 @@ +/* Copyright (c) 2016, Linaro Limited + * All rights reserved. + * + * SPDX-License-Identifier: BSD-3-Clause + */ + +/** + * @file + * + * ODP device + */ + +#ifndef ODP_API_DEV_H_ +#define ODP_API_DEV_H_ +#include <odp/visibility_begin.h> + +#ifdef __cplusplus +extern "C" { +#endif + +#include <odp/api/std_types.h> + +/** @defgroup odp_dev ODP DEVICE + * Operations on devices + * @{ + */ + +/** + * @typedef odp_dev_t + * ODP Device + */ + +/** + * @def ODP_DEV_NAME_LEN + * Maximum device name length in chars + */ + +/** + * @def ODP_DEV_ANY + * Any device + */ + +/** + * @def ODP_DEV_INVALID + * Invalid device + */ + +/** + * Get Device ID by Name + * + * Get an implementation-defined device identifier from a device name. Device + * names are supplied as parameter info (command line, file, etc.) to the + * application. This routine translates this symbolic name into an internal + * identifier that can be used to define a device connection hierarchy for + * NUMA or other purposes. + * + * The reserved id ODP_DEV_ANY may be used as a "don't care" placeholder + * wherever a device id is required. + * + * @param name Name of the device + * + * @return Device ID + * @retval ODP_DEV_INVALID Device is unknown + */ +odp_dev_t odp_dev_id(const char *name); + +/** + * Get printable value for an odp_dev_t + * + * @param hdl odp_dev_t handle to be printed + * @return uint64_t value that can be used to print/display this + * handle + * + * @note This routine is intended to be used for diagnostic purposes + * to enable applications to generate a printable value that represents + * an odp_dev_t handle. + */ +uint64_t odp_dev_to_u64(odp_dev_t hdl); + +/** + * @} + */ + +#ifdef __cplusplus +} +#endif + +#include <odp/visibility_end.h> +#endif
Add the odp_dev_id() API used for NUMA support Signed-off-by: Bill Fischofer <bill.fischofer@linaro.org> --- include/odp/api/spec/dev.h | 89 ++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 89 insertions(+) create mode 100644 include/odp/api/spec/dev.h -- 2.7.4