| Message ID | 1407165786-32589-1-git-send-email-maxim.uvarov@linaro.org |
|---|---|
| State | New |
| Headers | show |
On 08/04/2014 06:23 PM, Maxim Uvarov wrote: > Add ability to specify which odp layers should be > initialized. > > Signed-off-by: Maxim Uvarov <maxim.uvarov@linaro.org> > --- > v2: change logic according to v1. If flag set layer will be > initialized. > > example/generator/odp_generator.c | 2 +- > example/l2fwd/odp_l2fwd.c | 2 +- > example/odp_example/odp_example.c | 2 +- > example/packet/odp_pktio.c | 2 +- > example/packet_netmap/odp_pktio_netmap.c | 2 +- > example/timer/odp_timer_test.c | 2 +- > include/odp_init.h | 10 ++++++- > platform/linux-dpdk/odp_init.c | 50 ++++++++++++++++++++------------ > platform/linux-generic/odp_init.c | 50 ++++++++++++++++++++------------ > platform/linux-keystone2/odp_init.c | 50 ++++++++++++++++++++------------ > test/api_test/odp_common.c | 2 +- > 11 files changed, 109 insertions(+), 65 deletions(-) > [...] > diff --git a/include/odp_init.h b/include/odp_init.h > index 490324a..f119e99 100644 > --- a/include/odp_init.h > +++ b/include/odp_init.h > @@ -22,6 +22,13 @@ extern "C" { > > #include <odp_std_types.h> > > +#define ODP_INIT_F_ALL 0xff > +#define ODP_INIT_F_SHMINIT (1 << 0) > +#define ODP_INIT_F_GPOOL (1 << 1) > +#define ODP_INIT_F_GQUEUE (1 << 2) What 'G' in GPOOL in GQUEUE stands for? IMO buffer pools and queues are mandatory part of ODP application. Do you have example where you would want to avoid their initialization? > +#define ODP_INIT_F_SCHEDULE (1 << 3) > +#define ODP_INIT_F_PKTIO (1 << 4) > +#define ODP_INIT_F_TIMER (1 << 5) > > > > @@ -31,9 +38,10 @@ extern "C" { > * This function must be called once before calling > * any other ODP API functions. > * > + * @param mask Mask odp layers to be initialized > * @return 0 if successful > */ > -int odp_init_global(void); > +int odp_init_global(uint8_t mask); 8 bits may be not enough after some time.
On 08/04/2014 09:30 PM, Taras Kondratiuk wrote: > On 08/04/2014 06:23 PM, Maxim Uvarov wrote: >> Add ability to specify which odp layers should be >> initialized. >> >> Signed-off-by: Maxim Uvarov <maxim.uvarov@linaro.org> >> --- >> v2: change logic according to v1. If flag set layer will be >> initialized. >> >> example/generator/odp_generator.c | 2 +- >> example/l2fwd/odp_l2fwd.c | 2 +- >> example/odp_example/odp_example.c | 2 +- >> example/packet/odp_pktio.c | 2 +- >> example/packet_netmap/odp_pktio_netmap.c | 2 +- >> example/timer/odp_timer_test.c | 2 +- >> include/odp_init.h | 10 ++++++- >> platform/linux-dpdk/odp_init.c | 50 >> ++++++++++++++++++++------------ >> platform/linux-generic/odp_init.c | 50 >> ++++++++++++++++++++------------ >> platform/linux-keystone2/odp_init.c | 50 >> ++++++++++++++++++++------------ >> test/api_test/odp_common.c | 2 +- >> 11 files changed, 109 insertions(+), 65 deletions(-) >> > > [...] > >> diff --git a/include/odp_init.h b/include/odp_init.h >> index 490324a..f119e99 100644 >> --- a/include/odp_init.h >> +++ b/include/odp_init.h >> @@ -22,6 +22,13 @@ extern "C" { >> >> #include <odp_std_types.h> >> >> +#define ODP_INIT_F_ALL 0xff >> +#define ODP_INIT_F_SHMINIT (1 << 0) >> +#define ODP_INIT_F_GPOOL (1 << 1) >> +#define ODP_INIT_F_GQUEUE (1 << 2) > > What 'G' in GPOOL in GQUEUE stands for? G is Global, because pool can be created in application also. So I added G to specify that it's global. > > IMO buffer pools and queues are mandatory part of ODP application. Do > you have example where you would want to avoid their initialization? You can avoid queues in burst case (packet io example) where packets fetched directly without queues. You probably not need pool in case if you do some cryto app which uses buffer from memory chunk api. Not packet pool buffers. > >> +#define ODP_INIT_F_SCHEDULE (1 << 3) >> +#define ODP_INIT_F_PKTIO (1 << 4) >> +#define ODP_INIT_F_TIMER (1 << 5) >> >> >> >> @@ -31,9 +38,10 @@ extern "C" { >> * This function must be called once before calling >> * any other ODP API functions. >> * >> + * @param mask Mask odp layers to be initialized >> * @return 0 if successful >> */ >> -int odp_init_global(void); >> +int odp_init_global(uint8_t mask); > > 8 bits may be not enough after some time. When it will not be enough we will extend that. For now there is room for 3 more flags.
On 4 August 2014 13:39, Maxim Uvarov <maxim.uvarov@linaro.org> wrote: > On 08/04/2014 09:30 PM, Taras Kondratiuk wrote: > >> On 08/04/2014 06:23 PM, Maxim Uvarov wrote: >> >>> Add ability to specify which odp layers should be >>> initialized. >>> >>> Signed-off-by: Maxim Uvarov <maxim.uvarov@linaro.org> >>> --- >>> v2: change logic according to v1. If flag set layer will be >>> initialized. >>> >>> example/generator/odp_generator.c | 2 +- >>> example/l2fwd/odp_l2fwd.c | 2 +- >>> example/odp_example/odp_example.c | 2 +- >>> example/packet/odp_pktio.c | 2 +- >>> example/packet_netmap/odp_pktio_netmap.c | 2 +- >>> example/timer/odp_timer_test.c | 2 +- >>> include/odp_init.h | 10 ++++++- >>> platform/linux-dpdk/odp_init.c | 50 >>> ++++++++++++++++++++------------ >>> platform/linux-generic/odp_init.c | 50 >>> ++++++++++++++++++++------------ >>> platform/linux-keystone2/odp_init.c | 50 >>> ++++++++++++++++++++------------ >>> test/api_test/odp_common.c | 2 +- >>> 11 files changed, 109 insertions(+), 65 deletions(-) >>> >>> >> [...] >> >> diff --git a/include/odp_init.h b/include/odp_init.h >>> index 490324a..f119e99 100644 >>> --- a/include/odp_init.h >>> +++ b/include/odp_init.h >>> @@ -22,6 +22,13 @@ extern "C" { >>> >>> #include <odp_std_types.h> >>> >>> +#define ODP_INIT_F_ALL 0xff >>> +#define ODP_INIT_F_SHMINIT (1 << 0) >>> +#define ODP_INIT_F_GPOOL (1 << 1) >>> +#define ODP_INIT_F_GQUEUE (1 << 2) >>> >> >> What 'G' in GPOOL in GQUEUE stands for? >> > > G is Global, because pool can be created in application also. So I added G > to > specify that it's global. > > > >> IMO buffer pools and queues are mandatory part of ODP application. Do >> you have example where you would want to avoid their initialization? >> > > You can avoid queues in burst case (packet io example) where packets > fetched > directly without queues. You probably not need pool in case if you do some > cryto app > which uses buffer from memory chunk api. Not packet pool buffers. > > > >> +#define ODP_INIT_F_SCHEDULE (1 << 3) >>> +#define ODP_INIT_F_PKTIO (1 << 4) >>> +#define ODP_INIT_F_TIMER (1 << 5) >>> >>> >>> >>> @@ -31,9 +38,10 @@ extern "C" { >>> * This function must be called once before calling >>> * any other ODP API functions. >>> * >>> + * @param mask Mask odp layers to be initialized >>> * @return 0 if successful >>> */ >>> -int odp_init_global(void); >>> +int odp_init_global(uint8_t mask); >>> >> >> 8 bits may be not enough after some time. >> > When it will not be enough we will extend that. For now there is room for > 3 more flags. Would it not be better just to go with an int now ? I can see using up 3 more flags quite easily, but 32 bits will take a while. > > > _______________________________________________ > lng-odp mailing list > lng-odp@lists.linaro.org > http://lists.linaro.org/mailman/listinfo/lng-odp >
On 08/04/2014 08:39 PM, Maxim Uvarov wrote: > On 08/04/2014 09:30 PM, Taras Kondratiuk wrote: >> On 08/04/2014 06:23 PM, Maxim Uvarov wrote: >>> diff --git a/include/odp_init.h b/include/odp_init.h >>> index 490324a..f119e99 100644 >>> --- a/include/odp_init.h >>> +++ b/include/odp_init.h >>> @@ -22,6 +22,13 @@ extern "C" { >>> >>> #include <odp_std_types.h> >>> >>> +#define ODP_INIT_F_ALL 0xff >>> +#define ODP_INIT_F_SHMINIT (1 << 0) >>> +#define ODP_INIT_F_GPOOL (1 << 1) >>> +#define ODP_INIT_F_GQUEUE (1 << 2) >> >> What 'G' in GPOOL in GQUEUE stands for? > > G is Global, because pool can be created in application also. So I added > G to > specify that it's global. Do you mean 'predefined pools'? Better to use this name. We don't have a global queues defined yet.
On 08/04/2014 06:23 PM, Maxim Uvarov wrote: > diff --git a/example/packet/odp_pktio.c b/example/packet/odp_pktio.c > index edf8cfd..8079fd0 100644 > --- a/example/packet/odp_pktio.c > +++ b/example/packet/odp_pktio.c > @@ -285,7 +285,7 @@ int main(int argc, char *argv[]) > int core_count; > > /* Init ODP before calling anything else */ > - if (odp_init_global()) { > + if (odp_init_global(ODP_INIT_F_ALL & ~ODP_INIT_F_TIMER)) { > ODP_ERR("Error: ODP global init failed.\n"); > exit(EXIT_FAILURE); > } By passing ~ODP_INIT_F_TIMER you are effectively using 'disable' approach. The idea of having 'enable' flags instead of 'disable' is to explicitly inform implementation about API sets that will be used. Normally you know what you will use, but you have no idea as application writer which sets of API are available in an implementation that you won't use. For example if we will add new sets of API (IPC, classification, etc.) this application will have to be updated with ~ODP_INIT_F_IPC & ~ODP_INIT_F_CLASSIFICATION. Right?
How does this play in with Keiths resumed call for argv argc? Thread = "[lng-odp] Port of Pktgen to ODP" it looks like on two fronts we have come back to the need to add args to init that are more complex than basic flags. With argv argc, I assume we can pass these flags as options with getopt ? Some issues are 1. passing things through to underlying SDKs 2. Making some args mandatory 3. Keeping the argument parsing in sync across platforms 4. Not coupling all the platforms and their special args together 5. Not promoting the use of platform specific args in general as they will break portability. Mike On 5 August 2014 05:57, Taras Kondratiuk <taras.kondratiuk@linaro.org> wrote: > On 08/04/2014 06:23 PM, Maxim Uvarov wrote: > >> diff --git a/example/packet/odp_pktio.c b/example/packet/odp_pktio.c >> index edf8cfd..8079fd0 100644 >> --- a/example/packet/odp_pktio.c >> +++ b/example/packet/odp_pktio.c >> @@ -285,7 +285,7 @@ int main(int argc, char *argv[]) >> int core_count; >> >> /* Init ODP before calling anything else */ >> - if (odp_init_global()) { >> + if (odp_init_global(ODP_INIT_F_ALL & ~ODP_INIT_F_TIMER)) { >> ODP_ERR("Error: ODP global init failed.\n"); >> exit(EXIT_FAILURE); >> } >> > > By passing ~ODP_INIT_F_TIMER you are effectively using 'disable' > approach. The idea of having 'enable' flags instead of 'disable' is to > explicitly inform implementation about API sets that will be used. > Normally you know what you will use, but you have no idea as > application writer which sets of API are available in an implementation > that you won't use. For example if we will add new sets of API (IPC, > classification, etc.) this application will have to be updated with > ~ODP_INIT_F_IPC & ~ODP_INIT_F_CLASSIFICATION. Right? > > > _______________________________________________ > lng-odp mailing list > lng-odp@lists.linaro.org > http://lists.linaro.org/mailman/listinfo/lng-odp >
On 08/05/2014 02:59 PM, Mike Holmes wrote: > How does this play in with Keiths resumed call for argv argc? Thread = > "[lng-odp] Port of Pktgen to ODP" it looks like on two fronts we have > come back to the need to add args to init that are more complex than > basic flags. > > With argv argc, I assume we can pass these flags as options with getopt ? These flags are defined by API used by application, so they are hardcoded into application, but not passed from command line. If we are going to argv/argc way, then application can just insert one more parameter into the list, before passing it to odp_init_global(). > Some issues are > > 1. passing things through to underlying SDKs > 2. Making some args mandatory > 3. Keeping the argument parsing in sync across platforms > 4. Not coupling all the platforms and their special args together > 5. Not promoting the use of platform specific args in general as they > will break portability. I agree with an approach when only common platform independent arguments are passed to ODP application. We have to analyze which parameters are required by different platforms and come up with a common list. The rest platform specific parameters have to passed to implementation in some other way (environment variable, config file, hardcoded, etc.)
I failed to summarise yesterdays conversation, here is todays attempt. - argv, arc is not in principle opposed by anyone. - Need to push as much as possible into the implementation wherever possible - Arguments should try very hard to be platform agnostic - Arguments for a platform need to be marked and passed through verbatim - then duplication of existing flags on different sdks is ok and clear to the user. Maybe --dpdk followed by all the flags it expects. - VMs may need some differences passed in, so not all differences can be compiled into the implementation. - Need to avoid the case of two platforms claiming the same argument - A strong case for scripting like Lua was not exposed in this discussion. Could I ask everyone to reply with any agnostic and platform specific arguments they would like to see, maybe then we can form a clear picture of those that are needed. On 5 August 2014 08:31, Taras Kondratiuk <taras.kondratiuk@linaro.org> wrote: > On 08/05/2014 02:59 PM, Mike Holmes wrote: > >> How does this play in with Keiths resumed call for argv argc? Thread = >> "[lng-odp] Port of Pktgen to ODP" it looks like on two fronts we have >> come back to the need to add args to init that are more complex than >> basic flags. >> >> With argv argc, I assume we can pass these flags as options with getopt ? >> > > These flags are defined by API used by application, so they are > hardcoded into application, but not passed from command line. If we are > going to argv/argc way, then application can just insert one more > parameter into the list, before passing it to odp_init_global(). > > Some issues are >> >> 1. passing things through to underlying SDKs >> 2. Making some args mandatory >> 3. Keeping the argument parsing in sync across platforms >> 4. Not coupling all the platforms and their special args together >> 5. Not promoting the use of platform specific args in general as they >> will break portability. >> > > I agree with an approach when only common platform independent arguments > are passed to ODP application. We have to analyze which parameters are > required by different platforms and come up with a common list. The rest > platform specific parameters have to passed to implementation in some > other way (environment variable, config file, hardcoded, etc.) >
Here is an example of what I am suggesting be done for argc/argv. This is just an example for everyone to play with as it will need some testing and changes I would guess. I am not perfect, but I do play one on TV :-)
Keith Wiles, Principal Technologist with CTO office, Wind River
mobile 972-213-5533
[Powering 30 Years of Innovation]<http://www.windriver.com/announces/wr30/>
On Aug 7, 2014, at 7:52 AM, Mike Holmes <mike.holmes@linaro.org<mailto:mike.holmes@linaro.org>> wrote:
I failed to summarise yesterdays conversation, here is todays attempt.
* argv, arc is not in principle opposed by anyone.
* Need to push as much as possible into the implementation wherever possible
* Arguments should try very hard to be platform agnostic
* Arguments for a platform need to be marked and passed through verbatim - then duplication of existing flags on different sdks is ok and clear to the user. Maybe --dpdk followed by all the flags it expects.
* VMs may need some differences passed in, so not all differences can be compiled into the implementation.
* Need to avoid the case of two platforms claiming the same argument
* A strong case for scripting like Lua was not exposed in this discussion.
Could I ask everyone to reply with any agnostic and platform specific arguments they would like to see, maybe then we can form a clear picture of those that are needed.
On 5 August 2014 08:31, Taras Kondratiuk <taras.kondratiuk@linaro.org<mailto:taras.kondratiuk@linaro.org>> wrote:
On 08/05/2014 02:59 PM, Mike Holmes wrote:
How does this play in with Keiths resumed call for argv argc? Thread =
"[lng-odp] Port of Pktgen to ODP" it looks like on two fronts we have
come back to the need to add args to init that are more complex than
basic flags.
With argv argc, I assume we can pass these flags as options with getopt ?
These flags are defined by API used by application, so they are
hardcoded into application, but not passed from command line. If we are
going to argv/argc way, then application can just insert one more
parameter into the list, before passing it to odp_init_global().
Some issues are
1. passing things through to underlying SDKs
2. Making some args mandatory
3. Keeping the argument parsing in sync across platforms
4. Not coupling all the platforms and their special args together
5. Not promoting the use of platform specific args in general as they
will break portability.
I agree with an approach when only common platform independent arguments
are passed to ODP application. We have to analyze which parameters are
required by different platforms and come up with a common list. The rest
platform specific parameters have to passed to implementation in some
other way (environment variable, config file, hardcoded, etc.)
--
Mike Holmes
Linaro Technical Manager / Lead
LNG - ODP
On 7 August 2014 05:52, Mike Holmes <mike.holmes@linaro.org> wrote: > I failed to summarise yesterdays conversation, here is todays attempt. > > argv, arc is not in principle opposed by anyone. In what role? For functionality that involves serious production deployment they *must not* be used. The same goes to environment variables. All functionality must be available through APIs. If we are talking about some structure of argv,argc in ODP examples and test then it is fine. However it such context it is not clear why platforms have to be in agreement. Thanks, Victor > Need to push as much as possible into the implementation wherever possible > Arguments should try very hard to be platform agnostic > Arguments for a platform need to be marked and passed through verbatim - > then duplication of existing flags on different sdks is ok and clear to the > user. Maybe --dpdk followed by all the flags it expects. > VMs may need some differences passed in, so not all differences can be > compiled into the implementation. > Need to avoid the case of two platforms claiming the same argument > A strong case for scripting like Lua was not exposed in this discussion. > > Could I ask everyone to reply with any agnostic and platform specific > arguments they would like to see, maybe then we can form a clear picture of > those that are needed. > > > > On 5 August 2014 08:31, Taras Kondratiuk <taras.kondratiuk@linaro.org> > wrote: >> >> On 08/05/2014 02:59 PM, Mike Holmes wrote: >>> >>> How does this play in with Keiths resumed call for argv argc? Thread = >>> "[lng-odp] Port of Pktgen to ODP" it looks like on two fronts we have >>> come back to the need to add args to init that are more complex than >>> basic flags. >>> >>> With argv argc, I assume we can pass these flags as options with getopt ? >> >> >> These flags are defined by API used by application, so they are >> hardcoded into application, but not passed from command line. If we are >> going to argv/argc way, then application can just insert one more >> parameter into the list, before passing it to odp_init_global(). >> >>> Some issues are >>> >>> 1. passing things through to underlying SDKs >>> 2. Making some args mandatory >>> 3. Keeping the argument parsing in sync across platforms >>> 4. Not coupling all the platforms and their special args together >>> 5. Not promoting the use of platform specific args in general as they >>> will break portability. >> >> >> I agree with an approach when only common platform independent arguments >> are passed to ODP application. We have to analyze which parameters are >> required by different platforms and come up with a common list. The rest >> platform specific parameters have to passed to implementation in some >> other way (environment variable, config file, hardcoded, etc.) > > > > > -- > Mike Holmes > Linaro Technical Manager / Lead > LNG - ODP > > _______________________________________________ > lng-odp mailing list > lng-odp@lists.linaro.org > http://lists.linaro.org/mailman/listinfo/lng-odp >
Victor, did you have a specific set of API suggestions for this? I think that this whole area should be part of the ODP v2.0 planning discussions we'd like to have at LCU. It would be interesting to see if NFV has any recommendations in this area since this is all part of hooking into the larger management infrastructure. Bill On Thu, Aug 7, 2014 at 12:02 PM, Victor Kamensky <victor.kamensky@linaro.org > wrote: > On 7 August 2014 05:52, Mike Holmes <mike.holmes@linaro.org> wrote: > > I failed to summarise yesterdays conversation, here is todays attempt. > > > > argv, arc is not in principle opposed by anyone. > > In what role? For functionality that involves serious > production deployment they *must not* be used. > The same goes to environment variables. All > functionality must be available through APIs. > > If we are talking about some structure of > argv,argc in ODP examples and test then > it is fine. However it such context it is not > clear why platforms have to be in agreement. > > Thanks, > Victor > > > Need to push as much as possible into the implementation wherever > possible > > Arguments should try very hard to be platform agnostic > > Arguments for a platform need to be marked and passed through verbatim - > > then duplication of existing flags on different sdks is ok and clear to > the > > user. Maybe --dpdk followed by all the flags it expects. > > VMs may need some differences passed in, so not all differences can be > > compiled into the implementation. > > Need to avoid the case of two platforms claiming the same argument > > A strong case for scripting like Lua was not exposed in this discussion. > > > > Could I ask everyone to reply with any agnostic and platform specific > > arguments they would like to see, maybe then we can form a clear picture > of > > those that are needed. > > > > > > > > On 5 August 2014 08:31, Taras Kondratiuk <taras.kondratiuk@linaro.org> > > wrote: > >> > >> On 08/05/2014 02:59 PM, Mike Holmes wrote: > >>> > >>> How does this play in with Keiths resumed call for argv argc? Thread = > >>> "[lng-odp] Port of Pktgen to ODP" it looks like on two fronts we have > >>> come back to the need to add args to init that are more complex than > >>> basic flags. > >>> > >>> With argv argc, I assume we can pass these flags as options with > getopt ? > >> > >> > >> These flags are defined by API used by application, so they are > >> hardcoded into application, but not passed from command line. If we are > >> going to argv/argc way, then application can just insert one more > >> parameter into the list, before passing it to odp_init_global(). > >> > >>> Some issues are > >>> > >>> 1. passing things through to underlying SDKs > >>> 2. Making some args mandatory > >>> 3. Keeping the argument parsing in sync across platforms > >>> 4. Not coupling all the platforms and their special args together > >>> 5. Not promoting the use of platform specific args in general as they > >>> will break portability. > >> > >> > >> I agree with an approach when only common platform independent arguments > >> are passed to ODP application. We have to analyze which parameters are > >> required by different platforms and come up with a common list. The rest > >> platform specific parameters have to passed to implementation in some > >> other way (environment variable, config file, hardcoded, etc.) > > > > > > > > > > -- > > Mike Holmes > > Linaro Technical Manager / Lead > > LNG - ODP > > > > _______________________________________________ > > lng-odp mailing list > > lng-odp@lists.linaro.org > > http://lists.linaro.org/mailman/listinfo/lng-odp > > > > _______________________________________________ > lng-odp mailing list > lng-odp@lists.linaro.org > http://lists.linaro.org/mailman/listinfo/lng-odp >
On 7 August 2014 13:02, Victor Kamensky <victor.kamensky@linaro.org> wrote: > On 7 August 2014 05:52, Mike Holmes <mike.holmes@linaro.org> wrote: > > I failed to summarise yesterdays conversation, here is todays attempt. > > > > argv, arc is not in principle opposed by anyone. > > In what role? For functionality that involves serious > production deployment they *must not* be used. > The same goes to environment variables. All > functionality must be available through APIs. > If odp_global init is an API that takes argv and arg style arguments is that ok, I am not sure what you mean by it must be via an API unless you mean APIs that do not take variable argument lists > > If we are talking about some structure of > argv,argc in ODP examples and test then > it is fine. However it such context it is not > clear why platforms have to be in agreement. > I think this is a discussion about how to pass things into the init of ODP rather than examples, the recurring need to pass some platform specific data though to the implementation. I also assume the case or platform agnostic configuration that cannot be hard coded - I am imagining ODP running in an Enterprise situation in multiple VMs where you want the same app attached to different ports or something like that. > > Thanks, > Victor > > > Need to push as much as possible into the implementation wherever > possible > > Arguments should try very hard to be platform agnostic > > Arguments for a platform need to be marked and passed through verbatim - > > then duplication of existing flags on different sdks is ok and clear to > the > > user. Maybe --dpdk followed by all the flags it expects. > > VMs may need some differences passed in, so not all differences can be > > compiled into the implementation. > > Need to avoid the case of two platforms claiming the same argument > > A strong case for scripting like Lua was not exposed in this discussion. > > > > Could I ask everyone to reply with any agnostic and platform specific > > arguments they would like to see, maybe then we can form a clear picture > of > > those that are needed. > > > > > > > > On 5 August 2014 08:31, Taras Kondratiuk <taras.kondratiuk@linaro.org> > > wrote: > >> > >> On 08/05/2014 02:59 PM, Mike Holmes wrote: > >>> > >>> How does this play in with Keiths resumed call for argv argc? Thread = > >>> "[lng-odp] Port of Pktgen to ODP" it looks like on two fronts we have > >>> come back to the need to add args to init that are more complex than > >>> basic flags. > >>> > >>> With argv argc, I assume we can pass these flags as options with > getopt ? > >> > >> > >> These flags are defined by API used by application, so they are > >> hardcoded into application, but not passed from command line. If we are > >> going to argv/argc way, then application can just insert one more > >> parameter into the list, before passing it to odp_init_global(). > >> > >>> Some issues are > >>> > >>> 1. passing things through to underlying SDKs > >>> 2. Making some args mandatory > >>> 3. Keeping the argument parsing in sync across platforms > >>> 4. Not coupling all the platforms and their special args together > >>> 5. Not promoting the use of platform specific args in general as they > >>> will break portability. > >> > >> > >> I agree with an approach when only common platform independent arguments > >> are passed to ODP application. We have to analyze which parameters are > >> required by different platforms and come up with a common list. The rest > >> platform specific parameters have to passed to implementation in some > >> other way (environment variable, config file, hardcoded, etc.) > > > > > > > > > > -- > > Mike Holmes > > Linaro Technical Manager / Lead > > LNG - ODP > > > > _______________________________________________ > > lng-odp mailing list > > lng-odp@lists.linaro.org > > http://lists.linaro.org/mailman/listinfo/lng-odp > > >
On 7 August 2014 10:08, Bill Fischofer <bill.fischofer@linaro.org> wrote: > Victor, did you have a specific set of API suggestions for this? I think > that this whole area should be part of the ODP v2.0 planning discussions > we'd like to have at LCU. It would be interesting to see if NFV has any > recommendations in this area since this is all part of hooking into the > larger management infrastructure. I think I am missing some context here. Why we are talking about argc/argv on thread where init flags are proposed? And as far as original diff is concerned I don't see any sense in it. I consider ODP implementation as one unit of functionality and only one simple init call is needed to start it, maybe another call in threads created by user. Asking user to understand all these flags and meke it consistent between different implementation is very hard. Also what is about internal dependencies, i.e if I did not pass ODP_INIT_F_GPOOL but pass ODP_INIT_F_PKTIO (assuming that PKTIO depends on GPOOL) what will happen in this case? If there is a problem of cost of initializing certain resources it should not be driven by flags passed from user, but rather converted on first use initialization. Thanks, Victor > Bill > > > On Thu, Aug 7, 2014 at 12:02 PM, Victor Kamensky > <victor.kamensky@linaro.org> wrote: >> >> On 7 August 2014 05:52, Mike Holmes <mike.holmes@linaro.org> wrote: >> > I failed to summarise yesterdays conversation, here is todays attempt. >> > >> > argv, arc is not in principle opposed by anyone. >> >> In what role? For functionality that involves serious >> production deployment they *must not* be used. >> The same goes to environment variables. All >> functionality must be available through APIs. >> >> If we are talking about some structure of >> argv,argc in ODP examples and test then >> it is fine. However it such context it is not >> clear why platforms have to be in agreement. >> >> Thanks, >> Victor >> >> > Need to push as much as possible into the implementation wherever >> > possible >> > Arguments should try very hard to be platform agnostic >> > Arguments for a platform need to be marked and passed through verbatim - >> > then duplication of existing flags on different sdks is ok and clear to >> > the >> > user. Maybe --dpdk followed by all the flags it expects. >> > VMs may need some differences passed in, so not all differences can be >> > compiled into the implementation. >> > Need to avoid the case of two platforms claiming the same argument >> > A strong case for scripting like Lua was not exposed in this discussion. >> > >> > Could I ask everyone to reply with any agnostic and platform specific >> > arguments they would like to see, maybe then we can form a clear picture >> > of >> > those that are needed. >> > >> > >> > >> > On 5 August 2014 08:31, Taras Kondratiuk <taras.kondratiuk@linaro.org> >> > wrote: >> >> >> >> On 08/05/2014 02:59 PM, Mike Holmes wrote: >> >>> >> >>> How does this play in with Keiths resumed call for argv argc? Thread >> >>> = >> >>> "[lng-odp] Port of Pktgen to ODP" it looks like on two fronts we have >> >>> come back to the need to add args to init that are more complex than >> >>> basic flags. >> >>> >> >>> With argv argc, I assume we can pass these flags as options with >> >>> getopt ? >> >> >> >> >> >> These flags are defined by API used by application, so they are >> >> hardcoded into application, but not passed from command line. If we are >> >> going to argv/argc way, then application can just insert one more >> >> parameter into the list, before passing it to odp_init_global(). >> >> >> >>> Some issues are >> >>> >> >>> 1. passing things through to underlying SDKs >> >>> 2. Making some args mandatory >> >>> 3. Keeping the argument parsing in sync across platforms >> >>> 4. Not coupling all the platforms and their special args together >> >>> 5. Not promoting the use of platform specific args in general as they >> >>> will break portability. >> >> >> >> >> >> I agree with an approach when only common platform independent >> >> arguments >> >> are passed to ODP application. We have to analyze which parameters are >> >> required by different platforms and come up with a common list. The >> >> rest >> >> platform specific parameters have to passed to implementation in some >> >> other way (environment variable, config file, hardcoded, etc.) >> > >> > >> > >> > >> > -- >> > Mike Holmes >> > Linaro Technical Manager / Lead >> > LNG - ODP >> > >> > _______________________________________________ >> > lng-odp mailing list >> > lng-odp@lists.linaro.org >> > http://lists.linaro.org/mailman/listinfo/lng-odp >> > >> >> _______________________________________________ >> lng-odp mailing list >> lng-odp@lists.linaro.org >> http://lists.linaro.org/mailman/listinfo/lng-odp > >
On 7 August 2014 13:20, Victor Kamensky <victor.kamensky@linaro.org> wrote: > On 7 August 2014 10:08, Bill Fischofer <bill.fischofer@linaro.org> wrote: > > Victor, did you have a specific set of API suggestions for this? I think > > that this whole area should be part of the ODP v2.0 planning discussions > > we'd like to have at LCU. It would be interesting to see if NFV has any > > recommendations in this area since this is all part of hooking into the > > larger management infrastructure. > > I think I am missing some context here. Why we are talking about > argc/argv on thread where init flags are proposed? > The context is the discussions in the meetings over the last day or two about platform compile time configuration, vs run time ODP configuration and runtime platform specific configuration, the two topics have become merged. > > And as far as original diff is concerned I don't see any sense in it. > I consider ODP implementation as one unit of functionality and only > one simple init call is needed to start it, maybe another call in threads > created by user. Asking user to understand all these flags and meke > it consistent between different implementation is very hard. Also what > is about internal dependencies, i.e if I did not pass ODP_INIT_F_GPOOL > but pass ODP_INIT_F_PKTIO (assuming that PKTIO depends on > GPOOL) what will happen in this case? > > If there is a problem of cost of initializing certain resources it > should not be driven by flags passed from user, but rather converted > on first use initialization. > > Thanks, > Victor > > > Bill > > > > > > On Thu, Aug 7, 2014 at 12:02 PM, Victor Kamensky > > <victor.kamensky@linaro.org> wrote: > >> > >> On 7 August 2014 05:52, Mike Holmes <mike.holmes@linaro.org> wrote: > >> > I failed to summarise yesterdays conversation, here is todays attempt. > >> > > >> > argv, arc is not in principle opposed by anyone. > >> > >> In what role? For functionality that involves serious > >> production deployment they *must not* be used. > >> The same goes to environment variables. All > >> functionality must be available through APIs. > >> > >> If we are talking about some structure of > >> argv,argc in ODP examples and test then > >> it is fine. However it such context it is not > >> clear why platforms have to be in agreement. > >> > >> Thanks, > >> Victor > >> > >> > Need to push as much as possible into the implementation wherever > >> > possible > >> > Arguments should try very hard to be platform agnostic > >> > Arguments for a platform need to be marked and passed through > verbatim - > >> > then duplication of existing flags on different sdks is ok and clear > to > >> > the > >> > user. Maybe --dpdk followed by all the flags it expects. > >> > VMs may need some differences passed in, so not all differences can be > >> > compiled into the implementation. > >> > Need to avoid the case of two platforms claiming the same argument > >> > A strong case for scripting like Lua was not exposed in this > discussion. > >> > > >> > Could I ask everyone to reply with any agnostic and platform specific > >> > arguments they would like to see, maybe then we can form a clear > picture > >> > of > >> > those that are needed. > >> > > >> > > >> > > >> > On 5 August 2014 08:31, Taras Kondratiuk <taras.kondratiuk@linaro.org > > > >> > wrote: > >> >> > >> >> On 08/05/2014 02:59 PM, Mike Holmes wrote: > >> >>> > >> >>> How does this play in with Keiths resumed call for argv argc? > Thread > >> >>> = > >> >>> "[lng-odp] Port of Pktgen to ODP" it looks like on two fronts we > have > >> >>> come back to the need to add args to init that are more complex than > >> >>> basic flags. > >> >>> > >> >>> With argv argc, I assume we can pass these flags as options with > >> >>> getopt ? > >> >> > >> >> > >> >> These flags are defined by API used by application, so they are > >> >> hardcoded into application, but not passed from command line. If we > are > >> >> going to argv/argc way, then application can just insert one more > >> >> parameter into the list, before passing it to odp_init_global(). > >> >> > >> >>> Some issues are > >> >>> > >> >>> 1. passing things through to underlying SDKs > >> >>> 2. Making some args mandatory > >> >>> 3. Keeping the argument parsing in sync across platforms > >> >>> 4. Not coupling all the platforms and their special args together > >> >>> 5. Not promoting the use of platform specific args in general as > they > >> >>> will break portability. > >> >> > >> >> > >> >> I agree with an approach when only common platform independent > >> >> arguments > >> >> are passed to ODP application. We have to analyze which parameters > are > >> >> required by different platforms and come up with a common list. The > >> >> rest > >> >> platform specific parameters have to passed to implementation in some > >> >> other way (environment variable, config file, hardcoded, etc.) > >> > > >> > > >> > > >> > > >> > -- > >> > Mike Holmes > >> > Linaro Technical Manager / Lead > >> > LNG - ODP > >> > > >> > _______________________________________________ > >> > lng-odp mailing list > >> > lng-odp@lists.linaro.org > >> > http://lists.linaro.org/mailman/listinfo/lng-odp > >> > > >> > >> _______________________________________________ > >> lng-odp mailing list > >> lng-odp@lists.linaro.org > >> http://lists.linaro.org/mailman/listinfo/lng-odp > > > > >
We basically have three layers in the ODP framework Application, ODP and SDK. The Application layer needs arguments plus the SDK layer in the case of DPDK needs arguments. The ODP needs arguments or configuration at run time too. This is the reason for the patch file, just to show what I was suggesting. Here is the commit comment just so you do not need to open the file. Add the new ODP_INIT_GLOBAL() macro to hide some details. Which allows the arguments to be passed to the different levels. The levels are ODP, SDK and application using the getopt construct of '--' to delimit each layer. First arguments '-s foobar.lua' made up args for ODP layer. Second arguments '-c 3 -n 3' DPDK arguments layer. Third arguments '-i 0,1 -p 3' example args for applications layer. odp_foobar -s foobar.lua -- -c 3 -n 3 -- -i 0,1 -p 3 Always use '--' to delimit the layers. Now we have three different layers each being able to have its own flags and arguments. This way we do not need to register or make sure someone does not try to duplicate a argument. Thanks ++Keith Keith Wiles, Principal Technologist with CTO office, Wind River mobile 972-213-5533 [Powering 30 Years of Innovation]<http://www.windriver.com/announces/wr30/> On Aug 7, 2014, at 12:23 PM, Mike Holmes <mike.holmes@linaro.org<mailto:mike.holmes@linaro.org>> wrote: On 7 August 2014 13:20, Victor Kamensky <victor.kamensky@linaro.org<mailto:victor.kamensky@linaro.org>> wrote: On 7 August 2014 10:08, Bill Fischofer <bill.fischofer@linaro.org<mailto:bill.fischofer@linaro.org>> wrote: > Victor, did you have a specific set of API suggestions for this? I think > that this whole area should be part of the ODP v2.0 planning discussions > we'd like to have at LCU. It would be interesting to see if NFV has any > recommendations in this area since this is all part of hooking into the > larger management infrastructure. I think I am missing some context here. Why we are talking about argc/argv on thread where init flags are proposed? The context is the discussions in the meetings over the last day or two about platform compile time configuration, vs run time ODP configuration and runtime platform specific configuration, the two topics have become merged. And as far as original diff is concerned I don't see any sense in it. I consider ODP implementation as one unit of functionality and only one simple init call is needed to start it, maybe another call in threads created by user. Asking user to understand all these flags and meke it consistent between different implementation is very hard. Also what is about internal dependencies, i.e if I did not pass ODP_INIT_F_GPOOL but pass ODP_INIT_F_PKTIO (assuming that PKTIO depends on GPOOL) what will happen in this case? If there is a problem of cost of initializing certain resources it should not be driven by flags passed from user, but rather converted on first use initialization. Thanks, Victor > Bill > > > On Thu, Aug 7, 2014 at 12:02 PM, Victor Kamensky > <victor.kamensky@linaro.org<mailto:victor.kamensky@linaro.org>> wrote: >> >> On 7 August 2014 05:52, Mike Holmes <mike.holmes@linaro.org<mailto:mike.holmes@linaro.org>> wrote: >> > I failed to summarise yesterdays conversation, here is todays attempt. >> > >> > argv, arc is not in principle opposed by anyone. >> >> In what role? For functionality that involves serious >> production deployment they *must not* be used. >> The same goes to environment variables. All >> functionality must be available through APIs. >> >> If we are talking about some structure of >> argv,argc in ODP examples and test then >> it is fine. However it such context it is not >> clear why platforms have to be in agreement. >> >> Thanks, >> Victor >> >> > Need to push as much as possible into the implementation wherever >> > possible >> > Arguments should try very hard to be platform agnostic >> > Arguments for a platform need to be marked and passed through verbatim - >> > then duplication of existing flags on different sdks is ok and clear to >> > the >> > user. Maybe --dpdk followed by all the flags it expects. >> > VMs may need some differences passed in, so not all differences can be >> > compiled into the implementation. >> > Need to avoid the case of two platforms claiming the same argument >> > A strong case for scripting like Lua was not exposed in this discussion. >> > >> > Could I ask everyone to reply with any agnostic and platform specific >> > arguments they would like to see, maybe then we can form a clear picture >> > of >> > those that are needed. >> > >> > >> > >> > On 5 August 2014 08:31, Taras Kondratiuk <taras.kondratiuk@linaro.org<mailto:taras.kondratiuk@linaro.org>> >> > wrote: >> >> >> >> On 08/05/2014 02:59 PM, Mike Holmes wrote: >> >>> >> >>> How does this play in with Keiths resumed call for argv argc? Thread >> >>> = >> >>> "[lng-odp] Port of Pktgen to ODP" it looks like on two fronts we have >> >>> come back to the need to add args to init that are more complex than >> >>> basic flags. >> >>> >> >>> With argv argc, I assume we can pass these flags as options with >> >>> getopt ? >> >> >> >> >> >> These flags are defined by API used by application, so they are >> >> hardcoded into application, but not passed from command line. If we are >> >> going to argv/argc way, then application can just insert one more >> >> parameter into the list, before passing it to odp_init_global(). >> >> >> >>> Some issues are >> >>> >> >>> 1. passing things through to underlying SDKs >> >>> 2. Making some args mandatory >> >>> 3. Keeping the argument parsing in sync across platforms >> >>> 4. Not coupling all the platforms and their special args together >> >>> 5. Not promoting the use of platform specific args in general as they >> >>> will break portability. >> >> >> >> >> >> I agree with an approach when only common platform independent >> >> arguments >> >> are passed to ODP application. We have to analyze which parameters are >> >> required by different platforms and come up with a common list. The >> >> rest >> >> platform specific parameters have to passed to implementation in some >> >> other way (environment variable, config file, hardcoded, etc.) >> > >> > >> > >> > >> > -- >> > Mike Holmes >> > Linaro Technical Manager / Lead >> > LNG - ODP >> > >> > _______________________________________________ >> > lng-odp mailing list >> > lng-odp@lists.linaro.org<mailto:lng-odp@lists.linaro.org> >> > http://lists.linaro.org/mailman/listinfo/lng-odp >> > >> >> _______________________________________________ >> lng-odp mailing list >> lng-odp@lists.linaro.org<mailto:lng-odp@lists.linaro.org> >> http://lists.linaro.org/mailman/listinfo/lng-odp > > -- Mike Holmes Linaro Technical Manager / Lead LNG - ODP
On 7 August 2014 10:16, Mike Holmes <mike.holmes@linaro.org> wrote: > > > > On 7 August 2014 13:02, Victor Kamensky <victor.kamensky@linaro.org> wrote: >> >> On 7 August 2014 05:52, Mike Holmes <mike.holmes@linaro.org> wrote: >> > I failed to summarise yesterdays conversation, here is todays attempt. >> > >> > argv, arc is not in principle opposed by anyone. >> >> In what role? For functionality that involves serious >> production deployment they *must not* be used. >> The same goes to environment variables. All >> functionality must be available through APIs. > > If odp_global init is an API that takes argv and arg style arguments is that > ok, No, it is not OK. See below. >I am not sure what you mean by it must be via an API unless you mean > APIs that do not take variable argument lists I don't what odp_init_global take argc,argv it pushes meaning of action that has to be done into other dimension. I.e one need to look at API description and then look at list of possible argc/argv which described where? How portable are they? If you need to convey any information of odp_init_global define proper enumeration, structs etc and pass it to odp_init_global or other API. >> >> >> If we are talking about some structure of >> argv,argc in ODP examples and test then >> it is fine. However it such context it is not >> clear why platforms have to be in agreement. > > I think this is a discussion about how to pass things into the init of ODP > rather than examples, the recurring need to pass some platform specific data > though to the implementation. I think it is beyond of ODP platform independent specification. Implementation specific configuration could be delivered in variety of ways: o build time: platform header file with bunch of constants, which are defined by user of library o special implementation specific set of calls that app does in beginning. We can survive implementation specific ifdef in some portion of our code o config file that ODP implementation reads o ... In my opinion it is up to ODP implementer how to do that. > I also assume the case or platform agnostic > configuration that cannot be hard coded If there is a need for platform agnostic configuration in ODP we need to create proper API for that. That what I meant it should be APIs only. Pushing some meaning to set of strings IMHO is wrong way to go. > - I am imagining ODP running in an > Enterprise situation in multiple VMs where you want the same app attached to > different ports or something like that. Serious deployment situation is way more complicated configuration parameters do not always come from args/argv. And serious application has its own set of rules and convention how to handle argc/argv. Just simple example what if those configuration parameters should be changed on the fly of they hey come from config file. With array of strings API you telling me that I need to construct array of strings and pass it to some API ... I find it not very clean. If follow that direction why do we define enumeration for crypto protocols supported let's pass it as 'char *'. Thanks, Victor >> >> >> Thanks, >> Victor >> >> > Need to push as much as possible into the implementation wherever >> > possible >> > Arguments should try very hard to be platform agnostic >> > Arguments for a platform need to be marked and passed through verbatim - >> > then duplication of existing flags on different sdks is ok and clear to >> > the >> > user. Maybe --dpdk followed by all the flags it expects. >> > VMs may need some differences passed in, so not all differences can be >> > compiled into the implementation. >> > Need to avoid the case of two platforms claiming the same argument >> > A strong case for scripting like Lua was not exposed in this discussion. >> > >> > Could I ask everyone to reply with any agnostic and platform specific >> > arguments they would like to see, maybe then we can form a clear picture >> > of >> > those that are needed. >> > >> > >> > >> > On 5 August 2014 08:31, Taras Kondratiuk <taras.kondratiuk@linaro.org> >> > wrote: >> >> >> >> On 08/05/2014 02:59 PM, Mike Holmes wrote: >> >>> >> >>> How does this play in with Keiths resumed call for argv argc? Thread >> >>> = >> >>> "[lng-odp] Port of Pktgen to ODP" it looks like on two fronts we have >> >>> come back to the need to add args to init that are more complex than >> >>> basic flags. >> >>> >> >>> With argv argc, I assume we can pass these flags as options with >> >>> getopt ? >> >> >> >> >> >> These flags are defined by API used by application, so they are >> >> hardcoded into application, but not passed from command line. If we are >> >> going to argv/argc way, then application can just insert one more >> >> parameter into the list, before passing it to odp_init_global(). >> >> >> >>> Some issues are >> >>> >> >>> 1. passing things through to underlying SDKs >> >>> 2. Making some args mandatory >> >>> 3. Keeping the argument parsing in sync across platforms >> >>> 4. Not coupling all the platforms and their special args together >> >>> 5. Not promoting the use of platform specific args in general as they >> >>> will break portability. >> >> >> >> >> >> I agree with an approach when only common platform independent >> >> arguments >> >> are passed to ODP application. We have to analyze which parameters are >> >> required by different platforms and come up with a common list. The >> >> rest >> >> platform specific parameters have to passed to implementation in some >> >> other way (environment variable, config file, hardcoded, etc.) >> > >> > >> > >> > >> > -- >> > Mike Holmes >> > Linaro Technical Manager / Lead >> > LNG - ODP >> > >> > _______________________________________________ >> > lng-odp mailing list >> > lng-odp@lists.linaro.org >> > http://lists.linaro.org/mailman/listinfo/lng-odp >> > > > > > > -- > Mike Holmes > Linaro Technical Manager / Lead > LNG - ODP
On 7 August 2014 10:37, Wiles, Roger Keith <keith.wiles@windriver.com> wrote: > We basically have three layers in the ODP framework Application, ODP and > SDK. The Application layer needs arguments plus the SDK layer in the case > of DPDK needs arguments. The ODP needs arguments or configuration at run > time too. This is the reason for the patch file, just to show what I was > suggesting. > > Here is the commit comment just so you do not need to open the file. > > Add the new ODP_INIT_GLOBAL() macro to hide some details. Which > allows the arguments to be passed to the different levels. The > levels are ODP, SDK and application using the getopt construct > of '--' to delimit each layer. > > First arguments '-s foobar.lua' made up args for ODP layer. > Second arguments '-c 3 -n 3' DPDK arguments layer. > Third arguments '-i 0,1 -p 3' example args for applications layer. > > odp_foobar -s foobar.lua -- -c 3 -n 3 -- -i 0,1 -p 3 > > Always use '--' to delimit the layers. > > Now we have three different layers each being able to have its own flags > and arguments. This way we do not need to register or make sure someone > does not try to duplicate a argument. > IMO above is simplified method that is suitable for ODP example applications and tests. Please see my other responses. In production deployment ODP is just part of overall bigger system and such systems have itis own complicated set of rules, that we don't want to know. I am arguing that such system should be able to communicate with ODP through set of well defined structured APIs, rather than array of strings with some obscure meaning. I.e it seems to me that with argc/argv/ approach we are trying to create some mini language with its own semantics whereas normal structure C apis could be used. Thanks, Victor > Thanks > ++Keith > > *Keith **Wiles*, Principal Technologist with CTO office, *Wind River* > mobile 972-213-5533 > > [image: Powering 30 Years of Innovation] > <http://www.windriver.com/announces/wr30/> > > On Aug 7, 2014, at 12:23 PM, Mike Holmes <mike.holmes@linaro.org> wrote: > > > > > On 7 August 2014 13:20, Victor Kamensky <victor.kamensky@linaro.org> > wrote: > >> On 7 August 2014 10:08, Bill Fischofer <bill.fischofer@linaro.org> wrote: >> > Victor, did you have a specific set of API suggestions for this? I >> think >> > that this whole area should be part of the ODP v2.0 planning discussions >> > we'd like to have at LCU. It would be interesting to see if NFV has any >> > recommendations in this area since this is all part of hooking into the >> > larger management infrastructure. >> >> I think I am missing some context here. Why we are talking about >> argc/argv on thread where init flags are proposed? >> > > The context is the discussions in the meetings over the last day or two > about platform compile time configuration, vs run time ODP configuration > and runtime platform specific configuration, the two topics have become > merged. > >> >> And as far as original diff is concerned I don't see any sense in it. >> I consider ODP implementation as one unit of functionality and only >> one simple init call is needed to start it, maybe another call in threads >> created by user. Asking user to understand all these flags and meke >> it consistent between different implementation is very hard. Also what >> is about internal dependencies, i.e if I did not pass ODP_INIT_F_GPOOL >> but pass ODP_INIT_F_PKTIO (assuming that PKTIO depends on >> GPOOL) what will happen in this case? >> >> If there is a problem of cost of initializing certain resources it >> should not be driven by flags passed from user, but rather converted >> on first use initialization. >> >> Thanks, >> Victor >> >> > Bill >> > >> > >> > On Thu, Aug 7, 2014 at 12:02 PM, Victor Kamensky >> > <victor.kamensky@linaro.org> wrote: >> >> >> >> On 7 August 2014 05:52, Mike Holmes <mike.holmes@linaro.org> wrote: >> >> > I failed to summarise yesterdays conversation, here is todays >> attempt. >> >> > >> >> > argv, arc is not in principle opposed by anyone. >> >> >> >> In what role? For functionality that involves serious >> >> production deployment they *must not* be used. >> >> The same goes to environment variables. All >> >> functionality must be available through APIs. >> >> >> >> If we are talking about some structure of >> >> argv,argc in ODP examples and test then >> >> it is fine. However it such context it is not >> >> clear why platforms have to be in agreement. >> >> >> >> Thanks, >> >> Victor >> >> >> >> > Need to push as much as possible into the implementation wherever >> >> > possible >> >> > Arguments should try very hard to be platform agnostic >> >> > Arguments for a platform need to be marked and passed through >> verbatim - >> >> > then duplication of existing flags on different sdks is ok and clear >> to >> >> > the >> >> > user. Maybe --dpdk followed by all the flags it expects. >> >> > VMs may need some differences passed in, so not all differences can >> be >> >> > compiled into the implementation. >> >> > Need to avoid the case of two platforms claiming the same argument >> >> > A strong case for scripting like Lua was not exposed in this >> discussion. >> >> > >> >> > Could I ask everyone to reply with any agnostic and platform specific >> >> > arguments they would like to see, maybe then we can form a clear >> picture >> >> > of >> >> > those that are needed. >> >> > >> >> > >> >> > >> >> > On 5 August 2014 08:31, Taras Kondratiuk < >> taras.kondratiuk@linaro.org> >> >> > wrote: >> >> >> >> >> >> On 08/05/2014 02:59 PM, Mike Holmes wrote: >> >> >>> >> >> >>> How does this play in with Keiths resumed call for argv argc? >> Thread >> >> >>> = >> >> >>> "[lng-odp] Port of Pktgen to ODP" it looks like on two fronts we >> have >> >> >>> come back to the need to add args to init that are more complex >> than >> >> >>> basic flags. >> >> >>> >> >> >>> With argv argc, I assume we can pass these flags as options with >> >> >>> getopt ? >> >> >> >> >> >> >> >> >> These flags are defined by API used by application, so they are >> >> >> hardcoded into application, but not passed from command line. If we >> are >> >> >> going to argv/argc way, then application can just insert one more >> >> >> parameter into the list, before passing it to odp_init_global(). >> >> >> >> >> >>> Some issues are >> >> >>> >> >> >>> 1. passing things through to underlying SDKs >> >> >>> 2. Making some args mandatory >> >> >>> 3. Keeping the argument parsing in sync across platforms >> >> >>> 4. Not coupling all the platforms and their special args together >> >> >>> 5. Not promoting the use of platform specific args in general as >> they >> >> >>> will break portability. >> >> >> >> >> >> >> >> >> I agree with an approach when only common platform independent >> >> >> arguments >> >> >> are passed to ODP application. We have to analyze which parameters >> are >> >> >> required by different platforms and come up with a common list. The >> >> >> rest >> >> >> platform specific parameters have to passed to implementation in >> some >> >> >> other way (environment variable, config file, hardcoded, etc.) >> >> > >> >> > >> >> > >> >> > >> >> > -- >> >> > Mike Holmes >> >> > Linaro Technical Manager / Lead >> >> > LNG - ODP >> >> > >> >> > _______________________________________________ >> >> > lng-odp mailing list >> >> > lng-odp@lists.linaro.org >> >> > http://lists.linaro.org/mailman/listinfo/lng-odp >> >> > >> >> >> >> _______________________________________________ >> >> lng-odp mailing list >> >> lng-odp@lists.linaro.org >> >> http://lists.linaro.org/mailman/listinfo/lng-odp >> > >> > >> > > > > -- > *Mike Holmes* > Linaro Technical Manager / Lead > LNG - ODP > _______________________________________________ > lng-odp mailing list > lng-odp@lists.linaro.org > http://lists.linaro.org/mailman/listinfo/lng-odp > > >
Comments inline. Keith Wiles, Principal Technologist with CTO office, Wind River mobile 972-213-5533 [Powering 30 Years of Innovation]<http://www.windriver.com/announces/wr30/> On Aug 7, 2014, at 12:47 PM, Victor Kamensky <victor.kamensky@linaro.org<mailto:victor.kamensky@linaro.org>> wrote: On 7 August 2014 10:16, Mike Holmes <mike.holmes@linaro.org<mailto:mike.holmes@linaro.org>> wrote: On 7 August 2014 13:02, Victor Kamensky <victor.kamensky@linaro.org<mailto:victor.kamensky@linaro.org>> wrote: On 7 August 2014 05:52, Mike Holmes <mike.holmes@linaro.org<mailto:mike.holmes@linaro.org>> wrote: I failed to summarise yesterdays conversation, here is todays attempt. argv, arc is not in principle opposed by anyone. In what role? For functionality that involves serious production deployment they *must not* be used. The same goes to environment variables. All functionality must be available through APIs. If odp_global init is an API that takes argv and arg style arguments is that ok, No, it is not OK. See below. I am not sure what you mean by it must be via an API unless you mean APIs that do not take variable argument lists I don't what odp_init_global take argc,argv it pushes meaning of action that has to be done into other dimension. I.e one need to look at API description and then look at list of possible argc/argv which described where? How portable are they? The args for ODP should be pretty generic IMO, so we can list these in the comments or in the usage() /help function for the ODP layer. Maybe ODP does not require args, but we have began to define flags and that is just another form of arguments. These flags can be passed on the command line or a init routine developed by the developer can construct a argc/argv list. These are just for init time not run time changes. If the system needs to have runtime configuration then API’s should be created. We do not need to have every options or runtime configuration on the command line, but create API’s to allow them to be changed through ODP to the SDK. If you need to convey any information of odp_init_global define proper enumeration, structs etc and pass it to odp_init_global or other API. Using enums or flags or argc/argv provides the same basic result and command line args are reasonable as most of the platforms here run on Linux. Now ODP can provide a routine to convert some type of args to argc/argv format which is very simple do. If we are talking about some structure of argv,argc in ODP examples and test then it is fine. However it such context it is not clear why platforms have to be in agreement. I think this is a discussion about how to pass things into the init of ODP rather than examples, the recurring need to pass some platform specific data though to the implementation. I think it is beyond of ODP platform independent specification. Implementation specific configuration could be delivered in variety of ways: Yes, Implementation configuration can be done is a multiple ways, but the point is we do need and see a requirement for being able to apply arguments to all three layer in a application — Application layer — ODP layer — SDK layer each of these layers require a basic way to provide configuration at build time, init time and run time. Runtime is for configuration that needs to be define on the fly and init time configuration is for system initialization only in most case and does not change normally after initialization. o build time: platform header file with bunch of constants, which are defined by user of library o special implementation specific set of calls that app does in beginning. We can survive implementation specific ifdef in some portion of our code o config file that ODP implementation reads A config file requires a filesystem, but let ignore that point for now. If we did provide a init file for ODP then we can provide a ‘-f foobar.config’ option on the command line. Another option is the application open and parse the config file converting them into argc/argv options to pass to odp_init_global() routine. o ... In my opinion it is up to ODP implementer how to do that. I do see some of the configuration must be defined by the developer, but we are starting to see some common threads and configuration we can supply today. I also assume the case or platform agnostic configuration that cannot be hard coded If there is a need for platform agnostic configuration in ODP we need to create proper API for that. That what I meant it should be APIs only. Pushing some meaning to set of strings IMHO is wrong way to go. I do not see how we are not making this platform agnostic as we still allow the SDK and application to have its own configuration without having to alter ODP for each platform. I am seeing a lot of duplicate code in each of the platforms we have today and if you look at the code I broke out the DPDK routine odp_init_dpdk() into odp_init_skd() instead, which now should allow most of the ODP code for initing system to be generic and common between the platforms. I did not fully include those changes as it a bit more work. - I am imagining ODP running in an Enterprise situation in multiple VMs where you want the same app attached to different ports or something like that. Serious deployment situation is way more complicated configuration parameters do not always come from args/argv. And serious application has its own set of rules and convention how to handle argc/argv. Just simple example what if those configuration parameters should be changed on the fly of they hey come from config file. With array of strings API you telling me that I need to construct array of strings and pass it to some API ... I find it not very clean. If follow that direction why do we define enumeration for crypto protocols supported let's pass it as 'char *’. I have developed and design a number of systems here are WR and you are correct that a product needs a lot of work to be a real product. My point is we can help reduce the workload of the developer by helping define at least a basic configuration system he can build on for his product. As for the crypto protocols I have not looked at them, but I think you are taking an extreme view here. Do not mean to offend anyone, but I do believe we need a basic configuration in all three areas for ODP. Thanks, Victor Thanks, Victor Need to push as much as possible into the implementation wherever possible Arguments should try very hard to be platform agnostic Arguments for a platform need to be marked and passed through verbatim - then duplication of existing flags on different sdks is ok and clear to the user. Maybe --dpdk followed by all the flags it expects. VMs may need some differences passed in, so not all differences can be compiled into the implementation. Need to avoid the case of two platforms claiming the same argument A strong case for scripting like Lua was not exposed in this discussion. Could I ask everyone to reply with any agnostic and platform specific arguments they would like to see, maybe then we can form a clear picture of those that are needed. On 5 August 2014 08:31, Taras Kondratiuk <taras.kondratiuk@linaro.org<mailto:taras.kondratiuk@linaro.org>> wrote: On 08/05/2014 02:59 PM, Mike Holmes wrote: How does this play in with Keiths resumed call for argv argc? Thread = "[lng-odp] Port of Pktgen to ODP" it looks like on two fronts we have come back to the need to add args to init that are more complex than basic flags. With argv argc, I assume we can pass these flags as options with getopt ? These flags are defined by API used by application, so they are hardcoded into application, but not passed from command line. If we are going to argv/argc way, then application can just insert one more parameter into the list, before passing it to odp_init_global(). Some issues are 1. passing things through to underlying SDKs 2. Making some args mandatory 3. Keeping the argument parsing in sync across platforms 4. Not coupling all the platforms and their special args together 5. Not promoting the use of platform specific args in general as they will break portability. I agree with an approach when only common platform independent arguments are passed to ODP application. We have to analyze which parameters are required by different platforms and come up with a common list. The rest platform specific parameters have to passed to implementation in some other way (environment variable, config file, hardcoded, etc.) -- Mike Holmes Linaro Technical Manager / Lead LNG - ODP
On 08/07/2014 08:16 PM, Mike Holmes wrote: > > On 7 August 2014 13:02, Victor Kamensky <victor.kamensky@linaro.org > <mailto:victor.kamensky@linaro.org>> wrote: > > On 7 August 2014 05:52, Mike Holmes <mike.holmes@linaro.org > <mailto:mike.holmes@linaro.org>> wrote: > > I failed to summarise yesterdays conversation, here is todays > attempt. > > > > argv, arc is not in principle opposed by anyone. > > In what role? For functionality that involves serious > production deployment they *must not* be used. > The same goes to environment variables. All > functionality must be available through APIs. > > If odp_global init is an API that takes argv and arg style arguments is > that ok, I am not sure what you mean by it must be via an API unless you > mean APIs that do not take variable argument lists > > > If we are talking about some structure of > argv,argc in ODP examples and test then > it is fine. However it such context it is not > clear why platforms have to be in agreement. > > I think this is a discussion about how to pass things into the init of > ODP rather than examples, the recurring need to pass some platform > specific data though to the implementation. I also assume the case or > platform agnostic configuration that cannot be hard coded - I am > imagining ODP running in an Enterprise situation in multiple VMs where > you want the same app attached to different ports or something like that. Ports is a part of a common set of configuration parameters. And currently they are not passed to odp_init, but to odp_pktio_open(). What are other parameters that need to configured in runtime?
Mixing init mask and passing additional info is not really good. We actually need both. Some default configuration should work with all init layers turned on and no arguments provided. If some more deep tunning is required user can add additional args. tilera use "--" in cmline of tile-monitor in about the same way. And there might be conflict with also using "--". Other than that I like that approach. Maxim. On 08/07/2014 10:00 PM, Victor Kamensky wrote: > > > > On 7 August 2014 10:37, Wiles, Roger Keith <keith.wiles@windriver.com > <mailto:keith.wiles@windriver.com>> wrote: > > We basically have three layers in the ODP framework Application, > ODP and SDK. The Application layer needs arguments plus the SDK > layer in the case of DPDK needs arguments. The ODP needs arguments > or configuration at run time too. This is the reason for the patch > file, just to show what I was suggesting. > > Here is the commit comment just so you do not need to open the file. > > Add the new ODP_INIT_GLOBAL() macro to hide some details. Which > allows the arguments to be passed to the different levels. The > levels are ODP, SDK and application using the getopt construct > of '--' to delimit each layer. > > First arguments '-s foobar.lua' made up args for ODP layer. > Second arguments '-c 3 -n 3' DPDK arguments layer. > Third arguments '-i 0,1 -p 3' example args for applications layer. > > odp_foobar -s foobar.lua -- -c 3 -n 3 -- -i 0,1 -p 3 > > Always use '--' to delimit the layers. > > Now we have three different layers each being able to have its own > flags and arguments. This way we do not need to register or make > sure someone does not try to duplicate a argument. > > > IMO above is simplified method that is suitable for ODP > example applications and tests. Please see my other > responses. In production deployment ODP is just part > of overall bigger system and such systems have itis own > complicated set of rules, that we don't want to know. > I am arguing that such system should be able to communicate > with ODP through set of well defined structured APIs, rather > than array of strings with some obscure meaning. I.e > it seems to me that with argc/argv/ approach we are > trying to create some mini language with its own > semantics whereas normal structure C apis could be used. > > Thanks, > Victor > > Thanks > ++Keith > > *Keith **Wiles*, Principal Technologist with CTO office, *Wind River* > mobile 972-213-5533 <tel:972-213-5533> > > Powering 30 Years of Innovation > <http://www.windriver.com/announces/wr30/> > > On Aug 7, 2014, at 12:23 PM, Mike Holmes <mike.holmes@linaro.org > <mailto:mike.holmes@linaro.org>> wrote: > >> >> >> >> On 7 August 2014 13:20, Victor Kamensky >> <victor.kamensky@linaro.org <mailto:victor.kamensky@linaro.org>> >> wrote: >> >> On 7 August 2014 10:08, Bill Fischofer >> <bill.fischofer@linaro.org >> <mailto:bill.fischofer@linaro.org>> wrote: >> > Victor, did you have a specific set of API suggestions for >> this? I think >> > that this whole area should be part of the ODP v2.0 >> planning discussions >> > we'd like to have at LCU. It would be interesting to see >> if NFV has any >> > recommendations in this area since this is all part of >> hooking into the >> > larger management infrastructure. >> >> I think I am missing some context here. Why we are talking about >> argc/argv on thread where init flags are proposed? >> >> >> The context is the discussions in the meetings over the last day >> or two about platform compile time configuration, vs run time ODP >> configuration and runtime platform specific configuration, the >> two topics have become merged. >> >> >> And as far as original diff is concerned I don't see any >> sense in it. >> I consider ODP implementation as one unit of functionality >> and only >> one simple init call is needed to start it, maybe another >> call in threads >> created by user. Asking user to understand all these flags >> and meke >> it consistent between different implementation is very hard. >> Also what >> is about internal dependencies, i.e if I did not pass >> ODP_INIT_F_GPOOL >> but pass ODP_INIT_F_PKTIO (assuming that PKTIO depends on >> GPOOL) what will happen in this case? >> >> If there is a problem of cost of initializing certain >> resources it >> should not be driven by flags passed from user, but rather >> converted >> on first use initialization. >> >> Thanks, >> Victor >> >> > Bill >> > >> > >> > On Thu, Aug 7, 2014 at 12:02 PM, Victor Kamensky >> > <victor.kamensky@linaro.org >> <mailto:victor.kamensky@linaro.org>> wrote: >> >> >> >> On 7 August 2014 05:52, Mike Holmes >> <mike.holmes@linaro.org <mailto:mike.holmes@linaro.org>> wrote: >> >> > I failed to summarise yesterdays conversation, here is >> todays attempt. >> >> > >> >> > argv, arc is not in principle opposed by anyone. >> >> >> >> In what role? For functionality that involves serious >> >> production deployment they *must not* be used. >> >> The same goes to environment variables. All >> >> functionality must be available through APIs. >> >> >> >> If we are talking about some structure of >> >> argv,argc in ODP examples and test then >> >> it is fine. However it such context it is not >> >> clear why platforms have to be in agreement. >> >> >> >> Thanks, >> >> Victor >> >> >> >> > Need to push as much as possible into the implementation >> wherever >> >> > possible >> >> > Arguments should try very hard to be platform agnostic >> >> > Arguments for a platform need to be marked and passed >> through verbatim - >> >> > then duplication of existing flags on different sdks is >> ok and clear to >> >> > the >> >> > user. Maybe --dpdk followed by all the flags it expects. >> >> > VMs may need some differences passed in, so not all >> differences can be >> >> > compiled into the implementation. >> >> > Need to avoid the case of two platforms claiming the >> same argument >> >> > A strong case for scripting like Lua was not exposed in >> this discussion. >> >> > >> >> > Could I ask everyone to reply with any agnostic and >> platform specific >> >> > arguments they would like to see, maybe then we can form >> a clear picture >> >> > of >> >> > those that are needed. >> >> > >> >> > >> >> > >> >> > On 5 August 2014 08:31, Taras Kondratiuk >> <taras.kondratiuk@linaro.org >> <mailto:taras.kondratiuk@linaro.org>> >> >> > wrote: >> >> >> >> >> >> On 08/05/2014 02:59 PM, Mike Holmes wrote: >> >> >>> >> >> >>> How does this play in with Keiths resumed call for >> argv argc? Thread >> >> >>> = >> >> >>> "[lng-odp] Port of Pktgen to ODP" it looks like on >> two fronts we have >> >> >>> come back to the need to add args to init that are >> more complex than >> >> >>> basic flags. >> >> >>> >> >> >>> With argv argc, I assume we can pass these flags as >> options with >> >> >>> getopt ? >> >> >> >> >> >> >> >> >> These flags are defined by API used by application, so >> they are >> >> >> hardcoded into application, but not passed from command >> line. If we are >> >> >> going to argv/argc way, then application can just >> insert one more >> >> >> parameter into the list, before passing it to >> odp_init_global(). >> >> >> >> >> >>> Some issues are >> >> >>> >> >> >>> 1. passing things through to underlying SDKs >> >> >>> 2. Making some args mandatory >> >> >>> 3. Keeping the argument parsing in sync across platforms >> >> >>> 4. Not coupling all the platforms and their special >> args together >> >> >>> 5. Not promoting the use of platform specific args in >> general as they >> >> >>> will break portability. >> >> >> >> >> >> >> >> >> I agree with an approach when only common platform >> independent >> >> >> arguments >> >> >> are passed to ODP application. We have to analyze which >> parameters are >> >> >> required by different platforms and come up with a >> common list. The >> >> >> rest >> >> >> platform specific parameters have to passed to >> implementation in some >> >> >> other way (environment variable, config file, >> hardcoded, etc.) >> >> > >> >> > >> >> > >> >> > >> >> > -- >> >> > Mike Holmes >> >> > Linaro Technical Manager / Lead >> >> > LNG - ODP >> >> > >> >> > _______________________________________________ >> >> > lng-odp mailing list >> >> > lng-odp@lists.linaro.org <mailto:lng-odp@lists.linaro.org> >> >> > http://lists.linaro.org/mailman/listinfo/lng-odp >> >> > >> >> >> >> _______________________________________________ >> >> lng-odp mailing list >> >> lng-odp@lists.linaro.org <mailto:lng-odp@lists.linaro.org> >> >> http://lists.linaro.org/mailman/listinfo/lng-odp >> > >> > >> >> >> >> >> -- >> *Mike Holmes* >> Linaro Technical Manager / Lead >> LNG - ODP >> _______________________________________________ >> lng-odp mailing list >> lng-odp@lists.linaro.org <mailto:lng-odp@lists.linaro.org> >> http://lists.linaro.org/mailman/listinfo/lng-odp > > > > > _______________________________________________ > lng-odp mailing list > lng-odp@lists.linaro.org > http://lists.linaro.org/mailman/listinfo/lng-odp
Comments inline. Keith Wiles, Principal Technologist with CTO office, Wind River mobile 972-213-5533 [Powering 30 Years of Innovation]<http://www.windriver.com/announces/wr30/> On Aug 7, 2014, at 1:00 PM, Victor Kamensky <victor.kamensky@linaro.org<mailto:victor.kamensky@linaro.org>> wrote: On 7 August 2014 10:37, Wiles, Roger Keith <keith.wiles@windriver.com<mailto:keith.wiles@windriver.com>> wrote: We basically have three layers in the ODP framework Application, ODP and SDK. The Application layer needs arguments plus the SDK layer in the case of DPDK needs arguments. The ODP needs arguments or configuration at run time too. This is the reason for the patch file, just to show what I was suggesting. Here is the commit comment just so you do not need to open the file. Add the new ODP_INIT_GLOBAL() macro to hide some details. Which allows the arguments to be passed to the different levels. The levels are ODP, SDK and application using the getopt construct of '--' to delimit each layer. First arguments '-s foobar.lua' made up args for ODP layer. Second arguments '-c 3 -n 3' DPDK arguments layer. Third arguments '-i 0,1 -p 3' example args for applications layer. odp_foobar -s foobar.lua -- -c 3 -n 3 -- -i 0,1 -p 3 Always use '--' to delimit the layers. Now we have three different layers each being able to have its own flags and arguments. This way we do not need to register or make sure someone does not try to duplicate a argument. IMO above is simplified method that is suitable for ODP example applications and tests. Please see my other responses. In production deployment ODP is just part of overall bigger system and such systems have itis own complicated set of rules, that we don't want to know. I am arguing that such system should be able to communicate with ODP through set of well defined structured APIs, rather than array of strings with some obscure meaning. I.e it seems to me that with argc/argv/ approach we are trying to create some mini language with its own semantics whereas normal structure C apis could be used. ODP is a basic framework we have to define and setup the system from a simple configuration point of view as we can not see what each product will need in the future. It is very possible the SDK or APPLICATION layer does not require arguments and would load those arguments from a config file or provide a set of API’s to configure these layers. This is still possible as the SDK and application would ignore and pass these args to the next layer. In all of the application now in ODP we have arguments for the application and in the DPDK argc/argv is constructed in the ‘C’ and is not usable in that design except at a very basic level. I agree for most non-initialization time configuration we should use API’s, but for the basic system we can use argc/argv. I would like to see your suggestions as to how we can design ODP. Thanks, Victor Thanks ++Keith Keith Wiles, Principal Technologist with CTO office, Wind River mobile 972-213-5533<tel:972-213-5533> [Powering 30 Years of Innovation]<http://www.windriver.com/announces/wr30/> On Aug 7, 2014, at 12:23 PM, Mike Holmes <mike.holmes@linaro.org<mailto:mike.holmes@linaro.org>> wrote: On 7 August 2014 13:20, Victor Kamensky <victor.kamensky@linaro.org<mailto:victor.kamensky@linaro.org>> wrote: On 7 August 2014 10:08, Bill Fischofer <bill.fischofer@linaro.org<mailto:bill.fischofer@linaro.org>> wrote: > Victor, did you have a specific set of API suggestions for this? I think > that this whole area should be part of the ODP v2.0 planning discussions > we'd like to have at LCU. It would be interesting to see if NFV has any > recommendations in this area since this is all part of hooking into the > larger management infrastructure. I think I am missing some context here. Why we are talking about argc/argv on thread where init flags are proposed? The context is the discussions in the meetings over the last day or two about platform compile time configuration, vs run time ODP configuration and runtime platform specific configuration, the two topics have become merged. And as far as original diff is concerned I don't see any sense in it. I consider ODP implementation as one unit of functionality and only one simple init call is needed to start it, maybe another call in threads created by user. Asking user to understand all these flags and meke it consistent between different implementation is very hard. Also what is about internal dependencies, i.e if I did not pass ODP_INIT_F_GPOOL but pass ODP_INIT_F_PKTIO (assuming that PKTIO depends on GPOOL) what will happen in this case? If there is a problem of cost of initializing certain resources it should not be driven by flags passed from user, but rather converted on first use initialization. Thanks, Victor > Bill > > > On Thu, Aug 7, 2014 at 12:02 PM, Victor Kamensky > <victor.kamensky@linaro.org<mailto:victor.kamensky@linaro.org>> wrote: >> >> On 7 August 2014 05:52, Mike Holmes <mike.holmes@linaro.org<mailto:mike.holmes@linaro.org>> wrote: >> > I failed to summarise yesterdays conversation, here is todays attempt. >> > >> > argv, arc is not in principle opposed by anyone. >> >> In what role? For functionality that involves serious >> production deployment they *must not* be used. >> The same goes to environment variables. All >> functionality must be available through APIs. >> >> If we are talking about some structure of >> argv,argc in ODP examples and test then >> it is fine. However it such context it is not >> clear why platforms have to be in agreement. >> >> Thanks, >> Victor >> >> > Need to push as much as possible into the implementation wherever >> > possible >> > Arguments should try very hard to be platform agnostic >> > Arguments for a platform need to be marked and passed through verbatim - >> > then duplication of existing flags on different sdks is ok and clear to >> > the >> > user. Maybe --dpdk followed by all the flags it expects. >> > VMs may need some differences passed in, so not all differences can be >> > compiled into the implementation. >> > Need to avoid the case of two platforms claiming the same argument >> > A strong case for scripting like Lua was not exposed in this discussion. >> > >> > Could I ask everyone to reply with any agnostic and platform specific >> > arguments they would like to see, maybe then we can form a clear picture >> > of >> > those that are needed. >> > >> > >> > >> > On 5 August 2014 08:31, Taras Kondratiuk <taras.kondratiuk@linaro.org<mailto:taras.kondratiuk@linaro.org>> >> > wrote: >> >> >> >> On 08/05/2014 02:59 PM, Mike Holmes wrote: >> >>> >> >>> How does this play in with Keiths resumed call for argv argc? Thread >> >>> = >> >>> "[lng-odp] Port of Pktgen to ODP" it looks like on two fronts we have >> >>> come back to the need to add args to init that are more complex than >> >>> basic flags. >> >>> >> >>> With argv argc, I assume we can pass these flags as options with >> >>> getopt ? >> >> >> >> >> >> These flags are defined by API used by application, so they are >> >> hardcoded into application, but not passed from command line. If we are >> >> going to argv/argc way, then application can just insert one more >> >> parameter into the list, before passing it to odp_init_global(). >> >> >> >>> Some issues are >> >>> >> >>> 1. passing things through to underlying SDKs >> >>> 2. Making some args mandatory >> >>> 3. Keeping the argument parsing in sync across platforms >> >>> 4. Not coupling all the platforms and their special args together >> >>> 5. Not promoting the use of platform specific args in general as they >> >>> will break portability. >> >> >> >> >> >> I agree with an approach when only common platform independent >> >> arguments >> >> are passed to ODP application. We have to analyze which parameters are >> >> required by different platforms and come up with a common list. The >> >> rest >> >> platform specific parameters have to passed to implementation in some >> >> other way (environment variable, config file, hardcoded, etc.) >> > >> > >> > >> > >> > -- >> > Mike Holmes >> > Linaro Technical Manager / Lead >> > LNG - ODP >> > >> > _______________________________________________ >> > lng-odp mailing list >> > lng-odp@lists.linaro.org<mailto:lng-odp@lists.linaro.org> >> > http://lists.linaro.org/mailman/listinfo/lng-odp >> > >> >> _______________________________________________ >> lng-odp mailing list >> lng-odp@lists.linaro.org<mailto:lng-odp@lists.linaro.org> >> http://lists.linaro.org/mailman/listinfo/lng-odp > > -- Mike Holmes Linaro Technical Manager / Lead LNG - ODP
On 08/07/2014 10:21 PM, Taras Kondratiuk wrote: > On 08/07/2014 08:16 PM, Mike Holmes wrote: >> >> On 7 August 2014 13:02, Victor Kamensky <victor.kamensky@linaro.org >> <mailto:victor.kamensky@linaro.org>> wrote: >> >> On 7 August 2014 05:52, Mike Holmes <mike.holmes@linaro.org >> <mailto:mike.holmes@linaro.org>> wrote: >> > I failed to summarise yesterdays conversation, here is todays >> attempt. >> > >> > argv, arc is not in principle opposed by anyone. >> >> In what role? For functionality that involves serious >> production deployment they *must not* be used. >> The same goes to environment variables. All >> functionality must be available through APIs. >> >> If odp_global init is an API that takes argv and arg style arguments is >> that ok, I am not sure what you mean by it must be via an API unless you >> mean APIs that do not take variable argument lists >> >> >> If we are talking about some structure of >> argv,argc in ODP examples and test then >> it is fine. However it such context it is not >> clear why platforms have to be in agreement. >> >> I think this is a discussion about how to pass things into the init of >> ODP rather than examples, the recurring need to pass some platform >> specific data though to the implementation. I also assume the case or >> platform agnostic configuration that cannot be hard coded - I am >> imagining ODP running in an Enterprise situation in multiple VMs where >> you want the same app attached to different ports or something like >> that. > > Ports is a part of a common set of configuration parameters. And > currently they are not passed to odp_init, but to odp_pktio_open(). > > What are other parameters that need to configured in runtime? Numa? Packet i/o type for linux-generic (netmap, mmap, raw sockets); For K2 you can switch from CMA back to mmap if needed; Might be some hw modes if platform can support several... > > > _______________________________________________ > lng-odp mailing list > lng-odp@lists.linaro.org > http://lists.linaro.org/mailman/listinfo/lng-odp
I guess another option would be to remove the ‘—‘ delimiter for something like this:
odp_foobar —app-args=“-i 0,1 -p 3" —odp-args=“-s foobar.lua” —sdk-args=“-c 3 -n3”
But this is just a suggestion to help with requiring ‘—‘ for a delimiter. I could live with one or the other,but this above does help with layer that do not have args and then order is not required.
Thanks
++Keith
Keith Wiles, Principal Technologist with CTO office, Wind River
mobile 972-213-5533
[Powering 30 Years of Innovation]<http://www.windriver.com/announces/wr30/>
On Aug 7, 2014, at 1:23 PM, Maxim Uvarov <maxim.uvarov@linaro.org<mailto:maxim.uvarov@linaro.org>> wrote:
Mixing init mask and passing additional info is not really good. We actually need both. Some default configuration
should work with all init layers turned on and no arguments provided. If some more deep tunning is required user can
add additional args.
tilera use "--" in cmline of tile-monitor in about the same way. And there might be conflict with also using "--".
Other than that I like that approach.
Maxim.
On 08/07/2014 10:00 PM, Victor Kamensky wrote:
On 7 August 2014 10:37, Wiles, Roger Keith <keith.wiles@windriver.com<mailto:keith.wiles@windriver.com><mailto:keith.wiles@windriver.com>> wrote:
We basically have three layers in the ODP framework Application,
ODP and SDK. The Application layer needs arguments plus the SDK
layer in the case of DPDK needs arguments. The ODP needs arguments
or configuration at run time too. This is the reason for the patch
file, just to show what I was suggesting.
Here is the commit comment just so you do not need to open the file.
Add the new ODP_INIT_GLOBAL() macro to hide some details. Which
allows the arguments to be passed to the different levels. The
levels are ODP, SDK and application using the getopt construct
of '--' to delimit each layer.
First arguments '-s foobar.lua' made up args for ODP layer.
Second arguments '-c 3 -n 3' DPDK arguments layer.
Third arguments '-i 0,1 -p 3' example args for applications layer.
odp_foobar -s foobar.lua -- -c 3 -n 3 -- -i 0,1 -p 3
Always use '--' to delimit the layers.
Now we have three different layers each being able to have its own
flags and arguments. This way we do not need to register or make
sure someone does not try to duplicate a argument.
IMO above is simplified method that is suitable for ODP
example applications and tests. Please see my other
responses. In production deployment ODP is just part
of overall bigger system and such systems have itis own
complicated set of rules, that we don't want to know.
I am arguing that such system should be able to communicate
with ODP through set of well defined structured APIs, rather
than array of strings with some obscure meaning. I.e
it seems to me that with argc/argv/ approach we are
trying to create some mini language with its own
semantics whereas normal structure C apis could be used.
Thanks,
Victor
Thanks
++Keith
*Keith **Wiles*, Principal Technologist with CTO office, *Wind River*
mobile 972-213-5533 <tel:972-213-5533>
Powering 30 Years of Innovation
<http://www.windriver.com/announces/wr30/>
On Aug 7, 2014, at 12:23 PM, Mike Holmes <mike.holmes@linaro.org<mailto:mike.holmes@linaro.org>
<mailto:mike.holmes@linaro.org>> wrote:
On 7 August 2014 13:20, Victor Kamensky
<victor.kamensky@linaro.org<mailto:victor.kamensky@linaro.org> <mailto:victor.kamensky@linaro.org>>
wrote:
On 7 August 2014 10:08, Bill Fischofer
<bill.fischofer@linaro.org<mailto:bill.fischofer@linaro.org>
<mailto:bill.fischofer@linaro.org>> wrote:
> Victor, did you have a specific set of API suggestions for
this? I think
> that this whole area should be part of the ODP v2.0
planning discussions
> we'd like to have at LCU. It would be interesting to see
if NFV has any
> recommendations in this area since this is all part of
hooking into the
> larger management infrastructure.
I think I am missing some context here. Why we are talking about
argc/argv on thread where init flags are proposed?
The context is the discussions in the meetings over the last day
or two about platform compile time configuration, vs run time ODP
configuration and runtime platform specific configuration, the
two topics have become merged.
And as far as original diff is concerned I don't see any
sense in it.
I consider ODP implementation as one unit of functionality
and only
one simple init call is needed to start it, maybe another
call in threads
created by user. Asking user to understand all these flags
and meke
it consistent between different implementation is very hard.
Also what
is about internal dependencies, i.e if I did not pass
ODP_INIT_F_GPOOL
but pass ODP_INIT_F_PKTIO (assuming that PKTIO depends on
GPOOL) what will happen in this case?
If there is a problem of cost of initializing certain
resources it
should not be driven by flags passed from user, but rather
converted
on first use initialization.
Thanks,
Victor
> Bill
>
>
> On Thu, Aug 7, 2014 at 12:02 PM, Victor Kamensky
> <victor.kamensky@linaro.org<mailto:victor.kamensky@linaro.org>
<mailto:victor.kamensky@linaro.org>> wrote:
>>
>> On 7 August 2014 05:52, Mike Holmes
<mike.holmes@linaro.org<mailto:mike.holmes@linaro.org> <mailto:mike.holmes@linaro.org>> wrote:
>> > I failed to summarise yesterdays conversation, here is
todays attempt.
>> >
>> > argv, arc is not in principle opposed by anyone.
>>
>> In what role? For functionality that involves serious
>> production deployment they *must not* be used.
>> The same goes to environment variables. All
>> functionality must be available through APIs.
>>
>> If we are talking about some structure of
>> argv,argc in ODP examples and test then
>> it is fine. However it such context it is not
>> clear why platforms have to be in agreement.
>>
>> Thanks,
>> Victor
>>
>> > Need to push as much as possible into the implementation
wherever
>> > possible
>> > Arguments should try very hard to be platform agnostic
>> > Arguments for a platform need to be marked and passed
through verbatim -
>> > then duplication of existing flags on different sdks is
ok and clear to
>> > the
>> > user. Maybe --dpdk followed by all the flags it expects.
>> > VMs may need some differences passed in, so not all
differences can be
>> > compiled into the implementation.
>> > Need to avoid the case of two platforms claiming the
same argument
>> > A strong case for scripting like Lua was not exposed in
this discussion.
>> >
>> > Could I ask everyone to reply with any agnostic and
platform specific
>> > arguments they would like to see, maybe then we can form
a clear picture
>> > of
>> > those that are needed.
>> >
>> >
>> >
>> > On 5 August 2014 08:31, Taras Kondratiuk
<taras.kondratiuk@linaro.org<mailto:taras.kondratiuk@linaro.org>
<mailto:taras.kondratiuk@linaro.org>>
>> > wrote:
>> >>
>> >> On 08/05/2014 02:59 PM, Mike Holmes wrote:
>> >>>
>> >>> How does this play in with Keiths resumed call for
argv argc? Thread
>> >>> =
>> >>> "[lng-odp] Port of Pktgen to ODP" it looks like on
two fronts we have
>> >>> come back to the need to add args to init that are
more complex than
>> >>> basic flags.
>> >>>
>> >>> With argv argc, I assume we can pass these flags as
options with
>> >>> getopt ?
>> >>
>> >>
>> >> These flags are defined by API used by application, so
they are
>> >> hardcoded into application, but not passed from command
line. If we are
>> >> going to argv/argc way, then application can just
insert one more
>> >> parameter into the list, before passing it to
odp_init_global().
>> >>
>> >>> Some issues are
>> >>>
>> >>> 1. passing things through to underlying SDKs
>> >>> 2. Making some args mandatory
>> >>> 3. Keeping the argument parsing in sync across platforms
>> >>> 4. Not coupling all the platforms and their special
args together
>> >>> 5. Not promoting the use of platform specific args in
general as they
>> >>> will break portability.
>> >>
>> >>
>> >> I agree with an approach when only common platform
independent
>> >> arguments
>> >> are passed to ODP application. We have to analyze which
parameters are
>> >> required by different platforms and come up with a
common list. The
>> >> rest
>> >> platform specific parameters have to passed to
implementation in some
>> >> other way (environment variable, config file,
hardcoded, etc.)
>> >
>> >
>> >
>> >
>> > --
>> > Mike Holmes
>> > Linaro Technical Manager / Lead
>> > LNG - ODP
>> >
>> > _______________________________________________
>> > lng-odp mailing list
>> > lng-odp@lists.linaro.org<mailto:lng-odp@lists.linaro.org> <mailto:lng-odp@lists.linaro.org>
>> > http://lists.linaro.org/mailman/listinfo/lng-odp
>> >
>>
>> _______________________________________________
>> lng-odp mailing list
>> lng-odp@lists.linaro.org<mailto:lng-odp@lists.linaro.org> <mailto:lng-odp@lists.linaro.org>
>> http://lists.linaro.org/mailman/listinfo/lng-odp
>
>
-- *Mike Holmes*
Linaro Technical Manager / Lead
LNG - ODP
_______________________________________________
lng-odp mailing list
lng-odp@lists.linaro.org<mailto:lng-odp@lists.linaro.org> <mailto:lng-odp@lists.linaro.org>
http://lists.linaro.org/mailman/listinfo/lng-odp
Keith Wiles, Principal Technologist with CTO office, Wind River mobile 972-213-5533 [Powering 30 Years of Innovation]<http://www.windriver.com/announces/wr30/> On Aug 7, 2014, at 1:29 PM, Maxim Uvarov <maxim.uvarov@linaro.org<mailto:maxim.uvarov@linaro.org>> wrote: On 08/07/2014 10:21 PM, Taras Kondratiuk wrote: On 08/07/2014 08:16 PM, Mike Holmes wrote: On 7 August 2014 13:02, Victor Kamensky <victor.kamensky@linaro.org<mailto:victor.kamensky@linaro.org> <mailto:victor.kamensky@linaro.org>> wrote: On 7 August 2014 05:52, Mike Holmes <mike.holmes@linaro.org<mailto:mike.holmes@linaro.org> <mailto:mike.holmes@linaro.org>> wrote: > I failed to summarise yesterdays conversation, here is todays attempt. > > argv, arc is not in principle opposed by anyone. In what role? For functionality that involves serious production deployment they *must not* be used. The same goes to environment variables. All functionality must be available through APIs. If odp_global init is an API that takes argv and arg style arguments is that ok, I am not sure what you mean by it must be via an API unless you mean APIs that do not take variable argument lists If we are talking about some structure of argv,argc in ODP examples and test then it is fine. However it such context it is not clear why platforms have to be in agreement. I think this is a discussion about how to pass things into the init of ODP rather than examples, the recurring need to pass some platform specific data though to the implementation. I also assume the case or platform agnostic configuration that cannot be hard coded - I am imagining ODP running in an Enterprise situation in multiple VMs where you want the same app attached to different ports or something like that. Ports is a part of a common set of configuration parameters. And currently they are not passed to odp_init, but to odp_pktio_open(). What are other parameters that need to configured in runtime? Numa? Packet i/o type for linux-generic (netmap, mmap, raw sockets); For K2 you can switch from CMA back to mmap if needed; Might be some hw modes if platform can support several… This is a runtime configuration and a API’s is reasonable, but you may need a init-time configuration to select the initial one to use or just use the API’s. Everything is relative from a given point of view.
On 08/07/2014 09:29 PM, Maxim Uvarov wrote: > On 08/07/2014 10:21 PM, Taras Kondratiuk wrote: >> On 08/07/2014 08:16 PM, Mike Holmes wrote: >>> >>> On 7 August 2014 13:02, Victor Kamensky <victor.kamensky@linaro.org >>> <mailto:victor.kamensky@linaro.org>> wrote: >>> >>> On 7 August 2014 05:52, Mike Holmes <mike.holmes@linaro.org >>> <mailto:mike.holmes@linaro.org>> wrote: >>> > I failed to summarise yesterdays conversation, here is todays >>> attempt. >>> > >>> > argv, arc is not in principle opposed by anyone. >>> >>> In what role? For functionality that involves serious >>> production deployment they *must not* be used. >>> The same goes to environment variables. All >>> functionality must be available through APIs. >>> >>> If odp_global init is an API that takes argv and arg style arguments is >>> that ok, I am not sure what you mean by it must be via an API unless you >>> mean APIs that do not take variable argument lists >>> >>> >>> If we are talking about some structure of >>> argv,argc in ODP examples and test then >>> it is fine. However it such context it is not >>> clear why platforms have to be in agreement. >>> >>> I think this is a discussion about how to pass things into the init of >>> ODP rather than examples, the recurring need to pass some platform >>> specific data though to the implementation. I also assume the case or >>> platform agnostic configuration that cannot be hard coded - I am >>> imagining ODP running in an Enterprise situation in multiple VMs where >>> you want the same app attached to different ports or something like >>> that. >> >> Ports is a part of a common set of configuration parameters. And >> currently they are not passed to odp_init, but to odp_pktio_open(). >> >> What are other parameters that need to configured in runtime? > > Numa? I'm not sure I understand. Do you mean at which node to run an application? I think it should be configurable via API. > Packet i/o type for linux-generic (netmap, mmap, raw sockets); That's only a test/debug feature and can be passed over many other ways. > For K2 you can switch from CMA back to mmap if needed; CMA is a must for HW buffers, so not an option. > Might be some hw modes if platform can support several... That will be actually not a per application option, but rather a system configuration. Only one per system should exist stored or hardcoded somewhere.
On 08/07/2014 09:26 PM, Wiles, Roger Keith wrote: > Comments inline. > *Keith **Wiles*, Principal Technologist with CTO office, *Wind River* > mobile 972-213-5533 > > Powering 30 Years of Innovation <http://www.windriver.com/announces/wr30/> > > On Aug 7, 2014, at 1:00 PM, Victor Kamensky <victor.kamensky@linaro.org > <mailto:victor.kamensky@linaro.org>> wrote: > >> >> >> >> On 7 August 2014 10:37, Wiles, Roger Keith <keith.wiles@windriver.com >> <mailto:keith.wiles@windriver.com>> wrote: >> >> We basically have three layers in the ODP framework Application, >> ODP and SDK. The Application layer needs arguments plus the SDK >> layer in the case of DPDK needs arguments. The ODP needs arguments >> or configuration at run time too. This is the reason for the patch >> file, just to show what I was suggesting. >> >> Here is the commit comment just so you do not need to open the file. >> >> Add the new ODP_INIT_GLOBAL() macro to hide some details. Which >> allows the arguments to be passed to the different levels. The >> levels are ODP, SDK and application using the getopt construct >> of '--' to delimit each layer. >> >> First arguments '-s foobar.lua' made up args for ODP layer. >> Second arguments '-c 3 -n 3' DPDK arguments layer. >> Third arguments '-i 0,1 -p 3' example args for applications layer. >> >> odp_foobar -s foobar.lua -- -c 3 -n 3 -- -i 0,1 -p 3 >> >> Always use '--' to delimit the layers. >> >> Now we have three different layers each being able to have its own >> flags and arguments. This way we do not need to register or make >> sure someone does not try to duplicate a argument. >> >> >> IMO above is simplified method that is suitable for ODP >> example applications and tests. Please see my other >> responses. In production deployment ODP is just part >> of overall bigger system and such systems have itis own >> complicated set of rules, that we don't want to know. >> I am arguing that such system should be able to communicate >> with ODP through set of well defined structured APIs, rather >> than array of strings with some obscure meaning. I.e >> it seems to me that with argc/argv/ approach we are >> trying to create some mini language with its own >> semantics whereas normal structure C apis could be used. > > ODP is a basic framework we have to define and setup the system from a > simple configuration point of view as we can not see what each product > will need in the future. > > It is very possible the SDK or APPLICATION layer does not require > arguments and would load those arguments from a config file or provide a > set of API’s to configure these layers. This is still possible as the > SDK and application would ignore and pass these args to the next layer. > In all of the application now in ODP we have arguments for the > application and in the DPDK argc/argv is constructed in the ‘C’ and is > not usable in that design except at a very basic level. > > I agree for most non-initialization time configuration we should use > API’s, but for the basic system we can use argc/argv. I would like to > see your suggestions as to how we can design ODP. From implementation point it is not an issue to parse configuration strings, but it just feels wrong. The fact that DPDK does it doesn't mean it is a good approach. I'd prefer to pass generic parameters via structure to odp_init() and let implementation to figure out how to get other stuff if needed. In case of Linux-based implementations there are many ways to do it.
On 7 August 2014 11:16, Wiles, Roger Keith <keith.wiles@windriver.com> wrote: > Comments inline. > > *Keith **Wiles*, Principal Technologist with CTO office, *Wind River* > mobile 972-213-5533 > > [image: Powering 30 Years of Innovation] > <http://www.windriver.com/announces/wr30/> > > On Aug 7, 2014, at 12:47 PM, Victor Kamensky <victor.kamensky@linaro.org> > wrote: > > On 7 August 2014 10:16, Mike Holmes <mike.holmes@linaro.org> wrote: > > > > > On 7 August 2014 13:02, Victor Kamensky <victor.kamensky@linaro.org> > wrote: > > > On 7 August 2014 05:52, Mike Holmes <mike.holmes@linaro.org> wrote: > > I failed to summarise yesterdays conversation, here is todays attempt. > > argv, arc is not in principle opposed by anyone. > > > In what role? For functionality that involves serious > production deployment they *must not* be used. > The same goes to environment variables. All > functionality must be available through APIs. > > > If odp_global init is an API that takes argv and arg style arguments is > that > ok, > > > No, it is not OK. See below. > > I am not sure what you mean by it must be via an API unless you mean > APIs that do not take variable argument lists > > > I don't what odp_init_global take argc,argv it pushes > meaning of action that has to be done into other dimension. > I.e one need to look at API description and then look at > list of possible argc/argv which described where? How > portable are they? > > > The args for ODP should be pretty generic IMO, so we can list these in > the comments or in the usage() /help function for the ODP layer. Maybe ODP > does not require args, but we have began to define flags and that is just > another form of arguments. These flags can be passed on the command line or > a init routine developed by the developer can construct a argc/argv list. > These are just for init time not run time changes. If the system needs to > have runtime configuration then API’s should be created. > > We do not need to have every options or runtime configuration on the > command line, but create API’s to allow them to be changed through ODP to > the SDK. > > > If you need to convey any information of odp_init_global > define proper enumeration, structs etc and pass it to > odp_init_global or other API. > > > Using enums or flags or argc/argv provides the same basic result and > command line args are reasonable as most of the platforms here run on > Linux. Now ODP can provide a routine to convert some type of args to > argc/argv format which is very simple do. > > > > > If we are talking about some structure of > argv,argc in ODP examples and test then > it is fine. However it such context it is not > clear why platforms have to be in agreement. > > > I think this is a discussion about how to pass things into the init of ODP > rather than examples, the recurring need to pass some platform specific > data > though to the implementation. > > > I think it is beyond of ODP platform independent > specification. Implementation specific configuration > could be delivered in variety of ways: > > > Yes, Implementation configuration can be done is a multiple ways, but the > point is we do need and see a requirement for being able to apply arguments > to all three layer in a application > — Application layer > — ODP layer > — SDK layer > > each of these layers require a basic way to provide configuration at > build time, init time and run time. Runtime is for configuration that needs > to be define on the fly and init time configuration is for system > initialization only in most case and does not change normally after > initialization. > I am not concerned about 'Application layer' and 'SDK layer' at all. All that is beyond of ODP framework. ODP cannot dictate neither of these layers what to do and how to do init time configuration. I acknowledge that there are some consistency issues of command line arguments on 'Application layer' when it comes to ODP example and tests, but all that is not normative part of ODP and can be handled separate. As 'ODP layer' concerned, yes sure I can construct array of strings and pass it to some function. My worry is that makes ODP api to spread out ... Along with regular header files, enums, #defines, structs, function annotated by doxygen etc we would have some special language for such array of strings. These set of strings must be treated as API, i.e described precisely, agreed by all members, versioned along ODP spec life time. But what logistical means we have for that? For example doxygen handles all C API elments very well, how would you describe those strings in header files? Very long comment in odp_init_global function? IMO it creates logistical challenges and it is not worth it. For example how would you version such thing and do build time checks? With #define, struct, etc incomatibilties quit often will come out during build time (#define is not there, struct field is missing etc). But with array of strings it will be just run-time error. Anyway if difference between strict APIs and set of strings is so small why do you argue vs structured APIs the way followed by ODP everywhere else. I.e what benefits it brings to have API with array of strings API? Maybe I am missing this part. Thanks, Victor > > > o build time: platform header file with bunch of constants, > which are defined by user of library > > o special implementation specific set of calls that app > does in beginning. We can survive implementation > specific ifdef in some portion of our code > > o config file that ODP implementation reads > > > A config file requires a filesystem, but let ignore that point for now. > If we did provide a init file for ODP then we can provide a ‘-f > foobar.config’ option on the command line. Another option is the > application open and parse the config file converting them into argc/argv > options to pass to odp_init_global() routine. > > > o ... > > In my opinion it is up to ODP implementer how to > do that. > > > I do see some of the configuration must be defined by the developer, but > we are starting to see some common threads and configuration we can supply > today. > > > I also assume the case or platform agnostic > configuration that cannot be hard coded > > > If there is a need for platform agnostic configuration > in ODP we need to create proper API for that. That > what I meant it should be APIs only. Pushing some > meaning to set of strings IMHO is wrong way to go. > > > I do not see how we are not making this platform agnostic as we still > allow the SDK and application to have its own configuration without having > to alter ODP for each platform. > > I am seeing a lot of duplicate code in each of the platforms we have > today and if you look at the code I broke out the DPDK routine > odp_init_dpdk() into odp_init_skd() instead, which now should allow most of > the ODP code for initing system to be generic and common between the > platforms. I did not fully include those changes as it a bit more work. > > > - I am imagining ODP running in an > Enterprise situation in multiple VMs where you want the same app attached > to > different ports or something like that. > > > Serious deployment situation is way more complicated > configuration parameters do not always come from > args/argv. And serious application has its own set of > rules and convention how to handle argc/argv. Just > simple example what if those configuration parameters > should be changed on the fly of they hey come from > config file. With array of strings > API you telling me that I need to construct array of > strings and pass it to some API ... I find it not very > clean. If follow that direction why do we define enumeration > for crypto protocols supported let's pass it as 'char *’. > > > I have developed and design a number of systems here are WR and you are > correct that a product needs a lot of work to be a real product. My point > is we can help reduce the workload of the developer by helping define at > least a basic configuration system he can build on for his product. > > As for the crypto protocols I have not looked at them, but I think you > are taking an extreme view here. > > Do not mean to offend anyone, but I do believe we need a basic > configuration in all three areas for ODP. > > > Thanks, > Victor > > > > Thanks, > Victor > > Need to push as much as possible into the implementation wherever > possible > Arguments should try very hard to be platform agnostic > Arguments for a platform need to be marked and passed through verbatim - > then duplication of existing flags on different sdks is ok and clear to > the > user. Maybe --dpdk followed by all the flags it expects. > VMs may need some differences passed in, so not all differences can be > compiled into the implementation. > Need to avoid the case of two platforms claiming the same argument > A strong case for scripting like Lua was not exposed in this discussion. > > Could I ask everyone to reply with any agnostic and platform specific > arguments they would like to see, maybe then we can form a clear picture > of > those that are needed. > > > > On 5 August 2014 08:31, Taras Kondratiuk <taras.kondratiuk@linaro.org> > wrote: > > > On 08/05/2014 02:59 PM, Mike Holmes wrote: > > > How does this play in with Keiths resumed call for argv argc? Thread > = > "[lng-odp] Port of Pktgen to ODP" it looks like on two fronts we have > come back to the need to add args to init that are more complex than > basic flags. > > With argv argc, I assume we can pass these flags as options with > getopt ? > > > > These flags are defined by API used by application, so they are > hardcoded into application, but not passed from command line. If we are > going to argv/argc way, then application can just insert one more > parameter into the list, before passing it to odp_init_global(). > > Some issues are > > 1. passing things through to underlying SDKs > 2. Making some args mandatory > 3. Keeping the argument parsing in sync across platforms > 4. Not coupling all the platforms and their special args together > 5. Not promoting the use of platform specific args in general as they > will break portability. > > > > I agree with an approach when only common platform independent > arguments > are passed to ODP application. We have to analyze which parameters are > required by different platforms and come up with a common list. The > rest > platform specific parameters have to passed to implementation in some > other way (environment variable, config file, hardcoded, etc.) > > > > > > -- > Mike Holmes > Linaro Technical Manager / Lead > LNG - ODP > > _______________________________________________ > lng-odp mailing list > lng-odp@lists.linaro.org > http://lists.linaro.org/mailman/listinfo/lng-odp > > > > > > -- > Mike Holmes > Linaro Technical Manager / Lead > LNG - ODP > > > _______________________________________________ > lng-odp mailing list > lng-odp@lists.linaro.org > http://lists.linaro.org/mailman/listinfo/lng-odp > > >
Keith Wiles, Principal Technologist with CTO office, Wind River mobile 972-213-5533 On Aug 7, 2014, at 2:21 PM, Taras Kondratiuk <taras.kondratiuk@linaro.org<mailto:taras.kondratiuk@linaro.org>> wrote: On 08/07/2014 09:26 PM, Wiles, Roger Keith wrote: Comments inline. *Keith **Wiles*, Principal Technologist with CTO office, *Wind River* mobile 972-213-5533 Powering 30 Years of Innovation <http://www.windriver.com/announces/wr30/> On Aug 7, 2014, at 1:00 PM, Victor Kamensky <victor.kamensky@linaro.org<mailto:victor.kamensky@linaro.org> <mailto:victor.kamensky@linaro.org>> wrote: On 7 August 2014 10:37, Wiles, Roger Keith <keith.wiles@windriver.com<mailto:keith.wiles@windriver.com> <mailto:keith.wiles@windriver.com>> wrote: We basically have three layers in the ODP framework Application, ODP and SDK. The Application layer needs arguments plus the SDK layer in the case of DPDK needs arguments. The ODP needs arguments or configuration at run time too. This is the reason for the patch file, just to show what I was suggesting. Here is the commit comment just so you do not need to open the file. Add the new ODP_INIT_GLOBAL() macro to hide some details. Which allows the arguments to be passed to the different levels. The levels are ODP, SDK and application using the getopt construct of '--' to delimit each layer. First arguments '-s foobar.lua' made up args for ODP layer. Second arguments '-c 3 -n 3' DPDK arguments layer. Third arguments '-i 0,1 -p 3' example args for applications layer. odp_foobar -s foobar.lua -- -c 3 -n 3 -- -i 0,1 -p 3 Always use '--' to delimit the layers. Now we have three different layers each being able to have its own flags and arguments. This way we do not need to register or make sure someone does not try to duplicate a argument. IMO above is simplified method that is suitable for ODP example applications and tests. Please see my other responses. In production deployment ODP is just part of overall bigger system and such systems have itis own complicated set of rules, that we don't want to know. I am arguing that such system should be able to communicate with ODP through set of well defined structured APIs, rather than array of strings with some obscure meaning. I.e it seems to me that with argc/argv/ approach we are trying to create some mini language with its own semantics whereas normal structure C apis could be used. ODP is a basic framework we have to define and setup the system from a simple configuration point of view as we can not see what each product will need in the future. It is very possible the SDK or APPLICATION layer does not require arguments and would load those arguments from a config file or provide a set of API’s to configure these layers. This is still possible as the SDK and application would ignore and pass these args to the next layer. In all of the application now in ODP we have arguments for the application and in the DPDK argc/argv is constructed in the ‘C’ and is not usable in that design except at a very basic level. I agree for most non-initialization time configuration we should use API’s, but for the basic system we can use argc/argv. I would like to see your suggestions as to how we can design ODP. From implementation point it is not an issue to parse configuration strings, but it just feels wrong. The fact that DPDK does it doesn't mean it is a good approach. I'd prefer to pass generic parameters via structure to odp_init() and let implementation to figure out how to get other stuff if needed. In case of Linux-based implementations there are many ways to do it. I do understand, but DPDK is inited in this fashion and we would need to supply a argc/argv parameters to call its init routine. Now it is being done in the odp_init_dpdk() function and it can not support all of the args. We need to pass args to the SDK in some way and argc/argv is the best solution for DPDK. Now we can do as I suggested by adding ‘—sdk-args=“”’ to the command line or the odp_init_dpdk() routine will need to get the values some place. It just seems similar to add them to command line, even if ODP is not really launched from a command line. I do not have a feel for how many designs are not using some type of OS like Linux or FreeBSD, but I would guess it is not the standard today. It appears all of the platforms in the repo are based in some OS platform today. Even VxWorks has argc/argv support and able to launch applications from a command line :-)
On 7 August 2014 12:36, Wiles, Roger Keith <keith.wiles@windriver.com> wrote: > > Keith Wiles, Principal Technologist with CTO office, Wind River > mobile 972-213-5533 > On Aug 7, 2014, at 2:21 PM, Taras Kondratiuk <taras.kondratiuk@linaro.org> > wrote: > > On 08/07/2014 09:26 PM, Wiles, Roger Keith wrote: > > Comments inline. > *Keith **Wiles*, Principal Technologist with CTO office, *Wind River* > mobile 972-213-5533 > > Powering 30 Years of Innovation <http://www.windriver.com/announces/wr30/> > > On Aug 7, 2014, at 1:00 PM, Victor Kamensky <victor.kamensky@linaro.org > <mailto:victor.kamensky@linaro.org>> wrote: > > > > > On 7 August 2014 10:37, Wiles, Roger Keith <keith.wiles@windriver.com > <mailto:keith.wiles@windriver.com>> wrote: > > We basically have three layers in the ODP framework Application, > ODP and SDK. The Application layer needs arguments plus the SDK > layer in the case of DPDK needs arguments. The ODP needs arguments > or configuration at run time too. This is the reason for the patch > file, just to show what I was suggesting. > > Here is the commit comment just so you do not need to open the file. > > Add the new ODP_INIT_GLOBAL() macro to hide some details. Which > allows the arguments to be passed to the different levels. The > levels are ODP, SDK and application using the getopt construct > of '--' to delimit each layer. > > First arguments '-s foobar.lua' made up args for ODP layer. > Second arguments '-c 3 -n 3' DPDK arguments layer. > Third arguments '-i 0,1 -p 3' example args for applications layer. > > odp_foobar -s foobar.lua -- -c 3 -n 3 -- -i 0,1 -p 3 > > Always use '--' to delimit the layers. > > Now we have three different layers each being able to have its own > flags and arguments. This way we do not need to register or make > sure someone does not try to duplicate a argument. > > > IMO above is simplified method that is suitable for ODP > example applications and tests. Please see my other > responses. In production deployment ODP is just part > of overall bigger system and such systems have itis own > complicated set of rules, that we don't want to know. > I am arguing that such system should be able to communicate > with ODP through set of well defined structured APIs, rather > than array of strings with some obscure meaning. I.e > it seems to me that with argc/argv/ approach we are > trying to create some mini language with its own > semantics whereas normal structure C apis could be used. > > > ODP is a basic framework we have to define and setup the system from a > simple configuration point of view as we can not see what each product > will need in the future. > > It is very possible the SDK or APPLICATION layer does not require > arguments and would load those arguments from a config file or provide a > set of API’s to configure these layers. This is still possible as the > SDK and application would ignore and pass these args to the next layer. > In all of the application now in ODP we have arguments for the > application and in the DPDK argc/argv is constructed in the ‘C’ and is > not usable in that design except at a very basic level. > > I agree for most non-initialization time configuration we should use > API’s, but for the basic system we can use argc/argv. I would like to > see your suggestions as to how we can design ODP. How about the following approach: we would exclude 'ODP layer' from argc/argv discussion. And have structured APIs for its init time configuration. What is left Application and SDK layers. It seems to me that Application in current discussion are existing ODP examples and tests. And SDK layer is opaque. We could agree on some set of options that ODP examples and tests understand and rest as opaque stuff will be passed to SDK layer. odp_init_global could take argc/argv but it would be treated as opaque thing, outside of spec, only one of many possible ways to configure SDK layer. If some ODP implementation would choose other way to configure itself it is fine. All that ODP examples and tests and SDK config are not normative part of ODP. How it is done exactly I don't care. Applications that I have in mind would use structured APIs to talk with ODP layer, and in case of DPDK based implementation would pass SDK config as set of argc/argv and possibly do some other things with other implementations. Thanks, Victor > > From implementation point it is not an issue to parse configuration > strings, but it just feels wrong. The fact that DPDK does it doesn't > mean it is a good approach. > > I'd prefer to pass generic parameters via structure to odp_init() and let > implementation to figure out how to get other stuff if needed. > In case of Linux-based implementations there are many ways to do it. > > > I do understand, but DPDK is inited in this fashion and we would need to > supply a argc/argv parameters to call its init routine. Now it is being done > in the odp_init_dpdk() function and it can not support all of the args. We > need to pass args to the SDK in some way and argc/argv is the best solution > for DPDK. Now we can do as I suggested by adding ‘—sdk-args=“”’ to the > command line or the odp_init_dpdk() routine will need to get the values some > place. It just seems similar to add them to command line, even if ODP is not > really launched from a command line. > > I do not have a feel for how many designs are not using some type of OS like > Linux or FreeBSD, but I would guess it is not the standard today. It appears > all of the platforms in the repo are based in some OS platform today. Even > VxWorks has argc/argv support and able to launch applications from a command > line :-) >
On 08/07/2014 10:36 PM, Wiles, Roger Keith wrote: > > *Keith **Wiles*, Principal Technologist with CTO office, *Wind River* > mobile 972-213-5533 > On Aug 7, 2014, at 2:21 PM, Taras Kondratiuk > <taras.kondratiuk@linaro.org <mailto:taras.kondratiuk@linaro.org>> wrote: >> >> From implementation point it is not an issue to parse configuration >> strings, but it just feels wrong. The fact that DPDK does it doesn't >> mean it is a good approach. >> >> I'd prefer to pass generic parameters via structure to odp_init() and >> let implementation to figure out how to get other stuff if needed. >> In case of Linux-based implementations there are many ways to do it. > > I do understand, but DPDK is inited in this fashion and we would need to > supply a argc/argv parameters to call its init routine. Now it is being > done in the odp_init_dpdk() function and it can not support all of the > args. We need to pass args to the SDK in some way and argc/argv is the > best solution for DPDK. Now we can do as I suggested by adding > ‘—sdk-args=“”’ to the command line or the odp_init_dpdk() routine will > need to get the values some place. It just seems similar to add them to > command line, even if ODP is not really launched from a command line. The first approach that comes to my mind is an environment variable. DPDK_CMD_LINE='some configs' odp_foobar then linux-dpdk can getevn('DPDK_CMD_LINE') and everybody happy. API is not changed and DPDK have its configuration.
Keith Wiles, Principal Technologist with CTO office, Wind River mobile 972-213-5533 On Aug 7, 2014, at 2:35 PM, Victor Kamensky <victor.kamensky@linaro.org<mailto:victor.kamensky@linaro.org>> wrote: On 7 August 2014 11:16, Wiles, Roger Keith <keith.wiles@windriver.com<mailto:keith.wiles@windriver.com>> wrote: Comments inline. Keith Wiles, Principal Technologist with CTO office, Wind River mobile 972-213-5533<tel:972-213-5533> [Powering 30 Years of Innovation]<http://www.windriver.com/announces/wr30/> On Aug 7, 2014, at 12:47 PM, Victor Kamensky <victor.kamensky@linaro.org<mailto:victor.kamensky@linaro.org>> wrote: On 7 August 2014 10:16, Mike Holmes <mike.holmes@linaro.org<mailto:mike.holmes@linaro.org>> wrote: On 7 August 2014 13:02, Victor Kamensky <victor.kamensky@linaro.org<mailto:victor.kamensky@linaro.org>> wrote: On 7 August 2014 05:52, Mike Holmes <mike.holmes@linaro.org<mailto:mike.holmes@linaro.org>> wrote: I failed to summarise yesterdays conversation, here is todays attempt. argv, arc is not in principle opposed by anyone. In what role? For functionality that involves serious production deployment they *must not* be used. The same goes to environment variables. All functionality must be available through APIs. If odp_global init is an API that takes argv and arg style arguments is that ok, No, it is not OK. See below. I am not sure what you mean by it must be via an API unless you mean APIs that do not take variable argument lists I don't what odp_init_global take argc,argv it pushes meaning of action that has to be done into other dimension. I.e one need to look at API description and then look at list of possible argc/argv which described where? How portable are they? The args for ODP should be pretty generic IMO, so we can list these in the comments or in the usage() /help function for the ODP layer. Maybe ODP does not require args, but we have began to define flags and that is just another form of arguments. These flags can be passed on the command line or a init routine developed by the developer can construct a argc/argv list. These are just for init time not run time changes. If the system needs to have runtime configuration then API’s should be created. We do not need to have every options or runtime configuration on the command line, but create API’s to allow them to be changed through ODP to the SDK. If you need to convey any information of odp_init_global define proper enumeration, structs etc and pass it to odp_init_global or other API. Using enums or flags or argc/argv provides the same basic result and command line args are reasonable as most of the platforms here run on Linux. Now ODP can provide a routine to convert some type of args to argc/argv format which is very simple do. If we are talking about some structure of argv,argc in ODP examples and test then it is fine. However it such context it is not clear why platforms have to be in agreement. I think this is a discussion about how to pass things into the init of ODP rather than examples, the recurring need to pass some platform specific data though to the implementation. I think it is beyond of ODP platform independent specification. Implementation specific configuration could be delivered in variety of ways: Yes, Implementation configuration can be done is a multiple ways, but the point is we do need and see a requirement for being able to apply arguments to all three layer in a application — Application layer — ODP layer — SDK layer each of these layers require a basic way to provide configuration at build time, init time and run time. Runtime is for configuration that needs to be define on the fly and init time configuration is for system initialization only in most case and does not change normally after initialization. I am not concerned about 'Application layer' and 'SDK layer' at all. All that is beyond of ODP framework. ODP cannot dictate neither of these layers what to do and how to do init time configuration. I acknowledge that there are some consistency issues of command line arguments on 'Application layer' when it comes to ODP example and tests, but all that is not normative part of ODP and can be handled separate. You need to be concerned with the SDK and application layer at some point as ODP is not a standalone product and requires these two parts to even be usable or exist. ODP is trying to define a framework and part of that framework should be how it interacts with SDK and applications. I feel like we are at an impasse here and I would like to suggest another solution. int odp_init_global(int argc, char ** argv, odp_flags_t flags); This way ODP can get its values from the flags and can ignore the argc/argv, but just pass them alone to the SDK. I do not like this option and just passing flags is not reasonable IMO. Flags at some point have limits and the amount of information they can convey is limited. Even if you passed structures this has limits as well as anyone wanting to pass more information would need to modify or replace the structure. If each platform has it own structure that becomes a problem as well. In your suggestion the application would have to convert argc/argv values into flags to be passed to ODP/SDK, but why the args came into the applications via a command line why not pass them along. I know the application could be called from another application and not from a command line, but I feel the common case is going to be from some type of command line. ODP should define a framework attempting to remove as much work as it can from each platform and no more. By trying to define a clean framework we allow the developer to work on what is important SDK/application. ODP needs and must unify some of the code in each platform into a single common code base and provide clean methods to allow the SDK and application to add what is reasonable. As the three platforms stand today we have a lot of duplicate code and in some cases use code directly from the linux-generic platform in the other platform. These common set of routines need to be pulled into a common directory code base. As 'ODP layer' concerned, yes sure I can construct array of strings and pass it to some function. My worry is that makes ODP api to spread out ... Along with regular header files, enums, #defines, structs, function annotated by doxygen etc we would have some special language for such array of strings. These set of strings must be treated as API, i.e described precisely, agreed by all members, versioned along ODP spec life time. But what logistical means we have for that? For example doxygen handles all C API elments very well, how would you describe those strings in header files? Very long comment in odp_init_global function? IMO it creates logistical challenges and it is not worth it. For example how would you version such thing and do build time checks? With #define, struct, etc incomatibilties quit often will come out during build time (#define is not there, struct field is missing etc). But with array of strings it will be just run-time error. I do not understand the above statement as each layer is independent from the next in some ways except for common defines and prototypes. The ODP layer or application layer should never need to inspect or look at the SDK layer values, if they do then we have a bigger problem for portability. As for doxygen is very happy about including comments on function headers, file headers, defines, macros, arrays, typedefs or … so I do not see the problem with doxygen here. Anyway if difference between strict APIs and set of strings is so small why do you argue vs structured APIs the way followed by ODP everywhere else. I.e what benefits it brings to have API with array of strings API? Maybe I am missing this part. I guess I would ask you the same question :-) Thanks, Victor o build time: platform header file with bunch of constants, which are defined by user of library o special implementation specific set of calls that app does in beginning. We can survive implementation specific ifdef in some portion of our code o config file that ODP implementation reads A config file requires a filesystem, but let ignore that point for now. If we did provide a init file for ODP then we can provide a ‘-f foobar.config’ option on the command line. Another option is the application open and parse the config file converting them into argc/argv options to pass to odp_init_global() routine. o ... In my opinion it is up to ODP implementer how to do that. I do see some of the configuration must be defined by the developer, but we are starting to see some common threads and configuration we can supply today. I also assume the case or platform agnostic configuration that cannot be hard coded If there is a need for platform agnostic configuration in ODP we need to create proper API for that. That what I meant it should be APIs only. Pushing some meaning to set of strings IMHO is wrong way to go. I do not see how we are not making this platform agnostic as we still allow the SDK and application to have its own configuration without having to alter ODP for each platform. I am seeing a lot of duplicate code in each of the platforms we have today and if you look at the code I broke out the DPDK routine odp_init_dpdk() into odp_init_skd() instead, which now should allow most of the ODP code for initing system to be generic and common between the platforms. I did not fully include those changes as it a bit more work. - I am imagining ODP running in an Enterprise situation in multiple VMs where you want the same app attached to different ports or something like that. Serious deployment situation is way more complicated configuration parameters do not always come from args/argv. And serious application has its own set of rules and convention how to handle argc/argv. Just simple example what if those configuration parameters should be changed on the fly of they hey come from config file. With array of strings API you telling me that I need to construct array of strings and pass it to some API ... I find it not very clean. If follow that direction why do we define enumeration for crypto protocols supported let's pass it as 'char *’. I have developed and design a number of systems here are WR and you are correct that a product needs a lot of work to be a real product. My point is we can help reduce the workload of the developer by helping define at least a basic configuration system he can build on for his product. As for the crypto protocols I have not looked at them, but I think you are taking an extreme view here. Do not mean to offend anyone, but I do believe we need a basic configuration in all three areas for ODP. Thanks, Victor Thanks, Victor Need to push as much as possible into the implementation wherever possible Arguments should try very hard to be platform agnostic Arguments for a platform need to be marked and passed through verbatim - then duplication of existing flags on different sdks is ok and clear to the user. Maybe --dpdk followed by all the flags it expects. VMs may need some differences passed in, so not all differences can be compiled into the implementation. Need to avoid the case of two platforms claiming the same argument A strong case for scripting like Lua was not exposed in this discussion. Could I ask everyone to reply with any agnostic and platform specific arguments they would like to see, maybe then we can form a clear picture of those that are needed. On 5 August 2014 08:31, Taras Kondratiuk <taras.kondratiuk@linaro.org<mailto:taras.kondratiuk@linaro.org>> wrote: On 08/05/2014 02:59 PM, Mike Holmes wrote: How does this play in with Keiths resumed call for argv argc? Thread = "[lng-odp] Port of Pktgen to ODP" it looks like on two fronts we have come back to the need to add args to init that are more complex than basic flags. With argv argc, I assume we can pass these flags as options with getopt ? These flags are defined by API used by application, so they are hardcoded into application, but not passed from command line. If we are going to argv/argc way, then application can just insert one more parameter into the list, before passing it to odp_init_global(). Some issues are 1. passing things through to underlying SDKs 2. Making some args mandatory 3. Keeping the argument parsing in sync across platforms 4. Not coupling all the platforms and their special args together 5. Not promoting the use of platform specific args in general as they will break portability. I agree with an approach when only common platform independent arguments are passed to ODP application. We have to analyze which parameters are required by different platforms and come up with a common list. The rest platform specific parameters have to passed to implementation in some other way (environment variable, config file, hardcoded, etc.) -- Mike Holmes Linaro Technical Manager / Lead LNG - ODP
On 7 August 2014 13:40, Wiles, Roger Keith <keith.wiles@windriver.com> wrote: > > *Keith **Wiles*, Principal Technologist with CTO office, *Wind River* > mobile 972-213-5533 > On Aug 7, 2014, at 2:35 PM, Victor Kamensky <victor.kamensky@linaro.org> > wrote: > > > > > On 7 August 2014 11:16, Wiles, Roger Keith <keith.wiles@windriver.com> > wrote: > >> Comments inline. >> >> *Keith **Wiles*, Principal Technologist with CTO office, *Wind River* >> mobile 972-213-5533 >> >> [image: Powering 30 Years of Innovation] >> <http://www.windriver.com/announces/wr30/> >> >> On Aug 7, 2014, at 12:47 PM, Victor Kamensky < >> victor.kamensky@linaro.org> wrote: >> >> On 7 August 2014 10:16, Mike Holmes <mike.holmes@linaro.org> wrote: >> >> >> >> >> On 7 August 2014 13:02, Victor Kamensky <victor.kamensky@linaro.org> >> wrote: >> >> >> On 7 August 2014 05:52, Mike Holmes <mike.holmes@linaro.org> wrote: >> >> I failed to summarise yesterdays conversation, here is todays attempt. >> >> argv, arc is not in principle opposed by anyone. >> >> >> In what role? For functionality that involves serious >> production deployment they *must not* be used. >> The same goes to environment variables. All >> functionality must be available through APIs. >> >> >> If odp_global init is an API that takes argv and arg style arguments is >> that >> ok, >> >> >> No, it is not OK. See below. >> >> I am not sure what you mean by it must be via an API unless you mean >> APIs that do not take variable argument lists >> >> >> I don't what odp_init_global take argc,argv it pushes >> meaning of action that has to be done into other dimension. >> I.e one need to look at API description and then look at >> list of possible argc/argv which described where? How >> portable are they? >> >> >> The args for ODP should be pretty generic IMO, so we can list these in >> the comments or in the usage() /help function for the ODP layer. Maybe ODP >> does not require args, but we have began to define flags and that is just >> another form of arguments. These flags can be passed on the command line or >> a init routine developed by the developer can construct a argc/argv list. >> These are just for init time not run time changes. If the system needs to >> have runtime configuration then API’s should be created. >> >> We do not need to have every options or runtime configuration on the >> command line, but create API’s to allow them to be changed through ODP to >> the SDK. >> >> >> If you need to convey any information of odp_init_global >> define proper enumeration, structs etc and pass it to >> odp_init_global or other API. >> >> >> Using enums or flags or argc/argv provides the same basic result and >> command line args are reasonable as most of the platforms here run on >> Linux. Now ODP can provide a routine to convert some type of args to >> argc/argv format which is very simple do. >> >> >> >> >> If we are talking about some structure of >> argv,argc in ODP examples and test then >> it is fine. However it such context it is not >> clear why platforms have to be in agreement. >> >> >> I think this is a discussion about how to pass things into the init of ODP >> rather than examples, the recurring need to pass some platform specific >> data >> though to the implementation. >> >> >> I think it is beyond of ODP platform independent >> specification. Implementation specific configuration >> could be delivered in variety of ways: >> >> >> Yes, Implementation configuration can be done is a multiple ways, but >> the point is we do need and see a requirement for being able to apply >> arguments to all three layer in a application >> — Application layer >> — ODP layer >> — SDK layer >> >> each of these layers require a basic way to provide configuration at >> build time, init time and run time. Runtime is for configuration that needs >> to be define on the fly and init time configuration is for system >> initialization only in most case and does not change normally after >> initialization. >> > > I am not concerned about 'Application layer' and 'SDK layer' at all. > All that is beyond of ODP framework. ODP cannot dictate neither > of these layers what to do and how to do init time configuration. I > acknowledge that there are some consistency issues of command > line arguments on 'Application layer' when it comes to ODP > example and tests, but all that is not normative part of ODP and > can be handled separate. > > > You need to be concerned with the SDK and application layer at some point > as ODP is not a standalone product and requires these two parts to even be > usable or exist. ODP is trying to define a framework and part of that > framework should be how it interacts with SDK and applications. > > I feel like we are at an impasse here and I would like to suggest > another solution. > > int odp_init_global(int argc, char ** argv, odp_flags_t flags); > > This way ODP can get its values from the flags and can ignore the > argc/argv, but just pass them alone to the SDK. > > I do not like this option and just passing flags is not reasonable IMO. > Flags at some point have limits and the amount of information they can > convey is limited. Even if you passed structures this has limits as well as > anyone wanting to pass more information would need to modify or replace the > structure. If each platform has it own structure that becomes a problem as > well. > See my other response 'flags' proposed on this diff do not have any sense to me. At least it was not explained well. I am not arguing for flags. IMO if there is any need to do any ODP layer specific init time config, it should be discussed separately API (function, struct, etc) and added to normative ODP API. One example from the past, before your time, that was discussed a bit, but dropped on the floor was need to have function pointers callbacks that ODP layer and below should use to report error and debug messages. How would you pass function pointers to ODP layer through set of strings? > > In your suggestion the application would have to convert argc/argv > values into flags to be passed to ODP/SDK, but why the args came into the > applications via a command line why not pass them along. I know the > application could be called from another application and not from a command > line, but I feel the common case is going to be from some type of command > line. > In systems of company from where I am assignee none of the process receive any meaningful arguments, relevant to this discussion. It could be anything, other than process arguments: o build time o EEPROM o startup config file o XML config file o Run time licensing o Run time detection of platform type (for single image that runs on different h/w) o standby reconfigured by active o ... Clearly in the case of argc/argv strings array API we would need to construct those arguments from structured information which we receive by other means. I find that ugly. So I don't see normal process argc/argv argument as dominant and defining way. > > ODP should define a framework attempting to remove as much work as it > can from each platform and no more. By trying to define a clean framework > we allow the developer to work on what is important SDK/application. > > ODP needs and must unify some of the code in each platform into a single > common code base and provide clean methods to allow the SDK and application > to add what is reasonable. As the three platforms stand today we have a lot > of duplicate code and in some cases use code directly from the > linux-generic platform in the other platform. These common set of routines > need to be pulled into a common directory code base. > Above is arguable. As far as I know many members have no intention at all to make their ODP layer implementation to be open source. In such situation implementation sharing is problematic. We have few now that could share some code, but clearly implementation sharing is not the goal. Sharing of API is the goal. > > As 'ODP layer' concerned, yes sure I can construct array of strings > and pass it to some function. My worry is that makes ODP api to > spread out ... Along with regular header files, enums, #defines, structs, > function annotated by doxygen etc we would have some special > language for such array of strings. These set of strings must be treated > as API, i.e described precisely, agreed by all members, versioned along > ODP spec life time. But what logistical means we have for that? For > example doxygen handles all C API elments very well, how would you > describe those strings in header files? Very long comment in > odp_init_global > function? IMO it creates logistical challenges and it is not worth it. > For example how would you version such thing and do build time checks? > With #define, struct, etc incomatibilties quit often will come out during > build time (#define is not there, struct field is missing etc). But with > array of strings it will be just run-time error. > > > I do not understand the above statement as each layer is independent from > the next in some ways except for common defines and prototypes. The ODP > layer or application layer should never need to inspect or look at the SDK > layer values, if they do then we have a bigger problem for portability. As > for doxygen is very happy about including comments on function headers, > file headers, defines, macros, arrays, typedefs or … so I do not see the > problem with doxygen here. > As far as 'ODP layer' concerned all those parameters would constitute API. My question how such API described, versioned, discovered by user, etc. > > > Anyway if difference between strict APIs and set of strings is so > small why do you argue vs structured APIs the way followed by ODP > everywhere else. I.e what benefits it brings to have API with array of > strings API? Maybe I am missing this part. > > > I guess I would ask you the same question :-) > The rule of existing precedent :). We have all ODP APIs so far described as regular structured C APIs. Now we are trying to introduce parallel mechanism with set of strings and I argue that it is not worth it, because I don't see what benefits it brings (IMHO only issues). Please look at my previous response on the thread. If we would have ability to do init time configuration for 'ODP layer' through structured APIs and have some argc/argv convention for Application (== existing examples and tests) and SDK (one that wants to do init config in such way), where both areas are outside of normative part of ODP I am fine with it. IMO it could be reasonable compromise for current discussion. Thanks, Victor > > Thanks, > Victor > > >> >> >> o build time: platform header file with bunch of constants, >> which are defined by user of library >> >> o special implementation specific set of calls that app >> does in beginning. We can survive implementation >> specific ifdef in some portion of our code >> >> o config file that ODP implementation reads >> >> >> A config file requires a filesystem, but let ignore that point for now. >> If we did provide a init file for ODP then we can provide a ‘-f >> foobar.config’ option on the command line. Another option is the >> application open and parse the config file converting them into argc/argv >> options to pass to odp_init_global() routine. >> >> >> o ... >> >> In my opinion it is up to ODP implementer how to >> do that. >> >> >> I do see some of the configuration must be defined by the developer, >> but we are starting to see some common threads and configuration we can >> supply today. >> >> >> I also assume the case or platform agnostic >> configuration that cannot be hard coded >> >> >> If there is a need for platform agnostic configuration >> in ODP we need to create proper API for that. That >> what I meant it should be APIs only. Pushing some >> meaning to set of strings IMHO is wrong way to go. >> >> >> I do not see how we are not making this platform agnostic as we still >> allow the SDK and application to have its own configuration without having >> to alter ODP for each platform. >> >> I am seeing a lot of duplicate code in each of the platforms we have >> today and if you look at the code I broke out the DPDK routine >> odp_init_dpdk() into odp_init_skd() instead, which now should allow most of >> the ODP code for initing system to be generic and common between the >> platforms. I did not fully include those changes as it a bit more work. >> >> >> - I am imagining ODP running in an >> Enterprise situation in multiple VMs where you want the same app attached >> to >> different ports or something like that. >> >> >> Serious deployment situation is way more complicated >> configuration parameters do not always come from >> args/argv. And serious application has its own set of >> rules and convention how to handle argc/argv. Just >> simple example what if those configuration parameters >> should be changed on the fly of they hey come from >> config file. With array of strings >> API you telling me that I need to construct array of >> strings and pass it to some API ... I find it not very >> clean. If follow that direction why do we define enumeration >> for crypto protocols supported let's pass it as 'char *’. >> >> >> I have developed and design a number of systems here are WR and you are >> correct that a product needs a lot of work to be a real product. My point >> is we can help reduce the workload of the developer by helping define at >> least a basic configuration system he can build on for his product. >> >> As for the crypto protocols I have not looked at them, but I think you >> are taking an extreme view here. >> >> Do not mean to offend anyone, but I do believe we need a basic >> configuration in all three areas for ODP. >> >> >> Thanks, >> Victor >> >> >> >> Thanks, >> Victor >> >> Need to push as much as possible into the implementation wherever >> possible >> Arguments should try very hard to be platform agnostic >> Arguments for a platform need to be marked and passed through verbatim - >> then duplication of existing flags on different sdks is ok and clear to >> the >> user. Maybe --dpdk followed by all the flags it expects. >> VMs may need some differences passed in, so not all differences can be >> compiled into the implementation. >> Need to avoid the case of two platforms claiming the same argument >> A strong case for scripting like Lua was not exposed in this discussion. >> >> Could I ask everyone to reply with any agnostic and platform specific >> arguments they would like to see, maybe then we can form a clear picture >> of >> those that are needed. >> >> >> >> On 5 August 2014 08:31, Taras Kondratiuk <taras.kondratiuk@linaro.org> >> wrote: >> >> >> On 08/05/2014 02:59 PM, Mike Holmes wrote: >> >> >> How does this play in with Keiths resumed call for argv argc? Thread >> = >> "[lng-odp] Port of Pktgen to ODP" it looks like on two fronts we have >> come back to the need to add args to init that are more complex than >> basic flags. >> >> With argv argc, I assume we can pass these flags as options with >> getopt ? >> >> >> >> These flags are defined by API used by application, so they are >> hardcoded into application, but not passed from command line. If we are >> going to argv/argc way, then application can just insert one more >> parameter into the list, before passing it to odp_init_global(). >> >> Some issues are >> >> 1. passing things through to underlying SDKs >> 2. Making some args mandatory >> 3. Keeping the argument parsing in sync across platforms >> 4. Not coupling all the platforms and their special args together >> 5. Not promoting the use of platform specific args in general as they >> will break portability. >> >> >> >> I agree with an approach when only common platform independent >> arguments >> are passed to ODP application. We have to analyze which parameters are >> required by different platforms and come up with a common list. The >> rest >> platform specific parameters have to passed to implementation in some >> other way (environment variable, config file, hardcoded, etc.) >> >> >> >> >> >> -- >> Mike Holmes >> Linaro Technical Manager / Lead >> LNG - ODP >> >> _______________________________________________ >> lng-odp mailing list >> lng-odp@lists.linaro.org >> http://lists.linaro.org/mailman/listinfo/lng-odp >> >> >> >> >> >> -- >> Mike Holmes >> Linaro Technical Manager / Lead >> LNG - ODP >> >> >> _______________________________________________ >> lng-odp mailing list >> lng-odp@lists.linaro.org >> http://lists.linaro.org/mailman/listinfo/lng-odp >> >> >> > >
Hi Victor, I have tried to explain my point and I am not getting anywhere with you on the subject so we are at an impasse. I will try to comment on the statements below, but I think we need to figure out a method to resolve the issue. Keith Wiles, Principal Technologist with CTO office, Wind River mobile 972-213-5533 On Aug 7, 2014, at 4:26 PM, Victor Kamensky <victor.kamensky@linaro.org<mailto:victor.kamensky@linaro.org>> wrote: On 7 August 2014 13:40, Wiles, Roger Keith <keith.wiles@windriver.com<mailto:keith.wiles@windriver.com>> wrote: Keith Wiles, Principal Technologist with CTO office, Wind River mobile 972-213-5533<tel:972-213-5533> On Aug 7, 2014, at 2:35 PM, Victor Kamensky <victor.kamensky@linaro.org<mailto:victor.kamensky@linaro.org>> wrote: On 7 August 2014 11:16, Wiles, Roger Keith <keith.wiles@windriver.com<mailto:keith.wiles@windriver.com>> wrote: Comments inline. Keith Wiles, Principal Technologist with CTO office, Wind River mobile 972-213-5533<tel:972-213-5533> [Powering 30 Years of Innovation]<http://www.windriver.com/announces/wr30/> On Aug 7, 2014, at 12:47 PM, Victor Kamensky <victor.kamensky@linaro.org<mailto:victor.kamensky@linaro.org>> wrote: On 7 August 2014 10:16, Mike Holmes <mike.holmes@linaro.org<mailto:mike.holmes@linaro.org>> wrote: On 7 August 2014 13:02, Victor Kamensky <victor.kamensky@linaro.org<mailto:victor.kamensky@linaro.org>> wrote: On 7 August 2014 05:52, Mike Holmes <mike.holmes@linaro.org<mailto:mike.holmes@linaro.org>> wrote: I failed to summarise yesterdays conversation, here is todays attempt. argv, arc is not in principle opposed by anyone. In what role? For functionality that involves serious production deployment they *must not* be used. The same goes to environment variables. All functionality must be available through APIs. If odp_global init is an API that takes argv and arg style arguments is that ok, No, it is not OK. See below. I am not sure what you mean by it must be via an API unless you mean APIs that do not take variable argument lists I don't what odp_init_global take argc,argv it pushes meaning of action that has to be done into other dimension. I.e one need to look at API description and then look at list of possible argc/argv which described where? How portable are they? The args for ODP should be pretty generic IMO, so we can list these in the comments or in the usage() /help function for the ODP layer. Maybe ODP does not require args, but we have began to define flags and that is just another form of arguments. These flags can be passed on the command line or a init routine developed by the developer can construct a argc/argv list. These are just for init time not run time changes. If the system needs to have runtime configuration then API’s should be created. We do not need to have every options or runtime configuration on the command line, but create API’s to allow them to be changed through ODP to the SDK. If you need to convey any information of odp_init_global define proper enumeration, structs etc and pass it to odp_init_global or other API. Using enums or flags or argc/argv provides the same basic result and command line args are reasonable as most of the platforms here run on Linux. Now ODP can provide a routine to convert some type of args to argc/argv format which is very simple do. If we are talking about some structure of argv,argc in ODP examples and test then it is fine. However it such context it is not clear why platforms have to be in agreement. I think this is a discussion about how to pass things into the init of ODP rather than examples, the recurring need to pass some platform specific data though to the implementation. I think it is beyond of ODP platform independent specification. Implementation specific configuration could be delivered in variety of ways: Yes, Implementation configuration can be done is a multiple ways, but the point is we do need and see a requirement for being able to apply arguments to all three layer in a application — Application layer — ODP layer — SDK layer each of these layers require a basic way to provide configuration at build time, init time and run time. Runtime is for configuration that needs to be define on the fly and init time configuration is for system initialization only in most case and does not change normally after initialization. I am not concerned about 'Application layer' and 'SDK layer' at all. All that is beyond of ODP framework. ODP cannot dictate neither of these layers what to do and how to do init time configuration. I acknowledge that there are some consistency issues of command line arguments on 'Application layer' when it comes to ODP example and tests, but all that is not normative part of ODP and can be handled separate. You need to be concerned with the SDK and application layer at some point as ODP is not a standalone product and requires these two parts to even be usable or exist. ODP is trying to define a framework and part of that framework should be how it interacts with SDK and applications. I feel like we are at an impasse here and I would like to suggest another solution. int odp_init_global(int argc, char ** argv, odp_flags_t flags); This way ODP can get its values from the flags and can ignore the argc/argv, but just pass them alone to the SDK. I do not like this option and just passing flags is not reasonable IMO. Flags at some point have limits and the amount of information they can convey is limited. Even if you passed structures this has limits as well as anyone wanting to pass more information would need to modify or replace the structure. If each platform has it own structure that becomes a problem as well. See my other response 'flags' proposed on this diff do not have any sense to me. At least it was not explained well. I am not arguing for flags. IMO if there is any need to do any ODP layer specific init time config, it should be discussed separately API (function, struct, etc) and added to normative ODP API. One example from the past, before your time, that was discussed a bit, but dropped on the floor was need to have function pointers callbacks that ODP layer and below should use to report error and debug messages. How would you pass function pointers to ODP layer through set of strings? As we both know callbacks require a registration or the external code to have stub routines or fill in the function pointers directly. The only thing a string can do is add some requirement to the callback routine. The callbacks are outside the scope of the init-time configuration IMO and not a valid argument in this case. In your suggestion the application would have to convert argc/argv values into flags to be passed to ODP/SDK, but why the args came into the applications via a command line why not pass them along. I know the application could be called from another application and not from a command line, but I feel the common case is going to be from some type of command line. In systems of company from where I am assignee none of the process receive any meaningful arguments, relevant to this discussion. It could be anything, other than process arguments: o build time o EEPROM o startup config file o XML config file o Run time licensing o Run time detection of platform type (for single image that runs on different h/w) o standby reconfigured by active o ... Clearly in the case of argc/argv strings array API we would need to construct those arguments from structured information which we receive by other means. I find that ugly. So I don't see normal process argc/argv argument as dominant and defining way. This is taking the request to pass argc/argv to the extreme and expecting everything to be inited or set via a command line. This is also not a valid argument. ODP should define a framework attempting to remove as much work as it can from each platform and no more. By trying to define a clean framework we allow the developer to work on what is important SDK/application. ODP needs and must unify some of the code in each platform into a single common code base and provide clean methods to allow the SDK and application to add what is reasonable. As the three platforms stand today we have a lot of duplicate code and in some cases use code directly from the linux-generic platform in the other platform. These common set of routines need to be pulled into a common directory code base. Above is arguable. As far as I know many members have no intention at all to make their ODP layer implementation to be open source. In such situation implementation sharing is problematic. We have few now that could share some code, but clearly implementation sharing is not the goal. Sharing of API is the goal. I did not state you would need to make your secret sauce code open source in any fashion. All I stated was the three platforms I do see have a lot of common code and can be factored out to a set of common files between all platforms. As 'ODP layer' concerned, yes sure I can construct array of strings and pass it to some function. My worry is that makes ODP api to spread out ... Along with regular header files, enums, #defines, structs, function annotated by doxygen etc we would have some special language for such array of strings. These set of strings must be treated as API, i.e described precisely, agreed by all members, versioned along ODP spec life time. But what logistical means we have for that? For example doxygen handles all C API elments very well, how would you describe those strings in header files? Very long comment in odp_init_global function? IMO it creates logistical challenges and it is not worth it. For example how would you version such thing and do build time checks? With #define, struct, etc incomatibilties quit often will come out during build time (#define is not there, struct field is missing etc). But with array of strings it will be just run-time error. I do not understand the above statement as each layer is independent from the next in some ways except for common defines and prototypes. The ODP layer or application layer should never need to inspect or look at the SDK layer values, if they do then we have a bigger problem for portability. As for doxygen is very happy about including comments on function headers, file headers, defines, macros, arrays, typedefs or … so I do not see the problem with doxygen here. As far as 'ODP layer' concerned all those parameters would constitute API. My question how such API described, versioned, discovered by user, etc. Everyone system requires docs and comments plus a number of examples to teach everyone how to use that system ODP is no different. Anyway if difference between strict APIs and set of strings is so small why do you argue vs structured APIs the way followed by ODP everywhere else. I.e what benefits it brings to have API with array of strings API? Maybe I am missing this part. I guess I would ask you the same question :-) The rule of existing precedent :). We have all ODP APIs so far described as regular structured C APIs. Now we are trying to introduce parallel mechanism with set of strings and I argue that it is not worth it, because I don't see what benefits it brings (IMHO only issues). Please look at my previous response on the thread. If we would have ability to do init time configuration for 'ODP layer' through structured APIs and have some argc/argv convention for Application (== existing examples and tests) and SDK (one that wants to do init config in such way), where both areas are outside of normative part of ODP I am fine with it. IMO it could be reasonable compromise for current discussion. I think we are going around and around in circles now. It seems like my suggestion is being taken to the extreme and that was not my intent. Thanks ++Keith Thanks, Victor Thanks, Victor o build time: platform header file with bunch of constants, which are defined by user of library o special implementation specific set of calls that app does in beginning. We can survive implementation specific ifdef in some portion of our code o config file that ODP implementation reads A config file requires a filesystem, but let ignore that point for now. If we did provide a init file for ODP then we can provide a ‘-f foobar.config’ option on the command line. Another option is the application open and parse the config file converting them into argc/argv options to pass to odp_init_global() routine. o ... In my opinion it is up to ODP implementer how to do that. I do see some of the configuration must be defined by the developer, but we are starting to see some common threads and configuration we can supply today. I also assume the case or platform agnostic configuration that cannot be hard coded If there is a need for platform agnostic configuration in ODP we need to create proper API for that. That what I meant it should be APIs only. Pushing some meaning to set of strings IMHO is wrong way to go. I do not see how we are not making this platform agnostic as we still allow the SDK and application to have its own configuration without having to alter ODP for each platform. I am seeing a lot of duplicate code in each of the platforms we have today and if you look at the code I broke out the DPDK routine odp_init_dpdk() into odp_init_skd() instead, which now should allow most of the ODP code for initing system to be generic and common between the platforms. I did not fully include those changes as it a bit more work. - I am imagining ODP running in an Enterprise situation in multiple VMs where you want the same app attached to different ports or something like that. Serious deployment situation is way more complicated configuration parameters do not always come from args/argv. And serious application has its own set of rules and convention how to handle argc/argv. Just simple example what if those configuration parameters should be changed on the fly of they hey come from config file. With array of strings API you telling me that I need to construct array of strings and pass it to some API ... I find it not very clean. If follow that direction why do we define enumeration for crypto protocols supported let's pass it as 'char *’. I have developed and design a number of systems here are WR and you are correct that a product needs a lot of work to be a real product. My point is we can help reduce the workload of the developer by helping define at least a basic configuration system he can build on for his product. As for the crypto protocols I have not looked at them, but I think you are taking an extreme view here. Do not mean to offend anyone, but I do believe we need a basic configuration in all three areas for ODP. Thanks, Victor Thanks, Victor Need to push as much as possible into the implementation wherever possible Arguments should try very hard to be platform agnostic Arguments for a platform need to be marked and passed through verbatim - then duplication of existing flags on different sdks is ok and clear to the user. Maybe --dpdk followed by all the flags it expects. VMs may need some differences passed in, so not all differences can be compiled into the implementation. Need to avoid the case of two platforms claiming the same argument A strong case for scripting like Lua was not exposed in this discussion. Could I ask everyone to reply with any agnostic and platform specific arguments they would like to see, maybe then we can form a clear picture of those that are needed. On 5 August 2014 08:31, Taras Kondratiuk <taras.kondratiuk@linaro.org<mailto:taras.kondratiuk@linaro.org>> wrote: On 08/05/2014 02:59 PM, Mike Holmes wrote: How does this play in with Keiths resumed call for argv argc? Thread = "[lng-odp] Port of Pktgen to ODP" it looks like on two fronts we have come back to the need to add args to init that are more complex than basic flags. With argv argc, I assume we can pass these flags as options with getopt ? These flags are defined by API used by application, so they are hardcoded into application, but not passed from command line. If we are going to argv/argc way, then application can just insert one more parameter into the list, before passing it to odp_init_global(). Some issues are 1. passing things through to underlying SDKs 2. Making some args mandatory 3. Keeping the argument parsing in sync across platforms 4. Not coupling all the platforms and their special args together 5. Not promoting the use of platform specific args in general as they will break portability. I agree with an approach when only common platform independent arguments are passed to ODP application. We have to analyze which parameters are required by different platforms and come up with a common list. The rest platform specific parameters have to passed to implementation in some other way (environment variable, config file, hardcoded, etc.) -- Mike Holmes Linaro Technical Manager / Lead LNG - ODP
Hi Taras,
Looks like this maybe a possible direction, but I feel adding some init support into ODP is a good thing.
Keith Wiles, Principal Technologist with CTO office, Wind River mobile 972-213-5533
On Aug 7, 2014, at 3:08 PM, Taras Kondratiuk <taras.kondratiuk@linaro.org<mailto:taras.kondratiuk@linaro.org>> wrote:
On 08/07/2014 10:36 PM, Wiles, Roger Keith wrote:
*Keith **Wiles*, Principal Technologist with CTO office, *Wind River*
mobile 972-213-5533
On Aug 7, 2014, at 2:21 PM, Taras Kondratiuk
<taras.kondratiuk@linaro.org<mailto:taras.kondratiuk@linaro.org> <mailto:taras.kondratiuk@linaro.org>> wrote:
From implementation point it is not an issue to parse configuration
strings, but it just feels wrong. The fact that DPDK does it doesn't
mean it is a good approach.
I'd prefer to pass generic parameters via structure to odp_init() and
let implementation to figure out how to get other stuff if needed.
In case of Linux-based implementations there are many ways to do it.
I do understand, but DPDK is inited in this fashion and we would need to
supply a argc/argv parameters to call its init routine. Now it is being
done in the odp_init_dpdk() function and it can not support all of the
args. We need to pass args to the SDK in some way and argc/argv is the
best solution for DPDK. Now we can do as I suggested by adding
‘—sdk-args=“”’ to the command line or the odp_init_dpdk() routine will
need to get the values some place. It just seems similar to add them to
command line, even if ODP is not really launched from a command line.
The first approach that comes to my mind is an environment variable.
DPDK_CMD_LINE='some configs' odp_foobar
then linux-dpdk can getevn('DPDK_CMD_LINE') and everybody happy.
API is not changed and DPDK have its configuration.
Are we discussing solutions before we have agreed on the problem? On 8 August 2014 01:52, Wiles, Roger Keith <keith.wiles@windriver.com> wrote: > Hi Taras, > > Looks like this maybe a possible direction, but I feel adding some init > support into ODP is a good thing. > > *Keith **Wiles*, Principal Technologist with CTO office, *Wind River *mobile > 972-213-5533 > > On Aug 7, 2014, at 3:08 PM, Taras Kondratiuk <taras.kondratiuk@linaro.org> > wrote: > > On 08/07/2014 10:36 PM, Wiles, Roger Keith wrote: > > > *Keith **Wiles*, Principal Technologist with CTO office, *Wind River* > mobile 972-213-5533 > On Aug 7, 2014, at 2:21 PM, Taras Kondratiuk > <taras.kondratiuk@linaro.org <mailto:taras.kondratiuk@linaro.org > <taras.kondratiuk@linaro.org>>> wrote: > > > From implementation point it is not an issue to parse configuration > strings, but it just feels wrong. The fact that DPDK does it doesn't > mean it is a good approach. > > I'd prefer to pass generic parameters via structure to odp_init() and > let implementation to figure out how to get other stuff if needed. > In case of Linux-based implementations there are many ways to do it. > > > I do understand, but DPDK is inited in this fashion and we would need to > supply a argc/argv parameters to call its init routine. Now it is being > done in the odp_init_dpdk() function and it can not support all of the > args. We need to pass args to the SDK in some way and argc/argv is the > best solution for DPDK. Now we can do as I suggested by adding > ‘—sdk-args=“”’ to the command line or the odp_init_dpdk() routine will > need to get the values some place. It just seems similar to add them to > command line, even if ODP is not really launched from a command line. > > > The first approach that comes to my mind is an environment variable. > > DPDK_CMD_LINE='some configs' odp_foobar > > then linux-dpdk can getevn('DPDK_CMD_LINE') and everybody happy. > API is not changed and DPDK have its configuration. > > > > _______________________________________________ > lng-odp mailing list > lng-odp@lists.linaro.org > http://lists.linaro.org/mailman/listinfo/lng-odp > >
On 08/12/2014 05:21 PM, Ola Liljedahl wrote: > Are we discussing solutions before we have agreed on the problem? > +1 Lets discuss that on todays meeting. If there is time slot for that. Maxim. > > On 8 August 2014 01:52, Wiles, Roger Keith <keith.wiles@windriver.com > <mailto:keith.wiles@windriver.com>> wrote: > > Hi Taras, > > Looks like this maybe a possible direction, but I feel adding some > init support into ODP is a good thing. > > *Keith **Wiles*, Principal Technologist with CTO office, *Wind > River*mobile 972-213-5533 <tel:972-213-5533> > > On Aug 7, 2014, at 3:08 PM, Taras Kondratiuk > <taras.kondratiuk@linaro.org <mailto:taras.kondratiuk@linaro.org>> > wrote: > >> On 08/07/2014 10:36 PM, Wiles, Roger Keith wrote: >>> >>> *Keith **Wiles*, Principal Technologist with CTO office, *Wind >>> River* >>> mobile 972-213-5533 <tel:972-213-5533> >>> On Aug 7, 2014, at 2:21 PM, Taras Kondratiuk >>> <taras.kondratiuk@linaro.org >>> <mailto:taras.kondratiuk@linaro.org> >>> <mailto:taras.kondratiuk@linaro.org>> wrote: >>>> >>>> From implementation point it is not an issue to parse configuration >>>> strings, but it just feels wrong. The fact that DPDK does it >>>> doesn't >>>> mean it is a good approach. >>>> >>>> I'd prefer to pass generic parameters via structure to >>>> odp_init() and >>>> let implementation to figure out how to get other stuff if needed. >>>> In case of Linux-based implementations there are many ways to >>>> do it. >>> >>> I do understand, but DPDK is inited in this fashion and we would >>> need to >>> supply a argc/argv parameters to call its init routine. Now it >>> is being >>> done in the odp_init_dpdk() function and it can not support all >>> of the >>> args. We need to pass args to the SDK in some way and argc/argv >>> is the >>> best solution for DPDK. Now we can do as I suggested by adding >>> ‘—sdk-args=“”’ to the command line or the odp_init_dpdk() >>> routine will >>> need to get the values some place. It just seems similar to add >>> them to >>> command line, even if ODP is not really launched from a command >>> line. >> >> The first approach that comes to my mind is an environment variable. >> >> DPDK_CMD_LINE='some configs' odp_foobar >> >> then linux-dpdk can getevn('DPDK_CMD_LINE') and everybody happy. >> API is not changed and DPDK have its configuration. > > > _______________________________________________ > lng-odp mailing list > lng-odp@lists.linaro.org <mailto:lng-odp@lists.linaro.org> > http://lists.linaro.org/mailman/listinfo/lng-odp > > > > > _______________________________________________ > lng-odp mailing list > lng-odp@lists.linaro.org > http://lists.linaro.org/mailman/listinfo/lng-odp
diff --git a/example/generator/odp_generator.c b/example/generator/odp_generator.c index e4a72fa..af1ff89 100644 --- a/example/generator/odp_generator.c +++ b/example/generator/odp_generator.c @@ -530,7 +530,7 @@ int main(int argc, char *argv[]) int core_count; /* Init ODP before calling anything else */ - if (odp_init_global()) { + if (odp_init_global(ODP_INIT_F_ALL)) { ODP_ERR("Error: ODP global init failed.\n"); exit(EXIT_FAILURE); } diff --git a/example/l2fwd/odp_l2fwd.c b/example/l2fwd/odp_l2fwd.c index e331ff2..6b6c75a 100644 --- a/example/l2fwd/odp_l2fwd.c +++ b/example/l2fwd/odp_l2fwd.c @@ -288,7 +288,7 @@ int main(int argc, char *argv[]) odp_pktio_t pktio; /* Init ODP before calling anything else */ - if (odp_init_global()) { + if (odp_init_global(ODP_INIT_F_ALL)) { ODP_ERR("Error: ODP global init failed.\n"); exit(EXIT_FAILURE); } diff --git a/example/odp_example/odp_example.c b/example/odp_example/odp_example.c index be96093..f6197f8 100644 --- a/example/odp_example/odp_example.c +++ b/example/odp_example/odp_example.c @@ -934,7 +934,7 @@ int main(int argc, char *argv[]) memset(thread_tbl, 0, sizeof(thread_tbl)); - if (odp_init_global()) { + if (odp_init_global(ODP_INIT_F_ALL)) { printf("ODP global init failed.\n"); return -1; } diff --git a/example/packet/odp_pktio.c b/example/packet/odp_pktio.c index edf8cfd..8079fd0 100644 --- a/example/packet/odp_pktio.c +++ b/example/packet/odp_pktio.c @@ -285,7 +285,7 @@ int main(int argc, char *argv[]) int core_count; /* Init ODP before calling anything else */ - if (odp_init_global()) { + if (odp_init_global(ODP_INIT_F_ALL & ~ODP_INIT_F_TIMER)) { ODP_ERR("Error: ODP global init failed.\n"); exit(EXIT_FAILURE); } diff --git a/example/packet_netmap/odp_pktio_netmap.c b/example/packet_netmap/odp_pktio_netmap.c index 7d33b19..9c5ee6b 100644 --- a/example/packet_netmap/odp_pktio_netmap.c +++ b/example/packet_netmap/odp_pktio_netmap.c @@ -201,7 +201,7 @@ int main(int argc, char *argv[]) int i; /* Init ODP before calling anything else */ - if (odp_init_global()) { + if (odp_init_global(ODP_INIT_F_ALL & ~ODP_INIT_F_TIMER)) { ODP_ERR("Error: ODP global init failed.\n"); exit(EXIT_FAILURE); } diff --git a/example/timer/odp_timer_test.c b/example/timer/odp_timer_test.c index dbe0e5b..4e566bc 100644 --- a/example/timer/odp_timer_test.c +++ b/example/timer/odp_timer_test.c @@ -210,7 +210,7 @@ int main(int argc, char *argv[]) memset(thread_tbl, 0, sizeof(thread_tbl)); - if (odp_init_global()) { + if (odp_init_global(ODP_INIT_F_ALL)) { printf("ODP global init failed.\n"); return -1; } diff --git a/include/odp_init.h b/include/odp_init.h index 490324a..f119e99 100644 --- a/include/odp_init.h +++ b/include/odp_init.h @@ -22,6 +22,13 @@ extern "C" { #include <odp_std_types.h> +#define ODP_INIT_F_ALL 0xff +#define ODP_INIT_F_SHMINIT (1 << 0) +#define ODP_INIT_F_GPOOL (1 << 1) +#define ODP_INIT_F_GQUEUE (1 << 2) +#define ODP_INIT_F_SCHEDULE (1 << 3) +#define ODP_INIT_F_PKTIO (1 << 4) +#define ODP_INIT_F_TIMER (1 << 5) @@ -31,9 +38,10 @@ extern "C" { * This function must be called once before calling * any other ODP API functions. * + * @param mask Mask odp layers to be initialized * @return 0 if successful */ -int odp_init_global(void); +int odp_init_global(uint8_t mask); /** diff --git a/platform/linux-dpdk/odp_init.c b/platform/linux-dpdk/odp_init.c index ecc2066..76ac94d 100644 --- a/platform/linux-dpdk/odp_init.c +++ b/platform/linux-dpdk/odp_init.c @@ -50,7 +50,7 @@ int odp_init_dpdk(void) return 0; } -int odp_init_global(void) +int odp_init_global(uint8_t mask) { odp_thread_init_global(); @@ -61,34 +61,46 @@ int odp_init_global(void) return -1; } - if (odp_shm_init_global()) { - ODP_ERR("ODP shm init failed.\n"); - return -1; + if (mask & ODP_INIT_F_SHMINIT) { + if (odp_shm_init_global()) { + ODP_ERR("ODP shm init failed.\n"); + return -1; + } } - if (odp_buffer_pool_init_global()) { - ODP_ERR("ODP buffer pool init failed.\n"); - return -1; + if (mask & ODP_INIT_F_GPOOL) { + if (odp_buffer_pool_init_global()) { + ODP_ERR("ODP buffer pool init failed.\n"); + return -1; + } } - if (odp_queue_init_global()) { - ODP_ERR("ODP queue init failed.\n"); - return -1; + if (mask & ODP_INIT_F_GQUEUE) { + if (odp_queue_init_global()) { + ODP_ERR("ODP queue init failed.\n"); + return -1; + } } - if (odp_schedule_init_global()) { - ODP_ERR("ODP schedule init failed.\n"); - return -1; + if (mask & ODP_INIT_F_SCHEDULE) { + if (odp_schedule_init_global()) { + ODP_ERR("ODP schedule init failed.\n"); + return -1; + } } - if (odp_pktio_init_global()) { - ODP_ERR("ODP packet io init failed.\n"); - return -1; + if (mask & ODP_INIT_F_PKTIO) { + if (odp_pktio_init_global()) { + ODP_ERR("ODP packet io init failed.\n"); + return -1; + } } - if (odp_timer_init_global()) { - ODP_ERR("ODP timer init failed.\n"); - return -1; + if (mask & ODP_INIT_F_TIMER) { + if (odp_timer_init_global()) { + ODP_ERR("ODP timer init failed.\n"); + return -1; + } } return 0; diff --git a/platform/linux-generic/odp_init.c b/platform/linux-generic/odp_init.c index d4c2eb8..7649f9c 100644 --- a/platform/linux-generic/odp_init.c +++ b/platform/linux-generic/odp_init.c @@ -9,40 +9,52 @@ #include <odp_debug.h> -int odp_init_global(void) +int odp_init_global(uint8_t mask) { odp_thread_init_global(); odp_system_info_init(); - if (odp_shm_init_global()) { - ODP_ERR("ODP shm init failed.\n"); - return -1; + if (mask & ODP_INIT_F_SHMINIT) { + if (odp_shm_init_global()) { + ODP_ERR("ODP shm init failed.\n"); + return -1; + } } - if (odp_buffer_pool_init_global()) { - ODP_ERR("ODP buffer pool init failed.\n"); - return -1; + if (mask & ODP_INIT_F_GPOOL) { + if (odp_buffer_pool_init_global()) { + ODP_ERR("ODP buffer pool init failed.\n"); + return -1; + } } - if (odp_queue_init_global()) { - ODP_ERR("ODP queue init failed.\n"); - return -1; + if (mask & ODP_INIT_F_GQUEUE) { + if (odp_queue_init_global()) { + ODP_ERR("ODP queue init failed.\n"); + return -1; + } } - if (odp_schedule_init_global()) { - ODP_ERR("ODP schedule init failed.\n"); - return -1; + if (mask & ODP_INIT_F_SCHEDULE) { + if (odp_schedule_init_global()) { + ODP_ERR("ODP schedule init failed.\n"); + return -1; + } } - if (odp_pktio_init_global()) { - ODP_ERR("ODP packet io init failed.\n"); - return -1; + if (mask & ODP_INIT_F_PKTIO) { + if (odp_pktio_init_global()) { + ODP_ERR("ODP packet io init failed.\n"); + return -1; + } } - if (odp_timer_init_global()) { - ODP_ERR("ODP timer init failed.\n"); - return -1; + if (mask & ODP_INIT_F_TIMER) { + if (odp_timer_init_global()) { + ODP_ERR("ODP timer init failed.\n"); + return -1; + } } return 0; diff --git a/platform/linux-keystone2/odp_init.c b/platform/linux-keystone2/odp_init.c index f832551..de28fda 100644 --- a/platform/linux-keystone2/odp_init.c +++ b/platform/linux-keystone2/odp_init.c @@ -90,7 +90,7 @@ static int ti_init_hw_config(void) } -int odp_init_global(void) +int odp_init_global(uint8_t mask) { odp_thread_init_global(); @@ -99,34 +99,46 @@ int odp_init_global(void) ti_em_osal_core_init_global(); ti_init_hw_config(); - if (odp_shm_init_global()) { - ODP_ERR("ODP shm init failed.\n"); - return -1; + if (mask & ODP_INIT_F_SHMINIT) { + if (odp_shm_init_global()) { + ODP_ERR("ODP shm init failed.\n"); + return -1; + } } - if (odp_buffer_pool_init_global()) { - ODP_ERR("ODP buffer pool init failed.\n"); - return -1; + if (mask & ODP_INIT_F_GPOOL) { + if (odp_buffer_pool_init_global()) { + ODP_ERR("ODP buffer pool init failed.\n"); + return -1; + } } - if (odp_queue_init_global()) { - ODP_ERR("ODP queue init failed.\n"); - return -1; + if (mask & ODP_INIT_F_GQUEUE) { + if (odp_queue_init_global()) { + ODP_ERR("ODP queue init failed.\n"); + return -1; + } } - if (odp_schedule_init_global()) { - ODP_ERR("ODP schedule init failed.\n"); - return -1; + if (mask & ODP_INIT_F_SCHEDULE) { + if (odp_schedule_init_global()) { + ODP_ERR("ODP schedule init failed.\n"); + return -1; + } } - if (odp_pktio_init_global()) { - ODP_ERR("ODP packet io init failed.\n"); - return -1; + if (mask & ODP_INIT_F_PKTIO) { + if (odp_pktio_init_global()) { + ODP_ERR("ODP packet io init failed.\n"); + return -1; + } } - if (odp_timer_init_global()) { - ODP_ERR("ODP timer init failed.\n"); - return -1; + if (mask & ODP_INIT_F_TIMER) { + if (odp_timer_init_global()) { + ODP_ERR("ODP timer init failed.\n"); + return -1; + } } return 0; diff --git a/test/api_test/odp_common.c b/test/api_test/odp_common.c index 89ebd2d..dcb13ed 100644 --- a/test/api_test/odp_common.c +++ b/test/api_test/odp_common.c @@ -56,7 +56,7 @@ int odp_test_global_init(void) { memset(thread_tbl, 0, sizeof(thread_tbl)); - if (odp_init_global()) { + if (odp_init_global(ODP_INIT_F_ALL)) { ODP_ERR("ODP global init failed.\n"); return -1; }
Add ability to specify which odp layers should be initialized. Signed-off-by: Maxim Uvarov <maxim.uvarov@linaro.org> --- v2: change logic according to v1. If flag set layer will be initialized. example/generator/odp_generator.c | 2 +- example/l2fwd/odp_l2fwd.c | 2 +- example/odp_example/odp_example.c | 2 +- example/packet/odp_pktio.c | 2 +- example/packet_netmap/odp_pktio_netmap.c | 2 +- example/timer/odp_timer_test.c | 2 +- include/odp_init.h | 10 ++++++- platform/linux-dpdk/odp_init.c | 50 ++++++++++++++++++++------------ platform/linux-generic/odp_init.c | 50 ++++++++++++++++++++------------ platform/linux-keystone2/odp_init.c | 50 ++++++++++++++++++++------------ test/api_test/odp_common.c | 2 +- 11 files changed, 109 insertions(+), 65 deletions(-)