| Message ID | 20200615140137.21186-5-patrick.delaunay@st.com |
|---|---|
| State | New |
| Headers | show |
| Series | cmd: env: add option for quiet output on env info | expand |
On 6/15/20 8:01 AM, Patrick Delaunay wrote: > Add a pytest for testing the env info sub-command: > > test_env_info: test command with several option > > test_env_info_quiet: test the result of the sub-command with quiet option, > '-q' as used for support in shell test; for example: > if env info -p -d -q; then env save; fi > diff --git a/test/py/tests/test_env.py b/test/py/tests/test_env.py > + at pytest.mark.boardspec('sandbox') > + at pytest.mark.buildconfigspec('cmd_nvedit_info') > +def test_env_info(state_test_env): The body of these tests doesn't look like it tests something that's specific to sandbox, so I'm not sure why the test function is marked to only run on sandbox. Is it simply because other boards may store the environment differently and/or have valid saved environment in flash, so the responses to e.g. "env info" aren't the same everywhere? If so, I imagine that test_env_info_quiet() doesn't need to be sandbox-only, since there's no output in that case.
Hi Stephen, > From: Stephen Warren <swarren at wwwdotorg.org> > Sent: mardi 16 juin 2020 00:09 > > On 6/15/20 8:01 AM, Patrick Delaunay wrote: > > Add a pytest for testing the env info sub-command: > > > > test_env_info: test command with several option > > > > test_env_info_quiet: test the result of the sub-command with quiet > > option, '-q' as used for support in shell test; for example: > > if env info -p -d -q; then env save; fi > > > diff --git a/test/py/tests/test_env.py b/test/py/tests/test_env.py > > > + at pytest.mark.boardspec('sandbox') > > + at pytest.mark.buildconfigspec('cmd_nvedit_info') > > +def test_env_info(state_test_env): > > The body of these tests doesn't look like it tests something that's specific to > sandbox, so I'm not sure why the test function is marked to only run on sandbox. > Is it simply because other boards may store the environment differently and/or > have valid saved environment in flash, so the responses to e.g. "env info" aren't > the same everywhere? If so, I imagine that test_env_info_quiet() doesn't need to > be sandbox-only, since there's no output in that case. The test is not really sandbox specific but I don't have easy way to know on real board the ENV configuration (for the resut of command env info -p -d). In the test, I assume that at least CONFIG_ENV_IS_.... is activated (for persistent storage) and if this target is selected in the weak function env_get_location. And "env save" as be not be executed (default environment is used). And with quiet option, the test the environment if is persistent (result of "env -p -q" is 0) or using default ("env -d -q" result is 0). And in the next patch http://patchwork.ozlabs.org/project/uboot/patch/20200616074048.7898-10-patrick.delaunay at st.com/ As the command "env erase" is not always supported according he environment target. I could test on real hardware but I need to check if I test all the possible result. Patrick
On 6/16/20 2:01 AM, Patrick DELAUNAY wrote: > Hi Stephen, > >> From: Stephen Warren <swarren at wwwdotorg.org> >> Sent: mardi 16 juin 2020 00:09 >> >> On 6/15/20 8:01 AM, Patrick Delaunay wrote: >>> Add a pytest for testing the env info sub-command: >>> >>> test_env_info: test command with several option >>> >>> test_env_info_quiet: test the result of the sub-command with quiet >>> option, '-q' as used for support in shell test; for example: >>> if env info -p -d -q; then env save; fi >> >>> diff --git a/test/py/tests/test_env.py b/test/py/tests/test_env.py >> >>> + at pytest.mark.boardspec('sandbox') >>> + at pytest.mark.buildconfigspec('cmd_nvedit_info') >>> +def test_env_info(state_test_env): >> >> The body of these tests doesn't look like it tests something that's specific to >> sandbox, so I'm not sure why the test function is marked to only run on sandbox. >> Is it simply because other boards may store the environment differently and/or >> have valid saved environment in flash, so the responses to e.g. "env info" aren't >> the same everywhere? If so, I imagine that test_env_info_quiet() doesn't need to >> be sandbox-only, since there's no output in that case. > > The test is not really sandbox specific but I don't have easy way to know on real board > the ENV configuration (for the resut of command env info -p -d). > > In the test, I assume that at least CONFIG_ENV_IS_.... is activated (for persistent storage) > and if this target is selected in the weak function env_get_location. > And "env save" as be not be executed (default environment is used). > > And with quiet option, the test the environment if is persistent (result of "env -p -q" is 0) > or using default ("env -d -q" result is 0). > > And in the next patch > http://patchwork.ozlabs.org/project/uboot/patch/20200616074048.7898-10-patrick.delaunay at st.com/ > > As the command "env erase" is not always supported according he environment target. > > I could test on real hardware but I need to check if I test all the possible result. OK, I guess that makes sense for a start. For testing on real HW, the typical approach would be to require that the board's test configuration define some env__xxx variables that define its capabilities. Then, the test can be made to depend on those values, or whether those variables are set at all.
Hi Stephen, > From: Stephen Warren <swarren at wwwdotorg.org> > Sent: jeudi 18 juin 2020 00:32 > > On 6/16/20 2:01 AM, Patrick DELAUNAY wrote: > > Hi Stephen, > > > >> From: Stephen Warren <swarren at wwwdotorg.org> > >> Sent: mardi 16 juin 2020 00:09 > >> > >> On 6/15/20 8:01 AM, Patrick Delaunay wrote: > >>> Add a pytest for testing the env info sub-command: > >>> > >>> test_env_info: test command with several option > >>> > >>> test_env_info_quiet: test the result of the sub-command with quiet > >>> option, '-q' as used for support in shell test; for example: > >>> if env info -p -d -q; then env save; fi > >> > >>> diff --git a/test/py/tests/test_env.py b/test/py/tests/test_env.py > >> > >>> + at pytest.mark.boardspec('sandbox') > >>> + at pytest.mark.buildconfigspec('cmd_nvedit_info') > >>> +def test_env_info(state_test_env): > >> > >> The body of these tests doesn't look like it tests something that's > >> specific to sandbox, so I'm not sure why the test function is marked to only run > on sandbox. > >> Is it simply because other boards may store the environment > >> differently and/or have valid saved environment in flash, so the > >> responses to e.g. "env info" aren't the same everywhere? If so, I > >> imagine that test_env_info_quiet() doesn't need to be sandbox-only, since > there's no output in that case. > > > > The test is not really sandbox specific but I don't have easy way to > > know on real board the ENV configuration (for the resut of command env info -p > -d). > > > > In the test, I assume that at least CONFIG_ENV_IS_.... is activated > > (for persistent storage) and if this target is selected in the weak function > env_get_location. > > And "env save" as be not be executed (default environment is used). > > > > And with quiet option, the test the environment if is persistent > > (result of "env -p -q" is 0) or using default ("env -d -q" result is 0). > > > > And in the next patch > > http://patchwork.ozlabs.org/project/uboot/patch/20200616074048.7898-10 > > -patrick.delaunay at st.com/ > > > > As the command "env erase" is not always supported according he environment > target. > > > > I could test on real hardware but I need to check if I test all the possible result. > > OK, I guess that makes sense for a start. But I will propose a V5 to check command on real hardware Just modify test_env_info to check the all possible strings. > For testing on real HW, the typical approach would be to require that the board's > test configuration define some env__xxx variables that define its capabilities. > Then, the test can be made to depend on those values, or whether those variables > are set at all. For the next steps, I need to thinks about tests because ENV location is not only impacted by CONFIG_ENV_IS_IN_.... or CONFIG_ENV_IS_NOWHERE These defined can be activated simultaneously and env location is detected at run time: so it is difficult to predict the 'env info' result on real hardware. On sandbox it is fixed because ENVL_NOWHERE is selected by default. Patrick
diff --git a/test/py/tests/test_env.py b/test/py/tests/test_env.py index 6ff38f1020..cbdb41031c 100644 --- a/test/py/tests/test_env.py +++ b/test/py/tests/test_env.py @@ -336,3 +336,47 @@ def test_env_import_whitelist_delete(state_test_env): unset_var(state_test_env, 'foo2') unset_var(state_test_env, 'foo3') unset_var(state_test_env, 'foo4') + + at pytest.mark.boardspec('sandbox') + at pytest.mark.buildconfigspec('cmd_nvedit_info') +def test_env_info(state_test_env): + + """Test 'env info' command with several options. + """ + c = state_test_env.u_boot_console + + response = c.run_command('env info') + assert 'env_valid = invalid' in response + assert 'env_ready = true' in response + assert 'env_use_default = true' in response + + response = c.run_command('env info -p -d') + assert 'Default environment is used' in response + assert 'Environment cannot be persisted' in response + + response = c.run_command('env info -p -d -q') + assert response == "" + + at pytest.mark.boardspec('sandbox') + at pytest.mark.buildconfigspec('cmd_nvedit_info') + at pytest.mark.buildconfigspec('cmd_echo') +def test_env_info_quiet(state_test_env): + + """Test 'env info' quiet command result with several options for test. + """ + c = state_test_env.u_boot_console + + response = c.run_command('env info -d -q') + assert response == "" + response = c.run_command('echo $?') + assert response == "0" + + response = c.run_command('env info -p -q') + assert response == "" + response = c.run_command('echo $?') + assert response == "1" + + response = c.run_command('env info -d -p -q') + assert response == "" + response = c.run_command('echo $?') + assert response == "1"
Add a pytest for testing the env info sub-command: test_env_info: test command with several option test_env_info_quiet: test the result of the sub-command with quiet option, '-q' as used for support in shell test; for example: if env info -p -d -q; then env save; fi Signed-off-by: Patrick Delaunay <patrick.delaunay at st.com> --- Changes in v4: - rebase on master branch - move 5/7 stm32mp1: configs: activate CMD_ERASEENV in a new serie 183380 - move 2/7 and 4/7 in a new serie 183387 Changes in v3: - update commit message (sub-commandi) - rename test_env_info_test to test_env_info_quiet Changes in v2: - add pytest test_env_info and test_env_info_test (new) test/py/tests/test_env.py | 44 +++++++++++++++++++++++++++++++++++++++ 1 file changed, 44 insertions(+)