I'm currently overriding the test command from setuptools in my setup.py script to run my test suite, and as part of this I import pytest and call pytest.main() with the args I wish to pass in. Unfortunately, this function assumes that it's being run as a script, being passed in the arguments directly from the command-line. As a result, it can provide a confusing error output, for example:

$ python setup.py test
- snip -
running build_ext
usage: setup.py [options] [file_or_dir] [file_or_dir] [...]
setup.py: error: unrecognized arguments: --clear-cache
  inifile: /myproject/setup.cfg
  rootdir: /myproject

This error message would imply that the setup.py program can't recognise the option --clear-cache, even though it isn't present on the command-line. In fact, there's no indication that py.test is involved at all here.

A potential solution might be to only display these messages when the program running is the actual py.test script. In other cases, the error should not include usage notes, and should directly mention that pytest.main is being used:

$ python setup.py test
- snip -
running build_ext
error: unrecognized arguments passed to pytest.main(): --clear-cache
  inifile: /myproject/setup.cfg
  rootdir: /myproject

Alternatively, if the user isn't running py.test, this message might not be printed out at all, and instead an exception might be thrown that can be caught (if required) by the calling code, that has attributes that describe the error. The calling code can then decide how to display this error in the most appropriate format.