Pass the capability matrix's models as a pytest option
CI / checks (push) Skipped
CI / checks (pull_request) Successful in 51s
CI / checks (push) Skipped
CI / checks (pull_request) Successful in 51s
NISSE_CAPABILITY_MODELS was neither a secret nor configuration: which models to measure is a question for one run, so it is now an argument, --capability-models=a,b, declared in tests/conftest.py where pytest reads options at startup. With no models the matrix skips as before, so CI is unchanged. The last non-secret NISSE_* variable other than NISSE_CONFIG and NISSE_ENV is gone. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This commit is contained in:
1 parent
15d69799bf
commit
58b439088f
5 files changed
+53
-25
No files matched your search
@@ -0,0 +1,20 @@
|
||||
"""Command-line options for the repository-level test suites.
|
||||
|
||||
Declared here rather than beside the tests that use them: pytest only reads
|
||||
options from conftest files it loads at startup, and this directory is one
|
||||
of the configured test paths.
|
||||
"""
|
||||
|
||||
import pytest
|
||||
|
||||
|
||||
def pytest_addoption(parser: pytest.Parser) -> None:
|
||||
parser.addoption(
|
||||
"--capability-models",
|
||||
default="",
|
||||
help=(
|
||||
"comma-separated LiteLLM aliases to measure with the capability "
|
||||
"matrix (tests/integration/test_model_capability.py). Spends real "
|
||||
"tokens on every model listed; the matrix skips when this is empty."
|
||||
),
|
||||
)
|
||||
Reference in new issue
Block a user