

grep tiny tests examples
Here is a an example of a script that created the tiny model
stas/tiny-wmt19-en-de. You can easily adjust it to your specific
model's architecture.
It's easy to measure the run-time incorrectly if for example there is an overheard of downloading a huge model, but if
you test it locally the downloaded files would be cached and thus the download time not measured. Hence check the
execution speed report in CI logs instead (the output of pytest --durations=0 tests).
That report is also useful to find slow outliers that aren't marked as such, or which need to be re-written to be fast.
If you notice that the test suite starts getting slow on CI, the top listing of this report will show the slowest
tests.
Testing the stdout/stderr output
In order to test functions that write to stdout and/or stderr, the test can access those streams using the
pytest's capsys system. Here is how this is accomplished:
thon
import sys
def print_to_stdout(s):
    print(s)
def print_to_stderr(s):
    sys.stderr.write(s)
def test_result_and_stdout(capsys):
    msg = "Hello"
    print_to_stdout(msg)
    print_to_stderr(msg)
    out, err = capsys.readouterr()  # consume the captured output streams
    # optional: if you want to replay the consumed streams:
    sys.stdout.write(out)
    sys.stderr.write(err)
    # test:
    assert msg in out
    assert msg in err