I want to develop internal training content for writing good python unit and integration tests. There are three deliverables for this project:
* A longform document describing and exploring the subject matter in depth
* A 20 minute presentation walking through the content and examples
* An interactive tutorial / workshop applying the concepts in the document to concrete examples, with some automated evaluation based on coverage and hidden implementation bugs. This may be done either as a jupyter notebook, a dockerised training repository, an interactive webpage or similar, but ideally there should not be any backend necessary.
Familiarity with the subject matter is required, but I will be available to help develop and edit the content. The examples will walk through a simple microservices stack utilising the following python technologies:
* Flask HTTP server
* This interprets a REST query, validates the input and then dispatches the query to a microservice via a thrift RPC call
* Apache Thrift mid-layer microservice
* This implements permission checks and aggregates and interprets data into a series of database requests in one of the two data layers described below
* SQLAlchemy query constructed within the same service
* Translating a thrift request to a similarly-structured SQLAlchemy query
* Via another thrift microservice that uses mysql connector and raw SQL as a DAO service
A skeleton describing the content is provided below. It may be incomplete - if there is anything else to cover, please expand it or alter it. Please note that the examples do not have to be written from scratch - some existing skeleton code exists, just focus on the test structure and content.
Testing Best practices
* Test only public methods/functions
* Test only the external behaviour of a class or module, otherwise you are just making the implementation details harder to change
* Writing descriptive tests
* Each test should have a docstring describing the property being validated
* Quick note on TDD
* Keep tests DRY and isolated
* Test code is code too - share mocks and setup methods and keep it clean
* Avoid complex code in tests
* Tests need to be obviously correct - if you’re doing a lot of computation in a test, it is too complex.
* Data providers
* Don’t hard-code data, use data providers. This makes tests generalisable as property tests, too
* Golden Tests
* Quick note on hypothesis
* Avoid mocks & a note on DI
* Mocks are fragile - avoid them where you can. Your functions should be as much of a black box to your tests as possible. DI-like patterns help with this.
* Mock at the service boundaries
* This tests as much of the real behaviour of your code as possible.
* Wrap your service boundaries with an adapter that isolates your code from the interface of the external system.
* Python mock library
* Basic usage, common pitfalls, examples, etc
* How to write good mocks
* Unit tests vs Integration tests
* What’s the difference? Why would you use either?
* When not to test
* Type systems and other static tooling
* Coverage vs defensiveness
* Enforcing Contracts
* Nosetests, pytest
* Skipping tests, selecting tests, command line arguments, data providers[d][e][f]
* If it’s not covered, do you need it?
Tools: Pytest watch, pdb (and ptpdb), vscode integrations
Testing in the example stack
* creating and using the flask mocks
* A simple test
* Using a local thrift server mock
* Pure python
* Testing a simple permission check
* Testing using a local mysql instance
* Same as above, but with sqlalchemy set up
* Hypothesis[g] and Property based testing / state machine testing