Rooba
/
lightning
mirror of https://github.com/Lightning-AI/lightning.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

Updated Review guidelines (markdown)

edenlightning 2020-08-04 18:23:50 -04:00
parent 6ee879ca28
commit af50db9bae
1 changed files with 47 additions and 12 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

59
Review-guidelines.md

@ -1,17 +1,52 @@
Some recommended steps for reviewing open PRs. “Reviews allow for discussion of proposed changes and help ensure that the changes meet the repository’s contributing guidelines and other quality standards.“ (from GitHub documentation)
1. Check the **source issues if the fix solves it** ## How do I review a PR?
2. Read the changes if they are aligned with our code-base
3. Check if the changes (fixed issue, new feature) are **covered by tests** ### 1. Review the PR Description
4. Check if the change is mentioned in **CHANGELOG** (no need for typos, docs and CI) - Make sure that the PR title summarizes the change (not "fixes bug #2168").
5. If you checked all the above points, you may approve the PR in your review or request changes. Only approve it if you are confident that the issue or feature is fully addressed, otherwise leave a comment with your concerns. The actual `mergify` automation is set for N approvals so each approval matter and you shall be confident with your action... the rule of the tongue says: - Make sure there's an issue associated with this PR. No issue associated? Look for the relevant issue or ask the author to create one.
- Approve: if you completely agree with the content to be merged as it is now (no extra condition in comments) - Make sure PR has the right labels (bugfix, docs, tests, enhancements, etc.).
- Comment: not sure in any direction (your understanding, affect in global, etc.) but you want to express your opinion about a specific point - Make sure PR description includes:
- Request changes: if you are sure that the actual change will break something or has obvious issues * A detailed summary of what changed
* The motivation for the change
### 2. For new features or enhancements- ask yourself if we want this change in the product?
* We value everyone putting time and effort into PyTorch Lightning, but not every change should be included.
* We encourage everyone to open issues describing what they want to do before going off and doing it.
* Every feature we add takes attention and maintenance, but most importantly- might break something else.
* Discuss the tradeoffs in the PR or associated issue.
### 3. Make sure the PR's scope makes sense.
* Refactors should always be in separate PRs than changes.
* Every PR should change/fix **exactly one thing**. Otherwise, ask the author to split to smaller PRs.
### 4. Verify **the fix solves the issue properly.**
* It is usually good practice to clone the branch and try it out yourself.
### 5. Verify the code is **correct and readable**.
* Code is read way more often than it is written or modified, so readability is always important. You should aim to understand every changed line. If you don't understand it, it's likely someone else won't either. Ask questions and comments for things you don't understand.
* Make sure error scenarios and edge cases are well handled.
- Would this work for single node? multi node?
- CPU? GPU? TPU?
- Anything with spawn needs to be serializable. Spawn is basically like: a car is driving down a single lane- this is the main process. When spawn is called, that car gets copied into 8 other cars (8 gpus). That car then goes into its own lane. The car is trained on that lane, but the original car isn’t... so consider what happens in those scenarios.
- DDP is like the same analogy but instead of copying the car 8 times, we copy it 7 times and the main car then joins those 7 to train like all.
- Make sure this works with 16 bit precision.
* If you don’t think a variable name accurately represents what’s in it, suggest a better one.
* When making suggestions, mention that explicitly.
* When something **has** to change in the code, explain **what** should be changed and more importantly, **why** it should be changed (it is always useful to include links to external documentation/articles that provide more detailed information on the subject).
### 6. Verify the changes are **covered by tests**.
* Are there sufficient tests to ensure this change doesn’t break in the future?
* Read the tests and make sure they will break if the code does. The code in the tests is just as important as the code in the product.
* No tests? Ask the author to add them or add them yourself.
### 7. Check if the change is mentioned in **CHANGELOG** (no need for typos, docs and CI).
## Approving PRs
If you checked all the above and everything is looking good, you may approve the PR in your review or request changes.
As a rule of thumb:
- Approve: if you are confident that the issue or feature is fully addressed and the content can be merged as it is now (no extra conditions in comments).
- Comment: if there are any doubts (in your understanding, in the general direction, etc.) but you want to express your opinion about a specific point.
- Request changes: if you are sure that the actual change will break something or has obvious issues.
Remember that if you approve a PR and it reaches the minimum number of required approvals and given that all tests pass, the branch will be automatically merged into master. Remember that if you approve a PR and it reaches the minimum number of required approvals and given that all tests pass, the branch will be automatically merged into master.
## Hints and remarks
- **be nice and constructive**, remember your fist PR and what helped you / made you motivated to continue contributing ## Reviewers are friends!
- **use suggestions** for simple edits when you know what it shall be instead of leaving a long description (you can still add a short justification of your proposal) so the author can simply accept it.
- **Be nice and constructive**, remember your fist PR and what helped you / made you motivated to continue contributing.
- **Use suggestions** for simple edits when you know what it should be instead of leaving a long description (you can still add a short justification of your proposal) so the author can simply accept it.