Linting
As part of the build process we run a few linters to catch common programming errors, stylistic errors and possible security issues. Linters are run only when working with the package (directory) structure.
All linters are run via the demisto-sdk:
Note: this script is also used to run pytest. See: Unit Testing
An example of the result for running our lint checks on the HelloWorld package will look like:
Flake8#
This is a basic linter. It can be run without having all the dependencies available and will catch common errors. We also use this linter to enforce the standard python pep8 formatting style. On rare occasions you may encounter a need to disable an error/warning returned from this linter. Do this by adding an inline comment of the sort on the line you want to disable the error:
For example:
When adding an inline comment always also include the error code you are disabling for. That way if there are other errors on the same line they will be reported.
More info: https://flake8.pycqa.org/en/latest/user/violations.html#in-line-ignoring-errors
Pylint#
This linter is similar to flake8 but is able to catch some additional errors. We run this linter with error reporting only. It requires access to dependent modules and thus we run it within a docker image similar with all dependencies (similar to how we run pytest unit tests). On rare occasions you may encounter a need to disable an error/warning returned from this linter. Do this by adding an inline comment of the sort on the line you want to disable the error:
For example:
Is is also possible to disable and then enable a block of code. For example (taken from CommonServerPython.py):
Note: pylint can take both the error name and error code when doing inline comment disables. It is best to use the name which is clearer to understand.
More info: https://pylint.readthedocs.io/en/latest/user_guide/message-control.html
For classes that generate members dynamically (such as goolgeapi classes) pylint will generate multiple no-member errors as it won't be able to detect the members of the class. In this case it is best to add a .pylintrc file which will include the following:
See following example
Mypy#
Mypy uses type annotations to check code for common errors. It contains type information for many popular libraries (via typeshed project). Additionally, it allows you to define type annotations for your own functions and data structures. Type annotations are fully supported as a language feature in python 3.6 and above. In earlier versions type annotations are provided via the use of comments.
We run mypy in a relatively aggressive mode so it type checks also functions which don't contain type definitions. This may sometimes cause extra errors. If you receive errors you can always ignore the line with an inline comment of:
For example:
Note: mypy introduced the ignore[<error-name>] syntax only in version 0.730. See: error code docs. You may see in the code ignores of the form: type: ignore without the error-name. This would usually be from old code written before the support for error-name ignores. We do not recommend using this ignore style as it ignores all errors and increases the risk of ignoring unexpected serious errors.
Dealing with Need type annotation errors: If you receive such an error instead of simply adding an ignore comment it is better to define the type of the variable which is missing type annotation. This error is usually received when an empty dict or list is defined and mypy can not infer the type of the object. In this case it is better to define the type as dict or list. For example python 2 code:
Or with python 3 annotations
If you know the type that the list will hold use the type constructor List that can specify also what type it holds. For example a list which we know that will hold strings in python 2 code:
Or with python 3 annotations
Note: When using type constructors such as List or Dict there is need to import the type from the typing module in python 3. In python 2 as part of running mypy our wrapper script will include the typing module.
More info at: https://mypy.readthedocs.io/en/latest/index.html
Bandit#
Bandit is a tool designed to find common security issues in Python code.
We run bandit with a confidence level of HIGH. In the rare case that it reports a false positive, you can exclude the code by adding a comment of the sort: # nosec. See: https://github.com/PyCQA/bandit#exclusions .
XSOAR Linter#
This is a custom linter, based on pylint, whose main purpose is to catch errors regarding Cortex XSOAR code standards. The linter is activated using the pylint load plugins ability. We run this linter only with custom Cortex XSOAR error and warning messages (all other messages are disabled). On rare occasions, you may encounter a scenario in which you need to disable an error or warning message from being returned from by linter. To do this add an inline comment, as shown below, on the line you want to disable the error:
For example:
It is also possible to disable and then enable a block of code. The following example is taken from CommonServerPython.py:
Note: Pylint can take both the error name and error code when using an inline comment disable message. Please note, that it is best to use the name which is most clear to understand.