Bug 1391019 - Add py2 and py3 compatability linters, r=gps

check_compat.py was adapted from gps' check-py3-compat.py in mercurial:
https://www.mercurial-scm.org/repo/hg/file/tip/contrib/check-py3-compat.py

The py3 linter simply runs ast.parse(f) for each file being linted. Any syntax errors
are formatted as mozlint results and dumped to stdout as json. I looked into also
importing the file (using 3.5+'s importlib.util.spec_from_file_location), but there
were too many problems:

1. Lots of false positives (e.g module not found)
2. Some files seemed to run indefinitely on import

I decided to punt on importing for now, we can always investigate in a follow-up.

The py2 linter runs ast.parse(f), and also checks that the file has:
from __future__ import absolute_import, print_function

Initially every python file in the tree is excluded from the py2 check, though
at least this makes it easy to find+fix, and new files in un-excluded
directories will automatically be linted.

MozReview-Commit-ID: ABtq9dnPo9T

tools/lint/python/check_compat.py

@@ -0,0 +1,84 @@
+#!/usr/bin/env python
+# This Source Code Form is subject to the terms of the Mozilla Public
+# License, v. 2.0. If a copy of the MPL was not distributed with this
+# file, You can obtain one at http://mozilla.org/MPL/2.0/.
+
+from __future__ import absolute_import, print_function
+
+import ast
+import json
+import sys
+
+
+def parse_file(f):
+    with open(f, 'rb') as fh:
+        content = fh.read()
+    try:
+        return ast.parse(content)
+    except SyntaxError as e:
+        err = {
+            'path': f,
+            'message': e.msg,
+            'lineno': e.lineno,
+            'column': e.offset,
+            'source': e.text,
+            'rule': 'is-parseable',
+        }
+        print(json.dumps(err))
+
+
+def check_compat_py2(f):
+    """Check Python 2 and Python 3 compatibility for a file with Python 2"""
+    root = parse_file(f)
+
+    # Ignore empty or un-parseable files.
+    if not root or not root.body:
+        return
+
+    futures = set()
+    haveprint = False
+    future_lineno = 1
+    for node in ast.walk(root):
+        if isinstance(node, ast.ImportFrom):
+            if node.module == '__future__':
+                future_lineno = node.lineno
+                futures |= set(n.name for n in node.names)
+        elif isinstance(node, ast.Print):
+            haveprint = True
+
+    err = {
+        'path': f,
+        'lineno': future_lineno,
+        'column': 1,
+    }
+
+    if 'absolute_import' not in futures:
+        err['rule'] = 'require absolute_import'
+        err['message'] = 'Missing from __future__ import absolute_import'
+        print(json.dumps(err))
+
+    if haveprint and 'print_function' not in futures:
+        err['rule'] = 'require print_function'
+        err['message'] = 'Missing from __future__ import print_function'
+        print(json.dumps(err))
+
+
+def check_compat_py3(f):
+    """Check Python 3 compatibility of a file with Python 3."""
+    parse_file(f)
+
+
+if __name__ == '__main__':
+    if sys.version_info[0] == 2:
+        fn = check_compat_py2
+    else:
+        fn = check_compat_py3
+
+    manifest = sys.argv[1]
+    with open(manifest, 'r') as fh:
+        files = fh.read().splitlines()
+
+    for f in files:
+        fn(f)
+
+    sys.exit(0)