5fe1f3f6e3
Before, we'd open files and feed bytes to yaml.load(). When a str is fed to yaml.load(), it attempts to guess the encoding. It defaults to UTF-8 unless somebody set us up the BOM. This is probably OK. Except if the file isn't valid UTF-8, the exception will be raised in the bowels of YAML parsing and it may not be obvious the failure is due to invalid UTF-8 input versus say Python str/unicode coercion foo. We change all call sites that load YAML from a file to use codecs.open() to open the file in UTF-8 and perform UTF-8 decoding/validation at file read time. This should make any UTF-8 failures more obvious. Furthermore, it reinforces that our YAML files are UTF-8 and not some other encoding. I discovered this issue as part of trying to get emoji symbols to render on Treeherder. Unfortunately, it appears pyyaml detects many emoji as unprintable characters and refuses to load them. This makes me sad and makes me want to abandon pyyaml/YAML in favor of something that supports emoji :P MozReview-Commit-ID: AOvAruZFfnK
38 lines
1.2 KiB
Python
38 lines
1.2 KiB
Python
# -*- coding: utf-8 -*-
|
|
|
|
# This Source Code Form is subject to the terms of the Mozilla Public
|
|
# License, v. 2.0. If a copy of the MPL was not distributed with this
|
|
# file, You can obtain one at http://mozilla.org/MPL/2.0/.
|
|
|
|
from __future__ import absolute_import, print_function, unicode_literals
|
|
|
|
import codecs
|
|
import json
|
|
import yaml
|
|
from mozbuild.util import ReadOnlyDict
|
|
|
|
|
|
class Parameters(ReadOnlyDict):
|
|
"""An immutable dictionary with nicer KeyError messages on failure"""
|
|
def __getitem__(self, k):
|
|
try:
|
|
return super(Parameters, self).__getitem__(k)
|
|
except KeyError:
|
|
raise KeyError("taskgraph parameter {!r} not found".format(k))
|
|
|
|
|
|
def load_parameters_file(options):
|
|
"""
|
|
Load parameters from the --parameters option
|
|
"""
|
|
filename = options['parameters']
|
|
if not filename:
|
|
return Parameters()
|
|
with codecs.open(filename, 'rb', 'utf-8') as f:
|
|
if filename.endswith('.yml'):
|
|
return Parameters(**yaml.safe_load(f))
|
|
elif filename.endswith('.json'):
|
|
return Parameters(**json.load(f))
|
|
else:
|
|
raise TypeError("Parameters file `{}` is not JSON or YAML".format(filename))
|