Bug 1341214 - Add a small API to handle taskcluster queue and index requests. r=dustin

Various modules under taskcluster are doing ad-hoc url formatting or
requests to taskcluster services. While we could use the taskcluster
client python module, it's kind of overkill for the simple requests done
here. So instead of vendoring that module, create a smaller one with
a limited set of functions we need.

This changes the behavior of the get_artifact function to return a
file-like object when the file is neither a json nor a yaml, but that
branch was never used (and was actually returning an unassigned
variable, so it was broken anyways).

At the same time, make the function that does HTTP requests more
error-resistant, using urllib3's Retry with a backoff factor.

Also add a function that retrieves the list of artifacts, that while
currently unused, will be used by `mach artifact` shortly.
This commit is contained in:
Mike Hommey
2017-02-17 12:04:48 +09:00
parent eb5916da95
commit c7945d8ea1
11 changed files with 110 additions and 70 deletions

View File

@@ -62,9 +62,7 @@ def load_parameters_file(options):
Load parameters from the --parameters option
"""
import urllib
url_prefix = "https://queue.taskcluster.net/v1/task/"
url_postfix = "/artifacts/public/parameters.yml"
from taskgraph.util.taskcluster import get_artifact_url
filename = options['parameters']
@@ -78,7 +76,7 @@ def load_parameters_file(options):
# fetching parameters.yml using task task-id or supplied url
if filename.startswith("task-id="):
task_id = filename.split("=")[1]
filename = url_prefix + task_id + url_postfix
filename = get_artifact_url(task_id, 'public/parameters.yml')
f = urllib.urlopen(filename)
if filename.endswith('.yml'):