From patchwork Mon Nov 18 16:46:44 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ross Burton X-Patchwork-Id: 179594 Delivered-To: patch@linaro.org Received: by 2002:a92:38d5:0:0:0:0:0 with SMTP id g82csp3127057ilf; Mon, 18 Nov 2019 08:47:08 -0800 (PST) X-Google-Smtp-Source: APXvYqzVDE4s58B8H5uCzEjhYEqeA6gOSvxiRUhadiX9FJDWHhOScwWljX8fUE4BP3LDymXlxDfh X-Received: by 2002:a63:a449:: with SMTP id c9mr235925pgp.53.1574095628805; Mon, 18 Nov 2019 08:47:08 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1574095628; cv=none; d=google.com; s=arc-20160816; b=HYZ0KoyxNInpoZx3c6LOke0t2poLz9vxshB2mhH0oHwaMEk+IwTrlReNJuzCaO1NRc VRf7JTozIoUZO4lAOznKhlAGMHKdDX8zm1omlL7ieAEPo4/Hp6Lok5TdN6KH4Z+buS0s xrAYK3XmosjppiylsCL2bx31NroyTsXtC21j5UB2FRuhKGytSWXiCNl4RfsitCjgZf5o mcGb0OdM7RdiMxtfG9d7xahxgapqeIKwVVRpf8n8GNyVJbhaK/DpOAhYm/LJ38kEsCeU 2J4SHUKKvowpTslown18sX2IjxbN17x7T6oIVBmF+LXRAe3GsfoAMRgz9voTpu5MMDmx WPzA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=errors-to:sender:content-transfer-encoding:list-subscribe:list-help :list-post:list-archive:list-unsubscribe:list-id:precedence:subject :mime-version:references:in-reply-to:message-id:date:to:from :dkim-signature:delivered-to; bh=9qeccblzfXfJ9AYYLWYOdZVdu57gF/oN9TKnrguKsFw=; b=wLnWHn4EPLjnZtATJBOAb8QIMURZZBqO4YGksMu6Rm/8538VYJ3LUf3coogSo2Xccm 4qmhja+OKS5uwr324jRoVOyHE4ckkcsCgSiVXQTttj52idQqpjvlrtMt1mhet+fNKqt4 XRa3VKHkY7FAvUe1su+Bkcb4Csu2z0gR8KPM+tLZouwFTxot8lhh7AhTYyWvmxd97E+E eWloXRGMQccHTaxO9Guf8FuaTrPqoJW3P+D7Yyg3gorxCiPdn8QADmNZQ5Nm9w5XaFIu i1OMqET53jjo+RlCFf61ai/9Jd11YMr8PYSwBQ/+8MBMulV5N9Y5ZHYkCZFav0DDv390 Y7pg== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@intel-com.20150623.gappssmtp.com header.s=20150623 header.b="a9A7/Qoc"; spf=pass (google.com: best guess record for domain of openembedded-core-bounces@lists.openembedded.org designates 140.211.169.62 as permitted sender) smtp.mailfrom=openembedded-core-bounces@lists.openembedded.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=intel.com Return-Path: Received: from mail.openembedded.org (mail.openembedded.org. [140.211.169.62]) by mx.google.com with ESMTP id 67si20680423ple.265.2019.11.18.08.47.08; Mon, 18 Nov 2019 08:47:08 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of openembedded-core-bounces@lists.openembedded.org designates 140.211.169.62 as permitted sender) client-ip=140.211.169.62; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@intel-com.20150623.gappssmtp.com header.s=20150623 header.b="a9A7/Qoc"; spf=pass (google.com: best guess record for domain of openembedded-core-bounces@lists.openembedded.org designates 140.211.169.62 as permitted sender) smtp.mailfrom=openembedded-core-bounces@lists.openembedded.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from ec2-34-214-78-129.us-west-2.compute.amazonaws.com (localhost [127.0.0.1]) by mail.openembedded.org (Postfix) with ESMTP id 8C1487F947; Mon, 18 Nov 2019 16:46:58 +0000 (UTC) X-Original-To: openembedded-core@lists.openembedded.org Delivered-To: openembedded-core@lists.openembedded.org Received: from mail-wr1-f53.google.com (mail-wr1-f53.google.com [209.85.221.53]) by mail.openembedded.org (Postfix) with ESMTP id D035E7F8C6 for ; Mon, 18 Nov 2019 16:46:54 +0000 (UTC) Received: by mail-wr1-f53.google.com with SMTP id q15so7538138wrw.7 for ; Mon, 18 Nov 2019 08:46:56 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=intel-com.20150623.gappssmtp.com; s=20150623; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=GJwE60olmB34OdDBRJlGoCyRwjwAxCOyFJr0nPQzq5w=; b=a9A7/QocVtg879NJKdPegUY0gnzkRYFWwRie6EYfN2NAMwPZlz/tjeVp7AEF4ahZsO aeOxF8h0AB60ili0SopVqVCkgO/dnh4Y8RNkrG3BjGEcQ/FYE9hVw6PhlkFPDGuTkpkN dsEuGmAp4qfpIkLB3rNg+L1j1p7gDYlGTP0BfSzx9Je5K0S52TqjJU8M7MWp/F/LZ/Wq pleXYav2mVvBjQUvZvMB1V6cCH0TITj/u8FLD49a2ZfpaU6kH35Yr0WsgaxcXSTLx1Fa uMNl4BSGHDaA1rC2BomBmuOIARXLqQ3d5XMV7vTwO/H6J3hAylOJlNFYB+9eQGQWz28v UK4A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=GJwE60olmB34OdDBRJlGoCyRwjwAxCOyFJr0nPQzq5w=; b=kqajzjLsmhsZXQwX0r1e+qxKn3NHgp45Cy+itd3G2G/2REakGaz/aKdME4jCZmCu89 thUWx2UnpNhmoWZA43vIRqWwuhIQsFKD+kl4kO6uaGfJeUi+kQfdLS6hKopbqat0/n75 poGtEI770nLmSoDq7l38GdlwLevkOpP6jtvYuLh+b4PydYawqf39bIqdF6Jx6VjieHa0 KutWL1ZD36Q/pgH8sCXp3E2pgD4DswUQBmMQUBviaQhiRvo8pFY+ShIE5k/nCyhvvMdu Id7FZE+pzcs8LNYjSGO8MTXjubO9EQY6PSWkWF8p8wu1U+oVh3YKPoWroisevDjXB1/A A+gw== X-Gm-Message-State: APjAAAVxAykgZyyxvdeSVo5Xgt7l+WeAzmpkLdSR3e5uLCAB6okKEILN r5nKHkr2IYvC7TMjFEjfRemtM0lMPno= X-Received: by 2002:adf:cd0a:: with SMTP id w10mr17032533wrm.4.1574095615155; Mon, 18 Nov 2019 08:46:55 -0800 (PST) Received: from flashheart.burtonini.com (35.106.2.81.in-addr.arpa. [81.2.106.35]) by smtp.gmail.com with ESMTPSA id y189sm20936208wmb.13.2019.11.18.08.46.54 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 18 Nov 2019 08:46:54 -0800 (PST) From: Ross Burton To: openembedded-core@lists.openembedded.org Date: Mon, 18 Nov 2019 16:46:44 +0000 Message-Id: <20191118164647.29409-3-ross.burton@intel.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20191118164647.29409-1-ross.burton@intel.com> References: <20191118164647.29409-1-ross.burton@intel.com> MIME-Version: 1.0 Subject: [OE-core] [PATCH 3/6] cve-update-db-native: clean up proxy handling X-BeenThere: openembedded-core@lists.openembedded.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Patches and discussions about the oe-core layer List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: openembedded-core-bounces@lists.openembedded.org Errors-To: openembedded-core-bounces@lists.openembedded.org urllib handles adding proxy handlers if the proxies are set in the environment, so call bb.utils.export_proxies() to do that and remove the manual setup. Signed-off-by: Ross Burton --- .../recipes-core/meta/cve-update-db-native.bb | 31 +++---------------- 1 file changed, 5 insertions(+), 26 deletions(-) -- 2.20.1 -- _______________________________________________ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core diff --git a/meta/recipes-core/meta/cve-update-db-native.bb b/meta/recipes-core/meta/cve-update-db-native.bb index 08b18f064f0..db1d69a28e5 100644 --- a/meta/recipes-core/meta/cve-update-db-native.bb +++ b/meta/recipes-core/meta/cve-update-db-native.bb @@ -21,10 +21,12 @@ python do_populate_cve_db() { """ Update NVD database with json data feed """ - + import bb.utils import sqlite3, urllib, urllib.parse, shutil, gzip from datetime import date + bb.utils.export_proxies(d) + BASE_URL = "https://nvd.nist.gov/feeds/json/cve/1.0/nvdcve-1.0-" YEAR_START = 2002 @@ -40,16 +42,6 @@ python do_populate_cve_db() { except OSError: pass - proxy = d.getVar("https_proxy") - if proxy: - # instantiate an opener but do not install it as the global - # opener unless if we're really sure it's applicable for all - # urllib requests - proxy_handler = urllib.request.ProxyHandler({'https': proxy}) - proxy_opener = urllib.request.build_opener(proxy_handler) - else: - proxy_opener = None - cve_f = open(os.path.join(d.getVar("TMPDIR"), 'cve_check'), 'a') if not os.path.isdir(db_dir): @@ -67,15 +59,7 @@ python do_populate_cve_db() { json_url = year_url + ".json.gz" # Retrieve meta last modified date - - response = None - - if proxy_opener: - response = proxy_opener.open(meta_url) - else: - req = urllib.request.Request(meta_url) - response = urllib.request.urlopen(req) - + response = urllib.request.urlopen(meta_url) if response: for l in response.read().decode("utf-8").splitlines(): key, value = l.split(":", 1) @@ -95,12 +79,7 @@ python do_populate_cve_db() { # Update db with current year json file try: - if proxy_opener: - response = proxy_opener.open(json_url) - else: - req = urllib.request.Request(json_url) - response = urllib.request.urlopen(req) - + response = urllib.request.urlopen(json_url) if response: update_db(c, gzip.decompress(response.read()).decode('utf-8')) c.execute("insert or replace into META values (?, ?)", [year, last_modified])