From patchwork Mon Nov 11 22:13:37 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ross Burton X-Patchwork-Id: 179158 Delivered-To: patch@linaro.org Received: by 2002:a92:38d5:0:0:0:0:0 with SMTP id g82csp7264116ilf; Mon, 11 Nov 2019 14:14:47 -0800 (PST) X-Google-Smtp-Source: APXvYqxCyd/Qeng5ffc4NuPYR4cKEVtsgs316be8Ix4/bmH4FQyiSCukYSkoUqFckDZz5lCeTGRP X-Received: by 2002:a17:90a:6d64:: with SMTP id z91mr1846383pjj.44.1573510486907; Mon, 11 Nov 2019 14:14:46 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1573510486; cv=none; d=google.com; s=arc-20160816; b=gDkclW/hMp5Tbw6+seE4qUJ/dALc+BRUjqJCXjvZOSo6mb765WQ+osiM0kw3cpMu4e 6nN3FtqciPZmgpkVFj32I/XhF8wBVxczHSnCx7fKt143FXenhaW66nWhNDPnr1G+kmGL csMmDD2nB5dmFdZPsXpVzSh2BtUMwjgyFzaAH61OwkE2EoPqIzHheTE5hVBKAAYbJvuv xfSFwHqd7NuylLDOxk8R8xByYtGOrhsRlywO3G0gocWVSkdi7us3YclpypyUZyGZ8te+ 7804KJpLEy+F5DrxKoQ8ydyBHD2DNXiZ7ooCc37Ut3W0FhPREqa9c7KbGm1peExoMFxX YFnQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=errors-to:sender:content-transfer-encoding:list-subscribe:list-help :list-post:list-archive:list-unsubscribe:list-id:precedence:subject :mime-version:references:in-reply-to:message-id:date:to:from :dkim-signature:delivered-to; bh=LdhccsxCkKbecsaSOnQysQtRD/19yY0fbmV8FGp/Qvo=; b=aRZ93zK9iyIhdv6upe9aeNG1if8ywVacHpiC2r4aeS49DitSx2vYeHj9yLFvmRWElu o/POol98w0VCRs9woEO0rqL9yFf8xIrnmZSUyJIyld09CF1QSe9h5WEsXpB6ehBAVwa1 cZMmxtmV40tsZdNK8hyg2GFwhp3HU1wcm3mqR6ZNGiGtNWIFZ0VUwEyYIMoOPzjneDUD atrJ0PcjG/JDMVhyXWJuBt1mSiybvsyOfFG9cS00qGF5n8bf2FTBjzI8Odfgdh/OCNbV W9F/ul8rCEu4wPMsvO6WaWNK+K5kDaUSbxKl2nYHviJydQVFER4qGidEjP64fr3Pesmt n+Bw== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@intel-com.20150623.gappssmtp.com header.s=20150623 header.b=a+1ZiBvs; spf=pass (google.com: best guess record for domain of openembedded-core-bounces@lists.openembedded.org designates 140.211.169.62 as permitted sender) smtp.mailfrom=openembedded-core-bounces@lists.openembedded.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=intel.com Return-Path: Received: from mail.openembedded.org (mail.openembedded.org. [140.211.169.62]) by mx.google.com with ESMTP id f21si17836292plj.56.2019.11.11.14.14.46; Mon, 11 Nov 2019 14:14:46 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of openembedded-core-bounces@lists.openembedded.org designates 140.211.169.62 as permitted sender) client-ip=140.211.169.62; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@intel-com.20150623.gappssmtp.com header.s=20150623 header.b=a+1ZiBvs; spf=pass (google.com: best guess record for domain of openembedded-core-bounces@lists.openembedded.org designates 140.211.169.62 as permitted sender) smtp.mailfrom=openembedded-core-bounces@lists.openembedded.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from ec2-34-214-78-129.us-west-2.compute.amazonaws.com (localhost [127.0.0.1]) by mail.openembedded.org (Postfix) with ESMTP id CF4037FBEB; Mon, 11 Nov 2019 22:14:33 +0000 (UTC) X-Original-To: openembedded-core@lists.openembedded.org Delivered-To: openembedded-core@lists.openembedded.org Received: from mail-wr1-f65.google.com (mail-wr1-f65.google.com [209.85.221.65]) by mail.openembedded.org (Postfix) with ESMTP id 61FB67FAAA for ; Mon, 11 Nov 2019 22:13:43 +0000 (UTC) Received: by mail-wr1-f65.google.com with SMTP id l7so4774828wrp.6 for ; Mon, 11 Nov 2019 14:13:44 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=intel-com.20150623.gappssmtp.com; s=20150623; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=3GW7ecThMk1nx7wVwGkBEYGD9LfM1p2nWPbbrKeaqx0=; b=a+1ZiBvsCk5qO3aUQ/g1SXafP07uU/9V/RbWlCMTB12xcznkBPZoR29lzXXJzYtB7z 9xdAhNEJv9BQ+dGtyAcdBX5hDeDZl/AgSqlZBlR3n3SWKgwWt4RUcWSckAUKSYBXVamH KIqC22SWQtvuOlbG0DsYoqVt4ukmWSDUbDqa/Z6cXtLVU2q8CLrLeR6zhdDi3eNQNbiT UxGrlKfmgZBhx/Oej9ViJ7SVF0fFBmqdUaG3EKYSy74g4QY90D+VKx6Tiq03XJpQe2AI fFoI+NJRhyQ6yVScCXpT2P5DDgCjEGhCNqWqzTJXfbSFEjfMY+lEMd7GRrEyx36DYbI2 XfIQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=3GW7ecThMk1nx7wVwGkBEYGD9LfM1p2nWPbbrKeaqx0=; b=F/kqY/4I8H4Y3C+OLKvHlWIKPH7yOsF38rs0sThph42CbzAYfchIAF0awniCKVdcG2 S89ZgACNAalp43qkYekPT0v7AIEdrbEe7Mi8gbpD46juGlBc/rEuOml8AcDpqAy94k13 FaIAr3UxwxdH1C/8J5XvSqpEAbd8DJKrcVob8foMJVfkURHKu8VYTUS/waOEOdrIAPId tH+7wKQqEOWtHKZEmQjvOP6Whh7jqa0RqtvpgmdSCfkUAajLg/J7WH6+J9SOS2n+tLOh +wThZ74UjecFper9gj0yXo6zD7kNgghyVU3Fi1UryIbkrOaz97kwTW1hmjr0sri1XreI z//A== X-Gm-Message-State: APjAAAUoKFA3N2FxELJ3tQCOKq5atZAat+50NrCLnoazgr/L0eOrP8Xv LKCH/Ic2Of/bSJvLETO94Vp8bVEMQEs= X-Received: by 2002:a5d:5224:: with SMTP id i4mr3142288wra.303.1573510423684; Mon, 11 Nov 2019 14:13:43 -0800 (PST) Received: from flashheart.burtonini.com (35.106.2.81.in-addr.arpa. [81.2.106.35]) by smtp.gmail.com with ESMTPSA id d11sm24900083wrf.80.2019.11.11.14.13.42 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 11 Nov 2019 14:13:43 -0800 (PST) From: Ross Burton To: openembedded-core@lists.openembedded.org Date: Mon, 11 Nov 2019 22:13:37 +0000 Message-Id: <20191111221337.20332-4-ross.burton@intel.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20191111221337.20332-1-ross.burton@intel.com> References: <20191111221337.20332-1-ross.burton@intel.com> MIME-Version: 1.0 Subject: [OE-core] [PATCH 4/4] cve-check: rewrite look to fix false negatives X-BeenThere: openembedded-core@lists.openembedded.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Patches and discussions about the oe-core layer List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: openembedded-core-bounces@lists.openembedded.org Errors-To: openembedded-core-bounces@lists.openembedded.org A previous optimisation was premature and resulted in false-negatives in the report. Rewrite the checking algorithm to first get the list of potential CVEs by vendor:product, then iterate through every matching CPE for that CVE to determine if the bounds match or not. By doing this in two stages we can know if we've checked every CPE, instead of accidentally breaking out of the scan too early. Signed-off-by: Ross Burton --- meta/classes/cve-check.bbclass | 63 ++++++++++++++++++---------------- 1 file changed, 34 insertions(+), 29 deletions(-) -- 2.20.1 -- _______________________________________________ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core diff --git a/meta/classes/cve-check.bbclass b/meta/classes/cve-check.bbclass index 419b2c9148f..e95716d9ded 100644 --- a/meta/classes/cve-check.bbclass +++ b/meta/classes/cve-check.bbclass @@ -165,7 +165,6 @@ def check_cves(d, patched_cves): """ Connect to the NVD database and find unpatched cves. """ - import ast, csv, tempfile, subprocess, io from distutils.version import LooseVersion cves_unpatched = [] @@ -187,68 +186,74 @@ def check_cves(d, patched_cves): cve_whitelist = d.getVar("CVE_CHECK_WHITELIST").split() import sqlite3 - db_file = d.getVar("CVE_CHECK_DB_FILE") - conn = sqlite3.connect(db_file) + db_file = d.expand("file:${CVE_CHECK_DB_FILE}?mode=ro") + conn = sqlite3.connect(db_file, uri=True) + # For each of the known product names (e.g. curl has CPEs using curl and libcurl)... for product in products: - c = conn.cursor() if ":" in product: vendor, product = product.split(":", 1) - c.execute("SELECT * FROM PRODUCTS WHERE PRODUCT IS ? AND VENDOR IS ?", (product, vendor)) else: - c.execute("SELECT * FROM PRODUCTS WHERE PRODUCT IS ?", (product,)) + vendor = "%" - for row in c: - cve = row[0] - version_start = row[3] - operator_start = row[4] - version_end = row[5] - operator_end = row[6] + # Find all relevant CVE IDs. + for cverow in conn.execute("SELECT DISTINCT ID FROM PRODUCTS WHERE PRODUCT IS ? AND VENDOR LIKE ?", (product, vendor)): + cve = cverow[0] if cve in cve_whitelist: bb.note("%s-%s has been whitelisted for %s" % (product, pv, cve)) # TODO: this should be in the report as 'whitelisted' patched_cves.add(cve) + continue elif cve in patched_cves: bb.note("%s has been patched" % (cve)) - else: - to_append = False + continue + + vulnerable = False + for row in conn.execute("SELECT * FROM PRODUCTS WHERE ID IS ? AND PRODUCT IS ? AND VENDOR LIKE ?", (cve, product, vendor)): + (_, _, _, version_start, operator_start, version_end, operator_end) = row + #bb.debug(2, "Evaluating row " + str(row)) + if (operator_start == '=' and pv == version_start): - to_append = True + vulnerable = True else: if operator_start: try: - to_append_start = (operator_start == '>=' and LooseVersion(pv) >= LooseVersion(version_start)) - to_append_start |= (operator_start == '>' and LooseVersion(pv) > LooseVersion(version_start)) + vulnerable_start = (operator_start == '>=' and LooseVersion(pv) >= LooseVersion(version_start)) + vulnerable_start |= (operator_start == '>' and LooseVersion(pv) > LooseVersion(version_start)) except: bb.warn("%s: Failed to compare %s %s %s for %s" % (product, pv, operator_start, version_start, cve)) - to_append_start = False + vulnerable_start = False else: - to_append_start = False + vulnerable_start = False if operator_end: try: - to_append_end = (operator_end == '<=' and LooseVersion(pv) <= LooseVersion(version_end)) - to_append_end |= (operator_end == '<' and LooseVersion(pv) < LooseVersion(version_end)) + vulnerable_end = (operator_end == '<=' and LooseVersion(pv) <= LooseVersion(version_end)) + vulnerable_end |= (operator_end == '<' and LooseVersion(pv) < LooseVersion(version_end)) except: bb.warn("%s: Failed to compare %s %s %s for %s" % (product, pv, operator_end, version_end, cve)) - to_append_end = False + vulnerable_end = False else: - to_append_end = False + vulnerable_end = False if operator_start and operator_end: - to_append = to_append_start and to_append_end + vulnerable = vulnerable_start and vulnerable_end else: - to_append = to_append_start or to_append_end + vulnerable = vulnerable_start or vulnerable_end - if to_append: + if vulnerable: bb.note("%s-%s is vulnerable to %s" % (product, pv, cve)) cves_unpatched.append(cve) - else: - bb.note("%s-%s is not vulnerable to %s" % (product, pv, cve)) - patched_cves.add(cve) + break + + if not vulnerable: + bb.note("%s-%s is not vulnerable to %s" % (product, pv, cve)) + # TODO: not patched but not vulnerable + patched_cves.add(cve) + conn.close() return (list(patched_cves), cves_unpatched)