From patchwork Sun May 16 11:09:01 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Seongyong Park X-Patchwork-Id: 439992 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI, SPF_HELO_NONE, SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 20567C433ED for ; Sun, 16 May 2021 11:09:17 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id DB50D61166 for ; Sun, 16 May 2021 11:09:16 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230185AbhEPLK3 (ORCPT ); Sun, 16 May 2021 07:10:29 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:45090 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230062AbhEPLK2 (ORCPT ); Sun, 16 May 2021 07:10:28 -0400 Received: from mail-pl1-x636.google.com (mail-pl1-x636.google.com [IPv6:2607:f8b0:4864:20::636]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 84D4EC061573 for ; Sun, 16 May 2021 04:09:13 -0700 (PDT) Received: by mail-pl1-x636.google.com with SMTP id h7so1684597plt.1 for ; Sun, 16 May 2021 04:09:13 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:mime-version :content-transfer-encoding; bh=OSm6bfHH1ocv8+S+rwU7DxDHYh+fh9kq1bRy/0vMyG4=; b=RWKnnuAkFJ/qZx1bhozPOYQ1R1zAhkmKcbMXXkzYYp8BslbZ6baIgzt1rqAgINhdNY Q4a1I0otd4C4ipMrPp7sSP3T7BulNb3o9cMGZPIgArld9Xn1pl0b13pfPMR/dhrIEQPL +R3SmkYw13+3LC+PQo4fe/Ot4h3vyaOz6E6ywS+UQgialoU9DayI4PcB8X8l9TOKbPbH qCROFwmqvLrhLBrD9Fs7llfLv3ra7pX9Rbgn/FT5hEywjdH1KBFJAUZPEDIO8YjxU4ls 61W8ecIDhLgwkE8T8qcdVc+49UgkF8yPV0A5+hVtvzb9j4Ho+69CRRiP4KvRvJFpNm99 KveQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:mime-version :content-transfer-encoding; bh=OSm6bfHH1ocv8+S+rwU7DxDHYh+fh9kq1bRy/0vMyG4=; b=hS4Q7Szhf2isg4I2EKKlXbZVfAiR1WkwUpz7ISJhsCToim/624SHImS67mlCvBC1/1 Iu+tW+LM5Vau8goI3D+rfILRdamhGJvJkjO72F2aJN1RHtgQoDFRwcyNaa+gny0OH5jm buILuSTJ5j/xOYzvG5+Q/AlMQJyzKU4v5WmR/zPGBIsTkY1KyvBp6RaTw3NvFdAbfGwj U7DpvNipcYtcPsTPPOG1D+0xsV5gVN9toQUqnP8mdyi3ymQllpjrOaS2TSmzZ6YxGEmw khzVRKcbcinwJJBgtsVesuiLkWP2qglPfz93eVaXCciwtQSauAvdeJJpLdXUls/8ffZn 5lig== X-Gm-Message-State: AOAM530frymsY8CYvNQwWrJcPaNFs4nIwe8UVYfHC5rfokzyn43/91PV PgF+HumRp3Zo48p6eCVX7gK2FTAVcyJ68Q== X-Google-Smtp-Source: ABdhPJxNt0FFHfDTYWfPSwWzVdbdTMFPcBWPTb76Do6r36r8yUkCCZmk0DAl5dnUbyzzJ+ZoWJXtsQ== X-Received: by 2002:a17:90a:2:: with SMTP id 2mr6033453pja.107.1621163352382; Sun, 16 May 2021 04:09:12 -0700 (PDT) Received: from odkf.hopto.org.skbroadband ([211.58.213.153]) by smtp.gmail.com with ESMTPSA id v22sm7488842pff.105.2021.05.16.04.09.10 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 16 May 2021 04:09:11 -0700 (PDT) From: Seongyong Park To: linux-media@vger.kernel.org Cc: Matt Ranostay , Mauro Carvalho Chehab , Seongyong Park Subject: [PATCH 1/2] media: video-i2c: frame delay based on last frame's end time Date: Sun, 16 May 2021 20:09:01 +0900 Message-Id: <20210516110902.784-1-euphoriccatface@gmail.com> X-Mailer: git-send-email 2.31.1 MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-media@vger.kernel.org Current implementation calculates frame delay based on the time of start of the loop. This inevitably causes the loop delay to be slightly longer than the actual measurement period, thus skipping a frame every now and then. However, MLX90640 should ideally be working without a frame skip. Each measurement step updates half of the pixels in the frame (every other pixel in default "chess mode", and every other row in "interleave mode"), while additional coefficient data (25th & 26th row) updates every step. The compensational coefficient data only corresponds with the pixels updated in the same step. In short, if a frame is skipped, then half of a frame loses correction information and becomes garbage data. Signed-off-by: Seongyong Park --- drivers/media/i2c/video-i2c.c | 11 ++++++----- 1 file changed, 6 insertions(+), 5 deletions(-) -- 2.31.1 diff --git a/drivers/media/i2c/video-i2c.c b/drivers/media/i2c/video-i2c.c index 0465832a4..2ccb08335 100644 --- a/drivers/media/i2c/video-i2c.c +++ b/drivers/media/i2c/video-i2c.c @@ -445,14 +445,16 @@ static int video_i2c_thread_vid_cap(void *priv) struct video_i2c_data *data = priv; unsigned int delay = mult_frac(HZ, data->frame_interval.numerator, data->frame_interval.denominator); + unsigned long end_jiffies = jiffies; set_freezable(); do { - unsigned long start_jiffies = jiffies; struct video_i2c_buffer *vid_cap_buf = NULL; int schedule_delay; + end_jiffies += delay; + try_to_freeze(); spin_lock(&data->slock); @@ -477,10 +479,9 @@ static int video_i2c_thread_vid_cap(void *priv) VB2_BUF_STATE_ERROR : VB2_BUF_STATE_DONE); } - schedule_delay = delay - (jiffies - start_jiffies); - - if (time_after(jiffies, start_jiffies + delay)) - schedule_delay = delay; + schedule_delay = end_jiffies - jiffies; + while (schedule_delay <= 0) + schedule_delay += delay; schedule_timeout_interruptible(schedule_delay); } while (!kthread_should_stop()); From patchwork Wed May 19 03:45:08 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Seongyong Park X-Patchwork-Id: 442870 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI, SPF_HELO_NONE, SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id E2C8AC433ED for ; Wed, 19 May 2021 03:50:16 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id B29F96135C for ; Wed, 19 May 2021 03:50:16 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1353062AbhESDve (ORCPT ); Tue, 18 May 2021 23:51:34 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:37416 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1353035AbhESDve (ORCPT ); Tue, 18 May 2021 23:51:34 -0400 Received: from mail-pj1-x1029.google.com (mail-pj1-x1029.google.com [IPv6:2607:f8b0:4864:20::1029]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id BCAF5C06175F for ; Tue, 18 May 2021 20:50:14 -0700 (PDT) Received: by mail-pj1-x1029.google.com with SMTP id o17-20020a17090a9f91b029015cef5b3c50so2743147pjp.4 for ; Tue, 18 May 2021 20:50:14 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=IF9z/4CR4I9nODmhixle8IEYz+JA1I8SG+9FbLFfO/8=; b=mOJmigFjM68s+YZeY9hUbJh7NLZ/iS5i0LsfdpTKEZdQZR2D2Z19il570KhKV+r+43 u0BKRfUfXQguxfdnfJO3fh27J5mJmzJaLNUoPIZgc6KIMeF+/jCVEIbx7E4m8Qfl/ZNH dpgc+3tehog7G5QuP3fem6QzQfN25JPtVC0k6dQCTun03mHjSknYyqZzhC1q4Rrtichk pkNPB6d88cpIs54mAO5/ZbPPh06kIYmK2OoxJ7TF6VoKrcBcm8YKVANTBXnpG1LSXBZA oujTFWLaNrs/vgkiPcCvyeOk/PbxwzRg7fQDM1Fq1WWOoUJ+rE712KAoICMlXHx0orh/ mqsA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=IF9z/4CR4I9nODmhixle8IEYz+JA1I8SG+9FbLFfO/8=; b=aoLNsQFW7xRVKQjF6XErYkIfzyfVtKqidvLERSDUWOqJo6d5hlX+A/8z2V8fIiSJtS PQZO2PHrnobyT/Lm+ZN9gOEs8Wjj+eCkDcZno2IFqrQQ3l1Ao8GQk9ZSEBL7jyx3BUkM olvYpfAzt0vOgt2dJZ+PSDIYVeAmGTk8tBp1+ftdeQINRZIyAfzdDNchw3an61GB9wWl pML73zrqO6jKDghfNUmxR0rLR3t/1wC183OjnmYwFbevTtzXEg2tabK7XVeh36vkJLa4 mn9zqdcd8oL8PdShUV9JVEXQwo0M2Q6JPPgKs/gBuhMencCQ9Kkg8UtiGnhaAQN5BzrI aWFQ== X-Gm-Message-State: AOAM530Qe+xcpXC8YSTBoApi8eK89HgaPVomXDakYl7DG/lBej6ABveQ 91ErWXbCXgQFKYkNOZTwPy89MVPOw6zXhQ== X-Google-Smtp-Source: ABdhPJweBaVCNuEmaGJuJkpPNoaiXSjWnlfPWyge68Mr64fMjT12xK1cFDcoNel3ATbHncId4FJpxg== X-Received: by 2002:a17:902:b7c3:b029:ef:8d29:a7d1 with SMTP id v3-20020a170902b7c3b02900ef8d29a7d1mr8434918plz.55.1621396213903; Tue, 18 May 2021 20:50:13 -0700 (PDT) Received: from odkf.hopto.org ([211.58.213.153]) by smtp.gmail.com with ESMTPSA id o4sm14070038pjf.9.2021.05.18.20.50.11 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 18 May 2021 20:50:13 -0700 (PDT) From: Seongyong Park To: linux-media@vger.kernel.org Cc: Matt Ranostay , Mauro Carvalho Chehab , Seongyong Park Subject: [PATCH V2 2/2] media: video-i2c: append register data on MLX90640's frame Date: Wed, 19 May 2021 12:45:08 +0900 Message-Id: <20210519034507.9576-1-euphoriccatface@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210516110902.784-1-euphoriccatface@gmail.com> References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-media@vger.kernel.org On MLX90640, Each measurement step updates half of the pixels in the frame (every other pixel in default "chess mode", and every other row in "interleave mode"), while additional coefficient data (25th & 26th row) updates every step. The compensational coefficient data only corresponds with the pixels updated in the same step. Only way to know which "subpage" was updated on the last step is to read "status register" on address 0x8000. Without this data, compensation calculation may be able to detect which sets of pixels have been updated, but it will have to make assumptions when frame skip happens, and there is no way to do it correctly when the host simply cannot keep up with refresh rate. Signed-off-by: Seongyong Park --- drivers/media/i2c/video-i2c.c | 13 +++++++++---- 1 file changed, 9 insertions(+), 4 deletions(-) diff --git a/drivers/media/i2c/video-i2c.c b/drivers/media/i2c/video-i2c.c index 2ccb08335..f2313b446 100644 --- a/drivers/media/i2c/video-i2c.c +++ b/drivers/media/i2c/video-i2c.c @@ -74,7 +74,8 @@ static const struct v4l2_fmtdesc mlx90640_format = { static const struct v4l2_frmsize_discrete mlx90640_size = { .width = 32, - .height = 26, /* 24 lines of pixel data + 2 lines of processing data */ + .height = 27, + /* 24 lines of pixel data + 2 lines of processing data + 1 line of registers */ }; static const struct regmap_config amg88xx_regmap_config = { @@ -168,8 +169,12 @@ static int amg88xx_xfer(struct video_i2c_data *data, char *buf) static int mlx90640_xfer(struct video_i2c_data *data, char *buf) { - return regmap_bulk_read(data->regmap, 0x400, buf, - data->chip->buffer_size); + int ret = regmap_bulk_read(data->regmap, 0x400, buf, + data->chip->buffer_size - 64); + if (ret) + return ret; + return regmap_bulk_read(data->regmap, 0x8000, buf + (data->chip->buffer_size - 64), + 64); } static int amg88xx_setup(struct video_i2c_data *data) @@ -375,7 +380,7 @@ static const struct video_i2c_chip video_i2c_chip[] = { .format = &mlx90640_format, .frame_intervals = mlx90640_frame_intervals, .num_frame_intervals = ARRAY_SIZE(mlx90640_frame_intervals), - .buffer_size = 1664, + .buffer_size = 1728, .bpp = 16, .regmap_config = &mlx90640_regmap_config, .nvmem_config = &mlx90640_nvram_config,