diff mbox series

[2/4] dt-bindings: touchscreen: add virtual-touchscreen and virtual-buttons properties

Message ID 20230510-feature-ts_virtobj_patch-v1-2-5ae5e81bc264@wolfvision.net
State New
Headers show
Series Input: support virtual objects on touchscreens | expand

Commit Message

Javier Carrasco May 10, 2023, 1:50 p.m. UTC
The virtual-touchscreen object defines an area within the touchscreen
where touch events are reported and their coordinates get converted to
the virtual origin. This object avoids getting events from areas that
are physically hidden by overlay frames.

For touchscreens where overlay buttons on the touchscreen surface are
provided, the virtual-buttons object contains a node for every button
and the key event that should be reported when pressed.

Signed-off-by: Javier Carrasco <javier.carrasco@wolfvision.net>
---
 .../bindings/input/touchscreen/touchscreen.yaml    | 54 ++++++++++++++++++++++
 1 file changed, 54 insertions(+)

Comments

Krzysztof Kozlowski May 11, 2023, 9:45 a.m. UTC | #1
On 10/05/2023 15:50, Javier Carrasco wrote:
> The virtual-touchscreen object defines an area within the touchscreen
> where touch events are reported and their coordinates get converted to
> the virtual origin. This object avoids getting events from areas that
> are physically hidden by overlay frames.
> 
> For touchscreens where overlay buttons on the touchscreen surface are
> provided, the virtual-buttons object contains a node for every button
> and the key event that should be reported when pressed.

Hm, I don't understand - are these separate physical buttons? If so, why
would they be part of touchscreen? Where do the wires go?

> 
> Signed-off-by: Javier Carrasco <javier.carrasco@wolfvision.net>
> ---
>  .../bindings/input/touchscreen/touchscreen.yaml    | 54 ++++++++++++++++++++++
>  1 file changed, 54 insertions(+)
> 

Best regards,
Krzysztof
Javier Carrasco May 11, 2023, 10:28 a.m. UTC | #2
On 11.05.23 11:45, Krzysztof Kozlowski wrote:
> On 10/05/2023 15:50, Javier Carrasco wrote:
>> The virtual-touchscreen object defines an area within the touchscreen
>> where touch events are reported and their coordinates get converted to
>> the virtual origin. This object avoids getting events from areas that
>> are physically hidden by overlay frames.
>>
>> For touchscreens where overlay buttons on the touchscreen surface are
>> provided, the virtual-buttons object contains a node for every button
>> and the key event that should be reported when pressed.
> 
> Hm, I don't understand - are these separate physical buttons? If so, why
> would they be part of touchscreen? Where do the wires go?
Those buttons are actually printed on some overlays which are mounted on
top of the touchscreen and  they are used to provide a predefined
functionality to that touchscreen region. Any touchscreen with such a
physical overlay with buttons printed on them or clipped touchscreen
areas might use this functionality.

These buttons are actually physical i.e. printed and with a given
functionality, but still part of the touchscreen as the physical device
is not aware of them. Therefore they only make sense in the touchscreen
context.
> 
>>
>> Signed-off-by: Javier Carrasco <javier.carrasco@wolfvision.net>
>> ---
>>  .../bindings/input/touchscreen/touchscreen.yaml    | 54 ++++++++++++++++++++++
>>  1 file changed, 54 insertions(+)
>>
> 
> Best regards,
> Krzysztof
>
Krzysztof Kozlowski May 12, 2023, 7:20 a.m. UTC | #3
On 12/05/2023 09:08, Michael Riesch wrote:
> Hi Krzysztof,
> 
>>> These buttons are actually physical i.e. printed and with a given
>>> functionality, but still part of the touchscreen as the physical device
>>> is not aware of them. Therefore they only make sense in the touchscreen
>>> context.
>>
>> So basically you still have the same touchscreen under the buttons and
>> these are not separate devices. Whether someone put a sticker on
>> touchscreen, does not really change hardware behavior and it's up to
>> system to handle this. What you are trying to do here is to create
>> virtual buttons through Devicetree and DT is not for that.
> 
> I have already addressed a similar statement here:
> https://lore.kernel.org/lkml/20230504042927.GA1129520@quokka/t/#m1a93595c36535312df0061459a1da5e062de6c44
> but let me extend this comment a bit.
> 
> The notion of "someone putting a sticker on touchscreen" does not really
> reflect the use case we have in mind. We are talking about devices that
> are shipped from the factory in a particular configuration, e.g.,
> 
> +-----------------------+---------+
> |                       |  power  |
> |                       |  button |
> |   touchscreen         +---------+
> |   (behind: display)   |  return |
> |                       |  button |
> +-----------------------+---------+
> 
> Here, the real touchscreen is larger than the display. The display is
> behind the "touchscreen" part. Behind the buttons there are symbols
> engraved in metal or printed foils or what not. I just would like to
> make it clear that these symbols are not going to change.
> 
> We believe that the engraved or printed symbols actually define the
> (expected) hardware behavior. Of course there is a virtual notion to
> that, and of course it would be possible to let the power button work as
> return button and vice versa in software. However, the user sees the
> engraved or printed symbols (which are not going to change) and expects
> the device to react appropriately.

OK, I actually missed the concept that display is not equal to the
touchscreen ("screen" actually suggests display :) ). In your case here
it sounds good, but please put some parts of this description into this
common binding. The sketch above is nice, especially if you can point
where the virtual origin x/y are. Picture is thousands words.

> 
> Now if you suggest that the system (that is user space, right?) should
> handle this, then the question would be which component is supposed to
> handle the touchscreen events and react accordingly. I don't have an
> answer to that and hope I don't need to find one. But independent of
> that, a configuration file is required that defines the area of the
> virtual buttons etc. Wouldn't this be similar to the (mostly) beloved
> xorg.conf containing the definitions of input devices?

If the case was a bit different - e.g. display is everywhere - I could
easily imagine that on the device rotation you want to move
(re-position) the buttons. Or if user has some accessibility option
enabled, you want bigger buttons. Then it would be a prove that it must
be configured and managed from user-space. How, I don't know.

Best regards,
Krzysztof
Michael Riesch May 12, 2023, 8:13 a.m. UTC | #4
Hi Krzysztof,

On 5/12/23 09:20, Krzysztof Kozlowski wrote:
> On 12/05/2023 09:08, Michael Riesch wrote:
>> Hi Krzysztof,
>>
>>>> These buttons are actually physical i.e. printed and with a given
>>>> functionality, but still part of the touchscreen as the physical device
>>>> is not aware of them. Therefore they only make sense in the touchscreen
>>>> context.
>>>
>>> So basically you still have the same touchscreen under the buttons and
>>> these are not separate devices. Whether someone put a sticker on
>>> touchscreen, does not really change hardware behavior and it's up to
>>> system to handle this. What you are trying to do here is to create
>>> virtual buttons through Devicetree and DT is not for that.
>>
>> I have already addressed a similar statement here:
>> https://lore.kernel.org/lkml/20230504042927.GA1129520@quokka/t/#m1a93595c36535312df0061459a1da5e062de6c44
>> but let me extend this comment a bit.
>>
>> The notion of "someone putting a sticker on touchscreen" does not really
>> reflect the use case we have in mind. We are talking about devices that
>> are shipped from the factory in a particular configuration, e.g.,
>>
>> +-----------------------+---------+
>> |                       |  power  |
>> |                       |  button |
>> |   touchscreen         +---------+
>> |   (behind: display)   |  return |
>> |                       |  button |
>> +-----------------------+---------+
>>
>> Here, the real touchscreen is larger than the display. The display is
>> behind the "touchscreen" part. Behind the buttons there are symbols
>> engraved in metal or printed foils or what not. I just would like to
>> make it clear that these symbols are not going to change.
>>
>> We believe that the engraved or printed symbols actually define the
>> (expected) hardware behavior. Of course there is a virtual notion to
>> that, and of course it would be possible to let the power button work as
>> return button and vice versa in software. However, the user sees the
>> engraved or printed symbols (which are not going to change) and expects
>> the device to react appropriately.
> 
> OK, I actually missed the concept that display is not equal to the
> touchscreen ("screen" actually suggests display :) ). In your case here
> it sounds good, but please put some parts of this description into this
> common binding. The sketch above is nice, especially if you can point
> where the virtual origin x/y are. Picture is thousands words.

I think this can be arranged :-)

>> Now if you suggest that the system (that is user space, right?) should
>> handle this, then the question would be which component is supposed to
>> handle the touchscreen events and react accordingly. I don't have an
>> answer to that and hope I don't need to find one. But independent of
>> that, a configuration file is required that defines the area of the
>> virtual buttons etc. Wouldn't this be similar to the (mostly) beloved
>> xorg.conf containing the definitions of input devices?
> 
> If the case was a bit different - e.g. display is everywhere - I could
> easily imagine that on the device rotation you want to move
> (re-position) the buttons. Or if user has some accessibility option
> enabled, you want bigger buttons. Then it would be a prove that it must
> be configured and managed from user-space. How, I don't know.

I fully agree here. If user space is able to draw the symbols, then
naturally it is also responsible for handling the events appropriately.
This could happen in an application with a GUI or on display
manager/compositor level (e.g., weston controller or similar).

But I take it that for the situation sketched above the device tree
approach is the correct one. Documentation should be improved, though.

Best regards,
Michael
diff mbox series

Patch

diff --git a/Documentation/devicetree/bindings/input/touchscreen/touchscreen.yaml b/Documentation/devicetree/bindings/input/touchscreen/touchscreen.yaml
index 895592da9626..869be007eb6f 100644
--- a/Documentation/devicetree/bindings/input/touchscreen/touchscreen.yaml
+++ b/Documentation/devicetree/bindings/input/touchscreen/touchscreen.yaml
@@ -80,6 +80,60 @@  properties:
   touchscreen-y-plate-ohms:
     description: Resistance of the Y-plate in Ohms
 
+  virtual-touchscreen:
+    description: Clipped touchscreen area
+    type: object
+
+    properties:
+      x-origin:
+        description: horizontal origin of the clipped area
+        $ref: /schemas/types.yaml#/definitions/uint32
+
+      y-origin:
+        description: vertical origin of the clipped area
+        $ref: /schemas/types.yaml#/definitions/uint32
+
+      x-size:
+        description: horizontal resolution of the clipped area
+        $ref: /schemas/types.yaml#/definitions/uint32
+
+      y-size:
+        description: vertical resolution of the clipped area
+        $ref: /schemas/types.yaml#/definitions/uint32
+
+  virtual-buttons:
+    description: list of nodes defining the buttons on the touchscreen
+    type: object
+
+    patternProperties:
+      '^button-':
+        type: object
+        description:
+          Each button (key) is represented as a sub-node.
+
+        properties:
+          label:
+            $ref: /schemas/types.yaml#/definitions/string
+            description: descriptive name of the button
+
+          linux,code: true
+
+          x-origin:
+            description: horizontal origin of the button area
+            $ref: /schemas/types.yaml#/definitions/uint32
+
+          y-origin:
+            description: vertical origin of the button area
+            $ref: /schemas/types.yaml#/definitions/uint32
+
+          x-size:
+            description: horizontal resolution of the button area
+            $ref: /schemas/types.yaml#/definitions/uint32
+
+          y-size:
+            description: vertical resolution of the button area
+            $ref: /schemas/types.yaml#/definitions/uint32
+
 dependencies:
   touchscreen-size-x: [ touchscreen-size-y ]
   touchscreen-size-y: [ touchscreen-size-x ]