mbox series

[net-next,v1,0/3] vsock: Add flag field in the vsock address

Message ID 20201201152505.19445-1-andraprs@amazon.com
Headers show
Series vsock: Add flag field in the vsock address | expand

Message

Paraschiv, Andra-Irina Dec. 1, 2020, 3:25 p.m. UTC
vsock enables communication between virtual machines and the host they are
running on. Nested VMs can be setup to use vsock channels, as the multi
transport support has been available in the mainline since the v5.5 Linux kernel
has been released.

Implicitly, if no host->guest vsock transport is loaded, all the vsock packets
are forwarded to the host. This behavior can be used to setup communication
channels between sibling VMs that are running on the same host. One example can
be the vsock channels that can be established within AWS Nitro Enclaves
(see Documentation/virt/ne_overview.rst).

To be able to explicitly mark a connection as being used for a certain use case,
add a flag field in the vsock address data structure. The "svm_reserved1" field
has been repurposed to be the flag field. The value of the flag will then be
taken into consideration when the vsock transport is assigned.

This way can distinguish between nested VMs / local communication and sibling
VMs use cases. And can also setup one or more types of communication at the same
time.

Thank you.

Andra

---

Patch Series Changelog

The patch series is built on top of v5.10-rc6.

GitHub repo branch for the latest version of the patch series:

* https://github.com/andraprs/linux/tree/vsock-flag-sibling-comm-v1

---

Andra Paraschiv (3):
  vm_sockets: Include flag field in the vsock address data structure
  virtio_transport_common: Set sibling VMs flag on the receive path
  af_vsock: Assign the vsock transport considering the vsock address
    flag

 include/uapi/linux/vm_sockets.h         | 18 +++++++++++++++++-
 net/vmw_vsock/af_vsock.c                | 15 +++++++++++----
 net/vmw_vsock/virtio_transport_common.c |  8 ++++++++
 3 files changed, 36 insertions(+), 5 deletions(-)

Comments

Stefano Garzarella Dec. 1, 2020, 4:09 p.m. UTC | #1
On Tue, Dec 01, 2020 at 05:25:03PM +0200, Andra Paraschiv wrote:
>vsock enables communication between virtual machines and the host they
>are running on. With the multi transport support (guest->host and
>host->guest), nested VMs can also use vsock channels for communication.
>
>In addition to this, by default, all the vsock packets are forwarded to
>the host, if no host->guest transport is loaded. This behavior can be
>implicitly used for enabling vsock communication between sibling VMs.
>
>Add a flag field in the vsock address data structure that can be used to
>explicitly mark the vsock connection as being targeted for a certain
>type of communication. This way, can distinguish between nested VMs and
>sibling VMs use cases and can also setup them at the same time. Till
>now, could either have nested VMs or sibling VMs at a time using the
>vsock communication stack.
>
>Use the already available "svm_reserved1" field and mark it as a flag
>field instead. This flag can be set when initializing the vsock address
>variable used for the connect() call.

Maybe we can split this patch in 2 patches, one to rename the svm_flag 
and one to add the new flags.

>
>Signed-off-by: Andra Paraschiv <andraprs@amazon.com>
>---
> include/uapi/linux/vm_sockets.h | 18 +++++++++++++++++-
> 1 file changed, 17 insertions(+), 1 deletion(-)
>
>diff --git a/include/uapi/linux/vm_sockets.h b/include/uapi/linux/vm_sockets.h
>index fd0ed7221645d..58da5a91413ac 100644
>--- a/include/uapi/linux/vm_sockets.h
>+++ b/include/uapi/linux/vm_sockets.h
>@@ -114,6 +114,22 @@
>
> #define VMADDR_CID_HOST 2
>
>+/* This sockaddr_vm flag value covers the current default use case:
>+ * local vsock communication between guest and host and nested VMs setup.
>+ * In addition to this, implicitly, the vsock packets are forwarded to the host
>+ * if no host->guest vsock transport is set.
>+ */
>+#define VMADDR_FLAG_DEFAULT_COMMUNICATION	0x0000

I think we don't need this macro, since the next one can be used to 
check if it a sibling communication (flag 0x1 set) or not (flag 0x1 
not set).

>+
>+/* Set this flag value in the sockaddr_vm corresponding field if the vsock
>+ * channel needs to be setup between two sibling VMs running on the same host.
>+ * This way can explicitly distinguish between vsock channels created for nested
>+ * VMs (or local communication between guest and host) and the ones created for
>+ * sibling VMs. And vsock channels for multiple use cases (nested / sibling VMs)
>+ * can be setup at the same time.
>+ */
>+#define VMADDR_FLAG_SIBLING_VMS_COMMUNICATION	0x0001

What do you think if we shorten in VMADDR_FLAG_SIBLING?

Thanks,
Stefano

>+
> /* Invalid vSockets version. */
>
> #define VM_SOCKETS_INVALID_VERSION -1U
>@@ -145,7 +161,7 @@
>
> struct sockaddr_vm {
> 	__kernel_sa_family_t svm_family;
>-	unsigned short svm_reserved1;
>+	unsigned short svm_flag;
> 	unsigned int svm_port;
> 	unsigned int svm_cid;
> 	unsigned char svm_zero[sizeof(struct sockaddr) -
>-- 
>2.20.1 (Apple Git-117)
>
>
>
>
>Amazon Development Center (Romania) S.R.L. registered office: 27A Sf. Lazar Street, UBC5, floor 2, Iasi, Iasi County, 700045, Romania. Registered in Romania. Registration number J22/2621/2005.
>
Stefano Garzarella Dec. 1, 2020, 4:23 p.m. UTC | #2
On Tue, Dec 01, 2020 at 05:25:05PM +0200, Andra Paraschiv wrote:
>The vsock flag has been set in the connect and (listen) receive paths.
>
>When the vsock transport is assigned, the remote CID is used to
>distinguish between types of connection.
>
>Use the vsock flag (in addition to the CID) from the remote address to
>decide which vsock transport to assign. For the sibling VMs use case,
>all the vsock packets need to be forwarded to the host, so always assign
>the guest->host transport if the vsock flag is set. For the other use
>cases, the vsock transport assignment logic is not changed.
>
>Signed-off-by: Andra Paraschiv <andraprs@amazon.com>
>---
> net/vmw_vsock/af_vsock.c | 15 +++++++++++----
> 1 file changed, 11 insertions(+), 4 deletions(-)
>
>diff --git a/net/vmw_vsock/af_vsock.c b/net/vmw_vsock/af_vsock.c
>index d10916ab45267..bafc1cb20abd4 100644
>--- a/net/vmw_vsock/af_vsock.c
>+++ b/net/vmw_vsock/af_vsock.c
>@@ -419,16 +419,21 @@ static void vsock_deassign_transport(struct vsock_sock *vsk)
>  * (e.g. during the connect() or when a connection request on a listener
>  * socket is received).
>  * The vsk->remote_addr is used to decide which transport to use:
>- *  - remote CID == VMADDR_CID_LOCAL or g2h->local_cid or VMADDR_CID_HOST if
>- *    g2h is not loaded, will use local transport;
>- *  - remote CID <= VMADDR_CID_HOST will use guest->host transport;
>- *  - remote CID > VMADDR_CID_HOST will use host->guest transport;
>+ *  - remote flag == VMADDR_FLAG_SIBLING_VMS_COMMUNICATION, will always
>+ *    forward the vsock packets to the host and use guest->host transport;
>+ *  - otherwise, going forward with the remote flag default value:
>+ *    - remote CID == VMADDR_CID_LOCAL or g2h->local_cid or VMADDR_CID_HOST
>+ *      if g2h is not loaded, will use local transport;
>+ *    - remote CID <= VMADDR_CID_HOST or h2g is not loaded, will use
>+ *      guest->host transport;
>+ *    - remote CID > VMADDR_CID_HOST will use host->guest transport;
>  */
> int vsock_assign_transport(struct vsock_sock *vsk, struct vsock_sock *psk)
> {
> 	const struct vsock_transport *new_transport;
> 	struct sock *sk = sk_vsock(vsk);
> 	unsigned int remote_cid = vsk->remote_addr.svm_cid;
>+	unsigned short remote_flag = vsk->remote_addr.svm_flag;
> 	int ret;
>
> 	switch (sk->sk_type) {
>@@ -438,6 +443,8 @@ int vsock_assign_transport(struct vsock_sock *vsk, struct vsock_sock *psk)
> 	case SOCK_STREAM:
> 		if (vsock_use_local_transport(remote_cid))
> 			new_transport = transport_local;
>+		else if (remote_flag == VMADDR_FLAG_SIBLING_VMS_COMMUNICATION)

Others flags can be added, so here we should use the bitwise AND 
operator to check if this flag is set.

And what about merging with the next if clause?


Thanks,
Stefano

>+			new_transport = transport_g2h;
> 		else if (remote_cid <= VMADDR_CID_HOST || 
> 		!transport_h2g)
> 			new_transport = transport_g2h;
> 		else
>-- 
>2.20.1 (Apple Git-117)
>
Paraschiv, Andra-Irina Dec. 1, 2020, 6:02 p.m. UTC | #3
On 01/12/2020 18:27, Stefano Garzarella wrote:
>
>
> Hi Andra,
>
> On Tue, Dec 01, 2020 at 05:25:02PM +0200, Andra Paraschiv wrote:
>> vsock enables communication between virtual machines and the host 
>> they are
>> running on. Nested VMs can be setup to use vsock channels, as the multi
>> transport support has been available in the mainline since the v5.5 
>> Linux kernel
>> has been released.
>>
>> Implicitly, if no host->guest vsock transport is loaded, all the 
>> vsock packets
>> are forwarded to the host. This behavior can be used to setup 
>> communication
>> channels between sibling VMs that are running on the same host. One 
>> example can
>> be the vsock channels that can be established within AWS Nitro Enclaves
>> (see Documentation/virt/ne_overview.rst).
>>
>> To be able to explicitly mark a connection as being used for a 
>> certain use case,
>> add a flag field in the vsock address data structure. The 
>> "svm_reserved1" field
>> has been repurposed to be the flag field. The value of the flag will 
>> then be
>> taken into consideration when the vsock transport is assigned.
>>
>> This way can distinguish between nested VMs / local communication and 
>> sibling
>> VMs use cases. And can also setup one or more types of communication 
>> at the same
>> time.
>
> Thanks to work on this, I've left you a few comments, but I think this
> is the right way to support nested and sibling communication together.

Hi Stefano,

Thanks also for taking time to review and both you and Stefan for 
sharing an overview of this proposed option.

I'm going through the comments and will send out the v2 of the patch 
series as I have the changes done and validated.

Thanks,
Andra



Amazon Development Center (Romania) S.R.L. registered office: 27A Sf. Lazar Street, UBC5, floor 2, Iasi, Iasi County, 700045, Romania. Registered in Romania. Registration number J22/2621/2005.
Paraschiv, Andra-Irina Dec. 1, 2020, 6:15 p.m. UTC | #4
On 01/12/2020 18:09, Stefano Garzarella wrote:
>
> On Tue, Dec 01, 2020 at 05:25:03PM +0200, Andra Paraschiv wrote:
>> vsock enables communication between virtual machines and the host they
>> are running on. With the multi transport support (guest->host and
>> host->guest), nested VMs can also use vsock channels for communication.
>>
>> In addition to this, by default, all the vsock packets are forwarded to
>> the host, if no host->guest transport is loaded. This behavior can be
>> implicitly used for enabling vsock communication between sibling VMs.
>>
>> Add a flag field in the vsock address data structure that can be used to
>> explicitly mark the vsock connection as being targeted for a certain
>> type of communication. This way, can distinguish between nested VMs and
>> sibling VMs use cases and can also setup them at the same time. Till
>> now, could either have nested VMs or sibling VMs at a time using the
>> vsock communication stack.
>>
>> Use the already available "svm_reserved1" field and mark it as a flag
>> field instead. This flag can be set when initializing the vsock address
>> variable used for the connect() call.
>
> Maybe we can split this patch in 2 patches, one to rename the svm_flag
> and one to add the new flags.

Sure, I can split this in 2 patches, to have a bit more separation of 
duties.

>
>>
>> Signed-off-by: Andra Paraschiv <andraprs@amazon.com>
>> ---
>> include/uapi/linux/vm_sockets.h | 18 +++++++++++++++++-
>> 1 file changed, 17 insertions(+), 1 deletion(-)
>>
>> diff --git a/include/uapi/linux/vm_sockets.h 
>> b/include/uapi/linux/vm_sockets.h
>> index fd0ed7221645d..58da5a91413ac 100644
>> --- a/include/uapi/linux/vm_sockets.h
>> +++ b/include/uapi/linux/vm_sockets.h
>> @@ -114,6 +114,22 @@
>>
>> #define VMADDR_CID_HOST 2
>>
>> +/* This sockaddr_vm flag value covers the current default use case:
>> + * local vsock communication between guest and host and nested VMs 
>> setup.
>> + * In addition to this, implicitly, the vsock packets are forwarded 
>> to the host
>> + * if no host->guest vsock transport is set.
>> + */
>> +#define VMADDR_FLAG_DEFAULT_COMMUNICATION     0x0000
>
> I think we don't need this macro, since the next one can be used to
> check if it a sibling communication (flag 0x1 set) or not (flag 0x1
> not set).

Right, that's not particularly the use of the flag value, as by default 
comes as 0. It was more for sharing the cases this covers. But I can 
remove the define and keep this kind of info, with regard to the default 
case, in the commit message / comments.

>
>> +
>> +/* Set this flag value in the sockaddr_vm corresponding field if the 
>> vsock
>> + * channel needs to be setup between two sibling VMs running on the 
>> same host.
>> + * This way can explicitly distinguish between vsock channels 
>> created for nested
>> + * VMs (or local communication between guest and host) and the ones 
>> created for
>> + * sibling VMs. And vsock channels for multiple use cases (nested / 
>> sibling VMs)
>> + * can be setup at the same time.
>> + */
>> +#define VMADDR_FLAG_SIBLING_VMS_COMMUNICATION 0x0001
>
> What do you think if we shorten in VMADDR_FLAG_SIBLING?
>

Yup, this seems ok as well for me. I'll update the naming.

Thanks,
Andra

>
>> +
>> /* Invalid vSockets version. */
>>
>> #define VM_SOCKETS_INVALID_VERSION -1U
>> @@ -145,7 +161,7 @@
>>
>> struct sockaddr_vm {
>>       __kernel_sa_family_t svm_family;
>> -      unsigned short svm_reserved1;
>> +      unsigned short svm_flag;
>>       unsigned int svm_port;
>>       unsigned int svm_cid;
>>       unsigned char svm_zero[sizeof(struct sockaddr) -
>> -- 
>> 2.20.1 (Apple Git-117)
>>
>>
>>
>>
>> Amazon Development Center (Romania) S.R.L. registered office: 27A Sf. 
>> Lazar Street, UBC5, floor 2, Iasi, Iasi County, 700045, Romania. 
>> Registered in Romania. Registration number J22/2621/2005.
>>
>




Amazon Development Center (Romania) S.R.L. registered office: 27A Sf. Lazar Street, UBC5, floor 2, Iasi, Iasi County, 700045, Romania. Registered in Romania. Registration number J22/2621/2005.
Paraschiv, Andra-Irina Dec. 1, 2020, 7:06 p.m. UTC | #5
On 01/12/2020 18:23, Stefano Garzarella wrote:
>
> On Tue, Dec 01, 2020 at 05:25:05PM +0200, Andra Paraschiv wrote:
>> The vsock flag has been set in the connect and (listen) receive paths.
>>
>> When the vsock transport is assigned, the remote CID is used to
>> distinguish between types of connection.
>>
>> Use the vsock flag (in addition to the CID) from the remote address to
>> decide which vsock transport to assign. For the sibling VMs use case,
>> all the vsock packets need to be forwarded to the host, so always assign
>> the guest->host transport if the vsock flag is set. For the other use
>> cases, the vsock transport assignment logic is not changed.
>>
>> Signed-off-by: Andra Paraschiv <andraprs@amazon.com>
>> ---
>> net/vmw_vsock/af_vsock.c | 15 +++++++++++----
>> 1 file changed, 11 insertions(+), 4 deletions(-)
>>
>> diff --git a/net/vmw_vsock/af_vsock.c b/net/vmw_vsock/af_vsock.c
>> index d10916ab45267..bafc1cb20abd4 100644
>> --- a/net/vmw_vsock/af_vsock.c
>> +++ b/net/vmw_vsock/af_vsock.c
>> @@ -419,16 +419,21 @@ static void vsock_deassign_transport(struct 
>> vsock_sock *vsk)
>>  * (e.g. during the connect() or when a connection request on a listener
>>  * socket is received).
>>  * The vsk->remote_addr is used to decide which transport to use:
>> - *  - remote CID == VMADDR_CID_LOCAL or g2h->local_cid or 
>> VMADDR_CID_HOST if
>> - *    g2h is not loaded, will use local transport;
>> - *  - remote CID <= VMADDR_CID_HOST will use guest->host transport;
>> - *  - remote CID > VMADDR_CID_HOST will use host->guest transport;
>> + *  - remote flag == VMADDR_FLAG_SIBLING_VMS_COMMUNICATION, will always
>> + *    forward the vsock packets to the host and use guest->host 
>> transport;
>> + *  - otherwise, going forward with the remote flag default value:
>> + *    - remote CID == VMADDR_CID_LOCAL or g2h->local_cid or 
>> VMADDR_CID_HOST
>> + *      if g2h is not loaded, will use local transport;
>> + *    - remote CID <= VMADDR_CID_HOST or h2g is not loaded, will use
>> + *      guest->host transport;
>> + *    - remote CID > VMADDR_CID_HOST will use host->guest transport;
>>  */
>> int vsock_assign_transport(struct vsock_sock *vsk, struct vsock_sock 
>> *psk)
>> {
>>       const struct vsock_transport *new_transport;
>>       struct sock *sk = sk_vsock(vsk);
>>       unsigned int remote_cid = vsk->remote_addr.svm_cid;
>> +      unsigned short remote_flag = vsk->remote_addr.svm_flag;
>>       int ret;
>>
>>       switch (sk->sk_type) {
>> @@ -438,6 +443,8 @@ int vsock_assign_transport(struct vsock_sock 
>> *vsk, struct vsock_sock *psk)
>>       case SOCK_STREAM:
>>               if (vsock_use_local_transport(remote_cid))
>>                       new_transport = transport_local;
>> +              else if (remote_flag == 
>> VMADDR_FLAG_SIBLING_VMS_COMMUNICATION)
>
> Others flags can be added, so here we should use the bitwise AND
> operator to check if this flag is set.
>
> And what about merging with the next if clause?
>

Indeed, I'll update the codebase to use the bitwise operator. Then I can 
also merge all the checks corresponding to the g2h transport in a single 
if block.

Thanks,
Andra

>
>> +                      new_transport = transport_g2h;
>>               else if (remote_cid <= VMADDR_CID_HOST ||
>>               !transport_h2g)
>>                       new_transport = transport_g2h;
>>               else
>> -- 
>> 2.20.1 (Apple Git-117)
>>
>




Amazon Development Center (Romania) S.R.L. registered office: 27A Sf. Lazar Street, UBC5, floor 2, Iasi, Iasi County, 700045, Romania. Registered in Romania. Registration number J22/2621/2005.
Stefano Garzarella Dec. 2, 2020, 8:32 a.m. UTC | #6
On Tue, Dec 01, 2020 at 08:15:04PM +0200, Paraschiv, Andra-Irina wrote:
>

>

>On 01/12/2020 18:09, Stefano Garzarella wrote:

>>

>>On Tue, Dec 01, 2020 at 05:25:03PM +0200, Andra Paraschiv wrote:

>>>vsock enables communication between virtual machines and the host they

>>>are running on. With the multi transport support (guest->host and

>>>host->guest), nested VMs can also use vsock channels for communication.

>>>

>>>In addition to this, by default, all the vsock packets are forwarded to

>>>the host, if no host->guest transport is loaded. This behavior can be

>>>implicitly used for enabling vsock communication between sibling VMs.

>>>

>>>Add a flag field in the vsock address data structure that can be used to

>>>explicitly mark the vsock connection as being targeted for a certain

>>>type of communication. This way, can distinguish between nested VMs and

>>>sibling VMs use cases and can also setup them at the same time. Till

>>>now, could either have nested VMs or sibling VMs at a time using the

>>>vsock communication stack.

>>>

>>>Use the already available "svm_reserved1" field and mark it as a flag

>>>field instead. This flag can be set when initializing the vsock address

>>>variable used for the connect() call.

>>

>>Maybe we can split this patch in 2 patches, one to rename the svm_flag

>>and one to add the new flags.

>

>Sure, I can split this in 2 patches, to have a bit more separation of 

>duties.

>

>>

>>>

>>>Signed-off-by: Andra Paraschiv <andraprs@amazon.com>

>>>---

>>>include/uapi/linux/vm_sockets.h | 18 +++++++++++++++++-

>>>1 file changed, 17 insertions(+), 1 deletion(-)

>>>

>>>diff --git a/include/uapi/linux/vm_sockets.h 

>>>b/include/uapi/linux/vm_sockets.h

>>>index fd0ed7221645d..58da5a91413ac 100644

>>>--- a/include/uapi/linux/vm_sockets.h

>>>+++ b/include/uapi/linux/vm_sockets.h

>>>@@ -114,6 +114,22 @@

>>>

>>>#define VMADDR_CID_HOST 2

>>>

>>>+/* This sockaddr_vm flag value covers the current default use case:

>>>+ * local vsock communication between guest and host and nested 

>>>VMs setup.

>>>+ * In addition to this, implicitly, the vsock packets are 

>>>forwarded to the host

>>>+ * if no host->guest vsock transport is set.

>>>+ */

>>>+#define VMADDR_FLAG_DEFAULT_COMMUNICATION     0x0000

>>

>>I think we don't need this macro, since the next one can be used to

>>check if it a sibling communication (flag 0x1 set) or not (flag 0x1

>>not set).

>

>Right, that's not particularly the use of the flag value, as by 

>default comes as 0. It was more for sharing the cases this covers. But 

>I can remove the define and keep this kind of info, with regard to the 

>default case, in the commit message / comments.

>


Agree, you can add few lines in the comment block of VMADDR_FLAG_SIBLING 
describing the default case when it is not set.

>>

>>>+

>>>+/* Set this flag value in the sockaddr_vm corresponding field if 

>>>the vsock

>>>+ * channel needs to be setup between two sibling VMs running on 

>>>the same host.

>>>+ * This way can explicitly distinguish between vsock channels 

>>>created for nested

>>>+ * VMs (or local communication between guest and host) and the 

>>>ones created for

>>>+ * sibling VMs. And vsock channels for multiple use cases (nested 

>>>/ sibling VMs)

>>>+ * can be setup at the same time.

>>>+ */

>>>+#define VMADDR_FLAG_SIBLING_VMS_COMMUNICATION 0x0001

>>

>>What do you think if we shorten in VMADDR_FLAG_SIBLING?

>>

>

>Yup, this seems ok as well for me. I'll update the naming.

>


Thanks,
Stefano
Stefano Garzarella Dec. 2, 2020, 1:37 p.m. UTC | #7
Hi Andra,

On Tue, Dec 01, 2020 at 05:25:02PM +0200, Andra Paraschiv wrote:
>vsock enables communication between virtual machines and the host they are

>running on. Nested VMs can be setup to use vsock channels, as the multi

>transport support has been available in the mainline since the v5.5 Linux kernel

>has been released.

>

>Implicitly, if no host->guest vsock transport is loaded, all the vsock packets

>are forwarded to the host. This behavior can be used to setup communication

>channels between sibling VMs that are running on the same host. One example can

>be the vsock channels that can be established within AWS Nitro Enclaves

>(see Documentation/virt/ne_overview.rst).

>

>To be able to explicitly mark a connection as being used for a certain use case,

>add a flag field in the vsock address data structure. The "svm_reserved1" field

>has been repurposed to be the flag field. The value of the flag will then be

>taken into consideration when the vsock transport is assigned.

>

>This way can distinguish between nested VMs / local communication and sibling

>VMs use cases. And can also setup one or more types of communication at the same

>time.

>


Another thing worth mentioning is that for now it is not supported in 
vhost-vsock, since we are discarding every packet not addressed to the 
host.

What we should do would be:
- add a new IOCTL to vhost-vsock to enable sibling communication, by 
   default I'd like to leave it disabled

- allow sibling forwarding only if both guests have sibling 
   communication enabled and we should implement some kind of filtering 
   or network namespace support to allow the communication only between a 
   subset of VMs


Do you have plans to work on it?

Otherwise I put it in my to-do list and hope I have time to do it (maybe 
next month).

Thanks,
Stefano
Paraschiv, Andra-Irina Dec. 2, 2020, 4:18 p.m. UTC | #8
On 02/12/2020 15:37, Stefano Garzarella wrote:
>
> Hi Andra,
>
> On Tue, Dec 01, 2020 at 05:25:02PM +0200, Andra Paraschiv wrote:
>> vsock enables communication between virtual machines and the host 
>> they are
>> running on. Nested VMs can be setup to use vsock channels, as the multi
>> transport support has been available in the mainline since the v5.5 
>> Linux kernel
>> has been released.
>>
>> Implicitly, if no host->guest vsock transport is loaded, all the 
>> vsock packets
>> are forwarded to the host. This behavior can be used to setup 
>> communication
>> channels between sibling VMs that are running on the same host. One 
>> example can
>> be the vsock channels that can be established within AWS Nitro Enclaves
>> (see Documentation/virt/ne_overview.rst).
>>
>> To be able to explicitly mark a connection as being used for a 
>> certain use case,
>> add a flag field in the vsock address data structure. The 
>> "svm_reserved1" field
>> has been repurposed to be the flag field. The value of the flag will 
>> then be
>> taken into consideration when the vsock transport is assigned.
>>
>> This way can distinguish between nested VMs / local communication and 
>> sibling
>> VMs use cases. And can also setup one or more types of communication 
>> at the same
>> time.
>>
>
> Another thing worth mentioning is that for now it is not supported in
> vhost-vsock, since we are discarding every packet not addressed to the
> host.

Right, thanks for the follow-up.

>
> What we should do would be:
> - add a new IOCTL to vhost-vsock to enable sibling communication, by
>   default I'd like to leave it disabled
>
> - allow sibling forwarding only if both guests have sibling
>   communication enabled and we should implement some kind of filtering
>   or network namespace support to allow the communication only between a
>   subset of VMs
>
>
> Do you have plans to work on it?

Nope, not yet. But I can take some time in the second part of December / 
beginning of January for this. And we can catch up in the meantime if 
there is something blocking or more clarifications are needed to make it 
work.

Thanks,
Andra

>
>
> Otherwise I put it in my to-do list and hope I have time to do it (maybe
> next month).
>
> Thanks,
> Stefano
>




Amazon Development Center (Romania) S.R.L. registered office: 27A Sf. Lazar Street, UBC5, floor 2, Iasi, Iasi County, 700045, Romania. Registered in Romania. Registration number J22/2621/2005.
Stefano Garzarella Dec. 3, 2020, 8:51 a.m. UTC | #9
On Wed, Dec 02, 2020 at 06:18:15PM +0200, Paraschiv, Andra-Irina wrote:
>

>

>On 02/12/2020 15:37, Stefano Garzarella wrote:

>>

>>Hi Andra,

>>

>>On Tue, Dec 01, 2020 at 05:25:02PM +0200, Andra Paraschiv wrote:

>>>vsock enables communication between virtual machines and the host 

>>>they are

>>>running on. Nested VMs can be setup to use vsock channels, as the multi

>>>transport support has been available in the mainline since the 

>>>v5.5 Linux kernel

>>>has been released.

>>>

>>>Implicitly, if no host->guest vsock transport is loaded, all the 

>>>vsock packets

>>>are forwarded to the host. This behavior can be used to setup 

>>>communication

>>>channels between sibling VMs that are running on the same host. 

>>>One example can

>>>be the vsock channels that can be established within AWS Nitro Enclaves

>>>(see Documentation/virt/ne_overview.rst).

>>>

>>>To be able to explicitly mark a connection as being used for a 

>>>certain use case,

>>>add a flag field in the vsock address data structure. The 

>>>"svm_reserved1" field

>>>has been repurposed to be the flag field. The value of the flag 

>>>will then be

>>>taken into consideration when the vsock transport is assigned.

>>>

>>>This way can distinguish between nested VMs / local communication 

>>>and sibling

>>>VMs use cases. And can also setup one or more types of 

>>>communication at the same

>>>time.

>>>

>>

>>Another thing worth mentioning is that for now it is not supported in

>>vhost-vsock, since we are discarding every packet not addressed to the

>>host.

>

>Right, thanks for the follow-up.

>

>>

>>What we should do would be:

>>- add a new IOCTL to vhost-vsock to enable sibling communication, by

>>  default I'd like to leave it disabled

>>

>>- allow sibling forwarding only if both guests have sibling

>>  communication enabled and we should implement some kind of filtering

>>  or network namespace support to allow the communication only between a

>>  subset of VMs

>>

>>

>>Do you have plans to work on it?

>

>Nope, not yet. But I can take some time in the second part of December 

>/ beginning of January for this. And we can catch up in the meantime 

>if there is something blocking or more clarifications are needed to 

>make it work.

>


Good, it will be great!

Thanks,
Stefano