| Message ID | 20241125070633.8042-1-anuj20.g@samsung.com |
|---|---|
| Headers | show |
| Series | Read/Write with meta/integrity | expand |
On 11/25/24 07:06, Anuj Gupta wrote: > Add the ability to pass additional attributes along with read/write. > Application can populate attribute type and attibute specific information > in 'struct io_uring_attr' and pass its address using the SQE field: > __u64 attr_ptr; > > Along with setting a mask indicating attributes being passed: > __u64 attr_type_mask; > > Overall 64 attributes are allowed and currently one attribute > 'ATTR_TYPE_PI' is supported. > > With PI attribute, userspace can pass following information: > - flags: integrity check flags IO_INTEGRITY_CHK_{GUARD/APPTAG/REFTAG} > - len: length of PI/metadata buffer > - addr: address of metadata buffer > - seed: seed value for reftag remapping > - app_tag: application defined 16b value The API and io_uring parts look good, I'll ask to address the ATTR_TYPE comment below, the rest are nits, which that can be ignored and/or delayed. > Process this information to prepare uio_meta_descriptor and pass it down > using kiocb->private. I'm not sure using ->private is a good thing, but I assume it was discussed, so I'll leave it to Jens and other folks. > PI attribute is supported only for direct IO. > > Signed-off-by: Anuj Gupta <anuj20.g@samsung.com> > Signed-off-by: Kanchan Joshi <joshi.k@samsung.com> > --- > include/uapi/linux/io_uring.h | 31 +++++++++++++ > io_uring/io_uring.c | 2 + > io_uring/rw.c | 82 ++++++++++++++++++++++++++++++++++- > io_uring/rw.h | 14 +++++- > 4 files changed, 126 insertions(+), 3 deletions(-) > > diff --git a/include/uapi/linux/io_uring.h b/include/uapi/linux/io_uring.h > index aac9a4f8fa9a..bf28d49583ad 100644 > --- a/include/uapi/linux/io_uring.h > +++ b/include/uapi/linux/io_uring.h > @@ -98,6 +98,10 @@ struct io_uring_sqe { > __u64 addr3; > __u64 __pad2[1]; > }; > + struct { > + __u64 attr_ptr; /* pointer to attribute information */ > + __u64 attr_type_mask; /* bit mask of attributes */ > + }; > __u64 optval; > /* > * If the ring is initialized with IORING_SETUP_SQE128, then > @@ -107,6 +111,33 @@ struct io_uring_sqe { > }; > }; > > + > +/* Attributes to be passed with read/write */ > +enum io_uring_attr_type { > + ATTR_TYPE_PI, > + /* max supported attributes */ > + ATTR_TYPE_LAST = 64, ATTR_TYPE sounds too generic, too easy to get a symbol collision including with user space code. Some options: IORING_ATTR_TYPE_PI, IORING_RW_ATTR_TYPE_PI. If it's not supposed to be io_uring specific can be IO_RW_ATTR_TYPE_PI > +}; > + > +/* sqe->attr_type_mask flags */ > +#define ATTR_FLAG_PI (1U << ATTR_TYPE_PI) > +/* PI attribute information */ > +struct io_uring_attr_pi { > + __u16 flags; > + __u16 app_tag; > + __u32 len; > + __u64 addr; > + __u64 seed; > + __u64 rsvd; > +}; > + > +/* attribute information along with type */ > +struct io_uring_attr { > + enum io_uring_attr_type attr_type; I'm not against it, but adding a type field to each attribute is not strictly needed, you can already derive where each attr placed purely from the mask. Are there some upsides? But again I'm not against it. > + /* type specific struct here */ > + struct io_uring_attr_pi pi; > +}; > + > /* > * If sqe->file_index is set to this for opcodes that instantiate a new > * direct descriptor (like openat/openat2/accept), then io_uring will allocate > diff --git a/io_uring/io_uring.c b/io_uring/io_uring.c > index c3a7d0197636..02291ea679fb 100644 > --- a/io_uring/io_uring.c > +++ b/io_uring/io_uring.c > @@ -3889,6 +3889,8 @@ static int __init io_uring_init(void) > BUILD_BUG_SQE_ELEM(46, __u16, __pad3[0]); > BUILD_BUG_SQE_ELEM(48, __u64, addr3); > BUILD_BUG_SQE_ELEM_SIZE(48, 0, cmd); > + BUILD_BUG_SQE_ELEM(48, __u64, attr_ptr); > + BUILD_BUG_SQE_ELEM(56, __u64, attr_type_mask); > BUILD_BUG_SQE_ELEM(56, __u64, __pad2); > > BUILD_BUG_ON(sizeof(struct io_uring_files_update) != > diff --git a/io_uring/rw.c b/io_uring/rw.c > index 0bcb83e4ce3c..71bfb74fef96 100644 > --- a/io_uring/rw.c > +++ b/io_uring/rw.c > @@ -257,11 +257,54 @@ static int io_prep_rw_setup(struct io_kiocb *req, int ddir, bool do_import) > return 0; > } ... > @@ -902,6 +976,8 @@ static int __io_read(struct io_kiocb *req, unsigned int issue_flags) > * manually if we need to. > */ > iov_iter_restore(&io->iter, &io->iter_state); > + if (kiocb->ki_flags & IOCB_HAS_METADATA) > + io_meta_restore(io); That can be turned into a helper, but that can be done as a follow up. I'd also add a IOCB_HAS_METADATA into or around of io_rw_should_retry(). You're relying on O_DIRECT checks, but that sounds a bit fragile in the long run.
On Mon, Nov 25, 2024 at 02:58:19PM +0000, Pavel Begunkov wrote: > On 11/25/24 07:06, Anuj Gupta wrote: > > ATTR_TYPE sounds too generic, too easy to get a symbol collision > including with user space code. > > Some options: IORING_ATTR_TYPE_PI, IORING_RW_ATTR_TYPE_PI. > If it's not supposed to be io_uring specific can be > IO_RW_ATTR_TYPE_PI > Sure, will change to a different name in the next iteration. > > + > > +/* attribute information along with type */ > > +struct io_uring_attr { > > + enum io_uring_attr_type attr_type; > > I'm not against it, but adding a type field to each attribute is not > strictly needed, you can already derive where each attr placed purely > from the mask. Are there some upsides? But again I'm not against it. > The mask indicates what all attributes have been passed. But while processing we would need to know where exactly the attributes have been placed, as attributes are of different sizes (each attribute is of fixed size though) and they could be placed in any order. Processing when multiple attributes are passed would look something like this: attr_ptr = READ_ONCE(sqe->attr_ptr); attr_mask = READ_ONCE(sqe->attr_type_mask); size = total_size_of_attributes_passed_from_attr_mask; copy_from_user(attr_buf, attr_ptr, size); while (size > 0) { if (sizeof(io_uring_attr_type) > size) break; attr_type = get_type(attr_buf); attr_size = get_size(attr_type); process_attr(attr_type, attr_buf); attr_buf += attr_size; size -= attr_size; } We cannot derive where exactly the attribute information is placed purely from the mask, so we need the type field. Do you see it differently?
On 11/26/24 10:40, Anuj Gupta wrote: > On Mon, Nov 25, 2024 at 02:58:19PM +0000, Pavel Begunkov wrote: >> On 11/25/24 07:06, Anuj Gupta wrote: >> >> ATTR_TYPE sounds too generic, too easy to get a symbol collision >> including with user space code. >> >> Some options: IORING_ATTR_TYPE_PI, IORING_RW_ATTR_TYPE_PI. >> If it's not supposed to be io_uring specific can be >> IO_RW_ATTR_TYPE_PI >> > > Sure, will change to a different name in the next iteration. > >>> + >>> +/* attribute information along with type */ >>> +struct io_uring_attr { >>> + enum io_uring_attr_type attr_type; >> >> I'm not against it, but adding a type field to each attribute is not >> strictly needed, you can already derive where each attr placed purely >> from the mask. Are there some upsides? But again I'm not against it. >> > > The mask indicates what all attributes have been passed. But while > processing we would need to know where exactly the attributes have been > placed, as attributes are of different sizes (each attribute is of > fixed size though) and they could be placed in any order. Processing when > multiple attributes are passed would look something like this: > > attr_ptr = READ_ONCE(sqe->attr_ptr); > attr_mask = READ_ONCE(sqe->attr_type_mask); > size = total_size_of_attributes_passed_from_attr_mask; > > copy_from_user(attr_buf, attr_ptr, size); > > while (size > 0) { > if (sizeof(io_uring_attr_type) > size) > break; > > attr_type = get_type(attr_buf); > attr_size = get_size(attr_type); > > process_attr(attr_type, attr_buf); > attr_buf += attr_size; > size -= attr_size; > } > > We cannot derive where exactly the attribute information is placed > purely from the mask, so we need the type field. Do you see it > differently? In the mask version I outlined attributes go in the array in order of their types, max 1 attribute of each type, in which case the mask fully describes the placement. static attr_param_sizes[] = {[TYPE_PI] = sizeof(pi), ...}; mask = READ_ONCE(sqe->mask); off = 0; for (type = 0; type < NR_TYPE; type++) { // or find_next_bit trick if (!(mask & BIT(i))) continue; process(type=i, pointer= base + attr_param_sizes[i]); off += attr_param_sizes[i]; } Maybe it's a good idea to have a type field for double checking though, and with it we don't have to commit to one version or another yet.
On 11/26/24 13:01, Pavel Begunkov wrote: > On 11/25/24 07:06, Anuj Gupta wrote: > ... >> +/* sqe->attr_type_mask flags */ >> +#define ATTR_FLAG_PI (1U << ATTR_TYPE_PI) >> +/* PI attribute information */ >> +struct io_uring_attr_pi { >> + __u16 flags; >> + __u16 app_tag; >> + __u32 len; >> + __u64 addr; >> + __u64 seed; >> + __u64 rsvd; >> +}; >> + >> +/* attribute information along with type */ >> +struct io_uring_attr { >> + enum io_uring_attr_type attr_type; > > Hmm, I think there will be implicit padding, we need to deal > with it. And it's better to be explicitly sized, e.g. s/enum io_uring_attr_type/__u16/ >> + /* type specific struct here */ >> + struct io_uring_attr_pi pi; >> +}; > > This also looks PI specific but with a generic name. Or are > attribute structures are supposed to be unionised? >
On 11/26/24 13:54, Anuj Gupta wrote: > On Tue, Nov 26, 2024 at 01:01:03PM +0000, Pavel Begunkov wrote: >> On 11/25/24 07:06, Anuj Gupta wrote: >> ... >>> + /* type specific struct here */ >>> + struct io_uring_attr_pi pi; >>> +}; >> >> This also looks PI specific but with a generic name. Or are >> attribute structures are supposed to be unionised? > > Yes, attribute structures would be unionised here. This is done so that > "attr_type" always remains at the top. When there are multiple attributes > this structure would look something like this: > > /* attribute information along with type */ > struct io_uring_attr { > enum io_uring_attr_type attr_type; > /* type specific struct here */ > union { > struct io_uring_attr_pi pi; > struct io_uring_attr_x x; > struct io_uring_attr_y y; > }; > }; > > And then on the application side for sending attribute x, one would do: > > io_uring_attr attr; > attr.type = TYPE_X; > prepare_attr(&attr.x); Hmm, I have doubts it's going to work well because the union members have different sizes. Adding a new type could grow struct io_uring_attr, which is already bad for uapi. And it can't be stacked: io_uring_attr attrs[2] = {..., ...} sqe->attr_ptr = &attrs; ... This example would be incorrect. Even if it's just one attribute the user would be wasting space on stack. The only use for it I see is having ephemeral pointers during parsing, ala void parse(voud *attributes, offset) { struct io_uring_attr *attr = attributes + offset; if (attr->type == PI) { process_pi(&attr->pi); // or potentially fill_pi() in userspace } } But I don't think it's worth it. I'd say, if you're leaving the structure, let's rename it to struct io_uring_attr_type_pi or something similar. We can always add a new one later, it doesn't change the ABI.
On Tue, Nov 26, 2024 at 9:14 PM Pavel Begunkov <asml.silence@gmail.com> wrote: > > On 11/26/24 13:54, Anuj Gupta wrote: > > On Tue, Nov 26, 2024 at 01:01:03PM +0000, Pavel Begunkov wrote: > >> On 11/25/24 07:06, Anuj Gupta wrote: > >> ... > >>> + /* type specific struct here */ > >>> + struct io_uring_attr_pi pi; > >>> +}; > >> > >> This also looks PI specific but with a generic name. Or are > >> attribute structures are supposed to be unionised? > > > > Yes, attribute structures would be unionised here. This is done so that > > "attr_type" always remains at the top. When there are multiple attributes > > this structure would look something like this: > > > > /* attribute information along with type */ > > struct io_uring_attr { > > enum io_uring_attr_type attr_type; > > /* type specific struct here */ > > union { > > struct io_uring_attr_pi pi; > > struct io_uring_attr_x x; > > struct io_uring_attr_y y; > > }; > > }; > > > > And then on the application side for sending attribute x, one would do: > > > > io_uring_attr attr; > > attr.type = TYPE_X; > > prepare_attr(&attr.x); > > Hmm, I have doubts it's going to work well because the union > members have different sizes. Adding a new type could grow > struct io_uring_attr, which is already bad for uapi. And it > can't be stacked: > > io_uring_attr attrs[2] = {..., ...} > sqe->attr_ptr = &attrs; > ... > > This example would be incorrect. Even if it's just one attribute > the user would be wasting space on stack. The only use for it I > see is having ephemeral pointers during parsing, ala > > void parse(voud *attributes, offset) { > struct io_uring_attr *attr = attributes + offset; > > if (attr->type == PI) { > process_pi(&attr->pi); > // or potentially fill_pi() in userspace > } > } > > But I don't think it's worth it. I'd say, if you're leaving > the structure, let's rename it to struct io_uring_attr_type_pi > or something similar. We can always add a new one later, it > doesn't change the ABI. > In that case I can just drop the io_uring_attr_pi structure then. We can keep the mask version where we won't need the type and attributes would go in the array in order of their types as you suggested here [1]. Does that sound fine? [1] https://lore.kernel.org/io-uring/37ba07f6-27a5-45bc-86c4-df9c63908ef9@gmail.com/
On Tue, Nov 26, 2024 at 03:45:09PM +0000, Pavel Begunkov wrote: > On 11/26/24 13:54, Anuj Gupta wrote: > > On Tue, Nov 26, 2024 at 01:01:03PM +0000, Pavel Begunkov wrote: > > > On 11/25/24 07:06, Anuj Gupta wrote: > > Hmm, I have doubts it's going to work well because the union > members have different sizes. Adding a new type could grow > struct io_uring_attr, which is already bad for uapi. And it > can't be stacked: > How about something like this [1]. I have removed the io_uring_attr structure, and with the mask scheme the user would pass attributes in order of their types. Do you still see some cracks? [1] diff --git a/include/uapi/linux/io_uring.h b/include/uapi/linux/io_uring.h index aac9a4f8fa9a..38f0d6b10eaf 100644 --- a/include/uapi/linux/io_uring.h +++ b/include/uapi/linux/io_uring.h @@ -98,6 +98,10 @@ struct io_uring_sqe { __u64 addr3; __u64 __pad2[1]; }; + struct { + __u64 attr_ptr; /* pointer to attribute information */ + __u64 attr_type_mask; /* bit mask of attributes */ + }; __u64 optval; /* * If the ring is initialized with IORING_SETUP_SQE128, then @@ -107,6 +111,18 @@ struct io_uring_sqe { }; }; +/* sqe->attr_type_mask flags */ +#define IORING_RW_ATTR_FLAG_PI (1U << 0) +/* PI attribute information */ +struct io_uring_attr_pi { + __u16 flags; + __u16 app_tag; + __u32 len; + __u64 addr; + __u64 seed; + __u64 rsvd; +}; + /* * If sqe->file_index is set to this for opcodes that instantiate a new * direct descriptor (like openat/openat2/accept), then io_uring will allocate diff --git a/io_uring/io_uring.c b/io_uring/io_uring.c index c3a7d0197636..02291ea679fb 100644 --- a/io_uring/io_uring.c +++ b/io_uring/io_uring.c @@ -3889,6 +3889,8 @@ static int __init io_uring_init(void) BUILD_BUG_SQE_ELEM(46, __u16, __pad3[0]); BUILD_BUG_SQE_ELEM(48, __u64, addr3); BUILD_BUG_SQE_ELEM_SIZE(48, 0, cmd); + BUILD_BUG_SQE_ELEM(48, __u64, attr_ptr); + BUILD_BUG_SQE_ELEM(56, __u64, attr_type_mask); BUILD_BUG_SQE_ELEM(56, __u64, __pad2); BUILD_BUG_ON(sizeof(struct io_uring_files_update) != diff --git a/io_uring/rw.c b/io_uring/rw.c index 0bcb83e4ce3c..8d2ec89fd76b 100644 --- a/io_uring/rw.c +++ b/io_uring/rw.c @@ -257,11 +257,53 @@ static int io_prep_rw_setup(struct io_kiocb *req, int ddir, bool do_import) return 0; } +static inline void io_meta_save_state(struct io_async_rw *io) +{ + io->meta_state.seed = io->meta.seed; + iov_iter_save_state(&io->meta.iter, &io->meta_state.iter_meta); +} + +static inline void io_meta_restore(struct io_async_rw *io, struct kiocb *kiocb) +{ + if (kiocb->ki_flags & IOCB_HAS_METADATA) { + io->meta.seed = io->meta_state.seed; + iov_iter_restore(&io->meta.iter, &io->meta_state.iter_meta); + } +} + +static int io_prep_rw_pi(struct io_kiocb *req, struct io_rw *rw, int ddir, + u64 attr_ptr, u64 attr_type_mask) +{ + struct io_uring_attr_pi pi_attr; + struct io_async_rw *io; + int ret; + + if (copy_from_user(&pi_attr, u64_to_user_ptr(attr_ptr), + sizeof(pi_attr))) + return -EFAULT; + + if (pi_attr.rsvd) + return -EINVAL; + + io = req->async_data; + io->meta.flags = pi_attr.flags; + io->meta.app_tag = pi_attr.app_tag; + io->meta.seed = READ_ONCE(pi_attr.seed); + ret = import_ubuf(ddir, u64_to_user_ptr(pi_attr.addr), + pi_attr.len, &io->meta.iter); + if (unlikely(ret < 0)) + return ret; + rw->kiocb.ki_flags |= IOCB_HAS_METADATA; + io_meta_save_state(io); + return ret; +} + static int io_prep_rw(struct io_kiocb *req, const struct io_uring_sqe *sqe, int ddir, bool do_import) { struct io_rw *rw = io_kiocb_to_cmd(req, struct io_rw); unsigned ioprio; + u64 attr_type_mask; int ret; rw->kiocb.ki_pos = READ_ONCE(sqe->off); @@ -279,11 +321,28 @@ static int io_prep_rw(struct io_kiocb *req, const struct io_uring_sqe *sqe, rw->kiocb.ki_ioprio = get_current_ioprio(); } rw->kiocb.dio_complete = NULL; + rw->kiocb.ki_flags = 0; rw->addr = READ_ONCE(sqe->addr); rw->len = READ_ONCE(sqe->len); rw->flags = READ_ONCE(sqe->rw_flags); - return io_prep_rw_setup(req, ddir, do_import); + ret = io_prep_rw_setup(req, ddir, do_import); + + if (unlikely(ret)) + return ret; + + attr_type_mask = READ_ONCE(sqe->attr_type_mask); + if (attr_type_mask) { + u64 attr_ptr; + + /* only PI attribute is supported currently */ + if (attr_type_mask != IORING_RW_ATTR_FLAG_PI) + return -EINVAL; + + attr_ptr = READ_ONCE(sqe->attr_ptr); + ret = io_prep_rw_pi(req, rw, ddir, attr_ptr, attr_type_mask); + } + return ret; } int io_prep_read(struct io_kiocb *req, const struct io_uring_sqe *sqe) @@ -409,7 +468,9 @@ static inline loff_t *io_kiocb_update_pos(struct io_kiocb *req) static void io_resubmit_prep(struct io_kiocb *req) { struct io_async_rw *io = req->async_data; + struct io_rw *rw = io_kiocb_to_cmd(req, struct io_rw); + io_meta_restore(io, &rw->kiocb); iov_iter_restore(&io->iter, &io->iter_state); } @@ -744,6 +805,10 @@ static bool io_rw_should_retry(struct io_kiocb *req) if (kiocb->ki_flags & (IOCB_DIRECT | IOCB_HIPRI)) return false; + /* never retry for meta io */ + if (kiocb->ki_flags & IOCB_HAS_METADATA) + return false; + /* * just use poll if we can, and don't attempt if the fs doesn't * support callback based unlocks @@ -794,7 +859,7 @@ static int io_rw_init_file(struct io_kiocb *req, fmode_t mode, int rw_type) if (!(req->flags & REQ_F_FIXED_FILE)) req->flags |= io_file_get_flags(file); - kiocb->ki_flags = file->f_iocb_flags; + kiocb->ki_flags |= file->f_iocb_flags; ret = kiocb_set_rw_flags(kiocb, rw->flags, rw_type); if (unlikely(ret)) return ret; @@ -828,6 +893,18 @@ static int io_rw_init_file(struct io_kiocb *req, fmode_t mode, int rw_type) kiocb->ki_complete = io_complete_rw; } + if (kiocb->ki_flags & IOCB_HAS_METADATA) { + struct io_async_rw *io = req->async_data; + + /* + * We have a union of meta fields with wpq used for buffered-io + * in io_async_rw, so fail it here. + */ + if (!(req->file->f_flags & O_DIRECT)) + return -EOPNOTSUPP; + kiocb->private = &io->meta; + } + return 0; } @@ -902,6 +979,7 @@ static int __io_read(struct io_kiocb *req, unsigned int issue_flags) * manually if we need to. */ iov_iter_restore(&io->iter, &io->iter_state); + io_meta_restore(io, kiocb); do { /* @@ -1125,6 +1203,7 @@ int io_write(struct io_kiocb *req, unsigned int issue_flags) } else { ret_eagain: iov_iter_restore(&io->iter, &io->iter_state); + io_meta_restore(io, kiocb); if (kiocb->ki_flags & IOCB_WRITE) io_req_end_write(req); return -EAGAIN; diff --git a/io_uring/rw.h b/io_uring/rw.h index 3f432dc75441..2d7656bd268d 100644 --- a/io_uring/rw.h +++ b/io_uring/rw.h @@ -2,6 +2,11 @@ #include <linux/pagemap.h> +struct io_meta_state { + u32 seed; + struct iov_iter_state iter_meta; +}; + struct io_async_rw { size_t bytes_done; struct iov_iter iter; @@ -9,7 +14,14 @@ struct io_async_rw { struct iovec fast_iov; struct iovec *free_iovec; int free_iov_nr; - struct wait_page_queue wpq; + /* wpq is for buffered io, while meta fields are used with direct io */ + union { + struct wait_page_queue wpq; + struct { + struct uio_meta meta; + struct io_meta_state meta_state; + }; + }; }; int io_prep_read_fixed(struct io_kiocb *req, const struct io_uring_sqe *sqe);
On 11/26/24 16:23, Anuj gupta wrote: > On Tue, Nov 26, 2024 at 9:14 PM Pavel Begunkov <asml.silence@gmail.com> wrote: ... >> This example would be incorrect. Even if it's just one attribute >> the user would be wasting space on stack. The only use for it I >> see is having ephemeral pointers during parsing, ala >> >> void parse(voud *attributes, offset) { >> struct io_uring_attr *attr = attributes + offset; >> >> if (attr->type == PI) { >> process_pi(&attr->pi); >> // or potentially fill_pi() in userspace >> } >> } >> >> But I don't think it's worth it. I'd say, if you're leaving >> the structure, let's rename it to struct io_uring_attr_type_pi >> or something similar. We can always add a new one later, it >> doesn't change the ABI. >> > > In that case I can just drop the io_uring_attr_pi structure then. We can > keep the mask version where we won't need the type and attributes would go > in the array in order of their types as you suggested here [1]. Does that > sound fine? That should work, the approach in this patchset is fine as well. I'll take a look at the path a bit later today. > [1] https://lore.kernel.org/io-uring/37ba07f6-27a5-45bc-86c4-df9c63908ef9@gmail.com/
On 11/27/24 09:46, Anuj Gupta wrote: > On Tue, Nov 26, 2024 at 03:45:09PM +0000, Pavel Begunkov wrote: >> On 11/26/24 13:54, Anuj Gupta wrote: >>> On Tue, Nov 26, 2024 at 01:01:03PM +0000, Pavel Begunkov wrote: >>>> On 11/25/24 07:06, Anuj Gupta wrote: >> >> Hmm, I have doubts it's going to work well because the union >> members have different sizes. Adding a new type could grow >> struct io_uring_attr, which is already bad for uapi. And it >> can't be stacked: >> > > How about something like this [1]. I have removed the io_uring_attr > structure, and with the mask scheme the user would pass attributes in > order of their types. Do you still see some cracks? Looks good to me > --- a/io_uring/rw.c > +++ b/io_uring/rw.c ... > +static int io_prep_rw_pi(struct io_kiocb *req, struct io_rw *rw, int ddir, > + u64 attr_ptr, u64 attr_type_mask) > +{ > + struct io_uring_attr_pi pi_attr; > + struct io_async_rw *io; > + int ret; > + > + if (copy_from_user(&pi_attr, u64_to_user_ptr(attr_ptr), > + sizeof(pi_attr))) > + return -EFAULT; > + > + if (pi_attr.rsvd) > + return -EINVAL; > + > + io = req->async_data; > + io->meta.flags = pi_attr.flags; > + io->meta.app_tag = pi_attr.app_tag; > + io->meta.seed = READ_ONCE(pi_attr.seed); Seems an unnecessary READ_ONCE slipped here > + ret = import_ubuf(ddir, u64_to_user_ptr(pi_attr.addr), > + pi_attr.len, &io->meta.iter); > + if (unlikely(ret < 0)) > + return ret; > + rw->kiocb.ki_flags |= IOCB_HAS_METADATA; > + io_meta_save_state(io); > + return ret; > +} ...
This adds a new io_uring interface to exchange additional integrity/pi metadata with read/write. Example program for using the interface is appended below [1]. The patchset is on top of block/for-next. Block path (direct IO) , NVMe and SCSI driver are modified to support this. Patch 1 is an enhancement patch. Patch 2 is required to make the bounce buffer copy back work correctly. Patch 3 to 5 are prep patches. Patch 6 adds the io_uring support. Patch 7 gives us unified interface for user and kernel generated integrity. Patch 8 adds support in SCSI and patch 9 in NVMe. Patch 10 adds the support for block direct IO. Changes since v9: https://lore.kernel.org/linux-block/20241114104517.51726-1-anuj20.g@samsung.com/ - pass PI attribute information via pointer (Pavel) - fix kernel bot warnings Changes since v8: https://lore.kernel.org/io-uring/20241106121842.5004-1-anuj20.g@samsung.com/ - add option of the pass the PI information from user space via a pointer (Pavel) Changes since v7: https://lore.kernel.org/io-uring/20241104140601.12239-1-anuj20.g@samsung.com/ - change the sign-off order (hch) - add a check for doing metadata completion handling only for async-io - change meta_type name to something more meaningful (hch, keith) - add detail description in io-uring patch (hch) Changes since v6: https://lore.kernel.org/linux-block/20241030180112.4635-1-joshi.k@samsung.com/ - io_uring changes (bring back meta_type, move PI to the end of SQE128) - Fix robot warnings Changes since v5: https://lore.kernel.org/linux-block/20241029162402.21400-1-anuj20.g@samsung.com/ - remove meta_type field from SQE (hch, keith) - remove __bitwise annotation (hch) - remove BIP_CTRL_NOCHECK from scsi (hch) Changes since v4: https://lore.kernel.org/linux-block/20241016112912.63542-1-anuj20.g@samsung.com/ - better variable names to describe bounce buffer copy back (hch) - move defintion of flags in the same patch introducing uio_meta (hch) - move uio_meta definition to include/linux/uio.h (hch) - bump seed size in uio_meta to 8 bytes (martin) - move flags definition to include/uapi/linux/fs.h (hch) - s/meta/metadata in commit description of io-uring (hch) - rearrange the meta fields in sqe for cleaner layout - partial submission case is not applicable as, we are only plumbing for async case - s/META_TYPE_INTEGRITY/META_TYPE_PI (hch, martin) - remove unlikely branching (hch) - Better formatting, misc cleanups, better commit descriptions, reordering commits(hch) Changes since v3: https://lore.kernel.org/linux-block/20240823103811.2421-1-anuj20.g@samsung.com/ - add reftag seed support (Martin) - fix incorrect formatting in uio_meta (hch) - s/IOCB_HAS_META/IOCB_HAS_METADATA (hch) - move integrity check flags to block layer header (hch) - add comments for BIP_CHECK_GUARD/REFTAG/APPTAG flags (hch) - remove bio_integrity check during completion if IOCB_HAS_METADATA is set (hch) - use goto label to get rid of duplicate error handling (hch) - add warn_on if trying to do sync io with iocb_has_metadata flag (hch) - remove check for disabling reftag remapping (hch) - remove BIP_INTEGRITY_USER flag (hch) - add comment for app_tag field introduced in bio_integrity_payload (hch) - pass request to nvme_set_app_tag function (hch) - right indentation at a place in scsi patch (hch) - move IOCB_HAS_METADATA to a separate fs patch (hch) Changes since v2: https://lore.kernel.org/linux-block/20240626100700.3629-1-anuj20.g@samsung.com/ - io_uring error handling styling (Gabriel) - add documented helper to get metadata bytes from data iter (hch) - during clone specify "what flags to clone" rather than "what not to clone" (hch) - Move uio_meta defination to bio-integrity.h (hch) - Rename apptag field to app_tag (hch) - Change datatype of flags field in uio_meta to bitwise (hch) - Don't introduce BIP_USER_CHK_FOO flags (hch, martin) - Driver should rely on block layer flags instead of seeing if it is user-passthrough (hch) - update the scsi code for handling user-meta (hch, martin) Changes since v1: https://lore.kernel.org/linux-block/20240425183943.6319-1-joshi.k@samsung.com/ - Do not use new opcode for meta, and also add the provision to introduce new meta types beyond integrity (Pavel) - Stuff IOCB_HAS_META check in need_complete_io (Jens) - Split meta handling in NVMe into a separate handler (Keith) - Add meta handling for __blkdev_direct_IO too (Keith) - Don't inherit BIP_COPY_USER flag for cloned bio's (Christoph) - Better commit descriptions (Christoph) Changes since RFC: - modify io_uring plumbing based on recent async handling state changes - fixes/enhancements to correctly handle the split for meta buffer - add flags to specify guard/reftag/apptag checks - add support to send apptag [1] #define _GNU_SOURCE #include <stdio.h> #include <stdlib.h> #include <string.h> #include <linux/fs.h> #include <linux/io_uring.h> #include <linux/types.h> #include "liburing.h" /* * write data/meta. read both. compare. send apptag too. * prerequisite: * protected xfer: format namespace with 4KB + 8b, pi_type = 1 * For testing reftag remapping on device-mapper, create a * device-mapper and run this program. Device mapper creation: * # echo 0 80 linear /dev/nvme0n1 0 > /tmp/table * # echo 80 160 linear /dev/nvme0n1 200 >> /tmp/table * # dmsetup create two /tmp/table * # ./a.out /dev/dm-0 */ #define DATA_LEN 4096 #define META_LEN 8 struct t10_pi_tuple { __be16 guard; __be16 apptag; __be32 reftag; }; int main(int argc, char *argv[]) { struct io_uring ring; struct io_uring_sqe *sqe = NULL; struct io_uring_cqe *cqe = NULL; void *wdb,*rdb; char wmb[META_LEN], rmb[META_LEN]; char *data_str = "data buffer"; int fd, ret, blksize; struct stat fstat; unsigned long long offset = DATA_LEN * 10; struct t10_pi_tuple *pi; struct io_uring_sqe_ext *sqe_ext; struct io_uring_attr w_pi, r_pi; if (argc != 2) { fprintf(stderr, "Usage: %s <block-device>", argv[0]); return 1; }; if (stat(argv[1], &fstat) == 0) { blksize = (int)fstat.st_blksize; } else { perror("stat"); return 1; } if (posix_memalign(&wdb, blksize, DATA_LEN)) { perror("posix_memalign failed"); return 1; } if (posix_memalign(&rdb, blksize, DATA_LEN)) { perror("posix_memalign failed"); return 1; } memset(wdb, 0, DATA_LEN); fd = open(argv[1], O_RDWR | O_DIRECT); if (fd < 0) { printf("Error in opening device\n"); return 0; } ret = io_uring_queue_init(8, &ring, 0); if (ret) { fprintf(stderr, "ring setup failed: %d\n", ret); return 1; } /* write data + meta-buffer to device */ sqe = io_uring_get_sqe(&ring); if (!sqe) { fprintf(stderr, "get sqe failed\n"); return 1; } io_uring_prep_write(sqe, fd, wdb, DATA_LEN, offset); sqe->attr_type_mask = ATTR_FLAG_PI; w_pi.attr_type = ATTR_TYPE_PI; w_pi.pi.addr = (__u64)wmb; w_pi.pi.len = META_LEN; /* flags to ask for guard/reftag/apptag*/ w_pi.pi.flags = IO_INTEGRITY_CHK_GUARD | IO_INTEGRITY_CHK_REFTAG | IO_INTEGRITY_CHK_APPTAG; w_pi.pi.app_tag = 0x1234; w_pi.pi.seed = 10; w_pi.pi.rsvd = 0; sqe->attr_ptr = (__u64)&w_pi; pi = (struct t10_pi_tuple *)wmb; pi->guard = 0; pi->reftag = 0x0A000000; pi->apptag = 0x3412; ret = io_uring_submit(&ring); if (ret <= 0) { fprintf(stderr, "sqe submit failed: %d\n", ret); return 1; } ret = io_uring_wait_cqe(&ring, &cqe); if (!cqe) { fprintf(stderr, "cqe is NULL :%d\n", ret); return 1; } if (cqe->res < 0) { fprintf(stderr, "write cqe failure: %d", cqe->res); return 1; } io_uring_cqe_seen(&ring, cqe); /* read data + meta-buffer back from device */ sqe = io_uring_get_sqe(&ring); if (!sqe) { fprintf(stderr, "get sqe failed\n"); return 1; } io_uring_prep_read(sqe, fd, rdb, DATA_LEN, offset); sqe->attr_type_mask = ATTR_FLAG_PI; r_pi.attr_type = ATTR_TYPE_PI; r_pi.pi.addr = (__u64)rmb; r_pi.pi.len = META_LEN; r_pi.pi.flags = IO_INTEGRITY_CHK_GUARD | IO_INTEGRITY_CHK_REFTAG | IO_INTEGRITY_CHK_APPTAG; r_pi.pi.app_tag = 0x1234; r_pi.pi.seed = 10; r_pi.pi.rsvd = 0; sqe->attr_ptr = (__u64)&r_pi; ret = io_uring_submit(&ring); if (ret <= 0) { fprintf(stderr, "sqe submit failed: %d\n", ret); return 1; } ret = io_uring_wait_cqe(&ring, &cqe); if (!cqe) { fprintf(stderr, "cqe is NULL :%d\n", ret); return 1; } if (cqe->res < 0) { fprintf(stderr, "read cqe failure: %d", cqe->res); return 1; } pi = (struct t10_pi_tuple *)rmb; if (pi->apptag != 0x3412) printf("Failure: apptag mismatch!\n"); if (pi->reftag != 0x0A000000) printf("Failure: reftag mismatch!\n"); io_uring_cqe_seen(&ring, cqe); pi = (struct t10_pi_tuple *)rmb; if (strncmp(wmb, rmb, META_LEN)) printf("Failure: meta mismatch!, wmb=%s, rmb=%s\n", wmb, rmb); if (strncmp(wdb, rdb, DATA_LEN)) printf("Failure: data mismatch!\n"); io_uring_queue_exit(&ring); free(rdb); free(wdb); return 0; } Anuj Gupta (7): block: define set of integrity flags to be inherited by cloned bip block: modify bio_integrity_map_user to accept iov_iter as argument fs, iov_iter: define meta io descriptor fs: introduce IOCB_HAS_METADATA for metadata io_uring: introduce attributes for read/write and PI support block: introduce BIP_CHECK_GUARD/REFTAG/APPTAG bip_flags scsi: add support for user-meta interface Christoph Hellwig (1): block: copy back bounce buffer to user-space correctly in case of split Kanchan Joshi (2): nvme: add support for passing on the application tag block: add support to pass user meta buffer block/bio-integrity.c | 84 ++++++++++++++++++++++++++++------- block/blk-integrity.c | 10 ++++- block/fops.c | 45 ++++++++++++++----- drivers/nvme/host/core.c | 21 +++++---- drivers/scsi/sd.c | 4 +- include/linux/bio-integrity.h | 25 ++++++++--- include/linux/fs.h | 1 + include/linux/uio.h | 9 ++++ include/uapi/linux/fs.h | 9 ++++ include/uapi/linux/io_uring.h | 31 +++++++++++++ io_uring/io_uring.c | 2 + io_uring/rw.c | 82 +++++++++++++++++++++++++++++++++- io_uring/rw.h | 14 +++++- 13 files changed, 291 insertions(+), 46 deletions(-)