[DRAFT] userspace LL/audio test PR#10558
Draft
kv2019i wants to merge 54 commits intothesofproject:mainfrom
Draft
[DRAFT] userspace LL/audio test PR#10558kv2019i wants to merge 54 commits intothesofproject:mainfrom
kv2019i wants to merge 54 commits intothesofproject:mainfrom
Conversation
Collaborator
Author
|
Running the pipeline_two_components_user test added in this PR, on a Intel PTL, looks like this: |
kv2019i
force-pushed
the
feat/userspace-ll-wip
branch
from
9cdb1be to
37b0412
Compare
Collaborator
Author
|
V2 snapshot pushed:
|
If CONFIG_SOF_BOOT_TEST_STANDALONE is set, ipc_init() is terminated early. This ensures SOF will not start to generate or respond to IPC messages that could potentially interfere with standalone test cases (some of which send and receive IPCs). The current implementation leaves the component list uninitialized and this can cause trouble to standalone tests that want to utilzie common IPC code to build messages. Fix this problem by executing more of ipc_init() also in the standalone mode. Signed-off-by: Kai Vehmanen <kai.vehmanen@linux.intel.com>
Add a method to add access to a component object to a particular thread. This is required as component object state includes kernel objects and to use these from user-space, access needs to be granted. Signed-off-by: Kai Vehmanen <kai.vehmanen@linux.intel.com>
In userspace LL builds, use a mutex to protect component connections. This code shoudl work for kernel builds, but at least now add new code under a ifdef to avoid the cost of additional mutex object. Signed-off-by: Kai Vehmanen <kai.vehmanen@linux.intel.com>
Add a built option to make sof_heap_allo/free available as system calls to user-space. Add a test case for the functions that runs in a user-space thread. Signed-off-by: Kai Vehmanen <kai.vehmanen@linux.intel.com>
Separate the state for LL scheduler memory into kernel and user accessible resources. The pointer to the LL heap must be accessible from user-space, so that user space can allocate memory and pass the heap pointer as argument. Signed-off-by: Kai Vehmanen <kai.vehmanen@linux.intel.com>
Turn the pdata->sem into a dynamic object in userspace LL builds. Keep statically allocated semaphore for kernel LL builds. Signed-off-by: Kai Vehmanen <kai.vehmanen@linux.intel.com>
Move the user-accessible heap pointer to common partition defined in userspace_helper.h. Signed-off-by: Kai Vehmanen <kai.vehmanen@linux.intel.com>
Add a heap parameter to DMA scatter-gather allocation interface. This makes it possible to control how allocations are done for the DMA buffers. Signed-off-by: Kai Vehmanen <kai.vehmanen@linux.intel.com>
Continue the work in commit 9567234 ("buffer: allocate on specific heap") and add ability to specify the heap to all buffer interface functions. Signed-off-by: Kai Vehmanen <kai.vehmanen@linux.intel.com>
Allow a non-null pointer at the end of the DMA transfer block list, if and only if it points to the first entry in the block list. The SOF DAI module sets the DMA transfers blocks like this and this change is required to use DAI module from user-space. Signed-off-by: Kai Vehmanen <kai.vehmanen@linux.intel.com>
The platform data descriptions need to be accessible to all threads. These are e.g. used when setting up host/DAI copiers and they need platform DMA properties. Signed-off-by: Kai Vehmanen <kai.vehmanen@linux.intel.com>
Use separate heaps depending whether host copier is run in user or kernel space. Signed-off-by: Kai Vehmanen <kai.vehmanen@linux.intel.com>
For historical reasons, host-zephyr has somewhat complicated code to manage the DMA channel instance information. When a DMA channel is allocated, a pointer to the system DMA channel table is acquired and some additional information is stored per channel. This is however redundant as the only piece of information actually needed is the channel index. Simplify the code by not storing the channel pointer anymore, but rather just store the channel index and use that in all calls to the DMA driver. Signed-off-by: Kai Vehmanen <kai.vehmanen@linux.intel.com>
Make sure we use the same heap to allocate DMA SG buffers as we use for other component resources. Signed-off-by: Kai Vehmanen <kai.vehmanen@linux.intel.com>
These interfaces are no longer used anywhere, so they can be safely removed. Signed-off-by: Kai Vehmanen <kai.vehmanen@linux.intel.com>
Force to use user-space LL heap in host_common_new() if CONFIG_SOF_USERSPACE_LL is set. Signed-off-by: Kai Vehmanen <kai.vehmanen@linux.intel.com>
Ensure component heap is correctly passed whenever memory is allocated in the component. This allows to run the component both in kernel and user space. Signed-off-by: Kai Vehmanen <kai.vehmanen@linux.intel.com>
The pcm_converter depends on a set of global function tables to set up correct conversion functions. These objects need to be made available to user-space threads, so that pcm_converter can be also run in user-space. No impact to kernel-space use of pcm_converter. Signed-off-by: Kai Vehmanen <kai.vehmanen@linux.intel.com>
The notifier feature is not available when running in user-space, so disable the notify calls in comp_buffer_free() in these builds. Signed-off-by: Kai Vehmanen <kai.vehmanen@linux.intel.com>
Convert all memory allocations to use the sof_heap_alloc() interface and pass the dai_data specific heap object. This makes dai-zephyr code compatible with use from user-space, but does not effect kernel space use. Signed-off-by: Kai Vehmanen <kai.vehmanen@linux.intel.com>
The dai_get()/dai_put() provide a helper to access DAI devices. When used in user-space, the wrapper struct should be created in user-space memory. Signed-off-by: Kai Vehmanen <kai.vehmanen@linux.intel.com>
For historical reasons, dai-zephyr has somewhat complicated code to manage the DMA channel instance information. When a DMA channel is allocated, a pointer to the system DMA channel table is acquired and some additional information is stored per channel. This is however redundant as the only piece of information actually needed is the channel index. Simplify the code by not storing the channel pointer anymore, but rather just store the channel index and use that in all calls to the DMA driver. Signed-off-by: Kai Vehmanen <kai.vehmanen@linux.intel.com>
The spinlock used to protect access to DAI properties can be converted to a mutex as this is only accessed from IPC and LL threads and both are normal Zephyr threads. As an additional benefit, use of mutex allows to run the dai-zephyr module in user-space. Signed-off-by: Kai Vehmanen <kai.vehmanen@linux.intel.com>
Modify code to allocate DAI properties object on stack and use dai_get_properties_copy(). This is required when DAI code is run in user-space and a syscall is needed to talk to the DAI driver. It's not possible to return a pointer to kernel memory, so instead data needs to be copied to caller stack. Signed-off-by: Kai Vehmanen <kai.vehmanen@linux.intel.com>
Add counterpart to zephyr_ll_task_alloc() to allow freeing the task with correct heap. Signed-off-by: Kai Vehmanen <kai.vehmanen@linux.intel.com>
Add function zephyr_ll_grant_access() to allow other threads to access the scheduler mutex. This is needed if work is submitted from other threads to the scheduler. Signed-off-by: Kai Vehmanen <kai.vehmanen@linux.intel.com>
Only double-map the LL resources if CONFIG_CACHE_HAS_MIRRORED_MEMORY_REGIONS is set. Signed-off-by: Kai Vehmanen <kai.vehmanen@linux.intel.com>
Ensure the scheduler objects and lists of schedulers are allocated such that they can be used with both kernel and user-space LL scheduler implementations. Signed-off-by: Kai Vehmanen <kai.vehmanen@linux.intel.com>
Multiple temporary limitations to limit user LL scheduler to only work on core 0. This is mostly related to privileged cpu_get_id() that is used in many places to check the id of the core code is running on. This is not available to user-space at the moment, so temporary measures are needed. Signed-off-by: Kai Vehmanen <kai.vehmanen@linux.intel.com>
Make the scheduler list available to all user-space threads. Signed-off-by: Kai Vehmanen <kai.vehmanen@linux.intel.com>
Don't use sof_cycle_get_64() if SOF built for user-space LL. Signed-off-by: Kai Vehmanen <kai.vehmanen@linux.intel.com>
Add two optional ops that allow the schedule.h user to get access to the thread context that will be used for scheduling. This is critical when the callbacks are run in user-space context and schedule.h client needs to grant access to objects like locks to the callback thread. Signed-off-by: Kai Vehmanen <kai.vehmanen@linux.intel.com>
Implement the optional scheduler_init_context() and scheduler_free_context() ops for user-space LL scheduler. Signed-off-by: Kai Vehmanen <kai.vehmanen@linux.intel.com>
Add new domain ops that are called from privileged context and are used to set up resources like threads and initialize other kernel objects. Signed-off-by: Kai Vehmanen <kai.vehmanen@linux.intel.com>
Implement the new domain_thread_init/free() ops for the Zephyr LL scheduler implementation. Move all privileged operations to these methods. After this change, the domain_register() and domain_unregister() are now safe to call from user-space context. Signed-off-by: Kai Vehmanen <kai.vehmanen@linux.intel.com>
Add a new test to userspace_ll set that takes a step towards running full audio pipelines in user-space. The test creates a pipeline with two components (IPC4 host and DAI copiers), does pipeline prepare, one copy cycle in prepared state and tears down the pipeline. One user-space thread is created to manage the pipelines. This would be equivalent to user IPC handler thread. Another user-space thread is created for the LL scheduler. This test has some limitation, but is a useful test point to test resource allocations in pipeline, component and module adapter code. The code adds a reference test case where the same flow is fully run in kernel space. Signed-off-by: Kai Vehmanen <kai.vehmanen@linux.intel.com>
Start a separate user thread to handle application IPC messages when SOF is built with CONFIG_SOF_USERSPACE_LL. This is a in-progress implementation that only handles one IPC message type and is thus not a full implementation. This does allow to proceed to test IPC user thread creation and the basic mechanism to handle messages. Signed-off-by: Kai Vehmanen <kai.vehmanen@linux.intel.com>
Disable the code to set up pipeline position registers using the sof_get() singleton instance. This design is not feasible when pipelnes are running in user-space and an alternative implementation is needed. This is not required in initial LL user testing as position reporting via pipeline register is not used in all configurations. Signed-off-by: Kai Vehmanen <kai.vehmanen@linux.intel.com>
Add a test that sends SOF_IPC4_GLB_CREATE_PIPELINE message via ipc_cmd() interface. This test can be used to test SOF when built with CONFIG_SOF_USERSPACE_LL. Handling of audio pipeline IPC messages (like CREATE_PIPELINE) will be routed to a user-space IPC thread. Signed-off-by: Kai Vehmanen <kai.vehmanen@linux.intel.com>
The logic in comp_buffer_free() to check the module heap reference count, and if it goes to zero, free the heap itself - this does not belong in the buffer free method. TODO: needs cleanup before submission. Signed-off-by: Kai Vehmanen <kai.vehmanen@linux.intel.com>
This is a set of temporary changes to audio code to remove calls to privileged interfaces that are not mandatory to run simple audio tests. These need proper solutions to be able to run all use-cases in user LL version. Signed-off-by: Kai Vehmanen <kai.vehmanen@linux.intel.com>
pipeline_schedule_triggered() cannot use irq_local_disable() and irq_local_enable() when running in user-space. Comment out this for now as, but this needs a better solution for multicore scenarios. Signed-off-by: Kai Vehmanen <kai.vehmanen@linux.intel.com>
kv2019i
force-pushed
the
feat/userspace-ll-wip
branch
from
37b0412 to
7317b18
Compare
Collaborator
Author
|
V3 snapshot pushed:
|
Collaborator
Author
|
Test output for the new test added in V3: |
kv2019i
commented
Mar 17, 2026
| /* works? yes */ | ||
| //return 0; | ||
|
|
||
| printk("ipc %p\n", ipc); |
Collaborator
Author
There was a problem hiding this comment.
Oops, these shouldn't be here. :)
kv2019i
commented
Mar 17, 2026
|
|
||
| /* create the pipeline */ | ||
| pipe = pipeline_new(NULL, pipe_desc->primary.r.instance_id, | ||
| pipe = pipeline_new(heap, pipe_desc->primary.r.instance_id, |
Collaborator
Author
There was a problem hiding this comment.
@lyakh @jsarha @lgirdwood This is where you'd plug in the vregions stuff to pass a separate heap to each pipe (based on topology description of its needs). In this series, as a placeholder, I use the zephyr_ll_user_heap() instead.
Collaborator
There was a problem hiding this comment.
right, maybe put a comment there for now to make re-discovery easier
lyakh
reviewed
Mar 18, 2026
Collaborator
lyakh
left a comment
lyakh
left a comment
There was a problem hiding this comment.
last reviewed commit so far "schedule: zephyr_ll: implement thread_init/free domain ops"
| #ifdef CONFIG_SOF_USERSPACE_LL | ||
| void comp_grant_access_to_thread(const struct comp_dev *dev, struct k_thread *th) | ||
| { | ||
| assert(dev->list_mutex); |
Collaborator
There was a problem hiding this comment.
description a bit confusing - this is only granting access to a mutex. Also list_mutex is only added to comp_dev in the next commit.
| } | ||
|
|
||
| stream_addr = rballoc_align(flags, size, align); | ||
| stream_addr = sof_heap_alloc(heap, flags, size, align); |
Collaborator
There was a problem hiding this comment.
the commit, that is mentioned in the commit message, only moved buffer context objects to particular heaps. This commit moves actual data buffers to them too, which is different and (arguably) more risky
| #define HDA_DMA_BUFFER_PERIOD_COUNT 4 | ||
|
|
||
| SHARED_DATA struct sof_dma dma[] = { | ||
| APP_TASK_DATA SHARED_DATA struct sof_dma dma[] = { |
Collaborator
There was a problem hiding this comment.
do I understand correctly, that this kind of userspace access makes that data writable to userspace?
| comp_err(dev, "requested channel %d is busy", hda_chan); | ||
| return -ENODEV; | ||
| } | ||
| hd->chan = &hd->dma->chan[channel]; |
Collaborator
There was a problem hiding this comment.
is this also not needed for the legacy mode? Also below
| uint64_t next_sync; | ||
| uint64_t period_in_cycles; | ||
| #endif | ||
| struct k_heap *heap; |
Collaborator
There was a problem hiding this comment.
wasn't this already referenced in the previous commit?
|
|
||
| k_spinlock_init(&dd->dai->lock); | ||
| #ifdef CONFIG_SOF_USERSPACE_LL | ||
| dd->dai->lock = k_object_alloc(K_OBJ_MUTEX); |
Collaborator
There was a problem hiding this comment.
check for NULL? Possibly in other locations too
| k_mutex_lock(dai->lock, K_FOREVER); | ||
| props = dai_get_properties(dai->dev, direction, stream_id); | ||
| hs_id = props->dma_hs_id; | ||
| ret = dai_get_properties_copy(dai->dev, direction, stream_id, &props); |
Collaborator
There was a problem hiding this comment.
I'm guessing this is made a syscall in one of the commits
| mod_heap = &mod_heap_user->heap; | ||
| } else { | ||
| #ifdef CONFIG_SOF_USERSPACE_LL | ||
| mod_heap = zephyr_ll_user_heap(); |
Collaborator
There was a problem hiding this comment.
looks good, but this else is entered under multiple conditions, might need to double-check
| .schedule_task_before = zephyr_ll_task_schedule_before, | ||
| .schedule_task_after = zephyr_ll_task_schedule_after, | ||
| .schedule_task_free = zephyr_ll_task_free, | ||
| .schedule_task_free = zephyr_ll_task_sched_free, |
Collaborator
There was a problem hiding this comment.
let's "spend" 3 more characters and make it ..._schedule_free()
| return -ENOMEM; | ||
| } | ||
| tr_err(&ll_tr, "Failed to allocate thread object for core %d", core); | ||
| dt->handler = NULL; |
lyakh
reviewed
Mar 18, 2026
| { | ||
| const struct sof_man_fw_desc *desc = basefw_vendor_get_manifest(); | ||
| const struct sof_man_module *mod; | ||
| uint32_t i; |
| uint32_t i; | ||
|
|
||
| if (!desc) | ||
| return -1; |
| return (int)i; | ||
| } | ||
|
|
||
| return -1; |
| union ipc4_connector_node_id node_id; | ||
| uint32_t dma_buffer_size; | ||
| uint32_t config_length; | ||
| } __packed __aligned(4); |
Collaborator
There was a problem hiding this comment.
does __aligned actually make sense in a type definition?
| pipe_msg.extension.dat = ipc_user->ipc_msg_ext; | ||
|
|
||
| /* Execute pipeline creation in user context */ | ||
| ipc_user->result = ipc_pipeline_new(ipc_user->ipc, (ipc_pipe_new *)&pipe_msg); |
Collaborator
There was a problem hiding this comment.
that's brave! ;-) I'd put a huge "TODO" here to make sure not to ship this by chance :-)
|
|
||
| /* create the pipeline */ | ||
| pipe = pipeline_new(NULL, pipe_desc->primary.r.instance_id, | ||
| pipe = pipeline_new(heap, pipe_desc->primary.r.instance_id, |
Collaborator
There was a problem hiding this comment.
right, maybe put a comment there for now to make re-discovery easier
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
SOF has recently gained ability to run DP (=preemptable audio tasks) in Zephyr user-space.
This PR is an early stage pull-request for changes to extend this capability to all of the audio pipeline code, and specifically the LL (low-latency) tasks.
This early stage as the design is not set in stone and the PR uses a number of short cuts in order to move (and tests) incrementally larger sets of audio functionality.