Audio Sink

The Audio Sink HAL provides a standardized interface for managing and controlling the playback of non-tunnelled Pulse Code Modulation (PCM) audio streams. It acts as an abstraction layer between the audio processing hardware and higher-level software components, such as the RDK middleware. The primary function of the Audio Sink HAL is to receive PCM audio frame buffers, deliver them to the audio mixer, and expose controls for managing audio mixing levels, including volume, mute state, and fade effects.

This HAL is designed to handle PCM audio data that has been decoded or generated by software components within the system. It specifically excludes support for tunnelled audio (where the encoded audio stream is passed directly to the hardware for decoding) and audio passthrough modes (where the audio stream bypasses software processing altogether). These scenarios are handled differently within the audio pipeline.

The RDK middleware, specifically the GStreamer pipeline, includes a dedicated RDK Audio Sink element that is designed to interact seamlessly with the Audio Sink HAL. This element handles the task of receiving audio data from the pipeline, formatting it into appropriate frame buffers, and delivering it to the HAL for playback. It also manages the communication with the HAL to control mixing parameters and handle audio events.

The Audio Sink HAL plays a critical role in the overall audio playback architecture by:

• Abstracting Hardware: It provides a consistent interface to the audio hardware, shielding higher-level software from hardware-specific details. This allows for easier porting of the middleware to different platforms with varying audio hardware.
• Managing Resources: The HAL can manage multiple audio sink instances, each representing a distinct audio output or stream. This allows for concurrent playback of multiple audio sources.
• Controlling Mixing: It provides controls for volume, mute, and fading, enabling applications to adjust the audio output to their requirements.
• Handling Buffers: The HAL is responsible for receiving audio frame buffers, ensuring they are in the correct format, and delivering them to the audio mixer for processing.
• Event Handling: The HAL can generate events related to audio playback, such as changes in state, errors, or end-of-stream notifications, allowing the middleware to react accordingly.

The interaction between the RDK GStreamer Audio Sink element and the Audio Sink HAL is crucial for efficient and robust audio playback. The element handles the higher-level stream management, while the HAL takes care of the low-level hardware interaction and mixing control. This separation of concerns simplifies the design and maintenance of the audio subsystem.

References

Info

Interface Definition	audiosink/current
Interface Version	current
API Documentation	TBD - Doxygen
HAL Interface Type	AIDL and Binder
VTS Tests	TBC
Reference Implementation - vComponent	TBD

Related Pages

Related Pages

• Audio Decoder
• AV Buffer
• AV Clock
• Session State Management

Implementation Requirements

#	Requirement	Comments
HAL.AUDIOSINK.1	Starting and stopping audio streams shall never produce an audible click or pop artefact due to the audio waveform where the audio streaming was started or stopped.
HAL.AUDIOSINK.2	If any audio decoders support secure audio path processing (SAP) then the audio sink shall be capable of processing secure input buffers to maintain SAP.
HAL.AUDIOSINK.3	The default volume level for an audio sink session shall be 1.0 (full volume) and unmuted.
HAL.AUDIOSINK.4	The default reference level for an audio sink session shall be -31dB.
HAL.AUDIOSINK.5	If a client process exits, the Audio Sink server shall automatically stop and close any Audio Sink instance controlled by that client.

Interface Definition

Interface Definition File	Description
IAudioSinkManager.aidl	Audio Sink Manager HAL interface which provides access to the IAudioSink resource instances.
IAudioSink.aidl	Audio Sink interface for a single audio sink resource instance.
IAudioSinkController.aidl	Controller interface for an IAudioSink resource instance.
IAudioSinkControllerListener.aidl	Listener callbacks interface to clients from an IAudioSinkController.
IAudioSinkEventListener.aidl	Listener callbacks interface to clients from an IAudioSink.
Capabilities.aidl	Parcelable describing the capabilities of an IAudioSink resource instance.
ErrorCode.aidl	Enum list of audio sink error codes.
PlatformCapabilities.aidl	Parcelable describing the capabilities of the platform audio.
Property.aidl	Enum list of audio sink properties.
Volume.aidl	Parcelable for defining a volume and mute state.
VolumeRamp.aidl	Enum list of volume ramp types.

Initialization

The systemd hal-audios_sink_manager.service unit file is provided by the vendor layer to start the service and should include Wants or Requires directives to start any platform driver services it depends upon.

The Audio Sink Manager service depends on the Service Manager to register itself as a service.

Upon starting, the service shall register the IAudioSinkManager interface with the Service Manager using the string IAudioSinkManager.serviceName and immediately become operational.

Product Customization

The IAudioSinkManager.getAudioSinkIds() should return an array of IAudioSink.Id parcelables to uniquely represent all of the audio sink resources supported by the vendor layer. Typically, the ID value starts at 0 for the first audio sink and increments by 1 for each additional audio sink.

The Capabilities parcelable returned by the IAudioSink.getCapabilities() function lists all capabilities supported by this audio sink instance.

An audio sink instance can only operate on one audio stream in an open session. Concurrent audio streams require multiple audio sink instances to be opened.

System Context

The Audio Sink HAL can provide functionality to multiple clients.

Typically an RDK middleware GStreamer audio sink element will work with a single IAudioSink instance and pass it PCM audio in AV Buffer handles for mixing.

The RDK middleware resource management system will examine the number of audio sink resources and their capabilities, so they can be allocated to streaming sessions.

flowchart TD
    RDKClientComponent("RDKClientComponent")
    subgraph Listeners["Listeners"]
        IAudioSinkEventListener("IAudioSinkEventListener")
        IAudioSinkControllerListener("IAudioSinkControllerListener")
    end
    subgraph IAudioSinkHAL["Audio Decoder HAL"]
        IAudioSinkManager("IAudioSinkManager <br>(Service)")
        IAudioSink("IAudioSink <br>(Instance)")
        IAudioSinkController("IAudioSinkController <br>(Instance)")
    end
    subgraph OutputComponents["Output"]
        platformMixer("Platform Integrated Mixer")
    end
    RDKClientComponent -- createAudioPool() <br> alloc() <br> free() <br> destroyPool() --> IAVBuffer(IAVBuffer)
    AudioFramePool("Audio Frame Pool")
    IAVBuffer -- free() --> AudioFramePool
    RDKClientComponent -- getIAudioSinkIds() <br> getIAudioSink() --> IAudioSinkManager
    RDKClientComponent -- getCapabilities() <br> getProperty() <br> getState() <br> open() <br> close() <br> registerEventListener() <br> unregisterEventListener() --> IAudioSink
    RDKClientComponent -- setAudioDecoder() <br> getAudioDecoder() <br> start() <br> stop() <br> queueAudioFrame() <br> flush() <br> getVolume() <br> setVolume() <br> setVolumeRamp() --> IAudioSinkController
    IAudioSinkManager --> IAudioSink --> IAudioSinkController
    IAudioSink -- onStateChanged() <br> onFristFrameRendered() <br> onEndOfStream() <br> onAudioUnderflow() <br> onAudioResumed() <br> onFlushComplete()--> IAudioSinkEventListener
    IAudioSinkController -- onStateChanged() <br> onFristFrameRendered() <br> onEndOfStream() <br> onAudioUnderflow() <br> onAudioResumed() <br> onFlushComplete()--> IAudioSinkControllerListener
    IAudioSinkEventListener --> RDKClientComponent
    IAudioSinkControllerListener --> RDKClientComponent
    IAudioSinkManager -- free() --> IAVBuffer
    IAudioSinkController -. audio -.-> platformMixer

    classDef background fill:#121212,stroke:none,color:#E0E0E0;
    classDef blue fill:#1565C0,stroke:#E0E0E0,stroke-width:2px,color:#E0E0E0;
    classDef lightGrey fill:#616161,stroke:#E0E0E0,stroke-width:2px,color:#FFFFFF;
    classDef wheat fill:#FFB74D,stroke:#424242,stroke-width:2px,color:#000000;
    classDef green fill:#4CAF50,stroke:#E0E0E0,stroke-width:2px,color:#FFFFFF;
    classDef default fill:#1E1E1E,stroke:#E0E0E0,stroke-width:1px,color:#E0E0E0;

    RDKClientComponent:::blue
    IAudioSinkManager:::wheat
    IAudioSinkController:::wheat
    IAudioSink:::wheat
    IAVBuffer:::green
    IAudioSinkControllerListener:::wheat
    IAudioSinkEventListener:::wheat
    AudioFramePool:::green
    platformMixer:::green

Resource Management

The IAudioSinkManager provides access to one or more IAudioSink sub-interfaces which each represent an audio sink resource instance offered by the platform.

Each IAudioSink resource instance is assigned a unique integer ID, which is used in the IAudioSink.Id.value and can be read from RESOURCE_ID using the IAudioSink.getProperty() function.

To use an IAudioSink resource instance it must be opened by a client, which returns an IAudioSinkController sub-interface to access buffer queuing and additional state controls.

Important

Any number of clients can access the IAudioSinkManager service and get access to the IAudioSink sub-interfaces, but only 1 client can open() an IAudioSink and access its IAudioSinkController sub-interface.

The diagram below shows the relationship between the interfaces and resource instances.

graph LR

    %% --- Encapsulating Everything Inside "Audio Decoder HAL" ---
    IAudioSinkManager("IAudioSinkManager")

    %% --- Audio Decoder Manager Service Spawns Instances ---
    IAudioSinkManager --> ADI1("IAudioSink <br> ID = 0")
    IAudioSinkManager --> ADI2("IAudioSink <br> ID = 1")
    IAudioSinkManager --> ADI3("IAudioSink <br> ID = 2")

    %% --- Each Instance Has a Controller ---
    ADI1 --> ADIC1("IAudioSinkController")
    ADI2 --> ADIC2("IAudioSinkController")
    ADI3 --> ADIC3("IAudioSinkController")

    %% --- High Contrast Styling (Rounded Box Simulation) ---
    classDef background fill:#121212,stroke:none,color:#E0E0E0;
    classDef manager fill:#388E3C,stroke:#1B5E20,stroke-width:2px,color:#FFFFFF;
    classDef instance1 fill:#FFC107,stroke:#FF8F00,stroke-width:2px,color:#000000;
    classDef instance2 fill:#FF9800,stroke:#E65100,stroke-width:2px,color:#000000;
    classDef instance3 fill:#F44336,stroke:#B71C1C,stroke-width:2px,color:#FFFFFF;
    classDef controller fill:#00ACC1,stroke:#006064,stroke-width:2px,color:#000000;

    %% --- Apply Colors ---
    class IAudioSinkManager manager;
    class ADI1 instance1;
    class ADI2 instance2;
    class ADI3 instance3;
    class ADIC1 instance1;
    class ADIC2 instance2;
    class ADIC3 instance3;

    %% --- Consistent Link Colors Per Instance ---
    %% Yellow for Instance 0
    linkStyle 0,3 stroke:#AA8800,stroke-width:2px;
    %% Orange for Instance 1
    linkStyle 1,4 stroke:#CC5500,stroke-width:2px;
    %% Red for Instance 2
    linkStyle 2,5 stroke:#CC2200,stroke-width:2px;

Audio Buffers

Audio buffers entering the Audio Sink shall be delivered as AV Buffer handles (allocated from either an AV Buffer audio pool or the audio frame pool) with a presentation timestamp and metadata describing the audio frame through the IAudioSinkController.queueAudioFrame() function.

An AV Buffer audio pool would be used for PCM data which has come from system memory (e.g. PCM sound clip) or from a soft audio decoder. In this case the audio pool is created against IAudioDecoder.Id.UNDEFINED.

The audio data must be in the PCM audio format and sample rate, as reported in PlatformCapabilities returned from the IAudioSinkManager.getPlatformCapabilities() function.

Once the data in an audio frame buffer has been fully passed to or processed by the mixer, the Audio Sink shall free the handle by calling IAVBuffer.free().

Input Buffer Back-Pressure

IAudioSinkController.queueAudioFrame() returns false when the internal frame buffer queue is full. Buffer ownership remains with the caller and the frame must be retained for re-submission.

To avoid wasted binder transactions, the client SHOULD wait for IAudioSinkControllerListener.onFrameBufferAvailable() before calling queueAudioFrame() again. The callback fires exactly once per back-pressure episode: when the internal queue transitions from full to has-space. If the client continues to call queueAudioFrame() during back-pressure (receiving false repeatedly), only one callback is delivered per transition. It is not fired in steady-state operation.

Continuing to call queueAudioFrame() while the queue is full is permitted but will return false repeatedly until space is available.

Secure Audio Processing

If any audio decoder supports SAP in non-tunnelled mode then the Audio Sink HAL must also support SAP to be able to process secure AV buffers of decoded PCM data, otherwise SAP support is optional.

Clear PCM Audio Playback

PCM stream data can originate in the RDK media pipeline from multiple sources; from an application, from the RDK middleware or from a software audio decoder. In these cases the PCM data is passed directly to the Audio Sink HAL.

Clear PCM audio is copied into a non-secure AV Buffer and is routed to the IAudioSinkController where it is queued for mixing.

Tunnelled Audio & Passthrough Mode

The Audio Sink HAL cannot handle audio frame buffers when tunnelled audio is exclusively in use or when audio passthrough mode is enabled.

In these cases an Audio Decoder does not return audio frame buffer handles that can be passed to the Audio Sink.

The Audio Sink HAL is still used to control the audio stream volume, mute and volume ramping.

End of Stream Signalling

EOS is carried on the framework metadata parcelable. The RDK middleware client signals EOS to the Audio Sink by setting FrameMetadata.endOfStream = true on the final frame queued via IAudioSinkController.queueAudioFrame(). The buffer MUST be a valid final audio frame - there is no EOS-only marker form.

When FrameMetadata.endOfStream = true, the other fields of FrameMetadata describe the final frame as normal — there is no separate EOS-only marker form.

For non-tunnelled audio decoded by the Audio Decoder, the Audio Decoder delivers FrameMetadata.endOfStream = true on its final onFrameOutput() callback; the RDK middleware client forwards that frame and metadata to the Audio Sink via queueAudioFrame().

For PCM audio (no Audio Decoder in the path), the RDK middleware client generates the EOS metadata itself and queues it on the final frame.

All audio frame buffers queued up in the Audio Sink continue to be fed into the audio mixer in the usual way. After the final frame has been completely passed to the mixer, the sink fires IAudioSinkControllerListener.onEndOfStream() exactly once. Subsequent calls to queueAudioFrame() raise EX_ILLEGAL_STATE until the sink is flushed or stopped and restarted.

Audio Sink States

The Audio Sink HAL follows the standard Session State Management paradigm.

When an Audio Sink session enters a FLUSHING or STOPPING transitory state it shall free any AV buffers it is holding.

The sequence diagram below shows the behavior of the callbacks.

sequenceDiagram
    box rgb(30,136,229) RDK Audio Sink
        participant Client as RDK Client
        participant IAudioSinkEventListener
        participant IAudioSinkControllerListener
    end
    box rgb(249,168,37) Audio Sink Server
        participant ADC as IAudioSink
        participant Controller as IAudioSinkController
    end
    box rgb(67,160,71) Audio AV Buffer
        participant IAVBuffer as IAVBuffer
    end

    Client->>ADC: registerEventListener(IAudioSinkEventListener)

    Note over ADC: open() transitions from CLOSED -> OPENING -> READY

    Client->>ADC: open(IAudioSinkControllerListener)
    ADC-->>IAudioSinkEventListener: onStateChanged(CLOSED -> OPENING)
    ADC->>Controller: new
    ADC-->>IAudioSinkEventListener: onStateChanged(OPENING -> READY)
    ADC-->>Client: IAudioSinkController
    Client->>Controller: attachClock(clockId)

    Note over ADC: start() transitions from READY -> STARTING -> STARTED

    Client->>Controller: start()
    ADC-->>IAudioSinkEventListener: onStateChanged(READY -> STARTING)
    ADC-->>IAudioSinkEventListener: onStateChanged(STARTING -> STARTED)

    Note over Client: Client can now queue\n audio frame buffers

    Client->>Controller: queueAudioFrame(pts, bufferHandle=1000, metadata)
    Client->>Controller: queueAudioFrame(pts, bufferHandle=1001, metadata)
    Controller->>IAVBuffer: free(bufferHandle=1000)

    Note over ADC: flush() transitions from STARTED -> FLUSHING -> STARTED

    Client->>Controller: flush()
    ADC-->>IAudioSinkEventListener: onStateChanged(STARTED -> FLUSHING)
    Controller->>IAVBuffer: free(bufferHandle=1001)
    ADC-->>IAudioSinkEventListener: onStateChanged(FLUSHING -> STARTED)

    Client->>Controller: queueAudioFrame(pts, bufferHandle=1002, metadata)

    Note over ADC: stop() transitions from STARTED -> STOPPING -> READY

    Client->>Controller: stop()
    ADC-->>IAudioSinkEventListener: onStateChanged(STARTED -> STOPPING)
    Controller->>IAVBuffer: free(bufferHandle=1002)
    ADC-->>IAudioSinkEventListener: onStateChanged(STOPPING -> READY)

    Note over ADC: close() transitions from READY -> CLOSING -> CLOSED

    Client->>ADC: close()
    ADC-->>IAudioSinkEventListener: onStateChanged(READY -> CLOSING)
    ADC->>Controller: delete
    ADC-->>IAudioSinkEventListener: onStateChanged(CLOSING -> CLOSED)

    Client->>ADC: unregisterEventListener(IAudioSinkEventListener)