public final class MediaCodec extends Object
MediaCodec is generally used like this:
MediaCodec codec = MediaCodec.createDecoderByType(type); codec.configure(format, ...); codec.start(); ByteBuffer[] inputBuffers = codec.getInputBuffers(); ByteBuffer[] outputBuffers = codec.getOutputBuffers(); for (;;) { int inputBufferIndex = codec.dequeueInputBuffer(timeoutUs); if (inputBufferIndex >= 0) { // fill inputBuffers[inputBufferIndex] with valid data ... codec.queueInputBuffer(inputBufferIndex, ...); } int outputBufferIndex = codec.dequeueOutputBuffer(timeoutUs); if (outputBufferIndex >= 0) { // outputBuffer is ready to be processed or rendered. ... codec.releaseOutputBuffer(outputBufferIndex, ...); } else if (outputBufferIndex == MediaCodec.INFO_OUTPUT_BUFFERS_CHANGED) { outputBuffers = codec.getOutputBuffers(); } else if (outputBufferIndex == MediaCodec.INFO_OUTPUT_FORMAT_CHANGED) { // Subsequent data will conform to new format. MediaFormat format = codec.getOutputFormat(); ... } } codec.stop(); codec.release(); codec = null;Each codec maintains a number of input and output buffers that are referred to by index in API calls. The contents of these buffers is represented by the ByteBuffer[] arrays accessible through getInputBuffers() and getOutputBuffers(). After a successful call to
start()
the client "owns" neither
input nor output buffers, subsequent calls to dequeueInputBuffer(long)
and dequeueOutputBuffer(android.media.MediaCodec.BufferInfo, long)
then transfer ownership from the codec
to the client.The client is not required to resubmit/release buffers immediately to the codec, the sample code above simply does this for simplicity's sake.
Once the client has an input buffer available it can fill it with data
and submit it it to the codec via a call to queueInputBuffer(int, int, int, long, int)
.
The codec in turn will return an output buffer to the client in response
to dequeueOutputBuffer(android.media.MediaCodec.BufferInfo, long)
. After the output buffer has been processed
a call to releaseOutputBuffer(int, boolean)
will return it to the codec.
If a video surface has been provided in the call to configure(android.media.MediaFormat, android.view.Surface, android.media.MediaCrypto, int)
,
releaseOutputBuffer(int, boolean)
optionally allows rendering of the buffer
to the surface.
Input buffers (for decoders) and Output buffers (for encoders) contain encoded data according to the format's type. For video types this data is all the encoded data representing a single moment in time, for audio data this is slightly relaxed in that a buffer may contain multiple encoded frames of audio. In either case, buffers do not start and end on arbitrary byte boundaries, this is not a stream of bytes, it's a stream of access units.
Most formats also require the actual data to be prefixed by a number
of buffers containing setup data, or codec specific data, i.e. the
first few buffers submitted to the codec object after starting it must
be codec specific data marked as such using the flag BUFFER_FLAG_CODEC_CONFIG
in a call to queueInputBuffer(int, int, int, long, int)
.
Codec specific data included in the format passed to configure(android.media.MediaFormat, android.view.Surface, android.media.MediaCrypto, int)
(in ByteBuffer entries with keys "csd-0", "csd-1", ...) is automatically
submitted to the codec, this data MUST NOT be submitted explicitly by the
client.
Once the client reaches the end of the input data it signals the end of
the input stream by specifying a flag of BUFFER_FLAG_END_OF_STREAM
in the call to
queueInputBuffer(int, int, int, long, int)
. The codec will continue to return output buffers
until it eventually signals the end of the output stream by specifying
the same flag (BUFFER_FLAG_END_OF_STREAM
) on the BufferInfo returned in
dequeueOutputBuffer(android.media.MediaCodec.BufferInfo, long)
.
In order to start decoding data that's not adjacent to previously submitted
data (i.e. after a seek) it is necessary to flush()
the decoder.
Any input or output buffers the client may own at the point of the flush are
immediately revoked, i.e. after a call to flush()
the client does not
own any buffers anymore.
Note that the format of the data submitted after a flush must not change,
flush does not support format discontinuities,
for this a full stop(), configure(), start() cycle is necessary.
Modifier and Type | Class and Description |
---|---|
static class |
MediaCodec.BufferInfo
Per buffer metadata includes an offset and size specifying
the range of valid data in the associated codec buffer.
|
static class |
MediaCodec.CryptoException |
static class |
MediaCodec.CryptoInfo
Metadata describing the structure of a (at least partially) encrypted
input sample.
|
Modifier and Type | Field and Description |
---|---|
static int |
BUFFER_FLAG_CODEC_CONFIG
This indicated that the buffer marked as such contains codec
initialization / codec specific data instead of media data.
|
static int |
BUFFER_FLAG_END_OF_STREAM
This signals the end of stream, i.e.
|
static int |
BUFFER_FLAG_SYNC_FRAME
This indicates that the buffer marked as such contains the data
for a sync frame.
|
static int |
CONFIGURE_FLAG_ENCODE
If this codec is to be used as an encoder, pass this flag.
|
static int |
CRYPTO_MODE_AES_CTR |
static int |
CRYPTO_MODE_UNENCRYPTED |
static int |
INFO_OUTPUT_BUFFERS_CHANGED
The output buffers have changed, the client must refer to the new
set of output buffers returned by
getOutputBuffers() from
this point on. |
static int |
INFO_OUTPUT_FORMAT_CHANGED
The output format has changed, subsequent data will follow the new
format.
|
static int |
INFO_TRY_AGAIN_LATER
If a non-negative timeout had been specified in the call
to
dequeueOutputBuffer(android.media.MediaCodec.BufferInfo, long) , indicates that the call timed out. |
static int |
VIDEO_SCALING_MODE_SCALE_TO_FIT
The content is scaled to the surface dimensions
|
static int |
VIDEO_SCALING_MODE_SCALE_TO_FIT_WITH_CROPPING
The content is scaled, maintaining its aspect ratio, the whole
surface area is used, content may be cropped
|
Modifier and Type | Method and Description |
---|---|
void |
configure(MediaFormat format,
Surface surface,
MediaCrypto crypto,
int flags)
Configures a component.
|
static MediaCodec |
createByCodecName(String name)
If you know the exact name of the component you want to instantiate
use this method to instantiate it.
|
static MediaCodec |
createDecoderByType(String type)
Instantiate a decoder supporting input data of the given mime type.
|
static MediaCodec |
createEncoderByType(String type)
Instantiate an encoder supporting output data of the given mime type.
|
int |
dequeueInputBuffer(long timeoutUs)
Returns the index of an input buffer to be filled with valid data
or -1 if no such buffer is currently available.
|
int |
dequeueOutputBuffer(MediaCodec.BufferInfo info,
long timeoutUs)
Dequeue an output buffer, block at most "timeoutUs" microseconds.
|
protected void |
finalize()
Invoked when the garbage collector has detected that this instance is no longer reachable.
|
void |
flush()
Flush both input and output ports of the component, all indices
previously returned in calls to
dequeueInputBuffer(long) and
dequeueOutputBuffer(android.media.MediaCodec.BufferInfo, long) become invalid. |
ByteBuffer[] |
getInputBuffers()
Call this after start() returns.
|
ByteBuffer[] |
getOutputBuffers()
Call this after start() returns and whenever dequeueOutputBuffer
signals an output buffer change by returning
INFO_OUTPUT_BUFFERS_CHANGED |
MediaFormat |
getOutputFormat()
Call this after dequeueOutputBuffer signals a format change by returning
INFO_OUTPUT_FORMAT_CHANGED |
void |
queueInputBuffer(int index,
int offset,
int size,
long presentationTimeUs,
int flags)
After filling a range of the input buffer at the specified index
submit it to the component.
|
void |
queueSecureInputBuffer(int index,
int offset,
MediaCodec.CryptoInfo info,
long presentationTimeUs,
int flags)
Similar to
queueInputBuffer(int, int, int, long, int) but submits a buffer that is
potentially encrypted. |
void |
release()
Make sure you call this when you're done to free up any opened
component instance instead of relying on the garbage collector
to do this for you at some point in the future.
|
void |
releaseOutputBuffer(int index,
boolean render)
If you are done with a buffer, use this call to return the buffer to
the codec.
|
void |
setVideoScalingMode(int mode)
If a surface has been specified in a previous call to
configure(android.media.MediaFormat, android.view.Surface, android.media.MediaCrypto, int)
specifies the scaling mode to use. |
void |
start()
After successfully configuring the component, call start.
|
void |
stop()
Finish the decode/encode session, note that the codec instance
remains active and ready to be
start() ed again. |
public static final int BUFFER_FLAG_SYNC_FRAME
public static final int BUFFER_FLAG_CODEC_CONFIG
public static final int BUFFER_FLAG_END_OF_STREAM
flush()
follows.public static final int CONFIGURE_FLAG_ENCODE
public static final int CRYPTO_MODE_UNENCRYPTED
public static final int CRYPTO_MODE_AES_CTR
public static final int INFO_TRY_AGAIN_LATER
dequeueOutputBuffer(android.media.MediaCodec.BufferInfo, long)
, indicates that the call timed out.public static final int INFO_OUTPUT_FORMAT_CHANGED
getOutputFormat()
returns the new format.public static final int INFO_OUTPUT_BUFFERS_CHANGED
getOutputBuffers()
from
this point on.public static final int VIDEO_SCALING_MODE_SCALE_TO_FIT
public static final int VIDEO_SCALING_MODE_SCALE_TO_FIT_WITH_CROPPING
public static MediaCodec createDecoderByType(String type)
type
- The mime type of the input data.public static MediaCodec createEncoderByType(String type)
type
- The desired mime type of the output data.public static MediaCodec createByCodecName(String name)
MediaCodecList
name
- The name of the codec to be instantiated.protected void finalize()
Object
Note that objects that override finalize
are significantly more expensive than
objects that don't. Finalizers may be run a long time after the object is no longer
reachable, depending on memory pressure, so it's a bad idea to rely on them for cleanup.
Note also that finalizers are run on a single VM-wide finalizer thread,
so doing blocking work in a finalizer is a bad idea. A finalizer is usually only necessary
for a class that has a native peer and needs to call a native method to destroy that peer.
Even then, it's better to provide an explicit close
method (and implement
Closeable
), and insist that callers manually dispose of instances. This
works well for something like files, but less well for something like a BigInteger
where typical calling code would have to deal with lots of temporaries. Unfortunately,
code that creates lots of temporaries is the worst kind of code from the point of view of
the single finalizer thread.
If you must use finalizers, consider at least providing your own
ReferenceQueue
and having your own thread process that queue.
Unlike constructors, finalizers are not automatically chained. You are responsible for
calling super.finalize()
yourself.
Uncaught exceptions thrown by finalizers are ignored and do not terminate the finalizer thread. See Effective Java Item 7, "Avoid finalizers" for more.
public final void release()
public void configure(MediaFormat format, Surface surface, MediaCrypto crypto, int flags)
format
- The format of the input data (decoder) or the desired
format of the output data (encoder).surface
- Specify a surface on which to render the output of this
decoder.crypto
- Specify a crypto object to facilitate secure decryption
of the media data.flags
- Specify CONFIGURE_FLAG_ENCODE
to configure the
component as an encoder.public final void start()
public final void stop()
public final void flush()
dequeueInputBuffer(long)
and
dequeueOutputBuffer(android.media.MediaCodec.BufferInfo, long)
become invalid.public final void queueInputBuffer(int index, int offset, int size, long presentationTimeUs, int flags) throws MediaCodec.CryptoException
MediaExtractor
provides codec
specific data as part of
the returned track format in entries named "csd-0", "csd-1" ...
These buffers should be submitted using the flag BUFFER_FLAG_CODEC_CONFIG
.
To indicate that this is the final piece of input data (or rather that
no more input data follows unless the decoder is subsequently flushed)
specify the flag BUFFER_FLAG_END_OF_STREAM
.index
- The index of a client-owned input buffer previously returned
in a call to dequeueInputBuffer(long)
.offset
- The byte offset into the input buffer at which the data starts.size
- The number of bytes of valid input data.presentationTimeUs
- The time at which this buffer should be rendered.flags
- A bitmask of flags BUFFER_FLAG_SYNC_FRAME
,
BUFFER_FLAG_CODEC_CONFIG
or BUFFER_FLAG_END_OF_STREAM
.MediaCodec.CryptoException
- if a crypto object has been specified in
configure(android.media.MediaFormat, android.view.Surface, android.media.MediaCrypto, int)
public final void queueSecureInputBuffer(int index, int offset, MediaCodec.CryptoInfo info, long presentationTimeUs, int flags) throws MediaCodec.CryptoException
queueInputBuffer(int, int, int, long, int)
but submits a buffer that is
potentially encrypted.index
- The index of a client-owned input buffer previously returned
in a call to dequeueInputBuffer(long)
.offset
- The byte offset into the input buffer at which the data starts.info
- Metadata required to facilitate decryption, the object can be
reused immediately after this call returns.presentationTimeUs
- The time at which this buffer should be rendered.flags
- A bitmask of flags BUFFER_FLAG_SYNC_FRAME
,
BUFFER_FLAG_CODEC_CONFIG
or BUFFER_FLAG_END_OF_STREAM
.MediaCodec.CryptoException
public final int dequeueInputBuffer(long timeoutUs)
timeoutUs
- The timeout in microseconds, a negative timeout indicates "infinite".public final int dequeueOutputBuffer(MediaCodec.BufferInfo info, long timeoutUs)
info
- Will be filled with buffer meta data.timeoutUs
- The timeout in microseconds, a negative timeout indicates "infinite".public final void releaseOutputBuffer(int index, boolean render)
index
- The index of a client-owned output buffer previously returned
in a call to dequeueOutputBuffer(android.media.MediaCodec.BufferInfo, long)
.render
- If a valid surface was specified when configuring the codec,
passing true renders this output buffer to the surface.public final MediaFormat getOutputFormat()
INFO_OUTPUT_FORMAT_CHANGED
public ByteBuffer[] getInputBuffers()
public ByteBuffer[] getOutputBuffers()
INFO_OUTPUT_BUFFERS_CHANGED
public final void setVideoScalingMode(int mode)
configure(android.media.MediaFormat, android.view.Surface, android.media.MediaCrypto, int)
specifies the scaling mode to use. The default is "scale to fit".