FFmpeg: your powerful video helper / audio in your application

This article was published as part of the Data Science Blogathon

The name is inspired by MPEG, que significa Moving Picture Expert Group. FF stands for Fast Forward. The MPEG group is the alliance of working groups behind the establishment of many of today's essential video standards., including MP4 format, with which we are all familiar.

Results

1. What is FFmpeg?
2. Why do people use FFmpeg?
3. How to configure FFmpeg in Windows, Ubuntu and macOS.
4. Libraries are available from FFmpeg for media manipulations.
5. Tools that FFmpeg comes with for command line applications.
6. Usage and examples for FFprobe and FFplay.

Why do people use FFmpeg?

It is often referred to as the Swiss army knife of media transcoding or streaming.. We can use FFmpeg to perform many functions. The code is written in C language and optimized for better performance. Its commands are simple to execute. Once you are familiar with the concepts, it is very flexible for all the filters and options that you can use to suit your needs.

FFmpeg supports a wide range of codes, formats, devices and protocols, making it an ideal choice as a transcoding engine. Unlike many discontinued projects, is still in active development for more than 20 years. There is a large community of developers, users and contributors constantly developing new features and fixes.
FFmpeg has been used in central processing for video platforms such as YouTube and iTunes. Most of us use a media player like VLC to play video files. VLC uses FFmpeg libraries as its core. Some video editors and mobile apps also use FFmpeg under the hood.

Setting up FFmpeg on Windows, Ubuntu and macOS

There are a few things to know before setting up FFmpeg. There is no official build for any operating system. But it is an open source project, which you can get from the FFmpeg source code on the internet. Therefore, you can choose to create it yourself from source code. You can download a snapshot of the current source code here or make a payment from your git repository and follow the instructions to compile it on your machine. This can be a bit complex and time consuming and may require you to understand and choose some configuration options. It may also be necessary to install additional tools and compilers at the beginning. Then, unless you have a good reason to build it yourself, you may not want to go this route.

Instead, you can choose to download and install a pre-compiled FFmpeg package for your operating system. There are two ways to configure these precompiled binaries. If you have a package manager like APT on Ubuntu and Homebrew on macOS, it is easier to use as it takes care of downloading FFmpeg and its dependencies.

Let's keep things as simple as possible and choose the simplest way to start. The procedures below show how you can configure using precompiled packages.

Mac OS

On macOS, the easiest way to install FFmpeg is through Homebrew. Make sure you have installed Homebrew on your macOS, or you can continue this tutorial on how to install Homebrew.
We are using macOS 11.4 in this case.

• Open Terminal and type brew — version . You should see the prep version printed in Terminal if Homebrew installed successfully.

• For macOS in x86 architecture, use commandbrew install ffmpeg while for M1 please use the command aarch arm64 brew install ffmpeg .

• After the installation is complete, you can type ffmpeg -version in Terminal. If the FFmpeg version is printed, it means that FFmpeg installed successfully.

Windows

Unlike APT on Ubuntu and Homebrew on macOS, there is no known package manager in Windows, that you can use to quickly install FFmpeg. For Windows, you need to disable it and configure it manually.

There is 4 Windows FFmpeg build variants available that can be found here:

1. git full – built from a master branch with a large set of libraries.
2. git essentials – built from a master branch with commonly used libraries.
3. release full – built from the latest branch of the release with a large set of libraries.
4. release essentials – built from the latest branch of the release with commonly used libraries.

We will use FFmpeg release essentials in our case.

• Download the basics of the FFmpeg version here.
• Extract the files to a place from where you will use them in the future. In my case, I put them on my E Drive.

• Within compartment The folder contains the executables we want.

• Add compartment folder a VariableIn statistics and mathematics, a "variable" is a symbol that represents a value that can change or vary. There are different types of variables, and qualitative, that describe non-numerical characteristics, and quantitative, representing numerical quantities. Variables are fundamental in experiments and studies, since they allow the analysis of relationships and patterns between different elements, facilitating the understanding of complex phenomena.... Windows environment path so we can run this tool without having to specify the full path every time. Look for Environment Variables in the search bar and click Environment Variables…

• Click on Path in User variable, click New, and paste the full path of the compartment file.

• Opened Command prompt and type ffmpeg -version . If you see that the version of FFmpeg is printed, it means that FFmpeg installed successfully. We can also check ffprob -version Y ffplay -version .

FFmpeg Libraries

FFmpeg has several valuable libraries that you can use directly from your application code. Each library contains different functions related to a particular area.

Some of the famous libraries are:

1. libavcodec: contains all encoders and decoders that FFmpeg supports.
2. libavformat: has all muxers and demuxers to work with various container formats.
3. libavfilter: It consists of many filters that you can use to modify the audio or video according to the requirements.
4. libavdevice: supports several different input and output devices.
5. libavutil: aids portable multimedia programming.
6. libswscale: performs highly optimized image scaling and color space and pixel format conversion operations.
7. libswresample: performs resampling operations, highly optimized audio sample format remixing and conversion.

FFmpeg tools

In addition to libraries that libraries can use from other applications, the FFmpeg package contains some tools on its own.

Some of the tools are:

1. FFmpeg: main transcoding engine. Often invoked from a command line or other process.
2. ffplay: a minimal tool for playing audio or video.
3. ffprobe: quick media inspection to extract valuable information, like how many streams are there in the media, the frame rate of a video, etc.

Examples and use of FFprobe

Use

ffprobe [OPTIONS] [INPUT_FILE]

• man ffprobe– See the manual.
• ffprobe -h– Sample help included.
• -v error– Hide records unless there is an error.
• -show_format– Show video metadata formatted using Tag.
• show_streams– Shows all streams in the archive (both video and audio).
• -print_format —Print metadata in a specific format (CSV / XML / JSONJSON, o JavaScript Object Notation, It is a lightweight data exchange format that is easy for humans to read and write, and easy for machines to analyze and generate. It is commonly used in web applications to send and receive information between a server and a client. Its structure is based on key-value pairs, making it versatile and widely adopted in software development..).
• -select_streams
• -show_entries stream=- Show only the specific entity and its nested dependencies.
• -show_entries stream= default-noprint_wrappers=1- Show only the specific entity.

You can learn more about using FFprob here.

Basic examples

$ ffprobe -v error -show_format -show_streams input.mp4

Output:
[STREAM]
index=0
codec_name=h264
codec_long_name=H.264 / Stroke / MPEG-4 AVC / MPEG-4 part 10
profile=High
codec_type=video
codec_time_base=1/50
codec_tag_string=avc1
codec_tag=0x31637661
width=320
height=240
has_b_frames=2
sample_aspect_ratio=1:1
display_aspect_ratio=4:3
pix_fmt=yuv420p
level=13
color_range=N/A
color_space=unknown
color_transfer=unknown
color_primaries=unknown
chroma_location=left
timecode=N/A
refs=4
is_avc=1
nal_length_size=4
id=N/A
r_frame_rate=25/1
avg_frame_rate=25/1
time_base=1/12800
start_pts=0
start_time=0.000000
duration_ts=384000
duration=30.000000
bit_rate=34761
max_bit_rate=N/A
bits_per_raw_sample=8
nb_frames=750
nb_read_frames=N/A
nb_read_packets=N/A
DISPOSITION:default=1
DISPOSITION:dub=0
DISPOSITION:original=0
DISPOSITION:comment = 0
DISPOSITION:lyrics=0
DISPOSITION:karaoke=0
DISPOSITION:forced=0
DISPOSITION:hearing_impaired=0
DISPOSITION:visual_impaired=0
DISPOSITION:clean_effects=0
DISPOSITION:attached_pic=0
TAG:language = and
DAY:handler_name=VideoHandler
[/STREAM]
[STREAM]
index=1
codec_name=aac
codec_long_name=AAC (Advanced Audio Coding)
profile=LC
codec_type=audio
codec_time_base=1/44100
codec_tag_string=mp4a
codec_tag=0x6134706d
sample_fmt=fltp
sample_rate=44100
channels=1
channel_layout=mono
bits_per_sample=0
id=N/A
r_frame_rate=0/0
avg_frame_rate=0/0
time_base=1/44100
start_pts=-1024
start_time=-0.023220
duration_ts=1324024
duration=30.023220
bit_rate=56517
max_bit_rate=N/A
bits_per_raw_sample=N/A
nb_frames=1293
nb_read_frames=N/A
nb_read_packets=N/A
DISPOSITION:default=1
DISPOSITION:dub=0
DISPOSITION:original=0
DISPOSITION:comment = 0
DISPOSITION:lyrics=0
DISPOSITION:karaoke=0
DISPOSITION:forced=0
DISPOSITION:hearing_impaired=0
DISPOSITION:visual_impaired=0
DISPOSITION:clean_effects=0
DISPOSITION:attached_pic=0
TAG:language = and
DAY:handler_name=SoundHandler
[/STREAM]
[FORMAT]
filename=input.mp4
nb_streams=2
nb_programs=0
format_name=mov,mp4,m4a,3gp,3g2,mj2
format_long_name=QuickTime / MOV
start_time=-0.023220
duration=30.024000
size=368644
bit_rate=98226
probe_score=100
TAG:major_brand=isom
TAG:minor_version=512
TAG:compatible_brands=isomiso2avc1mp41
TAG:title=FFprobe Tips
TAG:encoder=Lavf56.15.101
[/FORMAT]

• From the example above, if you just want duration=30.024000 .

$ ffprobe -v error -show_entries format=duration -of default=noprint_wrappers=1 input.mp4

Output: size=368644

• If you only want the value without key.

$ ffprobe -v error -show_entries format=size -of default=noprint_wrappers=1:nokey=1 input.mp4

Output: 368644

• Getting width and height (resolutionThe "resolution" refers to the ability to make firm decisions and meet set goals. In personal and professional contexts, It involves defining clear goals and developing an action plan to achieve them. Resolution is critical to personal growth and success in various areas of life, as it allows you to overcome obstacles and keep your focus on what really matters....) Of video.

$ ffprobe -v error -select_streams v:0 -show_entries stream=height,width -of csv=s=x:p=0 input.mp4

Output: 1280x720

Use and examples of FFplay

Use

ffplay [OPTIONS] [INPUT_FILE]

• man ffplay- See the manual.
• ffplay -h- Sample help included.
• ffplay - Play video in full screen.
• -x -y - Play video with specific size of height and width pop-ups in the center. It will automatically fill the extra space between the video area and the window area if the aspect ratio of the popup window and the video are different.
• -noborder- Hide the title bar while playing videos in pop-up windows.
• -[top/left/bottom/right] - Make popup at specific coordinates. By default, will appear in the center of the screen.

You can learn more about using FFplay here.

Basic examples

• Play a video file called video.mp4 in full screen.

$ ffplay video.mp4

• Play a video file in a pop-up window with sizes of 600 widths, 600 tall and without title bar.

$ ffplay video.mp4 -x 600 -Y 600 -noborder



• Play a video file in a pop-up window in the upper left corner.

$ ffplay video.mp4 -x 600 -Y 600 -noborder -top 0 -left 0



Conclution

We had covered all the basics of FFmpeg. Feel free to read their official documentation here if you want to have a deeper understanding!

References

About the Author:

This article is written by Han Sheng, an AI Full Stack app developer from Cameron Highlands, Malaysia. He/She is passionate about it deep learningDeep learning, A subdiscipline of artificial intelligence, relies on artificial neural networks to analyze and process large volumes of data. This technique allows machines to learn patterns and perform complex tasks, such as speech recognition and computer vision. Its ability to continuously improve as more data is provided to it makes it a key tool in various industries, from health..., computer vision and also Edge devices. Made various web applications / AI-powered mobiles to help customers solve real-world problems. Feel free to read about it through his portfolio.

The media shown in this article is not the property of DataPeaker and is used at the author's discretion.

Related