Java 監控直播流rtsp協議轉rtmp、hls、httpflv協議返回瀏覽器 需求背景: 在做之前的項目的時候有一個對接攝像頭實時播放的需求,由於我們攝像頭的購買量不是很多,海康威視不給我們提供流媒體雲伺服器,所以需要我們自己去 一個去滿足我們能在瀏覽器看到監控畫面。項目源代碼在以前公司沒有拷貝 ...
Java 監控直播流rtsp協議轉rtmp、hls、httpflv協議返回瀏覽器
目錄需求背景:
在做之前的項目的時候有一個對接攝像頭實時播放的需求,由於我們攝像頭的購買量不是很多,海康威視不給我們提供流媒體雲伺服器,所以需要我們自己去 一個去滿足我們能在瀏覽器看到監控畫面。項目源代碼在以前公司沒有拷貝就不能複習,最近又在準備面試,所以寫了這個博客來複盤和擴展一下,由於我現在沒有Liunx,我就用Windows來演示,生產環境還是要使用Liunx,下麵這些操作在Liunx也是一樣的流程,大家自行百度。
一:瞭解音視頻流協議:
協議 | HttpFlv | RTMP | HLS | Dash |
---|---|---|---|---|
全稱 | FLASH VIDEO over HTTP | Real Time Message Protocol | HTTP Living Streaming | |
傳輸方式 | HTTP長連接 | TCP長連接 | HTTP短連接 | HTTP短連接 |
視頻封裝格式 | FLV | FLV TAG | TS文件 | Mp4 3gp webm |
原理 | 同RTMP,使用HTTP協議(80埠) | 每個時刻的數據收到後立刻轉發 | 集合一段時間的數據,生成TS切片文件(三片),並更新m3u8索引 | |
延時 | 低 1~3秒 | 低 1~3秒 | 高 5~20秒(依切片情況) | 高 |
數據分段 | 連續流 | 連續流 | 切片文件 | 切片文件 |
Html5播放 | 可通過HTML5解封包播放 (flv.js) | 不支持 | 可通過HTML5解封包播放 (hls.js) | 如果dash文件列表是MP4, webm文件,可直接播放 |
其它 | 需要Flash技術支持,不支持多音頻流、多視頻流,不便於seek(即拖進度條) | 跨平臺支持較差,需要Flash技術支持 | 播放時需要多次請求,對於網路質量要求高 |
二:方案一 rtsp 轉rtmp
1、下載nginx + nginx-rtmp-module
nginx:下載地址:http://nginx-win.ecsds.eu/download/
nginx-rtmp-module:nginx 的擴展,安裝後支持rtmp協議,下載地址:https://github.com/arut/nginx-rtmp-module
解壓nginx-rtmp-module到nginx根目錄下,並修改其文件夾名為nginx-rtmp-module(原名為nginx-rtmp-module-master)
2、nginx配置文件
到nginx根目錄下的conf目錄下複製一份nginx-win.conf 重命名 nginx-win-rtmp.conf
nginx-win-rtmp.conf:
#user nobody;
# multiple workers works !
worker_processes 2;
#error_log logs/error.log;
#error_log logs/error.log notice;
#error_log logs/error.log info;
#pid logs/nginx.pid;
events {
worker_connections 8192;
# max value 32768, nginx recycling connections+registry optimization =
# this.value * 20 = max concurrent connections currently tested with one worker
# C1000K should be possible depending there is enough ram/cpu power
# multi_accept on;
}
rtmp {
server {
listen 1935;
chunk_size 4000;
application live {
live on;
# 播放時進行回調,如果HttpRespone statusCode不等於200會斷開
# on_play http://localhost:8081/auth;
}
application hls {
live on;
# 開啟hls切片
hls on;
# m3u8地址
hls_path html/hls;
# 一個切片多少秒
hls_fragment 8s;
# on_play http://localhost:8081/auth;
# on_publish http://localhost:8081/auth;
# on_done http://localhost:8081/auth;
}
}
}
http {
#include /nginx/conf/naxsi_core.rules;
include mime.types;
default_type application/octet-stream;
#log_format main '$remote_addr:$remote_port - $remote_user [$time_local] "$request" '
# '$status $body_bytes_sent "$http_referer" '
# '"$http_user_agent" "$http_x_forwarded_for"';
#access_log logs/access.log main;
# # loadbalancing PHP
# upstream myLoadBalancer {
# server 127.0.0.1:9001 weight=1 fail_timeout=5;
# server 127.0.0.1:9002 weight=1 fail_timeout=5;
# server 127.0.0.1:9003 weight=1 fail_timeout=5;
# server 127.0.0.1:9004 weight=1 fail_timeout=5;
# server 127.0.0.1:9005 weight=1 fail_timeout=5;
# server 127.0.0.1:9006 weight=1 fail_timeout=5;
# server 127.0.0.1:9007 weight=1 fail_timeout=5;
# server 127.0.0.1:9008 weight=1 fail_timeout=5;
# server 127.0.0.1:9009 weight=1 fail_timeout=5;
# server 127.0.0.1:9010 weight=1 fail_timeout=5;
# least_conn;
# }
sendfile off;
#tcp_nopush on;
server_names_hash_bucket_size 128;
## Start: Timeouts ##
client_body_timeout 10;
client_header_timeout 10;
keepalive_timeout 30;
send_timeout 10;
keepalive_requests 10;
## End: Timeouts ##
#gzip on;
server {
listen 5080;
server_name localhost;
location /stat {
rtmp_stat all;
rtmp_stat_stylesheet stat.xsl;
}
location /stat.xsl {
root nginx-rtmp-module/;
}
location /control {
rtmp_control all;
}
location /hls {
# Serve HLS fragments
types {
application/vnd.apple.mpegurl m3u8;
video/mp2t ts;
}
expires -1;
add_header Access-Control-Allow-Origin *;
}
#charset koi8-r;
#access_log logs/host.access.log main;
## Caching Static Files, put before first location
#location ~* \.(jpg|jpeg|png|gif|ico|css|js)$ {
# expires 14d;
# add_header Vary Accept-Encoding;
#}
# For Naxsi remove the single # line for learn mode, or the ## lines for full WAF mode
location / {
#include /nginx/conf/mysite.rules; # see also http block naxsi include line
##SecRulesEnabled;
##DeniedUrl "/RequestDenied";
##CheckRule "$SQL >= 8" BLOCK;
##CheckRule "$RFI >= 8" BLOCK;
##CheckRule "$TRAVERSAL >= 4" BLOCK;
##CheckRule "$XSS >= 8" BLOCK;
root html;
index index.html index.htm;
}
# For Naxsi remove the ## lines for full WAF mode, redirect location block used by naxsi
##location /RequestDenied {
## return 412;
##}
## Lua examples !
# location /robots.txt {
# rewrite_by_lua '
# if ngx.var.http_host ~= "localhost" then
# return ngx.exec("/robots_disallow.txt");
# end
# ';
# }
#error_page 404 /404.html;
# redirect server error pages to the static page /50x.html
#
error_page 500 502 503 504 /50x.html;
location = /50x.html {
root html;
}
# proxy the PHP scripts to Apache listening on 127.0.0.1:80
#
#location ~ \.php$ {
# proxy_pass http://127.0.0.1;
#}
# pass the PHP scripts to FastCGI server listening on 127.0.0.1:9000
#
#location ~ \.php$ {
# root html;
# fastcgi_pass 127.0.0.1:9000; # single backend process
# fastcgi_pass myLoadBalancer; # or multiple, see example above
# fastcgi_index index.php;
# fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
# include fastcgi_params;
#}
# deny access to .htaccess files, if Apache's document root
# concurs with nginx's one
#
#location ~ /\.ht {
# deny all;
#}
}
# another virtual host using mix of IP-, name-, and port-based configuration
#
#server {
# listen 8000;
# listen somename:8080;
# server_name somename alias another.alias;
# location / {
# root html;
# index index.html index.htm;
# }
#}
# HTTPS server
#
#server {
# listen 443 ssl spdy;
# server_name localhost;
# ssl on;
# ssl_certificate cert.pem;
# ssl_certificate_key cert.key;
# ssl_session_timeout 5m;
# ssl_prefer_server_ciphers On;
# ssl_protocols TLSv1 TLSv1.1 TLSv1.2;
# ssl_ciphers ECDH+AESGCM:ECDH+AES256:ECDH+AES128:ECDH+3DES:RSA+AESGCM:RSA+AES:RSA+3DES:!aNULL:!eNULL:!MD5:!DSS:!EXP:!ADH:!LOW:!MEDIUM;
# location / {
# root html;
# index index.html index.htm;
# }
#}
}
3、cmd 到nginx根目錄啟動nginx
nginx.exe -c conf\nginx-win-rtmp.conf
測試:瀏覽器輸入 http://localhost:5080/stat,看到
代表安裝成功
4、下載ffmpeg安裝
ffmpeg:一個處理音視頻強大的庫,我們需要用它來轉協議,下載地址:https://www.gyan.dev/ffmpeg/builds/ ,這裡可以下載essential和full版本,essential就是簡版,只包含ffmpeg.exe、ffplay.exe、
ffprobe.exe, 而full版本就包含了動態庫和相關頭文件,方便我們在開發中調用。
5、配置ffmpeg環境變數
將ffmpeg解壓後裡面的bin路徑複製到Path裡面去
6、測試ffmpeg
cmd ffmpeg -version
命令看到代表成功
7、下載VLC播放器
下載地址:https://www.videolan.org/vlc/
8、查攝像頭的rtsp協議格式
我這裡截圖是海康威視的
現在沒有測試的流,我找了個點播的rtsp
rtsp://wowzaec2demo.streamlock.net/vod/mp4:BigBuckBunny_115k.mp4,用這個代替是一樣的
9、執行ffmpeg命令
ffmpeg強大,命令也是複雜,我們cmd 執行
ffmpeg -re -rtsp_transport tcp -stimeout 20000000 -i "rtsp://wowzaec2demo.streamlock.net/vod/mp4:BigBuckBunny_115k.mp4" -buffer_size 1024000 -max_delay 500000 -codec:v libx264 -r 25 -rtbufsize 10M -s 1280x720 -map:v 0 -an -f flv rtmp://127.0.0.1:1935/live/test
rtsp://wowzaec2demo.streamlock.net/vod/mp4:BigBuckBunny_115k.mp4,是輸入源頭
rtmp://127.0.0.1:1935/live/test 是輸出地址
如果沒有報錯的話,到現在rtsp就已經轉換好了
ffmpeg命令學習:https://www.jianshu.com/p/df3216a52e59 、https://blog.csdn.net/fuhanghang/article/details/123565920
10、測試rtmp是否轉換成功
我們打開VLC,媒體->打開網路串流->輸入 rtmp://127.0.0.1:1935/live/test
-> 播放
11、測試是否成功
等待幾秒鐘看到有視頻播放就是成功了
12、為什麼放棄了用rtmp
rtmp的優點是延遲低,效率高,但是在瀏覽器需要安裝flash才能放,也就老版本的瀏覽器在用,rtmp可能會在別的地方支持,所以還是把他方式方法貼出來了。
三:方案二 rtsp轉hls
1、nginx配置:
在前面已經貼出來了,其中這幾個是針對hls的
2、執行ffmepg命令
ffmpeg -i "rtsp://wowzaec2demo.streamlock.net/vod/mp4:BigBuckBunny_115k.mp4" -vcodec libx264 -acodec aac -f flv rtmp://127.0.0.1:1935/hls/test
3、查看nginx根目錄 -> hls -> test.m3u8 是否生成
生成了代表一切正常
4、m3u8在網頁上播放
<!DOCTYPE html>
<html lang="zh-CN">
<head>
<meta charset="UTF-8">
<title>前端播放m3u8格式視頻</title>
<!--https://www.bootcdn.cn/video.js/-->
<link href="https://cdn.bootcss.com/video.js/7.6.5/alt/video-js-cdn.min.css" rel="stylesheet">
<script src="https://cdn.bootcss.com/video.js/6.6.2/video.js"></script>
<!--https://www.bootcdn.cn/videojs-contrib-hls/-->
<script src="https://cdn.bootcss.com/videojs-contrib-hls/5.15.0/videojs-contrib-hls.min.js"></script>
</head>
<body>
<video id="myVideo" class="video-js vjs-default-skin vjs-big-play-centered" controls preload="auto" width="1080" height="708" data-setup='{}'>
<source id="source" src="http://127.0.0.1:5080/hls/test.m3u8" type="application/x-mpegURL">
</video>
</body>
<script>
// videojs 簡單使用
var myVideo = videojs('myVideo',{
bigPlayButton : true,
textTrackDisplay : false,
posterImage: false,
errorDisplay : false,
})
myVideo.play() // 視頻播放
myVideo.pause() // 視頻暫停
</script>
</html>
source標簽的src屬性:http://你的nginx ip:nginx http埠/hls/test.m3u8
rtsp轉HLS成功!
5、認識一下m3u8格式
m3u8文件裡面存儲了一個索引,以文本格式打開是這樣的
#EXTM3U
#EXT-X-VERSION:3
#EXT-X-MEDIA-SEQUENCE:56
#EXT-X-TARGETDURATION:13
#EXTINF:10.381,
test-56.ts
#EXTINF:10.422,
test-57.ts
#EXTINF:13.453,
test-58.ts
m3u8文件它不是視頻源,源頭是ts尾碼文件
6、為什麼放棄了用HLS
轉HLS協議及網頁載入過程:
ffmepg收到rtsp的流時候,會等一個切片的時間,一個切片時間到了,切片ts會放到伺服器中,同時m3u8文件中加一個索引,對應著新進入的切片。網頁在載入m3u8的時候,就是讀取m3u8中的的索引去載入ts文件,所以在不斷的請求ts,對ts進行解析,不斷的和TCP握手,這就是為什麼HLS延遲高和對網速的要求高的原因,我們監控肯定是要延遲低的,HLS相容性好,適合點播。
四:方案三rtsp 轉httpflv(採用)
1、安裝nginx-flv-module
這個插件需要編譯,教程:https://blog.csdn.net/KayChanGEEK/article/details/105095844
我這裡已經編譯好了,直接下載啟動:
https://gitee.com/isyuesen/nginx-flv-file
2、nginx配置
看我git裡面的https://gitee.com/isyuesen/nginx-flv-file/blob/master/conf/nginx.conf,和預設的config差別主要是添加了這幾個
rtmp {
server {
listen 1935;
# 流復用的最大塊大小
chunk_size 4000;
application liveapp {
live on;
# 推流開始
on_publish http://localhost:8081/auth;
# 推流關閉
on_publish_done http://localhost:8081/auth;
# 客戶端開始播放
on_play http://localhost:8081/auth;
# 客戶端結束播放
on_play_done http://localhost:8081/auth;
}
}
}
location /live {
flv_live on;
chunked_transfer_encoding on;
add_header 'Access-Control-Allow-Credentials' 'true'; #add additional HTTP header
add_header 'Access-Control-Allow-Origin' '*'; #add additional HTTP header
add_header Access-Control-Allow-Headers X-Requested-With;
add_header Access-Control-Allow-Methods GET,POST,OPTIONS;
add_header 'Cache-Control' 'no-cache';
}
3、做java許可權認證
nginx rtmp配置中有配置on_publish鉤子介面 http://localhost:8081/auth,這個回調HttpResponse stausCode如果不等於200會拒絕I/O,更多回調鉤子看:https://github.com/arut/nginx-rtmp-module/wiki/Directives#on_connect
@PostMapping("/auth")
public void getVideo(String token, HttpServletRequest httpServletRequest, HttpServletResponse httpServletResponse) throws Exception {
if (token.equals("tokenValue")) {
httpServletResponse.setStatus(200);
} else {
// 拒絕服務
httpServletResponse.setStatus(500);
}
}
4、執行ffmepg命令:
ffmpeg -re -rtsp_transport tcp -i "rtsp://wowzaec2demo.streamlock.net/vod/mp4:BigBuckBunny_115k.mp4" -f flv -vcodec h264 -vprofile baseline -acodec aac -ar 44100 -strict -2 -ac 1 -f flv -s 640*360 -q 10 "rtmp://127.0.0.1:1935/liveapp/test"
4.1 採用java代碼去執行ffmepg命令
依賴 javaCV
<dependency>
<groupId>org.bytedeco</groupId>
<artifactId>javacv-platform</artifactId>
<version>1.5.2</version>
</dependency>
public class App {
public static void main( String[] args ) throws IOException, InterruptedException {
String name = "test";
// rtsp地址
String rtspDir = "rtsp://wowzaec2demo.streamlock.net/vod/mp4:BigBuckBunny_115k.mp4";
// rtmp地址
String rtmpDir = "rtmp://192.168.0.140:1935/liveapp/" + name + "?token=tokenValue";
String ffmpeg = Loader.load(org.bytedeco.ffmpeg.ffmpeg.class);
ProcessBuilder pb = new ProcessBuilder(ffmpeg,
"-re",
"-rtsp_transport",
"tcp",
"-i",
rtspDir,
"-f",
"flv",
"-vcodec",
"h264",
"-vprofile",
"baseline",
"-acodec",
"aac",
"-ar",
"44100",
"-strict",
"-2",
"-ac",
"1",
"-f",
"flv",
"-s",
"640*360",
"-q",
"10",
rtmpDir
);
pb.inheritIO().start().waitFor();
}
}
5、測試http-flv鏈接
如果你跟著我做的,那鏈接就是 http://127.0.0.1:18080/live?port=1935&app=liveapp&stream=test&token=tokenValue,在VLC播放器中點擊媒體 -> 打開網路串流 -> 輸入http://127.0.0.1:18080/live?port=1935&app=liveapp&stream=test&token=tokenValue -> 播放
有視頻證明你離成功就差最後一步
6、前端使用flv.js播放:
<!doctype html>
<html lang="en">
<head>
<meta charset="UTF-8">
<meta name="viewport"
content="width=device-width, user-scalable=no, initial-scale=1.0, maximum-scale=1.0, minimum-scale=1.0">
<meta http-equiv="X-UA-Compatible" content="ie=edge">
<title>播放http-flv</title>
</head>
<body>
<video id="videoElement"></video>
<script src="https://cdn.bootcdn.net/ajax/libs/flv.js/1.6.2/flv.min.js"></script>
<script>
if (flvjs.isSupported()) {
const videoElement = document.getElementById('videoElement');
const flvPlayer = flvjs.createPlayer({
type: 'flv',
url: 'http://127.0.0.1:18080/live?port=1935&app=liveapp&stream=test&token=tokenValue'
});
flvPlayer.attachMediaElement(videoElement);
flvPlayer.load();
flvPlayer.play();
}
</script>
</body>
</html>