Video doorphone using RTSP camera
These examples / test setups show how to use tSIP as an endpoint combining
SIP/RTP audio and RTSP video from camera.
Background: I've created this examples responding to request describing setup with doorphone, FRITZ!Box and RTSP network camera. I don't think that FRITZ!Box supports video, I'm also not 100% sure if it is even necesary for this particular setup, maybe it would work better with plain peer-to-peer calls (there are some video doorphones that are working both as SIP video endpoint and RTSP server, in this case tSIP "video" build would also work). It still seems like a valid and non-unique use case.
I had prepared an example for similar scenario in the past: video doorphone, but it was based on different assumptions. tSIP was used just to start/stop separate RTSP video player, audio path was handled by separate "regular" SIP desk phone and BLF/dialog-info subscriptions were used to trigger RTSP playback. As FRITZ!Box is not supporting BLF, it would not work with it. As mpv video plugin for tSIP was created later, similar setup can also be now created in a more elegant way.
Yak shaving - testing RTSP
Initially I did not have RTSP network camera at home. At the time of writing this (end of July 2024), it was also not possible to find any working publicly available RTSP stream / test service. Really, I've tried few dozen links and all were dead at the moment. Perhaps some unsecured private cameras could be reached, but even if, they might dead as well the very next day. I needed a way to test it locallly.
VLC
This well-known video player has also built-in RTSP server functionality.
Media/Stream menu allows to use video file or local USB camera as a source.
Don't miss the "Add" button.
It looks like everything works here with transcoding disabled.
Issues found:
- Media/Stream function was confusing for me as this wizard allows to go forward without adding any output (RTSP) stream; don't forget to press "Add" button in the "Destination Setup" wizard step
- initially VLC responded with 461 error in the RTSP playback setup phase; turns out this likely mpv issue - TCP transport was used despite only UDP transport being listed in VLC media SDP; adding manual transport selection in the script starting mpv RTSP playback was needed
- I don't know why, but VLC fails after time or maybe after few subsequent RTSP connections - new playback cannot be started, VLC itself cannot be closed and has to be killed from task manager to restart
Adding option to select RTSP transport required updating tSIP_mpv.dll. Make sure to use version newer that 2.0, e.g. the one included in archives below.
Despite the reliability issue, VLC seems to be useful tool for testing.
live555
From what I understand live555 might be the key part of RTSP support in VLC.
While tSIP works pretty well, almost perfectly under Wine (thanks to Wine improvements, recently with lubuntu 24 / Wine 9.0
even network interfaces and audio devices are listed correctly), it is Windows native.
It doesn't seem like live555 Windows binaries are readily available, so I'm adding them here, along with a test WebM/VP8 file that can be used a video source:
live555_mediaserver_vs2022_bin_x64_release_static.7z. Source: https://github.com/tomek-o/live555_vs2022.
To use live555 mediaServer just start executable placing supported video file next to it and connect with rtsp://ip_address/file_name (including file extension). The list of supported file formats is limited comparing to VLC.
tSIP + tSIP_mpv.dll: video in the main window
Video is rendered on one of the buttons from "console" area.
mpv_doorphone.7z
Download also libmpv-2.dll from the link below and put it next to tSIP.exe!
tSIP + tSIP_mpv.dll: video in the tray notifier
Tray notifier started as a tiny notification window, displaying caller name and Answer/Hangup buttons,
but since tSIP 0.3.4 it can be resized and customized, working as a button container. This allows to separate
doorphone handling from regular calls and activities and it would also work well is answering doorphone is
a side job.
mpv_doorphone_tray.7z
Download also libmpv-2.dll from the link below and put it next to tSIP.exe!
Configuration
Both examples, using the main window and the tray notifier window are using similar "on call state" Lua script. This script starts or stops RTSP video playback depending on call state. What is important for a doorphone, video is displayed before call is answered (answering the call is not necesary to see the video). For displaying video, selected button (placed either in the main window or in the tray notifier window) is used as a canvas. Actual video handling is done by tSIP_mpv.dll (placed in /phone folder and activated in tSIP configuration) and libmpv-2.dll (placed next to tSIP.exe).
Lua script contains configuration section at the beginning. It includes:
- list (there can be more than one) of numbers or URIs of doorphones (other numbers/URIs would be ignored by the script); for testing you can call sip:3333@sip2sip.info right away (assigned to the speed dial button); full URIs would be used probably only for peer-to-peer setups and they can copied from tSIP call history
- RTSP address of a video source (normally network camera, in the test setup: localhost:8554 to be used with VLC as a test server); this address can contain login and password if needed
- RTSP transport selection (when testing with VLC I had to force udp, if there would be 461 error in log you might need to change it to tcp)
- ID of the button where video should be displayed (= 100 in my example)
- for a tray notifier variant: ID of the "Answer" button (to be hidden for outgoing calls or when incoming call is answered)
Some additional mpv properties reducing video delay:
PluginSendMessageText("tSIP_mpv.dll", PLAYER_NAME .. " SET_PROPERTY_STRING cache no")
PluginSendMessageText("tSIP_mpv.dll", PLAYER_NAME .. " SET_PROPERTY_STRING demuxer-readahead-secs 0")
PluginSendMessageText("tSIP_mpv.dll", PLAYER_NAME .. " SET_PROPERTY_STRING cache-pause no")
PluginSendMessageText("tSIP_mpv.dll", PLAYER_NAME .. " SET_PROPERTY_STRING untimed yes")
Example with 4 buttons switching RTSP cameras
mpv_doorphone_4btns.7z
Latency test
Glass-to-glass latency with two "real" cameras tested, modern Hanhwa dome camera and older Vivotek IP8133 was pretty consistent, close to 260 ms.
Recording video
Add
PluginSendMessageText("tSIP_mpv.dll", M.PLAYER_NAME .. " SET_PROPERTY_STRING stream-record record_test.ts")
after PLAY command (order seems to matter, same as selected output format).
Few ideas
- DTMF button with code or SendDtmf() Lua function can be used to open the door
- wave file playback function (switching audio source during the call) can be used to play instruction(s) for the person entering
- Answer() + BlindTransfer() can be used to redirect the call to someone else
- global hotkeys (e.g. assigned to buttons configured for door handling) might be useful for opening the door or rejecting the call with less interruptions of normal work
- local call recording can be enabled and optionally used for voicemail function when call is not picked up or rejected
- while decent doorphone should contain AGC / Automatic Gain Control for its microphone, similar function is also available in tSIP ("AGC for RX direction"), originally intended to deal with inconsistent volume levels when calling different mobile phones / mobile providers
- Lua SetButtonVisible() function can be used to show/hide button on which video is displayed; since buttons with higher IDs are displayed above buttons with lower IDs, big video button with ID = 100, covering most of the console area, can be shown only for doorphone calls
Other notes
- libmpv-2.dll is pretty heavy (~80 MB when uncompressed), so I have not included it in archives; download it from tSIP_plugin_mpv github and unzip next to tSIP.exe
- non-video tSIP build is included in the archives (just to make them smaller, RTSP video is handled by mpv anyway)
- while there are some decent doorphones supporting SIP/RTP video (and usually RTSP/RTP video same time) I haven't seen so far any that matches video quality of a dedicated network/security camera
- mpv property: profile low-latency (set in the Lua script) really works and might be essential to keep acceptable audio/video synchronization
- playing RTSP streams from multiple cameras (different camera angles) is be possible - see tSIP mpv plugin page
- last time I've checked Zenitel TKIV+ intercom kit, it was crashing when receiving direct call from tSIP and their support ignored it despite sending them stack trace obtained via syslog with precise crash description of a very obvious issue; this was not the first time I saw substandard software and support for SIP doorphones, even for core products from big brands - testing is really needed
Related
Back to howto list