Raspberry Pi MJPEG at ~30fps

You may also like...

24 Responses

  1. Did you try on this on the original Pi?
    Can you save to SD card as MJPEG instead of streaming? What speeds can you get if you have?

    I wanted to try the MIT motion magnification and they recommend using MJPEG as it does not use intra frame dependencies.

    Thanks.

    Rich Shumaker

  2. Lewis says:

    Hi Rich, I’ve got a couple of Raspberry Pi 2 – I only got in to the Pi when I heard about the extra grunt that the Pi 2 had so unfortunately I’ve never tried this on the original Pi.

    Honestly, I’m not sure that you’ll get it to record to the SD card at that rate. Most of the Class 10 SD cards can only just manage 40Mbps.

    Apologies I can’t be much help here. My Pi’s are out of reach of me right now.

    Oh, I should add, if you want to give it a try yourself, you’ll need to fettle with the information after

  3. Franklyn says:

    I’ve been looking for a solution exactly like this, for months. I was using MotionPie for awhile, but the image quality is horrible. RPi Cam Control was also a really nice option, but the settings are crazy and I spent hours just trying to do basic things. So your method is awesome and elegant. The only thing I’m looking for is to save a jpeg every 20 seconds instead of every minute. Which option would I change in ffmpeg to achieve this?

    Thanks!

  4. Lewis says:

    Hi Franklyn, thanks for the positive comments. This admittledy has been quite a journey just to get the quality (and field of view) I show in the final image of this updated post. The script that I use to grab the stills from the camera is simply executed from crontab so you wouldn’t make any changes to the script itself. Unfortunately crontab doesn’t do sub-minute executions so once per minute is the best frequency you can get without fettling it a little. This chap suggests a solution to add the script to crontab twice but make it sleep before executing. http://stackoverflow.com/questions/9619362/running-a-cron-every-30-seconds. In your case, to get it to run every 20 seconds, you’d add the script to crontab three times.

    To get to crontab run this at the command line.

    My crontab looks as follows:

    To get what you want and if I were to change mine, it would look like this instead:

    Hope that’s useful for you.
    -Lewis

  5. Franklyn says:

    Thanks so much! I’m going to try it out asap.

    Thanks again,

    Franklyn

  6. Jeremy says:

    Are you just using the stock Raspberry Pi Camera Module or a USB Webcam? Its hard to tell from the post.

  7. Lewis says:

    It’s the stock module but with a different lens. ☺

  8. Jeremy says:

    Sorry for my ignorance here, but had a question regarding this command. I’m using this for an aircraft application where my heads up display on the ground station only works with the MJPEG format (overlays flight metrics ect…), and since the stock RPi Cam Module is so light, it is perfect for my particular needs.

    That said, thank you for doing the leg work on getting this working, I’ve been trying to find a post where someone could get a high frame-rate with the stock RPi camera and is streaming it, not just saving it to the SD card.

    When I run the command (modified slightly for my own needs) I get quite a few errors. Maybe you can make sense of them. Here’s the command I run:

    cvlc –no-audio \
    v4l2:///dev/video0 \
    –v4l2-width 1920 \
    –v4l2-height 1440 \
    –v4l2-chroma MJPG \
    –v4l2-hflip 1 \
    –v4l2-vflip 1 \
    –sout ‘#standard{access=http{mime=multipart/x-mixed-replace;boundary=–7b3cc56e5f51db803f790dad720ed50a},mux=mpjpeg,dst=:8084/}’ \
    -I dummy

    And here’s the errors I get:
    VLC media player 2.0.3 Twoflower (revision 2.0.2-93-g77aa89e)
    [0x19f8e18] inhibit interface error: Failed to connect to the D-Bus session daem on: Unable to autolaunch a dbus-daemon without a $DISPLAY for X11
    [0x19f8e18] main interface error: no suitable interface module
    [0x76600d48] main interface error: no suitable interface module
    [0x196d8f0] main libvlc error: interface “globalhotkeys,none” initialization fai led
    [0x1a01b38] dummy interface: using the dummy interface module…

    Any ideas? I don’t have a display connected (streaming over network only), could that be the problem?

  9. Lewis says:

    Hi Jeremy, gald this could help you. I’d wager that’s because there’s an existing vlc.exe process (or something else) using the camera.

    If you don’t want to reboot use the “top” command on your Pi to see if anything else is running that could be accessing the camera – could be an abandoned vlc.exe process or raspivid/raspistill. Anything that attaches to the camera module will have dedicated access and any other process will get an error.

    Let me know how you get on.

    -Lewis

  10. Jeremy says:

    I got it working, and now it streams to my Windows 7 VLC without issues.

    Problem is it STILL doesn’t pipe into my ground station HUD which only supports the mjpeg format.

    I have some other webcam feeds that I can successfully pipe directly into my ground station HUD (but those just use mjpg_streamer … really bad quality and fps).

    Most people it seems are using a capture device and re-streaming the video using dshow// in the VLC GUI (guide here http://vps.oborne.me/gcs/Mjpeg%20Video%20Source%20with%20VLC.htm).

    Since the video stream I get from the Raspberry Pi (using your command) doesn’t work directly in the ground station HUD like my mjpg_streamer videos do, I thought I would follow the HUD developer’s suggestion and try to re-stream it again using the Windows VLC GUI in that guide.

    Instead of using dshow in VLC however since I’m not using a capture device, I’m just re-streaming the video from my Raspberry Pi Cam using your command above (for me that is http://192.168.2.250:8084). When I try to re-stream the Raspberry Pi output in VLC on my Windows 7 desktop to another port (8085) as in the guide, it just crashes the windows VLC GUI. The command I end up with after setting up VLC in the GUI as the guide suggests is:

    :sout=#transcode{vcodec=MJPG,vb=800,scale=1,acodec=none}:http{mux=mpjpeg,dst=:8085/} :sout-keep

    I didn’t think this would be so complicated as all my other video streams work using mjpg_streamer… the command I use with mjpg_streamer that actually works in the HUD is:

    /usr/bin/mjpg_streamer –input “input_uvc.so $yuv –device $device –fps $fps –resolution $resolution” –output “output_http.so –www $www –port $port $auth”

    where I’m using a config file to specify all $ options.

    Any advice? Appreciate any help 🙂

  11. Lewis says:

    Hi Jeremy, sounds like a cool little project you have on here. I get where you’re coming from with re-streaming it, that makes sense and it’s frustrating that it just crashes VLC. I have recently been tinkering with another little project of mine that’s kind of like a home intranet web application and one of the features is a proxy written in PHP for an MJPEG stream from one of my webcams. That webcam is a Samsung SNH-E6440BN and it only manages 2-3fps in MJPEG mode 1920×1080 at the best of times so it doesn’t really stress the proxy. I haven’t actually got to the point of extending it to use the Pi (which I’m having terrible longevity/reliability issues with currently) so you’re welcome to try it. As I mentioned, it’s written in PHP and uses the cURL extension (of PHP) and is actually quite basic in functionality so it shouldn’t be too much to wrap your head around.

    I’d be interested to see if this would execute inside the Pi using its own install of PHP/Apache or if it just buries the CPU. It isn’t doing any transcoding, just regurgitating what it receives from the MJPEG stream. If the reason your HUD isn’t displaying the stream is related to a format issue, like the MJPEG stream isn’t valid or something, this could give you an extra level of control that you don’t have with VLC.

    Let me know how you get on! Code follows:

  12. Lewis says:

    I should add that there’s stuff in there like authentication etc. that you won’t need. I’m interested to hear how you get on so do keep me updated and I’ll see if I can find something like an MJPEG stream validator to see if what my command is putting out is considered a valid MJPEG stream and VLC is just being kind by playing it.

  13. Lewis says:

    Jeremy, I just had a thought – I remember when i was tinkering with this that I had to specify the forward slash ‘/’ after the stream or VLC simply wouldn’t play it. I think that’s why I ended up specifically outputting it to webcam.mjpg in the end because I kept forgetting the slash. My final code for the output is:

  14. Jeremy says:

    It’s playing fine for me in VLC without the / at the end, the problem is when I am trying to re-stream the video it crashes the VLC gui. I had hoped that this command would output an mjpeg and that I wouldn’t have to re-stream.

    Because the ground station HUD plays the video from mjpg-streamer I mentioned earlier (off a webcam) no problem and the cvlc command doesn’t, there has to be something different with the video format that is output that it doesn’t like. One idea I had is to look at the packets with Wireshark as a “verification” but after that I’m out of ideas.

  15. Lewis says:

    I agree, a single stream would be the preference certainly. MJPEG itself isn’t complex, it’s literally a stream of JPEG pictures, separated by a boundary with a Content-type of multipart/x-mixed-replace sent at the outset. So long as the client application can understand and decode this, it should work. That’s why you see the content type and boundary declarations in the cvlc command – there’s not a lot else to it and certainly no room for tinkering other than with the boundary but the boundary is just a marker to define the start and end of JPEG images (as the frames).

    Can your HUD device deal with the size of the images being sent down in the stream? It may be worthwhile reducing image size, frame rate etc. to bring down throughput rates to those you get with mjpg streamer. VLC should give you information about the stream so check Tool > Codec Information to make sure it says Motion JPEG MJPG – I’m sure it does, it’s just another check. I’ve just had a read of this page too and it suggests using cURL to grab trace information from the stream (saves messing with WireShark).

    http://pdn.pelco.com/content/mjpeg-http-push

    -Lewis

  16. Rodney says:

    Hello there! I wanted to thank you for your guide, it helped me tremendously. I had a few Pi1’s lying around & thought I could use them as front & rear house cameras.

    I initially used MotionPie for my Pi Camera Module stream but I had to re-purpose a Pi2 to get between 7 to 10 fps @ 720p. 1080p was around 3-4 fps & zoomed in like you said.

    After reading your guide I am now currently getting a stable & solid 15fps @ 720p streaming from a Pi1 with the Pi Camera Module.
    I may even change back to the Pi2 to try for a 10fps 1920×1440 stream like yours. The quality would be nice.

    For remote viewing of the cameras your FTP upload of an image script is more than enough for my needs.

    For others with a Pi1, I used the following:

    v4l2-ctl --set-fmt-video=width=1280,height=720,pixelformat=5
    v4l2-ctl -c compression_quality=100,sharpness=30
    
    v4l2-ctl -p 15
    
    cvlc    --no-audio \
            v4l2:///dev/video0 \
            --v4l2-width 1280 \
            --v4l2-height 720 \
            --v4l2-chroma MJPG \
            --sout '#standard{access=http{mime=multipart/x-mixed-replace;boundary=--7b3cc56e5f51db803f790dad720ed50a},mux=mpjpeg,dst=:8554/}' \
            -I dummy
    

    Thanks very much once again!
    -Rodney

  17. Lewis says:

    You’re welcome Rodney. I’m really glad it was useful for you. Sorry if you saw an error a moment ago, I’m currently tinkering on my server 🙂

  18. Richard C Giroux says:

    I followed your directions and things worked but the speed it really slow. I am seeing about a 2 second lag along with maybe 5 fps on a Pi3.

    What am I doing wrong?

  19. Michael says:

    I don’t know what I’m doing wrong. Your first example works – and nothing after it.

    I’m able to take stills and video – so I know the camera works.

  20. Eric says:

    Update: I just submitted a comment/question on an error seen when implementing cvlc over VNC. I’m still getting the error – but I decided to fire up VLC on the receive side and see if the stream shows up and, sure enough, there it is. So apparently, the video is properly streamed. The error is still a curiosity … but at least not a show stopper!

  21. Eric says:

    This aligns actually with the very first comment in this thread. I’d like to write the 1920×1440 10 FPS video to a file (for ~10 seconds or so). I can stream across my network with the commands you suggest above just fine. So I updated the cvlc command as follows:

    cvlc –no-audio \
    v4l2:///dev/video0 \
    –v4l2-width 1920 \
    –v4l2-height 1440 \
    –v4l2-chroma MJPG \
    –v4l2-hflip 1 \
    –v4l2-vflip 1 \
    –sout=file/ts:filename.mpg

    It saves to a file just fine. However, if I open the VLC gui on the RPi, nothing shows up. I can pull the mpg file over to my Windows machine and open it up in VLC. The upper half of the video frames play just fine. However the lower half of the frame seems frozen to the 1st frame captured.

    Looking at the ‘Messages’ dialog in VLC – the following warnings show up:
    avcodec warning: plane 0 not aligned
    avcodec warning: disabling direct rendering
    core debug: End of video preroll
    core debug: Received first picture
    hqdn3d debug: Changing coefs to 6.94 6.00 2.56 4.50
    avcodec warning: cannot decode one frame (1777 bytes)
    core debug: Decoder wait done in 201 ms
    avcodec warning: cannot decode one frame (13009 bytes)
    avcodec warning: cannot decode one frame (65500 bytes)

    (it then proceeds to give a long string of not being able to decode one frame…)

    I thought the problem might be the data rate – i.e. that the SD card write speeds could not keep up – so I slowed the feed to 5 FPS. However, the result is the same … or even worse! I tested the write speed of the SD card on my raspberry pi using the command:

    sync; dd if=/dev/zero of=~/test.tmp bs=500k count=1024

    and it comes in at about 10.4 MB/s (‘mega-bytes per second). Looking at task manager, when streaming, the required bandwidth is ~ 11 Mbps (‘mega-bits-per-second’).
    So the SD card should be about 8x faster than required to keep up with the generated video stream.

    Am I doing something wrong with my write-to-file command? Is there another way to write the generated video to a file on the pi (or record to file from another computer) given that the streaming is working fine?

  22. Eric says:

    I figured out what my problem was. I was using the wrong ‘muxer’. When I switched to using the ‘ogg’ muxer, I was able to write to a file just fine. For the record, the command that is working for me is as follows:

    cvlc –no-audio \
    v4l2:///dev/video0 \
    –v4l2-width 1920 \
    –v4l2-height 1440 \
    –v4l2-chroma MJPG \
    –run-time=10 \
    –sout=file/ogg:/tmp/filename.mpg \
    vlc:\\quit
    Two things I added to this command are:

    The ‘–run-time’ specifier to capture just a limited duration of video (10 seconds in this case).

    The ‘vlc:\quit’ command at the end to close out vlc at the end and return control to the command prompt.

    One other thing I did before capturing any video was to mount the /tmp folder to the tmpfs. Then, by writing to this folder partition, we write directly to RAM, eliminating the SD Card from the file-write process (just in case that was adding a bottleneck to the process).

  23. Jontec says:

    Hello Lewis, I stumbled on the post because I have the same problem with MJPEG shrinking the field of view (FOV). I am using the software motion to record images whenever movement was detected.
    The FOV changed after I did “bcm2835-v4l2 max_video_width=2592 max_video_height=1944”.
    I know I can alter width and height via motion config or v4l2-ctl but both ways do not result in 2592×1944 images with a full FOV.
    Here is a more elaborate explanation of the problem https://www.raspberrypi.org/forums/viewtopic.php?f=43&t=165028
    Do you have a clue how to solve it? Cheers!

  24. Peter says:

    Hi Lewis,

    thank you for detailed tutorial! One question about playing the stream – when I am streaming over wifi, VLC keeps disconnecting. I am playing the stream on PC and it disconnects after couple seconds and reconnects immediately after I press play button. Is it a way how to force VLC to keep connected? I tried to play with VLC setting, buffering, etc.. but id did not help.
    Thank you,
    Peter.

Leave a Reply

Your email address will not be published. Required fields are marked *