Webcam control with WPF or how to create high framerate player with DirectShow by using InteropBitmap in WPF application
Did you ever see, that MediaElement “eats” about 30% of CPU while playing movie in WPF? Did you thought, that you can display live camera capture in WPF with 60 fps full screen (I have really high resolution 1920×1200) and 2% of CPU? You did not? Let’s see how it can be done. Today we’ll create simple WebCam player control that can show you live video capturing with high frame rate. In order to do it, we’ll use DirectShow, WPF and make them work together.
You, probably do not believe me. Let’s start. In order to build this application, we need to make DirectDraw working in C# managed code. We can use DirectShow.NET, but this time we’ll do it manually. Why? because I love to do things manually. So let’s understand what we need? Actually, not very much: one Sample Grabber (ISampleGrabber) and one Device input filter(IBaseFilter). Both we should connvert with Graph Builder (IGraphBuilder) and point to some grabber implementation (ISampleGrabberCB). Also, we do not want DirectShow to render video for use, thus we’ll send it’s default Video Window (IVideoWindow) to null with no AutoShow and then run the controller (IMediaControl). Do you tired enough to lost me? Let’s see the code. One Filter graph with one Device Filter and one Sample grabber.
graph = Activator.CreateInstance(Type.GetTypeFromCLSID(FilterGraph)) as IGraphBuilder;
sourceObject = FilterInfo.CreateFilter(deviceMoniker);grabber = Activator.CreateInstance(Type.GetTypeFromCLSID(SampleGrabber)) as ISampleGrabber;
grabberObject = grabber as IBaseFilter;graph.AddFilter(sourceObject, "source");
graph.AddFilter(grabberObject, "grabber");
Set media type for our grabber
using (AMMediaType mediaType = new AMMediaType())
{
mediaType.MajorType = MediaTypes.Video;
mediaType.SubType = MediaSubTypes.RGB32;
grabber.SetMediaType(mediaType);
And then connect device filter to out pin and grabber to in pin. Then get capabilities of video received (thiss stuff come from your web camera manufacturer)
if (graph.Connect(sourceObject.GetPin(PinDirection.Output, 0), grabberObject.GetPin(PinDirection.Input, 0)) >= 0)
{
if (grabber.GetConnectedMediaType(mediaType) == 0)
{
VideoInfoHeader header = (VideoInfoHeader)Marshal.PtrToStructure(mediaType.FormatPtr, typeof(VideoInfoHeader));
capGrabber.Width = header.BmiHeader.Width;
capGrabber.Height = header.BmiHeader.Height;
}
}
Out pin to grabber without buffering and callback to grabber object (this one will get all images from our source).
graph.Render(grabberObject.GetPin(PinDirection.Output, 0));
grabber.SetBufferSamples(false);
grabber.SetOneShot(false);
grabber.SetCallback(capGrabber, 1);
Dump output window
IVideoWindow wnd = (IVideoWindow)graph;
wnd.put_AutoShow(false);
wnd = null;
And run the controller
control = (IMediaControl)graph;
control.Run();
We done. Now our video is captured and can be accessed from BufferCB method of ISampleGrabberCB. Next step is to do WPF related stuff
First of all, we’ll use InteropBitmap. This one will provide us with real performance bust. So, one our DirectShow graph is ready and we know result image capabilities, we can create memory section and map it in order to provide ISampleGrabberCB with place to put images. This will be always the same pointer, so all we have to do is to .Invalidate interop image.
if (capGrabber.Width != default(int) && capGrabber.Height != default(int))
{uint pcount = (uint)(capGrabber.Width * capGrabber.Height * PixelFormats.Bgr32.BitsPerPixel / 8);
section = CreateFileMapping(new IntPtr(-1), IntPtr.Zero, 0×04, 0, pcount, null);
map = MapViewOfFile(section, 0xF001F, 0, 0, pcount);
BitmapSource = System.Windows.Interop.Imaging.CreateBitmapSourceFromMemorySection(section, capGrabber.Width, capGrabber.Height, PixelFormats.Bgr32,
capGrabber.Width * PixelFormats.Bgr32.BitsPerPixel / 8, 0) as InteropBitmap;
capGrabber.Map = map;
if (OnNewBitmapReady != null)
OnNewBitmapReady(this, null);
}
Now in capGrabber (ISampleGrabberCB) we’ll copy buffer, comes from our webcam to the mapped location for WPF usage
public int BufferCB(double sampleTime, IntPtr buffer, int bufferLen)
{
if (Map != IntPtr.Zero)
{
CopyMemory(Map, buffer, bufferLen);
OnNewFrameArrived();
}
return 0;
}
All we have to do is to call InteropBitmap.Invalidate() each frame to reread the image bytes from the mapped section.
if (BitmapSource != null)
{
BitmapSource.Invalidate();
How do display all this stuff? Simple – subclass from Image and set it’s Source property with the interop bitmap.
public class CapPlayer : Image,IDisposable
{
…void _device_OnNewBitmapReady(object sender, EventArgs e)
{
this.Source = _device.BitmapSource;
}
Now, the usage from XAML is really simple
<l:CapPlayer x:Name="player"/>
We done
As always, download full source code for this article
…and be good people and don’t tell anymore, that WPF performance in terms of imaging is sucks
P.S. small ‘r’ if you have more, then one WebCam connected. Inside CapDevice class there is member, public static FilterInfo[] DeviceMonikes, that provides you with all DirectShow devices installed. So, the only thing you should do in order to change the device is to set deviceMoniker = DeviceMonikes[0].MonikerString; with the moniker of your device. This sample works with first one.
You may also be interested with:
- WPF DataGrid CTP is here. It’s also open source!
- Nifty time savers for WPF development
- The new version of WPF Performance Profiling Tool is available for download
- Quick Silverlight (and WPF) tip: How to write program without XAML
- Capturing and streaming sound by using DirectSound with C#
April 23rd, 2008 · Comments (23)
23 Responses to “Webcam control with WPF or how to create high framerate player with DirectShow by using InteropBitmap in WPF application”
Leave a Reply
Discover other tags
My tools
- .NET Framework Detector
- Duplicate images finder
- Exchange Security Policy for Windows Mobile Devices Fix
- Gas Price Windows Vista SideBar gadget
- Israel Traffic Information Windows Vista SideBar gadget
- Localization fix for SAP ES Explorer for Visual Studio
- LocTester
- RTL and LTR in Windows Live Writer
- Silverlight controls library
- Snipping tool integration plugin for WLW
- USB FM receiver library
- Vista Battery Saver
- WebCam control for WPF
- Windows Live SkyDrive attachment for Windows Live Writer
- Wireless Migrator
- WPF Virtual Keyboard















January 1st, 2009 at 12:45 am
Very impressive. +1 for changing the resolution. I’ve been searching for a solution and could not find any that I understand.
January 1st, 2009 at 12:45 am
I have extended your CapPlayer class a little bit by adding 2 properties:
1) Rotation
2) CurrentBitmap
You can find a blog entry about it here:
blog.catenalogic.com/…/Retrieving-snapshots-from-a-webcam.aspx
January 1st, 2009 at 12:45 am
I’m starting in WPF. There is some method in the classes of this project that capture one frame of the CapPlayer control to put in one image control, for example?
January 1st, 2009 at 12:45 am
How would you specify the resolution you want the camera to produce? Sorry about the newbie question.
January 1st, 2009 at 12:45 am
What about webcam options? Is it possible to change them?
January 1st, 2009 at 12:45 am
Pingback from Webcam Control with WPF « Roman’s Blog
January 1st, 2009 at 12:45 am
Hi, I noticed someone posted a question about the video being upside down, and I must admit I have the same problem, however, I fixed it with an easy fix.
2 options.
First option is you can go into Expression Blend 2.0, and just rotate the video player 180 degrees.
if you don’t have Expression Blend 2.0 you can do the following
replace the following <1:CapPlayer x:Name="player"/>
with the following code
<l:CapPlayer x:Name="player" RenderTransformOrigin="0.5,0.5">
<l:CapPlayer.RenderTransform>
<TransformGroup>
<ScaleTransform ScaleX="1" ScaleY="1"/>
<SkewTransform AngleX="0" AngleY="0"/>
<RotateTransform Angle="180.294"/>
<TranslateTransform X="0" Y="0"/>
</TransformGroup>
</l:CapPlayer.RenderTransform>
</l:CapPlayer>
Hope it helped.
January 1st, 2009 at 12:45 am
How do you stop the feed of the images? I called the Stop() method located _device.Stop() and made a public method to access. When I call this it will stop but it takes a crazy long time and is very random, does it not dispose properly?
January 1st, 2009 at 12:45 am
Wow, excellent article/code, thanks!
I’m trying to get this to work with an IP Camera (Linksys WVC54GC) but can’t figure out how to get the FilterInfo for the ip camera (device enum won’t work since it’s not directly connected to my pc). I can get IFilterInfo for the ip cam by going through FilgraphManager instead, but connect fails when I try to use this filter. Any ideas on how I can get this to work with an IP Camera? Any help would be greatly appreciated.
Thanks!
Mike.
January 1st, 2009 at 12:45 am
Excellent article Tamir!
I was playing around with it using a firewire-video as source, and it worked just fine (except for beeing flipped upside down, but that was easily corrected with a a scale transform), BUT I have a question though, is there any route I could take to add de-interlacing of the video as well? Do you know? Thanks!
Cheers!
-Ted
January 1st, 2009 at 12:45 am
cindex, I do not understand and cannot repro the problem. It looks like a problem with your webcam software or drivers
January 1st, 2009 at 12:45 am
Please, answer my problem.
T.T
Inspite of "player.Dispose();"
webcam is not released.
Because of that, new window do not load webcam.
Help,me
January 1st, 2009 at 12:45 am
…hmm
Lamp turned off when debugging stop, not when application is closed.
January 1st, 2009 at 12:45 am
Web cam video window is opened by "showdialog()"
January 1st, 2009 at 12:45 am
Thank you for your answer,Tamir Khason.
But, I do not understand yours.
Inspite fo "player.Dispose();"
My webcam`s lamp is not turned off.(my webcam`lame turned on when it run)
Because of that, probably new window is non shown webcam video.
It`s lamp is turned off when application is closed.
January 6th, 2009 at 10:30 am
[...] Webcam control with WPF how to create high framerate player with DirectShow by using InteropBitmap in WPF application (tags: c# wpf webcam development) Share and Enjoy: [...]
February 9th, 2009 at 12:10 pm
Anybody got something like this in vb.net? I already have the whole app written in vb, i was just needing a webcam control and i would be golden…
February 13th, 2009 at 10:33 am
This solution works for me but i have a big problem: this is “eating” more than 30 of CPU. Why could it be possible?
Sorry for my english
February 23rd, 2009 at 10:52 am
How to increase resolution of video? I ve got hd web cam and got only 160/120 resolution?
With greetings, Anatoly
April 9th, 2009 at 3:36 pm
Good! It works fine! Thank you.
May 26th, 2009 at 6:47 am
Great job!
How can I change the Webcam resolution?
————————————
Rich Says:
January 1st, 2009 at 12:45 am
Very impressive. +1 for changing the resolution. I’ve been searching for a solution and could not find any that I understand.
————————————
June 19th, 2009 at 2:31 pm
Trying to use this to limit the video to just one of the channels (say… Red-only, don’t show the blue or green, so I have just the red part of the image…).
I’m guessing that it’ll be a tweak in the capGrabber_PropertyChanged function of CapDevice.cs. Because that’s where the BitmapSource is built from memory.
Or am I on completely the wrong track?
-Scott
June 30th, 2009 at 2:32 am
why the picture is