Jonathan Chambers Web Log

Profiling Mono with AMD CodeAnalyst

2011-02-22T22:45:00.013-05:00

A coworker wanted to profile his app on mono the other day. While the profiler was rewritten for 2.10 and is now greatly improved, there are a still a few issues. One is that we are not currently using 2.10 yet for our projects. Another issue is that there is no GUI to view results from the profiler.We frequently use tools like Intel VTune and AMD CodeAnalyst to profile C++ and C# at work. I knew VTune added an API to support profiling JIT'd code. Mono has a profiling API that allows custom profilers to be loaded by the runtime. I started coding a mono profiler for VTune a few months ago. I hit a few issues, and shelved the code.

After downloading AMD CodeAnalyst (it's free) and looking through the API headers, I found a similar API for JIT'd code support. A quick hour of hacking and the profiler was working. So, how can you use it? The profiler only is setup for usage on Windows right now, so I'll assume we are working in a Windows development environment.

Download AMD CodeAnalyst.

Grab fresh mono from GitHub.

Open mono\msvc\mono.sln in VS2010. Build target of choice.

At this point, you'll need a mono install to match the runtime. There's no easy way to build a mono install on Windows currently. There is work ongoing for this, but it's not quite ready yet. You'll either have to bite the cygwin bullet or wait for the next mono release that you can build a mono runtime in sync with.

Once you have an install up an running, copy the mono-profiler-codeanalyst.dll into the bin directory in the install.

Create a simple test case.

Start CodeAnalyst and launch the Session settings dialog.

Pass --profile=codeanalyst along with your normal mono flags.

After running, you should get a results pane. Note the JITCode item taking up the most time samples. Also note mono-2.0.dll coming in third. That's the mono runtime itself. Note you can mix profiling managed and native code within the same application.

Double click on JITCode and we can see the hits for the individual methods. This includes both our code, and the mono class libraries.

These results make sense, given our test case. The most interesting thing is that a non-trivial amount of time is spent in an internal method System.String.memcpy4. This is probably due to our inefficient string concatenation within a loop.

The CodeAnalyst JIT API is somewhat limited. It doesn't allow the specification of source files or line numbers. The API itself does not seem to be currently documented, or perhaps even supported. The Intel VTune API is much richer, but VTune is not free. I'll try to put together a VTune profiler, and do another post with a more interesting sample. I'm also considering producing binaries of the profiler compatible with the recent mono 2.10 release that can be dropped in an used, avoiding the issue of building a full mono install on windows.

Mono Debugger on Windows

2009-11-13T23:30:00.001-05:00

I need to cleanup the changes a bit more, but here's a preview.

Update: Code is now checked into svn.

ANSYS and Intel

2009-04-03T12:44:00.004-04:00

I consider my job at ANSYS quite exciting. I get to work on some amazing software with great people. We get to push the limits of engineering simulation and computer technology. In case you've ever wondered what our software does, here's a video shot by Intel showing how we leverage their latest hardware.

Why did I go to Best Buy?

2009-04-01T21:17:00.004-04:00

The rumors are true. Best Buy sucks.

I went to Best Buy today to look at new laptops. I wasn't really planning on buying one at Best Buy due to the wonderful reputation they have. After looking a few minutes I find a nice Dell XPS 1530 that s being discontinued for $739. I know this is a good deal as I had configured a similar laptop at dell.com a few days ago for almost $1400. A supervisor stops by and asks if I need help. I respond I'm interested in the laptop. I then ask if price is open box, floor model, etc. He laughs and assures me I'll get a new laptop in a box. I want to see some more specs. He takes me to a kiosk and punches in the SKU. It brings up a screen similar to this, and I notice two things:

1. The price is $1059 instead of $739.

2. Big red text at top of window says "Item out of stock at store."

Supervisor notices #1 and says, "Hmm, the price is more online. That's odd since they should be the same. Oh well, we price match anyway." Supervisor then walks away.

Five minutes pass. Salesman stops by.

"Do you need any help?"

"I'll take the laptop."

"We don't have any in stock. And I can't give you floor model since we don't have a power cable for it."

"Okay. Can you check another store?"

"Sure. I'll be right back."

...

"The Erie, PA store (over 3 hours away) has 7 left at same price. But, they won't send us any for two weeks since the are doing inventory. Sorry I can't help you."

So, I leave and figure I'll buy laptop online. But wait, online price is ~$300 higher. I call online customer service and describe situation.

"Sir, stores have more leeway than we do for pricing. We cannot match the store's prices online. How much cheaper was it in the store?"

"$300 cheaper."

"Man, that's a great deal. Sorry I can't help you."

I now begin ranting at work. A guy next to me mentions his sister lives in Erie, and is coming home this weekend. She could pickup the laptop for me. Great. I call Erie store to buy laptop over phone. I start explaining whole situation again.

"So, I'd like to buy laptop and have somone pick it up for me."

"I am sorry sir. To pickup laptop, whoever purchases the laptop needs to come to the store and show ID and credit card. Sorry I cannot help you."

Grrr. I am now mad enough to call corporate customer service and explain situation to them.

"Hmm. Wow. Hmm. It doesn't sound like there's a whole lot I can do."

"So, this laptop is available 3 hours from me and also online. But, in order to get one I need to drive over 6 hours roundtrip or pay an extra $300."

"Yes, well. I guess I could call the store for you and talk to them."

"They explicitly told me they needed ID and credit card for pickup."

"Hmm. Well, the ball seems to be in your court sir as to what you want me to do."

"It sounds like you can't do much."

Best Buy has lost another customer for life.

I dedicate this post to the blog post that inspired me to take time and describe my experience.

Biggest Flaw in .Net...Fixed?

2008-10-30T08:00:00.000-04:00

OK, so maybe I am biased about the default security settings for managed code on a network share being the biggest flaw in .Net. However, it has to be one of the most aggravating ones (especially when considering deployment options). I never understood how allowing unmanaged code to run on a network share, but not managed code by default was security.

Quoting Vance Morrison:

The problem is that failing to run managed code WHILE STILL ALLOWING UNMANAGED EXE's to run, does not provide any security (because hackers will simply use unmanaged code) but does cause nontrivial deployment headaches (manage apps can't be run from network locations).

Anyway, it's now fixed!!! Just grab .Net 3.5 SP1. Thanks to MS (and Vance) for making this change.

SIL, mono, and COM on Linux

2008-05-22T15:40:00.002-04:00

A developer named Mark Strobert contacted me a while back about the feasibility of using mono and it's COM Interop implementation to port Windows software written using C# and COM. I told him it should be possible, and we discussed one of the main problems being the lack of a COM infrastructure on Linux. I hadn't heard from him again for a while until...

he emailed and informed me that they were open sourcing their basic COM implementation, a.k.a. libcom. In looking over their blog/wiki, it seems as if the use of mono's COM Interop and their libcom is holding together fairly well. I won't say this is the first real world test of COM Interop in mono, as Shana has used it quite well to implement the System.Windows.Forms WebBrowser control. However, the fact that mono's COM Interop implementation is being used by others is exciting to me (if anyone is keeping their mono COM Interop use in the closet, drop me a line to let me in on the secret).

The developers are using mono to port and develop software in order to support a faith based organization that is involved in literacy, linguistics, and translation. I won't go on any further, because if you're interested you can click around their site (or Linux development site) to see better information than I could explain here.

Test

2007-10-30T10:35:00.001-04:00

This is only a test...

Mono.XPCOM

2007-08-07T08:09:00.000-04:00

With a little prodding from Miguel, I finally spent some time trying to play around with mono's COM Interop support and XPCOM. The important thing to note is that COM Interop in mono should (and does) work with XPCOM. Most of my troubles have been due to a lack of XPCOM knowledge, not any technical issues with COM Interop. I haven't worked out all the issues, but below is a little test case I am working on. (side note: there is a serious lack of decent, standalone XPCOM samples/tutorials available on the web IMO)


class XPCOMTester
{
[STAThread]
static int Main (string[] args) {
   int hr = NativeMethods.XPCOMGlueStartup (IntPtr.Zero);
   nsIServiceManager sm;
   hr = NativeMethods.NS_InitXPCOM2 (out sm, IntPtr.Zero, IntPtr.Zero);

   nsIComponentRegistrar cr = (nsIComponentRegistrar)sm;

   cr.autoRegister(IntPtr.Zero);

   Guid kEventQueueServiceCID = new Guid("be761f00-a3b0-11d2-996c-0080c7cb1080");
   IntPtr ptr;
   sm.getService(kEventQueueServiceCID, typeof(nsIEventQueueService).GUID, out ptr);

   nsIEventQueueService eqs = (nsIEventQueueService)Marshal.GetObjectForIUnknown(ptr);
   eqs.CreateThreadEventQueue();

   nsIEventQueue eq = eqs.GetThreadEventQueue(IntPtr.Zero);

   Guid IPC_SERVICE_CID = new Guid("9f12676a-5168-4a08-beb8-edf8a593a1ca");
   sm.getService(IPC_SERVICE_CID, typeof(ipcIService).GUID, out ptr);
   // or
   // sm.getServiceByContractID("@mozilla.org/ipc/service;1", typeof(ipcIService).GUID, out ptr);

   // Do more XPCOM stuff here
   // 

   NativeMethods.NS_ShutdownXPCOM(sm);
   NativeMethods.XPCOMGlueShutdown();
   return 0;
}

}

Interfaces are defined as follows (taking nsIServiceManager as an example):


[Guid("8bb35ed9-e332-462d-9155-4a002ab5c958")]
[InterfaceType (ComInterfaceType.InterfaceIsIUnknown)]
[ComImport()]
public interface nsIServiceManager
{
  [MethodImpl(MethodImplOptions.InternalCall, MethodCodeType=MethodCodeType.Runtime)]
  [PreserveSigAttribute]
  void getService ([MarshalAs (UnmanagedType.LPStruct)] Guid clsid,
     [MarshalAs (UnmanagedType.LPStruct)] Guid iid,
     out IntPtr result);

  [MethodImpl(MethodImplOptions.InternalCall, MethodCodeType=MethodCodeType.Runtime)]
  [PreserveSigAttribute]
  void getServiceByContractID ([MarshalAs (UnmanagedType.LPStr)] string contractID,
  [MarshalAs (UnmanagedType.LPStruct)] Guid iid,
  out IntPtr result);

  [MethodImpl(MethodImplOptions.InternalCall, MethodCodeType=MethodCodeType.Runtime)]
  [PreserveSigAttribute]
  PRBool isServiceInstantiated ([MarshalAs (UnmanagedType.LPStruct)] Guid clsid,
  [MarshalAs (UnmanagedType.LPStruct)] Guid iid);

  [MethodImpl(MethodImplOptions.InternalCall, MethodCodeType=MethodCodeType.Runtime)]
  [PreserveSigAttribute]
  PRBool isServiceInstantiatedByContractID ([MarshalAs (UnmanagedType.LPStr)] string contractID,
  [MarshalAs (UnmanagedType.LPStruct)] Guid iid);
}

The above samples does have one problem at this point; the all to GetThreadEventQueue fails with a class not registered error. Again, I think this is my XPCOM ignorance I am fighting. If anyone actually has any interest in using mono and XPCOM, or has any advice, please send it my way.

Also, I have started working on a Mono.XPCOM interop library that would contain definitions of standard XPCOM interfaces like the ones used above. Any advice as to where to put this code (google)?

Three COM Interop Updates

2007-02-26T12:22:00.000-05:00

COM Callable Wrappers

First, COM Callable Wrapper support is now in svn. This means that managed objects can be passed to C++ and interacted with as if they were unmanaged COM objects. It still needs some more work, but most basic use of this functionality should work.

User Base Doubles

Previously, I seemed to be the only person interested in using COM Interop, especially on Linux. However, I had an exchange with someone on IRC last week who is trying to port their C#/C++ COM application from Windows to Linux. They are investigating using COM Interop instead of having to write a C wrapper layer with pinvokes.

Mozilla/XPCOM

Theoretically, the COM Interop functionality should work with Mozilla/XPCOM or any COM like system at least on Windows and Linux on x86 and x86-64 architectures (COM Interop depends on the binary layout of vtables). I always planned on trying COM Interop on XPCOM but never had the time. This weekend I took an hour or so and tried to hack something together. Since Monodevelop has an ASP.Net designer that uses Mozilla and one of the future goals is better DOM access, I decided to hack something together inside of the editor.

The good news is that it works! I can traverse/access the DOM without any glue/wrapper library. I just need to define some interfaces in C# and get a pointer to the DOM. The bad news is that strings in Mozilla/XPCOM really stink. I couldn't figure out how to marshal the strings correctly, thus making the exercise less than complete. If anyone knows what a DOMString is, please post a comment. If it ends up being a fairly simple type it can be marshalled that would be great, however if it's just a C++ class (as I currently fear) some unmanaged glue code will be needed to convert the string to something usable by the marshaller.

The electric car resurrection

2007-01-08T08:40:00.001-05:00

The electric car has been back for awhile, but not in a form for mere mortals. I don't think most mortals could handle 0-60 in 4 seconds anyway.
However, it looks like mortals may get their shot. 40 miles on a charge (with a gasoline backup) means I could use it for traveling to/from work everyday without burning any gasoline. Let's see if GM can deliver and make it affordable to the middle class.

COM Interop - COM Callable Wrappers

2006-12-11T00:05:00.000-05:00

Support for Runtime Callable Wrappers (RCWs) has been in mono for a few months now. Support for COM Callable Wrappers (CCWs) is well underway. I have a patch pending, and am cleaning up some issues with it right now. CCWs basically allow you to safely expose managed objects to unmanaged code as COM objects.

I have to say the mono runtime is some cool code to hack around in; I also have to say thanks to the runtime hackers who continually help me out. Most of my time is spent translating the rules of a COM environment to the rules of the managed runtime. Essentially all of the technical functionality I need (getting a function pointer for a managed method, parameter marshalling, etc.) is already there. All I've had to do is implement some COM specific stuff in the runtime, and always remember to adhere to COM rules (AddRef the pointer returned from a QueryInterface call) and runtime rules when needed (don't hold onto an object pointer directly as that will cause big problems with the new moving garbage collector).

So what's left to do? Once the CCW patch gets in, not much from my point of view. I don't have support for IDispatch complete yet for either CCWs or RCWs, so that will get some attention. Beyond that it's bugs, performance, and what anyone requests (assuming someone will use this). And who would use this? I can think of a few cases:

This may be a good way to interact with Mozilla/XPCOM components from mono. I haven't tried this yet as my skills at getting a working Mozilla build/development setup are not very good. My goal is either to get the System.Windows.Forms WebBrowser control working, or try to get the ASP.Net Designer to work using COM Interop to communicate with Mozilla.
This may also be a good way to interact with Open Office. I think this has less potential than XPCOM, but I'm not ruling anything out yet.
In general, it seems to be an ok way to wrap a C++ library if you don't want to use a C API created either manually or using one of the various tools available.

If anyone can think of other cool applications for COM Interop, or has suggestions let me know.

COM Interop in SVN

2006-07-15T08:41:00.000-04:00

I just finished committing the first round of patches for COM Interop. It's been about a year since I started investigating COM Interop. This is actually the third implementation of COM Interop I've written and by far the best. It has the most compatiblity with .Net's implementation, with the least amount of code of my three implementations.

What's working?
Well, most of the functionality of RCWs (Runtime Callable Wrappers) is implemented. That means using unmanaged COM objects as managed objects. Nothing is done yet for the opposite direction for CCWs (COM Callable Wrappers); that is using managed objects as COM Objects in unmanaged code. The other main limitation is that I don't have the object marshalling done yet. That means you can't pass a COM object as a parameter (or return value) to a method on another COM object; this should be fixed shortly though. However, you can create as many COM objects as you want in managed code, and use methods/properties whose parameters (return value) are simple types. That's enough to test basic COM objects, and do stuff like start Internet Explorer and navigate to a specific url.

What's next?
Anyway, now that the big initial patch is in, I hope it is easier to get smaller things done faster. So, be looking for fast progress in the next few weeks. I'd like to get RCW's completely done in the next month. After that, I'll either look at CCWs or else trying to extend COM Interop to work for XPCOM.

Something Real

2005-12-03T01:15:00.000-05:00

Well, I've received no comments on my blog yet. Obviously, the lack of pictures is the cause. In order to remedy that, I'm posting a screenshot of the COM Interop in mono doing something real. The mono runtime is launched through Visual Studio. It runs a simple C# program that creates an Excel application, displays it, and adds a workbook.

Spent alot of time working on VARIANT marshalling. And this little attribute caused me major pain trying to get the Excel sample working.

COM Interop in Mono - progress

2005-11-30T00:12:00.000-05:00

I have been making some progress in the COM Interop realm over the past couple of days. A COM object can be created in managed code. It can be cast to the various interfaces that it supports, and methods and properties can be used. All calls now go through the marshaller, with support for BSTRs and COM interfaces. That leaves VARIANTs and SAFEARRAYs. There is a few other TODOs, but I believe the only one that would directly affect a user would be support for connection points.

As for XPCOM, I've been doing some research. MS COM is a binary standard. XPCOM does not seem to be a binary standard due to it's cross platform nature. Thus, I can't rely on the vtables being formatted like in MS COM. So, the consensus seems to be that the xptcall is the way to go for the XPCOM interop. Not sure about interop assemblies for XPCOM yet(they can of course be hand coded). Also, haven't figured out if XPCOM interop should use the same attributes that MS COM uses, or if not where the new attributes go.

After all that comes the other direction, i.e. unmanaged code using managed components as COM objects. Haven't actually done any coding for it, but I think it will be pretty easy. The marshalling code is the same as the above. Add a little bit of plumbing code and the basics should be there. I'd post a screenshot, but haven't figured out what to demo/show. Plus, my hope is that this will make it into mono in the near future anyway.

COM Interop in Mono (part 1)

2005-11-27T04:01:00.000-05:00

The past few months I have been looking into supporting COM Interop in Mono. Not just for Microsoft COM on Windows, but hopefully XPCOM on Windows and Linux, as well as COM ported to Linux by third parties (Mainsoft, for example). I initially tried to do everything in managed code and did come up with a solution; but it was ugly and lacked some of the functionality of MS's implementation. So, I took the plunge and began looking into the mono runtime.

COM Interop is a large topic. The definitive book for COM Interop, .Net and COM: The Complete Interoperability Guide, is a solid 1500+ pages). So I'll skip any general explanation, and instead focus on the implementation in mono. COM Interop is bidirectional. It provides a mechanism to expose unmanaged COM objects to managed clients. The managed wrapper is called a runtime callable wrapper (RCW). It also provides a way to expose managed objects as COM objects to unmanaged clients. The wrapper exposed to unmanaged code is called a COM callable wrapper (CCW). My initial focus has been on the former.

Currently, I am using the same interop assemblies that MS does. An interop assembly is generated from a COM type library, and essentially converts COM type information into metadata. This interop assembly is what is referenced by managed code. This assembly contains managed interfaces that correspond to COM interfaces, as well as managed classes that correspond to COM coclasses. The methods on each interface are in the same order as they are in the COM interface. Thus, we can determine the layout of the COM interface vtable. The usage of COM objects in managed code can be divided into two cases.

The first case is when an class in the interop assembly is created in managed code. When the user creates a class that is marked with the ComImportAttribute (all classes and interfaces in the interop assembly are marked with this attribute), extra space is reserved in the class for a pointer to the unmanaged COM object.

All methods on the RCW are marked as internal calls. When the runtime tries to resolve the internal call in mono, instead of looking it up in the internal call tables, a trampoline (forgive me if I'm using this term wrong) is emitted that will call the method on the underlying COM object. That trampoline first calls a helper method and passes it the MonoMethod and the MonoObject that the call is for. The pointer to the COM object is obtained from the MonoObject (stored in the extra space reserved for the pointer). Next, the interface that the method was defined on is determined. The GuidAttribute on this interface is used to call QueryInterfaces on the COM object. The returned interface pointer is the correct 'this' pointer for the unmanaged function. Then, the offset of the method in the interface is determined. This offset is used to obtain the function pointer via the vtable of the COM object. That function pointer is called using a stdcall calling convention with the 'this' pointer pushed as the first argument on the stack. Most COM methods return an HRESULT (int) that indicates success/failure. This will be translated into a managed exception.

The second case occurs when an interface pointer is returned from a method/property, or when a RCW is cast to an interface that is not listed in the metadata. RCW's are special in that a cast can succeed to an interface other than those listed in the class's metadata. The runtime calls QueryInterface on the underlying COM object, and if that call succeeds the runtime allows the cast to occur. At this point, the runtime knows nothing of the COM object's identity except that it supports the given interface. The managed object's type becomes a generic type that wraps COM objects, System.__ComObject.

To handle this in mono, a solution was built on top of the remoting infrastructure. First, a new internal class was defined, System.ComProxy, that derives from System.Runtime.Remoting.Proxies.RealProxy. The constructor for the ComProxy class takes an IntPtr argument, the pointer to the IUnknown interface of the COM object. System.ComProxy also implements System.Runtime.Remoting.IRemotingTypeInfo, which allows for special handling of casting via the CanCastTo method. When a COM interface pointer is returned, a new instance of ComProxy is created for the type System.__ComObject. Then a call to GetTransparentProxy is made. The transparent proxy (tp) object that is returned is cast to the expected managed interface. This in turn causes a call to CanCastTo, which calls QueryInterface on the COM object to determine whether the target interface is supported. If the interface is supported, the remoting infrastructure dynamically adds the interface and its methods to the proxy's vtable. Any method calls on this interface are now handled as in the previous case.

And there it is. Too much information for the casual reader, and not enough for those of you who care. Code will hopefully follow shortly, as I have time.

Beginning of the End

2005-11-23T15:14:00.000-05:00

Yes. I now have a blog.