In May of 2009 I discovered some significant performance problems that I blogged about. In summary, I had tracked it down to the issue of LINQ2SQ having to create a full expression tree on every instantiation of a LINQ2SQL query. I’m not a compiler write kind of guy but do respect the complexity of that and doing things like building expression trees, but still this really sucked. Using the compile syntax in LINQ2SQL is very awkward, and IMHO takes all the fun out of using LINQ2SQL. If you don’t remember my post, here is the graph showing the evil happening.
So, what is a Microsoft MVP to do? I complained to everyone at Microsoft I knew. I made this my mission for about a year. I was told time after time that the problem was hugely complicated and no one was working on making it any faster. I just could not believe it because I felt this was such a barrier to success to LINQ2SQL type technology (now Entity Framework really) that Microsoft just could not ignore my findings. I was actually surprised that I was the only one screaming about this.
Well, quietly, Microsoft has announced that in .Net Framework 4.5, Entity Framework will include “Auto-Compiled LINQ Queries”. This is awesome! When it comes out, I’ll be testing and it giving feedback.
For now, I’m a very happy camper.
Over the past couple years, the focus of the web development I’ve been doing involves building highly flexible, highly scalable and straight forward web sites to implement and maintain Line of Business (LOB) applications. As you can probably tell from my posts, I’m very “practical” focused, and at the same time have a desire to build awesome web applications.
The technology pairing I’ve chosen is Microsoft’s .Net platform with MVC on the server, and ExtJS on the client. Though it’s possible to still use ExtJS with standard html/aspx pages, I’ve found the best combination is to use 100% JavaScript on the client (ExtJS) and have all the server side technology be 100% service based. I’ve used LINQ2SQL extensively as well as Entity Framework in the latest Visual Studio 2010 release.
The learning curve was quite steep to actually be able to efficiently build highly flexible, highly scalable applications using these technologies, but now that I know it, I wouldn’t have it any other way.
I’m considering putting together a series of 4 Day Classes around the country (or even world) that would basically teach people the methods and patterns I’ve learned and essentially leap frog a development team into being able to quickly do what it has taken me years to figure out. I’ve been fortunate enough to know the top 1% instructors and I’m sure with the right incentive, can get them to join me in both putting together these classes as well as teaching them.
(more…)
Just a quick post in case anyone is wasting 10 minutes figuring out how to do this. For me, this came up because in Sql Server 2008, NVARCHAR(MAX) is 4000 characters and I needed more. The recommended datatype to use for more is VARBINARY. When I did that, LINQ2SQL converted that type to Linq.Binary. (using encoding)
So, in my C# code, here is what you need to convert to the Linq.Binary:
(more…)
I often speak very highly of LINQ and also LINQPad. This morning, I was struggling with some sql that would let me do a count by DateTime while stripping out the time portion. That is, I just want to know how many entries are in the table for each Date (regardless of what time). I tried lots of solutions I got from search, and they all gave not correct results, usually involving Casting and other non fun sql programming constructs.
(more…)
So, I’m really enjoying using LINQ and specifically LINQ2SQL. I’ve got a current problem where I want to get from a long list of loads (potentially hundreds per day), a short list of days in reverse order, that have a certain number or more of loads.
(more…)
We’ve built a data access layer on top of LINQ2SQL for dynamically building the layer we call for data access. It’s convenient because we pass in a query object as a parameter that has a bunch of nullable variables in it. Here is kind of what it looks like:
public partial class CodeGenTestQuery : QueryBase
{
public int? Id { get; set; }
public List<int> Ids { get; set; }
public bool? IsStarred { get; set; }
...
If I pass an empty list of int’s into Ids, I have a line of code that build the query as follows:
(more…)
Last night, I had the honor of having dinner with three other VSLive speakers including Jim Wooley, one of the authors of the awesome book on LINQ titled LINQ in Action. Jim is a wealth of knowledge as well as very entertaining. He had an interesting analogy for what using try/catch statements to handle expected issues. I’ll leave it up to you to post to his twitter account and find out what it was. it’s just a little off color and I don’t want to be rejected from search engines.
(more…)
*Note 10/22/2011 (2+ years later): Microsoft is fixing this problem in EntityFramework 4.5! See my post here about it: http://peterkellner.net/2011/10/22/microsoft-to-add-auto-compiled-linq-queries-to-entity-framework-v-next/
So, I’ve been on kind of a rant lately about how slow LINQ2SQL is if you don’t compile your queries before executing them. To be fair, if you are doing Windows Forms Programming, WPF or Silverlight it really does not have much impact. The reason is that a very complex LINQ query may take 50 milliseconds (1/20th of a second). No big deal if you just have a dozen or so of them to do. The story changes though if you are using LINQ2SQL in a web environment that has limited CPU resources. That is, unless you have unlimited money, if it takes more than one web server to handle your load, your throwing away money by using uncompiled LINQ2SQL.
So, to put some more substance behind my claims, I’ve written a small test application using Visual Studio 2008 that compares the performance of using LINQ2SQL compiled verses non-compiled on a trivial web page.
(more…)
I’ve been doing a lot of LINQ2SQL lately and just in case I have not said it loud enough how enthusiastic I am about LINQPAD written by Joseph Albahari. I have the $19 version which has intellisense, but even without, it would be totally awesome.
(more…)
So, I recently blogged about the huge penalty for not compiling your LINQ2SQL. This problem is so big that it occurred to us that maybe all of LINQ has the problem. So, time for a simple test. Below is a very simple program that basically generates a list of Ids. In one case, it’s using LINQ, and the other just Plain C#. The code is pretty self explanatory. Here are the results:
| Test Performed | Time to Do 100000 iterations |
| Using LINQ | 52ms |
| Using Simple C# | 35ms |
Well, my fears are put aside. Though LINQ is somewhat slower, for 100,000 iterations, .052 seconds is pretty good. (compared with processing a single not to complicated LINQ2SQL statement for 100,000 iterations would take about 20,000 seconds or 333 hours. Quite a difference to .052 seconds!
(more…)
Background
At my company, we have been running a workflow process that is taking 6 hours to complete. This is a new project so we’ve had the good fortune to be able to use the latest Microsoft technologies. We are a Microsoft shop, so that means Sql Server 2008, C#, ASP.NET, ExtJS, LINQ to SQL (hoping to move to Entity Framework someday) and others. After spending all day running performance analysis tools and basically performing a full audit of the 6 hour process, we sadly concluded that our time was being eaten up by LINQ to SQL processing. My experience has always been that anything you do on the compiled code side is usually overwhelmed by database access times, usually at least ten to one. Well, I learned a lot yesterday.
(more…)
