PeterKellner.net

In May of 2009 I discovered some significant performance problems that I blogged about.  In summary, I had tracked it down to the issue of LINQ2SQ having to create a full expression tree on every instantiation of a LINQ2SQL query.  I’m not a compiler write kind of guy but do respect the complexity of that and doing things like building expression trees, but still this really sucked.  Using the compile syntax in LINQ2SQL is very awkward, and IMHO takes all the fun out of using LINQ2SQL.  If you don’t remember my post, here is the graph showing the evil happening.

So, what is a Microsoft MVP to do?  I complained to everyone at Microsoft I knew.  I made this my mission for about a year.  I was told time after time that the problem was hugely complicated and no one was working on making it any faster.  I just could not believe it because I felt this was such a barrier to success to LINQ2SQL type technology (now Entity Framework really) that Microsoft just could not ignore my findings.  I was actually surprised that I was the only one screaming about this.

Well, quietly, Microsoft has announced that in .Net Framework 4.5, Entity Framework will include “Auto-Compiled LINQ Queries”.  This is awesome!   When it comes out, I’ll be testing and it giving feedback.

For now, I’m a very happy camper.

I’m constantly amazed by the insightfulness of ReSharper’s suggested refactorings (ReSharper is a Visual Studio Addin from JetBrains I use with C#). Today, I’ve been working on a threading problem where I’m getting crashes based on what seems like not proper locking across threads (they usually show up as some type of ugly update object or enum error).

My code starts like this:

public static List<DbProgressReport> DbProgressReportProperty { get; set; }

In this post, we’ll take a straight forward procedure based set of code and convert it to LINQ using a ReSharper from JetBrains suggestion.   I’ve found that in general, when I do things with foreach syntax, there is often a better way in Linq to do this.  The better does not jump out at me sometimes, however with ReSharper, it is often a button click away.

Here is a shorthand way for converting a list of strings defined as follows:

 List<string> strings = new List<string>
 
	{
		"a","b","c"

	};

To something that looks like:

In this section, we will talk about what happens when the users presses the “Sessions” hyperlink from the speakers page.  If you remember from the last article, we used a special converter on the Speakers.xaml page and bound that to the hyperlink titled sessions as shown below.

This one is just to amazing to not blog about.  I’ve been a die heart ReSharper user for quite a while and recently have started using their early access versions.  Primarily because they seem to have added a lot of LINQ support which I use heavily now.  I had the following code before the refactor.  I have to admit, it’s a little .Net 1.1/2.0 ish, but what can I say, I’m a production coder and it did the job.

private static int GetTransitTime(string parcelToCheck)
{
    int transitTime = 0;
    foreach (var parcelSCAC in Constant.UPS_SCAC_MAP)
    {
        if (parcelSCAC.Key.Equals(parcelToCheck))
        {
            transitTime = Constant.UPS_TRANSIT_TIME[parcelSCAC.Value];
            break;
        }
    }
    return transitTime;
}

Just a quick post in case anyone is wasting 10 minutes figuring out how to do this.  For me, this came up because in Sql Server 2008, NVARCHAR(MAX) is 4000 characters and I needed more.  The recommended datatype to use for more is VARBINARY.  When I did that, LINQ2SQL converted that type to Linq.Binary. (using encoding)

So, in my C# code, here is what you need to convert to the Linq.Binary:

This is just going to be a short post, but I bet it’s something I do a large number of times so I thought I’d blog it.  Say you get back from something like a web service an array of objects.

I often speak very highly of LINQ and also LINQPad.  This morning, I was struggling with some sql that would let me do a count by DateTime while stripping out the time portion.  That is, I just want to know how many entries are in the table for each Date (regardless of what time).  I tried lots of solutions I got from search, and they all gave not correct results, usually involving Casting and other non fun sql programming constructs.

I’ve done a bunch of Lat/Long type mapping programs over the years and one of the problems always ends up around performance.  A common problem is that you keep redrawing the same line over and over and over.  The pixels you are drawing don’t get any darker so all you are doing is wasting time.  In my current project, we were drawing approximately 1500 lines when we really only had about 150 unique lines.  I’ve always known how to solve this problem with a bunch of thrown together hacked up code, but now, LINQ gives me a very clean way to do it.

So, the problem is you have a record that looks like this:

 public class LoadSegmentInfoFlat
        {
            public double? OriginLattitude { get; set; }
            public double? OriginLongitude { get; set; }
            public double? DestinationLattitude { get; set; }
            public double? DestinationLongitude { get; set; }
            public string Color { get; set; }
        }

So, I’m really enjoying using LINQ and specifically LINQ2SQL.  I’ve got a current problem where I want to get from a long list of loads (potentially hundreds per day), a short list of days in reverse order, that have a certain number or more of loads.

We’ve built a data access layer on top of LINQ2SQL for dynamically building the layer we call for data access.  It’s convenient because we pass in a query object as a parameter that has a bunch of nullable variables in it.  Here is kind of what it looks like:

    public partial class CodeGenTestQuery : QueryBase

    {

        public int? Id { get; set; }

        public List<int> Ids { get; set; }

        public bool? IsStarred { get; set; }

        ...

If I pass an empty list of int’s into Ids, I have a line of code that build the query as follows:

Last night, I had the honor of having dinner with three other VSLive speakers including Jim Wooley, one of the authors of the awesome book on LINQ titled LINQ in Action.  Jim is a wealth of knowledge as well as very entertaining.  He had an interesting analogy for what using try/catch statements to handle expected issues.  I’ll leave it up to you to post to his twitter account and find out what it was.  it’s just a little off color and I don’t want to be rejected from search engines.

*Note 10/22/2011 (2+ years later):  Microsoft is fixing this problem in EntityFramework 4.5!  See my post here about it: http://peterkellner.net/2011/10/22/microsoft-to-add-auto-compiled-linq-queries-to-entity-framework-v-next/

I’ve been doing a lot of LINQ2SQL lately and just in case I have not said it loud enough how enthusiastic I am about LINQPAD written by Joseph Albahari.  I have the $19 version which has intellisense, but even without, it would be totally awesome.

So, I recently blogged about the huge penalty for not compiling your LINQ2SQL.  This problem is so big that it occurred to us that maybe all of LINQ has the problem.  So, time for a simple test.  Below is a very simple program that basically generates a list of Ids.  In one case, it’s using LINQ, and the other just Plain C#.  The code is pretty self explanatory.  Here are the results:

Well, my fears are put aside.  Though LINQ is somewhat slower, for 100,000 iterations, .052 seconds is pretty good.  (compared with processing a single not to complicated LINQ2SQL statement for 100,000 iterations would take about 20,000 seconds or 333 hours.  Quite a difference to .052 seconds!

So, this is very straight forward, but I sometimes forget it.  I figure I’ll do a short blog post on it so next time I search for it, I’ll probably hit my own blog post.

So, say you have a list of objects as follows:

Background

At my company, we have been running a workflow process that is taking 6 hours to complete.  This is a new project so we’ve had the good fortune to be able to use the latest Microsoft technologies.  We are a Microsoft shop, so that means Sql Server 2008, C#, ASP.NET, ExtJS, LINQ to SQL (hoping to move to Entity Framework someday) and others.  After spending all day running performance analysis tools and basically performing a full audit of the 6 hour process, we sadly concluded that our time was being eaten up by LINQ to SQL processing.  My experience has always been that anything you do on the compiled code side is usually overwhelmed by database access times, usually at least ten to one.  Well, I learned a lot yesterday.

Andrew Siemer has written an excellent book about how to best use .Net (specifically with asp.net as the front end web technology) to build a social networking site.  He literally starts at the beginning and builds service by service, screen by screen, data table by data table what is necessary to build the site.  A lot of Andrew shows is not only just web type stuff, but he shows many best practices for building .Net applications in general.

Among other things, the goes in to great detail on the following topics:

So, you want to do an update but are wondering what the hec LINQ is doing.  Turns out it is really easy.  All you have to do is run in the debugger and add the Log option to your data context.

Here is an example:

DataClassesGeneralDataContext db3pLogicContext;

db3pLogicContext.Log = Console.Out;

var companyQuery = from tbl in db3pLogicContext.Companies

                   where tbl.ParentId != 0

                   select tbl;

int totalCntParents = companyQuery.Count();

foreach (DBAccess.Company co in companyQuery)

{

    co.CreateDate = DateTime.Now;

    break;

}

db3pLogicContext.SubmitChanges();