Response: “Is WCF faster than ASP.NET Core? Of course not! Or is it?”

Benchmarking code is tough, you have to understand the nuance, the setup to get it right. Which is why when I read Is WCF faster than ASP.NET Core? Of course not! Or is it? I was very curious, since the .Net Core team has spent a lot of time tuning Kestrel, how could this be possible?

Starting out, I ran the code from Github to see, and the WCF scores were the same, but the WebApi scores were noticeably already lower. Only change was I moved the project to .Net 4.6.1, since I have been doing exclusively .Net Core development. The processor is a Ryzen 1700, which could lead to the difference.

First benchmark run – unmodified

Going into this benchmark, I knew the following things with my experience with WCF

  • WCF Internally keeps a series of connection, similar to SQL connection pooling. If your channel(socket) faults, you have to do the try { client.Close();} catch { client.Abort();} pattern or you lose available channels until your app pool recycles.
  • HttpClient is thread safe and i s recommended as a singleton while most developers instiate it once per a function call then dispose of it. Best practices is one instance of HttpClient per remote server.
  • Benchmarks are tough.

Looking over the code, I see that the WCF code is straight forward, creates a ChannelFactory, a switch statement for the different format types, and a simple synchronous Invoke call to the WCF operation. Wait….we created a client to load test the WCF service, Web API doesn’t have a default client, how does that work.

Apples and Oranges

While the WCF benchmark was using the built in .Net client, the Web Api project was using a HttpClient object, that was instantiated via the constructor (not really the constructor but the generated on build for variables that have a value assigned). Ok, switch that guy to a static, run the test, 20us drop, not much, but room for improvement. Also the WebApi calls have the same switch statement but uses a Func<> to run the function, those are just fancy delegates, shave that off, now we are down to 50us, first test now is at around 900us. Better, still strange. The URL was also being created (and allocated) on each call, moved that to the constructor as well, shaved some time off too.

This will add to each benchmark run versus formatting the string once.

Alright, let us crack open Fiddler and see if we can get some timings out of it. This, was all bad, trying to proxy requests and reconfigure Kestrel, not great. I was stumped, I remember load testing, with JMeter, a .Net Core microservice where I was able to get it to return faster than a straight SQL call for the same data, then maxed out a F5 Load Balancer doing it, I never had that with WCF, what is the deal?

When writing benchmarks, benchmark one thing, not two

I removed the async/await, it probably didn’t help that much.

Finally, I remember in my current project, I use “StringContent” with HttpClient, not this “PostAsJsonAsync” method, turns out that is an extension method. Great, maybe that is it? Ripped it out and for a last resort, just set the content to “[]” faking an empty JSON string.

WHAT JUST HAPPENED. The numbers shot down to the 400us range, that couldn’t be. JSON serialization can’t be that bad these days.  Ok, swap out the “[]” with a real JSON call to “JsonConvert.SerializeObject”, wait, that is from the Newtonsoft package, hmmmm. This is running on the full .Net Framework, I bet it is using the built in Javascript serializer (checked, yep) which now everyone favors the Newtonsoft.

 

 

Great, now the different is down to 120us for ItemCount of 0 and almost the same for item count of 10. All we changed was the client, we made one tiny change to the startup of Kestrel to remove some extra calls. I gave myself a time block on this, and ran past it. I feel good about where the test went.

Benchmark the server, not both

This benchmark test was testing HttpClient, an extension method, a Javascript seralizer, string formating, reflection (the type name call), and a lambda function, when the point was to see which server framework is faster. This is where tools like JMeter excel, same client, but can load test two different servers to truly test just the server. You still have to make sure your local CPU doesn’t have any contention issues, and your processor isn’t using, or can’t use, a turbo state for that one core. The lesson here is to be careful, and please peer review some of these benchmarks before making bold claims.

 

Wait, is WCF faster?

I won’t say, because I would need the time to setup a proper environment. The numbers for this benchmark are damn close now, and we are now 110us or 0.11 milliseconds difference.  0.11 millisecond is the amount of time it takes light to travel ~20.5 miles (3.0*10^8*0.0001s). In comparison CloudFlare tested two servers connected via 10gb links to one 10gb switch, latency was an average of 60us. Yeah, this is now in the “You have bigger issues to performance tune” category.

 

How to Diagnose a Test Failure on TeamCity

Have you ever had that unit test that works locally, but fails on team city. Turns out you can debug them and find out that it isn’t team city’s fault.

Failing Team City Builds

You run the test locally, pass, run them on team city, fail. Alright, time to get serious, 1, pull the TeamCity artifact down locally, 2, run MSTest from command line like TeamCity does, 3, start debugging. Most unit test frameworks throw exceptions when a test fails, great way to debug into the test.

Make sure you have an artifact from TeamCity or you can’t pull down the test. Well, unless you are an admin on the box.

TeamCity Test Artifact

TeamCity Download Artifact

Visual Studio Attach to Process

Open up Visual Studio Command Prompt for VS2015.

[code]
mstest /testcontainer:MyTests.dll
[/code]

Next, attach to the MSTest process QUICKLY. MSTest launches QTAgent32.exe for running the actual unit tests but it won’t stop to wait for you.

Attach To Process QTAgent32.exe

In the end, it turned out another test ran first, and our second tests couldn’t set a private static readonly because of its if null check.

[code langauge=”csharp”]
private static readonly List<int> Codes = new List<int>();
[/code]

Benchmarking MVC6 Beta 8

I have always been curious how fast MVC can be, and a previous article (http://blog.bitdiff.com/2012/06/performance-comparison-iis-75-and-iis-8.html) detailed the Web API  speeds with IIS 7.5 to IIS 8 along with self-hosted options. So I spun up a MVC 6 web  app with Beta-8 and used the Apache BenchMark to test it out. Running a i7-3740QM CPU with Windows 7.

My controller contained only this,

[csharp]

[Route("api/[controller]")]
public class ValuesController : Controller
{
// GET: api/values
[HttpGet]
public IEnumerable<string> Get()
{
return new string[] { "value1", "value2" };
}
}

[/csharp]

I fired up Apache Benchmark using the command below.

[code]ab -n 1000 -c 2 http://localhost:5000/api/Values [/code]

3.064MS! It beats out the 17.818 from Windows 8 Self Host and I am running Windows 7! This is a localhost call, so adding in SSL or a network would likely increase this. but it helps if it starts really low first.

The first “Time per request” is for each request * concurrent request number. The second “Time per request” is the actual individual requests. Wonderfully explained at this ServerFault question http://serverfault.com/questions/274252/apache-ab-please-explain-the-output

mvc6-beta8-batch1

For giggles, I pumped it up to 10,000 concurrent requests, I also tried 5,000 concurrent requests and got the same ~7ms response time.

mvc6-beta8-crazyconcurrent

So basically MVC6 is FAST! VS2015 diagnostic session for those who wanted to take a peek. I might have had around 100 chrome tabs open when running this benchmark.

 

 

vs2015 diagnostic session

 

Entity Framework MSSQL Sequence Numbers

I had an application where I wanted to have a sequence key that spanned multiple tables. Mainly to provide better logging support, so the log search would be “Find Object X” instead of “Find object X with type Y”. Then add in namespacing and the logs start to get wide, IE “Find Object 12 with type Company.Product.Service.Model.ModelName” repeated hundreds of times.

Step 1, make sure your object in Entity Framework is set to a StoreGeneratedPattern of “None”. (Right click table, select properties, go to the property window)

Entity Framework Properties

Change your Entities partial class to include this override to save changes.

[gist id=”400bb4513848ef85bf3a” synhi=true lang=”csharp”]

And done!

 

Failed to start shard

I have a project that uses ElasticSearch for searching and all the goodness that it has. When I test a new technology, I like to break it a few times.  Like adding a bunch of data, hitting the stop button on the service or killing the task in task manager.

So I did this with ElasticSearch, and I received this error message.

failed to recover shard]; nested: ElasticsearchIllegalArgumentException[No version type match

Everywhere that I read included some details about how the disk filled up, or was low on space.

Capture

 

Simple solution, flush the index. ElasticSearch has an internal and on disk transaction log.  Being used to SQL’s ACID based consistency, you can’t shutdown the service without ill-effects. 8-7-2014 10-58-01 AM

 

More details on the flush command and how to access it from the API versus the HEAD plugin, http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/indices-flush.html