Rx – Awaiting events & observables in C# 5

So everyone is now using C# 5 to write asynchronous methods to await tasks right ;)

Did you know you can also await other things? Like events & observables!?

The following code demonstrates how you can “await” items being added to an observable collection.

using System;
using System.Collections.ObjectModel;
using System.Collections.Specialized;
using System.Reactive;
using System.Reactive.Linq;

namespace ConsoleApplication11
{
    class Program
    {
        private static readonly ObservableCollection<int> Collection = new ObservableCollection<int>();

        static void Main()
        {
            Test();
            Collection.Add(42);
        }

        public static async void Test()
        {
            Console.WriteLine("awaiting event...");
            var itemsAdded = await
                (
                    from collectionChanged in Collection.ToNotifyCollectionChangedObservable()
                    where collectionChanged.EventArgs.Action == NotifyCollectionChangedAction.Add
                    select collectionChanged.EventArgs.NewItems
                )
                .Take(1);

            Console.WriteLine("items added;");
            foreach (var item in itemsAdded)
                Console.WriteLine(item);
        }
    }

    public static class Extensions
    {
        public static IObservable<EventPattern<NotifyCollectionChangedEventArgs>>
            ToNotifyCollectionChangedObservable(this INotifyCollectionChanged source)
        {
            return Observable.FromEventPattern<NotifyCollectionChangedEventHandler, NotifyCollectionChangedEventArgs>
                (h => source.CollectionChanged += h,
                h => source.CollectionChanged -= h);
        }
    }
}

Enjoy!

*UPDATE* make sure you have Rx-experimental release. The GetAwaiter method is not in the stable release yet!

 

Pop Quiz: Asynchronous Methods & Contract Errors

This is for people wanting to expand their asynchronous brains!

Pop Quiz

Here is a question for you; What will this seemly trivial program segment do?

void X()
{
    try
    {
        Y();
    }
    catch (Exception ex)
    {
        Console.WriteLine(ex.Message);
    }
}

async void Y()
{
    throw new NotImplementedException();
}

Before you answer, consider another question; Is there a behavioral difference between these two methods?

IEnumerable<int> J()
{
    throw new NotImplementedException();
}

IEnumerable<int> K()
{
    throw new NotImplementedException();
    yield return -1;
}

Don’t worry some Rx posts are coming soon!

Reactive Extensions utilizing 64 cores

Saw this on Facebook… welcome to the future;

image

Hostile to friendly type names

Have you ever written some code that dumps type names using reflection and run into annoyances like this?

Query
new[]
{
    typeof(List<int>),
    typeof(List<List<int>>),
    typeof(int?),
    typeof(bool),
}.Select(t => t.Name).Dump("hostile type names");
Output

image

 

This can be problematic if you are trying to generate C# code via T4 templates or something similar.

A colleague and I recently solved this problem using the C# CodeDom (handy extension method included).

Query
new[]
{
    typeof(List<int>),
    typeof(List<List<int>>),
    typeof(int?),
    typeof(bool),
}
.Select(t => new
{
    HostileName = t.Name,
    FriendlyName = t.GetFriendlyName()
})
.Dump("hostile -> friendly type names");
Output

image

 
Extension method
public static class TypeEx
{
    public static string GetFriendlyName(this Type t)
    {
        using (var provider = new CSharpCodeProvider())
        {
            var typeRef = new CodeTypeReference(t);
            return provider.GetTypeOutput(typeRef);
        }
    }
}
Posted in .NET, C#. 4 Comments »

Structs that implement IEnumerator

OK. Lets just revise the query we had in the previous post. I’m going to change it so that is finds enumerators that are also value types;

from f in Directory.GetFiles (
       Path.Combine (
             Environment.GetFolderPath (System.Environment.SpecialFolder.ProgramFilesX86),
             @"Reference Assemblies\Microsoft\Framework\.NETFramework\v4.0"),
       "*.dll")
where !f.ToLowerInvariant().Contains ("thunk")
where !f.ToLowerInvariant().Contains ("wrapper")
select Assembly.LoadFrom (f) into a
from t in a.GetExportedTypes()
where t.IsValueType && typeof(IEnumerator).IsAssignableFrom (t)
select new { a.GetName().Name, t.FullName }

Running the query in LINQPad will yield the following results.

image

It would seem like a CRAZY! thing to do given that an enumerator, is by its very nature, mutable… I’ll try to reason why this might be beneficial; then we will look at why it sucks.

Possible Reasoning

I can think of two possible reasons why they might have chosen to do this.

1. It could be considered a “feature” in that if you ever wanted to remember or save the current state of a collection’s enumerator, you could just copy it to another field. I’m going to rule this out, the behaviour isn’t consistent with other enumerators (arrays, non-generic collections and then later, iterator blocks).

2. Another possibility is performance. Perhaps the BCL team were looking to avoid the heap allocation caused by calling GetEnumerator.

It would seem like a “micro optimisation”; does that tiny heap allocation really cause an issue? I’m going to look at a popular design pattern that I think might expose the problem they were trying to solve.

The Composite Design Pattern

Consider a C# implementation of the Composite pattern;

// Composite design pattern
class Node
{
    public readonly ArrayList Children = new ArrayList();

    public void Recurse()
    {
        foreach(Node child in Children)
        {
            child.Recurse();
        }
    }
}

Now lets consider a simple usage (1000 nodes with 1000 children).

var root = new Node();
for (int i = 0; i < 1000; i++)
{
    var child = new Node();
    for (int j = 0; j < 1000; j++)
    {
        child.Children.Add(new Node());
    }
    root.Children.Add(child);
}

var before = GC.CollectionCount(0);
root.Recurse();
var after = GC.CollectionCount(0);
Console.WriteLine("Collection Count: " + (after - before));

OUTPUT

image

That’s right 7 GCs just to recurse our composite tree structure! Yikes!

If we change from ArrayList to List<object>, you can see the difference.

image

Is this a realistic scenario? Can anyone think of a LARGE instance of the composite design pattern present in many of today’s .NET applications? I’ll give you a clue, it starts with W… and ends in PF.

WPF & Silverlight are text book implementations of the composite design pattern; can you imagine how many times the tree is traversed in this manner? Can you imagine how deep the tree is for a complex user interface? In my mind, this is a good theory for explaining why, at the very least the “presentation core” enumerators have been implemented in this manner. I think most developers would have implemented a composite design pattern using List<T> at some stage or another.

Possible Problems

So is it likely that someone would trip over this optimisation? Lets start with the obvious ones before moving to hell.

List<int>.Enumerator e1 = new List<int>{1,2,3,4,5}.GetEnumerator();
List<int>.Enumerator e2 = e1;
e1.MoveNext();
Console.WriteLine(e1.Current);
Console.WriteLine(e2.Current);

This will output 1 & 0. The team probably decided to take this hit, after all any normal person would write this.

IEnumerator<int> e1 = new List<int>{1,2,3,4,5}.GetEnumerator();
IEnumerator<int> e2 = e1;
e1.MoveNext();
Console.WriteLine(e1.Current);
Console.WriteLine(e2.Current);

This causes the value type to be boxed, meaning the “copy by value” semantics disappear and we get two references to the same enumerator, giving us the expected output of 1 & 1. But wait a second, most developers in .NET 3.5 / C# 3.0 will write this;

var e1 = new List<int>{1,2,3,4,5}.GetEnumerator();
var e2 = e1;
e1.MoveNext();
Console.WriteLine(e1.Current);
Console.WriteLine(e2.Current);

Uh oh! GetEnumerator returns List<int>.Enumerator meaning the inferred type is the value type, we’re back to square one!

I guess you could argue, well at least the problem is localised to this statement; it isn’t like people are passing these around from one method to another.

Oh wait, but what about the interference of generic type parameters! This is heavily leveraged by technologies like LINQ & Rx (NOTE: This is how I came up with my recent pop quiz!).

Consider the following;

    public class Wrapper<T> where T : IEnumerator
    {
        private readonly T enumerator;

        public Wrapper(T enumerator)
        {
            this.enumerator = enumerator;
        }

        public object Current
        {
            get { return enumerator.Current; }
        }

        public bool MoveNext()
        {
            return enumerator.MoveNext();
        }
    }

 

Impromptu Pop Quiz

I don’t want to box the enumerator… we only have one storage location.

private readonly T enumerator;

Will this wrapper work for enumerators that are value types?

Here is a complete program for you to play with.

using System;
using System.Collections.Generic;
using System.Collections;

namespace ConsoleApplication30
{
    class Program
    {
        private static void Main()
        {
            var w = Wrapper.Create(new List<int> { 1, 2, 3, 4, 5, 6, 7, 8, 9, 10 }.GetEnumerator());
            w.MoveNext();
            Console.WriteLine("Expected value: 1");
            Console.WriteLine("Actual value: " + w.Current);
        }
    }

    public static class Wrapper
    {
        public static Wrapper<T> Create<T>(T enumerator)
            where T : IEnumerator
        {
            return new Wrapper<T>(enumerator);
        }
    }

    public class Wrapper<T> where T : IEnumerator
    {
        private readonly T enumerator;

        public Wrapper(T enumerator)
        {
            this.enumerator = enumerator;
        }

        public object Current
        {
            get { return enumerator.Current; }
        }

        public bool MoveNext()
        {
            return enumerator.MoveNext();
        }
    }
}

Look forward to your replies!

Structs that implement IDisposable

A few weeks ago some friends and I were at the pub discussing value types that implement IDisposable. Consider the IObservable interface.

interface IObservable<T>
{
    IDisposable Subscribe(IObserver<T> observer);
}

One *might* consider using a disposable value type to take some pressure off the GC.

struct SlimDisposable : IDisposable
{
  // ....    
}

Unfortunately this line of thinking is flawed. The value type will be boxed as an IDisposable, resulting in the heap allocation we were trying to avoid. Lets pretend our interface looked more like this;

interface IObservable<T>
{
    SlimDisposable Subscribe(IObserver<T> observer);
}

Or maybe;

interface IObservable<TValue,TDisposable> where TDisposable : IDisposable
{
    TDisposable Subscribe(IObserver<TValue> observer);
}

Now we could reference SlimDisposable without it being boxed! But is this a good thing? Unfortunately creating disposable value types is fraught with danger!

Eric Lippert has a great post on the subject here; To box or not to box, that is the question

To quote MSDN: “To help ensure that resources are always cleaned up appropriately, a Dispose method should be callable multiple times without throwing an exception.”

This means that a disposable object usually needs to mutate & track some state to determine if it has been disposed or not. But we are using a value type this state will be copied whenever we do an assignment. Consider the following;

public struct Disposable : IDisposable
{
    public bool IsDisposed;

    public void Dispose()
    {
        if(!IsDisposed)
        {
            Console.WriteLine("Disposing Resources");
            IsDisposed = true;
        }
    }
}

Now lets say you did something like this;

var d1 = new Disposable();
var d2 = d1;
d1.Dispose();
Console.WriteLine("d1 == {0}, d2 == {1}", d1.IsDisposed, d2.IsDisposed);

OUTPUT

Disposing Resources

d1 == True, d2 == False

We now have two states & only 1 underlying resource!

Purely as an educational exercise, we decided to write a query that would find all the value types in the .NET framework that implement IDisposable. It turns out there are in fact quite a few

See for yourself;

from f in Directory.GetFiles (
       Path.Combine (
             Environment.GetFolderPath (System.Environment.SpecialFolder.ProgramFilesX86),
             @"Reference Assemblies\Microsoft\Framework\.NETFramework\v4.0"),
       "*.dll")
where !f.ToLowerInvariant().Contains ("thunk")
where !f.ToLowerInvariant().Contains ("wrapper")
select Assembly.LoadFrom (f) into a
from t in a.GetExportedTypes()
where t.IsValueType && typeof(IDisposable).IsAssignableFrom (t)
select new { a.GetName().Name, t.FullName }

Capture

We’ve got some interesting types here;

System.Thread.AsyncFlowControl

System.Thread.CancellationTokenRegistration

System.Windows.Threading.DispatchProcessingDisabled

Both CancallationTokenRegistration & DispatchProcessingDisabled get around the mutable state problem; they are in fact immutable value types. They use the state of a parent object (a reference type) to determine if they have been disposed or not. This means they can be assigned/copied safely.

AsyncFlowControl is in fact a mutable value type! It does however use some state on the tread that is references to determine if the control flow is suppressed or not. However it does mean you can do weird things like this;

var afc1 = ExecutionContext.SuppressFlow();
var afc2 = afc1;
afc1.Dispose();
var afc3 = ExecutionContext.SuppressFlow();
afc2.Dispose();

This will result in the flow being restored, even though we never disposed of afc3. The BCL developers may have decided that in this case, the performance and pressure on the GC was so critical, that they’d commit this sin. Transferring ExecutionContext information from one thread to another is probably considered somewhat of a performance hotspot. I suspect it is also the case that very few developers have ever used this API let alone know what an ExecutionContext is!

Anyway, the real point of interest here is, “oh my god, what are all these enumerators!”

Next time – Structs that implement IEnumerator.

More Reactive Extensions Performance Improvements

I blogged about performance when Rx was officially released a few months ago.

Last week the team released a new version (1.1.11111) where “The major focus of this release is performance-related work”.

Subject<T> now uses non-blocking synchronization!

Internally a subject has a list of subscribed observers. Traditionally, concurrent access to the list has been synchronised internally with a C# lock (Monitor). The latest release uses compare & swap (CAS) operation to modify it’s internal state. As OnNext doesn’t modify the internal state it doesn’t even need a CAS operation, it simply dispatches the notifications to all the observers.

Test code (from Rx team)

for (int n = 0; n < 10; n++)
{
    var c = new CountdownEvent(n);

    var s = new Subject<int>();
    for (int i = 0; i < n; i++)
        s.Subscribe(_ => { }, () => c.Signal());

    var sw = Stopwatch.StartNew();

    Scheduler.ThreadPool.Schedule(() =>
    {
        for (int i = 0; i < 100000000; i++)
            s.OnNext(42);
        s.OnCompleted();
    });

    c.Wait();
    sw.Stop();
    Console.WriteLine(sw.Elapsed);
}

Results

image

Reactive Extensions Extensions 1.2

 
What’s New?
  • Internalized or removed all interactive extensions that were unnecessary or that shadowed Microsoft’s Ix Experimental library.
  • Added DependencyObjectExtensions.DependencyPropertyChanged and UIElementExtensions.RoutedEventRaised extension methods and a corresponding lab.
  • Added CollectionNotification, CollectionModification, IListSubject, IDictionarySubject and concrete types.
  • Added Collect extension methods to Observable2, ObservableDirectory, DirectoryInfoExtensions and FileSystemWatcherExtensions.
  • Added ObservableDirectory lab.
  • Moved various extension methods into new classes: SmtpClientExtensions, PingExtensions, HttpListenerExtensions, WebClientExtensions, WebRequestExtensions and SocketExtensions.
  • Adjusted trace identity format. Also adjusted the default text for Ix tracing.
  • Added ObservableSyndication for RSS 2.0 and Atom 1.0, with a corresponding UI lab that also uses DictionarySubject and the Collect extension method.
  • Added ApplicationSettingsBase extensions for observing setting changes.
  • Added ICommand extensions, CommandSubject, AnonymousCommand and a corresponding lab.
  • Added the Subscription XAML markup extension for WPF, which is similar to Binding and supports observable data sources. Includes a corresponding lab.
  • Added EventSubscription trigger, which supports event handler bindings from FrameworkElement to IObserver, delegate or ICommand properties. Includes a corresponding lab.
  • Major performance and memory improvements for parsers; now avoids stack overflows due to recursion in quantifiers.
  • Added ICursor<T> and IObservableCursor<T> types with concrete implementations, including CursorSubject<T> and ToCursor extension methods.
  • Added full support for reactive XML parsers in WP7. Now there’s complete parser parity across all platforms.
  • Added view model support for all platforms, with corresponding UI labs. Includes optional IViewModel interface and optional Rxx.ViewModel base class.
  • Added Exactly parser operator.
  • Added non-greedy variants to some of the quantifying parser operators.
  • Added an overload to the AtLeast parser operator that accepts a maximum parameter, with behavior similar to the {n,m} regex pattern.
  • Added AndUnordered and AllUnordered parser operators, fixed the XML parsers so that attributes are matched in any order and reversed the order of the attributes in the XML schema labs.
  • Added Consume extensions that generalize the producer/consumer pattern over observables.
  • Added Stream, FileStream and TextReader extensions.
  • Added ObservableFile class with a corresponding lab.
  • Added n-ary Zip and CombineLatest combinators.
  • Improved lab application.

Full Release Notes Here!

Download here

NuGet Packages are also available

Observable.Generate Pop Quiz

What would be the output of the following program;

var xs = new int[]{1,2,3};
Observable.Generate(
    xs.GetEnumerator(),    // initial state
    e => e.MoveNext(),    // break condition
    e => e,            // iterate
    e => e.Current        // result selector
).Subscribe(Console.WriteLine);

The program (correctly) outputs;

1

2

3

What about this program? Would the output be the same? If not why not?

var xs = new List<int>{1,2,3};
Observable.Generate(
    xs.GetEnumerator(),    // initial state
    e => e.MoveNext(),    // break condition
    e => e,                // iterate
    e => e.Current        // result selector
).Subscribe(Console.WriteLine);

Stay tuned.

Asus U36SD notebook running Windows 8 developer preview

I’ve just setup a new laptop running Windows 8 & Visual Studio 11 developer preview.

Hardware

Asus U36SD
Intel Core i7
4GB RAM
160GB SSD
1GB NVidia 520M

http://www.asus.com/Notebooks/Superior_Mobility/U36SD/

Installation

It was really easy to get Window 8 up and running. The laptop came with Windows 7 pre installed.

  1. Boot into Windows 7
  2. Download 64 bit developer preview (with developer tools)
  3. Unpack the ISO onto the desktop (I used 7zip)
  4. Run “Setup.exe”
  5. It will ask if you want to check internet for latest version – “Yes”
  6. It will ask what you want to keep – “Nothing”!

In what seemed like less than 5 minutes (I didn’t time it), the machine had rebooted and I was running Windows 8!

Visual Studio 11

It comes pre-installed, so there is nothing to do there.

I’ve create a simple hello world application. I chose to build a “C# metro application”, you are presented with the familiar XAML designer & C# code behind. There is not really much to say here. Visual Studio 11 looks and feels exactly the same as 2010 except there are a bunch of new projects for “Metro applications”. WPF & Silverlight developers won’t have any difficultly slotting into this development environment. Apart from the namespaces everything is very similar.

I’ll post more once I’ve something interesting to talk about…

Follow

Get every new post delivered to your Inbox.