async materialization of IQueryable

[csantero]

At the moment, if the media type formatter receives an IQueryable, it will do a foreach over them inside of SerializeMany, which will synchronously run the query. We should probably do a ToArrayAsync() to fix this. We could either do this inside WriteToStreamAsync(), or add another action filter that transforms an IQueryable into an array. I actually have a filter that does the latter inside my project which I could port over. The drawback is that it's Yet Another action filter the user has to add.

[SphtKr]

Agreed, I favor the WriteToStreamAsync approach, as it seems like that's what that method is for. Right now the implementations of WriteToStreamAsync and ReadFromStreamAsync are both completely synchronous and just wrapped in a task.

I haven't gotten into that because I'm not an expert on .NET's async programming model(s), and I understand it can potentially be perilous. It's worth noting that we already require .NET 4.5, so at least we have the option to use await, which might simplify things?

[csantero]

I just discovered that the WriteToStreamAsync approach is not viable. The IQueryable Async extension methods (like ToArrayAsync()) are all part of EntityFramework, and we don't want an EF dependency in the formatter. So we'll have to define an action filter in JSONAPI.EntityFramework.

[SphtKr]

Ew...yeah, I hadn't thought about that. So, how do we make sure that ActionFilter runs after the filtering and sorting ones? Does the order of the attributes determine that?

[csantero]

The async queryable materializer filter has to be inserted BEFORE our filtering and sorting ones. This is because of the onion-style nesting of filtering handlers.

Given the following directives against your HttpConfiguration:

config.Filters.Add(new ResolveQueryablesAsyncAttribute()); // Or whatever we wind up calling it
config.Filters.Add(new EnableSortingAttribute());
config.Filters.Add(new EnableFilteringAttribute());

The order of operations becomes:

ResolveQueryablesAsyncAttribute: OnActionExecutingAsync
EnableSortingAttribute: OnActionExecutingAsync
EnableFilteringAttribute: OnActionExecutingAsync
Action runs
EnableFilteringAttribute: OnActionExecutedAsync
EnableSortingAttribute: OnActionExecutedAsync
ResolveQueryablesAsyncAttribute: OnActionExecutedAsync

Getting this order wrong could be a potential pain point for our users. Maybe at some point it might be worthwhile adding an extension method that adds them all together. Of course if you're not adding the filters in HttpConfiguration but rather as attributes, then a different order is followed which I'm not intimately familiar with.

[SphtKr]

Getting this order wrong could be a potential pain point for our users.

Quite. Possibly dumb question, but... Remind me what the gain is here? I don't think it's possible or meaningful to stream the JSON response as it's processed, so the client is essentially waiting for the whole operation to complete in any case. The database connection is going to be held open for the same amount of time regardless of whether the IQueryable is resolved sync or async. So, maybe the thread could be used for something else while it's waiting (I'm still fuzzy on how this is possible)... In any case, the serialization code can't really continue until the whole IQueryable is processed, so what's the cost of doing it synchronously?

[csantero]

Yeah, it frees up the web api thread to handle other requests while the data is being fetched. It's the .NET equivalent to node.js's async callback philosophy, which is what makes that platform perform well for high-throughput concurrency.

[csantero]

TBH I don't know the real performance gain from this, but I already am using the attribute in my project so I figured I would just port it over here and then at least users have the option. Maybe someday we can invent a stress-testing suite.

[SphtKr]

Sure, I suppose we can support that as an option in the meantime at least. "If you're running out of threads or see slow performance with large result sets, use this..."

I have to wonder if there's another solution (dynamically looking for ToArrayAsync? add our own IQueryable extension that defines ToArrayAsync and defaults to the existing implementation if it exists?).