What's The Point Of Using WCF In A Web App?

A very common approach of building web applications in .NET is to put most of the non-UI related code behind an internal WCF service layer. I used to be a fan of this approach as well, but these days I just don't see the benefit of that internal service layer anymore. The overhead that an internal WCF service layer adds to development, deployment and runtime performance just doesn't stack up favorably to the supposed benefits IMO. To be clear: I'm talking about WCF services that will only be used by the front end of your web application.

Let's talk about the overhead on development first. If you're using WCF services in your web app, you need proxies to access those services. Some people prefer to generate the proxies based on the WSDL of the services that will be used. In the worst case, this leads to regenerating proxies and all of the types that are defined in the WSDL every time you change a service contract or one of the types that are used by the services. If multiple people need to make changes to any of these concurrently, this easily leads to merging problems when people need to commit their changes. Another way is to share the same types on both sides (client & server), and implement your service proxies by inheriting from ClientBase and manually keeping the implementation of the proxies up to date with the definitions of their service contracts. This is better than regenerating a bunch of code all the time, but you're still writing a lot of redirection code for the purpose of, well, what exactly? Another possibility is to use dynamic proxies which automatically implement the service contracts but this increases the amount of infrastructure code you need to put in place and it's not always clear to everyone how exactly communication with the services happens. There's also a lot of WCF configuration for each service that you need to maintain, and it can quickly grow unwieldy.

Then there's the overhead on performance. I hope we can all agree that any operation that goes out of process is at least an order of magnitude slower than a similar operation that can be executed in process. First of all, there's the networking overhead (even if your services are hosted on the same machine as the web app) that you have to keep into account. Secondly, there is the cost of serializing and deserializing everything that is transferred between the client and the server. Even with the most efficient bindings and serializers, the cost of all of this quickly adds up on high-traffic web apps. That's not to say that WCF services are inherently slow. They can be very fast and efficient, but they'll never be as fast and efficient as executing that logic in process within the web app.

Finally, there's the extra overhead it introduces to the deployment phase:

more endpoints to set up and transfer artifacts to
more configuration
more monitoring of endpoints
more servers if you're not hosting the services and the web app on the same machine

Of course, people will argue that there a plenty of benefits to using a WCF service layer in a web app. The ones I hear about most often are the forced separation of business logic and UI logic and improved scalability and reliability. I really disagree that you need a physical separation of business and UI logic. I much prefer approaches where the separation is based on abstractions. A good example was recently posted by Ayende (here and here). And when it comes to scalability/reliability, a web app that isn't dependent on a WCF service layer is as easy (or even easier depending on your setup) to scale than one that is entirely dependent on WCF services. First of all, if you care about scalability/reliability your web app should already be prepared to run behind a load balancer. If you already have a load balancer in place, you can just add more web servers to your setup when needed. If you'd host the WCF services on the same machines that are hosting the web front end, you'd get less total throughput from one server than you would if that one server could just host a web app that fully executes in process (not including the database obviously). If you're hosting the WCF services on separate machines, you'd end up with more servers to handle the load and to achieve the reliability you need than you would with just being able to add more web servers to your setup. That also increases your licensing costs. And of course, it also means increased networking overhead on every service call, which also implies that the threads on your web servers will be blocked for longer periods while they wait for those service calls to return. Unless you're calling those services asynchronously, but most people simply don't. Also, if you have serious scalability and reliability requirements you're probably better off with asynchronous messaging solutions than with SOAP services.

WCF has its benefits (though I prefer Web API's or asynchronous messaging over SOAP services these days) and it has its use cases. I just don't think internal service layers for web apps is one of them.

What are the benefits that you think an internal WCF service layer brings to your web app? And what's your opinion on how they stack up versus the downsides?

Written by Davy Brion, published on 3/18/2012 5:28:28 PM
Categories: architecture , code-quality , performance , wcf

Architectural Drivers

There are many different opinions and preferences when it comes to how we should deal with software architecture. My personal preference is to try not to make things more complex than they should be, while striving to maintain enough flexibility to be able to deal with future changes in both functional and non-functional requirements in a way that hopefully doesn't require extensive effort. I've always thought of good architecture essentially being something that enables developers to keep implementing new features over time at a similar cost than what it took to implement features in the early stages of development. Thus I prefer to try to find a balance between simplicity of the codebase, and making sure that the architecture scales nicely (from a code quality point of view) as the application grows in functionality. I also like to make sure that we can scale out (from a performance/throughput point of view) relatively easily if that need to scale comes up. These are just my general preferences though, and aren't always relevant to every project or suitable for every situation. I'm now in the 10th year of my career and have seen a variety of architectural styles in the wild, and of course I've read and heard about many more since the subject has always interested me a lot. I've learned that it's important to keep architectural drivers of a project in mind when thinking about what kind of architecture is suitable for a certain project or situation. Those drivers are often more important than whatever your personal preferences are. I've come up with a list of drivers that I think should influence the architecture used by teams or organizations.

Simplicity/Complexity of the application

A common mistake is to decide to use a fashionable architectural style in a project that doesn't really have enough complexity to warrant the use of said architectural style. For instance, there's no reason whatsoever to go the DDD route if your application is mostly a forms-over-data CRUD app. You won't really get any benefit from it, and you'll only increase the total cost of the required development effort. On the other hand, an application which will feature a lot of behavior in a complex domain will certainly benefit from using a DDD approach versus a data-driven approach that implements the behavior in a more procedural, transaction-script based approach. Be honest to your customer/employer and yourself and make an objective decision based on the simplicity or the complexity of the required solution that you're being paid to build. Don't let enthusiasm for a fashionable architectural style influence your decision.

Expected lifetime of the application

Some applications live for years and years, and some are only temporary solutions that will be phased out a year or two after having been put in production. Of course, some of those temporary solutions end up being used much longer than what was originally expected due to a variety of reasons but that certainly isn't always a given. Unless you're familiar enough with how things typically go at your client or your company to predict these things, your best bet is to just go with the initially expected lifetime of the application. Does a temporary solution require the best possible architecture out there, and the resulting impact that would have on the cost to build it? I'd say it most often doesn't. In these cases, there's nothing wrong with going for the approach that is simply good enough for what is required. Striving for perfection is expensive, and generally not worth it for short-term solutions. Conversely, if the solution is expected to last a long time, a solid architecture becomes more important and is worth the extra investment to (try to) get it right if that makes it possible to keep the costs of long-term development under control.

Strategic importance of the application

Some applications are meant to improve the core business of a company, while others play a more supportive role in something that isn't actually part of the company's core business. Some applications generate revenue (directly or by improving efficiency of revenue-generating activities) while others are meant to reduce costs, typically administrative in nature. Cost reduction is important, but it's not quite as important as revenue generation and that's a factor that should be taken under consideration. A strategically important application almost always warrants putting in the effort to come up with a good architecture because it's highly likely that the application will have to evolve in whichever direction the business evolves. That's not to say that the same kind of effort wouldn't be important for cost-reducing applications. But it just might be less important than you'd like to think it is. Again, good architecture increases the cost of the project, and the return on that investment in the big picture of the company should not be ignored.

Skill-level/discipline of team

Good code and good architecture requires skill and discipline. A strong team is capable of letting good architecture grow organically and keep it at a high quality level. For a mediocre team, letting it grow organically is too often a recipe for disaster. Ideally, we'd always be working with strong teams but as we all know, that simply isn't always possible. For mediocre teams, it often makes sense to put architecture in place that is more restrictive in nature and where everyone knows how the code should be structured in advance. The downside is that this typically reduces flexibility and individual creativity, and often introduces more ceremony and indirection than what many of us would ideally like to see. But it's likely that those downsides are offset by having everyone on the same page and taking some possibly difficult decisions away from people who might not be strong enough to make the right decisions. I know that sounds harsh, but that's a reality that many of us have to deal with.

Conformity/Continuity

A lot of IT departments or software development companies prefer to use the same architecture and frameworks/libraries for most (or even all) of their projects because it makes it easier to have people work on multiple projects. In this situation, it's easier to move people between projects or have them do maintenance work on a project they weren't initially involved in. While this often prevents going with the most suitable approach for every project, it does introduce a few benefits that are hard to argue with. People will need less time to get familiar with a code base. In-house training and sharing of knowledge get easier. Bringing in new people (especially for temporary assignments) also gets somewhat easier because you have a baseline of required skills/knowledge that should go a long way within the organization. A huge downside however, is that it creates an environment that will quickly frustrate creative developers. Also, when new strong developers are brought in, they could quickly tune out once they realize they got there too 'late' to have any influence on how projects are developed. The interesting part about this driver is that it is influenced by the skill-level-discipline driver, while simultaneously influencing it towards the future.

Non-functional requirements

This is the last driver on the list, but it is certainly not the least important. Obviously, architecture is greatly influenced by some important non-functional requirements. There are many factors that can have a profound influence on what the most suitable architecture of a system could be, and it's important to think about those as quickly as possible. Do you need to respond in real time? Do you need to support mobile devices? Will there be connectivity issues? Will you need to scale massively? What are the auditing requirements? Are you dependent on third-party services? Do you need to minimize resource consumption? What kind of accessibility do you have to keep into account? The list goes on and on and many of these issues can end up becoming huge problems if you don't think about them ahead of time. Of course, you can't think of everything in advance, and many non-functional requirements can be introduced during the lifetime of the application instead of being known in advance. But the more you know in advance, the better you can prepare your software for them from the beginning.

Conclusion

There is a virtually endless variety of architectural choices and styles that can influence how you develop software. And unfortunately, there's a lot of dogma surrounding it as well. I hope this post made it clear that there is no definitive 'right' or 'wrong' when it comes to architecture, and that (as pretty much everything else in this business) it all really depends :)

I'm sure you can think of more architectural drivers, and I'd love to hear about any that you think I've forgotten.

Written by Davy Brion, published on 2/26/2012 6:10:50 PM
Categories: architecture

DTO's Should Transfer Data, Not Entities

I've read a couple of posts recently where the authors were complaining about excessive mapping to DTO's and whether or not it's worth it. I've been a fan of DTO's for a long time, but I'm not a fan of how I frequently see them being used. More specifically, I very much dislike the all-too-common approach of creating a DTO for each entity in your domain model. In the worst cases, the DTO's actually reference each other and try too hard to mimic the structure of the entities. For example, an OrderDto which has a reference to a CustomerDto and a collection of OrderLineDto instances. In those cases, I absolutely agree that all of the mapping involved introduces way too much ceremony to the codebase without really offering any concrete benefits.

Of course, it really depends on the architecture of your system. If you're using services that are only used by the front-end of your system, then I'd still advise against making those entities available outside of the service boundary for reasons that I've discussed earlier. That doesn't mean that you should go the entity-mimicking-DTO-route though. In fact, entity-mimicking-DTO's introduce a few of the same downsides you'd get from exposing entities directly through your services. For an 'internal' service (i.e. only used by your application), I think it makes a lot more sense to use DTO's per service operation which are optimized for the use case that that service operation is meant to implement.

Essentially, an internal service's operations will be either queries or commands. When it comes to queries, why not just do the simplest thing possible? In most cases, it's sufficient to just return the data in the exact same form that the data will be displayed in. Quite often, that means a denormalized set of data where the data comes from more than one type of entity/table. There's no reason to send a bunch of entity-mimicking DTO's that reference each other to the client if you're going to use the data to populate a grid. Just send a list of DTO's which are already in the most optimal form. Populating those DTO's can then simply be done through straight SQL, a view, a stored procedure, a projection through your ORM, a map/reduce operation, or whatever else that makes sense. The point is that in most cases, you should just populate those DTO's as directly as you can instead of mapping to those DTO's. In this case, the DTO's offer a clear-cut benefit and don't really introduce tedious ceremony code in your codebase.

As for the commands (inserts, updates, anything that creates something or results in something happening), why even use DTO's in the first place? People will often argue that the ability to reuse the entity-mimicking-DTO's for these operations is a benefit. Personally, I don't really see the benefit. The mere presence of entity-mimicking-DTO's only encourages people to use them for the queries as well. Instead, I prefer to go with types that encapsulate all of the data relevant to the current command (the data to be inserted/updated, or whatever else the command needs). In a very simple scenario, this could be an InsertCustomerCommand type which simply has properties for the data that needs to be inserted. Nobody will be confusing these types with any other purpose than what their name communicates.

If you're building a web app where the server-side code does everything in process (i.e. without a WCF service), then you can in many cases just use the entities directly without a real drawback, though I'd recommend keeping an eye on unexpected select N+1 problems, since those will frequently come up when you're preparing your views. But even in this case, using DTO's that are optimized for the scenarios in which they're being used can really simplify the query-side of your system a lot.

There's nothing inherently wrong with DTO's if you use them to simply transfer data. If you're using them to transfer entities, you're robbing yourself of their biggest benefit while only making things more complex than they need to be.

Written by Davy Brion, published on 2/19/2012 8:32:17 PM
Categories: architecture , code-quality

Highly Recommended Book: REST In Practice

A couple of months ago, I knew very little about REST and Restful services. Since it was increasingly getting more attention in the developer community, I wanted to find out what it was about. Luckily for me, O'Reilly had recently released REST In Practice and when it was listed as an O'Reilly Deal Of The Day, I bought it without thinking twice. Unfortunately, I temporarily stopped reading it about halfway through because I needed to finish some other books first, but I recently picked up where I left off.

Coming from the Microsoft world were SOAP services have for a long time been the norm in most projects, it's very interesting to see what REST is all about and how these Restful services are built. If you don't know anything about REST, you might want to check out this article by Martin Fowler first. Obviously, you don't need to read that before you can read the book but it might pique your interest. The first chapter does a great job of introducing to the architecture of the Web and how the REST architectural style fits within it. After that, you're introduced to the Restbucks example, which is used throughout the book to show you how Restful services can work.

The book then gradually starts diving deeper and deeper into the implementation of the Restful services for the Restbucks example. Everything is explained very clearly, and the authors continuously show both the requests and the responses that are going over the wire to illustrate what is going on at the HTTP level, which makes it even easier to understand everything that's being discussed. After the explanations and request/response examples, there's typically a code-based example in either Java or .NET to show how you can implement these systems. After a while I just started skipping these examples entirely because I thought they didn't really bring any added benefit. Though I'm sure some people will appreciate those examples being included as well.

Another great thing about this book is that it remains very clear and easy to follow, even though it covers a lot of ground. Here's a brief overview of the things you'll see implemented in the next couple of chapters:

CRUD
Hypermedia (the engine of application state transformations)
Everything you need to know about caching
Event-driven services with Atom
Atom publishing
Security: authentication as well as authorization

After that, there's a chapter on the semantic web and microformats, which I didn't find very interesting but others likely will. The final 2 chapters make for a great conclusion of the book. First, there's a pretty extensive comparison between the benefits and drawbacks of SOAP versus REST. And finally, the last chapter covers when it makes sense to use REST and when it doesn't and ends with a recap of the major selling points of using the Web as a central building block of your architecture.

I grok the REST architectural style now, and definitely like it. If you read this book with an open mind, I'd bet you'll like it a lot as well.

Written by Davy Brion, published on 3/5/2011 7:12:42 PM
Categories: architecture , books

The MVVM Pattern Is Highly Overrated

Update: check out my MVP In Silverlight/WPF series which discusses the MVP approach as an alternative to MVVM

If you're doing Silverlight or WPF, you've no doubt come across the MVVM (Model-View-ViewModel) pattern. It seems to be the most popular client-side architecture pattern used among Silverlight/WPF developers. I find the pattern to be highly overrated, and actually have some big issues with the whole thing.

First, let's briefly cover what MVVM is about for those of you who don't know yet. MVVM virtually eliminates all of the code that would typically be placed in the code-behind file of your View (a user control, a screen, whatever) by putting all of that logic in the ViewModel. The ViewModel itself is never tightly coupled to the View. If it does have a reference to it, it's typically through an interface that the View implements instead of a reference of the actual type of the View. This increases, or better yet, introduces testability to a large part of your UI code that you normally wouldn't be able to cover with unit tests if you'd go with the traditional "put it all in the code-behind"-approach.

The ViewModel typically contains properties for the data that is to be shown in the View, and also raises notification events when the data in those properties changes. The View uses the powerful data-binding capabilities of Silverlight/WPF to bind the properties of the ViewModel to other user controls that the View is composed of. User-events that are captured by the View are sent to the ViewModel through Commands. Typically, these commands execute a method in the ViewModel which contains some kind of logic... usually to either send the updated data in the ViewModel's properties to the Model (usually a Service Layer in Silverlight, in WPF it could be either a true Domain Model or also a Service Layer), to perform some business logic in the Model, or to retrieve data from the Model so it can be placed in the ViewModel's properties so the View can bind to it.

That is, in a nutshell, how the MVVM pattern works. So why do I have issues with this? You can develop and test most of the application's logic without being dependent on a physical View and that is typically a Good Thing, no? It sure is! However, my problem with this approach is that too many responsibilities are concentrated within the ViewModel. Its main responsibilities are to facilitate databinding and to interact with the Model. And that, in my opinion, isn't very clean. In some ways, you could actually think of the ViewModel as a glorified code-behind, with the only difference being that it's not tightly coupled to the (physical) View.

In most (if not all) MVVM implementations, a ViewModel has many possible reasons to be changed. It might need to be changed because of different data-binding requirements. Then again, it might also require changes when a part of the Model is modified. Now, I'm sure many of you can agree with me when I say that two of the most important principles in software development are Seperation of Concerns (SoC) and the Single Responsibility Principle (SRP). Obviously, the ViewModel doesn't fare well when it comes to both of these principles.

Lets forget about MVVM for a second and focus on the concerns and responsibilities that a user control can have... say, a user control that shows customer information and allows the user to edit that data so it can be persisted:

visualization of the actual control (its own layout as well drawing other user controls that it is composed of)
communication/interaction with the Model
making data (from the Model) available so it can be displayed
outputting data in the correct user controls (for instance: various textboxes)
(simple validation) of modified/inputted data (for instance: no string values for numeric fields, etc...)

Without MVVM, all of these would be taken care of in the View. Obviously, not a good idea right? After all, if it were a good idea, we'd never have had a reason to start using MVVM in the first place.

Now, with MVVM, a lot of people would divide these concerns and responsibilities like this:

View:

visualization of the actual control (its own layout as well drawing other user controls that it is composed of)
outputting data in the correct user controls (for instance: various textboxes)
(simple) validation of modified/inputted data (for instance: no string values for numeric fields, etc...)

ViewModel:

communication/interaction with the Model
making data (from the Model) available so it can be displayed

In this case, the View still has 3 responsibilities which is still too much according to 'the guidelines', but it's not that big of a deal (though plenty of people would argue that the simple validation would be better placed in the ViewModel). You're highly unlikely to actually want to write automated tests for pure visualization anyway and the SRP is not something that you absolutely need to follow to the extreme in every single place. For the View, this is really not a problem and very much acceptable.

The ViewModel however has 2 important responsibilities in this case, and I'd argue that these 2 things should not be done in the same place. Making data available is done through data-binding. To do this, you need a set of properties and you need to raise the necessary events. In most cases, raising those events is very straightforward, but in more complex controls you might need a bit of additional logic to determine which event should be raised at what point. The other important responsibility is the communication/interaction with the Model. In most Silverlight applications, the Model will be a Service Layer. To communicate with this Service Layer, you need Service Proxies. That means that your ViewModel is essentially responsible for communicating with the Service Layer, dealing with business exceptions that might be thrown by some service calls, and dealing with technical exceptions that can occur simply because of network-related problems. Group all of those together and I don't think I'm going out on a limb here by saying that that is a lot of logic to put in one class, no?

(Sidenote: what I don't really understand is that many people who vigorously advocate adherence to SRP and SoC in their domain and business code don't seem to hold their UI code to the same standards. I do.)

At work, we do a lot of Silverlight development. We typically have around 5 Silverlight projects in active development at the same time. And it's been that way for over a year now. That equals a lot of Silverlight code that we've written and experience and knowledge that we've built up. And we haven't used MVVM for any of it. All this time, we've been using the MVP pattern (Supervising Controller variant) with a slight twist. That twist being a slimmed down version of a Presentation Model. Our Presentation Model's sole responsibility is to facilitate data-binding, and in some cases, a touch of validation is added as well.

If we go back to our previous example of the customer screen, the responsibilities and concerns would be divided like this in our MVP-PMlight approach:

View:

visualization of the actual control (its own layout as well drawing other user controls that it is composed of)
outputting data in the correct user controls (for instance: various textboxes)
(simple) validation of modified/inputted data (for instance: no string values for numeric fields, etc...)

Presenter:

communication/interaction with the Model based on the contents of the Presentation Model

Presentation Model:

making data (from the Model) available so it can be displayed

Which leads to classes which are more focused on their task instead of trying to focus on too many things at the same time. In my opinion, this approach is much better/cleaner than MVVM. Not only is there a noticeable benefit in code quality (classes are more focused), there is also increased potential to reuse our 'light Presentation Models' in multiple controls. Testability is increased over MVVM as well since it's always easier to test classes which are focused versus testing classes which have too many responsibilities. All in all, a couple of important benefits and we still haven't thought of a real drawback compared to MVVM.

Update: check out my MVP In Silverlight/WPF series which discusses the MVP approach as an alternative to MVVM

Written by Davy Brion, published on 7/21/2010 3:23:58 PM
Categories: architecture , code-quality , silverlight

« Older Entries