TerryR my friend, you and I should have a drink. We have some similar problems.
1. Project Structure: I agree with Eduardo that the folder structure in an MVC app leaves something to be desired. You have your standard Controllers, Models, and Views folders. But then the Views folder gets broken down into a different folder for each Controller, plus a Shared folder. And each Views/ControllerName or Views/Shared can be broken down into EditorTemplates and DisplayTemplates. But it lets you decide how to organize your Models folder (you can do with or without subfolders & additional namespace declarations).
God forbid you are using Areas, which duplicate the Controllers, Models, and Views folder structure for each area.
/Areas
/Area1Name
/Controllers
FirstController.cs
SecondController.cs
ThirdController.cs
/Models
(can organize all in here or in separate folders / namespaces)
/Views
/First
/DisplayTemplates
WidgetAbc.cshtml <-- to be used by views in Views/First
/EditorTemplates
WidgetAbc.cshtml <-- to be used by views in Views/First
PartialViewAbc.cshtml <-- to be used by FirstController
/Second
PartialViewDef.cshtml <-- to be used by SecondController
/Third
PartialViewMno.cshtml <-- to be used by ThirdController
/Shared
/DisplayTemplates
WidgetXyz.cshtml <-- to be used by any view in Area1
/EditorTemplates
WidgetXyz.cshtml <-- to be used by any view in Area1
PartialViewXyz.cshtml <-- to be used anywhere in Area1
_ViewStart.cshtml <-- area needs its own _ViewStart.cshtml
Web.config <-- put custom HTML Helper namespaces in here
Area1NameRegistration.cs <-- define routes for area1 here
/Area2Name
/Controllers
/Models
/Views
Area2NameRegistration.cs <-- define routes for area2 here
/Controllers
AccountController.cs
HomeController.cs
/Models
/Views
/Account
/DisplayTemplates
WidgetGhi.cshtml <-- to be used views in Views/Account
/EditorTemplates
WidgetGhi.cshtml <-- to be used views in Views/Account
PartialViewGhi.cshtml <-- to be used by AccountController
/Home
(same pattern as Account, views & templates are controller-specific)
/Shared
/DisplayTemplates
EmailAddress.cshtml <-- to be used by any view in any area
Time.cshtml <-- to be used by any view in any area
Url.cshtml <-- to be used by any view in any area
/EditorTemplates
EmailAddress.cshtml <-- to be used by any view in any area
Time.cshtml <-- to be used by any view in any area
Url.cshtml <-- to be used by any view in any area
_Layout.cshtml <-- master layout page with sections
Error.cshtml <-- custom page to show if unhandled exception occurs
_ViewStart.cshtml <-- won't be used automatically in an area
Web.config <-- put custom HTML Helper namespaces in here
This means if you are working with something like a WidgetController, you have to look in other folders to find the related WidgetViewModels, WidgetViews, WidgetEditorTemplates, WidgetDisplayTemplates, etc. As cumbersome as this may be, I stick to it and don't deviate from these MVC conventions. As far as putting a model, controller, and view in the same folder but with different namespaces, I avoid this because I use ReSharper. It will squiggly underline a namespace that doesn't match the folder where the class is located. I know I could turn this R# feature off, but it helps in other parts of the project.
For non-class files, MVC gives you Content and Scripts out of the box. We try to keep all of our static / non-compiled files in these places, again, to follow convention. Any time we incorporate a js library that uses themes (images and or css), the theme files all go somewhere under /content. For script, we just put all of them directly into /scripts. Originally this was to get JS intellisense from VS, but now that we get JS intellisense from R# regardless of placement in /scripts, I suppose we could deviate from that and divide scripts by folder to better organize. Are you using ReSharper? It is pure gold IMO.
Another little piece of gold that helps a lot with refactoring is T4MVC. Using this, we don't need to type in string paths for area names, controller names, action names, even files in content & scripts. T4MVC strongly types all of the magic strings for you. Here's a small sample of how your project structure doesn't matter as much if you're using T4MVC:
// no more magic strings in route definitions
context.MapRoutes(null,
new[] { string.Empty, "features", "features/{version}" },
new
{
area = MVC.PreviewArea.Name,
controller = MVC.PreviewArea.Features.Name,
action = MVC.PreviewArea.Features.ActionNames.ForPreview,
version = "december-2011-preview-1",
},
new { httpMethod = new HttpMethodConstraint("GET") }
);
@* T4MVC renders .min.js script versions when project is targeted for release *@
<link href="@Url.Content(Links.content.Site_css)?r=201112B" rel="stylesheet" />
<script src="@Url.Content(Links.scripts.jquery_1_7_1_js)" type="text/javascript">
</script>
@* render a route URL as if you were calling an action method directly *@
<a href="@Url.Action(MVC.MyAreaName.MyControllerName.MyActionName
(Model.SomeId))">@Html.DisplayFor(m => m.SomeText)</a>
// call action redirects as if you were executing an action method
return RedirectToAction(MVC.Area.MyController.DoSomething(obj1.Prop, null));
2. Data access: I have no experience with PetaPoco, but I'm sure it's worth checking out. For your complex reports, have you considered SQL Server Reporting services? Or, are you running on a different db? Sorry I'm not clear on what exactly you are asking for. We use EF + LINQ, but we also put certain knowledge about how to generate reports in domain classes. Thus, we have controller call domain service call repository, instead of having controller call repository directly. For ad-hoc reports we use SQL Reporting Services, which again isn't perfect, but our users like to be able to bring data into Excel easily, and SSRS makes that easy on us.
3. Client-Side Code Organization and UI Rendering: This is where I think I may be able to offer some help. Take a page from the book of MVC unobtrusive validation and unobtrusive AJAX. Consider this:
<img id="loading_spinner" src="/path/to/img" style="display:none;" />
<h2 id="loading_results" style="display:none;">
Please wait, this may take a while...
</h2>
<div id="results">
</div>
<input id="doSomethingDangerous" class="u-std-ajax"
type="button" value="I'm feeling lucky"
data-myapp-confirm="Are you sure you want to do this?"
data-myapp-show="loading_spinner,loading_results"
data-myapp-href="blah/DoDangerousThing" />
Ignore the ajax success function for now (more on this later). You can get away with a single script for some of your actions:
$('.u-std-ajax').click(function () {
// maybe confirm something first
var clicked = this;
var confirmMessage = $(clicked).data('myapp-confirm');
if (confirmMessage && !confirm(confirmMessage )) { return; }
// show a spinner? something global would be preferred so
// I dont have to repeat this on every page
// maybe the page should notify the user of what's going on
// in addition to the dialog?
var show = $(clicked).data('myapp-show');
if (show) {
var i, showIds = show.split(',');
for (i = 0; i < showIds.length; i++) {
$('#' + showIds[i]).show();
}
}
var url = $(clicked).data('myapp-href');
if (url) {
$.ajax({
url: url,
complete: function () {
// Need to hide the spinner, again would prefer to
// have this done elsewhere
if (show) {
for (i = 0; i < showIds.length; i++) {
$('#' + showIds[i]).hide();
}
}
}
});
}
});
The above code will take care of the confirmation, showing the spinner, showing the wait message, and hiding the spinner / wait message after the ajax call is complete. You configure the behaviors using data-* attributes, like the unobtrusive libraries.
General Questions
- Client MVC vs. server MVC? I didn't try to librarify the actions you took in the success function because it looks like your controller is returning JSON. If your controllers are returning JSON, you might want to look at KnockoutJS. Knockout JS version 2.0 was released today. It can plug right into your JSON, so that an observable click can automatically bind data to your javascript templates. On the other hand if you don't mind having your ajax action methods return HTML instead of JSON, they can return the already-constructed UL with its LI children, and you can append that to an element by using data-myapp-response="results". Your success function would then just look like this:
success: function(html) {
var responseId = $(clicked).data('myapp-response');
if (responseId) {
$('#' + responseId).empty().html(html);
}
}
To sum up my best answer for this, if you must return JSON from your action methods, you are skipping the server-side View, so this really isn't server MVC -- it's just MC. If you return PartialViewResult with html to ajax calls, this is server MVC. So if your app must return JSON data for ajax calls, use client MVVM like KnockoutJS.
Either way, I don't like the JS you posted because it mixes your layout (html tags) with behavior (asynchronous data load). Choosing either server MVC with partial html views or client MVVM with pure JSON viewmodel data will solve this problem for you, but manually constructing DOM/HTML in javascript violates separation of concerns.
- Javascript file creation Apparently minification features are coming in .NET 4.5. If you go the unobtrusive route, there shouldn't be anything stopping you from loading all of your JS in 1 script file. I would be careful about creating different JS files for each entity type, you will end up with JS file explosion. Remember, once your script file is loaded, the browser should cache it for future requests.
- Complex queries I don't consider having feature like pagination, sorting, etc, as being complex. My preference is to handle this with URL's and server-side logic, to make the db queries as limited as needed. However we are deployed to Azure, so query optimization is important to us. For example: /widgets/show-{pageSize}-per-page/page-{pageNumber}/sort-by-{sortColumn}-{sortDirection}/{keyword}. EF and LINQ to Entities can handle pagination and sorting with methods like .Take(), .Skip(), .OrderBy(), and .OrderByDescending(), so you get what you need during the db trip. I haven't found the need for a clientlib yet, so I honestly don't know much about them. Look to other answers for more advice on that.
- Project silk Never heard of this one, will have to check it out. I am a big fan of Steve Sanderson, his books, his BeginCollectionItem HtmlHelper, and his blog. That said, I don't have any experience with KnockoutJS in production. I have checked out its tutorials, but I try not to commit to something until it's at least version 2.0. Like I mentioned, KnockoutJS 2.0 was just released.
- N-tier If by tier you mean different physical machine, then no, I don't think anything goes out any windows. Generally 3-tier means you have 3 machines. So you might have a fat client as your presentation tier, that runs on a user's machine. The fat client might access a service tier, that runs on an application server and returns XML or whatever to the fat client. And the service tier might get its data from a SQL server on a 3rd machine.
MVC is one layer, on 1 tier. Your controllers, models, and views are all part of your Presentation Layer, which is 1 tier in the physical architecture. MVC implements the Model-View-Controller pattern, which is where you might be seeing additional layers. However, try not to think of these 3 aspects as tiers or layers. Try to think of all 3 of them as Presentation Layer Concerns.
Update after pres/bus/data comment
Okay, so you are using tier and layer interchangably. I usually use the term "layer" for logical / project / assembly divisions, and tier for physical network separation. Sorry for the confusion.
You will find quite a few people in the MVC camp who say you should not use the "Models" in MVC for your entity data model, nor should you use your Controllers for business logic. Ideally your models should be view-specific ViewModels. Using something like Automapper, you take your entities from your domain model and DTO them into ViewModels, sculpted specifically for use by the view.
Any business rules should also be part of your domain, and you can implement them using domain services / factory pattern / whatever is appropriate in your domain layer, not in the MVC presentation layer. Controllers should be dumb, though not quite as dumb as models, and should give responsibility to the domain for anything that requires business knowledge. Controllers manage the flow of HTTP requests and responses, but anything with real business value should be above the controller's pay grade.
So, you can still have a layered architecture, with MVC as the presentation layer. It is a client of your application layer, service layer, or domain layer, depending on how you architect it. But ultimately your entity model should be part of the domain, not models in MVC.
Best Answer
Maybe your applications aren't very big? I remember the first big app I co-wrote, it consisted only of a large number of complex jQuery plugins. There was no separation of concerns, the code was convoluted, testing it was almost impossible.
The next app we wrote in Backbone. It gave us a basic structure to build upon. Obviously, Backbone introduced its own share of problem but it solved a lot more.
Our problems mainly concerned reacting to model & view changes, we started to have a large number of events & handlers hard to control. Even then, it was heaven compared to what we had in the previous app!
We decided to rewrite the app in AngularJS (personal preference, you can substitute Angular for Ember in this paragraph and arguments will probably stay). It allowed us to make our client code 40% smaller and so much easier to maintain; testability was a bonus too. We could focus more on the app logic than on manual DOM modifications.
To summarize, my experience is contrary to yours. I'd think similarly to you in case of simple sites since MVC always introduce some boilerplate and you need certain application size to experience the benefits.