Entity Framework – Best Architecture for Entities with Web Service Data

asp.net-mvcentityentity-frameworkweb services

We are currently using Entity Framework as an ORM across a few web applications, and until now, it has suited us well as all our data is stored in a single database. We are using the repository pattern, and have services (the domain layer) which use these, and return the EF entities directly to the ASP.NET MVC controllers.

However, a requirement has come up to utilise a 3rd party API (through a web service) which will give us extra information that relates to the user in our database. In our local User database, we will store an external ID which we can provide the to API to get additional information. There is quite a bit of information available, but for the sake of simplicity, one of them relates to the user's company (name, manager, room, job title, location etc). This information will be used in various places throughout our web apps – as opposed to being used in a single place.

So my question is, where is the best place to populate and access this information? As it is used in various places, it's not really sensible to fetch it on an ad-hoc basis wherever we use in the web application – so it makes sense to return this additional data from the domain layer.

My initial thought was just to create a wrapper model class which would contain the EF entity (EFUser), and a new 'ApiUser' class containing the new information – and when we get a user, we get the EFUser, and then get the additional info from the API, and populate the ApiUser object. However, whilst this would be fine for getting single users, it falls over when getting multiple users. We can't hit the API when getting a list of users.

My second thought was just to add a singleton method to the EFUser entity which returns the ApiUser, and just populate it when needed. This solves the above problem as we only access it when we need it.

Or the final thought was to keep a local copy of the data in our database, and synchronise it with the API when the user logs in. This is minimal work as it's just a synchronisation process – and we don't have the overhead of hitting the DB and API every time we want to get user information. However, these means storing the data in two places, and also means the data is out of date for any user that hasn't logged in for a while.

Does anyone have any advice or suggestions on how best to handle this kind of scenario?

Best Answer

Your case

In your case all three options are viable. I think that the best option is probably to sync your data sources someplace the asp.net application is not even aware of. That is, avoid the two fetches in the foreground every time, sync the API with the db silently). So if that's a viable option in your case - I say do it.

A solution where you make the fetch 'once' like the other answer suggests doesn't seem very viable since it doesn't persist the response anywhere and ASP.NET MVC will just make the fetch for every request over and over.

I'd avoid the singleton, I don't think it's a good idea at all for plenty of the usual reasons.

If the third option is not viable - one option is to lazy load it . That is, have a class extend the entity, and have it hit the API on a need to basis. That's a very dangerous abstraction though since it's even more magic and non-obvious state.

I guess it really boils down to several questions:

How often does the API call data change? Not often? Third option. Often? Suddenly the third option is not too viable. I'm not sure I'm as against ad-hoc calls as you.
How expensive is an API call? Do you pay per call? Are they fast? Free? If they're fast, making a call each time might work, if they're slow you need to have some sort of prediction in place and make the calls. If they cost money - that's a big incentive for caching.
How fast does the response time have to be? Obviously faster is better, but sacrificing speed for simplicity might be worth it in some cases, especially if it's not directly facing a user.
How different is the API data from your data? Are they two conceptually different things? If so, it might be even better to just expose the API outside rather than return the API result with the result directly and let the other side make the second call and handle managing it.

A word or two about separation of concerns

Allow me to argue against what Bobson is saying about separation of concerns here. At the end of the day - putting that logic in the entities like that violates separation of concerns just as bad.

Having such a repository violates separation of concerns just as bad by putting presentation centric logic in the business logic layer. Your repository is now suddenly aware of the presentation related things like how you display the user in your asp.net mvc controllers.

In this related question I've asked about accessing entities directly from a controller. Allow me to quote one of the answers there:

"Welcome to BigPizza, the custom Pizza shop, may I take your order?" "Well, I'd like to have a Pizza with olives, but tomato sauce on top and cheese at the bottom and bake it in the oven for 90 minutes until it's black and hard like a flat rock of granite." "OK, Sir, custom Pizzas are our profession, we'll make it."

The cashier goes to the kitchen. "There is a psycho at the counter, he wants to have a Pizza with... it's a rock of granite with ... wait ... we need to have a name first", he tells the cook.

"No!", the cook screams, "not again! You know we tried that already." He takes a stack of paper with 400 pages, "here we have rock of granite from 2005, but... it didn't have olives, but paprica instead... or here is top tomato ... but the customer wanted it baked only half a minute." "Maybe we should call it TopTomatoGraniteRockSpecial?" "But it doesn't take the cheese at the bottom into account..." The cashier: "That's what Special is supposed to express." "But having the Pizza rock formed like a pyramid would be special as well", the cook replies. "Hmmm ... it is difficult...", the desparate cashier says.

"IS MY PIZZA ALREADY IN THE OVEN?", suddenly it shouts through the kitchen door. "Let's stop this discussion, just tell me how to make this Pizza, we are not going to have such a Pizza a second time", the cook decides. "OK, it's a Pizza with olives, but tomato sauce on top and cheese at the bottom and bake it in the oven for 90 minutes until it's black and hard like a flat rock of granite."

(Read the rest of the answer, it's really nice imo).

It's naive to ignore the fact there is a database - there is a database, and no matter how hard you want to abstract that, it's not going anywhere. Your application will be aware of the data source. You won't be able to 'hot swap it'. ORMs are useful but they leak because of how complicated the problem they solve is and for plenty of performance reasons (Like Select n+1 for example).

Related Solutions

Is Moving Entity Framework objects over a webservice really the best way

There's no doubt you could optimize this application - any application can be optimized. But before you dive in are you sure you need to do this? Is there a problem with the current process - is it too slow, too expensive, is someone complaining? If you're just doing this as an iterative improvement & imagine you'll be a hero if you can reduce data transfer by 30%, the risks are far bigger than the benefits. Rewriting your service contracts will mean you'll need to add transformation code at each end, which means rehydrating EF objects and ensuring they have the correct state to reattach to the data context. It sounds easy but it's a big change.

You should definitely profile how expensive the EF objects are compared to equivalent DTOs. You're running on a hunch at the moment. How much data will you save by making this change?

Are there simpler, more obvious improvements? When optimizing WCF services in the past I've identified that a huge overhead in service request size was Windows Authentication headers - enormous security tokens being passed between client and server - which can be replaced with a much smaller certificate. Is all the data being sent completely necessary? I assume you're sending binary (net.tcp) rather than text (http), but if you're not that's an obvious improvement.

DTOs are a useful pattern, and they are championed heavily by MVC guys, but this isn't because of any data-saving concern - it's because they provide a service interface, an abstraction from the database. Without a DTO you add a dependency to your database model. This doesn't apply in your case because it seems you have the same database model on both ends. The simplest approach will be to send the EF objects over the wire and directly insert them, just as you're currently doing.

Reducing data traffic will save some money. How much money? Enough to warrant your time developing this solution, additional maintenance time due to increased application complexity?

Xamarin Development – Web API vs Entity Framework with Repository Pattern

Do we need to transition to a Web API based call to the database to support cross platform?

Yes.

Here's why: you'll need a common, web-enabled API for your three front-ends to talk to. Entity Framework won't execute queries to your DB server over the Internet.

Is there any reason we should get away from Entity Framework?

No. You can use it to build the Web API. You can even retain your Repository/Unit of Work layers, if you want to. Here's what your architecture might conceptually look like:

DB <--> Entity Framework <--> Repository/UOW <--> Web API Endpoint <==> Frontend
                                                                    ^
                                                                    JSON over Internet