You should not use $objectManager directly because it takes control from clients of your code and leads to higher code coupling

We split all objects to two groups: injectables & non-injectables.

Injectables - all services, mostly stateless classes like FrontController or EventManager should be requested in object constructors.

Non-Injectables - all entities, mostly stateful objects like Product or Category should be created through Factories. And Factories should be requested in constructors.

You can read more here: http://devdocs.magento.com/guides/v2.0/extension-dev-guide/depend-inj.html#dep-inj-mod-type-life-mgmt

If there is a repository and it does what you need well, always prefer the repository.

Repositories are part of the Service Contracts (they are implementations of interfaces in Api), this means they are meant as a public interface to other modules.

Use Repositories for full loading

$model->load() is not part of the service contract. I had a question on that particular topic, you might find the answers useful: Is there ever a reason to prefer $model->load() over service contracts?

Use Factories to create new entities

Repositories do not come with methods to create a new entity, so in that case, you will need a factory. But use the factory for the interface, such as Magento\Catalog\Api\Data\ProductInterfaceFactory - it will create the right implementation based on DI configuration.

Then use the repository->save() method to save it.

Use Collection Factories if you need more control

The following is not official Magento best practice, but currently, repositories do not give you fine control over what to load. The search criteria API lets you define filters, but for example, there is no way to select particular EAV attributes or specify which index tables to join.

These are implementation details, hidden from the service contract APIs, but often these implementation details matter and you get poor performance if you ignore them. For that reason, as soon as the repositories are limiting me I don't hesitate anymore to use the underlying collections.