Magento 1.9 – Best Practices for Unit Tests

For unit tests you should use Magento\Framework\TestFramework\Unit\Helper\ObjectManager which allows to create any object with needed arguments (if they aren't passed, mocks will be created instead).

The Magento\TestFramework\ObjectManager is used only by integration tests and you get the PHP Fatal error: Class 'Magento\TestFramework\ObjectManager' not found error because the different autoload.php file should be used.

For more details, please, see the testing section from official the Magento 2 dev docs.

Using the bundled PHPUnit version, even if it's ancient, is probably the best way to go since that will allow running the tests for all modules together during CI.

I think writing tests in a manner that is incompatible with the bundled test framework greatly reduces the value of the tests.
Of course you could set up CI to run your tests with a different version of PHPUnit, but that adds a lot of complexity to the build system.

That said, I do agree with you that it's not worth supporting PHP 5.6. I use PHP7 scalar type hinting and return type hinting as much as possible (plus, I don't care about the market place).

In order to work around the limitations of the PHPUnit 4.1 mocking library there are at least two rather straight forward workarounds that I've used in the past:

1. Use anonymous or regular classes to build your test doubles, for example

   $fooIsFalseStub = new class extends Foo implements BarInterface() {
       public function __construct(){};
       public function isSomethingTrue(string $something): bool
       {
           return false;
       }
   };
   

2. Use the bundled PHPUnit but a third party mocking library that can be included via composer with require-dev, for example https://github.com/padraic/mockery. All mocking libraries I tried can very easily be used with any testing framework, even a very old version of PHPUnit like 4.1.

Neither of those has any technical advantage over the other. You can implement any required test double logic with either.

Personally I prefer using anonymous classes because that doesn't add to the number of external dependencies, and also it's more fun to write them that way.

EDIT:
To answer your questions:

Does Mockery 'solve' the problem of PHPUnit 4.1.0 not being able to properly handle PHP7 type hinting?

Yes, see the example below.

And what are the benefits of anonymous classes over mocking?

Using anonymous classes to create test doubles is also "mocking", it's not really different from using a mocking library such as PHPUnits or Mockery or another.
A mock is just on specific type of test double, regardless of how it is created.
One small difference between using anonymous classes or a mocking library is that anonymous classes don't have an external library dependency, since it's just plain PHP. Otherwise there are no benefits or drawbacks. It's simply a matter of preference. I like it because it illustrates that testing is not about any test framework or mocking library, testing is just writing code that executes the system under test and automatically validates it works.

And how about updating the PHPUnit version in the main composer.json file to 5.3.5 (the latest version supporting PHP7 ánd having public mocking-methods (required by Magento 2's own tests))?

This might be problematic since the tests in other modules and the core are only tested with PHPUnit 4.1, and as such you might encounter false failures in CI. I think it's best to stick with the bundled version of PHPUnit for that reason. @maksek said they will be updating PHPUnit, but there is no ETA for that.

Example of a test with a test double of a class that requires PHP7 running with PHPUnit 4.1, using the Mockery library:

<?php

declare(strict_types = 1);

namespace Example\Php7\Test\Unit;

// Foo is a class that will not work with the mocking library bundled with PHPUnit 4.1 
// The test below creates a mock of this class using mockery and uses it in a test run by PHPUnit 4.1
class Foo
{
    public function isSomethingTrue(string $baz): bool
    {
        return 'something' === $baz; 
    }
}

// This is another class that uses PHP7 scalar argument types and a return type.
// It is the system under test in the example test below.
class Bar
{
    private $foo;

    public function __construct(Foo $foo)
    {
        $this->foo = $foo;
    }

    public function useFooWith(string $s): bool
    {
        return $this->foo->isSomethingTrue($s);
    }
}

// This is an example test that runs with PHPUnit 4.1 and uses mockery to create a test double
// of a class that is only compatible with PHP7 and younger.
class MockWithReturnTypeTest extends \PHPUnit_Framework_TestCase
{
    protected function tearDown()
    {
        \Mockery::close();
    }

    public function testPHPUnitVersion()
    {
        // FYI to show this test runs with PHPUnit 4.1
        $this->assertSame('4.1.0', \PHPUnit_Runner_Version::id());
    }

    public function testPhpVersion()
    {
        // FYI this test runs with PHP7
        $this->assertSame('7.0.15', \PHP_VERSION);
    }

    // Some nonsensical example test using a mock that has methods with
    // scalar argument types and PHP7 return types.
    public function testBarUsesFoo()
    {
        $stubFoo = \Mockery::mock(Foo::class);
        $stubFoo->shouldReceive('isSomethingTrue')->with('faz')->andReturn(false);
        $this->assertFalse((new Bar($stubFoo))->useFooWith('faz'));
    }
}