I could be waaaaaay off, but...

It seems that a model is necessary to use the library, you just don't care which one, right? Further, the library doesn't care how the model gets its data, or where the data comes from, it just wants an object that meets the criteria of the interface.

If the library doesn't need the connection object, then make that the responsibility of the model.

interface MyFooModelInterface 
{
    public function getItem($id);
    public function addItem($item);
    ...
}

And here is a simple class example:

class MyLib
{
    protected $model;

    public function __construct(MyFooModelInterface $model)
    {
        $this->model = $model;
    }

    public function getItemById($id)
    {
        return $this->model->getItem($id);
    }
}

And a Model example:

class MyModel implements MyFooModelInterface
{
    protected $conn;

    public function __construct(\PDO $conn)
    {
         $this->conn = $conn;
    }

    public function getItem($id)
    {
        ...
    }
    public function addItem($item)
    {
        ...
    }
}

And here is an example of using it:

// the model is responsible for handling the data retrieval, not the library
$model = new Model($conn);

$lib = new MyLib($model);
return $lib->getItem($id);

Now, if I decide that I want a file-based solution, I can easily swap the model. To test the model, the connection obj is injected, so that's easy. To test the library, the model is also injected, so, again, that's easy.

In this case, the model is only responsible for getting the data, while the library is responsible for handling and processing the data. Neither cares what or how the other one does it.

EDIT

the library with a factory method to create the model if none was given...

class MyLib
{
    protected $model;

    public function __construct(MyFooModelInterface $model = null, $conn = null)
    {
        // if we have a model, use it
        if (!is_null($model)) { 
            $this->model = $model; 
        }

        // if we don't have a model, but have a connection obj, make a model
        if (is_null($model) && !is_null($conn)) { 
            $this->model = $this->factoryModel($conn); 
        }

        // if we don't have either, quit and go home
        if (is_null($model) && is_null($conn)) {
            throw new Exception('need moar inputs');
        }
    }

    protected factoryModel($conn)
    {
        $this->model = new MyModel($conn);
    }

    public function getItemById($id)
    {
        return $this->model->getItem($id);
    }
}

Indeed, you could have been using PDO directly. The only real thing it does is to rethrow PDOException as RunTimeException, a practice which should be avoided at all costs.

This being said, PDO doesn't keep you from having to create special adapters for a given database, since different databases have different syntax and functionality which cannot possibly be handled by PDO. It works for simple stuff like PDOStatement::rowCount(), but PDO won't help you if, for example, you need to LIMIT/OFFSET the number of results in MySQL and Microsoft SQL, forcing you to write LIMIT/OFFSET query for one and WHERE rowNumber BETWEEN ... AND ... for another.

Moreover, the presence of PDO doesn't mean you shouldn't have a database layer either: your business layer shouldn't call PDO directly.

Example

If a website needs to display the number of users, business layer will call database layer IDatabase similarly to this:

$countUsers = $this->data->countAllUsers();

Then, you'll have an implementation for Microsoft SQL Server:

class SqlServerDatabase implements IDatabase
{
    ...
    public function countAllUsers()
    {
        $query = 'select count(1) from [Community].[User]';
        $statement = $this->connection->prepare($query);
        $statement->execute();
        return $statement->fetchColumn();
    }
    ...
}

and another one for MySQL:

class SqlServerDatabase implements IDatabase
{
    ...
    public function countAllUsers()
    {
        $query = 'select count(*) from `user`';
        $statement = $this->connection->prepare($query);
        $statement->execute();
        return $statement->fetchColumn();
    }
    ...
}

If you have a lot of table-counting, you may want to refactor that to reduce code duplication. Without PDO, you won't be able to do that. With PDO, you can:

abstract class DatabaseCommon
{
    protected function countAllRows(TableName $tableName, $preferOneToAsterisk = false)
    {
        $query = $preferOneToAsterisk ?
            'select count(1) from ' . $tableName->sanitize() :
            'select count(*) from ' . $tableName->sanitize();

        $statement = $this->connection->prepare($query);
        $statement->execute();
        return $statement->fetchColumn();
    }
}

class SqlServerDatabase extends DatabaseCommon implements IDatabase
{
    public function countAllUsers()
    {
        return $this->countAllRows(
            new MicrosoftSqlTableName('Community', 'User'),
            true
        );
    }
}

class SqlServerDatabase extends DatabaseCommon implements IDatabase
{
    public function countAllUsers()
    {
        return $this->countAllRows(new MySqlTableName('user'));
    }
}