I need a queue which multiple threads can put stuff into, and multiple threads may read from.
Python has at least two queue classes, Queue.Queue
and collections.deque
, with the former seemingly using the latter internally. Both claim to be thread-safe in the documentation.
However, the Queue docs also state:
collections.deque is an alternative
implementation of unbounded queues
with fast atomic append() and
popleft() operations that do not
require locking.
Which I guess I don't quite unterstand: Does this mean deque isn't fully thread-safe after all?
If it is, I may not fully understand the difference between the two classes. I can see that Queue adds blocking functionality. On the other hand, it loses some deque features like support for the in-operator.
Accessing the internal deque object directly, is
x in Queue().deque
thread-safe?
Also, why does Queue employ a mutex for it's operations when deque is thread-safe already?
Best Answer
Queue.Queue
andcollections.deque
serve different purposes. Queue.Queue is intended for allowing different threads to communicate using queued messages/data, whereascollections.deque
is simply intended as a datastructure. That's whyQueue.Queue
has methods likeput_nowait()
,get_nowait()
, andjoin()
, whereascollections.deque
doesn't.Queue.Queue
isn't intended to be used as a collection, which is why it lacks the likes of thein
operator.It boils down to this: if you have multiple threads and you want them to be able to communicate without the need for locks, you're looking for
Queue.Queue
; if you just want a queue or a double-ended queue as a datastructure, usecollections.deque
.Finally, accessing and manipulating the internal deque of a
Queue.Queue
is playing with fire - you really don't want to be doing that.