![]() ![]() As soon as the first URL is added to the queue, one of the worker threads picks it up and starts downloading it. ![]() The next step is to retrieve the feed contents using the feedparser module and enqueue the URLs of the enclosures. # Set up some threads to fetch the enclosures. That means it is safe to start the threads before there is anything in the queue. When download_enclosures() processes the statement url = q.get(), it blocks and waits until the queue has something to return. Once the target function for the threads is defined, the worker threads can be started. ![]() # A real app wouldn't use hard-coded data. Listing 2.59: fetch_podcasts.py from queue import Queue The example uses hard-coded values for the number of threads and list of URLs to fetch. Usually, these would come from user inputs (e.g., preferences or a database). It does not have enough error handling for production use, but the skeleton implementation illustrates the use of the queue module.įirst, some operating parameters are established. The program reads one or more RSS feeds, queues up the enclosures for the five most recent episodes from each feed to be downloaded, and processes several downloads in parallel using threads. The source code for the podcasting client in this section demonstrates how to use the Queue class with multiple threads. The order of processing for items added to the queue while the consumer threads are running depends on thread context switching. This example has multiple threads consuming the jobs, which are processed based on the priority of items in the queue at the time get() was called. Threading.Thread(target=process_job, args=(q,)), Print('Processing job:', next_job.description) Listing 2.58: queue_priority.py import functoolsĭef _init_(self, priority, description): PriorityQueue uses the sort order of the contents of the queue to decide which item to retrieve. For example, print jobs from the payroll department may take precedence over a code listing that a developer wants to print. Sometimes the processing order of the items in a queue needs to be based on characteristics of those items, rather than just the order they are created or added to the queue. The item most recently put into the queue is removed by get. In contrast to the standard FIFO implementation of Queue, the LifoQueue uses last-in, first-out ordering (normally associated with a stack data structure). This example uses a single thread to illustrate that elements are removed from the queue in the same order in which they are inserted. Elements are added to one “end” of the sequence using put(), and removed from the other end using get(). The Queue class implements a basic first-in, first-out container. If you do not, you may want to read some of the references before continuing. This discussion assumes you already understand the general nature of a queue. ![]()
0 Comments
Leave a Reply. |
AuthorWrite something about yourself. No need to be fancy, just an overview. ArchivesCategories |