保存最后N个元素

Posted 2020-09-08 dahu的菜园子

cookbook系列

问题：对要搜索的值的最后几项做个有限的历史记录。

方案：

 1 #coding=utf-8
 2 from collections import deque
 3 
 4 def search(lines, pattern, history=5):
 5     previous_lines = deque(maxlen=history)  #deque双端队列
 6     for line in lines:
 7         if pattern in line:
 8             yield line, previous_lines      #生成器，双端队列保存5个，意味着在找到pattern之前会记录5行数据，pattern就在line里面
 9         previous_lines.append(line)
10 
11 # Example use on a file
12 if __name__ == \'__main__\':
13     with open(\'somefile.txt\') as f:
14         for line, prevlines in search(f, \'python\', 5):
15             for pline in prevlines:
16                 print pline,  #先输出队列里的5个
17             print line,     #在输出pattern的line
18             print(\'-\'*20)

案例文件：somefile.txt

=== Keeping the Last N Items

==== Problem

You want to keep a limited history of the last few items seen
during iteration or during some other kind of processing.

==== Solution

Keeping a limited history is a perfect use for a `collections.deque`.
For example, the following code performs a simple text match on a
sequence of lines and prints the matching line along with the previous
N lines of context when found:

[source,python]
----
from collections import deque

def search(lines, pattern, history=5):
    previous_lines = deque(maxlen=history)
    for line in lines:
        if pattern in line:
            for pline in previous_lines:
                print(lline, end=\'\')
            print(line, end=\'\')
            print()
        previous_lines.append(line)

# Example use on a file
if __name__ == \'__main__\':
    with open(\'somefile.txt\') as f:
         search(f, \'python\', 5)
----

==== Discussion

Using `deque(maxlen=N)` creates a fixed size queue.  When new items
are added and the queue is full, the oldest item is automatically
removed.   For example:

[source,pycon]
----
>>> q = deque(maxlen=3)
>>> q.append(1)
>>> q.append(2)
>>> q.append(3)
>>> q
deque([1, 2, 3], maxlen=3)
>>> q.append(4)
>>> q
deque([2, 3, 4], maxlen=3)
>>> q.append(5)
>>> q
deque([3, 4, 5], maxlen=3)
----

Although you could manually perform such operations on a list (e.g.,
appending, deleting, etc.), the queue solution is far more elegant and
runs a lot faster.

More generally, a `deque` can be used whenever you need a simple queue
structure.  If you don\'t give it a maximum size, you get an unbounded
queue that lets you append and pop items on either end.  For example:

[source,pycon]
----
>>> q = deque()
>>> q.append(1)
>>> q.append(2)
>>> q.append(3)
>>> q
deque([1, 2, 3])
>>> q.appendleft(4)
>>> q
deque([4, 1, 2, 3])
>>> q.pop()
3
>>> q
deque([4, 1, 2])
>>> q.popleft()
4
----

Adding or popping items from either end of a queue has O(1)
complexity.   This is unlike a list where inserting or removing
items from the front of the list is O(N).    

View Code

运行结果：

H:\\Python27_64\\python.exe H:/myfile/python-cookbook-master/src/1/keeping_the_last_n_items/example.py
Keeping a limited history is a perfect use for a `collections.deque`.
For example, the following code performs a simple text match on a
sequence of lines and prints the matching line along with the previous
N lines of context when found:

[source,python]
--------------------
        previous_lines.append(line)

# Example use on a file
if __name__ == \'__main__\':
    with open(\'somefile.txt\') as f:
         search(f, \'python\', 5)
--------------------

进程已结束,退出代码0

View Code

讨论：

第5行双端队列deque的用法

固定长度队列：
>>> q = deque(maxlen=3)
>>> q.append(1)
>>> q.append(2)
>>> q.append(3)
>>> q
deque([1, 2, 3], maxlen=3)
>>> q.append(4)
>>> q
deque([2, 3, 4], maxlen=3)
>>> q.append(5)
>>> q
deque([3, 4, 5], maxlen=3)
双端队列：
>>> q = deque()
>>> q.append(1)
>>> q.append(2)
>>> q.append(3)
>>> q
deque([1, 2, 3])
>>> q.appendleft(4)
>>> q
deque([4, 1, 2, 3])
>>> q.pop()
3
>>> q
deque([4, 1, 2])
>>> q.popleft()
4

两者皆有：设置好固定长度时，左右两端均可添加和删除

 

使用双端队列要比列表 方便、简洁！