05-Leveldb实现-Memtable

Posted 2023-02-21 anda0109

Memtable

leveldb数据写入时并非直接落盘，而是先保存在内存中，在内存中的数据按key进行排序。当内存中的数据达到一定大小时，再将这批数据批量写入磁盘。在内存中的数据结构我们称之为Memtable，本节将介绍Memtable的实现。
先看代码：

class MemTable 
 public:
  // MemTables are reference counted.  The initial reference count
  // is zero and the caller must call Ref() at least once.
  explicit MemTable(const InternalKeyComparator& comparator);

  MemTable(const MemTable&) = delete;
  MemTable& operator=(const MemTable&) = delete;

  // Increase reference count.
  void Ref()  ++refs_; 

  // Drop reference count.  Delete if no more references exist.
  void Unref() 
    --refs_;
    assert(refs_ >= 0);
    if (refs_ <= 0) 
      delete this;
    
  

  // Returns an estimate of the number of bytes of data in use by this
  // data structure. It is safe to call when MemTable is being modified.
  size_t ApproximateMemoryUsage();

  // Return an iterator that yields the contents of the memtable.
  //
  // The caller must ensure that the underlying MemTable remains live
  // while the returned iterator is live.  The keys returned by this
  // iterator are internal keys encoded by AppendInternalKey in the
  // db/format.h,cc module.
  Iterator* NewIterator();

  // Add an entry into memtable that maps key to value at the
  // specified sequence number and with the specified type.
  // Typically value will be empty if type==kTypeDeletion.
  void Add(SequenceNumber seq, ValueType type, const Slice& key,
           const Slice& value);

  // If memtable contains a value for key, store it in *value and return true.
  // If memtable contains a deletion for key, store a NotFound() error
  // in *status and return true.
  // Else, return false.
  bool Get(const LookupKey& key, std::string* value, Status* s);

 private:
  friend class MemTableIterator;
  friend class MemTableBackwardIterator;

  struct KeyComparator 
    const InternalKeyComparator comparator;
    explicit KeyComparator(const InternalKeyComparator& c) : comparator(c) 
    int operator()(const char* a, const char* b) const;
  ;

  typedef SkipList<const char*, KeyComparator> Table;

  ~MemTable();  // Private since only Unref() should be used to delete it

  KeyComparator comparator_;
  int refs_;
  Arena arena_;
  Table table_;
;

上述代码中Add和Get函数即为向Memtable中写入和读取数据接口。
void Add(SequenceNumber seq, ValueType type, const Slice& key,
const Slice& value);第一个参数seq即全局唯一的序号，每进行一次更新操作序号加1，第二个参数type是表示该操作的类型：包括PUT和DELETE两种，key表示要添加或删除的key，value表示要添加的key对应的value(当type为DELETE时value为空）。
bool Get(const LookupKey& key, std::string* value, Status* s);当Memtable中存在所查找的key时，value存储其值，函数返回true；当Memtable中存在所查找key的删除标记时，s中存储NotFound的状态值，函数返回true；其他情况，函数返回false。
Memtable具体如何实现呢？主要在它的成员变量Table table_，即SkipList。

class SkipList 
 private:
  struct Node;

 public:
  // Create a new SkipList object that will use "cmp" for comparing keys,
  // and will allocate memory using "*arena".  Objects allocated in the arena
  // must remain allocated for the lifetime of the skiplist object.
  explicit SkipList(Comparator cmp, Arena* arena);

  SkipList(const SkipList&) = delete;
  SkipList& operator=(const SkipList&) = delete;

  // Insert key into the list.
  // REQUIRES: nothing that compares equal to key is currently in the list.
  void Insert(const Key& key);

  // Returns true iff an entry that compares equal to key is in the list.
  bool Contains(const Key& key) const;

  // Iteration over the contents of a skip list
  class Iterator 
   public:
    // Initialize an iterator over the specified list.
    // The returned iterator is not valid.
    explicit Iterator(const SkipList* list);

    // Returns true iff the iterator is positioned at a valid node.
    bool Valid() const;

    // Returns the key at the current position.
    // REQUIRES: Valid()
    const Key& key() const;

    // Advances to the next position.
    // REQUIRES: Valid()
    void Next();

    // Advances to the previous position.
    // REQUIRES: Valid()
    void Prev();

    // Advance to the first entry with a key >= target
    void Seek(const Key& target);

    // Position at the first entry in list.
    // Final state of iterator is Valid() iff list is not empty.
    void SeekToFirst();

    // Position at the last entry in list.
    // Final state of iterator is Valid() iff list is not empty.
    void SeekToLast();

   private:
    const SkipList* list_;
    Node* node_;
    // Intentionally copyable
  ;

 private:
  enum  kMaxHeight = 12 ;

  inline int GetMaxHeight() const 
    return max_height_.load(std::memory_order_relaxed);
  

  Node* NewNode(const Key& key, int height);
  int RandomHeight();
  bool Equal(const Key& a, const Key& b) const  return (compare_(a, b) == 0); 

  // Return true if key is greater than the data stored in "n"
  bool KeyIsAfterNode(const Key& key, Node* n) const;

  // Return the earliest node that comes at or after key.
  // Return nullptr if there is no such node.
  //
  // If prev is non-null, fills prev[level] with pointer to previous
  // node at "level" for every level in [0..max_height_-1].
  Node* FindGreaterOrEqual(const Key& key, Node** prev) const;

  // Return the latest node with a key < key.
  // Return head_ if there is no such node.
  Node* FindLessThan(const Key& key) const;

  // Return the last node in the list.
  // Return head_ if list is empty.
  Node* FindLast() const;

  // Immutable after construction
  Comparator const compare_;
  Arena* const arena_;  // Arena used for allocations of nodes

  Node* const head_;

  // Modified only by Insert().  Read racily by readers, but stale
  // values are ok.
  std::atomic<int> max_height_;  // Height of the entire list

  // Read/written only by Insert().
  Random rnd_;
;

Skiplist类提供了两个主要方法，Insert和Contains。前者即向Skiplist中插入一个key，后者则是判断Skiplist中是否存在这个key。当然还提供了一个Iterator类，用于迭代Skiplist中的所有key值。

Skiplist

接下来介绍一下Skiplist:
Skiplist故名思义，即跳跃的链表。传统的链表我们要查找时只能依次遍历，效率很低；而skiplist在查找时，则可以跳跃的方式遍历，从而提高了查询效率。其原理是怎么样的呢？
跳表是由William Pugh发明。他在 Communications of the ACM June 1990, 33(6) 668-676 发表了Skip lists: a probabilistic alternative to balanced trees，在该论文中详细解释了跳表的数据结构和插入删除操作。
跳表是平衡树的一种替代的数据结构，但是和红黑树不相同的是，跳表对于树的平衡的实现是基于一种随机化的算法的，这样也就是说跳表的插入和删除的工作是比较简单的。

下面来研究一下跳表的核心思想：

先从链表开始，如果是一个简单的链表，那么我们知道在链表中查找一个元素I的话，需要将整个链表遍历一次。

如果是说链表是排序的，并且节点中还存储了指向前面第二个节点的指针的话，那么在查找一个节点时，仅仅需要遍历N/2个节点即可。

这基本上就是跳表的核心思想，其实也是一种通过“空间来换取时间”的一个算法，通过在每个节点中增加了向前的指针，从而提升查找的效率。而leveldb的Memtable即使用了基于该思想的skiplist。