On teh way

LevelDB Source Reading (1): Structure LevelDB “is an open source on-disk key-value store.” After I read some documents, I have some basic understanding of LevelDB. So I come up with some questions about structure of LevelDB to answer when reading the source code. Structure Log File: repair/recover db A log file (*.log) stores a sequence of recent updates. Each update is appended to the current log file. The log file contents are a sequence of 32KB blocks. The only exception is that the tail of the file may contain a partial block. Block format: Each block consists of a sequence of records: block := record* trailer? record := checksum: uint32 // crc32c of type and data[] ; little-endian length: uint16 // little-endian type: uint8 // One of FULL, FIRST, MIDDLE, LAST data: uint8[length] // data is LengthPrefixedSlice with type from batch data definition in Block : data: also named `writeBatch` in levelDB // WriteBatch header has an 8-byte ...

LevelDB Source Reading (3): Compaction In the last blog that analyzes read/write process of Leveldb, we can see writing only happens to log file and memory table, then it relies on the compaction process to move the new updates into persistent sorted table for future use. So the compaction is a crucial part for the design, and we will dive into it in this blog. Compaction LevelDB compacts its underlying storage data in the background to improve read performance. The upper sentence is cited from the document of Leveldb , and we will see how it is implemented via code review. Background compaction // db_impl.cc void DBImpl :: MaybeScheduleCompaction ( ) { // use background thread to run compaction env_ - > Schedule ( & DBImpl :: BGWork , this ) ; } Two main aspects // arrange background compaction when Get, Open, Write void DBImpl :: BackgroundCompaction ( ) { // compact memtable CompactMemTable ( ) ; // compact ...