Performance Optimization: CPU and Memory Usage in Pipelined DML

[ekexium]

Enhancement

There are some usages of CPU and memory allocations that could be optimized. Some of them may be intentional trade-offs and some are unintentional.

In the context of pipelined DML, the majority of latency comes from TiDB, making the execution speed in TiDB more critical to overall performance.

Sub-tasks

• Batch frequent tracker.Consume #53973
• Save unnecessary RuneCountInString #53942
• Optimize slice allocations in the hot path #53837
• session.SetMemoryFootprintChangeHook is called for each call of Txn() #53763
• tikvTxn.GetMembuffer() does not have to create new memBuffer objects #53714
• TransactionContext.UpdateDeltaForTable is inefficient #53660
• TableCommon.getCols() can be memorized #53604
• *: optimize error generation, debug log and AllocHandle in the hot write path #54308
• Optimize memdb traversals for repeated key access #54604
• GenKeyExistsErr, in insert on duplicate update key
• growSlice in genIndexValueVersion0, in insert on duplicate update key
• strings.Join in RequestSource.GetRequestSource, in insert on duplicate update key
• index.WithCtx
• Allocating rows in getRow
• Lower the frequency of onMemChange
• batch more rows in UpdateDeltaForTableFromColSlice
• SetMemoryFootprintChangeHook in Txn
• growSlice in Codec.decodeColumn, in delete
• debug logs, even if they are not enabled
• Error.Equal and IsErrNotFound