TiDB 架构解析与实践.pptx

1、TiDB Architecture and PracticeAbout Me Shen Li (申砾) VP of Engineering PingCAP Infrastructure Engineer / Open-source advocator Netease/360/PingCAP Tech lead of TiDB About PingCAP Since 2015 Office Beijing, Shanghai, Guangzhou, Hangzhou, Chengdu, Shenzhen,Silicon Valley Open-Source infrastructure soft

2、ware TiDB, TiKV, TiSpark, TiDB-OperatorAgenda Introduction Architecture Evolution TiDB in Real WorldPart I - IntroductionWhy we want to build a “New”SQL Database From the beginning RDBMSRDBMSNoSQLNewSQL How to scale? Middleware1970s20102015Present NoSQL NewSQL: F1 & SpannerMySQLPostgreSQLOracleRedis

3、HBaseCassandraMongoDBGoogleSpannerGoogle F1TiDBDB2.Whats TiDB?TiDB - A Distributed, Consistent, Scalable, SQL Database that supports the bestfeatures of both traditional RDBMS and NoSQL. Build from scratch Key features: Horizontal Scalability High Availability ACID Transaction SQL at ScaleOpen Sourc

4、e From first-day Active community 300+ contributorsOne of the most popularOpen Source DistributedRelational Database inthe world!Part II - ArchitectureTiDB ArchitecturePDPDData locationTSO/Data locationPDPD ClusterMetadataSparkDriverTiDBTiDBTiDBTiDBTiKVTiKVTiKVTiKVTiKVTiKVJobMySQL ClientsSyncerDistS

5、QL APIDistSQL APIWorkerWorkerWorker.TiDB.Spark ClusterTiDB ClusterTiKV Cluster (Storage)TiSparkThe SQL Layer Stateless SQL layer Client can connect to any existingtidb-server instanceLogicalPlanOptimizedLogical PlanSQLAST TiDB *will not* re-shuffle the dataacross different tidb-serversStatisticsCost

6、 Model Full-featured SQL Layer Speak MySQL wire protocol Why not reusing MySQL? Homemade parser & lexer RBO & CBOSelectedPhysical Plantidb-serverTiKVTiKVTiKVTiKVTiKVTiKV Secondary index support DML & DDLTiKV ClusterPingCAP.comThe Storage Layer (1/2)Client The storage layer for TiDB Distributed Key-V

7、alue storage engineDataflow Support ACID Transactions Replicate logs by Raft Range partitioning Split / merge dynamically SQL operators pushdownPDMetadataTiKVTiKVTiKVTiKVTiKVTiKVTiKVTiKVTiKVTiKVTiKVTiKVPDPDPlacement DriverTiKV NodesPingCAP.comThe Storage Layer (2/2)ClientPlacementDriverRPCRPCRPCRPCP

8、D 1PD 2PD 3TiKV node 1Store 1TiKV node 2Store 2TiKV node 3Store 3TiKV node 4Store 4Region 1Region 1Region 1Region 1Region 3Region 2Region 5Region 3Region 2Region 5Region 4Region 4Region 3Region 5Region 4Raft GroupPingCAP.comSchedulingPingCAP.comEcosystem ToolsPart III - EvolutionHTAP Hybrid transact

9、ional/analytical platform Real-time Data No more ETL! Cascades Optimizer Vectorized+Parallel Execution Engine Row-Column mixed Storage Engine Isolation of resourcesCloud NativeTiDB OperatorTiDB Controller ManagerTiDB SchedulerTiDBSchedulerTiDB Cluster ControllerTiDBControllerPDControllerTiKVControllerKubeSchedulerKubernetes CoreSchedulerController ManagerAPI ServerPart IV - TiDB in Real WorldSay goodbye to shardingReal-time Data Platform Real-time Data Convergence Platform Middle-End SystemPingCAP.comGeo ReplicationThank You !PingCAP.com