高级数据库课件：09-MongoDB.pptx

1、open-source, high-performance, document-oriented database1 OVERVIEWConceptsWho provides MongoDB in the cloud?http:/ is using MongoDB?http:/ DBMSDatabase ModelScore1OracleRelational DBMS1466.362MySQLRelational DBMS1278.363Microsoft SQL ServerRelational DBMS1118.054PostgreSQLRelational DBMS280.95Mongo

2、DB Document store279.056DB2Relational DBMS198.77Microsoft AccessRelational DBMS146.498Cassandra Wide column store108.919SQLiteRelational DBMS107.9710RedisKey-value store95.49200k+Education RegistrationsConceptsCommunity30k+MongoDB User Group Members8 Million +DownloadsConceptsDrivershttp:/docs.mongo

3、db.org/ecosystem/drivers/MongoDBDrivernCnC+nC#nJavanNode.jsnPerlnPHPnPythonnRubynScalaAppHorizontally ScalableArchitecturesno joinsno complex transactions+1.1 基本概念JSON-style Documents hello: “world” x16x00 x00 x00 x02hellox00 x06x00 x00 x00worldx00 x00http:/bsonspec.orgrepresented as BSONJust like a

4、 light and friendly XMLFlexible “Schemas”In collection db.posts:author: “mike”, links: 3, date: Sun Jul 18 2010 14:40:20 GMT-0700 (PDT) text: “blah blah”author: “eliot”, date: Sun Jul 18 2010 14:40:22 GMT-0700 (PDT) text: “Here is MongoDB .”, views: 10Potentially all documents in the same collection

5、Embedded Document _id: ObjectId(4d1009c7262bb4b94af1cea4) author_id: “1346”, date: Sun Jul 18 2010 14:40:20 GMT-0700 (PDT), title: “my story” text: “once upon a time .”, tags: “novel”,”english”, Comments: user_id: 234, text: “awesome dude”, user_id: 1235, text: “that made me cry”little need for join

6、sor transactions across documents!FeaturesComplex queryingAtomic updates with modifiersIndexing (unique, compound, Geo)Aggregation and Map / ReduceCapped CollectionsPowerful Shell (Javascript)GridFS: file storageMongoDB Pluggable Storage EnginesReplicationmasterslaveslaveUsing Replica Set:- pool of

7、servers with 1 master- automatic master election and failover- distributed reads (slaveOk)slaveClientClientShardingclientmongos.mongosmongod.ShardsmongodmongodmongodConfigServersmongodmongodmongodmongodmongodmongodmongodmongod.For large datasets, or write heavy systemSupportOS: Mac OS X, Windows, Li

8、nux, Solaris, 32/64 bitsDrivers: C, C#, C+, Haskell, Java, Javascript, Perl, PHP, Python, Ruby, Scala. + community driversOpen-source project with active community, Wiki, Google Group, 10gen consulting / supportProduction Examples Shutterfly Fourquare Craigslist bit.ly IGN Sourceforge Etsy the New Y

9、ork Times Business Insider Gilt Groupe Intuit College humor Evite Disqus Justin.tv Heartbeat Hot Potato Eventbrite Sugar crm Electronic Arts .New Post post = author: mike,. date: new Date(),. text: my blog post,. tags: mongodb, intro db.posts.save(post) db.posts.findOne() _id : ObjectId(4d2f944103e8

10、fdbb36f6d205),author : mike,date : ISODate(2011-01-14T00:08:49.933Z),text : my blog post,tags : mongodb,introA Quick Asidespecial keypresent in all documentsunique across a Collectionany type you want_id1.2 基本操作Update db.posts.update(_id: post._id,. $set: author: tony) c = author: eliot, date: new D

11、ate(), text: great post! db.posts.update(_id: post._id,. $push: comments: c) db.posts.update(_id: post._id,. $inc: views: 1)Update db.posts.update(_id: post._id,. $set: author: tony) c = author: eliot, date: new Date(), text: great post! db.posts.update(_id: post._id,. $push: comments: c) db.posts.u

12、pdate(_id: post._id,. $inc: views: 1)Querying db.posts.findOne()_id : ObjectId(4d2f944103e8fdbb36f6d205),author : tony,comments : author : eliot,date : ISODate(2011-01-14T00:13:52.463Z),text : great post!,date : ISODate(2011-01-14T00:08:49.933Z),tags : mongodb,intro,text : my blog post,views : 1More

13、 QueryingFind by Author db.posts.find(author: tony)10 most recent posts: db.posts.find().sort(date: -1).limit(10)Posts since April 1st: april_1 = new Date(2010, 3, 1) db.posts.find(date: $gt: april_1)Adding an index to speed up: db.posts.ensureIndex(author: 1) db.posts.ensureIndex(date: 1)More Query

14、ingFind with regexp: db.posts.find(text: /post$/)Find within array: db.posts.find(tags: intro) db.posts.ensureIndex(tags: 1)Find within embedded object: db.posts.find(comments.author: eliot) db.posts.ensureIndex(comments.author: 1)More QueryingCounting: db.posts.find().count() db.posts.find(author:

15、tony).count()Paging: page = 2 page_size = 15 db.post.find().limit(page_size).skip(page * page_size)Advanced operators:$gt, $lt, $gte, $lte, $ne, $all, $in, $nin, $where db.posts.find($where: this.author = tony | this.title = foo)A data structure that can be used to make certain queries more efficien

16、t.Index on a: 1a: 0, b: 9a: 2, b: 0a: 3, b: 2a: 3, b: 5a: 3, b: 7a: 7, b: 1a: 9, b: 1Index on a: 1, b: -1a: 0, b: 9a: 2, b: 0a: 3, b: 7a: 3, b: 5a: 3, b: 2a: 7, b: 1a: 9, b: 1B-tree StructureQuery for a: 7One index per query *One range operator per query ($)Range operator must be last field in index

17、Using RAM wellmongostat for MongoDB Behaviortail the logs for current optionsiostat for disk utiltop c for CPU usageReferences to your documents, efficiently ordered by keyMaintained in a tree structure, allowing fast lookupx:1y:1x:0.5,y:0.5x:2,y:0.5x:5,y:2x:-4,y:10 x:3,y:fdb.c.findOne( _id:2 ), usi

18、ng index _id:1db.c.find( x:2 ), using index x:1db.c.find( x:$in:2,3 ), using index x:1db.c.find( x.a:1 ), using index x.a:1Matches _id:1,x:a:1db.c.find( x:a:1 ), using index x:1Matches _id:1,x:a:1, but not _id:2,x:a:1,b:2QUESTION: What about db.c.find( $where:“this.x = this.y” ), using index x:1?Ind

19、exes cannot be used for $where type queries, but if there are non-where elements in the query then indexes can be used for the non-where elements.db.c.find( x:$gt:2 ), using index x:1db.c.find( x:$gt:2,$lt:5 ), using index x:1db.c.find( x:/a/ ), using index x:1QUESTION: What about db.c.find( x:/a/ )

20、, using index x:1?The letter a can appear anywhere in a matching string, so lexicographic ordering on strings wont help. However, we can use the index to find the range of documents where x is string (eg not a number) or x is the regular expression /a/.db.c.count( x:2 ) using index x:1db.c.distinct(

21、 x:2 ) using index x:1db.c.update( x:2, x:3 ) using index x:1db.c.remove( x:2 ) using index x:1QUESTION: What about db.c.update( x:2, $inc:x:3 ), using index x:1?Older versions of mongoDB didnt support modifiers on indexed fields, but we now support this.db.c.find( ).sort( x:1 ), using index x:1db.c

22、.find( ).sort( x:-1 ), using index x:1db.c.find( x:$gt:4 ).sort( x:-1 ), using index x:1db.c.find( ).sort( x.a:1 ), using index x.a:1QUESTION: What about db.c.find( y:1 ).sort( x:1 ), using index x:1?The index will be used to ensure ordering, provided there is no better index.db.c.find( x:null ), us

23、ing index x:1Matches _id:5db.c.find( x:$exists:false ), using index x:1Matches _id:5, but not _id:6,x:nullQUESTION: What about db.c.find( x:$exists:true ), using index x:1?The index is not currently used, though we may use the index in a future version of mongoDB.All the following match _id:6,x:2,10

24、 and use index x:1db.c.find( x:2 )db.c.find( x:10 )db.c.find( x:$gt:5 )db.c.find( x:2,10 )db.c.find( x:$in:2,5 )QUESTION: What about db.c.find( x:$all:2,10 )?The index will be used to look up all documents matching x:2.db.c.find( x:10,y:20 ), using index x:1,y:1db.c.find( x:10,y:20 ), using index x:

25、1,y:-1db.c.find( x:$in:10,20,y:20 ), using index x:1,y:1db.c.find().sort( x:1,y:1 ), using index x:1,y:1db.c.find().sort( x:-1,y:1 ), using index x:1,y:-1db.c.find( x:10 ).sort( y:1 ), using index x:1,y:1QUESTION: What about db.c.find( y:10 ).sort( x:1 ), using index x:1,y:1?The index will be used t

26、o ensure ordering, provided no better index is available.db.c.find( x:$ne:1 )db.c.find( x:$mod:10,1 )Uses index x:1 to scan numbers onlydb.c.find( x:$not:/a/ )db.c.find( x:$gte:0,$lte:10,y:5 ) using index x:1,y:1Currently must scan all elements from x:0,y:5 to x:10,y:5, but some improvements may be

27、possibledb.c.find( $where:this.x = 5 )QUESTION: What about db.c.find( x:$not:/a/ ), using index x:1?The index is not used currently, but will be used in mongoDB 1.6db.c.find( a:50,50 ) using index a:2ddb.c.find( a:$near:50,50 ) using index a:2dResults are sorted closest - farthestdb.c.find( a:$withi

28、n:$box:40,40,60,60 ) using index a:2ddb.c.find( a:$within:$center:50,50,10 ) using index a:2ddb.c.find( a:$near:50,50,b:2 ) using index a:2d,b:1QUESTION: Most queries can be performed with or without an index. Is this true of geospatial queries?No. A geospatial query requires an index._id:1 index cr

29、eated automaticallyFor non-capped collectionsdb.c.ensureIndex( x:1 )Can create an index at any time, even when you already have plenty of data in your collectionCreating an index will block mongoDB unless you specify background index creationdb.c.ensureIndex( x:1, background:true )Background index c

30、reation is a still impacts performance run at non peak times if youre concerned QUESTION: Can an index be removed during background creation?Not at this time.db.c.ensureIndex( x:1, unique:true )Dont allow _id:10,x:2 and _id:11,x:2Dont allow _id:12 and _id:13 (both match x:nullWhat if duplicates exis

31、t before index is created?Normally index creation fails and the index is removeddb.ensureIndex( x:1, unique:true,dropDups:true )QUESTION: In dropDups mode, which duplicates will be removed?The first document according to the collections “natural order” will be preserved.db.system.indexes.find( ns:db

32、.c )db.c.dropIndex( x:1 )db.c.dropIndexes()db.c.reIndex()Rebuilds all indexes, removing index cruft that has built up over large numbers of updates and deletes. Index cruft will not exist in mongoDB 1.6, so this command will be deprecated.QUESTION: Why would you want to drop an index?See next slideM

33、ax 40 indexes per collectionLogically equivalent indexes are not prevented (eg x:1 and x:-1)Indexes can improve speed of queries, but make inserts slowerMore specific indexes a:1,b:1,c:1 can be more helpful than less specific indexes a:1, but sorting compound keys may not be as fast as sorting simpl

34、e keysQUESTION: Do indexes make updates slower? How about deletes?It depends finding your document might be faster, but if any indexed fields are changed the indexes must be updated.In charge of picking which index to use for a query/count/update/delete/etcImplementation is part of the magic of mong

35、o (you can read about it online not covering today)Usually it does a good job, but if you know what youre doing you can override itdb.c.find( x:2,y:3 ).hint( y:1 )Use index y:1 and avoid trying out x:1As your data changes, different indexes may be chosen. Ordering requirements should be made explici

36、t using sort().QUESTION: How can you force a full collection scan instead of using indexes?db.c.find( x:2,y:3 ).hint( $natural:1 )query test.c ntoreturn:1 reslen:69 nscanned: 100000 i: 99999.0 nreturned: 1 157msquery test.$cmd ntoreturn:1 command: count: c, query: type: 0.0, i: $gt: 99000.0 , fields

37、: reslen:64 256msquery: query: , orderby: i: 1.0 . query test.c ntoreturn:0 exception 1378ms . User Exception 10128:too much key data for sort() with no index. add an index or specify a smaller limitquery test.c ntoreturn:0 reslen:4783 nscanned:100501 query: type: 500.0 , orderby: i: 1.0 nreturned:1

38、01 390msOccasionally may see a slow operation as a result of disk activity or mongo cleaning things up some messages about slow ops are spuriousKeep this in mind when running the same op a massive number of times, and it appears slow very rarelyRecord same info as with log messages, but in a databas

39、e collection db.system.profile.find()ts : Thu Jan 29 2009 15:19:32 GMT-0500 (EST) , info : query test.$cmd ntoreturn:1 reslen:66 nscanned:0 query: profile: 2 nreturned:1 bytes:50 , millis : 0. db.system.profile.find( info: /test.foo/ ) db.system.profile.find( millis : $gt : 5 ) db.system.profile.fin

40、d().sort($natural:-1)Enable explicitly using levels (0:off, 1:slow ops (100ms), 2:all ops) db.setProfilingLevel(2);was : 0 , ok : 1 db.getProfilingLevel()2 db.setProfilingLevel( 1 , 10 ); / slow means 10msProfiling impacts performance, but not severely db.c.find( x: 1000,y:0 ).explain()cursor : Btre

41、eCursor x_1,indexBounds : x : 1000 , x : 1000,nscanned : 10,nscannedObjects : 10,n : 10,millis : 0,oldPlan : cursor : BtreeCursor x_1,indexBounds : x : 1000 , x : 1000,allPlans : cursor : BtreeCursor x_1,indexBounds : x : 1000,x : 1000, cursor : BtreeCursor y_1,indexBounds : y : 0,y : 0,cursor : Bas

42、icCursor,indexBounds : db.c.findOne( i:99999 ) _id : ObjectId(4bb962dddfdcf5761c1ec6a3), i : 99999 query test.c ntoreturn:1 reslen:69 nscanned:100000 i: 99999.0 nreturned:1 157ms db.c.find( i:99999 ).limit(1).explain()cursor : BasicCursor,indexBounds : ,nscanned : 100000,nscannedObjects : 100000,n :

43、 1,millis : 161,allPlans : cursor : BasicCursor, indexBounds : db.c.ensureIndex( i:1 ); for( i = 0; i db.c.count( type:0,i:$gt:99000 )499query test.$cmd ntoreturn:1 command: count: c, query: type: 0.0, i: $gt: 99000.0 , fields: reslen:64 256ms db.c.find( type:0,i:$gt:99000 ).limit(1).explain()cursor

44、 : BtreeCursor type_1,indexBounds : type : 0 , type : 0 ,nscanned : 49502,nscannedObjects : 49502,n : 1,millis : 349,. db.c.ensureIndex( type:1,i:1 ); for( i = 0; i db.c.find().sort( i:1 )error: $err : too much key data for sort() with no index. add an index or specify a smaller limit db.c.find().so

45、rt( i:1 ).explain()JS Error: uncaught exception: error: $err : too much key data for sort() with no index. add an index or specify a smaller limit db.c.ensureIndex( i:1 ); for( i = 0; i db.c.find( type:500 ).sort( i:1 ) _id : ObjectId(4bba4904dfdcf5761c2f917e), i : 500, type : 500 _id : ObjectId(4bb

46、a4904dfdcf5761c2f9566), i : 1500, type : 500 .query test.c ntoreturn:0 reslen:4783 nscanned:100501 query: type: 500.0 , orderby: i: 1.0 nreturned:101 390ms db.c.find( type:500 ).sort( i:1 ).explain()cursor : BtreeCursor i_1,indexBounds : i : $minElement : 1 , i : $maxElement : 1 ,nscanned : 1000000,

47、nscannedObjects : 1000000,n : 1000,millis : 5388,. db.c.ensureIndex( type:1,i:1 ); for( i = 0; i 1000000; +i ) db.c.save( i:i,type:i%1000 ); Were storing our data in MongoDBOur applications need to run ad-hoc queries for grouping, summarizations, reporting, etc.We must have a way to reshape data eas

48、ily to support these access patternsYou can use Aggregation Framework for this!A Series of Document TransformationsExecuted in stagesOriginal input is a collectionOutput as a cursor or a collectionRich Library of FunctionsFilter, compute, group, and summarize dataOutput of one stage sent to input of

49、 nextOperations executed in sequential order$matchFilter documents$project$group$unwindExpand documents$sort$limit / $skipPaginate documents$redactRestrict documents$geoNearProximity sort documents$let, $mapDefine variables_id : ObjectId(54523d2d25784427c6fabce1),From : ,To : mongodb-,Date : ISODate

50、(2012-08-15T22:32:34Z),body : text/plain : ”Hello Munich, nice to see yalll!,Subject : ”Live From MongoDB Worldsubject: Hello There,words: 218,from: subject: I love Hofbrauhaus,words: 90,from: subject: MongoDB Rules!,words: 100,from: $match: from: subject: MongoDB Rules!,words: 100,from: subject: He