OSIsoftPI时序数据库核心技术课件.ppt

1、OSIsoft OverviewDevelopments in the PI System Family of ProductsGregg Le BlancPI System Product ManagerWhy Scalability?More data can be tracked every day Phones have GPS location data Real-time monitoring in the home Electrical usage Security systems Fire/Flood detection IT Applications Every machin

2、e on a network needs monitoring Example:700,000 homes x 10 data points=7,000,000 pointsBenefits IT Monitoring can detect Intrusions Impending hardware problems Diagnose software problems Virus traffic Example:Because WiredCity could detect and record the amount of inbound traffic from the Nimda worm

3、,they got a refund from MCI WorldcomWhere is PI Today?100,000+DataStreams 100s of simultaneous clients 300+Interfaces 6,500+Customers 1,000s of events recorded/secIssues Within Scalability A scalable system has several aspects Users served Calculations and analysis Integration issues Data capacity D

4、ata throughput Security InfrastructureUser Load&Integration Users need fast access to the data Client/Server users Stateless Users Client/Server users connect directly to PI Stateless Users Broader audience PI ICE Web services and integrationServer Manager(PI SDK)Safely exposes PI data The PI Trust

5、Table retains point by point security Connection management balances user requestsInternet Information Server(IIS)BrowserSecurity in ICEPI Web ServicePI SDKSOAP MessagesPI Enterprise ServerPI Point SecurityWindowsAuthentication(Aggregated Queries)PI Trust TableConnections in ICEElvispidemoNortonpiad

6、minAlicepialiceRickypiadminPI SystemLucypidemoICE Norton,RickyAliceElvis,LucyPI SDK Server Objects1 per unique PI user1 TCP/IP Connectionper serverPISDKSession ManagerUnderlying Web Service StructurePI ICEWeb ServicesLayerICE Web PartsICE Web PartsICE Web PartsICE Web PartsPI SDKPI ServerOtherHistor

7、iansRelationalDatabasesDCSSCADALIMSLooselyCoupledApplicationsTightlyCoupledWebServicesLayerLoosely Coupled ComputingThe idea of Web servicesAllows for computing in an environment that is:AsynchronousStatelessPlatform independentGeographically independentWill replace some“tightly coupled”computingTig

8、htly Coupled ComputingInfrastructure-level ties between systemsUses API callsUsually proprietaryNot as flexibleExamples:PI Interfaces to other information systemsOLEDB/ODBC connectorsPoint-to-Point,Middleware,and other integrationWeb Service IntegrationLoosely coupled applications built onWeb Servic

9、e Integration SpaceTraditional Tightly Coupled Integration SpaceICEKPIReportsAuditingERPQualityWeb ServicesPI BatchWeb servicesRelational DBWeb servicesPI AuditingWeb servicesERPWeb servicesPI ICE Web servicesWeb Services Integration SpaceInfrastructure OSIsoft is moving to.NET Older applications wi

10、ll migrate New applications will leverage the.NET Framework Management tools will use.NET New tools coming for SMT(Systems Management Tools)Parts of the PI System will be accessible through.NETDistributed Analysis Server Based Performance Equations Desktop Based Spreadsheets based on DataLink Distri

11、buted Calculations PI ACE Server load reduced Calculations can be reusedSecurity Concerns Isnt Microsoft a vulnerability?Over the last year 26 vulnerabilities in Apache 22 in Microsofts IIS Key issue:system maintenance What about Nimda?250,000 systems in 9 hours At least$2.4B in damages Infection st

12、arted July 13,2001 Patch was available June 18,2001(MS01-033)Security Response More accountability:Microsoft NIPC(National Infrastructure Protection Center)works with NERC and Microsoft to develop security procedures and standards Key precautions still remain Guard against social engineering System

13、maintenance Unknown vulnerabilities WiFi,wardriving Unprotected modem lines used for supportOSIsoft Security PI Point Security Trust Table Single direction PI to PI transfers Auditing A new database that records changes to PI Points Values Module changesWhats Been Covered Handling user load Calculat

14、ions Integration demand Security Infrastructure Million point PI SystemsHistory of the Historian PI 1 1983 HP-1000 1985 Vax-VMS 1988 PINet Client/Server Architecture PI 2 1992 Vax-VMS 1994 Alpha-OpenVMS PI 3 1993 Design WorkHistory of the Historian PI 3 Big 4 Unix HP-UX IBM-AIX Dec-OSF/1 Sun-Solaris

15、 Windows NT 3.51 Development Language C+History of the Historian PI 3 Key concepts of PI 2 were used Snapshot Compression Archive cache Archive navigation PINetHistory of the Historian PI 3 Key new concepts Multi-process RPC Based Inter-process communication PI SDK Historization of many different da

16、ta types Doubles Strings BLOBs Common code base for all platforms 64 bit supportPI 3 Original Release PI 3.0 Released November 1995 1,000 to 100,000+Points Up to 2000 events per secondsPI 3.0 to PI 3.3 How much can you do in 7 years?Concentrated on features Alarm ACE Totalizer PI SDK Batch Database

17、Module Database Audit NT Security COM Connectors Development Infra-structure Automated builds and testing Bug fixes(just a few)Scaling 1996 Typical system 100K points Data rates 2000 Events/Second 2002 Many systems approaching 150K points Up to 10,000 Events/SecondScaling 1995 Intel Pentium Pro Proc

18、essor 2P 200MHz 2002 Intel Xeon Processor 4P 1.6 GHzScaling The PI data rates reflect increase in processor speed.Point Count does not.Limitation is not processor speed.Limitations in PI 3.3 Memory 2 GB of virtual memory per process Archive cache Inter-process communication Compressed events from sn

19、apshot to archive Serialization of RPCs Archive sub-system handles one call at a timeMemory PI Databases are memory resident Point database Module database Snapshot PI Archive CachePI 3.3 Cache Memory Issues Cache record can be quite large.Adding a single event requires entire cache record Systems r

20、eceiving data for most points will have a large memory footprint Lots of cache activity Pushing records out of memory Reading records into memoryPI 3.4 Archive Cache Typical PI System Majority of points receive data regularly Add events New events received in order,and near current timeThe PI 3.4 Ca

21、che Addresses Memory Issues Smaller memory footprint More efficient handling of new events Same efficiency reading data Design Tradeoffs More complicated3.3 Snapshot Archive Inter-process Communication Current Mechanism Event received is processed by the compression algorithm.If passes compression e

22、vent is added to in-memory event queue Events in queue are packaged and sent via an RPC to the archive process3.3 Snapshot Buffering Archive unable to process compressed events Backups Extremely high data rates System problems Snapshot must write to“event queue”file Archive must read from the Snapsh

23、ot event queue Result 2 physical disk I/Os happen on same file3.4 Snapshot Archive Inter-process Communication Changed physical file to memory mapped file File system feature Mapped by Snapshot and Archive Snapshot writes compressed events to“file”Archive reads compressed events from“file”Synchroniz

24、ation techniques required to prevent corruption Buffering case handled by file systemLimitation 3:RPC Serialization PI 3.3 all sub-systems are single threaded Requests to sub-systems are serialized Archive example3 simultaneous calls Archive Summary Plot values Compressed valuesRPC Serialization Sol

25、ution Multiple Threads Threads are the obvious approach Unix and NT have very powerful and easy to use threading models Conceptually Simple But,easy to write bugsMulti-Threading in PI 3.4 PI 3 had some threading since the original release PI Net Manager Sub-systems use a thread for reading messages

26、PI 3.4 Sub-system Level thread model Main“house keeping”thread Read thread Message pump Pool of worker threadsMulti-Threading in PI 3.4 PI 3.4 Sub-system Level thread model Every sub-system is multi-threaded Sub-system must implement locks for optimal performance Worker thread pool is configurable R

27、untime Kill threads Suspend threads Change priority Add worker threads Delete worker threadsMulti-Threading in PI 3.4 What does this get us?The archive call from.Wont kill your system.What will 10 archive calls form.do to my system?Long time1=100Long time2=1134289piar_summary(1023453,&time1,&time2,&

28、rval,&pctgood,ARCTOTAL)PI on 64 Bit Windows One advantage:larger addressable memory 32 bit:2 GB 64 bit:16 128 GB XP-64:16 GB.Net Enterprise Server:64 GB.Net Data Center:128 GBPI on 64 Bit Windows Disadvantages:Slower Very sensitive to properly optimized code Servers only 64 bit Windows is not intend

29、ed to replace 32 bit desktops.Only makes sense to port server applications 64 bit development will be done on 32 bit machinesPI on 64 Bit Windows PI 3 is designed for 64 bit Supports Dec OSF/1 Original release of PI supported Decs 64 bit Unix and still doesHP Tru64 Unix.PI 3.4 will be 64 bit ready M

30、inor differences between Windows and Unix 64 bit 3.4 will address these minor issues 3.4 will compile cleanly Summary Point Count limited by archive cache memory 3.4 archive cache is significantly smaller and more efficient Memory mapped files for configuration data Point Database Tested million poi

31、nt system on typical 32 bit server class machine 2P 1 GHz Pentium 4/4 GB Memory Several million points possible 64 bit,perhaps 50 Million points?Summary Snapshot Archive inter-process communication Significantly faster the RPC mechanism Memory mapped file shared by both processes Backed by disk to insure no data loss RPC Serialization Eliminated by threading Process several calls simultaneouslyQuestions?