机器视觉三维测量技术-ppt课件.ppt

1、Institute of Opto-mechatronics Engineering, Hefei University of Technology, Hefei, Anhui, 230009, China卢卢 荣荣 胜胜Tel: , 2905872 Email: 机器视觉三维测量技术机器视觉三维测量技术3D Machine Vision Measurement TechniquesInstitute of Opto-mechatronics Engineering, Hefei University of Technology, Hefei, Anhui, 230009, China大 纲1

2、. 机器视觉的概念2. 摄像机模型3. 三维视觉传感器的类型4. 多视觉传感器测量系统的组成5. 摄像机与视觉传感器的标定6. 测量系统的全局标定7. 应用举例Institute of Opto-mechatronics Engineering, Hefei University of Technology, Hefei, Anhui, 230009, China机器视觉技术的概念机器视觉技术的概念测量测量检测检测识别识别 引导引导光源传感器光学镜组传感单元信号处理单元控制单元 采用光学成像技术（通常使用摄像机）获取被测目标的图像，再经过快速图像处理与图形识别算法，从摄取图像中获取目标的尺寸、

3、位置、方向、光谱特征、结构以及缺陷等信息，从而可以执行产品检验、分类与分组，装配线上的机械手运动引导、零部件的识别与定位，生产过程中质量监控与过程控制反馈等等任务。Institute of Opto-mechatronics Engineering, Hefei University of Technology, Hefei, Anhui, 230009, China应用领域举例应用领域举例 1. IC及一般电子业：PCB、BGA、LCD屏幕、被动组件形状脚位及定位、生产插件、晶元(Wafer)镜面研磨、生产组装、被动组件辨识2. MEMS 组件：微小器件的尺寸、形貌、表面缺陷的检测，组件动态

4、特性的测量等。3. 机械工具/自动化机械：零件尺寸、外形、瑕疵检测、零件分类比对、装配定位、加工定位、熔焊检测4. 电机工业：控制器红外线热像仪检测、电线瑕疵、裂缝检测、缆线配置检测5. 金属钢铁业：钢板尺寸检测、表面瑕疵检测、铸件瑕疵检测、材料金像检测6. 橡胶/塑料制品：保特瓶口尺寸检测、制品颜色分类检测7. 食品加工/包装业：瓶内液位高度、异物或灰尘检测、包装印刷辨识、打印字形及零件编号检测与识别8. 纺织皮革工业：表面针织纹路检测、色差检测、皮革表面特性检测9. 汽车工业：陶轴裁切定位、零件涂黄油检测、白车身检验10. 家电/办公：产品外壳印刷检测、1维/2维条形码辨识11. 保全/监

5、视系统：人像特征辨识、指纹辨识等Institute of Opto-mechatronics Engineering, Hefei University of Technology, Hefei, Anhui, 230009, China机器视觉在精密测量中的应用照明光源 图像传感器成像内部三维表面表面轮廓表面轮廓粗糙度粗糙度缺陷缺陷纹理纹理内部结构内部结构应力应力变形变形缺陷缺陷被测场景方位方位三维形貌三维形貌尺寸尺寸坐标坐标Institute of Opto-mechatronics Engineering, Hefei University of Technology, Hefei, A

6、nhui, 230009, China二、摄像机的建模1.影射几何与坐标变换2.摄像机建模3.镜头畸变4.摄像机内部参数的修正5.摄像机的精确模型6.摄像机的简化模型Institute of Opto-mechatronics Engineering, Hefei University of Technology, Hefei, Anhui, 230009, China欧式几何与影射几何的概念 xyzxyz不变量不变量: Distance and angle不变量不变量: Cross ratiobacdbdadbcacd , c;b ,aRInstitute of Opto-mechatron

7、ics Engineering, Hefei University of Technology, Hefei, Anhui, 230009, China齐次矩阵fzYyXx101000000001zyxffYXzyxffzYX00001zyfYzxfXInstitute of Opto-mechatronics Engineering, Hefei University of Technology, Hefei, Anhui, 230009, China坐标变换xyz110001333231232221131211wwwzyxzyxtrrrtrrrtrrrzyxwxwywzzyxwwwtttz

8、yxrrrrrrrrrzyx333231232221131211Institute of Opto-mechatronics Engineering, Hefei University of Technology, Hefei, Anhui, 230009, China眼睛的光学模型眼睛的光学模型像平面物体透镜fImage planeobjectlensInstitute of Opto-mechatronics Engineering, Hefei University of Technology, Hefei, Anhui, 230009, China摄像机的光学投影变换摄像机的光学投影变

9、换Image planeobjectlensfzYyXxxyXYzImage planeFocus planefz , y, xInstitute of Opto-mechatronics Engineering, Hefei University of Technology, Hefei, Anhui, 230009, China摄像机的光学投影变换摄像机的光学投影变换 （续）（续）fzYyXxfzYyXxxyXYzImage planefY,Xz,y,xxXYzImage planeFocus planefz ,y, xy101000000001zyxffYXInstitute of Op

10、to-mechatronics Engineering, Hefei University of Technology, Hefei, Anhui, 230009, China像平面与帧存体的对应关系110000)(10101YXvdudsvuyxxxyXYzImage planefY,Xz,y,xuvO00,vuv ,uComputer framexdyd)( )(00uudYuudsXyxxInstitute of Opto-mechatronics Engineering, Hefei University of Technology, Hefei, Anhui, 230009, Chi

11、na帧存体与摄像机坐标的关系1010000000010000)(10101zyxffvdudsvuyxxxyXYzImage planefY,Xz,y,xuvO00,vuv ,uComputer frameInstitute of Opto-mechatronics Engineering, Hefei University of Technology, Hefei, Anhui, 230009, China2.4 摄像机建模xyXYzImage planefY,Xz,y,xuvO00,vuv ,uComputer framewywxwz110001333231232221131211wwwz

12、yxzyxtrrrtrrrtrrrzyx11000010000000010000)(13332312322211312110101wwwzyxyxxzyxtrrrtrrrtrrrffvdudsvuInstitute of Opto-mechatronics Engineering, Hefei University of Technology, Hefei, Anhui, 230009, China摄像机建模（续）110000)(13332312322211312110101wwwzyxyxxzyxtrrrtrrrtrrrvfdufdsvuMtRAm| 未知参数：00,vusddfxyx(1)

13、 Intrinsic parameters:(2) Extrinsic parameters333231232221131211r ,r ,r ,r ,r ,r ,r ,r ,rOrientation: Translation:zyxt ,t ,t69, independent 33Institute of Opto-mechatronics Engineering, Hefei University of Technology, Hefei, Anhui, 230009, China镜头畸变的概念镜头畸变的概念xyXYzImage planefddY,Xz,y,xRadial distort

14、ionTangential distortionDecentering distortionThin prism distortionInstitute of Opto-mechatronics Engineering, Hefei University of Technology, Hefei, Anhui, 230009, China径向畸变5231rkrkr22ddYXrsinrYcosrXdd 22222212222221ddddddyrddddddxrYXYkYXYkYXXkYXXkInstitute of Opto-mechatronics Engineering, Hefei U

15、niversity of Technology, Hefei, Anhui, 230009, China偏心畸变0012011cossinjpjp偏心畸变既含有径向畸变又有切向畸变04221042213cosrjrjsinrjrjtdrdtdrdydxdcossinsincos最大切向畸变与 x 轴的夹角, 设在直角坐标系下22212221223ddddydddddxdYXpYXpYXpYXpInstitute of Opto-mechatronics Engineering, Hefei University of Technology, Hefei, Anhui, 230009, Chin

16、a薄棱镜畸变1422114221cosririsinriritprp11211cosissinis薄棱镜畸变是由镜头与摄像机面有小的倾角造成的1最大切向畸变与 x 轴的夹角, 设221221ddypddxpYXsYXsInstitute of Opto-mechatronics Engineering, Hefei University of Technology, Hefei, Anhui, 230009, China镜头总畸变镜头总畸变 ypydyrdxpxdxrdYYXX,s ,p,p,k ,k ,Y ,XYY,s ,p,p,k ,k ,Y ,XXXddydddxd12121121211

17、1ydxdYYXX11)()(2)1 ()(2)3 ()1 (2222221422122122214221dddddddyddddddddxdYXsYXpYXprkrkYYXsYXpYXprkrkX若考虑分项的前两项化简Institute of Opto-mechatronics Engineering, Hefei University of Technology, Hefei, Anhui, 230009, China摄像机内部参数的修正101000000001)1 ()1 (zyxffYXydxd1010000)1/(0000)1/(1zyxffYXyxdd1001/001/)(A010

18、1vfdufdsyyxxx,1212100sppkkvusddfxyx Intrinsic parameters7Institute of Opto-mechatronics Engineering, Hefei University of Technology, Hefei, Anhui, 230009, China摄像机的精确模型计算机屏幕uvO00,vuv ,uwywxwzm11001/001/)(13332312322211312110101wwwzyxyyxxxzyxtrrrtrrrtrrrvfdufdsvuxyXYzImage planefddY,Xz,y,xInstitute o

19、f Opto-mechatronics Engineering, Hefei University of Technology, Hefei, Anhui, 230009, China摄像机的精确模型 （续）)1 ()1 ()()(98765498732100ydzwwwywwwxdzwwwxwwwydxxdYtzryrxrtzryrxrfXtzryrxrtzryrxrfvvdYuudsX计算机屏幕uvO00,vuv ,uwywxwzmdydddxXprkXYXprk12121212/3一般情况下，取径向与切向畸变的的第一项就可以达到满意的效果xyXYzImage planefddY,Xz,y

20、,xInstitute of Opto-mechatronics Engineering, Hefei University of Technology, Hefei, Anhui, 230009, China6 摄像机的简化模型1000A00vfusfvuMtRAm| 1vum1wwwzyxM333231232221131211rrrrrrrrrRzyxttttxyXYz像平面fY,Xz,y,x计算机屏幕uvO00,vuv ,uwywxwzMmInstitute of Opto-mechatronics Engineering, Hefei University of Technology,

21、 Hefei, Anhui, 230009, China三、视觉传感器的类型1.主动与被动2.单目与双目3.点、线、面结构光4.编码结构光Institute of Opto-mechatronics Engineering, Hefei University of Technology, Hefei, Anhui, 230009, China三角法测量原理激光器镜头CCD 镜头xCCD 激光器镜头镜头Institute of Opto-mechatronics Engineering, Hefei University of Technology, Hefei, Anhui, 230009,

22、China视觉传感器的坐标系统zwxwywTyTxTzxw0yw0 xsyszsosPointOXYOzxyLaserImage planeOrigin Sensor coordinate system (SCS)Global Coordinate system (GCS)Camera coordinate system (CCS)Institute of Opto-mechatronics Engineering, Hefei University of Technology, Hefei, Anhui, 230009, China单一摄像机二维传感器单一摄像机二维传感器Image plan

23、esssz ,y,xPscyyscxxscooY ,XPXYfMeasured plane(CCS)SCSsczzInstitute of Opto-mechatronics Engineering, Hefei University of Technology, Hefei, Anhui, 230009, China点结构光视觉传感器syszsxcocxcyXYZpqPQOCCSSCSLaserInstitute of Opto-mechatronics Engineering, Hefei University of Technology, Hefei, Anhui, 230009, Ch

24、ina线结构光视觉传感器Laser Light planepPXYcxcyImage planesxsyszCCSSCSczInstitute of Opto-mechatronics Engineering, Hefei University of Technology, Hefei, Anhui, 230009, China条纹结构光视觉传感器条纹结构光视觉传感器The k-th light stripe planeImage planeProjector patternkxzyxzywFvuCFoknkF3Institute of Opto-mechatronics Engineerin

25、g, Hefei University of Technology, Hefei, Anhui, 230009, ChinaXYcxcyImage planesyszCCSSCScz00110010011001sxProjector条纹编码三维视觉传感器Institute of Opto-mechatronics Engineering, Hefei University of Technology, Hefei, Anhui, 230009, China彩色编码视觉传感器彩色编码视觉传感器cXcYcxcyspyypXpYczspxxprojectorcameraCCSSCSobjectspz

26、zInstitute of Opto-mechatronics Engineering, Hefei University of Technology, Hefei, Anhui, 230009, China被动双目视觉传感器syy11X2Ysxx1lamp2X2Y2x2y1zcamera 2CCS 2(CCS1) SCSobjectcamera 12zInstitute of Opto-mechatronics Engineering, Hefei University of Technology, Hefei, Anhui, 230009, China编码照明双目视觉传感器 syy11X2

27、Ysxx1projector2X2Y2x2yszz1camera 2CCS 2(CCS 1) SCSobjectcamera 12zInstitute of Opto-mechatronics Engineering, Hefei University of Technology, Hefei, Anhui, 230009, China多视觉传感器测量系统的组成Institute of Opto-mechatronics Engineering, Hefei University of Technology, Hefei, Anhui, 230009, ChinaCAN总线网络控制视觉测量系统

28、总线网络控制视觉测量系统图像采集卡图像采集卡计算机计算机多路视频切换开关多路视频切换开关CAN 接口卡接口卡CAN 网络控制总线网络控制总线传感器控制单元传感器控制单元传感器传感器视频信号数据总线视频信号数据总线 Institute of Opto-mechatronics Engineering, Hefei University of Technology, Hefei, Anhui, 230009, China1394总线并行图像处理系统总线并行图像处理系统传感器控制单元传感器控制单元传感器传感器计算机计算机接口卡接口卡数据与控制总线数据与控制总线 Institute of Opto-m

29、echatronics Engineering, Hefei University of Technology, Hefei, Anhui, 230009, China基于工控PC机的多计算机分布式并行图像处理架构PC控制器运动驱动机构可编程光源可编程光源多PC并行处理器HUB生产与管理系统主机办公电脑远端大型资料库视觉系统主控计算机PC三轴控制器摄像机Institute of Opto-mechatronics Engineering, Hefei University of Technology, Hefei, Anhui, 230009, China常用总线的比较 1.RS232 小于1

30、5m传输速率25kb/s2.RS485 最大传输速度10Mb/s3.CAN 小于40m传输速率1Mb/s4.USB 最高速率480Mb/s，可挂接127个外设5.CameraLink 最高4.8Gbps，最远可传输10米。6.Gigabit ethernet 带宽可达到1000 Mbps， 最远可传输100米。7. Firewire，最远可传输100米，带宽3.2GInstitute of Opto-mechatronics Engineering, Hefei University of Technology, Hefei, Anhui, 230009, China双总线架构总线接口PC多路

31、视频切换器视觉传感器1视觉传感器N视觉传感器2控制总线视频图像图像采集处理平台数据总线Institute of Opto-mechatronics Engineering, Hefei University of Technology, Hefei, Anhui, 230009, China单总线架构单总线架构计算机总线接口视觉传感器 1总线总线视觉传感器 2视觉传感器 N 传感器控制、图像采集与并行处理模块 传感器控制、图像采集与并行处理模块 传感器控制、图像采集与并行处理模块Institute of Opto-mechatronics Engineering, Hefei Universi

32、ty of Technology, Hefei, Anhui, 230009, China摄像机内部参数的标定Institute of Opto-mechatronics Engineering, Hefei University of Technology, Hefei, Anhui, 230009, China1 摄像机摄标定靶标摄像机摄标定靶标Institute of Opto-mechatronics Engineering, Hefei University of Technology, Hefei, Anhui, 230009, China标定靶标的控制点类型标定靶标的控制点类型几

33、何中心与投影中心的偏移Institute of Opto-mechatronics Engineering, Hefei University of Technology, Hefei, Anhui, 230009, China2 直接线性变换法(DLT)1 用一个齐次矩阵建立场景与控制点的对应关系11343332312423222114131211wiwiwiiiizyxaaaaaaaaaaaavuw2 从上式中解出DLT矩阵参数3411aa0La nnwnnwnnwnwnwnwnnnwnnwnnwnwnwnwniiwiiwiiwiwiwiwiiiwiiwiiwiwiwiwiwwwwwwww

34、wwwwvvzvyvxzyxuuzuyuxzyxvvzvyvxzyxuuzuyuxzyxvvzvyvxzyxuuzuyuxzyxLaaaaaaaaa1000000001100000000110000000010,11111111111111111111333231232221131211aInstitute of Opto-mechatronics Engineering, Hefei University of Technology, Hefei, Anhui, 230009, China直接线性变换法(DLT)在最小二乘意义下可求出参数 ，为了避免解很小的 ，必须使用合适的正则化方法，一种

35、方法是令 然后用伪逆法求解，但当 ， 这种正则化可能产生奇异性，为了克服这个缺点，可以令3411aa3411aa034a134a1234232231aaa3 矩阵参数分解：RQ 方法FMTBVA11100100100vuV10001011221bbbbB1000000ffF矩阵M和T分别表示靶标世界坐标系相对于摄像机坐标系的旋转矢量与平移矢量（b1 b2) 为仿射失真系数Institute of Opto-mechatronics Engineering, Hefei University of Technology, Hefei, Anhui, 230009, China3 RAC 两步法径

36、向准直约束：1）径向畸变不影响矢量 的方向2）有效焦距不影响矢量 的方向3）光轴沿Z轴平移不影响矢量 的方向uOPuOPuOP该方法第一步采用最小二乘方法求解超定方程组求出外部参数，第二步求解内部参数 , 91rrxyx, s ,tt1k , f , zInstitute of Opto-mechatronics Engineering, Hefei University of Technology, Hefei, Anhui, 230009, China 计算：0PPOP0d0yxYXddyxYXddywwwxwwwtzryrxrytzryrxrx654321)()(00vvdYYuudXs

37、XyddxdxddndidyyyyxxyxyxyxdnwndnwndnwndndnwndnwndnwndiwidiwidiwididiwidiwidwidwdwdwddwdwdwXXXtrtrtrttstrstrstrsYzYyYxYYzYyYxYzYyYxYYzYyYxYzYyYxYYzYyYx165432111111111111111/, 91rrxyx, s ,tt（1）（2）yddwdwdwxddwdwdwTXrYzrYyrYxtYrYzrYyrYx654321（3）令把（2）代入（1）Institute of Opto-mechatronics Engineering, Hefei

38、University of Technology, Hefei, Anhui, 230009, China传感器坐标系与摄像机外部参数的标定Laser Light planepPXYcxcyImage planesxsyszCCSSCSczuizwwwywwwuizwwwxwwwydxxdYtzryrxrtzryrxrfXtzryrxrtzryrxrfuudYuudsX98765498732100)()()1 ()1 ()()(00yduixduiiydixxdiYYXXvvdYuudsX000111986532976431875421292623282522272421rrrrrrrrrrr

39、rrrrrrrrrrrrrrrrInstitute of Opto-mechatronics Engineering, Hefei University of Technology, Hefei, Anhui, 230009, China 测量系统的全局标定Institute of Opto-mechatronics Engineering, Hefei University of Technology, Hefei, Anhui, 230009, ChinaTarget1 标定的基本原理标定的基本原理TCSTheodolitesGCSMeasured object SCSVisual sen

40、sorInstitute of Opto-mechatronics Engineering, Hefei University of Technology, Hefei, Anhui, 230009, ChinaTarget标定的基本原理标定的基本原理TCSLaser TrackerGCSMeasured object SCSVisual sensorInstitute of Opto-mechatronics Engineering, Hefei University of Technology, Hefei, Anhui, 230009, China经纬仪的标定经纬仪的标定abRulerT

41、CSABSensor 2Sensor 1Sensor 4Sensor 3Sensor 6Sensor 5SCS (WCS)targetlight planeGCSInstitute of Opto-mechatronics Engineering, Hefei University of Technology, Hefei, Anhui, 230009, China直接变换标定法直接变换标定法 sgssggTPRP987654321rrrrrrrrrsgR321tttsgTSCSGCSPtheodolitestargetgPsPsgsgsgTRH PSCSCSCLaserCameraInsti

42、tute of Opto-mechatronics Engineering, Hefei University of Technology, Hefei, Anhui, 230009, China间接变换标定法间接变换标定法ccwwPHPwwggPHPssccPHPsccwwgsgHHHHssccwwggPHHHPTzyx1PPSCSCCSLaserCamerascHCCS = SCSGCSTheodolitesTargetWCSCamerasgHcwHwgHInstitute of Opto-mechatronics Engineering, Hefei University of Tech

43、nology, Hefei, Anhui, 230009, China应用举例Institute of Opto-mechatronics Engineering, Hefei University of Technology, Hefei, Anhui, 230009, China钢管直线度视觉在线检测技术钢管直线度视觉在线检测技术Move left or rightPositioning switchGuidanceOptoelectronic detectorFluid driver Five laser visual sensorsInstitute of Opto-mechatron

44、ics Engineering, Hefei University of Technology, Hefei, Anhui, 230009, China机器视觉汽车轮胎安装定位技术机器视觉汽车轮胎安装定位技术 Thrust AngleGeometric Center LineFront Toe (+)Rear Toe (-)F r o n t WheelsRear WheelsThrust LineHLR Steering WheelSteering Wheel AngleCamber Angle (+)Vertical Gravity LineSteering Wheel SpokeInstitute of Opto-mechatronics Engineering, Hefei University of Technology, Hefei, Anhui, 230009, China自由曲面三维测量自由曲面三维测量SCScXcYcxcyspyypXpYczspxx闪光投影仪数码相机CCS物体spzzInstitute of Opto-mechatronics Engineering, Hefei University of Technology, Hefei, Anhui, 230009, China三维面形重建