迁移学习算法研究课件.ppt
- 【下载声明】
1. 本站全部试题类文档,若标题没写含答案,则无答案;标题注明含答案的文档,主观题也可能无答案。请谨慎下单,一旦售出,不予退换。
2. 本站全部PPT文档均不含视频和音频,PPT中出现的音频或视频标识(或文字)仅表示流程,实际无音频或视频文件。请谨慎下单,一旦售出,不予退换。
3. 本页资料《迁移学习算法研究课件.ppt》由用户(三亚风情)主动上传,其收益全归该用户。163文库仅提供信息存储空间,仅对该用户上传内容的表现方式做保护处理,对上传内容本身不做任何修改或编辑。 若此文所含内容侵犯了您的版权或隐私,请立即通知163文库(点击联系客服),我们立即给予删除!
4. 请根据预览情况,自愿下载本文。本站不保证下载资源的准确性、安全性和完整性, 同时也不承担用户因使用这些下载资源对自己和他人造成任何形式的伤害或损失。
5. 本站所有资源如无特殊说明,都需要本地电脑安装OFFICE2007及以上版本和PDF阅读器,压缩文件请下载最新的WinRAR软件解压。
- 配套讲稿:
如PPT文件的首页显示word图标,表示该PPT已包含配套word讲稿。双击word图标可打开word文档。
- 特殊限制:
部分文档作品中含有的国旗、国徽等图片,仅作为作品整体效果示例展示,禁止商用。设计者仅对作品中独创性部分享有著作权。
- 关 键 词:
- 迁移 学习 算法 研究 课件
- 资源描述:
-
1、INSTITUTE OF COMPUTING TECHNOLOGYINSTITUTE OF COMPUTING TECHNOLOGYINSTITUTE OF COMPUTING TECHNOLOGY迁移学习迁移学习算法研究算法研究庄福振庄福振中国科学院计算技术研究所中国科学院计算技术研究所2019 年年 4 月月 18 日日INSTITUTE OF COMPUTING TECHNOLOGYTrainingDataOccPalm LinesDragonStarFortune?ProflongTgoodLawyershortFbadPhD StubrokenTgoodDoclongFbadClas
2、sifierUnseen Data(,long,T)good!What if2传统监督机器学习传统监督机器学习(1/2)(1/2)2022-7-28from Prof.Qiang YangINSTITUTE OF COMPUTING TECHNOLOGY传统监督机器学习传统监督机器学习(2/2)(2/2)32022-7-28l传统监督学习在实际应用中在实际应用中通常不能满足!通常不能满足!训练集测试集分类器训练集测试集分类器INSTITUTE OF COMPUTING TECHNOLOGY迁移学习迁移学习42022-7-28l实际应用学习场景HP 新闻新闻Lenovo 新闻新闻迁移迁移学习学习
3、 运用已有的知识对运用已有的知识对不同但相关领域不同但相关领域问题问题进行求解的一种新的机器学习方法进行求解的一种新的机器学习方法 放宽了传统机器学习的两个基本假设放宽了传统机器学习的两个基本假设INSTITUTE OF COMPUTING TECHNOLOGY迁移学习场景迁移学习场景(1/4)(1/4)52022-7-28l迁移学习场景无处不在迁移迁移知识知识迁移迁移知识知识图像分类图像分类HP 新闻新闻Lenovo 新闻新闻新闻网页分类新闻网页分类INSTITUTE OF COMPUTING TECHNOLOGY异构特征空间6The apple is the pomaceous fruit
4、 of the apple tree,species Malus domestica in the rose family Rosaceae.Banana is the common name for a type of fruit and also the herbaceous plants of the genus Musa which produce this commonly eaten fruit.Training:TextFuture:ImagesApplesBananas迁移学习场景迁移学习场景(2/4)(2/4)2022-7-28from Prof.Qiang YangXin
5、Jin,Fuzhen Zhuang,Sinno Jialin Pan,Changying Du,Ping Luo,Qing He:Heterogeneous Multi-task Semantic Feature Learning for Classification.CIKM 2019:1847-1850.INSTITUTE OF COMPUTING TECHNOLOGY Test Test Training TrainingClassifierClassifier72.65%DVDElectronicsElectronics84.60%ElectronicsDrop!迁移学习场景迁移学习场
6、景(3/4)(3/4)72022-7-28from Prof.Qiang YangINSTITUTE OF COMPUTING TECHNOLOGY8DVDElectronicsBookKitchenClothesVideo gameFruitHotelTeaImpractical!迁移学习场景迁移学习场景(4/4)(4/4)2022-7-28from Prof.Qiang YangINSTITUTE OF COMPUTING TECHNOLOGYOutlinepConcept Learning for Transfer Learning Concept Learning based on N
7、on-negative Matrix Tri-factorization for Transfer Learning Concept Learning based on Probabilistic Latent Semantic Analysis for Transfer LearningpTransfer Learning using Auto-encodersTransfer Learning from Multiple Sources with Autoencoder RegularizationSupervised Representation Learning:Transfer Le
8、arning with Deep Auto-encoders92022-7-28INSTITUTE OF COMPUTING TECHNOLOGYConcept Learning based on Non-negative Matrix Tri-factorization for Transfer LearningConcept Learning for Transfer Learning102022-7-28INSTITUTE OF COMPUTING TECHNOLOGYIntroduction Many traditional learning techniques work well
9、only under the assumption:Training and test data follow the same distribution Training(labeled)ClassifierTest(unlabeled)Enterprise News Classification:including the classes“Product Announcement”,“Business scandal”,“Acquisition”,Product announcement:HPs just-released LaserJet Pro P1100 printer and th
10、e LaserJet Pro M1130 and M1210 multifunction printers,price performance.Announcement for Lenovo ThinkPad ThinkCentre price$150 off Lenovo K300 desktop using coupon code.Lenovo ThinkPad ThinkCentre price$200 off Lenovo IdeaPad U450p laptop using.their performanceHP newsLenovo newsDifferent distributi
11、onFail!11Concept Learning for Transfer Learning2022-7-28INSTITUTE OF COMPUTING TECHNOLOGYMotivation(1/3)Example AnalysisProduct announcement:HPs just-released LaserJet Pro P1100 printer and the LaserJet Pro M1130 and M1210 multifunction printers,price performance.Announcement for Lenovo ThinkPad Thi
12、nkCentre price$150 off Lenovo K300 desktop using coupon code.Lenovo ThinkPad ThinkCentre price$200 off Lenovo IdeaPad U450p laptop using.their performanceHP newsLenovo newsProductword conceptLaserJet,printer,price,performance ThinkPad,ThinkCentre,price,performance RelatedProductannouncementdocument
13、class:12Share some common words:announcement,price,performance indicateConcept Learning for Transfer Learning2022-7-28INSTITUTE OF COMPUTING TECHNOLOGYMotivation(2/3)Example Analysis:HPLaserJet,printer,price,performance et al.LenovoThinkpad,Thinkcentre,price,performance et al.The words expressing th
14、e same word concept are domain-dependent 13ProductProductannouncementword conceptindicatesThe association between word concepts and document classes is domain-independent Concept Learning for Transfer Learning2022-7-28INSTITUTE OF COMPUTING TECHNOLOGYMotivation(3/3)14 Further observations:Different
15、domains may use same key words to express the same concept(denoted as identical concept)Different domains may also use different key words to express the same concept(denoted as alike concept)Different domains may also have their own distinct concepts(denoted as distinct concept)The identical and al
16、ike concepts are used as the shared concepts for knowledge transfer We try to model these three kinds of concepts simultaneously for transfer learning text classificationConcept Learning for Transfer Learning2022-7-28INSTITUTE OF COMPUTING TECHNOLOGYPreliminary Knowledge Basic formula of matrix tri-
17、factorization:where the input X is the word-document co-occurrence matrixFGS15Concept Learning for Transfer Learning2022-7-28INSTITUTE OF COMPUTING TECHNOLOGYPrevious method-MTrick in SDM 2019(1/2)lSketch map of MTrickSource domain Xs FsGsFtGtTargetdomain XtSKnowledge Transfer16Concept Learning for
18、Transfer Learning2022-7-28lConsidering the alike conceptsINSTITUTE OF COMPUTING TECHNOLOGYMTrick(2/2)lOptimization problem for MTrickG0 is the supervision informationthe association S is shared as bridge to transfer knowledge17Concept Learning for Transfer LearninglDual Transfer Learning(Long et al.
19、,SDM 2019),considering identical and alike concepts2022-7-28INSTITUTE OF COMPUTING TECHNOLOGYTriplex Transfer Learning(TriTL)(1/5)lFurther divide the word concepts into three kinds:18F1,identical concepts;F2,alike concepts;F3,distinct concepts Input:s source domain Xr(1rs)with label information,t ta
20、rget domain Xr(s+1rs+t)We propose Triplex Transfer Learning framework based on matrix tri-factorization(TriTL for short)2022-7-28Concept Learning for Transfer LearningINSTITUTE OF COMPUTING TECHNOLOGYF1,S1 and S2 are shared as the bridge for knowledge transfer across domainsThe supervision informati
21、on is integrated by Gr(1rs)in source domainsTriTL(2/5)lOptimization Problem192022-7-28Concept Learning for Transfer LearningINSTITUTE OF COMPUTING TECHNOLOGYTriTL(3/5)lWe develop an alternatively iterative algorithm to derive the solution and theoretically analyze its convergence 202022-7-28Concept
22、Learning for Transfer LearningINSTITUTE OF COMPUTING TECHNOLOGYTriTL(4/5)lClassification on target domainsWhen 1rs,Gr contains the label information,so we remain it unchanged during the iterations when xi belongs to class j,then Gr(i,j)=1,else Gr(i,j)=0After the iteration,we obtain the output Gr(s+1
23、rs+t),then we can perform classification according to Gr212022-7-28Concept Learning for Transfer LearningINSTITUTE OF COMPUTING TECHNOLOGYTriTL(5/5)lAnalysis of Algorithm ConvergenceAccording to the methodology of convergence analysis in the two works Lee et al.,NIPS01 and Ding et al.,KDD06,the foll
24、owing theorem holds.Theorem(Convergence):After each round of calculating the iterative formulas,the objective function in the optimization problem will converge monotonically.222022-7-28Concept Learning for Transfer LearningINSTITUTE OF COMPUTING TECHNOLOGY232022-7-28rec.autosrec.motorcyclesrec.base
25、ballrec.hockeysci.cryptsic.electronicssci.medsci.spacecomp.graphicscomp.sys.ibm.pc.hardwarecomp.sys.mac.hardwarecomp.windows.xtalk.politics.misctalk.politics.gunstalk.politics.mideasttalk.religion.miscrecscicomptalkData Preparation(1/3)l20NewsgroupsFour top categories,each top category contains four
展开阅读全文