行人再识别的若干问题课件.pptx

1、Person Re-identification:Recent Challenges1My Research2Human Identification&Activity Understandingq BackgroundThe whole story1Detect an event2track persons across camera view3Identify who he/she is3Human Identification&Activity Understandingq BackgroundActivityPerson Re-identificationFaceRecognition

2、Person Re-identificationWhat ishedoing?Matching,TrackingCamera NetworkUnderstandingDetecting target objects(cars,pedestrian,bags etc.)5Person Re-identification6Recent Development&QuestionPose-guided,Local,Attention-based,GAN-based,a ppt:https:/ should we do?I would guess we will soon have 99%matchin

3、g rate this year or early next year on benchmarksqHave we already solved it?q7My Todays FocusTell less about performanceqAim to tell something of my understandingabout Re-IDq8Person Re-identification:Challenges9Person Re-identification:Challengesq Some Main VariationsView Lighting Occlusion Low Reso

4、lution Clothing Change101.Connection with Cross Domain?11Person Re-ID vs.Cross-ModalityView Biasq12Asymmetric Metric for Re-IDLearning universal featuretransformationLearning view-specificfeature transformation13Asymmetric Metric for Re-IDLearn different featuretransformation for differentcamera vie

5、wsPseudometricNon-negativity SymmetryTriangle InequalityCoincidence14Asymmetric Metric for Re-IDq Re-ID Reformulation by AugmentationView-specifictransformationYingcong Chen,Xiatian Zhu,Wei-Shi Zheng*,and Jian-Huang Lai.Person Re-Identificationby Camera Correlation Aware Feature Augmentation.IEEE Tr

6、ans.on Pattern Analysis andMachine Intelligence(PAMI),2017.15Asymmetric Metric for Re-IDq Re-ID Reformulation by AugmentationNot able to measure the relationshipbetween different view-specifictransformation matricesView-specifictransformationDo not constraint the discrepancybetween feature transform

7、ation acrossview:CoincidenceYingcong Chen,Xiatian Zhu,Wei-Shi Zheng*,and Jian-Huang Lai.Person Re-Identificationby Camera Correlation Aware Feature Augmentation.IEEE Trans.on Pattern Analysis andMachine Intelligence(PAMI),2017.16Asymmetric Metric for Re-IDAdaptive feature augmentationqgeneralisedcon

8、trol thediscrepancyBetweenfa and fbYingcong Chen,Xiatian Zhu,Wei-Shi Zheng*,and Jian-Huang Lai.Person Re-Identificationby Camera Correlation Aware Feature Augmentation.IEEE Trans.on Pattern Analysis andMachine Intelligence(PAMI),2017.17Asymmetric Metric for Re-IDLearning:qCamera coRrelation Aware Fe

9、ature augmenTation(CRAFT)Generalize any symmetric metric learning models to asymmetricones:e.g.MFA18Asymmetric Metric for Re-IDLearning:qCamera coRrelation Aware Feature augmenTation(CRAFT)Camera ViewDiscrepancyRegularization:ReduceCoincidenceBregman discrepancy of a projection19Asymmetric Metric fo

10、r Re-IDA frameworkqto extractdomain-genericand more viewinvariant personfeatures20Asymmetric Metric for Re-IDEvaluation:augmentation or not augmentation?qEvaluation:augmentation vs.domain adaptationqqEvaluation:whether using Camera View Discrepancy21Does the Asymmetric Metric Modelling Workfor other

11、 setting:unsupervised,semi-supervised,.22Asymmetric Metric for Re-ID:UnsupervisedUnsupervised Learningqo Clustering-based Asymmetric MEtric Learning(CAMEL)Hongxing Yu,Ancong Wu,Wei-Shi Zheng*.-Learning for Unsupervised Person Re-identification.In IEEE Conf.on ComputerVision(ICCV),2017.23Asymmetric M

12、etric for Re-ID:UnsupervisedUnsupervised Learningq24Hash Re-ID for Fast SearchFAST Re-ID on Numbers of Camerasqo Learning view-specific hash code for each cameraXiatian Zhu,Botong Wu,Dongcheng Huang,Wei-Shi Zheng*(PI)Identification.IEEE Transactions on Image Processing,2017.Fast Open-World Person Re

13、-Wei-Shi Zheng,Shaogang Gong,and Tao Xiang.Towards Open-World Person Re-Identificationby One-Shot Group-based Verification.IEEE Transactions on Pattern Analysis and MachineIntelligence(PAMI),vol.38,no.3,pp.591-606,2016.25Hash Re-ID for Fast SearchIdea of the FormulationqCross-view IdentityVerificati

14、on RegularisationCross-view IdentityCorrelation HashingView Context DiscrepancyRegularisation26Hash Re-ID for Fast SearchFAST Searchqo Comparison to other related Hashing functions27Hash Re-ID for Fast SearchFAST Searchqo When using more powerful features?282.How to match heterogeneousperson images

15、across camera views?29Person Re-ID vs.Cross-ModalityMatching between Heterogeneous Imagesq30RGB-Infrared Re-IDCross-Modality Learning:RGB-IR Re-IDqo Deep zero-paddingAncong Wu,Wei-Shi Zheng*(PI),Hongxing Yu,Shaogang Gong,Jianhuang Lai.RGB-InfraredCross-Modality Person Re-Identification.In IEEE Conf.

16、on Computer Vision(ICCV),2017.31RGB-Infrared Re-IDCross-Modality Learning:RGB-IR Re-IDq32RGB-Infrared Re-IDCross-Modality Learning:RGB-IR Re-IDq33RGB-Infrared Re-IDCross-Modality Learning:RGB-IR Re-IDqo SYSU RGB-IR Re-ID Dataset34When the input is not image?35Attribute-Image Person Re-IDMatch person

17、 images with specific attributedescriptions in surveillance environment.qZhou Yin,Wei-Shi Zheng*(PI),et al.Adversarial Attribute-Image Person Re-identification,IJCAI 201836Attribute-Image Person Re-IDIntuitively,when we hold some attribute description in mind,e.g.,qq“carrying backpack”,we generate a

18、n obscure and vagueimagination on how a backpack may look like,which we refer to asa concept.We model this generation process and match the generatedconcepts with image perceptions.37Attribute-Image Person Re-IDImage Concept Extraction loss:Our model learns a semanticallyqq!discriminative structure

19、of low-level person images.Semantic Consistency Constraint+Adversary loss:Our model!#$%generates the corresponding aligned image-analogous concept forhigh-level attribute.38Attribute-Image Person Re-IDOur model:Outperforms traditional cross modality retrieval methods(DeepCCAE,DeepCCA,2WayNet,CMCE).q

20、Outperforms classical pedestrian attribute recognition model(DeepMAR).Outperforms other variants of our model,which also generate homogenous distributions undersemantic consistency regularization for the two modalities(MMD,DeepCoral).qq39Attribute-Image Person Re-IDWrong samples40Attribute-Image Per

21、son Re-IDEffects of different generation strategies:Generation from attributes to image is better than generation fromqqimage to attributes.(A2Img vs.Img2A):Estimating the manifold of images from the training data is more reliable thanestimating that of attributesoGeneration in feature space is bett

22、er than generation in real imagespace.(A2Img vs.Real Images):Generating real pedestrian image is difficult.Generating noisy low-level imagesand then eliminating these noise to extract discriminative concepts is notnecessaryo41When dressing differently?42Depth Re-IDSomething to seeqIlluminationchange

23、ClotheschangeIn these cases,appearance cues are not reliable.43Depth Re-IDDepth descriptorsq Within-patch Covariance Between-patch Covariance Eigen-depth featureEigen-depth feature is rotation invariant.44Depth Re-IDMetricqxixjOExtracting Eigen-depth feature converts covariance matrices onRiemannian

24、 manifold to feature vectors in Euclidean space.45Depth Re-ID46Depth Re-IDTransferring Depthq()Ancong Wu,Wei-Shi Zheng*(PI),and Jian-HuangLai.Robust Depth-based Person Re-identification.IEEE Transactions on Image Processing,2017Depth Re-ID483.Low-resolutionPerson Re-identificationVaryingResolutionsC

25、amera ACamera B49Low-resolution Re-ID50Low-resolution Re-IDLow-resolution Re-IDqo JUDEA:joint multi-scale discriminant componentanalysisXiang Li,Wei-Shi Zheng*,Xiaojuan Wang,Tao Xiang,Shaogang Gong.Multi-scale(PI)Learning for Low-resolution Person Re-identification.IEEE Conf.on Computer Vision(ICCV)

26、,2015.51Low-resolution Re-IDq Super-resolution and Identity joiNt learninG(SING)Jiening Jiao,Wei-Shi Zheng*(PI),Ancong Wu,Xiatian Zhu,and Shaogang Gong.Deep Low-resolution Person Re-identification.AAAI 201852Low-resolution Re-ID53Low-resolution Re-IDResultsq54Low-resolution Re-IDResultsq55More 56Cro

27、ss-set Re-IDGalleryProbeLabelling images across camera views is costly57Cross-scenario Re-IDTransferring between setsAnqAsymmetricMulti-taskModellingXiaojuan Wang,Wei-Shi Zheng*(PI),Xiang Li,and Jianguo Zhang.Cross-scenario Transfer Person Re-identification.IEEE Transactions on Circuits and Systems

28、for Video Technology,vol.26,no.8,pp.1447-1460,2016.58Partial Re-ID59Partial Re-IDAnnotating PartialPart by Operatoror Detecting itautomaticallyLocal-to-localMatchingMatchingFusionWei-Shi Zheng,Xiang Li,Tao Xiang,Shengcai Liao,JianHuang Lai,ShaogangGong.Partial Person Re-identification.ICCV,2015.Glob

29、al-to-localMatching60Partial Re-IDExample of partial person matching61One-Shot Open-World Group-based Re-idMotivationqOpe world personr identification setting1)A large amount of non-targetimposters captured alongwith the target people on thewatch list.2)Their images will also appearin the probe set

30、and some ofthem will look visually similarto the target peopleWei-Shi Zheng,Shaogang Gong,and Tao Xiang.Towards Open-World Person Re-Identification by One-Shot Group-based Verification.IEEE Transactions on PatternAnalysis and Machine Intelligence(PAMI),vol.38,no.3,pp.591-606,2016.62One-Shot Open-Wor

31、ld Group-based Re-idKnowledge to transferqEnrich intr class variationApproximate target intra-inter class pair(magenta line and green line)Enrich inte class variationTarget specific non-target intra-inter classpair(magenta line and yellow line)Enrich group separationGroup separation intra-inter clas

32、s pair(green line and grey line)63MoreLingxiao He,et al.,CVPR 2018.S.Li et al.,CVPR 2017Weijian Deng,et al,.CVPR,2018.Longhui Wei,et al,.CVPR,2018.64SummaryConnection with Machine Learning forCross Modalities/Cross DomainsAsymmetricHash Re-IDPartial Re-IDAttribute-Metric LearningImage Re-IDUnsupervisedRe-IDRGB-InfraredRe-IDDepth Re-IDCross-scenarioRe-IDLow-resolution Re-IDOpen-world Re-IDPerson Re-identification65Thanks to my studentshttp:/ MEwszhengieee.org66