Texture-Synthesis-by-Non-parametric-Sampling：通过非参数化采样的纹理合成-课件.ppt

1、Alexei Efros and Thomas LeungUC Berkeley Given a finite sample of some texture,the goal is to synthesize other samples from that same texture.The sample needs to be large enoughTrue(infinite)textureSYNTHESISgenerated imageinput image Texture analysis:how to capture the essence of texture?Need to mod

2、el the whole spectrum:from repeated to stochastic texture This problem is at intersection of vision,graphics,statistics,and image compressionrepeatedstochasticBoth?multi-scale filter response histogram matching Heeger and Bergen,95 sampling from conditional distribution over multiple scales DeBonet,

3、97 filter histograms with Gibbs sampling Zhu et al,98 matching 1st and 2nd order properties of wavelet coefficients Simoncelli and Portilla,98 N-gram language model Shannon,48 clustering pixel neighbourhood densities Popat and Picard,93 Our goals:preserve local structure model wide range of real tex

4、tures ability to do constrained synthesis Our method:Texture is“grown”one pixel at a time conditional pdf of pixel given its neighbors synthesized thus far is computed directly from the sample image Shannon,48 proposed a way to generate English-looking text using N-grams:Assume a generalized Markov

5、model Use a large text to compute probability distributions of each letter given N-1 previous letters precompute or sample randomly Starting from a seed repeatedly sample this Markov chain to generate new letters One can use whole words instead of letters too:WE NEED TO EAT CAKE Results(using alt.si

6、ngles corpus):“As Ive commented before,really relating to someone involves standing next to impossible.”One morning I shot an elephant in my arms and kissed him.”I spent an interesting evening recently with a grain of salt Notice how well local structure is preserved!Now lets try this in 2D.Infinite

7、 sample imageGenerated image Assuming Markov property,what is conditional probability distribution of p,given the neighbourhood window?Instead of constructing a model,lets directly search the input image for all such neighbourhoods to produce a histogram for p To synthesize p,just pick one match at

8、randomSAMPLEpfinite sample imageGenerated imagep However,since our sample image is finite,an exact neighbourhood match might not be present So we find the best match using SSD error(weighted by a Gaussian to emphasize local structure),and take all samples within some distance from that matchSAMPLE S

9、tarting from the initial configuration,we “grow”the texture one pixel at a time The size of the neighbourhood window is a parameter that specifies how stochastic the user believes this texture to be To grow from scratch,we use a random 3x3 patch from input image as seed Growing is in“onion skin”orde

10、r Within each“layer”,pixels with most neighbors are synthesized first If no close match can be found,the pixel is not synthesized until the end Using Gaussian-weighted SSD is very important to make sure the new pixel agrees with its closest neighbors Approximates reduction to a smaller neighborhood

11、window if data is too sparseIncreasing window sizealuminum wirereptile skinfrench canvasrafia weavewoodgranitewhite breadbrick wallDeBonet,97Our approachSimple tilingSynthetic tilabletextureGrowing garbage Verbatim copying Occlusion fill-in for 3D reconstruction region-based image and video compress

12、ion a small sample of textured region is stored Texturing non-developable objects growing texture directly on surface Motion synthesis2D3DSample image Advantages:conceptually simple models a wide range of real-world textures naturally does hole-filling Disadvantages:its greedy its slow its a heuristic Not an answer to texture analysis,but hopefully some inspiration!Thanks to:Alex Berg Elizaveta Levina Jitendra Malik Yair Weiss Funding agencies NSF Graduate Fellowship Berkeley Fellowship ONR MURI California MIRCOAlexei Efros and Thomas LeungUC Berkeley