以藝術風格轉移之技術將視訊融入繪畫情境

Video Cloning for Paintings via Artistic Style Transfer

杜寧(Ning Tu)

指導教授 ： 劉興民

國立中正大學/工學院/資訊工程學系/碩士(2016年)

摘要

在過去，視覺藝術一詞往往代表的只是屬於靜態表示的繪畫、攝影、雕塑的美術作品。近年來，我們可以看到不管在博物館、美術館、甚至是許多藝術的展覽中，都有展示出動態風格的作品供遊客欣賞。其中最著名的莫過於「會動的清明上河圖」，但是製作時間共耗時兩年，必須先設計好畫中每個角色的動作，經由動畫師製成影片後，還需要利用大量投影機來投影以達到無縫拼接的效果。
在此研究中，我們對於產生藝術畫的動畫提出了一種方法，只需要利用許多網路上現有的影片資料庫，並要求使用者做一些簡單的輔助操作，即可達到動畫合成的效果。首先，我們的系統會讓使用者在影片的第一個影像中選取一個相同種類的物件，然後我們採用random forest(隨機森林)作為機器學習的演算法，幫助我們學習並取得使用者要加入到繪畫中的影片內的物件。接著我們利用風格轉換的技術，讓影片中的影像可以跟繪畫裡的風格一致。最後按照無縫合成的方法，把擷取出來的物件融入到繪畫中，以達到無縫合成的結果。
我們的方法可以讓不同使用者依照自己的喜好來把自己想要的動畫合成到繪畫中，不僅可以維持原作者的繪畫風格，甚至可以產生與原圖不一樣的意境來供人欣賞。

關鍵字

非真實感繪製 ； 風格轉換 ； 動畫合成 ； 視覺藝術

並列摘要

In the past, visual arts usually represented the static art like paintings, photography and sculptures. In recent years, many museums, artwork galleries, and even art exhibitions demonstrated dynamic artworks for visitors to relish. The most famous dynamic artwork is “The moving painting of Along the River During the Qingming Festival”. Nevertheless, it took two years to complete this work. They had to plan each action for every character at first, then drew each video frame by animators. Finally, it could achieve seamless stitching by using lots of projectors to render scene on the screen.
In our research, we propose a method for generating animated paintings. It only needs millions of videos on a network of existing databases and requires users to perform some simple auxiliary operations to achieve the effect of animation synthesis. First, our system lets users select an object with the same class from the first video frame. We then employ random forests as learning algorithm to retrieve from a video the object which users want to insert into an artwork. Second, we utilize style transferring, which enables the video frames to be consistent with the style of painting. At last, we use the seamless image cloning algorithm to yield seamless synthesizing result.
Our approach allows different users to synthesize animating paintings up to their own preferences. The resulting work not only maintains the original author's painting style, but also generates a variety of artistic conception for people to enjoy.

並列關鍵字

style transfer ； animation synthesis ； visual arts ； non-photorealistic rendering

參考文獻

[3] M. Okabe, K. Anjyo, T. Igarashi, and H. P. Seidel, “Animating pictures of fluid using video examples,” in Computer Graphics Forum, vol. 28, no. 2, pp. 677–686, 2009.

[4] M. Okabe, K. Anjyor, and R. Onai, “Creating fluid animation from a single image using video database,” in Computer Graphics Forum, vol. 30, no. 7, pp. 1973-1982, 2011.

[5] S. Bansal, S. Chaudhury, and B. Lall, “Dynamic texture synthesis for video compression,” in the IEEE National Conference on Communications, pp. 1-5, 2013.

[6] M. Sarim, A. Hilton, and J. Y. Guillemaut, “Temporal trimap propagation for video matting using inferential statistics,” in Proceedings of the 18th IEEE International Conference on Image Processing, pp. 1745-1748, 2011.

[12] M. Ashikhmin, “Fast texture transfer,” in IEEE Computer Graphics and Applications, vol. 23, no. 4, pp. 38-43, 2003.