故事可视化技术在三维场景构建中的应用研究

项目名称： 故事可视化技术在三维场景构建中的应用研究

项目编号： No.60873189

项目类型： 面上项目

立项/批准年度： 2009

项目学科： 无线电电子学、电信技术

项目作者： 曾新

作者单位： 中南大学

项目金额： 20万元

中文摘要： 本研究介绍了一种通过运用文字可视化技术使非专业人员实现建立一个三维虚拟环境途径，其基本思想是如何使计算机能理解语言的描述并从中提取相关的视觉信息而自动生成相应的三维虚拟场景。本研究通过总结和分析当前国外文字可视化领域的研究动向和人机界面的研究现状，明确和界定本课题的具体研究问题的范围。提出拟用基于儿童故事书的文字所包含的视觉信息作为虚拟场景视觉再现的首要的资源；在明确了可视化系统所涉及的相关理论与技术的基础上，完成了3DSV系统的原始构架和模块职能设计；在研究与扩展视觉感知理论、语言学空间认知相关理论的基础上，结合计算机图形技术，提出面向语义的视觉参数化定义方法。通过对包含视觉信息的特定文字进行定性和定量分析，根据文字-概念-视觉的直接联想模式建立一个新的、可拓展性的数据表现模型来联结语义和视觉形态，从而使概念上的再现成为可能并简化了整个可视化过程。同时在系统中整合相关的语言推理技术和现实世界的规则解决关于刚性物体的空间关系，提出在一阶逻辑基础上使用假言推理整合决定结构、及物体几何体的限制对输入语言所涉及的物体关系进行空间推理；创建了实时交互性的语言命令界面，实现了系统功能性的突破。

中文关键词： 人机界面;自然语言处理;虚拟场景构建;语义视觉表现;文字可视化

英文摘要： This research introduces an approach that enables non-professionals to create an interactive 3D virtual scene through manipulating visual features of the aspects of environment by language input. Based on a comprehensive study of current research on text to visualization and HCI, the methodology adopted was to develop a prototype system called 3DVS that takes a simplified story-based natural language as premier input source to produce reliable interpretations for existing NLP techniques. The main challenges of this work are to encapsulate related theory such as visual perception and language spatial cognitive, incorporate natural language understanding and computer graphic technologies to generate appropriate graphical output. An original semantic-oriented formalism was proposed to convert various semantic elements associated information into parameterized data, and this direct association word-concept-visual is used for knowledge representations. An extendable intermediate visual semantic representation was developed to provide a link between meaning of language and graphic visualization, the use of graphic constraints makes the conceptual visualization possible and simplifies the entire process. In order to enhance our system to generate the visual scenes based on relatively limited descriptions. The integration of language inference technology is accomplished by implementing real world knowledge and modus ponens based inference method to deduce spatial relations of the virtual environment from the semantic representation. Our approach also provides solutions for data accessibility, consistency and avoids redundancy. After a series of prototype system evaluation, the overall system was functionally sound and shown that the natural language and graphical interfaces developed have complementary strengths that contribute significantly to the value of the integrated system. Furthermore, an object-oriented character animation methodology was proposed and the application of 3D graphic engine was attempted to improve the effect of graphic representation. It is expected that further development of this prototype system could offer a flexible and easy-to-use aid to non-specialists and storytellers for generating their interactive 3D virtual environments.

英文关键词： Human-computer interface; Natural language processing; Virtual scene generation; Semantic representation; Text to visualization