Twin-to-twin transfusion syndrome treatment requires fetoscopic laser photocoagulation of placental vascular anastomoses to regulate blood flow to both fetuses. Limited field-of-view (FoV) and low visual quality during fetoscopy make it challenging to identify all vascular connections. Mosaicking can align multiple overlapping images to generate an image with increased FoV, however, existing techniques apply poorly to fetoscopy due to the low visual quality, texture paucity, and hence fail in longer sequences due to the drift accumulated over time. Deep learning techniques can facilitate in overcoming these challenges. Therefore, we present a new generalized Deep Sequential Mosaicking (DSM) framework for fetoscopic videos captured from different settings such as simulation, phantom, and real environments. DSM extends an existing deep image-based homography model to sequential data by proposing controlled data augmentation and outlier rejection methods. Unlike existing methods, DSM can handle visual variations due to specular highlights and reflection across adjacent frames, hence reducing the accumulated drift. We perform experimental validation and comparison using 5 diverse fetoscopic videos to demonstrate the robustness of our framework.
翻译:双对双输血综合症的治疗要求对胎盘血管血管血管肛门进行风化激光激光光化,以调控血液流向胎儿。 Fetoscop 期间的有限视野和低视觉质量使得辨别所有血管连接具有挑战性。 Mosaick 可将多重重叠图像与增加的FovV相匹配,但是,由于视觉质量低、纹理贫乏,现有技术对胎盘检查适用得很差,因此由于随时间的流逝导致更长时间的序列不合格。深层学习技术有助于克服这些挑战。因此,我们为从模拟、幻影和真实环境等不同环境中摄取的胎儿视频展示一个新的普遍深度深度测序(DSM)框架。DSM通过提出控制数据扩增和外部拒绝方法,将现有的深图像同系模型推广到序列数据。DSM可以处理由于相貌突出和反射跨近框而导致的视觉变异,从而减少了累积的漂移。我们用5种不同的图像进行实验性验证和比较。