Unwind: Interactive Fish Straightening

The ScanAllFish project is a large-scale effort to scan all the world's 33,100 known species of fishes. It has already generated thousands of volumetric CT scans of fish species which are available on open access platforms such as the Open Science Framework. To achieve a scanning rate required for a project of this magnitude, many specimens are grouped together into a single tube and scanned all at once. The resulting data contain many fish which are often bent and twisted to fit into the scanner. Our system, Unwind, is a novel interactive visualization and processing tool which extracts, unbends, and untwists volumetric images of fish with minimal user interaction. Our approach enables scientists to interactively unwarp these volumes to remove the undesired torque and bending using a piecewise-linear skeleton extracted by averaging isosurfaces of a harmonic function connecting the head and tail of each fish. The result is a volumetric dataset of a individual, straight fish in a canonical pose defined by the marine biologist expert user. We have developed Unwind in collaboration with a team of marine biologists: Our system has been deployed in their labs, and is presently being used for dataset construction, biomechanical analysis, and the generation of figures for scientific publication.

[1]  Jernej Barbic,et al.  FEM simulation of 3D deformable solids: a practitioner's guide to theory, discretization and model reduction , 2012, SIGGRAPH '12.

[2]  Cláudio T. Silva,et al.  HistoryTracker: Minimizing Human Interactions in Baseball Game Annotation , 2019, CHI.

[3]  Sungkil Lee,et al.  Heterogeneous volume deformation and animation authoring with density‐aware moving least squares , 2018, Comput. Animat. Virtual Worlds.

[4]  Min Chen,et al.  Volume Deformation via Scattered Data Interpolation , 2007, VG@Eurographics.

[5]  Mengchen Liu,et al.  A survey on information visualization: recent advances and challenges , 2014, The Visual Computer.

[6]  Mariano Alcañiz Raya,et al.  Real-time deformable models for surgery simulation: a survey , 2005, Comput. Methods Programs Biomed..

[7]  Megumi Nakao,et al.  Adaptive proxy geometry for direct volume manipulation , 2010, 2010 IEEE Pacific Visualization Symposium (PacificVis).

[8]  H BarrAlan Global and local deformations of solid primitives , 1984 .

[9]  Jaegul Choo,et al.  AILA: Attentive Interactive Labeling Assistant for Document Classification through Attention-Based Deep Neural Networks , 2019, CHI.

[10]  Deborah Silver,et al.  Illustrative Deformation for Data Exploration , 2007, IEEE Transactions on Visualization and Computer Graphics.

[11]  Julie Winchester,et al.  The Walking Dead: Status Report, Data Workflow and Best Practices of the oVert Thematic Collections Network , 2018, Biodiversity Information Science and Standards.

[12]  Marco Attene,et al.  Explicit cylindrical maps for general tubular shapes , 2017, Comput. Aided Des..

[13]  Alec Jacobson,et al.  Skinning: real-time shape deformation , 2014, SIGGRAPH ASIA Courses.

[14]  Adam P. Summers,et al.  The evolution of underwater flight: The redistribution of pectoral fin rays, in manta rays and their relatives (Myliobatidae) , 2018, Journal of morphology.

[15]  Daniele Panozzo,et al.  Half-Space Power Diagrams and Discrete Surface Offsets , 2020, IEEE Transactions on Visualization and Computer Graphics.

[16]  David C. Blackburn,et al.  The earliest equatorial record of frogs from the Late Triassic of Arizona , 2019, Biology Letters.

[17]  Cláudio T. Silva,et al.  Bijective maps from simplicial foliations , 2016, ACM Trans. Graph..

[18]  K. Hormann,et al.  MIPS: An Efficient Global Parametrization Method , 2000 .

[19]  Scott Schaefer,et al.  Isometry‐Aware Preconditioning for Mesh Parameterization , 2017, Comput. Graph. Forum.

[20]  Teseo Schneider,et al.  Smooth bijective maps between arbitrary planar polygons , 2015, Comput. Aided Geom. Des..

[21]  Sabine Coquillart,et al.  Extended free-form deformation: a sculpturing tool for 3D geometric modeling , 1990, SIGGRAPH.

[22]  Thomas W. Sederberg,et al.  Free-form deformation of solid geometric models , 1986, SIGGRAPH.

[23]  Charles Hansen,et al.  The Visualization Handbook , 2011 .

[24]  Olga Sorkine-Hornung,et al.  Locally Injective Mappings , 2013 .

[25]  Jack Snoeyink,et al.  Computing contour trees in all dimensions , 2000, SODA '00.

[26]  J. Shewchuk,et al.  Isosurface stuffing: fast tetrahedral meshes with good dihedral angles , 2007, SIGGRAPH 2007.

[27]  Reinhard Klein,et al.  An Adaptable Surface Parameterization Method , 2003, IMR.

[28]  Ronen Basri,et al.  Large-scale bounded distortion mappings , 2015, ACM Trans. Graph..

[29]  Marc Alexa,et al.  As-rigid-as-possible surface modeling , 2007, Symposium on Geometry Processing.

[30]  Yalin Wang,et al.  Volumetric Harmonic Map , 2003, Commun. Inf. Syst..

[31]  Yaron Lipman,et al.  Accelerated quadratic proxy for geometric optimization , 2016, ACM Trans. Graph..

[32]  Ryan Cross New 3D scanning campaign will reveal 20,000 animals in stunning detail , 2017 .

[33]  Scott Schaefer,et al.  Bijective parameterization with free boundaries , 2015, ACM Trans. Graph..

[34]  W SederbergThomas,et al.  Free-form deformation of solid geometric models , 1986 .

[35]  David Salesin,et al.  Interactive cutaway illustrations of complex 3D models , 2007, ACM Trans. Graph..

[36]  Andrew Nealen,et al.  Physically Based Deformable Models in Computer Graphics , 2006, Comput. Graph. Forum.

[37]  Olga Sorkine-Hornung,et al.  Scalable locally injective mappings , 2017, TOGS.

[38]  Megumi Nakao,et al.  Direct volume manipulation for visualizing intraoperative liver resection process , 2014, Comput. Methods Programs Biomed..

[39]  Yaron Lipman,et al.  Bounded distortion mapping spaces for triangular meshes , 2012, ACM Trans. Graph..

[40]  CoquillartSabine Extended free-form deformation: a sculpturing tool for 3D geometric modeling , 1990 .

[41]  Mark W. Jones,et al.  Volume Wires: A Framework for Empirical Nonlinear Deformation of Volumetric Datasets , 2006, J. WSCG.

[42]  Pierre Alliez,et al.  Polygon Mesh Processing , 2010 .

[43]  Olga Sorkine-Hornung,et al.  Geometric optimization via composite majorization , 2017, ACM Trans. Graph..

[44]  Cecilia R. Aragon,et al.  Traffigram: distortion for clarification via isochronal cartography , 2014, CHI.

[45]  Zhigang Deng,et al.  Structured Volume Decomposition via Generalized Sweeping , 2016, IEEE Transactions on Visualization and Computer Graphics.

[46]  Adam P. Summers,et al.  Specialized specialists and the narrow niche fallacy: a tale of scale-feeding fishes , 2018, Royal Society Open Science.

[47]  Olga Sorkine-Hornung,et al.  Autocuts: simultaneous distortion and cut optimization for UV mapping , 2017, ACM Trans. Graph..

[48]  Alexander Bock,et al.  TopoAngler: Interactive Topology-Based Extraction of Fishes , 2018, IEEE Transactions on Visualization and Computer Graphics.

[49]  Andrew S. Winter,et al.  Spatial transfer functions: a unified approach to specifying deformation in volume modeling and animation , 2003, VG.

[50]  Yang Liu,et al.  Computing inversion-free mappings by simplex assembly , 2016, ACM Trans. Graph..

[51]  PanozzoDaniele,et al.  Scalable Locally Injective Mappings , 2017 .

[52]  Yingcai Wu,et al.  A Survey of Visual Analytics Techniques and Applications: State-of-the-Art Research and Future Challenges , 2013, Journal of Computer Science and Technology.

[53]  Andrea Tagliasacchi,et al.  3D Skeletons: A State‐of‐the‐Art Report , 2016, Comput. Graph. Forum.

[54]  Timo Ropinski,et al.  Interactive Cutting Operations for Generating Anatomical Illustrations from Volumetric Data Sets , 2008, J. WSCG.

[55]  Mason N. Dean,et al.  On the jaws of lamniform sharks , 2012 .

[56]  Hong Qin,et al.  Harmonic volumetric mapping for solid modeling applications , 2007, Symposium on Solid and Physical Modeling.

[57]  Alan H. Barr,et al.  Global and local deformations of solid primitives , 1984, SIGGRAPH.

[58]  Baining Guo,et al.  Computing locally injective mappings by advanced MIPS , 2015, ACM Trans. Graph..