Computational Vegetation Group

Computational Vegetation Group focuses on digital twins of plants, particularly on digital tree models. Building on Dr. Benes’s 25 years of experience, we research the shape, structure, and function of vegetation. We study the plant’s shape, development over time, and interaction with its environment (light, water, wind, temperature). Most of the work has been done in collaboration with our colleagues and past members, listed below.

3D Vegetation Reconstruction - Digital Twins

Together with our collaborators, we have developed algorithms for AI-based 3D tree reconstruction from photographs, point clouds, inverse procedural models of trees as L-systems, and developmental models programmed as simulation systems. The input to this problem is an existing tree represented as an image, point cloud, or geometry. The goal is to find a functional-structural plant model (digital twin) that can be used in simulating “what-if” scenarios. The figure below from inverse procedural modeling of trees shows such a scenario. The input tree on the left is analyzed, and the simulation parameters that can reproduce it are found. Having the digital twin, the tree can be regrown in a different environment (right).

See the video of the reconstruction from images:

Crops in Silico and in Silico Plants

Our paper from 2020 describes the importance of multiscale computational models for crops, an initiative called Crops in Silico.

Together with our collaborators, we have developed algorithms that allow for finding genetic traits and 3D reconstruction for phenotyping facilities. In particular, our voxel carving algorithm for sorghum has been used to identify the gene that controls its branching angle.

The figure below shows the 3D reconstruction of maize, where (a) is the side and top view of a procedurally generated 3D model of a maize plant, (b) shows a voxelized version of the original 3D model, and (c) a reconstructed 3D model generated using the voxel carving.

A recent paper discusses current challenges and future of agricultural genomes to phenomes in the USA. It is a result of a three day workshop at Agricultural Genome to Phenome Initiative.

Deep Neural Models of Vegetation

We have developed algorithms that encode tree structure and behavior as a deep neural model. In particular, it encodes tree branchlets as a latent space that can be then used to create novel trees or reconstruct existing from point clouds.

Vegetation Simulation and Modeling

The video below shows the tree’s developmental and immediate response to wind. This model captures the physics response of the tree shape to the long and short-term influence of wind.

The developmental model can show climbing plants (Honorable mention from the Best Papers Committee Eurographics 2017), and there was also a paper in 2002 on the same topic.

The tree environmental response of the tree can be estimated from its geometry. The response can then be used to simulate the tree response to a different environment interactively, an approach we called plastic trees. The reconstructed model is environmentally sensitive and responds to light or obstacles, as shown in this video.

Ecosystems and Urban Forests

Entire landscapes can be authored by combining ecosystems and erosion, as shown in this paper. An urban forest can also be considered a special case of an ecosystem, as shown in this, where we find tree location, and we generate an inverse procedural model showing where and how to plant the trees. An early paper considered the urban forest as an unbalanced ecosystem, where the disturbance is caused by pruning, removing unwanted plants, planting in order, etc. Several earlier works studied the effect of virtual agents on ecosystems. An interesting approach is to simulate ecosystems interactively and even while considering animals

We have also developed the first algorithm that finds the location of trees in an urban environment (urban forest) that has also been used and extended for tree counting. The figure below shows the result of the procedural tree positioning in a hypothetical scenario of Central Park in New York. An older paper also shows urban forests as affected by how people care about the trees.

One of the critical aspects is the plant development. Our developmental model allows for creating windy trees that show the urban forest under the influence of wind.

Orchards and IMApple

Together with our collaborators, we have developed a source-sink developmental model for Golden Delicious apple trees. The trees consume water and use light for photosynthesis. The result is growth and apples on the trees, as shown in this image:

We also studied the effect of pruning on orchards and co-developed an automatic pruning optimization algorithm that finds what branch should be removed to maximize light intake of a tree.

Validation

One of the open problems is the validation of the computer-generated models. Together with our collaborators, we have developed the first perceptual model called ICTree for assessing the perceived realism of tree models. An AI-based model was trained on one million user evaluations and can predict the perceived quality of the tree model either from its geometry or an image. An example below shows an editing session and the feedback on the metric.

Current Members

Jae Joong Lee

Bosheng Li

Ian Andrew Ostermann

Zhiquan Wang

Xiaochen Zhou

Past Members

Javier Abdul Cordoba (2003)


Institution:

Michel Abdul-Massih, Ph.D. (2014)


Institution: Electronic Arts

Marek Fiser, M.S. (2015)


Institution: Google Inc

Alejandro Guayaquil, Ph.D. (2014)


Institution: Siemens Inc

Mathieu Gaillard, Ph.D. (2022)


Institution: Adobe Research

David Hrusa, M.S. (2021)


Institution:

Hao Kang, Ph.D. (2019)


Institution: Wormplex AI Research

Vojtech Krs, Ph.D. (2019)


Institution: Adobe Research

Juan Miguel Soto, M.S.


Institution:

Eric Millan, M.S.


Institution:

Ondrej Stava, Ph.D. (2012)


Institution: Google Inc

Juraj Vanek, Ph.D. (2014)


Institution: Relativity Space

Collaborators

Martin Čadík

Marie-Paule Cani

Guillaume Cordonnier

Oliver Deussen

Pierre Ecormier-Nocca

Arnaud Emilien

Jianwei Guo

Rado Gzao

Torsten Hädrich

John Hart

Eva Haviarova

Peter Hirst

Hui Huang

Jacek Kaluzny

Jonathan Klein

Stefan Kohek

Simon Kolmanič

Julian Kratt

Yanchao Liu

Amy Marshall-Collon

Radomir Mech

Chenyong Miao

Dominik Michels

Till Niese

Wojteck Palubicki

Soeren Pirk

Tomáš Polášek

James Schnable

Damjan Strnad

Michael Tross

Logan Wells

Fanyou Wu

Borut Zalik

Xiaopen Zhang

Computational Vegetation Publications

2024

  1. Lee, J. J., Li, B., & Benes, B. (2024). Latent L-Systems: Transformer-Based Tree Generator. ACM Trans. Graph., 43(1). https://doi.org/10.1145/3627101
  2. Tuggle, C. K., Clarke, J. L., Murdoch, B. M., Lyons, E., Scott, N. M., Benes, B., Campbell, J. D., Chung, H., Daigle, C. L., Choudhury, S. D., Dekkers, J. C. M., Dorea, J. R. R., Ertl, D. S., Feldman, M., Fragomeni, B. O., Fulton, J. E., Guadagno, C. R., Hagen, D. E., Hess, A. S., … Schnable, P. S. (2024). Current challenges and future of agricultural genomes to phenomes in the USA. Genome Biology, 25(8). https://doi.org/10.1186/s13059-023-03155-w
  3. Zarei, A., Li, B., Schnable, J. C., Lyons, E., Pauli, D., Barnard, K., & Benes, B. (2024). PlantSegNet: 3D point cloud instance segmentation of nearby plant organs with identical semantics. Computers and Electronics in Agriculture, 221, 108922. https://doi.org/https://doi.org/10.1016/j.compag.2024.108922
  4. Warner, C., Wu, F., Gazo, R., Benes, B., Kong, N., & Fei, S. (2024). CentralBark Image Dataset and Tree Species Classification Using Deep Learning. Algorithms, 17(5). https://doi.org/10.3390/a17050179

2023

  1. Rhizomorph: The Coordinated Function of Shoots and Roots. (2023). ACM Transaction on Graphics, 42(4). https://doi.org/https://doi.org/10.1145/3592145
  2. Zhou, X., Li, B., Benes, B., Fei, S., & Pirk, S. (2023). DeepTree: Modeling Trees with Situated Latents. IEEE Transactions on Visualization & Computer Graphics, 01, 1–14. https://doi.org/10.1109/TVCG.2023.3307887
  3. Firoze, A., Wingren, C., Yeh, R. A., Benes, B., & Aliaga, D. (2023). Tree Instance Segmentation With Temporal Contour Graph. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2193–2202.

2022

  1. Niese, T., Pirk, S., Albrecht, M., Benes, B., & Deussen, O. (2022). Procedural Urban Forestry. ACM Trans. Graph., 41(2). https://doi.org/10.1145/3502220
  2. Firoze, A., Benes, B., & Aliaga, D. (2022). Urban tree generator: spatio-temporal and generative deep learning for urban tree localization and modeling. The Visual Computer. https://doi.org/10.1007/s00371-022-02526-x

2021

  1. Polasek, T., Hrusa, D., Benes, B., & Cadik, M. (2021). ICTree: Automatic Perceptual Metrics for Tree Models. ACM Transaction on Graphics, 40(6), 1–15. https://doi.org/10.1145/3478513.3480519
  2. Li, B., Kałużny, J., Klein, J., Michels, D. L., Pałubicki, W., Benes, B., & Pirk, S. (2021). Learning to Reconstruct Botanical Trees from Single Images. ACM Transaction on Graphics, 40(6), 1–15. https://doi.org/10.1145/3478513.3480525
  3. Liu, Y., Guo, J., Benes, B., Deussen, O., Zhang, X., & Huang, H. (2021). TreePartNet: Neural Decomposition of Point Clouds for 3D Tree Reconstruction. ACM Transaction on Graphics, 40(6), 1–16. https://doi.org/10.1145/3478513.3480486
  4. Ecormier-Nocca, P., Cordonnier, G., Carrez, P., Moigne, A.-marie, Memari, P., Benes, B., & Cani, M.-P. (2021). Authoring Consistent Landscapes with Flora and Fauna. ACM Transactions on Graphics, 40(4). https://doi.org/10.1145/3450626.3459952
  5. Tross, M. C., Gaillard, M., Zweiner, M., Miao, C., Grove, R. J., Li, B., Benes, B., & Schnable, J. C. (2021). 3D reconstruction identifies loci linked to variation in angle of individual sorghum leaves. PeerJ, 9:e12628(1). https://doi.org/10.7717/peerj.12628
  6. Wu, F., Gazo, R., Benes, B., & Haviarova, E. (2021). Deep BarkID: a portable tree bark identification system by knowledge distillation. European Journal of Forest Research. https://doi.org/10.1007/s10342-021-01407-7
  7. Wu, F., Gazo, R., Haviarova, E., & Benes, B. (2021). Wood identification based on longitudinal section images by using deep learning. Wood Science and Technology, 55, 553–563. https://doi.org/10.1007/s00226-021-01261-1
  8. Kolmanič, S., Strnad, D., Kohek, Š., Benes, B., Hirst, P., & Žalik, B. (2021). An algorithm for automatic dormant tree pruning. Applied Soft Computing, 99, 106931. https://doi.org/10.1016/j.asoc.2020.106931

2020

  1. Guo, J., Jiang, H., Benes, B., Deussen, O., Zhang, X., Lischinski, D., & Huang, H. (2020). Inverse Procedural Modeling of Branching Structures by Inferring L-Systems. ACM Trans. Graph., 39(5). https://doi.org/10.1145/3394105
  2. Benes, B., Guan, K., Lang, M., Long, S. P., Lynch, J. P., Marshall-Colón, A., Peng, B., Schnable, J., Sweetlove, L. J., & Turk, M. J. (2020). Multiscale computational models can guide experimentation and targeted measurements for crop improvement. The Plant Journal, 103(1), 21–31. https://doi.org/10.1111/tpj.14722
  3. Strnad, D., Štefan Kohek, Benes, B., Kolmanič, S., & Žalik, B. (2020). A framework for multi-objective optimization of virtual tree pruning based on growth simulation. Expert Systems with Applications, 162, 113792. https://doi.org/https://doi.org/10.1016/j.eswa.2020.113792
  4. Gaillard, M., Miao, C., Schnable, J. C., & Benes, B. (2020). Voxel carving-based 3D reconstruction of sorghum identifies genetic determinants of light interception efficiency. Plant Direct, 4(10), e00255. https://doi.org/10.1002/pld3.255
  5. Gaillard, M., Miao, C., Schnable, J., & Benes, B. (2020). Sorghum Segmentation by Skeleton Extraction. In A. Bartoli & A. Fusiello (Eds.), Computer Vision – ECCV 2020 Workshops (pp. 296–311). Springer International Publishing. https://doi.org/ 10.1007/978-3-030-65414-6_21
  6. Gazo, R., Vanek, J., Abdul-Massih, M., & Benes, B. (2020). A fast pith detection for computed tomography scanned hardwood logs. Computers and Electronics in Agriculture, 170, 105107. https://doi.org/https://doi.org/10.1016/j.compag.2019.105107

2019

  1. Gaillard, M., Benes, B., Guérin, E., Galin, E., Rohmer, D., & Cani, M.-P. (2019). Dendry: A Procedural Model for Dendritic Patterns. Proceedings of the ACM SIGGRAPH Symposium on Interactive 3D Graphics and Games, 16:1–16:9. https://doi.org/10.1145/3306131.3317020

2018

  1. Gazo, R., Wells, L., Krs, V., & Benes, B. (2018). Validation of automated hardwood lumber grading system. Computers and Electronics in Agriculture, 155, 496–500. https://doi.org/https://doi.org/10.1016/j.compag.2018.06.041
  2. Wells, L., Gazo, R., Re, R. D., Krs, V., & Benes, B. (2018). Defect detection performance of automated hardwood lumber grading system. Computers and Electronics in Agriculture, 155, 487–495. https://doi.org/https://doi.org/10.1016/j.compag.2018.09.025

2017

  1. Cordonnier, G., Galin, E., Gain, J., Benes, B., Guérin, E., Peytavie, A., & Cani, M.-P. (2017). Authoring Landscapes by Combining Ecosystem and Terrain Erosion Simulation. ACM Trans. Graph., 36(4), 134:1–134:12. https://doi.org/10.1145/3072959.3073667
  2. Hädrich, T., Benes, B., Deussen, O., & Pirk, S. (2017). Interactive modeling and authoring of climbing plants. Computer Graphics Forum, 36(2), 49–61. https://doi.org/10.1111/cgf.13106
    Honorable mention from the Best Papers Committee
  3. Marshall-Colon, A., Long, S. P., Allen, D. K., Allen, G., Beard, D. A., Benes, B., von Caemmerer, S., Christensen, A. J., Cox, D. J., Hart, J. C., Hirst, P. M., Kannan, K., Katz, D. S., Lynch, J. P., Millar, A. J., Panneerselvam, B., Price, N. D., Prusinkiewicz, P., Raila, D., … Zhu, X.-G. (2017). Crops In Silico: Generating Virtual Crops Using an Integrative and Multi-scale Modeling Platform. Frontiers in Plant Science, 8, 786. https://doi.org/10.3389/fpls.2017.00786
  4. Fišer, M., Ravi, J., Benes, B., Shi, B., & Hirst, P. (2017). IMapple: a source-sink developmental model for’Golden Delicious’ apple trees. Acta Hortic, 51–60. https://doi.org/doi.org/10.17660/ActaHortic.2017.1160.8

2016

  1. Pirk, S., Benes, B., Ijiri, T., Li, Y., Deussen, O., Chen, B., & Měch, R. (2016). Modeling Plant Life in Computer Graphics. ACM SIGGRAPH 2016 Courses, 18:1–18:180. https://doi.org/10.1145/2897826.2927332
  2. Kang, H., Fiser, M., Shi, B., Sheibani, F., Hirst, P., & Benes, B. (2016). IMapple - Functional structural model of apple trees. 2016 IEEE International Conference on Functional-Structural Plant Growth Modeling, Simulation, Visualization and Applications (FSPMA), 90–97. https://doi.org/10.1109/FSPMA.2016.7818293

2015

  1. Emilien, A., Vimont, U., Cani, M.-P., Poulin, P., & Benes, B. (2015). WorldBrush: Interactive Example-Based Synthesis of Procedural Virtual Worlds. ACM Trans. Graph., 34(4). https://doi.org/10.1145/2766975
  2. Kratt, J., Spicker, M., Guayaquil, A., Fiser, M., Pirk, S., Deussen, O., Hart, J. C., & Benes, B. (2015). Woodification: User-Controlled Cambial Growth Modeling. Computer Graphics Forum, 34(2), 361–372. https://doi.org/10.1111/cgf.12566

2014

  1. Pirk, S., Niese, T., Hädrich, T., Benes, B., & Deussen, O. (2014). Windy Trees: Computing Stress Response for Developmental Tree Models. ACM Trans. Graph., 33(6). https://doi.org/10.1145/2661229.2661252
  2. Šťava, O., Pirk, S., Kratt, J., Chen, B., Měch, R., Deussen, O., & Benes, B. (2014). Inverse Procedural Modelling of Trees. Computer Graphics Forum, 33(6), 118–131. https://doi.org/10.1111/cgf.12282

2012

  1. Pirk, S., Stava, O., Kratt, J., Said, M. A. M., Neubert, B., Měch Radomı́r, Benes, B., & Deussen, O. (2012). Plastic Trees: Interactive Self-Adapting Botanical Tree Models. ACM Trans. Graph., 31(4). https://doi.org/10.1145/2185520.2185546

2011

  1. Benes, B., Massih, M. A., Jarvis, P., Aliaga, D. G., & Vanegas, C. A. (2011). Urban Ecosystem Design. Symposium on Interactive 3D Graphics and Games, 167–174. https://doi.org/10.1145/1944745.1944773

2010

  1. Št’ava, O., Benes, B., Měch, R., Aliaga, D. G., & Krištof, P. (2010). Inverse Procedural Modeling by Automatic Generation of L-systems. Computer Graphics Forum, 29(2), 665–674. https://doi.org/10.1111/j.1467-8659.2009.01636.x

2009

  1. Benes, B., Andrysco, N., & Št’ava, O. (2009). Interactive Modeling of Virtual Ecosystems. Proceedings of the Fifth Eurographics Conference on Natural Phenomena, 9–16. https://doi.org/10.5555/2381692.2381694

2004

  1. Benes, B., & Guerrero, J. M. S. (2004). Clustering in virtual plant ecosystems. WSCG SHORT Communication Papers Proceedings.

2003

  1. Benes, B., & Cordóba, J. A. (2003). Modeling virtual gardens by autonomous procedural agents. Proceedings of Theory and Practice of Computer Graphics, 2003., 58–65. https://doi.org/10.1109/TPCG.2003.1206931
  2. Benes, B., & Espinosa, E. D. (2003). Modeling virtual ecosystems with the proactive guidance of agents. Computer Animation and Social Agents, 2003. 16th International Conference On, 126–131. https://doi.org/10.1109/CASA.2003.1199313
  3. Benes, B., Cordoba, J. A., & Soto, J. M. (2003). Interacting agents with memory in virtual ecosystems. Journal of WSCG.

2002

  1. Beneš, B. (2002). A stable modeling of large plant ecosystems. Proceedings of the International Conference on Computer Vision and Graphics, 94–101.
  2. Benes, B., & Millán, E. U. (2002). Virtual Climbing Plants Competing for Space. Computer Animation, 33. https://doi.org/10.1109/CA.2002.1017504

Older

  1. Benes, B. (1997). Visual Model of Plant Development with Respect to Influence of Light. In D. Thalmann & M. van de Panne (Eds.), Computer Animation and Simulation ’97 (pp. 125–136). Springer Vienna. https://doi.org/10.1007/978-3-7091-6874-5_9
  2. Benes, B. (1996). An efficient estimation of light in simulation of plant development. In R. Boulic & G. Hégron (Eds.), Computer Animation and Simulation ’96 (pp. 153–165). Springer Vienna. https://doi.org/10.1007/978-3-7091-7486-9_11
  3. Benes, B. (1998). Skylight approximation for simulation of plant development. Proceedings. 1998 IEEE Conference on Information Visualization. An International Conference on Computer Visualization and Graphics (Cat. No. 98TB100246), 146–150. https://doi.org/10.1109/IV.1998.694212

Sponsors

2021-2021

Agricultural Genome to Phenome Initiative, (PI)

Optimizing 3D Canopy Architecture for Better Crops,


2019-2022

FFAR

Crops in Silico (see the news here)


2019-2022

National Science Foundation, (co-PI) 10001387

CHS Small Functional Proceduralization of 3D Geometric Models


2017-2019

National Science Foundation, (PI) 1608762

Collaborative Research-Inverse Procedural Material Modeling for Battery Design,


2015-2016

Intel, (PI)

Enhancing Computer Graphics Education with Many Integrated Core Computing


2014-2016

Ford Inc, (co-PI)

Tomographic data reconstruction,


2014-2015

Siemens Inc, (PI)

Software analysis for 3-D printing


2013-2015

United States Department of Agriculture(co-PI)

Integrating Spatial Educational Experiences (Isee) – Mapping a New Approach to Teaching and Learning Soil Science


2011-2014

NASA (CO-PI)

A Global High-Resolution Fossil Fuel CO2 Inventory Built From Assimilation of in Situ and Remotely-Sensed Datasets to Advance Satellite Greenhouse Gas Detection Support Systems


2010-2013

National Science Foundation (CO-PI)

Integrating Behavioral, Geometrical and Graphical Modeling to Simulate and Visualize Urban Areas


2010-present

Adobe Inc (PI)

Procedural Modeling


2008-2010

United States Department of Agriculture(co-PI)

Integrating Spatial Education Experience (ISEE) into Crop, Soil, and Environmental Science Curricula


Contact

Office: Lawson Builidng
Address: 305 N University St., Purdue University, West Lafayette, IN 47907-2107
Email: bbenes [at] purdue.edu
Phone: (765) 496-2954