Create a simple parallel model

Unsupported Processes

So far, the parallel FEM scheme with PETSc cannot be used by the all processes using linear and quadratic function spaces simultaneously. They are

• HydroMechanics (including that with LIE)
• ThermoHydroMechanics
• RichardsMechanics

Software Modules and Compilation on EVE

• Checkout the OGS-6 source code
  • Source the environment scripts scripts/env/eve/cli.sh
• Configure with utils enabled to:
  • Create the binary for generation of the structured mesh
  • Create the binary for the mesh partioner tool partmesh
  • cmake <path_to_source> -DOGS_BUILD_UTILS=ON
• Configure OGS for parallel usage with PETSc:
  • Source the environment scripts scripts/env/eve/petsc.sh
  • cmake <path_to_source> -DOGS_USE_PETSC=ON
• Build the source code with make

Create a structured mesh

In order to discretize the domain of a unit cube do bin/generateStructuredMesh -o cube_1x1x1_hex_axbxc.vtu -e hex --lx 1 --ly 1 --lz 1 --nx a --ny b --nz c where a, b, and c should be choosen according to the needs.

For the boundary conditions, if they are simple (_i.e._ homogeneous), the simplest .gml file is sufficient for the serial case, but for heterogeneous boundary conditions and parallelization boundary mesh files are needed. There are two possibilities to create such files:

• If there is a .gml file, use constructMeshesFromGeometry tool which takes the mesh file and geometry and creates all the boundaries which are named in the .gml file with the required bulk_node_ids and bulk_element_ids mappings.
• If there is a boundary mesh (generated by gmsh or salome, or extracted in paraview, for example) use the identifySubdomains tool to create or verify the needed bulk_node_ids and bulk_element_ids mappings.

Partition the mesh with partmesh

1. Convert vtu mesh into metis input mesh
   bin/partmesh -i cube_1x1x1_hex_axbxc.vtu --ogs2metis
   => results in a file cube_1x1x1_hex_axbxc.mesh
2. partition the mesh and the corresponding boundaries.
   bin/partmesh -n number_of_partitions -m -i cube_1x1x1_hex_axbxc.vtu -- boundary_meshes*.vtu
   This will result in a bunch of .bin files.

Deciding the part to benchmark

• Usually, for solving elliptic problems (for instance, the steady state groundwater flow) the time for the linear solver is the main part of the simulation.
• The hydro-thermal process can be used for testing of the IO capabilities.
• In the mechanics process large parts of the computational resources are spent for the assembly.

ToDo

• script to create models and partitions
• submit script to cluster
• Metrics to take into account could be
  • weak scaling (problem size increases as the number of processors increases)
  • strong scaling (fixed problem size, use increasing number of processors => speedup and parallel efficiency)
  • fixed number of processors, increasing problem size