MESHDECOMP

The MESHDECOMP is an open source project for decomposition of meshes, which will be used in parallel computing. The MESHDECOMP code is a part of SIFEL open source project and is located in PREP/PARTITIONING/ of SIFEL folder sturcture. The code uses keywords from SIFEL. The MESHDECOMP code has four modes.

  1. preprocessing - creates dual graph from finite element mesh. This dual graph is then used in graph partitioners - METIS, CHACO, PARTY and JOSTLE.
  2. postprocessing - creates decomposed mesh (several files with mesh topology) form output of mentioned partitioners.
  3. all - is complex and directly use METIS library for partitioning. In such a case, sequential mesh is input and decomposed mesh is output.
  4. aggregates - is for sequential computation with aggregates - BOSS preconditioning.

The syntax for usage is:

./meshdecomp file.dec

 where file.dec is a file with informations about mesh and decomposition. File is split into two parts - sec_files and sec_part. The structure of the part sec_files is following:

begsec_files # section with informations about sequential meshes and mesh description
topology_file file.top # file with sequential mesh
mesh_format sifel # format of sequential mesh - sifel or t3d
edge_numbering 1 # edge numbering
endsec_files

The sec_part has different structure for each option of the processing. The first line includes the type of processing. Syntax of sec_part section:

begsec_part # section with partitioning data
processing {preprocessing | postprocessing | all | aggregates}
endsec_part

 All types of processing can be used in case of METIS. In all other cases, preprocessing or postprocessing can be only used. Preprocessing creates dual graph for graph partitioners. There is structure sec_part:

partType {METIS| JOSTLE| CHACO| PARTY}
weigh_graph {noweight| nodalweight |edgeweight |nodealedgeweight |multiconstrain}

 where the options have the following meaning:

  • noweight - unweighted graph
  • nodalweight - vertices of graph have only own weights
  • edgeweight - edges of graph have only own weights
  • nodealedgeweight - vertices and edges have own weights
  • multiconstrain - for multi constrained graphs
weight_option {default | user}

 where the options have the following meaning:
  • default - weights are computed in code
  • user - user defined weights
file_with_weights weightf

 where weightf is file with weights.

graph_output_file graphf

where graphf is file for output dual graph.

Several examples of file.dec follows for the case of preprocessing. Topology file.top with mesh is in the SIFEL format and dual unweighted graph will be created for JOSTLE. Corresponding input file for the MESHDECOMP is in file.dec produces resulting file file-graph.grf. Another examples of input files for MESHDECOMP follows:

Input file for the mesh given in SIFEL format and dual graph with nodal weights created for JOSTLE with computed weights:

begsec_files
mesh description
topology_file file.top
mesh_format sifel
edge_numbering 1
endsec_files
begsec_part
processing preprocessing
partType JOSTLE
weight_graph nodalweight
weight_option default
graph_output_file file-graph
endsec_part

 Input file for the mesh given in SIFEL format and dual graph with nodal weights created for JOSTLE with weights defined by user.

begsec_files
mesh description
topology_file file.top
mesh_format sifel
edge_numbering 1
endsec_files
begsec_part
processing preprocessing
partType JOSTLE
weight_graph nodalweight
weight_option user
graph_output_file file-graph
endsec_part

Postprocessing creates files with meshes for parallel computing. Creation of such meshes is based on output from graph partitioners and on original sequential mesh. There is structure sec_part which contains following records:

  • mesh_description {all_nodes| bound_nodes| neg_bound_nodes} - type of mesh description of parallel meshes - bound_nodes or all_nodes (see PARGEF manual),
  • partType {METIS| JOSTLE| CHACO| PARTY},
  • number_of_partitions npart where npart is number of partitions into which original mesh was decomposed,
  • partitioning_file partf where partf is file with decomposition of the sequential mesh,
  • output_file_name outputfname where outputfname is filename for files with decomposed mesh.

Example of file_post.dec in case of postprocessing:
Mesh is in the SIFEL format and JOSTLE graph partitioner was used for decomposition. The mesh was decomposed into 4 subdomains and results are stored in files file-decomp1.top - file-decomp4.top. The file.top is the same as in above examples. The file.grf.ptn contains output from JOSTLE.

begsec_files
mesh description
topology_file file.top
mesh_format sifel
mesh_description bound_nodes
edge_numbering 1
endsec_files
begsec_part
processing postprocessing
mesh_description bound_nodes
partType JOSTLE
number_of_partitions 4
partitioning_file file.grf.ptn
output_file_name file-decomp.top

The option All in processing can be use only with METIS partitioning and creates parallel topology directly from sequential mesh with the help of METIS library. There is structure sec_part with this option:

  • number_of_partitions npart where npart is number of partitions into which original mesh was decomposed
  • weight_graph {noweight| nodalweight| edgeweight| nodealedgeweight| multiconstrain}
  • weight_option {default | user}
  • file_with_weights weightf
  • output_file_name outputfname
  • partitioning_technique {recursive | kway}
The difference between kway and recursive bisection is described in METIS manual. If the number of partition is smaller than eight recursive bisection is better than kway.

Example of file.dec in case of all:
Mesh is in SIFEL format and METIS graph partitioner will be used for decomposition. The topology file file.top is the same as in the above examples. The mesh will be decomposed into 4 subdomains and results are stored in files file-dec1.top - file-dec4.top. The input file for the MESHDECOMP file_metis.dec contains:

begsec_files
mesh description
topology_file file.top
mesh_format sifel
mesh_description bound_nodes
edge_numbering 1
endsec_files
begsec_part
processing all
mesh_description bound_nodes
number_of_partitions 4
weight_graph noweight
output_file_name file-dec.top
partitioning_technique recursive
endsec_part

Aggregates in processing can be use only with METIS partitioning and reprint sequential topology and uses mesh description for aggregates computation. The structure of section is similar as in All processing. Example of sec_part:

begsec_part
processing aggregates
number_of_aggregates naggreg where naggreg is number of aggregates
output_file_name newfile.top
partitioning_technique recursive
endsec_part

Written by Jaroslav Broz 2010-11-11


Image