****************************************************************************
*  Copyright (C) IAC3-UIB. Licensed under BSD. Please see details at       *
*  http://www.iac3.eu/simflowny-copyright-and-license                      *
****************************************************************************
****************************************************************************
*                                                                          *
*                      Code generated for AMReX                            *
*     Structured Adaptive Mesh Refinement Applications Infrastructure      *
*                 https://amrex-codes.github.io/amrex/                     *
*                                                                          *
****************************************************************************

Overview
--------

This document describes the Simflowny's generated code for AMReX framework as well as the parameters for the simulation.

The downloaded code is already compiled in the machine running Simflowny server.
The user may want to manually modify the code and compile it again using the MAKEFILE providen.

As a C++ code, files may have .cpp (or .C) and .h extension, both refering to the same class. The .h files declares the functions and variables used within the class, and the .cpp implements the functions. 

Simulation parameters
---------------------

Simflowny generates code for AMReX framework for mesh based problems. The following sections explain all the parameters.

First of all, AMReX  classifies parameter scope by a preffix. An example may be:

.. code-block:: 

    problem.tend = 0.0
    geometry.prob_lo     =  -15.0 -15.0 -15.0 
    amr.v   = 0

where each preffix encapsulates a common behaviour managed by a C++ class.
    
Problem parameters
******************

The first parameters are the problem ones, those specified by the user during the problem creation. They must be set conveniently for a model, problem and discretization schema desired behaviour.

Restart parameters
******************

AMReX provides the capability to stop a simulation and restart it from stored checkpoints. To start from a checkpoint, the following parameter must be uncommented and set:


    - amr.restart: if the simulation is restarting from checkpoint, this parameter indicates from which checkpoint folder.

Geometry parameters
*******************

In these parameters the simulation domain is set. Simflowny automatically sets the physical domain with the data set in the problem.

    - prob_lo, prob_hi: indicates the domain size in problem units. Simflowny sets the problem values, but it can be also modified by the user.
    - is_periodic: are already set by Simflowny depending on the problem boundary conditions. Indicates the dimensions that have periodical boundary. It should not be changed by a user since boundaries that are not periodic are implemented within the code. If the user wants to change the boundary conditions is strongly advised to modify the problem and generate the code again.

Resolution and refinement parameters
************************************

These parameter are used for resolution and regridding purposes. 

    - v: activates a more detailed console output.
    - n_cell: sets the mesh size. The ghost cells needed for calculations are automatically added to the mesh afterwards. Must be multiple of amr.blocking_factor.
    - max_level: indicates the maximum number of mesh levels. A value of 0 indicates that regridding is disabled. When a value is 1 or greater regridding is active.
    - ref_ratio: is used to set the regridding factor for the next level of refinement. More than one value is allowed to provide a different ratio for the levels.
    - regrid_int: in case of adaptive regridding it indicates when the regridding must be reset.
    - interpolator: spatial interpolator to fill ghost cells in finer levels. Allowed values are: "LINEAR_REFINE", "CUBIC_REFINE", "QUINTIC_REFINE".
    - fmr_levels: sets the number of fixed mesh refinement (FMR) levels. Must be lower or equal to max_level. If the value is lower than max_level, the remain levels will use adaptive mesh refinement criteria to create be created.

The following parameters represent the domain for each expected FMR level. For a new FMR level, two parameters are expected:
    - fmr_level_x_lo: lower domain for the refinement level x+1.
    - fmr_level_0_hi: upper domain for the refinement level x+1.
    
Grid control parameters
***********************

AMReX grid needs some parameters that help performance.

    - blocking_factor: this ensures that every grid is coarsenable by this factor for performance in multigrid.
    - max_grid_size: set up the maximum size of boxes in each level.
    - n_error_buf: indicates the minimum number of cells between regridding levels.

Evolution time parameters
*************************

The simulation time step cannot be set directly. The Courant–Friedrichs–Lewy condition must be set:

    - cfl: sets Courant–Friedrichs–Lewy condition.

AMR simulations can be shortened in execution time when using subcycling. The following parameter is used to activate it when needed.

    - do_subcycle: sets to 1 to activate subcycling. This is the classical subcycling algorithm and fastest implemented in Simflowny. The time step for every level in the AMR hierarchy is proportional to the refinement ratio, so coarser level steps covers more time than finer ones. Therefore, sophisticated synchronization is used between levels. 

AMR paremeters
**************

AMReX provides Adaptive Mesh Refinement (and also Fixed Mesh Refinement). A programmer can provide any algorithm to tag the mesh cells to refine. In the current version, Simflowny provides two tagging algorithms that are configured in this database. In the example file automatically created when generatig code, the two regridding configurations are available.

    - regridding_type: used to select the specific regridding criteria.

Regarding the specific regridding parameters:

    - Function:. The cell is refined if the provided field is greater than a given threshold. The parameters are
        - tag_field: is the field in which to check the threshold.
        - tag_err: is the threshold for the field.
    
    - Shadow:. Two timesteps from a field are compared, if the relative difference is greater than a given error, the cell is to be refined.
        - tag_field: is the main field to shadow.
        - tag_shadow_time: shadow time to compare. By default previous time step.
        - tag_err: is the error relative threshold.

Note: The fields used in the refinement criteria cannot be from analysis.

Plotting parameters
*******************

Simflowny provides a series of different plots. Some of them are only avaliable for 3D problems.

    - plot_overwrite: whether the plots are overwritten or old ones are renamed.

Full plot
---------

Plot all cells in the domain. (plt_full preffix)

    - dir: output directory
    - varnames: variables to store
    - int: interval of output, in finer cycle units. Set a negative value to disable output.

Slice plot
----------

When available, there will be a slicing capability for writing 2D slices of 3D data. The user can set as many slices as desired. Each slice has its own properties and must be defined inside its own preffix plt_slice_x, being x the number of the slice, beginning with 1 and being correlative. The properties of a slice are the following:

    - dir: output directory
    - varnames: variables to store
    - int: interval of output, in finer cycle units. Set a negative value to disable output.
    - plane_normal_axis:. To select the axis used as the normal of the slice plane. Allowed values are 1, 2, or 3.
    - distance_to_origin:. Used to set the location of the plane. The user must set the value in domain units.

Sphere plot
-----------

Another extra capability is providen to extract a spherical surface from 3D problems. The user can set as many spheres as desired. Each sphere has its own properties and must be defined inside its own section plt_sphere_x, being x the number of the sphere, beginning with 1 and being correlative. The properties of a sphere are the following:

    - dir: output directory
    - varnames: variables to store
    - int: interval of output, in finer cycle units. Set a negative value to disable output.
    - center:. The center of the sphere in domain units
    - radius:. The radius of the sphere in domain units.
    - resolution:. The number of cells for the 2D projection. Projection domain is :math:`[2 * \Pi, \Pi]`, corresponding to angles theta and phi (Description in https://en.wikipedia.org/wiki/Sphere#Equations_in_three-dimensional_space).

Integral plot
-------------

It is also possible to output  integration from the whole domain cells. The user can set as many integrations as desired. Each integration has its own properties and must be defined inside its own section plt_integral_x, being x the number of the sphere, beginning with 1 and being correlative. The properties of integration are the following:


    - dir: output directory
    - varnames: variables to store
    - int: interval of output, in finer cycle units. Set a negative value to disable output.
    - calculation:. The list of calculations to apply to variables. *INTEGRAL*, *L2NORM*, *MIN*, *MAX*, *ABSMIN* and *ABSMAX* are available

Puntual plot
------------

The last plot capability is output point data. The user can set as many points as desired. Each point has its own properties and must be defined inside its own section plt_point_x, being x the number of the point, beginning with 1 and being correlative. The properties of a point are the following:

    - dir: output directory
    - varnames: variables to store
    - int: interval of output, in finer cycle units. Set a negative value to disable output.
    - coordinates:. Coordinate of the point in domain untis

Checkpoint parameters
*********************

AMReX provides the capability to stop a simulation and restart it from stored checkpoints. The creation of checkpoints are set by the following parameters:
    
    - file: sets the name of the directory for the restart snapshots.    
    - int: if this value is greater than zero it will store snapshots for a future restart with the providen frequency.

Compilation
-----------

If the user modifies the source code or wants to run the code in a different machine from Simflowny server, the Makefile provided eases the task.
First of all, open it with a text editor and check that the library paths are the same of your system configuration. 
Once the Makefile is properly configured, open a terminal and execute:
    $ make


Execution
---------

To run a simulation in a single processor run in a terminal:
    $ ./binFile parameterFile.input
where binName is the name of the binary created in the compilation, and parameter.input is the name of the parameter file.

To run multiprocessor run:
    $ mpirun -np X ./binFile parameterFile.input

where X is the number of processors in which the simulation will run.

File Description
----------------

The following files are the most important ones which the user may be interested in look into or modify.

* Functions - These files may contain any function from the discretization schema which is not a MACRO function and the extrapolation functions if needed. One-line functions are automatically translated to MACRO functions by Simflowny for performance purposes and are located in the Problem class.

* AdvanceLevel - This file implements the main evolution algorithm.

* AmrCoreProblem - This class manages the simulation using AMReX routines.

* InitialCondition - It has the function for initial condition.

* Makefile - Makefile is a special format file that together with the make utility will help you to automatically compile this code. It contains the references to the required libraries and should be adapted if any library is installed in a different location from Simflowny server.


Problem modification
--------------------

The following sections describe the most significative functions for an user wanting to manually modify part of the code.

    - AdvanceLevel::AdvanceLevel. This section includes the whole algorithm (discretized problem) that is executed at one time step.
    - AmrCoreProblem::postInitialization. There are the post-initialization calculations.
    - AmrCoreProblem::mapDataOnPatch. This section maps the problem regions in the mesh.
    - InitialCondition::initdata. Initialization of the fields from the problem equations.
    - AmrCoreProblem::checkFinalization. Where the finalization condition is implemented.
