Solver

Each model and some filters are configurable with a set of parameters. For example, the pattern prediction parameters encode the learned patterns of glucose level change. As yet another example, in-silico patient model has parameters, which describes initial quantities and rates of metabolic quantities. To personalize such parameters to fit a particular patient, we need to find them using a combination of a solver and one or more error metrics. To the solver, the parameters are just a set of opaque data. Metrics provide fitness functions. Hence, it is up to the solver to generate and test new set of parameters based on the fitness function calls.

A solver is, in fact, not a SmartCGMS entity, but rather a service provided by an implementing library. The solver itself thus does not need to define an interoperable class.

Descriptor

A solver is described by the scgms::TSolver_Descriptor structure:

struct TSolver_Descriptor {
    const GUID id;
    const wchar_t *description;
    const BOOL specialized;	//if false, can be applied to any model
    const size_t specialized_count;		//can be zero with specialized == false, usually 1
    const GUID *specialized_models;		//array of models, whose parameters the solver can solve
};

• id is an unique GUID assigned to this solver. The solver is invoked using this identifier.
• description is a string representing a name of the solver; it must not be nullptr and must contain a valid zero-terminated string
• specialized is a flag, marking that the solver is able to optimize just a selected subset of models; if this is set to TRUE, the following two fields apply:
• specialized_count is a size of the following array of models
• specialized_models is an array of size specialized_count, containing the GUIDs of models, that the solver is able to optimize

A shared object exporting a solver must:

• export a do_get_solver_descriptors function
  • this function returns a continuous array of solver descriptors identified by its first and one-after-last element
• export a do_solve_generic function
  • this function calls the solver service

An example of do_get_solver_descriptors may be as follows:

const std::array solver_descriptors = { { descriptor_1, descriptor_2 } };

HRESULT IfaceCalling do_get_solver_descriptors(scgms::TSolver_Descriptor **begin, scgms::TSolver_Descriptor **end) {
    *begin = solver_descriptors.data();
    *end = solver_descriptors.data() + solver_descriptors.size();
    return S_OK;
}

An example of do_solve_generic may be as follows:

HRESULT IfaceCalling do_solve_generic(const GUID *solver_id, solver::TSolver_Setup *setup, solver::TSolver_Progress *progress) {
    if (*solver_id == descriptor_1.id) {
        CExample_Solver solver(*setup);
        return solver.Solve(*progress);
    }

    return E_NOTIMPL;
}

Solver progress

The solver::TSolver_Progress container defines the following fields:

• current_progress (size_t) - a field containing the current progress value (e.g. a number of past generations)
• max_progress (size_t) - a field containing the maximum progress value (e.g. a maximum number of generations)
• best_metric (TFitness) - an array of best fitnesses; the SmartCGMS supports multi-criterial optimalization with maximum of solver::Maximum_Objectives_Count criterias (10 at the moment of writing of this document)
• cancelled (BOOL) - by setting this field to TRUE (or essentially any other value than FALSE (zero)), the solver is cancelled on a next checkpoint (e.g. between the generations)

Solver setup

The solver retains a configuration in a solver::TSolver_Setup container:

struct TSolver_Setup {
    const size_t problem_size;
    const size_t objectives_count;
    const double *lower_bound, *upper_bound;
    const double **hints;
    const size_t hint_count;
    double * const solution;

    const void *data;
    const solver::TObjective_Function objective;

    const size_t max_generations;
    const size_t population_size;
    const double tolerance;
}; 

• problem_size - the size of the optimized problem, i.e.; the count of parameters of a given model (or model set)
• objectives_count - count of objectives to optimize by
• lower_bound - lower bound of parameter values
• upper_bound - upper bound of parameter values
• hints - an array of solution hints
• hint_count - a size of above array (number of hints)
• solution - an array of double precision floating point values with the size of problem_size, which is used to store the best result
• data - a type-agnostic pointer to data, that are passed to the objective function
• objective - an objective function pointer, see below for details
• max_generations - a maximum number of generations (iterations) requested by the caller; this may apply depending on the solver logic
• population_size - population size (swarm size, ...) requested by the caller; this may apply depending on the solver logic
• tolerance - a maximum tolerance of parameter improvement; this may apply depending on the solver logic

Objective function

The solver needs an objective function to evaluate given solution. The solver::TObjective_Function type is defined as follows:

using TObjective_Function = BOOL(IfaceCalling*)(const void *data, const size_t count, const double *solution, double* const fitness);

• data retains the data field value from the solver::TSolver_Setup container - the solver does not need to know the interpretation of the contents
• count - is the number of solutions, which are coalesced in the following field
• solution - is an array of parameters, which will get evaluated
  • when multiple solutions is to be evaluated (count is greater than 1), the parameters need to be stored in a continuous memory (coalesced to a single array)
• fitness - is the target container, where to store the calculated fitnesses
  • similar to above - when multiple solutions is to be evaluated, this effectively points to an array of coalesced fitness arrays

SmartCGMS supports up to solver::Maximum_Objectives_Count objectives, and thus the fitness array must point to an array of this size.

The multi-criterial selection of best solution is completely dependent on the solver logic. Solvers implemented in the SmartCGMS core uses a simplified Pareto front to determine the best solution. Nonetheless, the simplest approach is to select the dominant error metric (e.g. the fitness at index 0) and ignore the rest.

To obtain the fitness function (and eventually the implicitly constructed objective function), include either the Signal Error or Fast Signal Error filter in the chain. Alternatively, it can be any filter implementing the scgms::ISignal_Error_Inspection with properly implemented Promise_Metric method able to be called with parameters segment_id = scgms::All_Segments_Id and defer_to_dtor = TRUE. Then, SmartCGMS will use this method to register storage places for the fitness values. SmartCGMS will execute a filter chain it optimizes with a particular solver. On destroying the chain, the error metric filter has to store the fitness value to the designated pointer. Then, SmartCGMS picks it up and passes it to the chosen solver.

The SmartCGMS SDK implements a simplifying calls when an optimalization of a filter or model parameters are desired. For such a higher-level wrapper of the solver call, please, refer to the optimizer SDK page.

Example implementation

To demonstrate a simple implementation of solver interface, let us implement a solver, that attempts to randomly change the parameters of a single parameter vector, hoping to obtain a better parameter set. This is, of course, not an ideal approach, should you take this just as a simple example of the interface implementation, code structure and conventions.

Similarly to any other entities, let us create a descriptor header file, despite containing solely the solver GUID:

#include <rtl/guid.h>

namespace example_solver {
    constexpr GUID id = { 0x4e24dbc6, 0xb4f4, 0x458c, { 0xba, 0xe3, 0x97, 0x13, 0x77, 0x60, 0xb0, 0x13 } };	// {4E24DBC6-B4F4-458C-BAE3-97137760B013}
}

Although it is not necessarily needed, let us define a solver class. This helps code readability and maintainability. We split the definition and implementation:

#include <rtl/FilterLib.h>
#include <rtl/referencedImpl.h>
#include <rtl/SolverLib.h>
#include <rtl/UILib.h>

class CExample_Solver {

    protected:
        static constexpr size_t Default_Max_Generations = 1000;

        protected:
            solver::TSolver_Setup &mSetup;

            // NOTE: you may want to store solver::TFitness instead, optionally initialized with solver::Nan_Fitness
            double mBest_Metric = std::numeric_limits<double>::quiet_NaN();

            void Evaluate_Solution(const double* solution, solver::TSolver_Progress &progress);

	public:
            CExample_Solver(solver::TSolver_Setup &setup) : mSetup(setup) {
                //
            }
            virtual ~CExample_Solver() = default;

            HRESULT Solve(solver::TSolver_Progress &progress);
};

Now we need to implement the Evaluate_Solution and Solve methods we just defined. Let us start with the former one, that evaluates the current solution, and if it finds its fitness function value better, it copies the results to a globally best solution:

void CExample_Solver::Evaluate_Solution(const double* solution, solver::TSolver_Progress &progress) {

    // objective function computes a new metric values of given solution
    TFitness metrics;
    auto result = mSetup.objective(mSetup.data, 1, solution, metrics.data());

    // let us select just the first metric for simplicity; you may want to compare fitnesses with a more profound approach
    const double metricValue = metrics[0];

    // if the new metric value is a valid number and is better than the current best metric, update our current best solution
    if (!std::isnan(metricValue) && metricValue < mBest_Metric) {

        // for our calculations:
        mBest_Metric = metricValue;

        // for GUI to display in a visual element:
        progress.best_metric = metrics;
    
        // update current best solution in setup
        std::copy(solution, solution + mSetup.problem_size, mSetup.solution);
    }
}

The Solve method mainly just retains the values from solver setup and performs computation within a loop:

HRESULT CExample_Solver::Solve(solver::TSolver_Progress &progress) {

    // determine generation count
    const size_t genCount = mSetup.max_generations ? mSetup.max_generations : Default_Max_Generations;

    // set max_progress, so the GUI could display correct progress bar value
    progress.max_progress = genCount;

    // initialize (pseudo-)random number generator for our example solver
    static std::random_device rdev;
    std::default_random_engine reng(rdev());
    std::uniform_real_distribution<double> rdist(0.0, 1.0);

    // prepare vector for our temporary solution
    std::vector<double> current_solution(mSetup.problem_size);

    // NOTE: your solver should also respect mSetup.population_size parameter
    //       therefore, there should be a vector of vectors instead of just a single vector;
    //       for the sake of demonstration, we ignore this parameter


    mBest_Metric = std::numeric_limits<double>::max();
    // outer code may help you with some initial parameter estimation called "hints"
    // there may not be any hints present
    for (size_t i = 0; i < mSetup.hint_count; i++) {
        Evaluate_Solution(mSetup.hints[i], progress);
    }

    const double preSolveMetric = mBest_Metric; // currently, we hold the best metric of all hints given

    // iterate through all generations
    // we must respect the "cancelled" parameter of progress, as outer code and another thread may set it to true - in this case,
    // the solver must interrupt as soon as possible and return
    for (size_t generation = 0; generation < genCount && !progress.cancelled; generation++) {

        // we just "emulate" some kind of a solver, so the following lines serves just as an example of "some work" done by a solver
        // of course, one would choose a better strategy, than introduced here

        // generate a random number vector - every vector coordinate within given bounds!
        for (size_t i = 0; i < mSetup.problem_size; i++) {
            current_solution[i] = mSetup.lower_bound[i] + rdist(reng) * (mSetup.upper_bound[i] - mSetup.lower_bound[i]);
        }

        // evaluate newly "computed" solution - this also sets proper status, etc.
        Evaluate_Solution(current_solution.data(), progress);

        // pretend we're doing something - just to demonstrate the progress bar function in GUI
        // NOTE: obviously you don't want the following line of code to appear in your production code
        std::this_thread::sleep_for(std::chrono::milliseconds(10));

        // update current progress, so the GUI would update the visual progress bar element
        progress.current_progress = generation;
    }

    // we may return a different code if the solver was cancelled
    if (progress.cancelled) {
        return S_FALSE;
    }

    // the solver did all it could, but it has not found a better solution, that is already known - in this case, always return S_FALSE
    if (preSolveMetric < mBest_Metric) {
        return S_FALSE;
    }

    // a better solution has been found
    return S_OK;
}

To export the solver out of the library, we need to first create the descriptor. This solver is certainly not specialized to any model, so we set the corresponding attributes to false/zero/nullptr:

namespace example_solver {
    const scgms::TSolver_Descriptor descriptor = {
        id,
        L"Example randomizing solver",
        false,
        0,
        nullptr
    };
}

As a final step, let us define, implement and export the needed library interface:

const std::array<scgms::TSolver_Descriptor, 1> solver_descriptions = { example_solver::descriptor };

HRESULT IfaceCalling do_get_solver_descriptors(scgms::TSolver_Descriptor **begin, scgms::TSolver_Descriptor **end) {
    return do_get_descriptors(solver_descriptions, begin, end);
}

HRESULT IfaceCalling do_solve_generic(const GUID *solver_id, solver::TSolver_Setup *setup, solver::TSolver_Progress *progress) {
    if (*solver_id == example_solver::descriptor.id) {
        CExample_Solver solver(*setup);
        return solver.Solve(*progress);
    }

    return E_NOTIMPL;
}