26  Testing Large Numbers of Scenarios

Credit for this solution goes to Anna Laws and Mike Allen of the PenCHORD team.

When working out the best possible configuration for a service, you may wish to try out a large number of scenarios.

Let’s return to our branching model (with the reproducibility set via sim-tools as described in chapter Chapter 13).

We have a number of parameters available to us in this model:

class g:
    patient_inter = 5
    mean_reception_time = 2
    mean_n_consult_time = 6
    mean_d_consult_time = 20
    number_of_receptionists = 1
    number_of_nurses = 1
    number_of_doctors = 2
    prob_seeing_doctor = 0.6
    sim_duration = 600
    number_of_runs = 2

We can first create a python dictionary of the possible parameter values.

Warning

Be careful - the total number of possible permutations starts to grow very rapidly when you have lots of parameters with multiple options for each!

scenarios = {
    'patient_inter': [4, 8, 12],
    'mean_reception_time': [2, 3],
    'mean_n_consult_time': [6, 10, 14],
    'mean_d_consult_time': [10, 20],
    'number_of_receptionists': [1, 2],
    'number_of_nurses': [1, 2, 3],
    'number_of_doctors': [2, 3, 4],
    'prob_seeing_doctor': [0.6, 0.8]
}
Tip

Make sure to use exactly the same naming for the dictionary keys as is used in your g class.

This is because we will reset the values of the g class for each Trial programmatically.

Tip

For a small number of possibilities, setting the variables by hand will be fine.

For a larger number, you may want to use the range function.

e.g. to get 6, 10, 14 you would do

[i for i in range(6, 15, 4)]
[6, 10, 14]

Next we use the itertools package to create every possible permutation of the scenarios.

import itertools

# Generate all scenarios:
all_scenarios_tuples = [
    x for x in itertools.product(*scenarios.values())]
# Convert list of tuples back to list of dictionaries:
all_scenarios_dicts = [
    dict(zip(scenarios.keys(), p)) for p in all_scenarios_tuples]

Let’s take a look at the first 3 scenario dictionaries.

all_scenarios_dicts[0:3]
[{'patient_inter': 4,
  'mean_reception_time': 2,
  'mean_n_consult_time': 6,
  'mean_d_consult_time': 10,
  'number_of_receptionists': 1,
  'number_of_nurses': 1,
  'number_of_doctors': 2,
  'prob_seeing_doctor': 0.6},
 {'patient_inter': 4,
  'mean_reception_time': 2,
  'mean_n_consult_time': 6,
  'mean_d_consult_time': 10,
  'number_of_receptionists': 1,
  'number_of_nurses': 1,
  'number_of_doctors': 2,
  'prob_seeing_doctor': 0.8},
 {'patient_inter': 4,
  'mean_reception_time': 2,
  'mean_n_consult_time': 6,
  'mean_d_consult_time': 10,
  'number_of_receptionists': 1,
  'number_of_nurses': 1,
  'number_of_doctors': 3,
  'prob_seeing_doctor': 0.6}]

We can see that all that has changed is the probability of seeing a doctor (the last key-value pair in each dictionary).

How many scenarios have we created?

len(all_scenarios_dicts)
1296

Now let’s update our g class. We’ll just modify it to add in a space to add a scenario name.

class g:
    patient_inter = 5
    mean_reception_time = 2
    mean_n_consult_time = 6
    mean_d_consult_time = 20
    number_of_receptionists = 1
    number_of_nurses = 1
    number_of_doctors = 2
    prob_seeing_doctor = 0.6
    sim_duration = 600
    number_of_runs = 2
    scenario_name = 0 ## New

Let’s now create all of the scenario objects.

results = []

for index, scenario_to_run in enumerate(all_scenarios_dicts):
    g.scenario_name = index

    # Overwrite defaults from the passed dictionary

    for key in scenario_to_run:
        setattr(g, key, scenario_to_run[key])

    my_trial = Trial()

    # Call the run_trial method of our Trial object
    results.append(my_trial.run_trial())

pd.concat(results).groupby("scenario").mean().head(20)
average_inter_arrival num_recep num_nurses num_doctors average_reception_time average_nurse_time average_doctor_time prob_need_doctor Arrivals Mean Q Time Recep Mean Q Time Nurse Mean Q Time Doctor
scenario
0.0 4.0 1.0 1.0 2.0 2.0 6.0 10.0 0.6 144.0 1.099929 133.505891 0.676479
1.0 4.0 1.0 1.0 2.0 2.0 6.0 10.0 0.8 144.0 1.099929 105.505442 1.263302
2.0 4.0 1.0 1.0 3.0 2.0 6.0 10.0 0.6 144.0 1.099929 129.216818 0.014072
3.0 4.0 1.0 1.0 3.0 2.0 6.0 10.0 0.8 144.0 1.099929 107.602443 0.147246
4.0 4.0 1.0 1.0 4.0 2.0 6.0 10.0 0.6 144.0 1.099929 129.216818 0.000000
5.0 4.0 1.0 1.0 4.0 2.0 6.0 10.0 0.8 144.0 1.099929 95.026033 0.032971
6.0 4.0 1.0 2.0 2.0 2.0 6.0 10.0 0.6 144.0 1.099929 5.215946 2.477255
7.0 4.0 1.0 2.0 2.0 2.0 6.0 10.0 0.8 144.0 1.099929 7.497472 1.842969
8.0 4.0 1.0 2.0 3.0 2.0 6.0 10.0 0.6 144.0 1.099929 4.266968 0.592513
9.0 4.0 1.0 2.0 3.0 2.0 6.0 10.0 0.8 144.0 1.099929 9.260408 0.089449
10.0 4.0 1.0 2.0 4.0 2.0 6.0 10.0 0.6 144.0 1.099929 5.405535 0.063385
11.0 4.0 1.0 2.0 4.0 2.0 6.0 10.0 0.8 144.0 1.099929 8.370422 0.015803
12.0 4.0 1.0 3.0 2.0 2.0 6.0 10.0 0.6 144.0 1.099929 0.833707 1.786491
13.0 4.0 1.0 3.0 2.0 2.0 6.0 10.0 0.8 144.0 1.099929 0.871273 2.689420
14.0 4.0 1.0 3.0 3.0 2.0 6.0 10.0 0.6 144.0 1.099929 1.155603 0.142991
15.0 4.0 1.0 3.0 3.0 2.0 6.0 10.0 0.8 144.0 1.099929 0.697876 0.615490
16.0 4.0 1.0 3.0 4.0 2.0 6.0 10.0 0.6 144.0 1.099929 0.925463 0.009072
17.0 4.0 1.0 3.0 4.0 2.0 6.0 10.0 0.8 144.0 1.099929 0.540083 0.186033
18.0 4.0 2.0 1.0 2.0 2.0 6.0 10.0 0.6 144.0 0.091630 134.540431 0.676479
19.0 4.0 2.0 1.0 2.0 2.0 6.0 10.0 0.8 144.0 0.091630 106.515917 1.263302

Finally the following will give you a nice dictionary of all of your scenarios.

pd.DataFrame.from_dict(all_scenarios_dicts)
patient_inter mean_reception_time mean_n_consult_time mean_d_consult_time number_of_receptionists number_of_nurses number_of_doctors prob_seeing_doctor
0 4 2 6 10 1 1 2 0.6
1 4 2 6 10 1 1 2 0.8
2 4 2 6 10 1 1 3 0.6
3 4 2 6 10 1 1 3 0.8
4 4 2 6 10 1 1 4 0.6
... ... ... ... ... ... ... ... ...
1291 12 3 14 20 2 3 2 0.8
1292 12 3 14 20 2 3 3 0.6
1293 12 3 14 20 2 3 3 0.8
1294 12 3 14 20 2 3 4 0.6
1295 12 3 14 20 2 3 4 0.8

1296 rows × 8 columns