To Change the PMG Problem Configuration

1 1 Selecting A Problem Configuration 1.1 Prior to simulation To change the PMG problem configuration (i.e., the graph structure and the target and agent parameters) before the simulation, use the Problem Configuration Toolbox 1 shown in Fig. 5. Step 1 This toolbox should be used only at the very beginning before simulating anything. So the very first step is to do a clean refresh to the web page. Step 2 Three ways to use this toolbox are as follows: 1. To generate a random graph with 3 Agents, use the “Refresh” button (push repeatedly to re-generate). 2. To select one of the default problem configurations, use the drop-down menu. 3. To modify the existing setup, use the “Start Modifying” button. If one of the first two options above were used, there are no additional steps required. Hence can proceed to simulate the chosen problem configuration. Otherwise, (i.e., if the third option above was used) follow the steps below. Figure 5: Problem Configuration Toolbox 1. Step 3 Now, the Problem Configuration Toolbox 2 shown in Fig. 6 should appear on the web page. This can be used to 1. Add or Remove Targets and Agents. 2. Customize each of their parameters. 3. Move target locations and select graph edges (using the mouse pointer). Once finished, click the “Finish” button in the Problem Configuration Toolbox 1. Then can proceed to simulate the customized problem configuration. 2 Figure 6: Problem Configuration Toolbox 2. 1.2 While simulating (in the “Play” mode) To change problem configuration parameters (only the parameters related to targets and agents) while the simulation is executed in the “Play” mode, use Problem Configuration Toolbox 3 shown in Fig. 7 found at the bottom of the web page. Agents will directly react to these changes (no need to use the “Update” button). Figure 7: Problem Configuration Toolbox 3. 3 2 Executing Different Simulation Modes 2.1 “Play” Mode To see how agents behave in real-time (under a fixed Threshold Control Policy (TCP) or any other Receding Horizon Control (RHC) strategy), use the Run Simulations menu options shown in Figure 8. In there: 1. “Play” button starts the simulation. Then, it will turn into a “Pause” button which can be used to halt the simulation temporarily. 2. “Reset” button rests everything to the initial condition. In this menu (Fig. 8) some generic features available are: 1. “Frame Rate” slider can be used to see the simulation at a faster rate. 2. “Time” and “Cost” values will display the current simulation time and the cost. 3. “∆t” is the resolution of the simulation and “T ” is the simulation period, both of which can be edited using the respective text fields. Figure 8: Run Simulations Toolbox: For “Play” Mode. 2.2 “Fast Forward” Mode To see the final cost value and the final state of the system directly (under a fixed TCP or any other RHC strategy), use the Run Simulations menu options shown in Figure 9. 1. Use the “Fast Forward” button to run the simulation over the entire period [0, T ] at once and get the final cost and agent, target state results. 2. Use the “Generate Trajectory Plots” button (See Fig. 10) to further examine the agent and target state trajectories. The remaining options in this menu (Fig. 9) have the same functionalities as described before. 4 Figure 9: Run Simulations Toolbox: For “Fast Forward” Mode. Figure 10: Run Simulations Toolbox: The Button for Generating Trajectory Plots. 2.3 “Iterative Update” Mode To see the variation of the final cost value and the final state of the system when the threshold control policy is updated iteratively via Infinitesimal Perturbation Analysis (IPA) based gradient, the Run Simulations menu options shown in Figure 9 can be used. Figure 11: Run Simulations Toolbox: For “Iterative update” Mode. Note that this menu is applicable only when simulating a TCP. Therefore, make sure the use of RHC solutions is disabled (use the drop-down menu in the RHC toolbox shown in Fig. ??). Main steps involved in simulating iterative TCP updates are as follows: 1. “IPA-Play” button will initiate executing gradient updates to the current thresholds. Once clicked, the final cost and the final state after each iteration will be displayed continuously. A plot as shown in Fig 12 will appear - which will also get updated automatically with iterations go on. 2. “Θ-Refresh” can be used to execute a random update step to the threshold 5 control policy Θ. This may be used to perturb the gradient-based TCP update process - if needed. 3. “IPA-Next” button can be used to execute one update step at a time. Click this button multiple times to closely observe how each IPA gradient step affects the cost, state and TCP Θ. 4. “Stop” button will stop the iterative update process. 5. “Reset” button will reset the state of the system to its initial state (will not change the TCP Θ). Figure 12: Variation of final cost with iterative TCP updates. There are a few more options that can be customized related to this iterative update mode. Those are found in both in the Run Simulations menu shown in Fig. 11 and in the Boosting Menu shown in Fig. 13. Their purposes are as follows: Figure 13: Boosting Menu Toolbox: For “Iterative update” Mode. 6 1. “Step sizes” drop-down (in Fig. 13) can be used to change the step size method (between “Constant”, “Diminishing” and “Square-Summable”) used in gradient updates. 2. “Step Size” slider and the m“Multi-plier” slider (in Fig. 11) can be used to change the step size of the gradient update process. 3. “K” (in Fig. 11) value is the total number of update steps to be executed. 4. “∆t” and “T ” (in Fig. 11) have the same functionality as before (i.e., the time resolution and the total simulation period - per iteration). 5. “Best cost found so far” option (in Fig. 13) shows the best cost observed so far and the “Rollback!” button can be used to make the current threshold control policy the best one observed so far. 6. “Randomly perturb thresholds” sub menu (in Fig. 13) is another way to randomly perturb the TCP Θ - but now with an adjustable magnitude. 7 3 Executing different solution techniques In this section, it is assumed that the interested problem configuration is already selected (following the instructions given in Section 1). Also, it is assumed that the user requirement is to obtain terminal cost and state values after executing a simulation in “Fast Forward” mode (Described in Section 2.2) - under the interested solution technique (IPA-TCP, Greedy-IPA-TCP or any other RHC method). 3.1 IPA-TCP solution [4] The minimal set of steps are as follows: 1. Clean refresh the web page. 2. Select the interested problem configuration (following the instructions in Section 1). 3. Make sure the RHC methods are disabled using the drop-down menu in the RHC Toolbox shown in Fig. 14. 4. Make sure the Boosting methods are disabled using the drop-down menu in the Boosting Toolbox shown in Fig. 15. 5. Click the “IPA-Play” button in Run Simulations menu shown in 11 - to begin the IPA based gradient descent. And wait till the cost value goes down over the iterations (follow the graph - see Fig. 12). 6. Click the ”Stop” button and then the ”Reset” button both in the Run Simulations menu shown in 11. This will terminate the gradient descent and reset the system state. 7. Click the “Rollback!” button in the Boosting Menu shown in Fig. 13 to select the best TCP Θ observed during the gradient descent process. 8. Click the “Fast Forward” button in the Run Simulations menu shown in Fig. 9 to obtain the terminal cost and state of the system when the chosen TCP Θ is used. 9. Click the “Generate Trajectory Plots” button shown in 10 to see the state variation throughout the interested time period. Some more options that can be customized related to the IPA based gradient descent process (e.g., the step size) have already been discussed in Section 2.3. 8 Figure 14: Receding Horizon Control (RHC) Toolbox. Figure 15: Boosting Menu. 3.2 Greedy-IPA-TCP solution [2] This method needs a few more steps apart from the steps given above. The minimal set of steps are as follows: (New steps are Steps 5-8, steps 9-11 are optional) 1. Clean refresh the web page. 2. Select the interested problem configuration (following the instructions in Section 1). 3. Make sure the RHC methods are disabled using the drop-down menu in the RHC Toolbox shown in Fig. 14. 4. Make sure the Boosting methods are disabled using the drop-down menu in the Boosting Toolbox shown in Fig. 15. 5. Click the “Cluster !” button in the Greedy Threshold Control menu shown in Fig. 16 to cluster the targets among the agents. 6. Click the ”Search !” button in the Greedy Threshold Control menu shown in Fig. 16 to search for cyclic trajectories on each sub-graph for each agent to traverse. 9 7. Click the ’‘Adjust !” button in the Greedy Threshold Control menu shown in Fig. 16 to trade targets among agents so as to fine-tune the trajectories generated in the previous step. 8. Click the ”Apply !” button in the Greedy Threshold Control menu shown in Fig. 16 to convert the generated trajectories into a TCP Θ(0) . 9. Click the “IPA-Play” button in Run Simulations menu shown in 11 - to begin the IPA based gradient descent. And wait till the cost value goes down over the iterations (follow the graph - see Fig. 12). 10. Click the ”Stop” button and then the ”Reset” button both in the Run Simulations menu shown in 11. This will terminate the gradient descent and reset the system state. 11. Click the “Rollback!” button in the Boosting Menu shown in Fig. 13 to select the best TCP Θ observed during the gradient descent process. 12. Click the “Fast Forward” button in the Run Simulations menu shown in Fig. 9 to obtain the terminal cost and state of the system when the chosen TCP Θ is used. 13. Click the “Generate Trajectory Plots” button shown in 10 to see the state variation throughout the interested time period. Figure 16: Greedy TCP Construction Menu. The used graph clustering method can be customized using the Spectral Graph Clustering Parameters Menu shown in Fig. 17. Usage of that menu, as well as the remaining options on the Greedy TCP construction menu (Fig. 16), is intuitive (see [1] for more details). 10 Figure 17: Spectral Graph Clustering Parameters Menu. 3.3 RHC solution [3] Compared to the previous two solutions, simulating an RHC solution is more straightforward. The required minimal steps are as follows: (only new step in Step 3) 1. Clean refresh the web page. 2. Select the interested problem configuration (following the instructions in Section 1). 3. Select the interested RHC method from the drop-down menu in the RHC Toolbox shown in Fig. 14. 4. Click the “Fast Forward” button in the Run Simulations menu shown in Fig. 9 to obtain the terminal cost and state of the system when the chosen RHC solution is used at each agent. 5. Click the “Generate Trajectory Plots” button shown in 10 to see the state variation throughout the interested time period. If the interested RHC method is “Event-Driven RHC” or the “Event-Driven RHCα,” the “Planning Horizon” slider in the RHC Menu (shown in Fig. 14) can be used to try different planning horizon values. The “Auto-Tune” button will choose the optimal planning horizon H automatically.