3 @brief Phase oscillatory network for patten recognition based on modified Kuramoto model. 4 @details Implementation based on paper @cite article::nnet::syncpr::1. 6 @authors Andrei Novikov (pyclustering@yandex.ru) 8 @copyright GNU Public License 10 @cond GNU_PUBLIC_LICENSE 11 PyClustering is free software: you can redistribute it and/or modify 12 it under the terms of the GNU General Public License as published by 13 the Free Software Foundation, either version 3 of the License, or 14 (at your option) any later version. 16 PyClustering is distributed in the hope that it will be useful, 17 but WITHOUT ANY WARRANTY; without even the implied warranty of 18 MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the 19 GNU General Public License for more details. 21 You should have received a copy of the GNU General Public License 22 along with this program. If not, see <http://www.gnu.org/licenses/>. 35 import pyclustering.core.syncpr_wrapper
as wrapper
37 from pyclustering.core.wrapper
import ccore_library
41 except Exception
as error_instance:
42 warnings.warn(
"Impossible to import PIL (please, install 'PIL'), pyclustering's visualization " 43 "functionality is partially not available (details: '%s')." % str(error_instance))
46 import matplotlib.pyplot
as plt
47 import matplotlib.animation
as animation
48 except Exception
as error_instance:
49 warnings.warn(
"Impossible to import matplotlib (please, install 'matplotlib'), pyclustering's visualization " 50 "functionality is not available (details: '%s')." % str(error_instance))
55 @brief Represents output dynamic of syncpr (Sync for Pattern Recognition). 61 @brief Constructor of syncpr dynamic. 63 @param[in] phase (list): Dynamic of oscillators on each step of simulation. If ccore pointer is specified than it can be ignored. 64 @param[in] time (list): Simulation time. 65 @param[in] ccore (ctypes.pointer): Pointer to CCORE sync_dynamic instance in memory. 68 super().
__init__(phase, time, ccore);
73 @brief Visualizer of output dynamic of syncpr network (Sync for Pattern Recognition). 78 def show_pattern(syncpr_output_dynamic, image_height, image_width):
80 @brief Displays evolution of phase oscillators as set of patterns where the last one means final result of recognition. 82 @param[in] syncpr_output_dynamic (syncpr_dynamic): Output dynamic of a syncpr network. 83 @param[in] image_height (uint): Height of the pattern (image_height * image_width should be equal to number of oscillators). 84 @param[in] image_width (uint): Width of the pattern. 87 number_pictures = len(syncpr_output_dynamic);
88 iteration_math_step = 1.0;
89 if (number_pictures > 50):
90 iteration_math_step = number_pictures / 50.0;
93 number_cols = int(numpy.ceil(number_pictures ** 0.5));
94 number_rows = int(numpy.ceil(number_pictures / number_cols));
97 double_indexer =
True;
98 if ( (number_cols == 1)
or (number_rows == 1) ):
100 double_indexer =
False;
102 (_, axarr) = plt.subplots(number_rows, number_cols);
104 if (number_pictures > 1):
105 plt.setp([ax
for ax
in axarr], visible =
False);
107 iteration_display = 0.0;
108 for iteration
in range(len(syncpr_output_dynamic)):
109 if (iteration >= iteration_display):
110 iteration_display += iteration_math_step;
113 if (number_pictures > 1):
114 ax_handle = axarr[real_index];
116 syncpr_visualizer.__show_pattern(ax_handle, syncpr_output_dynamic, image_height, image_width, iteration);
118 if (double_indexer
is True):
119 real_index = real_index[0], real_index[1] + 1;
120 if (real_index[1] >= number_cols):
121 real_index = real_index[0] + 1, 0;
131 @brief Shows animation of pattern recognition process that has been preformed by the oscillatory network. 133 @param[in] syncpr_output_dynamic (syncpr_dynamic): Output dynamic of a syncpr network. 134 @param[in] image_height (uint): Height of the pattern (image_height * image_width should be equal to number of oscillators). 135 @param[in] image_width (uint): Width of the pattern. 136 @param[in] animation_velocity (uint): Interval between frames in milliseconds. 137 @param[in] title (string): Title of the animation that is displayed on a figure if it is specified. 138 @param[in] save_movie (string): If it is specified then animation will be stored to file that is specified in this parameter. 141 figure = plt.figure();
144 return frame_generation(0);
146 def frame_generation(index_dynamic):
149 if (title
is not None):
150 figure.suptitle(title, fontsize = 26, fontweight =
'bold')
152 ax1 = figure.add_subplot(121, projection=
'polar');
153 ax2 = figure.add_subplot(122);
155 dynamic = syncpr_output_dynamic.output[index_dynamic];
157 artist1, = ax1.plot(dynamic, [1.0] * len(dynamic), marker =
'o', color =
'blue', ls =
'');
158 artist2 = syncpr_visualizer.__show_pattern(ax2, syncpr_output_dynamic, image_height, image_width, index_dynamic);
160 return [ artist1, artist2 ];
162 cluster_animation = animation.FuncAnimation(figure, frame_generation, len(syncpr_output_dynamic), interval = animation_velocity, init_func = init_frame, repeat_delay = 5000);
164 if (save_movie
is not None):
168 cluster_animation.save(save_movie, writer =
'ffmpeg', fps = 15, bitrate = 1500);
174 def __show_pattern(ax_handle, syncpr_output_dynamic, image_height, image_width, iteration):
176 @brief Draws pattern on specified ax. 178 @param[in] ax_handle (Axis): Axis where pattern should be drawn. 179 @param[in] syncpr_output_dynamic (syncpr_dynamic): Output dynamic of a syncpr network. 180 @param[in] image_height (uint): Height of the pattern (image_height * image_width should be equal to number of oscillators). 181 @param[in] image_width (uint): Width of the pattern. 182 @param[in] iteration (uint): Simulation iteration that should be used for extracting pattern. 184 @return (matplotlib.artist) Artist (pattern) that is rendered in the canvas. 188 current_dynamic = syncpr_output_dynamic.output[iteration];
189 stage_picture = [(255, 255, 255)] * (image_height * image_width);
190 for index_phase
in range(len(current_dynamic)):
191 phase = current_dynamic[index_phase];
193 pixel_color = math.floor( phase * (255 / (2 * math.pi)) );
194 stage_picture[index_phase] = (pixel_color, pixel_color, pixel_color);
196 stage = numpy.array(stage_picture, numpy.uint8);
197 stage = numpy.reshape(stage, (image_height, image_width) + ((3),));
199 image_cluster = Image.fromarray(stage);
201 artist = ax_handle.imshow(image_cluster, interpolation =
'none');
202 plt.setp(ax_handle, visible =
True);
204 ax_handle.xaxis.set_ticklabels([]);
205 ax_handle.yaxis.set_ticklabels([]);
206 ax_handle.xaxis.set_ticks_position(
'none');
207 ax_handle.yaxis.set_ticks_position(
'none');
214 @brief Model of phase oscillatory network for pattern recognition that is based on the Kuramoto model. 215 @details The model uses second-order and third-order modes of the Fourier components. 217 CCORE option can be used to use the pyclustering core - C/C++ shared library for processing that significantly increases performance. 221 # Network size should be equal to size of pattern for learning. 222 net = syncpr(size_network, 0.3, 0.3); 224 # Train network using list of patterns (input images). 225 net.train(image_samples); 227 # Recognize image using 10 steps during 10 seconds of simulation. 228 sync_output_dynamic = net.simulate(10, 10, pattern, solve_type.RK4, True); 230 # Display output dynamic. 231 syncpr_visualizer.show_output_dynamic(sync_output_dynamic); 233 # Display evolution of recognition of the pattern. 234 syncpr_visualizer.show_pattern(sync_output_dynamic, image_height, image_width); 240 def __init__(self, num_osc, increase_strength1, increase_strength2, ccore = True):
242 @brief Constructor of oscillatory network for pattern recognition based on Kuramoto model. 244 @param[in] num_osc (uint): Number of oscillators in the network. 245 @param[in] increase_strength1 (double): Parameter for increasing strength of the second term of the Fourier component. 246 @param[in] increase_strength2 (double): Parameter for increasing strength of the third term of the Fourier component. 247 @param[in] ccore (bool): If True simulation is performed by CCORE library (C++ implementation of pyclustering). 251 if ( (ccore
is True)
and ccore_library.workable() ):
257 self.
_coupling = [ [0.0
for i
in range(num_osc)]
for j
in range(num_osc) ];
259 super().
__init__(num_osc, 1, 0, conn_type.ALL_TO_ALL, conn_represent.MATRIX, initial_type.RANDOM_GAUSSIAN, ccore)
264 @brief Default destructor of syncpr. 275 @brief Returns size of the network. 287 @brief Trains syncpr network using Hebbian rule for adjusting strength of connections between oscillators during training. 289 @param[in] samples (list): list of patterns where each pattern is represented by list of features that are equal to [-1; 1]. 294 for pattern
in samples:
301 number_samples = len(samples);
303 for i
in range(length):
304 for j
in range(i + 1, len(self), 1):
307 for p
in range(number_samples):
308 value1 = samples[p][i];
309 value2 = samples[p][j];
317 def simulate(self, steps, time, pattern, solution = solve_type.RK4, collect_dynamic = True):
319 @brief Performs static simulation of syncpr oscillatory network. 320 @details In other words network performs pattern recognition during simulation. 322 @param[in] steps (uint): Number steps of simulations during simulation. 323 @param[in] time (double): Time of simulation. 324 @param[in] pattern (list): Pattern for recognition represented by list of features that are equal to [-1; 1]. 325 @param[in] solution (solve_type): Type of solver that should be used for simulation. 326 @param[in] collect_dynamic (bool): If True - returns whole dynamic of oscillatory network, otherwise returns only last values of dynamics. 328 @return (list) Dynamic of oscillatory network. If argument 'collect_dynamic' = True, than return dynamic for the whole simulation time, 329 otherwise returns only last values (last step of simulation) of dynamic. 331 @see simulate_dynamic() 332 @see simulate_static() 336 return self.
simulate_static(steps, time, pattern, solution, collect_dynamic);
339 def simulate_dynamic(self, pattern, order = 0.998, solution = solve_type.RK4, collect_dynamic = False, step = 0.1, int_step = 0.01, threshold_changes = 0.0000001):
341 @brief Performs dynamic simulation of the network until stop condition is not reached. 342 @details In other words network performs pattern recognition during simulation. 343 Stop condition is defined by input argument 'order' that represents memory order, but 344 process of simulation can be stopped if convergance rate is low whose threshold is defined 345 by the argument 'threshold_changes'. 347 @param[in] pattern (list): Pattern for recognition represented by list of features that are equal to [-1; 1]. 348 @param[in] order (double): Order of process synchronization, distributed 0..1. 349 @param[in] solution (solve_type): Type of solution. 350 @param[in] collect_dynamic (bool): If True - returns whole dynamic of oscillatory network, otherwise returns only last values of dynamics. 351 @param[in] step (double): Time step of one iteration of simulation. 352 @param[in] int_step (double): Integration step, should be less than step. 353 @param[in] threshold_changes (double): Additional stop condition that helps prevent infinite simulation, defines limit of changes of oscillators between current and previous steps. 355 @return (list) Dynamic of oscillatory network. If argument 'collect_dynamic' = True, than return dynamic for the whole simulation time, 356 otherwise returns only last values (last step of simulation) of dynamic. 359 @see simulate_static() 366 ccore_instance_dynamic = wrapper.syncpr_simulate_dynamic(self.
_ccore_network_pointer, pattern, order, solution, collect_dynamic, step);
369 for i
in range(0, len(pattern), 1):
370 if (pattern[i] > 0.0):
373 self.
_phases[i] = math.pi / 2.0;
385 if (collect_dynamic ==
True):
386 dyn_phase.append(self.
_phases);
390 while (current_order < order):
395 time_counter += step;
398 if (collect_dynamic ==
True):
399 dyn_phase.append(self.
_phases);
400 dyn_time.append(time_counter);
403 previous_order = current_order;
407 if (abs(current_order - previous_order) < threshold_changes):
410 if (collect_dynamic !=
True):
411 dyn_phase.append(self.
_phases);
412 dyn_time.append(time_counter);
415 return output_sync_dynamic;
418 def simulate_static(self, steps, time, pattern, solution = solve_type.FAST, collect_dynamic = False):
420 @brief Performs static simulation of syncpr oscillatory network. 421 @details In other words network performs pattern recognition during simulation. 423 @param[in] steps (uint): Number steps of simulations during simulation. 424 @param[in] time (double): Time of simulation. 425 @param[in] pattern (list): Pattern for recognition represented by list of features that are equal to [-1; 1]. 426 @param[in] solution (solve_type): Type of solution. 427 @param[in] collect_dynamic (bool): If True - returns whole dynamic of oscillatory network, otherwise returns only last values of dynamics. 429 @return (list) Dynamic of oscillatory network. If argument 'collect_dynamic' = True, than return dynamic for the whole simulation time, 430 otherwise returns only last values (last step of simulation) of dynamic. 433 @see simulate_dynamic() 440 ccore_instance_dynamic = wrapper.syncpr_simulate_static(self.
_ccore_network_pointer, steps, time, pattern, solution, collect_dynamic);
443 for i
in range(0, len(pattern), 1):
444 if (pattern[i] > 0.0):
447 self.
_phases[i] = math.pi / 2.0;
449 return super().
simulate_static(steps, time, solution, collect_dynamic);
454 @brief Calculates function of the memorized pattern. 455 @details Throws exception if length of pattern is not equal to size of the network or if it consists feature with value that are not equal to [-1; 1]. 457 @param[in] pattern (list): Pattern for recognition represented by list of features that are equal to [-1; 1]. 459 @return (double) Order of memory for the specified pattern. 472 def __calculate_memory_order(self, pattern):
474 @brief Calculates function of the memorized pattern without any pattern validation. 476 @param[in] pattern (list): Pattern for recognition represented by list of features that are equal to [-1; 1]. 478 @return (double) Order of memory for the specified pattern. 483 for index
in range(len(self)):
484 memory_order += pattern[index] * cmath.exp( 1j * self.
_phases[index] );
486 memory_order /= len(self);
487 return abs(memory_order);
490 def _phase_kuramoto(self, teta, t, argv):
492 @brief Returns result of phase calculation for specified oscillator in the network. 494 @param[in] teta (double): Phase of the oscillator that is differentiated. 495 @param[in] t (double): Current time of simulation. 496 @param[in] argv (tuple): Index of the oscillator in the list. 498 @return (double) New phase for specified oscillator (don't assign it here). 509 phase_delta = self.
_phases[k] - teta;
511 phase += self.
_coupling[index][k] * math.sin(phase_delta);
516 term += (term1 - term2);
518 return ( phase + term / len(self) );
521 def __validate_pattern(self, pattern):
523 @brief Validates pattern. 524 @details Throws exception if length of pattern is not equal to size of the network or if it consists feature with value that are not equal to [-1; 1]. 526 @param[in] pattern (list): Pattern for recognition represented by list of features that are equal to [-1; 1]. 529 if (len(pattern) != len(self)):
530 raise NameError(
'syncpr: length of the pattern (' + len(pattern) +
') should be equal to size of the network');
532 for feature
in pattern:
533 if ( (feature != -1.0)
and (feature != 1.0) ):
534 raise NameError(
'syncpr: patten feature (' + feature +
') should be distributed in [-1; 1]');
def _calculate_phases(self, solution, t, step, int_step)
Calculates new phases for oscillators in the network in line with current step.
def __init__(self, num_osc, increase_strength1, increase_strength2, ccore=True)
Constructor of oscillatory network for pattern recognition based on Kuramoto model.
Represents output dynamic of Sync.
def simulate_static(self, steps, time, solution=solve_type.FAST, collect_dynamic=False)
Performs static simulation of oscillatory network.
def animate_pattern_recognition(syncpr_output_dynamic, image_height, image_width, animation_velocity=75, title=None, save_movie=None)
Shows animation of pattern recognition process that has been preformed by the oscillatory network...
def __init__(self, phase, time, ccore)
Constructor of syncpr dynamic.
def __validate_pattern(self, pattern)
Validates pattern.
def __calculate_memory_order(self, pattern)
Calculates function of the memorized pattern without any pattern validation.
Neural Network: Oscillatory Neural Network based on Kuramoto model.
Model of phase oscillatory network for pattern recognition that is based on the Kuramoto model...
Represents output dynamic of syncpr (Sync for Pattern Recognition).
def simulate(self, steps, time, pattern, solution=solve_type.RK4, collect_dynamic=True)
Performs static simulation of syncpr oscillatory network.
def memory_order(self, pattern)
Calculates function of the memorized pattern.
def __len__(self)
Returns size of the network.
def train(self, samples)
Trains syncpr network using Hebbian rule for adjusting strength of connections between oscillators du...
Visualizer of output dynamic of sync network (Sync).
def simulate_static(self, steps, time, pattern, solution=solve_type.FAST, collect_dynamic=False)
Performs static simulation of syncpr oscillatory network.
Visualizer of output dynamic of syncpr network (Sync for Pattern Recognition).
def simulate_dynamic(self, pattern, order=0.998, solution=solve_type.RK4, collect_dynamic=False, step=0.1, int_step=0.01, threshold_changes=0.0000001)
Performs dynamic simulation of the network until stop condition is not reached.
def show_pattern(syncpr_output_dynamic, image_height, image_width)
Displays evolution of phase oscillators as set of patterns where the last one means final result of r...
def __del__(self)
Default destructor of syncpr.
Neural and oscillatory network module.
Model of oscillatory network that is based on the Kuramoto model of synchronization.