Note
Go to the end to download the full example code.
Training the SOM representation estimator#
RANDOM_SEED = 42
VERBOSE = True
# generate train / test data
from examples.utils.dataset import generate_train_data
X_train, _ = generate_train_data(n_normal=100, n_abnormal=20, random_seed=RANDOM_SEED)
from pan import SomRepresentationEstimator
from minisom_representation import SomRepresentation
# train with derived SOM representation
estimator = SomRepresentationEstimator.with_derived_som_representation(X=X_train, random_seed=RANDOM_SEED, verbose=VERBOSE)
estimator.fit(X_train)
--------------
Based on 120 instances, the recommended SOM hyperparameters are the following:
Total node count (M): 55
Recommended sides (d1 x d2): 8 x 8
Initial neighborhood radius (sigma): 4.0
--------------
[ 0 / 20 ] 0% - ? it/s
[ 0 / 20 ] 0% - ? it/s
[ 1 / 20 ] 5% - 0:00:00 left
[ 2 / 20 ] 10% - 0:00:00 left
[ 3 / 20 ] 15% - 0:00:00 left
[ 4 / 20 ] 20% - 0:00:00 left
[ 5 / 20 ] 25% - 0:00:00 left
[ 6 / 20 ] 30% - 0:00:00 left
[ 7 / 20 ] 35% - 0:00:00 left
[ 8 / 20 ] 40% - 0:00:00 left
[ 9 / 20 ] 45% - 0:00:00 left
[ 10 / 20 ] 50% - 0:00:00 left
[ 11 / 20 ] 55% - 0:00:00 left
[ 12 / 20 ] 60% - 0:00:00 left
[ 13 / 20 ] 65% - 0:00:00 left
[ 14 / 20 ] 70% - 0:00:00 left
[ 15 / 20 ] 75% - 0:00:00 left
[ 16 / 20 ] 80% - 0:00:00 left
[ 17 / 20 ] 85% - 0:00:00 left
[ 18 / 20 ] 90% - 0:00:00 left
[ 19 / 20 ] 95% - 0:00:00 left
[ 20 / 20 ] 100% - 0:00:00 left Quantization Error: 0.0010
An SOM representation has been fitted as follows:
-------------------------------------------------------
Hyperparameters of SOM:
{'input_len': 2, 'x': 8, 'y': 8, 'sigma': np.float64(4.0), 'topology': 'rectangular', 'learning_rate': 0.5, 'decay_function': 'asymptotic_decay', 'sigma_decay_function': 'asymptotic_decay', 'neighborhood_function': 'gaussian', 'activation_distance': 'euclidean', 'random_seed': 42, 'num_iteration': 20, 'verbose': True}
Quality of SOM:
Quantization Error (QE): 0.0009728796386848607
Topographic Error (TE): 0.016666666666666666
An SOM representation estimator has been fitted as follows:
-----------------------------------------------------------------
Hyperparameters:
{'nu': 0.5, 'random_seed': 42, 'som_representation': <minisom_representation.som_representation.SomRepresentation object at 0x7fc2904b80e0>, 'verbose': True}
Learned parameters:
Offset: -0.00047117853857801643
# train with SOM representation hyperparameters specified
som_rep = SomRepresentation.with_derived_params(
X_train, sigma=3.3, topology="rectangular", learning_rate=0.25, num_iteration=44,
decay_function="asymptotic_decay", sigma_decay_function="asymptotic_decay",
neighborhood_function='gaussian', activation_distance='euclidean',
random_seed=RANDOM_SEED, verbose=VERBOSE
)
estimator = SomRepresentationEstimator.from_som_representation(som_rep, nu=.01, random_seed=RANDOM_SEED, verbose=VERBOSE)
estimator.fit(X_train)
--------------
Based on 120 instances, the recommended SOM hyperparameters are the following:
Total node count (M): 55
Recommended sides (d1 x d2): 8 x 8
Initial neighborhood radius (sigma): 4.0
--------------
[ 0 / 44 ] 0% - ? it/s
[ 0 / 44 ] 0% - ? it/s
[ 1 / 44 ] 2% - 0:00:00 left
[ 2 / 44 ] 5% - 0:00:00 left
[ 3 / 44 ] 7% - 0:00:00 left
[ 4 / 44 ] 9% - 0:00:00 left
[ 5 / 44 ] 11% - 0:00:00 left
[ 6 / 44 ] 14% - 0:00:00 left
[ 7 / 44 ] 16% - 0:00:00 left
[ 8 / 44 ] 18% - 0:00:00 left
[ 9 / 44 ] 20% - 0:00:00 left
[ 10 / 44 ] 23% - 0:00:00 left
[ 11 / 44 ] 25% - 0:00:00 left
[ 12 / 44 ] 27% - 0:00:00 left
[ 13 / 44 ] 30% - 0:00:00 left
[ 14 / 44 ] 32% - 0:00:00 left
[ 15 / 44 ] 34% - 0:00:00 left
[ 16 / 44 ] 36% - 0:00:00 left
[ 17 / 44 ] 39% - 0:00:00 left
[ 18 / 44 ] 41% - 0:00:00 left
[ 19 / 44 ] 43% - 0:00:00 left
[ 20 / 44 ] 45% - 0:00:00 left
[ 21 / 44 ] 48% - 0:00:00 left
[ 22 / 44 ] 50% - 0:00:00 left
[ 23 / 44 ] 52% - 0:00:00 left
[ 24 / 44 ] 55% - 0:00:00 left
[ 25 / 44 ] 57% - 0:00:00 left
[ 26 / 44 ] 59% - 0:00:00 left
[ 27 / 44 ] 61% - 0:00:00 left
[ 28 / 44 ] 64% - 0:00:00 left
[ 29 / 44 ] 66% - 0:00:00 left
[ 30 / 44 ] 68% - 0:00:00 left
[ 31 / 44 ] 70% - 0:00:00 left
[ 32 / 44 ] 73% - 0:00:00 left
[ 33 / 44 ] 75% - 0:00:00 left
[ 34 / 44 ] 77% - 0:00:00 left
[ 35 / 44 ] 80% - 0:00:00 left
[ 36 / 44 ] 82% - 0:00:00 left
[ 37 / 44 ] 84% - 0:00:00 left
[ 38 / 44 ] 86% - 0:00:00 left
[ 39 / 44 ] 89% - 0:00:00 left
[ 40 / 44 ] 91% - 0:00:00 left
[ 41 / 44 ] 93% - 0:00:00 left
[ 42 / 44 ] 95% - 0:00:00 left
[ 43 / 44 ] 98% - 0:00:00 left
[ 44 / 44 ] 100% - 0:00:00 left Quantization Error: 0.0008
An SOM representation has been fitted as follows:
-------------------------------------------------------
Hyperparameters of SOM:
{'input_len': 2, 'x': 8, 'y': 8, 'sigma': 3.3, 'topology': 'rectangular', 'learning_rate': 0.25, 'decay_function': 'asymptotic_decay', 'sigma_decay_function': 'asymptotic_decay', 'neighborhood_function': 'gaussian', 'activation_distance': 'euclidean', 'random_seed': 42, 'num_iteration': 44, 'verbose': True}
Quality of SOM:
Quantization Error (QE): 0.0008264950770934449
Topographic Error (TE): 0.016666666666666666
An SOM representation estimator has been fitted as follows:
-----------------------------------------------------------------
Hyperparameters:
{'nu': 0.01, 'random_seed': 42, 'som_representation': <minisom_representation.som_representation.SomRepresentation object at 0x7fc2a826da00>, 'verbose': True}
Learned parameters:
Offset: -0.004763263703869526
Total running time of the script: (0 minutes 2.584 seconds)