.. DO NOT EDIT. .. THIS FILE WAS AUTOMATICALLY GENERATED BY SPHINX-GALLERY. .. TO MAKE CHANGES, EDIT THE SOURCE PYTHON FILE: .. "auto_examples/demonstrate_02_train_pan_estimator.py" .. LINE NUMBERS ARE GIVEN BELOW. .. only:: html .. note:: :class: sphx-glr-download-link-note :ref:`Go to the end ` to download the full example code. .. rst-class:: sphx-glr-example-title .. _sphx_glr_auto_examples_demonstrate_02_train_pan_estimator.py: =============================================================== Training the PAN estimator =============================================================== .. GENERATED FROM PYTHON SOURCE LINES 8-11 .. code-block:: Python RANDOM_SEED = 42 VERBOSE = True .. GENERATED FROM PYTHON SOURCE LINES 12-18 .. code-block:: Python # generate train / test data from examples.utils.dataset import generate_train_data, generate_test_data X_train, y_train = generate_train_data(n_normal=100, n_abnormal=20, random_seed=RANDOM_SEED) X_test, y_test = generate_test_data(n_normal=100, n_abnormal=20, random_seed=RANDOM_SEED*2) .. GENERATED FROM PYTHON SOURCE LINES 19-23 .. code-block:: Python from pan import ParallelAnomalousNudge, SomRepresentationEstimator from minisom_representation import SomRepresentation from sklearn.preprocessing import StandardScaler .. GENERATED FROM PYTHON SOURCE LINES 24-29 .. code-block:: Python # train with default settings aka. derived SOM representation estimators model = ParallelAnomalousNudge.with_derived_estimators(X_train, y_train, random_seed=RANDOM_SEED, verbose=VERBOSE) model.fit(X_train, y_train) .. rst-class:: sphx-glr-script-out .. code-block:: none -------------- Based on 100 instances, the recommended SOM hyperparameters are the following: Total node count (M): 50 Recommended sides (d1 x d2): 7 x 8 Initial neighborhood radius (sigma): 4.0 -------------- Quantization Error: 0.3830 An SOM representation has been fitted as follows: ------------------------------------------------------- Hyperparameters of SOM: {'input_len': 2, 'x': 7, 'y': 8, 'sigma': np.float64(4.0), 'topology': 'rectangular', 'learning_rate': 0.5, 'decay_function': 'asymptotic_decay', 'sigma_decay_function': 'asymptotic_decay', 'neighborhood_function': 'gaussian', 'activation_distance': 'euclidean', 'random_seed': 42, 'num_iteration': 20, 'verbose': True} Quality of SOM: Quantization Error (QE): 0.38299310020419547 Topographic Error (TE): 0.0 An SOM representation estimator has been fitted as follows: ----------------------------------------------------------------- Hyperparameters: {'nu': 0.5, 'random_seed': 42, 'som_representation': , 'verbose': True} Learned parameters: Offset: -0.18478122783640794 -------------- Based on 20 instances, the recommended SOM hyperparameters are the following: Total node count (M): 23 Recommended sides (d1 x d2): 5 x 6 Initial neighborhood radius (sigma): 3.0 -------------- Quantization Error: 0.3288 An SOM representation has been fitted as follows: ------------------------------------------------------- Hyperparameters of SOM: {'input_len': 2, 'x': 5, 'y': 6, 'sigma': np.float64(3.0), 'topology': 'rectangular', 'learning_rate': 0.5, 'decay_function': 'asymptotic_decay', 'sigma_decay_function': 'asymptotic_decay', 'neighborhood_function': 'gaussian', 'activation_distance': 'euclidean', 'random_seed': 42, 'num_iteration': 20, 'verbose': True} Quality of SOM: Quantization Error (QE): 0.3287636715642034 Topographic Error (TE): 0.0 An SOM representation estimator has been fitted as follows: ----------------------------------------------------------------- Hyperparameters: {'nu': 0.5, 'random_seed': 42, 'som_representation': , 'verbose': True} Learned parameters: Offset: -0.17774894405431318 Quantization Error: 0.3830 An SOM representation has been fitted as follows: ------------------------------------------------------- Hyperparameters of SOM: {'input_len': 2, 'x': 7, 'y': 8, 'sigma': np.float64(4.0), 'topology': 'rectangular', 'learning_rate': 0.5, 'decay_function': 'asymptotic_decay', 'sigma_decay_function': 'asymptotic_decay', 'neighborhood_function': 'gaussian', 'activation_distance': 'euclidean', 'random_seed': 42, 'num_iteration': 20, 'verbose': True} Quality of SOM: Quantization Error (QE): 0.38299310020419547 Topographic Error (TE): 0.0 An SOM representation estimator has been fitted as follows: ----------------------------------------------------------------- Hyperparameters: {'nu': 0.5, 'random_seed': 42, 'som_representation': , 'verbose': True} Learned parameters: Offset: -0.18478122783640794 Quantization Error: 0.3288 An SOM representation has been fitted as follows: ------------------------------------------------------- Hyperparameters of SOM: {'input_len': 2, 'x': 5, 'y': 6, 'sigma': np.float64(3.0), 'topology': 'rectangular', 'learning_rate': 0.5, 'decay_function': 'asymptotic_decay', 'sigma_decay_function': 'asymptotic_decay', 'neighborhood_function': 'gaussian', 'activation_distance': 'euclidean', 'random_seed': 42, 'num_iteration': 20, 'verbose': True} Quality of SOM: Quantization Error (QE): 0.3287636715642034 Topographic Error (TE): 0.0 An SOM representation estimator has been fitted as follows: ----------------------------------------------------------------- Hyperparameters: {'nu': 0.5, 'random_seed': 42, 'som_representation': , 'verbose': True} Learned parameters: Offset: -0.17774894405431318 A PAN estimator has been fitted as follows: ----------------------------------------------------------------- Hyperparameters: {'abnormal_label': 1, 'estimators': {np.int64(0): SomRepresentationEstimator(random_seed=42, som_representation=, verbose=True), np.int64(1): SomRepresentationEstimator(random_seed=42, som_representation=, verbose=True)}, 'normal_label': 0, 'nu': 0.5, 'omega': 2.0, 'random_seed': 42, 'scaler__copy': True, 'scaler__with_mean': True, 'scaler__with_std': True, 'scaler': StandardScaler(), 'verbose': True} Learned parameters: Offset: -0.18478122783640794 .. raw:: html 
ParallelAnomalousNudge(estimators={np.int64(0): SomRepresentationEstimator(random_seed=42,
                                                                               som_representation=<minisom_representation.som_representation.SomRepresentation object at 0x7fc2910ce3f0>,
                                                                               verbose=True),
                                       np.int64(1): SomRepresentationEstimator(random_seed=42,
                                                                               som_representation=<minisom_representation.som_representation.SomRepresentation object at 0x7fc2a826d250>,
                                                                               verbose=True)},
                           random_seed=42, scaler=StandardScaler(), verbose=True)
In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook.
On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.


.. GENERATED FROM PYTHON SOURCE LINES 30-39 .. code-block:: Python # train with SOM representation estimator hyperparameters specified X_train_scaled = StandardScaler().fit_transform(X_train) estimator_0 = SomRepresentationEstimator.with_derived_som_representation(X=X_train_scaled[y_train == 0], nu=.05, random_seed=RANDOM_SEED, verbose=VERBOSE) estimator_1 = SomRepresentationEstimator.with_derived_som_representation(X=X_train_scaled[y_train == 1], nu=.05, random_seed=RANDOM_SEED, verbose=VERBOSE) estimators = {0: estimator_0, 1: estimator_1} model = ParallelAnomalousNudge.from_estimators(estimators=estimators, scaler=StandardScaler(), nu=estimator_0.nu, omega=3.5, random_seed=RANDOM_SEED, verbose=VERBOSE) model.fit(X_train, y_train) .. rst-class:: sphx-glr-script-out .. code-block:: none -------------- Based on 100 instances, the recommended SOM hyperparameters are the following: Total node count (M): 50 Recommended sides (d1 x d2): 8 x 8 Initial neighborhood radius (sigma): 4.0 -------------- -------------- Based on 20 instances, the recommended SOM hyperparameters are the following: Total node count (M): 23 Recommended sides (d1 x d2): 5 x 5 Initial neighborhood radius (sigma): 2.5 -------------- Quantization Error: 0.3590 An SOM representation has been fitted as follows: ------------------------------------------------------- Hyperparameters of SOM: {'input_len': 2, 'x': 8, 'y': 8, 'sigma': np.float64(4.0), 'topology': 'rectangular', 'learning_rate': 0.5, 'decay_function': 'asymptotic_decay', 'sigma_decay_function': 'asymptotic_decay', 'neighborhood_function': 'gaussian', 'activation_distance': 'euclidean', 'random_seed': 42, 'num_iteration': 20, 'verbose': True} Quality of SOM: Quantization Error (QE): 0.3589908984142133 Topographic Error (TE): 0.0 An SOM representation estimator has been fitted as follows: ----------------------------------------------------------------- Hyperparameters: {'nu': 0.05, 'random_seed': 42, 'som_representation': , 'verbose': True} Learned parameters: Offset: -1.203157467698538 Quantization Error: 0.3143 An SOM representation has been fitted as follows: ------------------------------------------------------- Hyperparameters of SOM: {'input_len': 2, 'x': 5, 'y': 5, 'sigma': np.float64(2.5), 'topology': 'rectangular', 'learning_rate': 0.5, 'decay_function': 'asymptotic_decay', 'sigma_decay_function': 'asymptotic_decay', 'neighborhood_function': 'gaussian', 'activation_distance': 'euclidean', 'random_seed': 42, 'num_iteration': 20, 'verbose': True} Quality of SOM: Quantization Error (QE): 0.31432479303447436 Topographic Error (TE): 0.0 An SOM representation estimator has been fitted as follows: ----------------------------------------------------------------- Hyperparameters: {'nu': 0.05, 'random_seed': 42, 'som_representation': , 'verbose': True} Learned parameters: Offset: -1.0382313050926588 A PAN estimator has been fitted as follows: ----------------------------------------------------------------- Hyperparameters: {'abnormal_label': 1, 'estimators': {0: SomRepresentationEstimator(nu=0.05, random_seed=42, som_representation=, verbose=True), 1: SomRepresentationEstimator(nu=0.05, random_seed=42, som_representation=, verbose=True)}, 'normal_label': 0, 'nu': 0.05, 'omega': 3.5, 'random_seed': 42, 'scaler__copy': True, 'scaler__with_mean': True, 'scaler__with_std': True, 'scaler': StandardScaler(), 'verbose': True} Learned parameters: Offset: -1.2031574486382641 .. raw:: html 
ParallelAnomalousNudge(estimators={0: SomRepresentationEstimator(nu=0.05,
                                                                     random_seed=42,
                                                                     som_representation=<minisom_representation.som_representation.SomRepresentation object at 0x7fc29016eb10>,
                                                                     verbose=True),
                                       1: SomRepresentationEstimator(nu=0.05,
                                                                     random_seed=42,
                                                                     som_representation=<minisom_representation.som_representation.SomRepresentation object at 0x7fc290489460>,
                                                                     verbose=True)},
                           nu=0.05, omega=3.5, random_seed=42,
                           scaler=StandardScaler(), verbose=True)
In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook.
On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.


.. GENERATED FROM PYTHON SOURCE LINES 40-50 .. code-block:: Python # train with SOM representation estimator and underlying SOM hyperparameters specified X_train_scaled = StandardScaler().fit_transform(X_train) som_rep_0 = SomRepresentation.with_derived_params(X=X_train_scaled[y_train == 0], sigma=2.5, learning_rate=0.25, num_iteration=25, random_seed=RANDOM_SEED, verbose=VERBOSE) som_rep_1 = SomRepresentation.with_derived_params(X=X_train_scaled[y_train == 1], sigma=1.0, learning_rate=0.10, num_iteration=10, random_seed=RANDOM_SEED, verbose=VERBOSE) estimator_0 = SomRepresentationEstimator.from_som_representation(som_rep_0, nu=0.25, random_seed=RANDOM_SEED, verbose=VERBOSE) estimator_1 = SomRepresentationEstimator.from_som_representation(som_rep_1, nu=0.1, random_seed=RANDOM_SEED, verbose=VERBOSE) estimators = {0: estimator_0, 1: estimator_1} model = ParallelAnomalousNudge.from_estimators(estimators=estimators, scaler=StandardScaler(), nu=estimator_0.nu, omega=3.5, random_seed=RANDOM_SEED, verbose=VERBOSE) model.fit(X_train, y_train) .. rst-class:: sphx-glr-script-out .. code-block:: none -------------- Based on 100 instances, the recommended SOM hyperparameters are the following: Total node count (M): 50 Recommended sides (d1 x d2): 8 x 8 Initial neighborhood radius (sigma): 4.0 -------------- -------------- Based on 20 instances, the recommended SOM hyperparameters are the following: Total node count (M): 23 Recommended sides (d1 x d2): 5 x 5 Initial neighborhood radius (sigma): 2.5 -------------- Quantization Error: 0.3461 An SOM representation has been fitted as follows: ------------------------------------------------------- Hyperparameters of SOM: {'input_len': 2, 'x': 8, 'y': 8, 'sigma': 2.5, 'topology': 'rectangular', 'learning_rate': 0.25, 'decay_function': 'asymptotic_decay', 'sigma_decay_function': 'asymptotic_decay', 'neighborhood_function': 'gaussian', 'activation_distance': 'euclidean', 'random_seed': 42, 'num_iteration': 25, 'verbose': True} Quality of SOM: Quantization Error (QE): 0.3460562864603404 Topographic Error (TE): 0.05 An SOM representation estimator has been fitted as follows: ----------------------------------------------------------------- Hyperparameters: {'nu': 0.25, 'random_seed': 42, 'som_representation': , 'verbose': True} Learned parameters: Offset: -0.4297918328330103 Quantization Error: 0.2510 An SOM representation has been fitted as follows: ------------------------------------------------------- Hyperparameters of SOM: {'input_len': 2, 'x': 5, 'y': 5, 'sigma': 1.0, 'topology': 'rectangular', 'learning_rate': 0.1, 'decay_function': 'asymptotic_decay', 'sigma_decay_function': 'asymptotic_decay', 'neighborhood_function': 'gaussian', 'activation_distance': 'euclidean', 'random_seed': 42, 'num_iteration': 10, 'verbose': True} Quality of SOM: Quantization Error (QE): 0.2509881031085331 Topographic Error (TE): 0.6 An SOM representation estimator has been fitted as follows: ----------------------------------------------------------------- Hyperparameters: {'nu': 0.1, 'random_seed': 42, 'som_representation': , 'verbose': True} Learned parameters: Offset: -0.5539597797686613 A PAN estimator has been fitted as follows: ----------------------------------------------------------------- Hyperparameters: {'abnormal_label': 1, 'estimators': {0: SomRepresentationEstimator(nu=0.25, random_seed=42, som_representation=, verbose=True), 1: SomRepresentationEstimator(nu=0.1, random_seed=42, som_representation=, verbose=True)}, 'normal_label': 0, 'nu': 0.25, 'omega': 3.5, 'random_seed': 42, 'scaler__copy': True, 'scaler__with_mean': True, 'scaler__with_std': True, 'scaler': StandardScaler(), 'verbose': True} Learned parameters: Offset: -0.4297918269277225 .. raw:: html 
ParallelAnomalousNudge(estimators={0: SomRepresentationEstimator(nu=0.25,
                                                                     random_seed=42,
                                                                     som_representation=<minisom_representation.som_representation.SomRepresentation object at 0x7fc2a819fc20>,
                                                                     verbose=True),
                                       1: SomRepresentationEstimator(nu=0.1,
                                                                     random_seed=42,
                                                                     som_representation=<minisom_representation.som_representation.SomRepresentation object at 0x7fc2910ce3f0>,
                                                                     verbose=True)},
                           nu=0.25, omega=3.5, random_seed=42,
                           scaler=StandardScaler(), verbose=True)
In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook.
On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.


.. rst-class:: sphx-glr-timing **Total running time of the script:** (0 minutes 0.301 seconds) .. _sphx_glr_download_auto_examples_demonstrate_02_train_pan_estimator.py: .. only:: html .. container:: sphx-glr-footer sphx-glr-footer-example .. container:: sphx-glr-download sphx-glr-download-jupyter :download:`Download Jupyter notebook: demonstrate_02_train_pan_estimator.ipynb ` .. container:: sphx-glr-download sphx-glr-download-python :download:`Download Python source code: demonstrate_02_train_pan_estimator.py ` .. container:: sphx-glr-download sphx-glr-download-zip :download:`Download zipped: demonstrate_02_train_pan_estimator.zip ` .. only:: html .. rst-class:: sphx-glr-signature `Gallery generated by Sphinx-Gallery `_